The Role
Design, build, and optimize scalable batch and real-time data ingestion, transformation, and analytics pipelines (Spark/Delta Lake). Migrate timeseries data, create analytical models and KPIs, implement monitoring/automation, and support platform APIs and DevOps practices to enable analytics and MLOps.
Summary Generated by Built In
Are you passionate about helping people improve the impact of their business through the use of data and analytics? If you enjoy the challenges associated with making complex information systems more user-friendly and effective, we are looking for you to join a team of creative and highly motivated professionals who are driving innovation in the upstream oil field services industry!
At National Oilwell Varco, we strive to lead technology innovation that delivers significant value to our customers. We are hiring a Big Data Engineer to help us build and support our data and analytics delivery pipeline.
Responsibilities
- Design, develop, and optimize scalable data ingestion, transformation, and analytics pipelines for IoT, operational, and business data using Spark and Delta Lake technologies.
- Create and tune Spark transformation, aggregation, streaming, and machine learning workloads, including optimization of Spark clusters and processing infrastructure.
- Develop reusable frameworks for ingesting, validating, enriching, and processing large-scale IoT and event-driven datasets.
- Build and maintain analytical data models, curated datasets, and KPI calculation frameworks that enable product analytics, operational reporting, business intelligence, and self-service analytics.
- Collaborate with Product, Engineering, Operations, and business stakeholders to define, implement, and maintain meaningful KPIs, metrics, and data products that support decision-making.
- Design and implement real-time and batch processing solutions using event-based and streaming technologies.
- Build and execute large-scale data migration solutions for transferring historical and operational data from source systems such as OSI PI and TimescaleDB into enterprise data platforms.
- Develop monitoring, observability, reconciliation, and automation tools that ensure the reliability, quality, and performance of data pipelines, migrations, analytics workloads, and MLOps/DataOps processes.
- Support the delivery of related platform capabilities, including APIs, data services, and integrations, while working within Agile and DevOps methodologies.
- Document solution architecture, data lineage, business logic, KPI definitions, and operational procedures to support long-term maintainability and governance.
Requirements
- Bachelor’s degree in computer science, information systems, or a related field. However, relevant experience will be considered.
- A minimum of 5 years of relevant experience required.
- Advanced understanding with spark and/or Databricks(preferred), including spark cluster tuning.
- Great understanding of distributed systems and partitioning.
- Automation and scripting using .net, C#, Python, Javascript, GoLang, AWS Cloud APIs.
- Database technologies and Timeseries databases like OSIPI, PostgreSQL, Timescale and SQL..
- Developing applications and/or scripts utilizing Timeseries data.
- Linux, Windows OS, Containers.
- Data management and data integration strategies.
- Amazon Web services and/or cloud technologies.
- Visualization/reporting tools such as Tableau, Spotfire, Grafana, etc.
- Understanding of CI/CD practices and technologies. GitHub and GitHub Actions(preferred), Jenkins, TeamCity, Udeploy, etc.
- Optimally communicating technical information.
- Working knowledge of ML and AI model development and the data science development life cycle is an added advantage.
- Experience with large datasets and understanding of stream and batch processing , Datadog, Brokers like Kafka or NATS , Container technologies, and terraform.
- Ability to relate architectural decisions and recommendations to business needs.
- Strong analytical and problem-solving skills.
- Highly independent and ability to communicate across all levels of the organization and work with diverse projects teams.
- Willingness to demonstrate a growth-mindset when faced with new challenges and opportunities.
We Power the Industry that Powers the World
Throughout every region in the world and across every area of drilling and production, our family of companies has provided the technical expertise, advanced equipment, and operational support necessary for success—now and in the future.
Global Family
We are a global family of thousands of individuals, working as one team to create a lasting impact for ourselves, our customers, and the communities where we live and work.
Purposeful Innovation
Through purposeful business innovation, product creation, and service delivery, we are driven to power the industry that powers the world better.
Service Above All
This drives us to anticipate our customers’ needs and work with them to deliver the finest products and services on time and on budget.
Skills Required
- Bachelor's degree in computer science, information systems, or related field (relevant experience considered)
- Minimum of 5 years of relevant experience
- Spark (advanced understanding, including cluster tuning)
- Databricks (preferred)
- Strong understanding of distributed systems and partitioning
- Automation and scripting using .NET, C#, Python, JavaScript, GoLang, and AWS Cloud APIs
- Timeseries and database technologies (OSI PI, TimescaleDB, PostgreSQL) and SQL
- Experience developing applications or scripts that utilize timeseries data
- Experience with Linux, Windows, and container technologies
- Experience with AWS or other cloud platforms
- Experience with visualization/reporting tools (Tableau, Spotfire, Grafana)
- Understanding of CI/CD practices and tools (GitHub, GitHub Actions preferred; Jenkins, TeamCity, Udeploy)
- Experience with large datasets, stream and batch processing, brokers (Kafka or NATS), Datadog, and Terraform
- Working knowledge of ML/AI model development and the data science lifecycle
- Strong analytical, problem-solving, communication, and ability to relate architectural decisions to business needs
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
NOV delivers technology-driven solutions to empower the global energy industry. For more than 150 years, NOV has pioneered innovations that enable its customers to safely produce abundant energy while minimizing environmental impact. The energy industry depends on NOV’s deep expertise and technology to continually improve oilfield operations and assist in efforts to advance the energy transition towards a more sustainable future. NOV powers the industry that powers the world.
