26-159.B - Data Engineer

Job Objective:
We are seeking a highly versatile and experienced Data Engineer to join our team. The ideal candidate thrives in a fast-paced environment and can manage a broad range of responsibilities across our data platform. You will play a critical role in managing and optimizing our Databricks environment, designing and maintaining data pipelines, performing ETL development, tracking platform costs and system health, and building data applications using both personal expertise and AI-driven approaches. Your work will directly contribute to the success of our asset teams by ensuring the availability, accuracy, reliability, and performance of our data and AI systems.

Essential Job Responsibilities:

• Serve as an administrator of the Databricks platform, managing workspaces, clusters, jobs, permissions, and configurations across environments.
• Monitor and optimize Databricks cluster performance, compute usage, and storage to ensure cost-effective and efficient operations.
• Track and report on platform costs, identifying opportunities for cost reduction and resource optimization across Databricks and associated cloud services.
• Proactively monitor the health, availability, and performance of data pipelines and platform components, implementing alerting and observability best practices.
• Develop, manage, and maintain scalable and reliable data pipelines for collecting, processing, and storing large volumes of structured and unstructured data.
• Create and optimize ETL processes to ensure high-quality data is available for analysis and reporting, leveraging tools such as dbt, Spark, and Databricks workflows.
• Integrate data from multiple sources, ensuring data consistency, quality, and accessibility across the organization.
• Implement and monitor data quality checks and validation processes to ensure accuracy and reliability of data throughout the pipeline lifecycle.
• Architect and maintain robust, scalable, and efficient data infrastructure to support data-driven applications and analytics initiatives.
• Build and deploy data applications using a combination of personal technical expertise and AI-assisted development tools and methodologies.
• Support the deployment and maintenance of data-driven applications, ensuring they are optimized for performance and scalability.
• Monitor and improve the performance of data systems and queries, ensuring efficient data processing and minimal downtime.

Other Job Responsibilities:

• Maintains employee confidence and protects company assets, including intellectual property, by keeping information confidential.
• Maintains technical knowledge by attending educational workshops, reviewing professional publications, establishing personal networks, and participating in professional associations.
• Other duties as assigned by management.

Qualifications:

• Three (3) years minimum experience in a data engineering role, prefer oil and gas experience.
• Strong working knowledge of Databricks, including workspace administration, cluster management, Delta Lake, Unity Catalog, and Databricks SQL.
• Experience with cloud application infrastructure, particularly AWS (S3, IAM, EC2, Lambda, etc.).
• Proficiency with data engineering tools such as dbt, Apache Spark, SQL, and Python.
• Experience tracking and optimizing cloud platform costs and resource utilization.
• Demonstrated ability to build data applications using modern development practices, including leveraging AI-assisted development tools.
• Exceptional ability in diagnosing and solving technical issues related to data platforms and pipelines.
• Ability to establish and maintain effective working relationships with employees, supervisors, other departments, officials, and the public.
• Ability to complete multiple, diverse tasks of differing priorities with minimum supervision.
• Excellent written and verbal communication skills, with specific ability to translate complex data systems information into an organized and presentable manner.
• Ability to travel to remote locations via large and small planes, helicopter, or boat. Expectation is quarterly trips to our asset locations.
• Required: Working knowledge of oil and gas industry applications and processes.

Education Requirements:

• Bachelor’s Degree from an accredited four-year university or college with a degree in Computer Science, Data Engineering, Information Systems, or a closely related field. Additional experience may substitute for education on a year-for-year basis.