Join a project shaping the future of sustainable energy through cutting-edge data engineering and scientific innovation. We are looking for a Data Engineer to help build scalable cloud-native data platforms for a deep-tech Customer working on one of the most ambitious fusion energy initiatives in the world.
You will work with modern distributed data technologies, including Databricks, Apache Spark, AWS, and Azure, while solving complex engineering challenges in a research-intensive environment.
We at Sigma Software value engineering excellence, long-term partnerships, and continuous learning. This is an opportunity to contribute to breakthrough scientific initiatives while working with modern technologies and highly skilled global teams.
PROJECT
The project focuses on building and evolving a modern cloud-based data platform supporting scientific research, analytics, and operational workflows in the fusion energy domain. The team develops scalable distributed data processing solutions while ensuring high standards of data quality, lineage, governance, reproducibility, and operational reliability.
You will work closely with scientists and engineers to support experimentation, analytics, and machine learning workflows in a highly innovative R&D environment.
• Design, develop, and maintain scalable and reliable data platforms and pipelines for analytics and operational use cases
• Build distributed data processing workflows using Apache Spark, pySpark, and Databricks
• Develop and evolve cloud-native data solutions leveraging Azure and AWS services
• Model and optimize data structures for analytics, reporting, and downstream consumption
• Integrate data from multiple heterogeneous sources while ensuring data quality, consistency, and traceability
• Implement data quality controls, schema validation, lineage tracking, and monitoring processes
• Optimize performance, scalability, reliability, and cost-efficiency of data pipelines and infrastructure
• Collaborate closely with scientists, analysts, and engineering teams to translate business and research requirements into technical solutions
• Contribute to CI/CD processes, infrastructure automation, and engineering best practices
• Research and introduce new technologies and approaches within the data engineering ecosystem
• 3+ years of commercial experience in Data Engineering
• 2+ years of hands-on production experience with Databricks
• Strong expertise with Delta Lake, including medallion architecture and table optimization
• Advanced proficiency in Python, pySpark, and SQL
• Deep practical knowledge of Databricks components including Unity Catalog, Lakeflow Jobs / Workflows, and job optimization
• Hands-on experience with AWS services including S3, IAM, KMS, and at least one of AWS Batch, ECR, EventBridge, Lambda, or API Gateway
• Experience designing event-driven and asynchronous data pipelines
• Strong knowledge of Infrastructure as Code and CI/CD practices
• Practical experience with Terraform and Databricks Asset Bundles
• Experience working with Git workflows, pull request reviews, and automated testing
• Understanding of secure authentication approaches such as OIDC and service principals
• Practical experience with MLflow, experiment tracking, model registry, and reproducibility practices
• Strong understanding of data quality, lineage, schema contracts, and provenance concepts
• Experience with data quality frameworks such as Great Expectations, Soda, pyDeequ, or Databricks DQX
• Experience monitoring and optimizing data platform reliability and cloud costs
• Upper-Intermediate or higher English level
WILL BE A PLUS:
• Experience packaging workloads using Docker and running batch workloads in AWS
• Familiarity with ECR and containerized data processing solutions
• Experience with experiment optimization libraries such as scikit-optimize, BoTorch, Ax, or scipy
• Understanding of ML feature engineering and leakage prevention techniques
• Experience working in regulated or high-security environments
• Knowledge of data governance, compliance, and data residency requirements
• Experience disabling or restricting AI-assisted tooling/features in secure environments
Skills Required
- 3+ years of commercial experience in data engineering
- 2+ years of hands-on production experience with Databricks
- Strong expertise with Delta Lake, including medallion architecture and table optimization
- Advanced proficiency in Python, PySpark, and SQL
- Practical knowledge of Databricks Unity Catalog, Lakeflow Jobs or Workflows, and job optimization
- Hands-on AWS experience with S3, IAM, KMS, and at least one of AWS Batch, ECR, EventBridge, Lambda, or API Gateway
- Experience designing event-driven and asynchronous data pipelines
- Strong knowledge of Infrastructure as Code and CI/CD practices
- Practical experience with Terraform and Databricks Asset Bundles
- Experience with Git workflows, pull request reviews, and automated testing
- Understanding of secure authentication approaches such as OIDC and service principals
- Practical experience with MLflow, experiment tracking, model registry, and reproducibility practices
- Strong understanding of data quality, lineage, schema contracts, and provenance
- Experience with data quality frameworks such as Great Expectations, Soda, pyDeequ, or Databricks DQX
- Experience monitoring and optimizing data platform reliability and cloud costs
- Upper-Intermediate or higher English level
- Experience packaging workloads with Docker and running batch workloads in AWS
- Familiarity with Amazon ECR and containerized data processing solutions
- Experience with scikit-optimize, BoTorch, Ax, or SciPy
- Understanding of ML feature engineering and leakage prevention
- Experience working in regulated or high-security environments
- Knowledge of data governance, compliance, and data residency requirements
- Experience disabling or restricting AI-assisted tooling or features in secure environments
What We Do
Sigma Software Group, an award-winning and trusted IT partner, has been serving customers for over 21 years, providing comprehensive IT solutions to various businesses, ranging from startups to established software product houses. As one of Europe's substantial IT consultancies, it brings together a dedicated workforce of over 2,100 professionals in 40 offices across 19 countries. With a diverse client base, including more than 300 enterprises, including Fortune 500 stalwarts, Sigma Software Group is a preferred choice for developing solutions that help businesses create cutting-edge products while meeting their unique needs. Sigma Software Group operates as a dynamic ecosystem of tech companies, offering 25 ready-to-implement innovative products and 40+ value-added services. Furthermore, Sigma Software Group is committed to fostering innovation through initiatives such as the Sigma Software Labs business incubator, Sigma Software University, the SID Venture Partners VC Fund, UA Tech Network, Techosystem, the European Business Association, and other collaborative efforts. Since 2015, Sigma Software Group has consistently earned recognition on the IAOP's prestigious World's Top 100 Outsourcing list. The company's accomplishments have also been acknowledged by prominent global media outlets such as Forbes, CNBC, The Times, and Reuters
