We are looking for a Senior Data Engineer to help drive the evolution of our Azure-based Data Platform and Lakehouse architecture.
This role is focused on Data Products development. The successful candidate will build pipelines in ADF, DAGs in Airflow and work closely with POs to refine business requirements and transform business needs into reliable Data Products in Databricks.
Key Responsibilites
Implement scalable, reliable and reusable data pipelines using Azure Data Factory and Apache Airflow.
Develop data transformations in Azure Databricks (PySpark / SQL) to create/extend data products following the Medallion Architecture layers:
Bronze (raw ingestion)
Silver (cleansed & conformed)
Gold (business-ready data products)
Optimize performance, cost, and reliability of data workloads.
Raise important questions during business maturation in order to detect risks before development stage.
Participate in architecture reviews and technical decision-making.
Promote engineering excellence through code reviews and design reviews.
Collaborate with Architects, Product Owners in data product technical maturation.
Provide support to the Run/Operations team for production pipelines and data products for complex topics.
Troubleshoot and resolve data incidents, failures, and performance issues
Participate in incident management, root cause analysis, and post-mortems.
Ensure proper handover, documentation, and operational readiness of developed solutions.
Contribute to improving runbooks, monitoring, and operational processes.
Hands-on Data Product development
Technical Leadership
Support & Operations (Run Team Collaboration)
Skills Required
- Experience developing scalable data pipelines with Azure Data Factory
- Experience developing DAGs with Apache Airflow
- Experience developing data transformations in Azure Databricks using PySpark and SQL
- Experience with Lakehouse and Medallion Architecture
- Experience optimizing data workload performance, cost, and reliability
- Experience with architecture reviews, technical decision-making, code reviews, and design reviews
- Experience troubleshooting production data incidents and conducting root cause analysis
What We Do
More than a simple IT consulting group, MARGO is an alchemy of talents with constantly enriched expertise, always encouraged to be dynamic and to flourish. For more than 17 years, we have been carrying out missions of the highest technical complexity, enabling the digital acceleration of our clients, while ensuring a perpetual intellectual and collective stimulation for all our talents. At MARGO it's Consultant First. By joining MARGO, your talent today will remain relevant tomorrow. We bring you into the group not only for what you are today but for the potential you wish to develop. At MARGO, you will be able to build your own path to excellence within our teams and our missions. Tailor-made to serve your ambition and to match your talent. Today, Margo has over 400 employees in 8 entities: ► MARGO Trading Systems: our full stack development offer in market finance ► MARGO Capital Markets: our business offer in market finance ► MARGO Analytics: our expertise in data science and data engineering ► A CAPELLA Consulting: our business offer dedicated to insurance ► CODE BUSTERS: our community of developers ► DELIVERED by MARGO: our 100% remote full stack development entity ► MARGO UK: expertise in complex coding and application support on the English market. ► MARGO Poland: expertise in complex coding in retail Banking on the Polish market
