The Role
Design, build, and maintain scalable Azure-based data pipelines and ETL/ELT workflows using Azure Data Factory, Databricks, PySpark, and Python. Implement CI/CD in Azure DevOps, optimize performance, ensure data quality, security, and governance, and collaborate with stakeholders to translate requirements into production-ready data solutions.
Summary Generated by Built In
As an Azure Data Engineer, you will:
- Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF) and Azure Databricks.
- Build and optimize large-scale data processing solutions using PySpark and Databricks.
- Develop robust ETL/ELT workflows using Python for data extraction, transformation, and loading.
- Implement and maintain CI/CD pipelines in Azure DevOps to automate deployment and testing of data solutions.
- Collaborate with business stakeholders, data architects, and engineering teams to translate data requirements into scalable solutions.
- Monitor, troubleshoot, and optimize data pipelines to ensure high performance, reliability, and scalability.
- Ensure data quality, security, governance, and compliance with industry best practices.
- Document technical solutions and contribute to continuous improvement of cloud data engineering practices.
What You Bring to the Table:
- 5+ years of experience in Data Engineering with a strong focus on Microsoft Azure.
- Hands-on experience with Azure Data Factory (ADF) for orchestrating data movement and transformation workflows.
- Strong expertise in Azure Databricks for developing scalable big data processing solutions.
- Proficiency in Python and PySpark for distributed data processing and ETL development.
- Experience implementing CI/CD pipelines using Azure DevOps.
- Strong knowledge of SQL for querying, transforming, and managing large datasets.
- Experience working with cloud-based data platforms and modern data engineering best practices.
- Good understanding of data security, performance tuning, and optimization techniques.
- Strong analytical, problem-solving, and debugging skills.
- Excellent communication, collaboration, and technical documentation skills.
You should possess the ability to:
- Design and develop scalable, high-performance cloud data solutions.
- Build and optimize complex ETL/ELT pipelines for large datasets.
- Process and analyze big data efficiently using PySpark and Databricks.
- Automate deployments through Azure DevOps CI/CD pipelines.
- Troubleshoot and resolve data pipeline failures and performance bottlenecks.
- Collaborate effectively with cross-functional technical and business teams.
- Work independently while managing multiple priorities in an Agile environment.
- Follow cloud security, governance, and data quality best practices.
- Continuously improve data engineering processes through automation and optimization.
What we bring to the table:
- Opportunity to work on enterprise-scale Azure cloud data engineering projects.
- Exposure to modern cloud-native technologies including Azure Data Factory, Azure Databricks, PySpark, and Azure DevOps.
- Collaborative environment that encourages innovation and knowledge sharing.
- Opportunities to work with large-scale data platforms and cloud transformation initiatives.
- Continuous learning and professional development in Azure cloud technologies.
Let’s Connect
Want to discuss this opportunity in more detail? Feel free to reach out.
Recruiter: Aswin Dhanvandhar
Phone: +31 20 369 0609 ; Extn :141
LinkedIn:https://www.linkedin.com/in/aswin-dhanvandhar/
Skills Required
- 5+ years of Data Engineering experience with Microsoft Azure
- Hands-on experience with Azure Data Factory (ADF)
- Strong expertise in Azure Databricks
- Proficiency in Python and PySpark for ETL and big data processing
- Experience implementing CI/CD pipelines using Azure DevOps
- Strong knowledge of SQL for querying and managing large datasets
- Experience with cloud-based data platforms and modern data engineering best practices
- Understanding of data security, governance, and compliance best practices
- Strong analytical, problem-solving, debugging, and communication skills
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
STAFIDE is a Netherlands-based niche technology talent consulting company operating across Europe. It helps organizations identify, recruit, and deploy technology professionals in areas including cybersecurity, cloud engineering, software development, data analytics, ERP, infrastructure, and digital transformation. Its services include recruitment, workforce engagement, secondment, onboarding support, workforce deployment, and workforce analytics that support technology hiring and expansion.
.jpg)
