The Role
Designs, develops, and maintains scalable Azure data pipelines and ETL/ELT solutions using Azure Data Factory, Databricks, PySpark, Python, and SQL. Builds CI/CD workflows with Azure DevOps, optimizes large-scale data processing, monitors and troubleshoots pipelines, and ensures data security, governance, compliance, and reliability. Collaborates with technical and business stakeholders to translate requirements into enterprise data engineering solutions.
Summary Generated by Built In
As an Azure Data Engineer, you will:
- Design, develop, and maintain complex and scalable data pipelines using Azure Data Factory (ADF) and Azure Databricks.
- Build and optimize large-scale data processing solutions using PySpark and Databricks.
- Develop robust ETL/ELT solutions in Python for data extraction, transformation, and loading.
- Collaborate with cross-functional teams to understand data requirements and deliver efficient data engineering solutions.
- Implement and maintain CI/CD pipelines using Azure DevOps for automated testing, integration, and deployment.
- Develop, test, and maintain reliable data solutions aligned with technical and business requirements.
- Monitor, troubleshoot, and optimize data pipelines to ensure performance, scalability, and reliability.
- Work with large datasets and use SQL for data querying and data management.
- Ensure data solutions follow applicable security, compliance, and engineering best practices.
- Stay updated with the latest trends, technologies, and best practices in Azure cloud data engineering.
What You Bring to the Table:
- 8–10 years of overall professional experience in data engineering, cloud data engineering, or related technology roles.
- Strong hands-on experience with Azure Data Factory for data orchestration, movement, and transformation.
- Strong experience with Azure Databricks for developing and managing scalable data processing solutions.
- Strong programming skills in Python.
- Hands-on experience with PySpark for large-scale data processing.
- Experience implementing CI/CD pipelines using Azure DevOps.
- Good knowledge of SQL and experience working with large datasets.
- Experience developing and maintaining ETL/ELT data pipelines.
- Understanding of data security, compliance, and cloud data engineering best practices.
- Strong problem-solving and analytical skills.
- Experience working independently as well as collaboratively within cross-functional teams.
You Should Possess the Ability To:
- Design and develop scalable data pipelines using Azure Data Factory and Azure Databricks.
- Develop efficient PySpark applications for processing and transforming large volumes of data.
- Write clean, maintainable, and reusable Python-based data engineering solutions.
- Build and manage automated CI/CD workflows in Azure DevOps.
- Perform data extraction, transformation, loading, validation, and integration across different data sources.
- Optimize data pipelines and Databricks workloads for performance and scalability.
- Monitor pipelines, identify failures or bottlenecks, and implement appropriate solutions.
- Troubleshoot data processing and pipeline-related issues effectively.
- Write SQL queries for data analysis, transformation, and validation.
- Work with stakeholders to understand requirements and translate them into technical data solutions.
- Follow appropriate data security, governance, compliance, and development standards.
- Document technical solutions, data processes, pipelines, and deployment procedures.
- Adapt to new Azure services, cloud data engineering technologies, and industry best practices.
What We Bring to the Table:
- An opportunity to work on enterprise-scale Azure cloud data engineering projects.
- Exposure to Azure Data Factory, Azure Databricks, PySpark, Python, Azure DevOps, and SQL.
- Opportunities to design and optimize large-scale data pipelines and processing frameworks.
- A collaborative environment involving cross-functional technical and business teams.
- Opportunities to contribute to cloud data modernization, automation, and CI/CD initiatives.
Let’s Connect
Want to discuss this opportunity in more detail? Feel free to reach out.
Recruiter: Hema Murali
Phone: +31 20 369 0609 ; Extn :148
LinkedIn: https://www.linkedin.com/in/hema-murali-315999329/
Skills Required
- 8-10 years of professional experience in data engineering, cloud data engineering, or related technology roles
- Strong hands-on experience with Azure Data Factory
- Strong experience with Azure Databricks
- Strong programming skills in Python
- Hands-on experience with PySpark
- Experience implementing CI/CD pipelines using Azure DevOps
- Good knowledge of SQL and experience with large datasets
- Experience developing and maintaining ETL/ELT data pipelines
- Understanding of data security, compliance, and cloud data engineering best practices
- Strong problem-solving and analytical skills
- Ability to work independently and collaboratively with cross-functional teams
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
STAFIDE is a Netherlands-based niche technology talent consulting company operating across Europe. It helps organizations identify, recruit, and deploy technology professionals in areas including cybersecurity, cloud engineering, software development, data analytics, ERP, infrastructure, and digital transformation. Its services include recruitment, workforce engagement, secondment, onboarding support, workforce deployment, and workforce analytics that support technology hiring and expansion.
