The Role
Design, develop, and maintain scalable batch and streaming data-processing solutions using Azure Databricks, Python, PySpark, and SQL. Build data models and architectures, manage parallel workloads, implement Azure YAML CI/CD pipelines, and use ARM/Bicep for infrastructure as code. Improve reliability, performance, maintainability, and technical debt while collaborating in an Agile engineering environment.
Summary Generated by Built In
As an Azure Databricks Engineer, you will:
- Design, develop, and maintain robust and reliable data processing solutions using Azure Databricks, Python, and PySpark.
- Build and manage both batch and streaming data processing workflows within Databricks.
- Design appropriate data models and processing architectures within the Databricks environment.
- Manage multiple parallel data processes working on shared source data.
- Develop scalable and maintainable data solutions while ensuring high standards of quality, reliability, and performance.
- Implement and maintain CI/CD pipelines using Azure and YAML.
- Work with Infrastructure as Code (IaC) using ARM/Bicep.
- Continuously improve existing solutions, reduce technical debt, and make pragmatic technical decisions.
- Collaborate effectively within an Agile engineering environment.
What You Bring to the Table:
- 8+ years of overall professional experience in data engineering/software engineering.
- Strong hands-on experience with Azure Databricks as a core technology.
- Deep expertise in Python, PySpark, and SQL.
- Proven experience designing and managing data processing solutions within Databricks.
- Strong experience handling both streaming and batch data.
- Experience designing data models specifically for Databricks-based processing environments.
- Solid experience with Microsoft Azure cloud services.
- Hands-on experience with CI/CD pipelines using YAML.
- Experience with Infrastructure as Code, particularly ARM/Bicep.
- Experience working in an Agile environment.
You should possess the ability to:
- Build stable, scalable, reliable, and maintainable Databricks solutions.
- Apply strong problem-solving and analytical skills to complex data-processing challenges.
- Make well-founded technical decisions in complex and evolving architectures.
- Work effectively with large-scale batch and streaming data processes.
- Manage and coordinate multiple parallel data-processing workloads.
- Identify and address technical debt proactively.
- Continuously improve data-processing solutions and engineering practices.
- Collaborate and communicate effectively with technical and cross-functional teams.
- Follow a structured and pragmatic engineering approach without unnecessary over-engineering.
- Balance engineering best practices with practical business and technical requirements.
What We Bring to the Table:
- An opportunity to work on complex Azure Databricks and data-processing solutions.
- Exposure to modern cloud, data engineering, CI/CD, and Infrastructure as Code practices.
- A collaborative Agile engineering environment focused on quality and continuous improvement.
- Opportunities to work with both batch and streaming data-processing architectures.
- A technically challenging environment where pragmatic, scalable, and maintainable solutions are valued
Let’s Connect
Want to discuss this opportunity in more detail? Feel free to reach out.
Recruiter: Giftson Paul Davidson
Phone: +31 20 369 0609 ; Extn : 151
LinkedIn: https://www.linkedin.com/in/giftsonpauldavidson/
Skills Required
- 8+ years of professional experience in data engineering or software engineering
- Strong hands-on experience with Azure Databricks
- Deep expertise in Python, PySpark, and SQL
- Experience designing and managing Databricks data-processing solutions
- Experience with batch and streaming data processing
- Experience designing data models for Databricks-based processing environments
- Solid experience with Microsoft Azure cloud services
- Hands-on experience with YAML-based CI/CD pipelines
- Experience with Infrastructure as Code, particularly ARM and Bicep
- Experience working in an Agile environment
- Strong problem-solving, analytical, communication, and collaboration skills
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
STAFIDE is a Netherlands-based niche technology talent consulting company operating across Europe. It helps organizations identify, recruit, and deploy technology professionals in areas including cybersecurity, cloud engineering, software development, data analytics, ERP, infrastructure, and digital transformation. Its services include recruitment, workforce engagement, secondment, onboarding support, workforce deployment, and workforce analytics that support technology hiring and expansion.
.jpg)
