The Azure Data Engineer will be responsible for designing, building, optimizing, and maintaining data pipelines and data solutions within the Microsoft Azure ecosystem. The role requires strong experience in cloud-based data engineering, modern data architectures, and scalable data processing frameworks.
- Design, develop, and maintain Azure-based data pipelines (batch and streaming).
- Implement ETL/ELT solutions using Azure Data Factory (ADF), Azure Databricks, Synapse Analytics, and related Azure services.
- Build and optimize data models, including Data Lakehouse architectures (Medallion, Delta Lake).
- Ensure data quality, integrity, and governance across all data processes.
- Collaborate with data scientists, analysts, and business teams to provide clean, reliable datasets.
- Perform pipeline monitoring, troubleshooting, performance optimization, and cost optimization.
- Develop CI/CD pipelines using Azure DevOps for data-related deployments.
- Implement security best practices, including RBAC, encryption, and data access governance.
- Strong proficiency with Azure Data Factory (ADF), Azure Databricks (PySpark), and Azure Synapse Analytics.
- Expert knowledge of Python, SQL, and data transformation frameworks.
- Experience with Delta Lake, Spark, Data Lake Storage (ADLS), and distributed compute patterns.
- Familiarity with Data Vault, Dimensional Modelling, or Medallion architecture.
- Experience with cloud DevOps tools (Azure DevOps, GitHub Actions).
- Knowledge of streaming technologies (Kafka, Event Hubs) is an advantage.
Requirements
- Azure certifications such as:
- AZ‑900 Azure Fundamentals
- DP‑203 Azure Data Engineer Associate
- Databricks Data Engineer certifications
- AZ‑900 Azure Fundamentals
- Bachelor’s degree in Computer Science, Engineering, or related field.
- 5+ years of data engineering experience (Azure preferred).
- Proven experience building and supporting production-scale cloud data solutions.
- Location: Johannesburg (Hybrid working model)
- Ways of Working: Standard SAST business hours
- Candidate: Preferably a South African Citizen or Permanent Resident
Skills Required
- 5+ years of data engineering experience
- Bachelor's degree in Computer Science, Engineering, or related field
What We Do
Blue Pearl is a market-leading CLOUD Solutions developer with extensive knowledge and insight into the latest technologies, standardised processes, advanced technical capabilities and consulting processes available, ensuring wholistic success for our clientele. We offer professional consulting to compliment your business strategy and overall management and make it our priority to add value to any business by listening, analysing and creating a conducive solution that will empower our client. We implement a Data Analysis Process that includes inspecting, cleansing, transforming, and modelling data with the end-goal of discovering useful information, informing conclusions, and relevant information to support your decision-making. Your business cannot afford not to engage with us, allowing our data analysis to play a role in making your business decisions more scientific and helping your business achieve effective operation. Blue Pearl’s team of experts include BI strategists, BI analysts, Data Warehouse Architects, Data Scientists, Implementation and Development experts. With the use of BI, Analytics and Big Data, we effectively partner with our customers on their mission to achieve a competitive business advantage and real ROI from the structured information we collect.
