Our client is seeking a capable intermediate-level Data Engineer with strong MLOps and analytics experience to support the design, optimisation, governance, and monitoring of enterprise data and machine learning pipelines.
The successful candidate will play a critical role in ensuring scalable, sustainable, and efficient data processes while supporting analytics, ML model deployment, integrations, and reporting initiatives within a Databricks ecosystem.
This opportunity offers strong long-term potential, as contractors are typically retained for multi-year engagements.
RequirementsKey Responsibilities
Data Engineering & Pipeline Management
- Design, optimise, and maintain scalable data pipelines within Databricks.
- Ensure pipelines are efficient, sustainable, easy to debug, and user-friendly.
- Implement and maintain Delta Tables and Databricks notebooks.
- Perform data validation and basic data quality checks.
- Monitor and improve process governance and operational efficiency.
- Train, deploy, and monitor machine learning models using MLflow.
- Analyse model performance and business impact.
- Support model lifecycle management and deployment best practices.
- Develop Power BI dashboards and business insight reporting.
- Support data-driven decision-making through analytics solutions.
- Monitor API data integrations and data sends.
- Troubleshoot integration failures and ensure data consistency.
- Manage Git-based workflows including:
- Pull requests
- Branch syncing
- Merge conflict resolution
- Pull requests
- Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders.
Qualifications
- Degree or Diploma in:
- Computer Science
- Data Engineering
- Information Systems
- Mathematics
- Statistics
- or related field
- Computer Science
- 3–5 years’ experience in Data Engineering or related roles.
- Hands-on experience with Databricks.
- Experience with MLflow and machine learning deployment processes.
- Experience with Power BI dashboard development.
- Strong experience with Git version control workflows.
- Exposure to API integrations and monitoring.
- Databricks
- Delta Tables
- Databricks Notebooks
- MLflow
- Python
- SQL
- Power BI
- Git / Azure DevOps
- API Monitoring & Integration
- Data Pipeline Optimisation
- Data Quality & Governance
- Azure Data Services
- CI/CD for ML Pipelines
- Spark / PySpark
- Cloud-based data platforms
- MLOps best practices
- Strong analytical and problem-solving abilities
- Attention to detail
- Strong communication skills
- Ability to work in collaborative environments
- Self-driven and proactive mindset
Skills Required
- Degree or Diploma in related field
- 3-5 years' experience in Data Engineering
- Hands-on experience with Databricks
- Experience with MLflow
- Experience with Power BI dashboard development
- Strong experience with Git workflows
- Exposure to API integrations
What We Do
Blue Pearl is a market-leading CLOUD Solutions developer with extensive knowledge and insight into the latest technologies, standardised processes, advanced technical capabilities and consulting processes available, ensuring wholistic success for our clientele. We offer professional consulting to compliment your business strategy and overall management and make it our priority to add value to any business by listening, analysing and creating a conducive solution that will empower our client. We implement a Data Analysis Process that includes inspecting, cleansing, transforming, and modelling data with the end-goal of discovering useful information, informing conclusions, and relevant information to support your decision-making. Your business cannot afford not to engage with us, allowing our data analysis to play a role in making your business decisions more scientific and helping your business achieve effective operation. Blue Pearl’s team of experts include BI strategists, BI analysts, Data Warehouse Architects, Data Scientists, Implementation and Development experts. With the use of BI, Analytics and Big Data, we effectively partner with our customers on their mission to achieve a competitive business advantage and real ROI from the structured information we collect.
