We are looking for a Senior Data Engineer to design, build, and maintain enterprise-scale data platforms on Azure. You will own end-to-end data pipelines from raw ingestion to curated, analytics-ready gold layers while driving data quality standards, mentoring junior engineers, and collaborating directly with product and AI teams.
Requirements
Pipeline Design & Architecture
▸ Architect and manage scalable ETL/ELT pipelines using Medallion Architecture (Bronze → Silver → Gold)
▸ Design ADF pipelines, data models, and schemas optimised for performance and maintainability
▸ Build near real-time and batch ingestion solutions across structured and semi-structured sources
Data Governance & Quality
▸ Implement data quality checks, validation frameworks, and lineage tracking across pipeline layers
▸ Enforce security controls, access policies, and compliance requirements on ADLS Gen 2 and Azure SQL
▸ Monitor pipeline SLAs, diagnose performance bottlenecks, and own incident resolution
Engineering Excellence
▸ Drive CI/CD adoption for data pipelines using Azure DevOps or equivalent tooling
▸ Define and enforce coding standards, peer-review practices, and documentation norms
▸ Contribute to Al agent development initiatives as a data platform subject-matter expert
Collaboration & Mentorship
▸ Partner with data scientists, analysts, and product managers to translate requirements into reliable data products
▸ Mentor junior engineers through code reviews, design discussions, and structured knowledge sharing
▸ Participate in Agile ceremonies and contribute to sprint planning, backlog refinement, and technical estimation
EXPERIENCE & REQUIREMENTS
Must-Have
▸ 5-6 years of hands-on Data Engineering experience in production environments
▸ Deep expertise in Azure ecosystem — ADLS Gen 2, ADF, Azure SQL Database, and Delta Lake
▸ Strong command of PySpark, Python, and SQL for large-scale data transformation
▸ Proven track record building and operating Medallion Architecture (Bronze/Silver/Gold) on Azure
▸ Experience designing data warehouses and data lake solutions at enterprise scale
▸ Familiarity with CI/CD pipelines, version control (Git), and DevOps practices in a data context
Nice-to-Have
▸ Exposure to AWS or GCP in addition to Azure
▸ Experience supporting or building Al/ML data pipelines and feature stores
▸ Knowledge of streaming frameworks — Kafka, Event Hubs, or Spark Structured Streaming
▸ Azure certifications (DP-203, AZ-900 or equivalent)
Skills Required
- 5-6 years of hands-on Data Engineering experience in production environments
- Deep expertise in Azure ecosystem: ADLS Gen2, ADF, Azure SQL Database, Delta Lake
- Strong command of PySpark, Python, and SQL for large-scale data transformation
- Proven track record building and operating Medallion Architecture (Bronze/Silver/Gold) on Azure
- Experience designing data warehouses and data lake solutions at enterprise scale
- Familiarity with CI/CD pipelines, version control (Git), and DevOps practices in a data context
- Architect and manage scalable ETL/ELT pipelines and design ADF pipelines, models, and schemas
- Implement data quality checks, validation frameworks, lineage tracking, and enforce security controls
- Exposure to AWS or GCP
- Experience supporting or building AI/ML data pipelines and feature stores
- Knowledge of streaming frameworks: Kafka, Event Hubs, or Spark Structured Streaming
- Azure certifications (DP-203, AZ-900 or equivalent)
What We Do
Trickle Up is a Digital marketing and HR service provider for businesses across the UK. With operations based in Pakistan, our team has a pool of highly talented individuals, coming from diverse fields and specialties, capable of providing your business with tailored solutions designed to meet your unique needs. With Trickle Up as your digital partner, you can cut down on hiring costs, optimise workflow, boost efficiency and overall upscale your business without having to worry about the complexities of managing in-house teams. __________________________________________________________________________________ What makes Trickle Up truly stand out is our commitment to making your vision a reality. With a wealth of expertise from our diverse team, we prioritise efficiency, transparency and innovation. Our approach is client-driven and result-centric, ensuring your business stays ahead in the competitive landscape.
