We are seeking an experienced Senior DataOps Engineer to implementation, and maintenance of our data infrastructure and pipelines. The ideal candidate will have a strong background in data engineering and DevOps principles, with a passion for automation, quality, and governance. You will act as a bridge between technical and business teams, ensuring our data platform is not only efficient and scalable but also reliable and compliant. This role is crucial for enabling data-driven decisions and accelerating the development lifecycle for all data initiatives.
- Design & Implement Data Platforms: Design, develop, and maintain robust, scalable data pipelines and ETL processes, with a focus on automation and operational excellence.
- Ensure Data Quality and Governance: Implement automated data validation, quality checks, and monitoring systems to ensure data accuracy, consistency, and reliability.
- Manage CI/CD for Data: Develop, own and optimize the CI/CD pipelines for data engineering workflows, including automated testing and deployment of data transformations and schema changes.
- Architect & Implement IaC: Use Infrastructure as Code (IaC) with Terraform to manage data infrastructure across various cloud platforms (Azure, AWS, GCP).
- Performance & Optimization: Proactively monitor and optimize query performance, data storage, and resource utilization to manage costs and enhance efficiency.
- Provide Technical Leadership: Mentor junior engineers, provide technical leadership, and champion best practices in data engineering and DataOps methodologies.
- Collaborate with Stakeholders: Manage communication with technical and business teams to understand requirements, assess technical and business impact, and deliver effective data solutions.
- Strategic Design: Possess the ability to see the big picture in architectural design, conduct thorough risk assessments, and plan for future scalability and growth.
Requirements
- Experience: 3+ years of experience in data engineering, data warehousing, and ETL processes, with a significant portion of that time focused on DataOps or a similar operational role.
- Platform Expertise: Strong experience with data platforms such as Databricks and exposure to multiple cloud environments (Azure, AWS, or GCP).
- Data Processing: Extensive experience with Apache Spark for large-scale data processing.
- Orchestration: Proficiency with data orchestration tools like Azure Data Factory (ADF), Apache Airflow, or similar.
- CI/CD & Version Control: Expert-level knowledge of version control (Git) and experience with CI/CD pipelines (GitLab CI/CD, GitHub Actions).
- IaC: Extensive hands-on experience with Terraform.
- Programming: Strong programming skills in Python and advanced proficiency in SQL.
- Data Modeling: Experience with data modeling tools and methodologies, specifically with dbt (data build tool).
- AI & ML: Experience with AI-related technologies like Retrieval-Augmented Generation (RAG) and frameworks such as LangChain.
- Data Observability: Hands-on experience with data quality and observability tools such as Great Expectations, Monte Carlo, or Soda Core.
- Data Governance: Familiarity with data governance principles, compliance requirements, and data catalogs (e.g., Unity Catalog).
- Streaming Technologies: Experience with stream processing technologies like Kafka or Flink.
- BI Tools: Experience with data visualization tools (e.g. Power BI).
- Containerization: Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes).
Top Skills
What We Do
Makro PRO is an exciting new digital venture by the iconic Makro. Our proud purpose is to build a technology platform that will help make business possible for restaurant owners, hotels, and independent retailers, and open the door for sellers. Makro PRO brings together the best talent across multi-nationals to transform the B2B marketplace ecosystem. We welcome bold, energetic, and thoughtful people who share our belief in collaboration, diversity, excellence, and putting customers at the heart of our work.
