Data Engineer

Posted 11 Days Ago
Be an Early Applicant
Dubai, ARE
In-Office
Senior level
Information Technology • Consulting
The Role
Build and support scalable Azure data solutions, including ingestion pipelines, SAP and non-SAP integrations, PySpark and SQL transformations, enterprise data models, and curated datasets. Develop Bronze, Silver, and Gold layers in Databricks, enable Power BI analytics, implement data quality and governance controls, monitor production pipelines, resolve incidents, and support deployments. Collaborate with analysts, BI developers, and architects to deliver trusted, performant data products.
Summary Generated by Built In
We are looking for a Data Engineer to build, enhance, and support scalable data solutions on Azure. The role involves developing data pipelines, integrating SAP and non-SAP source systems, implementing data transformations in Databricks, and delivering trusted datasets for reporting, analytics, and business intelligence.
The ideal candidate should have strong experience in Azure Data Factory, Databricks, SQL, and Python, along with a passion for solving complex data challenges.

Key Responsibilities

Data Ingestion & Integration
• Develop and maintain data ingestion pipelines using Azure Data Factory (ADF).
• Integrate data from SAP, databases, APIs, flat files, and cloud applications.
• Implement incremental and CDC-based data loading patterns.
• Troubleshoot data integration and pipeline issues.

Data Transformation & Processing
• Develop PySpark and SQL-based transformations in Azure Databricks.
• Build and maintain Bronze, Silver, and Gold layer datasets.
• Cleanse, validate, and enrich data to meet business requirements.
• Optimize jobs for performance, scalability, and cost efficiency.

Data Warehousing & Modeling
• Support implementation of enterprise data models and Common Data Models (CDM).
• Develop curated datasets for reporting and analytics.
• Ensure data consistency across multiple business domains.

Analytics Enablement
• Deliver trusted datasets for Power BI and self-service analytics.
• Work closely with Business Analysts, BI Developers, and Data Architects to understand data requirements.
• Support report and dashboard performance improvements.

Governance & Quality
• Implement data quality checks and validation frameworks.
• Adhere to data governance standards and security policies.
• Document data flows, business rules, and technical processes.

Support & Operations
• Monitor scheduled jobs and data pipelines.
• Investigate and resolve production incidents.
• Perform root cause analysis and implement corrective actions.
• Participate in deployment and release activities.


Requirements

Preferred Skills

·  SAP ECC / SAP S/4HANA data integration

·  REST API integrations

·  Event-driven architectures

·  Unity Catalog

·  Microsoft Fabric

·  Retail or Automotive domain experience

Experience

·  Overall 5-8 years of experience , 3-6 years of Data Engineering experience.

·  2+ years of experience with Azure Databricks.

·  2+ years of experience with Azure Data Factory.

·  Experience building enterprise-scale cloud data pipelines.


Technical Skills
• Azure Data Factory (ADF)
• Azure Databricks
• Python and PySpark
• SQL
• Azure Data Lake Storage (ADLS)
• Delta Lake

Data Engineering Concepts
• ETL / ELT development
• Data warehousing
• Medallion Architecture
• Incremental loading
• Data modeling
• Data quality management

Reporting & Analytics
• Power BI
• Dataset optimization
• Dimensional modeling

Version Control & DevOps
• Git
• Azure DevOps
• CI/CD concepts
Education
​Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.


Skills Required

  • 5-8 years of overall professional experience
  • 3-6 years of Data Engineering experience
  • At least 2 years of Azure Databricks experience
  • At least 2 years of Azure Data Factory experience
  • Experience building enterprise-scale cloud data pipelines
  • Proficiency with Azure Data Factory, Azure Databricks, Python, PySpark, SQL, Azure Data Lake Storage, and Delta Lake
  • Experience with ETL/ELT, data warehousing, medallion architecture, incremental loading, data modeling, and data quality management
  • Experience with Power BI, dimensional modeling, dataset optimization, Git, Azure DevOps, and CI/CD concepts
  • Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field
  • SAP ECC or SAP S/4HANA data integration
  • REST API integrations
  • Event-driven architectures
  • Unity Catalog
  • Microsoft Fabric
  • Retail or Automotive domain experience
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Dubai
242 Employees
Year Founded: 2016

What We Do

TechCarrot is a global IT services provider and digital-transformation partner, delivering cost-effective, customer-centric digital solutions to clients worldwide. Its 200-plus technology specialists provide services across multiple industries, aiming to create seamless customer experiences and support client growth. Founded in 2016 and headquartered in Dubai, the company serves the Middle East and Africa through UAE operations, APAC through India, and has also expanded into the United States and the Netherlands.

Similar Jobs

Datamatics Technologies Logo Datamatics Technologies

Data Engineer

Artificial Intelligence • Information Technology • Software • Analytics
In-Office
Dubai, ARE
65 Employees
In-Office
Dubai, ARE
353 Employees
In-Office
Dubai, ARE
353 Employees

techcarrot FZ LLC Logo techcarrot FZ LLC

Data Engineer

Information Technology • Consulting
In-Office
Dubai, ARE
242 Employees

Similar Companies Hiring

NODA AI Thumbnail
Artificial Intelligence • Information Technology • Software • Cybersecurity • Defense • Defense Technology
Austin, TX
54 Employees
Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
567 Employees
RapidScale Thumbnail
Cloud • Information Technology • Consulting
Raleigh, NC
600 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account