The Role
Design, build, automate, optimize, and support complex cloud data pipelines and ETL/ELT processes. Develop unified Databricks data platforms, data lakes, lakehouses, warehouses, and dimensional models. Integrate and migrate data across Oracle, Azure, Databricks, and Microsoft Fabric. Create Azure Data Factory pipelines, implement CI/CD and DataOps practices, and perform performance tuning, monitoring, troubleshooting, and data provisioning for large-scale enterprise environments.
Summary Generated by Built In
Responsibilities:
- Designing and developing data pipelines from source to end
user
- Optimizing data pipelines
General Skills:
- Experience with Cloud data platforms, data management and
data exchange tools and technologies
- Experience with Commercial and opensource data and database
development and management and specializing in data storage setting and
managing cloud Data As a Service (DaaS). application Database as a Service
(DBaaS), Data Warehouses as a Service (DWaaS), and other storage platforms
(both in the cloud and on-premise).
- Experience with Data pipeline and workflow development,
orchestration, deployment, and automation and specializing in programming and
pipelines to create and managing the dataflow and movement of data.
- Experience with Cloud data engineer must be familiar and
experienced with different programming languages and be able to integrate with
many different platforms to create data pipelines, automate tasks, and write
scripts.
- Experience in DataOPS principle, best practice, and
implementation and Agile project development and deployment
- Experience in Continuous Integration/Continuous
Development/Deployment (CI/CD) and Data provisioning Automation
- Experience with digital product, data analysis, data
exchange, data provisioning, and data security
- Extensive expert experience in designing/developing and
implementing data conversation and migration of VLD (Very large Data) of Online
analytical processing (OLAP) and online transaction processing (OLTP)
environments to Cloud Software-as-a-service (SaaS), Platform-as-a-service
(PaaS) and Infrastructure-as-a-service (IaaS) environments.
- Experience in design, development and implementation of
fact/dimension model, data mapping, data warehouse, data lake and data
lakehouse for enterprise
- Experience managing Cloud Data services for project
delivery, including storage, repositories, Data Lake, Data Lakehouse, key
vault, virtual machine, disk, etc.
- Experience with structured, semi-structured, unstructured
data collection, ingestion, provisioning and exchange technological development
of enterprise data warehouse and data lake and data lakehouse solutions and
operational support.
- Experience with DataOPS performance monitoring and tuning
- Excellent analytical, problem-solving and decision-making
skills; verbal and written communication skills; presentation skills;
interpersonal and negotiation skills
- A team player with a track record for meeting deadlines
Requirements
Experience and Skill Set Requirements:
Must Haves:
- Design,
build, automate and optimize complex data ETL/ELT processes using
Databricks, Delta Live Tables (DLT) and equivalent.
- Development
of unified data platforms using Databricks and Unity catalog or its
equivalent.
- Relational
databases (Oracle, MySQL, SQL Server), data modelling (relational &
dimensional), advanced SQL, query optimization, data replication,
administration.
- Advanced
SQL skills (PL/SQL, TSQL).
- Big
Data processing Frameworks (PySpark).
- Proficient
in creating Azure Data Factory pipelines for copy activity and custom data
pipelines in Databricks with CI CD (Azure DevOPS Git Integration) .
- Data
migration and data integration across different platforms Oracle to Azure
Data Lake, Oracle to Databricks, Oracle to Microsoft Fabric.
Skill Set Requirements:
Mandatory Skills and Experience (8+ years experience):
- Design,
build, automate and optimize complex data ETL/ELT processes using
Databricks, Delta Live Tables (DLT) and equivalent.
- Development
of unified data platforms using Databricks and Unity catalog or its
equivalent.
- Relational
databases (Oracle, MySQL, SQL Server), data modelling (relational &
dimensional), advanced SQL, query optimization, data replication,
administration.
- Advanced
SQL skills (PL/SQL, TSQL).
- Big
Data processing Frameworks (PySpark).
- Proficient
in creating Azure Data Factory pipelines for copy activity and custom data
pipelines in Databricks with CI CD (Azure DevOPS Git Integration) .
- Data
migration and data integration across different platforms Oracle to Azure
Data Lake, Oracle to Databricks, Oracle to Microsoft Fabric.
Desirable Skills and Experience:
- Data
lake design and development with focus on extracting, transforming data
coming from various data sources and loading into Lakehouse in medallion
architecture.
- Advanced
skills and hands-on experience with Oracle databases, Azure SQL Server and
Azure Data Factory.
- Cloud
technologies (Azure, Google, AWS).
- PL/SQL
for data extraction, transformation, and loading, initial setup and other
ETL experience management and support, such as troubleshooting,
performance tuning, failover and recovery.
- Scripting
languages (Python, Unix shell, Scala).
Skills Required
- 8+ years of experience in data engineering or closely related work
- Design, build, automate, and optimize complex ETL/ELT processes using Databricks and Delta Live Tables or equivalent
- Develop unified data platforms using Databricks and Unity Catalog or equivalent
- Experience with relational databases including Oracle, MySQL, and SQL Server
- Experience with relational and dimensional data modeling
- Advanced SQL, including PL/SQL and T-SQL
- Experience with query optimization, data replication, and database administration
- Experience with big data processing frameworks, including PySpark
- Create Azure Data Factory pipelines for copy activities and custom Databricks data pipelines
- Implement CI/CD using Azure DevOps and Git integration
- Perform data migration and integration across Oracle, Azure Data Lake, Databricks, and Microsoft Fabric
- Design and develop data lakes using medallion architecture and load data into lakehouses
- Hands-on experience with Oracle databases, Azure SQL Server, and Azure Data Factory
- Experience with cloud technologies including Azure, Google Cloud, and AWS
- Use PL/SQL for data extraction, transformation, loading, ETL support, troubleshooting, performance tuning, failover, and recovery
- Experience with Python, Unix shell, or Scala scripting
- Experience with DataOps principles, implementation, monitoring, and tuning
- Strong analytical, problem-solving, decision-making, communication, presentation, interpersonal, and negotiation skills
- Track record of teamwork and meeting deadlines
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Maarut Inc. is a Canadian company specializing in IT services, technology staffing, and software development, dedicated to assisting businesses with digital transformation and solving business challenges through technology.
