Software Developer - ETL - Senior
for our direct client Province of Ontario. The position details are given below:RQ11258 - Software Developer - ETL - Senior
10+ years experience in:
· Designing and developing scalable Medallion Data Lakehouse architectures.
· Expertise in data ingestion, transformation, and curation using Delta Lake and Databricks.
· Experience integrating structured and unstructured data sources into star/snowflake schemas.
· Building, automating, and optimizing complex ETL/ELT pipelines using Azure Data Factory (ADF), Databricks (PySpark, SQL, Delta Live Tables), and dbt.
· Implementing orchestrated workflows and job scheduling in Azure environments.
· Strong knowledge of relational (SQL Server, Synapse, PostgreSQL) and dimensional modeling.
· Advanced SQL query optimization, indexing, partitioning, and data replication strategies.
· Experience with Apache Spark, Delta Lake, and distributed computing frameworks in Azure Databricks.
· Working with Parquet, ORC, and JSON formats for optimized storage and retrieval.
· Deep expertise in Azure Data Lake Storage (ADLS), Azure Synapse Analytics, Azure SQL, Event Hubs, and Azure Functions.
· Strong understanding of cloud security, RBAC, and data governance.
· Proficiency in Python (PySpark), SQL, and PowerShell for data engineering workflows.
· Experience with CI/CD automation (Azure DevOps, GitHub Actions) for data pipelines.
· Implementing data lineage, cataloging, metadata management, and data quality frameworks.
· Experience with Unity Catalog for managing permissions in Databricks environments.
· Expertise in Power BI (DAX, data modeling, performance tuning).
· Experience in integrating Power BI with Azure Synapse and Databricks SQL Warehouses.
· Familiarity with MLflow, AutoML, and embedding AI-driven insights into data pipelines.
Skills Required
- 10+ years designing and developing Medallion Data Lakehouse architectures
- Expertise in Delta Lake and Databricks
- Integration of structured and unstructured data into star/snowflake schemas
- Building, automating, and optimizing ETL/ELT pipelines using Azure Data Factory, Databricks (PySpark, SQL, Delta Live Tables), and dbt
- Implementing orchestrated workflows and job scheduling in Azure environments
- Strong knowledge of relational databases (SQL Server, Synapse, PostgreSQL) and dimensional modeling
- Advanced SQL query optimization, indexing, partitioning, and data replication strategies
- Experience with Apache Spark, Delta Lake, and distributed computing frameworks in Azure Databricks
- Experience working with Parquet, ORC, and JSON file formats
- Deep expertise in Azure Data Lake Storage (ADLS), Azure Synapse Analytics, and Azure SQL
- Experience with Event Hubs and Azure Functions
- Strong understanding of cloud security, RBAC, and data governance
- Proficiency in Python (PySpark), SQL, and PowerShell
- Experience with CI/CD automation for data pipelines (Azure DevOps, GitHub Actions)
- Implementing data lineage, cataloging, metadata management, and data quality frameworks
- Experience with Unity Catalog for permissions in Databricks
- Expertise in Power BI including DAX, data modeling, and performance tuning
- Experience integrating Power BI with Azure Synapse and Databricks SQL Warehouses
- Familiarity with MLflow, AutoML, and embedding AI-driven insights into data pipelines
What We Do
ITProFound Inc. is a global technology services and solutions provider specializing in software development, IT consulting, and staff augmentation. They deliver end-to-end business practice solutions, including ERP, cloud migrations, and data analytics, across various industries.
