This role is for one of Weekday’s clients
Salary range: Rs 400000 - Rs 2600000 (ie INR 4 - 26 LPA)
Min Experience: 6+ years
Location: Bengaluru, Hyderabad, Chennai, Tamil Nadu, India
JobType: full-time
We are looking for an experienced Azure Data Engineer with 6–15 years of hands-on experience in designing, developing, and maintaining scalable data engineering solutions on Microsoft Azure. The ideal candidate will have strong expertise in Azure Data Factory, Azure Databricks, PySpark/Scala, Azure SQL, ADLS Gen2, and CI/CD pipelines.
You will be responsible for building robust data pipelines, developing data transformation workflows, integrating cloud-based data services, and ensuring data platforms are secure, scalable, reliable, and optimized for performance.
RequirementsKey Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory.
- Build and optimize large-scale data processing solutions using Azure Databricks and PySpark/Scala.
- Develop data ingestion and transformation workflows across structured and unstructured data sources.
- Work extensively with Azure Data Lake Storage Gen2 (ADLS Gen2) for data storage, organization, and processing.
- Develop and optimize solutions using Azure SQL, T-SQL, and Stored Procedures.
- Design and implement integrations and event-driven workflows using Azure Logic Apps.
- Implement secure access and credential management using Azure Key Vault.
- Create and maintain CI/CD pipelines for data engineering applications, notebooks, workflows, and infrastructure.
- Monitor data pipelines and troubleshoot performance, reliability, and production issues.
- Collaborate with data architects, software engineers, analysts, and business stakeholders to translate requirements into scalable technical solutions.
- Follow best practices for data security, governance, performance optimization, and cloud architecture.
- Participate in code reviews, technical documentation, deployment planning, and production support activities.
- 6–15 years of professional experience in data engineering, with significant experience working on Microsoft Azure.
- Strong hands-on expertise with Azure Data Factory (ADF).
- Strong experience with Azure Databricks.
- Proficiency in PySpark and/or Scala for large-scale data processing.
- Strong knowledge of Azure Data Lake Storage Gen2 (ADLS Gen2).
- Hands-on experience with Azure SQL and T-SQL.
- Strong experience developing and optimizing Stored Procedures.
- Experience designing and implementing CI/CD pipelines for data engineering workloads.
- Practical experience with Azure Logic Apps and cloud-based integrations.
- Experience using Azure Key Vault for secrets, credentials, and secure configuration management.
- Strong understanding of data engineering principles, ETL/ELT architecture, data transformation, and distributed processing.
- Experience with Azure Synapse Analytics.
- Hands-on experience with Azure DevOps.
- Familiarity with Azure Log Analytics and cloud monitoring solutions.
- Experience with infrastructure creation and automation using Bicep, Terraform, or Ansible.
- Understanding of Infrastructure as Code (IaC) practices.
- Experience working in Agile/Scrum development environments.
- Knowledge of data security, governance, and Azure cloud best practices.
Skills Required
- 6-15 years of professional data engineering experience on Microsoft Azure
- Azure Data Factory (ADF) - design and develop ETL/ELT pipelines
- Azure Databricks for large-scale data processing
- PySpark (and/or Scala) for data transformation
- Azure Data Lake Storage Gen2 (ADLS Gen2) for data storage and processing
- Azure SQL and T-SQL development
- Designing and optimizing Stored Procedures
- Design and maintain CI/CD pipelines for data engineering workloads
- Azure Logic Apps for integrations and event-driven workflows
- Azure Key Vault for secrets and credential management
- Strong understanding of ETL/ELT architecture, data transformation, and distributed processing
- Monitoring, troubleshooting, and production support of data pipelines
- Azure Synapse Analytics
- Azure DevOps
- Azure Log Analytics and cloud monitoring
- Infrastructure automation using Bicep, Terraform, or Ansible (IaC)
- Experience working in Agile/Scrum environments
- Knowledge of data security, governance, and Azure best practices
What We Do
Weekday is an AI-powered recruitment platform that helps startups hire top-tier engineering and product talent. By leveraging a massive database of white-collar professionals and advanced outreach tools, the company streamlines the hiring process through automated sourcing, AI-driven resume screening, and white-glove contingency services. Their mission is to modernize recruitment by enabling companies to discover and engage passive candidates efficiently, ensuring high-quality hires for critical roles.
