This role is for one of the Weekday's clients
Min Experience: 4+ years
Location: Bengaluru, Pune
JobType: full-time
We are looking for a highly skilled Databricks Engineer to design, develop, and optimize data pipelines and analytics solutions. The ideal candidate will have strong expertise in Databricks, PySpark, and SQL, along with experience in creating scalable data solutions across cloud platforms.
Requirements
Key Responsibilities
- Design and develop data pipelines and ETL workflows using Databricks and PySpark.
- Create and enhance SQL queries for data transformation and analytical purposes.
- Work closely with data architects, analysts, and business stakeholders to deliver high-quality data solutions.
- Maintain data quality, governance, and compliance throughout all processes.
- Integrate data from diverse sources and effectively handle large-scale datasets.
- Diagnose issues and improve the performance of data workflows and queries.
Primary Skills (Must-Have)
- Strong proficiency in Databricks, including Delta Lake and machine learning capabilities.
- Advanced expertise in PySpark for distributed data processing.
- Excellent SQL skills for data manipulation and analytics.
Good-to-Have Skills
- Experience with cloud services on Azure (ADF, Synapse, Purview, Fabric).
- Knowledge of AWS services such as Glue, Lambda, and Step Functions.
- Familiarity with workflow orchestration tools like Airflow.
- Experience using data transformation and integration tools including DBT, Fivetran, and Informatica.
- Understanding of streaming and messaging platforms like Kafka.
- BI and data visualization skills with Power BI.
- Experience with data governance tools such as Collibra and Alation.
- Knowledge of Google Cloud Platform’s BigQuery.
Qualifications
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related discipline.
- Demonstrated experience in constructing and managing data pipelines within cloud environments.
- Strong analytical and problem-solving abilities.
- Excellent communication and teamwork skills.
Nice to Have
- Experience operating in multi-cloud environments such as Azure, AWS, and GCP.
- Familiarity with modern data stack technologies and data governance frameworks.
Must-have skills
Databricks, PySpark, Microsoft Azure
Good-to-have skillsSQL
Skills Required
- Minimum 4+ years experience in data engineering or related roles
- Proficiency with Databricks including Delta Lake and ML capabilities
- Advanced expertise in PySpark for distributed data processing
- Excellent SQL skills for data transformation and analytics
- Experience designing and developing data pipelines and ETL workflows in cloud environments
- Experience maintaining data quality, governance, and compliance
- Bachelor's or Master's degree in Computer Science, Data Engineering, or related discipline
- Strong analytical, problem-solving, communication, and teamwork skills
- Experience with Azure cloud services (ADF, Synapse, Purview, Fabric)
- Familiarity with AWS services (Glue, Lambda, Step Functions)
- Familiarity with workflow orchestration tools such as Airflow
- Experience with DBT, Fivetran, or Informatica
- Knowledge of streaming platforms like Kafka
- BI and visualization skills with Power BI
- Experience with data governance tools such as Collibra or Alation
- Familiarity with GCP BigQuery and multi-cloud environments
What We Do
Weekday is an AI-powered recruitment platform that helps startups hire top-tier engineering and product talent. By leveraging a massive database of white-collar professionals and advanced outreach tools, the company streamlines the hiring process through automated sourcing, AI-driven resume screening, and white-glove contingency services. Their mission is to modernize recruitment by enabling companies to discover and engage passive candidates efficiently, ensuring high-quality hires for critical roles.
