This role is for one of the Weekday's clients
Min Experience: 4+ years
Location: Bengaluru, Pune
JobType: full-time
We are looking for a highly skilled Databricks Engineer to design, develop, and optimize data pipelines and analytics solutions. The ideal candidate will have strong expertise in Databricks, PySpark, and SQL, along with experience in creating scalable data solutions across cloud platforms.
Requirements
Key Responsibilities
- Design and develop data pipelines and ETL workflows using Databricks and PySpark.
- Create and enhance SQL queries for data transformation and analytical purposes.
- Work closely with data architects, analysts, and business stakeholders to deliver high-quality data solutions.
- Maintain data quality, governance, and compliance throughout all processes.
- Integrate data from diverse sources and effectively handle large-scale datasets.
- Diagnose issues and improve the performance of data workflows and queries.
Primary Skills (Must-Have)
- Strong proficiency in Databricks, including Delta Lake and machine learning capabilities.
- Advanced expertise in PySpark for distributed data processing.
- Excellent SQL skills for data manipulation and analytics.
Good-to-Have Skills
- Experience with cloud services on Azure (ADF, Synapse, Purview, Fabric).
- Knowledge of AWS services such as Glue, Lambda, and Step Functions.
- Familiarity with workflow orchestration tools like Airflow.
- Experience using data transformation and integration tools including DBT, Fivetran, and Informatica.
- Understanding of streaming and messaging platforms like Kafka.
- BI and data visualization skills with Power BI.
- Experience with data governance tools such as Collibra and Alation.
- Knowledge of Google Cloud Platform’s BigQuery.
Qualifications
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related discipline.
- Demonstrated experience in constructing and managing data pipelines within cloud environments.
- Strong analytical and problem-solving abilities.
- Excellent communication and teamwork skills.
Nice to Have
- Experience operating in multi-cloud environments such as Azure, AWS, and GCP.
- Familiarity with modern data stack technologies and data governance frameworks.
Must-have skills
Databricks, PySpark, Microsoft Azure
Good-to-have skillsSQL
Skills Required
- Databricks (including Delta Lake and ML capabilities)
- PySpark for distributed data processing
- SQL for data transformation and analytics
- Microsoft Azure cloud experience
- Minimum 4+ years relevant experience
- Bachelor's or Master's degree in Computer Science, Data Engineering, or related discipline
- Experience constructing and managing data pipelines in cloud environments
- Maintain data quality, governance, and compliance
- Strong analytical and problem-solving abilities
- Excellent communication and teamwork skills
- Azure Data Factory, Synapse, Purview, Fabric
- AWS services such as Glue, Lambda, Step Functions
- Workflow orchestration tools like Airflow
- Data transformation/integration tools: DBT, Fivetran, Informatica
- Streaming/messaging platforms such as Kafka
- BI and visualization with Power BI
- Data governance tools such as Collibra, Alation
- Familiarity with Google Cloud BigQuery
- Experience operating in multi-cloud environments (Azure, AWS, GCP)
What We Do
Weekday is an AI-powered recruitment platform that helps startups hire top-tier engineering and product talent. By leveraging a massive database of white-collar professionals and advanced outreach tools, the company streamlines the hiring process through automated sourcing, AI-driven resume screening, and white-glove contingency services. Their mission is to modernize recruitment by enabling companies to discover and engage passive candidates efficiently, ensuring high-quality hires for critical roles.
