Senior Data Engineer - Azure Databricks

Reposted 4 Days Ago
Be an Early Applicant
Hyderabad, Telangana, IND
In-Office
Senior level
Database • Analytics
The Role
Design, build, and maintain scalable ETL/ELT data pipelines and platforms on Azure Databricks. Ingest and transform structured and unstructured data using PySpark and Spark SQL, manage ADLS Gen2 storage, ensure data quality and performance, and collaborate with data science and BI teams to support analytics and reporting.
Summary Generated by Built In
Company Description

Blend is a premier AI services provider, committed to co-creating meaningful impact for its clients through the power of data science, AI, technology, and people. We help organisations solve complex business challenges by combining deep domain understanding with modern data and AI capabilities.

Our teams work across strategy, analytics, engineering, and product delivery to create scalable, high-value solutions that improve decision-making, efficiency, and growth.

Job Description

We are looking for a skilled Data Engineer with 5+ years of experience to design, build, and maintain scalable data pipelines and platforms using Azure and Databricks. The ideal candidate will work closely with analytics, data science, and business teams to deliver reliable, high-performance data solutions that enable data-driven decision-making.

Key Responsibilities

  • Design, develop, and optimize end-to-end data pipelines using Azure Databricks
  • Build and maintain scalable ETL/ELT frameworks for structured and unstructured data
  • Ingest data from multiple sources including APIs, databases, files, and streaming systems
  • Work with Azure Data Lake Storage (ADLS Gen2) for efficient data storage and retrieval
  • Develop data transformation logic using PySpark and Spark SQL
  • Ensure data quality, reliability, and performance across pipelines
  • Implement data modeling concepts to support analytics and reporting use cases
  • Collaborate with Data Scientists, BI teams, and business stakeholders to understand data requirements
  • Monitor, troubleshoot, and optimize Databricks jobs for performance and cost efficiency
  • Apply best practices around security, governance, and compliance within Azure
  • Participate in code reviews, documentation, and continuous improvement initiatives

Qualifications

  • 5+ years of hands-on experience as a Data Engineer
  • Strong experience with Azure Databricks
  • Proficiency in PySpark, Spark SQL, and Python
  • Experience working with Azure Data Lake Storage (ADLS Gen2)
  • Solid understanding of data warehousing and data modeling concepts
  • Experience building batch and/or streaming data pipelines
  • Familiarity with Azure Data Factory or similar orchestration tools
  • Strong SQL skills and experience with relational databases
  • Experience using Git or similar version control systems
  • Strong analytical, problem-solving, and communication skills

Additional Information

  • Experience with Delta Lake
  • Knowledge of streaming technologies (Kafka, Azure Event Hubs, Spark Structured Streaming)
  • Exposure to CI/CD pipelines using Azure DevOps
  • Experience working in Agile/Scrum environments
  • Understanding of cloud cost optimization strategies

Skills Required

  • 5+ years of hands-on experience as a Data Engineer
  • Strong experience with Azure Databricks
  • Proficiency in PySpark
  • Proficiency in Spark SQL
  • Proficiency in Python
  • Experience with Azure Data Lake Storage (ADLS Gen2)
  • Solid understanding of data warehousing and data modeling concepts
  • Experience building batch and/or streaming data pipelines
  • Familiarity with Azure Data Factory or similar orchestration tools
  • Strong SQL skills and experience with relational databases
  • Experience using Git or similar version control systems
  • Apply best practices around security, governance, and compliance within Azure
  • Strong analytical, problem-solving, and communication skills
  • Experience with Delta Lake
  • Knowledge of streaming technologies (Kafka, Azure Event Hubs, Spark Structured Streaming)
  • Exposure to CI/CD pipelines using Azure DevOps
  • Experience working in Agile/Scrum environments
  • Understanding of cloud cost optimization strategies

Blend360 Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Blend360 and has not been reviewed or approved by Blend360.

  • Fair & Transparent Compensation Pay is considered fair-to-good by many, and public salary postings for common data roles indicate competitive packages in numerous markets. Feedback suggests overall company sentiment aligns with acceptable compensation relative to peers in consulting and analytics.
  • Flexible Benefits Flexible and remote/hybrid work arrangements are consistently highlighted in official materials and role descriptions. Feedback suggests flexibility is a meaningful part of the total rewards experience.
  • Retirement Support A 401(k) with company match is part of the core package. Feedback suggests retirement offerings are standard and contribute to a complete benefits set.

Blend360 Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Columbia, MD
390 Employees
Year Founded: 2016

What We Do

Our Vision is to build a company of world-class people that helps our clients optimize business performance through data, technology and analytics. Blend360 has two divisions: Data Science Solutions: We work at the intersection of data, technology and analytics. Talent Solutions: We live and breathe the digital and talent marketplace.

Similar Jobs

Optum Logo Optum

Senior Data Engineer

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office
Hyderabad, Telangana, IND
160000 Employees

Wise Logo Wise

Head of SOX Testing

Fintech • Mobile • Payments • Software • Financial Services
Hybrid
Hyderabad, Telangana, IND
9000 Employees

Capco Logo Capco

Liquidity - Design Analyst

Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Remote or Hybrid
India
6000 Employees
Hybrid
Hyderabad, Telangana, IND
205000 Employees

Similar Companies Hiring

Northslope Thumbnail
Artificial Intelligence • Information Technology • Software • Analytics • Consulting • Generative AI
London, GB
100 Employees
Scotch Thumbnail
Artificial Intelligence • eCommerce • Fintech • Payments • Retail • Software • Analytics
US
35 Employees
Milestone Systems Thumbnail
Artificial Intelligence • Security • Software • Analytics • Big Data Analytics
Lake Oswego, OR
1500 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account