Databricks Engineer - Senior/ Lead

Posted 2 Days Ago
Be an Early Applicant
Hiring Remotely in Hyderabad, Telangana, IND
In-Office or Remote
Senior level
Database • Analytics
The Role
Design, build, and optimize scalable data pipelines and ETL/ELT workflows using Azure Databricks, Python, SQL, PySpark, and Spark. Manage Delta Lake tables and integrate Azure Data Lake Storage. Work with structured and semi-structured data, optimize performance and cost, ensure data quality and governance, and support analytics and machine learning use cases. Collaborate with data scientists, analysts, and stakeholders while applying security, access control, monitoring, and modern data engineering practices.
Summary Generated by Built In
Company Description

Blend is a premier AI services provider, committed to co-creating meaningful impact for its clients through the power of data science, AI, technology, and people. With a mission to fuel bold visions, Blend tackles significant challenges by seamlessly aligning human expertise with artificial intelligence. The company is dedicated to unlocking value and fostering innovation for its clients by harnessing world-class people and data-driven strategy. We believe that the power of people and AI can have a meaningful impact on your world, creating more fulfilling work and projects for our people and clients. For more information, visit www.blend360.com.

Job Description

We are looking for an experienced Azure Databricks Engineer with strong hands-on expertise in Python, SQL, and Apache Spark to design, build, and optimize scalable data pipelines and analytics solutions on the Azure cloud platform. The ideal candidate should have experience working with large datasets, distributed data processing, and modern data engineering practices.  

Responsibilities 

  • Design, develop, and maintain scalable data pipelines using Azure Databricks 

  • Implement ETL/ELT workflows using PySpark, Spark SQL, and Python 

  • Optimize Spark jobs for performance, cost, and scalability 

  • Work with structured and semi-structured data (Parquet, Delta, JSON, CSV) 

  • Build and manage Delta Lake tables (ACID, time travel, schema evolution) 

  • Integrate Databricks with Azure Data Lake Storage (ADLS Gen2) 

  • Develop complex queries and transformations using SQL 

  • Collaborate with data scientists, analysts, and stakeholders to support analytics and ML use cases 

  • Ensure data quality, validation, and monitoring 

  • Follow best practices for security, access control, and governance in Azure 

Qualifications

  • 6+ years of experience in Data Engineering 

  • Strong hands-on experience with Azure Databricks 

  • Proficiency in Python for data processing 

  • Strong knowledge of SQL (joins, window functions, performance tuning) 

  • Hands-on experience with Apache Spark / PySpark 

  • Experience working with Delta Lake 

  • Knowledge of Azure Data Lake Storage (ADLS Gen2) 

  • Understanding of distributed computing concepts 

  • Experience with Git version control 

Additional Information

  • Experience with Azure Data Factory 

  • Exposure to CI/CD pipelines (Azure DevOps, GitHub Actions) 

  • Basic understanding of data modeling 

  • Familiarity with cloud security and RBAC in Azure 

  • Exposure to streaming data (Spark Structured Streaming, Event Hub, Kafka) 

Skills Required

  • 6+ years of experience in data engineering
  • Strong hands-on experience with Azure Databricks
  • Proficiency in Python for data processing
  • Strong knowledge of SQL, including joins, window functions, and performance tuning
  • Hands-on experience with Apache Spark and PySpark
  • Experience working with Delta Lake
  • Knowledge of Azure Data Lake Storage Gen2
  • Understanding of distributed computing concepts
  • Experience with Git version control
  • Experience with Azure Data Factory
  • Exposure to CI/CD pipelines using Azure DevOps or GitHub Actions
  • Basic understanding of data modeling
  • Familiarity with cloud security and role-based access control in Azure
  • Exposure to streaming data using Spark Structured Streaming, Event Hubs, or Kafka

Blend360 Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Blend360 and has not been reviewed or approved by Blend360.

  • Fair & Transparent Compensation — Pay is considered fair-to-good by many, and public salary postings for common data roles indicate competitive packages in numerous markets. Feedback suggests overall company sentiment aligns with acceptable compensation relative to peers in consulting and analytics.
  • Flexible Benefits — Flexible and remote/hybrid work arrangements are consistently highlighted in official materials and role descriptions. Feedback suggests flexibility is a meaningful part of the total rewards experience.
  • Retirement Support — A 401(k) with company match is part of the core package. Feedback suggests retirement offerings are standard and contribute to a complete benefits set.

Blend360 Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Columbia, MD
390 Employees
Year Founded: 2016

What We Do

Our Vision is to build a company of world-class people that helps our clients optimize business performance through data, technology and analytics. Blend360 has two divisions: Data Science Solutions: We work at the intersection of data, technology and analytics. Talent Solutions: We live and breathe the digital and talent marketplace.

Similar Jobs

CSC Logo CSC

Accountant

Fintech • Legal Tech • Software • Financial Services • Cybersecurity • Data Privacy
Remote or Hybrid
2 Locations
8500 Employees

Hilton Logo Hilton

F&B Associate

Software • Hospitality
Remote
India
121228 Employees

Hilton Logo Hilton

Assistant Manager Restaurant

Software • Hospitality
Remote
India
121228 Employees

Hilton Logo Hilton

Restaurant Manager

Software • Hospitality
Remote
India
121228 Employees

Similar Companies Hiring

Northslope Thumbnail
Artificial Intelligence • Information Technology • Software • Analytics • Consulting • Generative AI
London, GB
100 Employees
Scotch Thumbnail
Artificial Intelligence • eCommerce • Fintech • Payments • Retail • Software • Analytics
US
35 Employees
Milestone Systems Thumbnail
Artificial Intelligence • Security • Software • Analytics • Big Data Analytics
Lake Oswego, OR
1500 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account