DATAECONOMY Jobs

Databricks Engineer

DATAECONOMY

Databricks Engineer

Posted 14 Days Ago

Be an Early Applicant

Hyderabad, Telangana, IND

In-Office

Senior level

Big Data • Cloud • Analytics • Consulting

The Role

Design, develop, and optimize scalable Databricks data engineering solutions. Build ETL/ELT pipelines, manage clusters, implement Lakehouse, Delta Lake, Delta Live Tables, Medallion Architecture, and Unity Catalog. Optimize Spark jobs for performance and cost, monitor pipelines, resolve data quality issues, and apply DevOps and CI/CD practices while collaborating with architects, analysts, and stakeholders.

Summary Generated by Built In

Job Title: Databricks Engineer
Experience Level: 5 - 10 years
Location: Hyderabad / Pune

Qualification: BTech/MTech/MCA
Mode of Work: Hybrid

We are seeking a skilled and motivated Databricks Engineer to design, develop, and optimize scalable data solutions on the Databricks platform. The ideal candidate will have strong expertise in Apache Spark, Databricks architecture, ETL development, and modern data lakehouse concepts. This role will be responsible for building robust data pipelines, implementing best practices in data engineering, and supporting analytics and machine learning workloads.

Key Responsibilities:

Design, develop, and maintain scalable data engineering solutions using Databricks and Apache Spark.

Create and deploy custom ETL/ELT pipelines to ingest, transform, and process large volumes of structured and unstructured data.

Configure, manage, and optimize Databricks clusters for performance, scalability, and cost efficiency.

Implement and manage Lakehouse architectures using Databricks best practices.

Develop and maintain data workflows utilizing Delta Lake and Delta Live Tables (DLT).

Design and implement Medallion Architecture (Bronze, Silver, Gold layers) for reliable and scalable data processing.

Configure and govern data assets using Unity Catalog, ensuring data security, access control, and compliance.

Collaborate with data architects, analysts, and business stakeholders to translate business requirements into technical solutions.

Monitor data pipelines and troubleshoot performance or data quality issues.

Optimize Spark jobs and workflows to improve processing efficiency and reduce execution time.

Follow DevOps and CI/CD best practices for deployment and release management.

Requirements

5-10 years of experience in Data Engineering or Big Data technologies.

Strong understanding of Databricks architecture and core platform capabilities.

Hands-on experience in setting up and managing Databricks clusters.

Strong expertise in Apache Spark (PySpark/Scala Spark).

Experience developing and deploying custom ETL pipelines in Databricks.

Experience implementing Medallion Architecture within Databricks.

Hands-on experience with Delta Live Tables (DLT).

Strong understanding and implementation experience with Unity Catalog.

Experience working with Delta Lake and Lakehouse architecture principles.

Knowledge of data modeling, data quality, and performance tuning techniques.

Familiarity with version control tools such as Git.

Benefits

Comprehensive Medical Coverage:
Health insurance of INR 5.0 Lakhs for you and your family (up to 6 members), ensuring complete peace of mind.
Robust Protection Plans:
Group Personal Accident Insurance and Group Term Life Insurance to safeguard you and your loved ones.
Retirement Benefits:
PF and Gratuity provided as per standard government regulations.
Flexible Work Options:
Enjoy hybrid work arrangements & flexible working hours
Generous Leave Policy:
21 days of annual leave, in addition to 10 company-declared holidays.
Employee Well-being Spaces:
Access to a dedicated break-out area with round-the-clock refreshments for relaxation and rejuvenation.

Skills Required

5-10 years of experience in Data Engineering or Big Data technologies
BTech, MTech, or MCA degree
Strong understanding of Databricks architecture and core platform capabilities
Hands-on experience setting up and managing Databricks clusters
Strong expertise in Apache Spark, including PySpark or Scala Spark
Experience developing and deploying custom ETL pipelines in Databricks
Experience implementing Medallion Architecture within Databricks
Hands-on experience with Delta Live Tables
Strong understanding and implementation experience with Unity Catalog
Experience with Delta Lake and Lakehouse architecture principles
Knowledge of data modeling, data quality, and performance tuning techniques
Familiarity with version control tools such as Git

View all jobs at DATAECONOMY

View DATAECONOMY Profile

Report Job

Am I A Good Fit?

beta

Get Personalized Job Insights.

Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company

414 Employees

Year Founded: 2018

What We Do

DATAECONOMY is a global, cloud-first data and AI consultancy delivering enterprise-grade solutions through an innovative intellectual-property suite. Its work spans data and BI platform modernization, self-service AI, data mesh and fabric, master data management, governance, cloud enablement, digital engineering, knowledge graphs, and machine lakes supporting cybersecurity and financial-crime use cases for enterprise clients.