Datavail

Data Engineer

Posted Yesterday

Be an Early Applicant

Bengaluru, Bengaluru Urban, Karnataka, IND

Hybrid

Senior level

Database

The Role

Design and build enterprise data warehouse assets, including dimensional models, conformed dimensions, fact tables, and scalable Azure Databricks pipelines. Develop PySpark, Spark SQL, and Delta Lake transformations; implement incremental processing and CDC; translate SQL Server logic; optimize distributed workloads; and reconcile data for accuracy and auditability. Collaborate with Databricks leadership to deliver production data assets.

Summary Generated by Built In

Title: Senior Technical Specialist - PBI Databricks

Location: Remote

Duration: 12+ Months

Education: Any Degree

Exp Level: 8+ Years

Job Responsibilities:

Design and build conformed dimensions and transactional fact tables for enterprise reporting.
Develop and maintain data pipelines on Azure Databricks using PySpark, Spark SQL, and Delta Lake.
Implement best practices in dimensional modeling, including:
Star schemas
Conformed dimensions
Surrogate and deterministic hash keys (e.g., SHA-256)
Transactional fact design
Translate existing SQL Server and stored procedure logic into efficient Spark-based transformations.
Implement incremental data processing and change data capture (CDC) pipelines.
Ensure data reconciliation with source systems to meet accuracy and audit standards.
Collaborate closely with a senior Databricks lead to deliver core data assets within defined timelines.

Required Skills:

8+ years of experience in data engineering, with strong focus on Azure Databricks.
Proficiency in:

PySpark
Spark SQL
Delta Lake

Deep expertise in dimensional modelling and data warehousing concepts.
Experience with:

Building scalable production pipelines
Incremental data loads and CDC patterns
Performance optimization in distributed systems

Strong understanding of relational databases and ability to interpret complex SQL/stored procedures

Good to have

Experience with:

Lakeflow / Spark Declarative Pipelines or dbt
Unity Catalog
Exposure to financial/General Ledger data models
Familiarity with JD Edwards systems, including:

F0902 (balances)
F0911 (journal details)

About UsDatavail is a leading provider of data management, application development, analytics, and cloud services, with more than 1,000 professionals helping clients build and manage applications and data via a world-class tech-enabled delivery platform and software solutions across all leading technologies. For more than 17 years, Datavail has worked with thousands of companies spanning different industries and sizes, and is an AWS Advanced Tier Consulting Partner, a Microsoft Solutions Partner for Data & AI and Digital & App Innovation (Azure), an Oracle Partner, and a MySQL Partner. About the Team

Datavail’s Data Management Services:

Datavail’s Data Management and Analytics practice is made up of experts who provide a variety of data services including initial consulting and development, designing and building complete data systems, as well as ongoing support and management of database, data warehouse, data lake, data integration, and virtualization and reporting environments. Datavail’s team is comprised of not just excellent BI & analytics consultants, but great people as well. Datavail’s data intelligence consultants are experienced, knowledgeable and certified in the best in breed BI and analytics software applications and technologies. We ascertain your business objectives, goals and requirements, assess your environment, and recommend the tools which best fit your unique situation. Our proven methodology can help your project succeed, regardless of stage. With the combination of a proven delivery model and top-notch experience ensures that Datavail will remain the Data Management experts on demand you desire. Datavail’s flexible and client focused services always add value to your organization.

Skills Required

Any degree
8+ years of data engineering experience
Strong Azure Databricks experience
Proficiency in PySpark
Proficiency in Spark SQL
Proficiency in Delta Lake
Expertise in dimensional modeling and data warehousing
Experience building scalable production pipelines
Experience with incremental data loads and change data capture patterns
Experience with distributed-system performance optimization
Strong understanding of relational databases and complex SQL/stored procedures
Experience with Lakeflow, Spark Declarative Pipelines, or dbt
Experience with Unity Catalog
Exposure to financial or General Ledger data models
Familiarity with JD Edwards, including F0902 and F0911

Datavail Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Datavail and has not been reviewed or approved by Datavail.

Fair & Transparent Compensation — Pay is considered competitive for many roles at the time of hire, with some noting significant initial increases and timely pay. Feedback suggests compensation can be strong in particular regions or positions even as overall satisfaction varies.
Wellbeing & Lifestyle Benefits — Remote-friendly roles and flexible or unlimited time off policies are highlighted as part of the employment experience. Feedback suggests these arrangements support work flexibility, though experiences differ by team and location.

Learn more about Datavail's Compensation & Benefits →

Datavail Insights

What's It Like to Work at Datavail? Datavail Culture & Values Datavail Career Growth & Development What's the Work-Life Balance Like at Datavail? Datavail Leadership & Management Datavail Company Growth, Stability & Outlook

View all jobs at Datavail

View Datavail Profile

Report Job

Am I A Good Fit?

beta

Get Personalized Job Insights.

Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company

HQ: Broomfield, CO

263 Employees

Year Founded: 2007

What We Do

A premiere data services company serving clients in North America, Datavail has 1,000 data professionals, data engineers, developers, project managers, consultants, and business experts, supported by industry-leading automation and intellectual property. For more than 17 years, Datavail has worked with thousands of companies spanning different industries and sizes. At Datavail, we look for more than smarts, experience and proficiency. On top of those requirements, we seek people who mesh with our corporate values. We seek brilliance without bravado and know-how without a know-it-all attitude. We hold low ego in high regard, embrace problem-solving as a passion and welcome every day as a new opportunity to learn. We’re flexible and hard working. We’re committed to our clients and colleagues. We help our people grow so they can help our clients grow. That makes us grow so we can help even more customers leverage organizational data for business value. Our Core Values: 1. We desire to serve. 2. We embody flexibility for availability 3. We exemplify low ego. 4. We work hard. 5. We strive for continuous improvement. 6. We are growth-oriented.