Title: Senior Technical Specialist - PBI Databricks
Location: Remote
Duration: 12+ Months
Education: Any Degree
Exp Level: 8+ Years
Job Responsibilities:
Design and build conformed dimensions and transactional fact tables for enterprise reporting.
Develop and maintain data pipelines on Azure Databricks using PySpark, Spark SQL, and Delta Lake.
Implement best practices in dimensional modeling, including:
Star schemas
Conformed dimensions
Surrogate and deterministic hash keys (e.g., SHA-256)
Transactional fact design
Translate existing SQL Server and stored procedure logic into efficient Spark-based transformations.
Implement incremental data processing and change data capture (CDC) pipelines.
Ensure data reconciliation with source systems to meet accuracy and audit standards.
Collaborate closely with a senior Databricks lead to deliver core data assets within defined timelines.
Required Skills:
8+ years of experience in data engineering, with strong focus on Azure Databricks.
Proficiency in:
PySpark
Spark SQL
Delta Lake
Deep expertise in dimensional modelling and data warehousing concepts.
Experience with:
Building scalable production pipelines
Incremental data loads and CDC patterns
Performance optimization in distributed systems
Strong understanding of relational databases and ability to interpret complex SQL/stored procedures
Good to have
Experience with:
Lakeflow / Spark Declarative Pipelines or dbt
Unity Catalog
Exposure to financial/General Ledger data models
Familiarity with JD Edwards systems, including:
F0902 (balances)
F0911 (journal details)
About UsDatavail is a leading provider of data management, application development, analytics, and cloud services, with more than 1,000 professionals helping clients build and manage applications and data via a world-class tech-enabled delivery platform and software solutions across all leading technologies. For more than 17 years, Datavail has worked with thousands of companies spanning different industries and sizes, and is an AWS Advanced Tier Consulting Partner, a Microsoft Solutions Partner for Data & AI and Digital & App Innovation (Azure), an Oracle Partner, and a MySQL Partner. About the Team
Skills Required
- 8+ years of experience in data engineering with strong focus on Azure Databricks
- Proficiency in PySpark
- Proficiency in Spark SQL
- Proficiency in Delta Lake
- Experience translating SQL Server and stored procedure logic into Spark-based transformations
- Deep expertise in dimensional modeling and data warehousing concepts (star schemas, conformed dimensions)
- Experience building scalable production pipelines, incremental data loads and CDC patterns
- Performance optimization in distributed systems
- Strong understanding of relational databases and ability to interpret complex SQL/stored procedures
- Any Degree
- Experience with Lakeflow / Spark Declarative Pipelines or dbt
- Experience with Unity Catalog
- Exposure to financial/General Ledger data models
- Familiarity with JD Edwards systems (F0902, F0911)
What We Do
A premiere data services company serving clients in North America, Datavail has 1,000 data professionals, data engineers, developers, project managers, consultants, and business experts, supported by industry-leading automation and intellectual property. For more than 17 years, Datavail has worked with thousands of companies spanning different industries and sizes. At Datavail, we look for more than smarts, experience and proficiency. On top of those requirements, we seek people who mesh with our corporate values. We seek brilliance without bravado and know-how without a know-it-all attitude. We hold low ego in high regard, embrace problem-solving as a passion and welcome every day as a new opportunity to learn. We’re flexible and hard working. We’re committed to our clients and colleagues. We help our people grow so they can help our clients grow. That makes us grow so we can help even more customers leverage organizational data for business value. Our Core Values: 1. We desire to serve. 2. We embody flexibility for availability 3. We exemplify low ego. 4. We work hard. 5. We strive for continuous improvement. 6. We are growth-oriented.
