City/State
Virginia Beach, VAWork Shift
First (Days)Overview:
Sentara Health is looking for a Remote Lead Databricks Engineer.
This is a remote position: Remote opportunities available in the following states: Virginia, North Carolina, Alabama, Delaware, Florida, Georgia, Idaho, Indiana, Kansas, Louisiana, Maine, Maryland, Minnesota, Nebraska, Nevada, New Hampshire, North Dakota, Ohio, Oklahoma, Pennsylvania, South Carolina, South Dakota, Tennessee, Texas, Utah, Washington (state), West Virginia, Wisconsin, Wyoming
Lead Specialty Engineer
Overview
We are looking for a Senior Data Engineer to help build and scale a modern, metadata-driven data platform on Azure. This role focuses on designing reusable data ingestion frameworks, enabling governance at scale, and integrating with modern data catalog and access patterns.
The ideal candidate is hands-on, thinks in terms of platforms rather than pipelines, and has strong experience with Databricks, Python, and cloud-native data architectures.
Key Responsibilities
- Design and develop scalable, reusable data ingestion frameworks using Azure Databricks and PySpark
- Build metadata-driven pipelines that decouple configuration from execution
- Implement and maintain Bronze, Silver, and Gold data layers following Medallion architecture
- Integrate data pipelines with data catalog and governance tools (e.g., DataHub, Purview)
- Implement schema enforcement, data contracts, and data quality checks within pipelines
- Work with ingestion tools (e.g., Fivetran) and file-based ingestion patterns
- Enable automation for dataset onboarding using configuration-driven approaches
- Collaborate with architecture, governance, and business teams to ensure scalable and governed data access
- Contribute to evolving toward event-driven and near real-time ingestion patterns
- Support CI/CD and DevOps practices for data platform components
Education:
- Bachelor's degree
Required Skills & Experience
- 8+ years of experience in data engineering or data platform development with degree
- 10+ years of relevant experience without a degree
- Strong hands-on experience with Azure Databricks and PySpark
- Proficiency in Python for data engineering and framework development
- Experience with Delta Lake and Lakehouse architecture
- Strong understanding of Medallion architecture (Bronze, Silver, Gold)
- Experience designing reusable and scalable data ingestion frameworks
- Experience working with data catalog / governance tools (DataHub, Purview, Collibra, etc.)
- Solid understanding of:
- data lineage
- schema evolution
- data contracts
- data quality frameworks
- Experience working with cloud storage (ADLS Gen2) and distributed data processing
- Familiarity with CI/CD pipelines and Git-based workflows
We provide market-competitive compensation packages, inclusive of base pay, incentives, and benefits. The base pay rate for Full Time employment is: $106,080 – $176,820.8. Additional compensation may be available for this role such as shift differentials, standby/on-call, overtime, premiums, extra shift incentives, or bonus opportunities
•Legal Resources Plan
•Colleagues have the opportunity to earn an annual discretionary bonus if established system and employee eligibility criteria is met.
Sentara Health is an equal opportunity employer and prides itself on the diversity and inclusiveness of its close to an almost 30,000-member workforce. Diversity, inclusion, and belonging is a guiding principle of the organization to ensure its workforce reflects the communities it serves.
In support of our mission “to improve health every day,” this is a tobacco-free environment.
For positions that are available as remote work, Sentara Health employs associates in the following states:
Alabama, Delaware, Florida, Georgia, Idaho, Indiana, Kansas, Louisiana, Maine, Maryland, Minnesota, Nebraska, Nevada, New Hampshire, North Carolina, North Dakota, Ohio, Oklahoma, Pennsylvania, South Carolina, South Dakota, Tennessee, Texas, Utah, Virginia, Washington, West Virginia, Wisconsin, and Wyoming.
Skills Required
- 8+ years of experience in data engineering with degree
- 10+ years of relevant experience without a degree
- Strong hands-on experience with Azure Databricks and PySpark
- Proficiency in Python for data engineering
- Experience with Delta Lake and Lakehouse architecture
- Experience designing reusable data ingestion frameworks
- Experience working with data catalog / governance tools
- Solid understanding of data lineage, schema evolution, data contracts, and data quality frameworks
- Experience working with cloud storage (ADLS Gen2)
- Familiarity with CI/CD pipelines and Git-based workflows
Sentara Healthcare Compensation & Benefits Highlights
The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Sentara Healthcare and has not been reviewed or approved by Sentara Healthcare.
-
Parental & Family Support — Four weeks of paid parental leave at full base pay and two weeks of job‑protected family caregiver leave support major life and care needs. Emergency back‑up care and reimbursements for infertility, adoption, and surrogacy further bolster family support.
-
Retirement Support — A 401(A) plan alongside 403(B)/401(K) employer matching is designed to strengthen long‑term financial security. Company‑paid life insurance with buy‑up options adds additional protection for families.
-
Flexible Benefits — Choice of medical plan designs and dental/vision options enables tailoring coverage to individual needs. An annual election between tuition assistance and student‑loan repayment offers flexibility to align with financial or education priorities.
Sentara Healthcare Insights
What We Do
Sentara Healthcare celebrates a 130-year history of innovation, compassion and community benefit. Based in Norfolk, VA, Sentara is a diverse not-for-profit family of 12 hospitals, an array of integrated services and a team of nearly 30,000 strong on a mission to improve health every day. This mandate is pursued through a disciplined strategy to achieve Top 10% performance in key measures through shared best practices, transformation of primary care through clinical integration and strategic growth that adds value to the communities we serve in Virginia and North Carolina.
%20(1).png)
