Sr. Data Engineer

Posted 25 Minutes Ago
Be an Early Applicant
Hiring Remotely in United States
Remote
140K-155K Annually
Senior level
Big Data • Healthtech • Database • Analytics
The Role
Build and operate production data pipelines and models for provider data using Azure Databricks and Azure SQL. Develop PySpark, Python, SQL, and DDL code; implement master data management, entity resolution, history, lineage, survivorship, quality checks, and observability. Own modeling standards and CI/CD enforcement, review internal and vendor work, maintain technical documentation, and ensure PHI protection and HIPAA, HITRUST, and SOC 2 compliance.
Summary Generated by Built In


Position Summary
DataSpring runs the largest healthcare data foundation in the U.S. The Sr. Data Engineer builds and owns the pipelines and data models behind our provider data platform and the ProView suite. You will spend most of your time writing production code on Azure Databricks and Azure SQL: ingesting provider data from many sources, modeling it with history and lineage, and deciding which source wins when they disagree. You will also own our data modeling standards and the automation that enforces them, and review the data model work of internal engineers and vendor delivery teams.


The Sr. Data Engineer is a full-time, remote, exempt position and reports to the Senior Director, Data Engineering & Architecture.

Specific Responsibilities

  • Design, build, and operate production pipelines on Azure Databricks and Azure SQL that ingest, integrate, and publish provider data from internal and third-party sources.
  • Design the data models those pipelines load, including integration and master-data layers with history (SCD Type 2), source lineage, and field-level survivorship rules.
  • Write production-grade SQL, PySpark, Python, and DDL, and tune performance and cost on large datasets.
  • Build data quality checks, reconciliation, monitoring, and alerting so problems surface before customers find them.
  • Own the data modeling and naming standards for SQL Server, Azure SQL, and Databricks (Delta Lake, Unity Catalog), and build the automated checks in Azure DevOps CI/CD pipelines that enforce them.
  • Review data models, schema changes, and pull requests from internal engineers and vendor teams, and record modeling decisions so they can be traced later.
  • Maintain source-to-target mappings, data dictionaries, and ERDs, generated from the systems themselves wherever possible so they stay current.
  • Partner with Data Science, product, billing, and application teams to turn business requirements into data models and clear data contracts.
  • Protect PHI (Protected Health Information), including classification of sensitive columns, and maintain compliance with HIPAA, HITRUST, and SOC 2 audit requirements.

Supervisory Responsibilities

None. This role sets technical direction for internal engineers and vendor delivery teams without direct reports.

Skills

  • Expert SQL and strong PySpark and Python, with a track record of building and running production pipelines, not only designing them.
  • Deep relational and dimensional modeling skills, including history (SCD), entity resolution, and knowing when each approach is the wrong choice.
  • Hands-on experience with Azure Databricks: Delta Lake, Unity Catalog, medallion architecture, job orchestration (e.g., Databricks Asset Bundles), and performance tuning.
  • Git-based workflows and CI/CD in Azure DevOps or an equivalent platform, including database deployment automation with gated promotion across environments.
  • Experience with data quality testing and pipeline observability.
  • Ability to explain design decisions clearly to engineers, product owners, and leadership, and to hold a technical line with vendor teams while keeping the relationship productive.

Experience

Required:

  • 8+ years in data engineering, including at least 4 years designing data models that reached production at scale.
  • Recent, hands-on production engineering. You will be expected to write and ship code, not only review it.
  • Demonstrated experience building master data management, entity resolution, or survivorship logic across multiple data sources.
  • Experience defining data modeling standards and automating their enforcement.
  • Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent experience.

Preferred:

  • Healthcare data experience, especially provider data, credentialing, or member eligibility data.
  • Experience reviewing or directing the work of vendor or offshore delivery teams.
  • Experience with data quality platforms (e.g., Ataccama) and data modeling tools (e.g., ER/Studio).
  • Experience using AI-assisted development tools to build engineering automation.

Who We Are

DataSpring is the trusted data connector at the core of healthcare. For more than 25 years, we have powered the industry with the largest and most complete healthcare data foundation in the U.S., including more than 4.8 million provider data records sourced directly from providers and member data representing 75% of covered lives supplied by health plans. By improving how essential information flows across the system, DataSpring helps healthcare operate more efficiently, accurately, and with greater confidence.

What You Get

At DataSpring, you will do meaningful work at the intersection of healthcare, data, and technology, helping solve complex problems that make the healthcare system work better. You will collaborate with experienced professionals who care deeply about accuracy, trust, and meaningful impact in a fully remote environment.

DataSpring offers competitive compensation and a comprehensive benefits package for full-time employees, including medical, dental, and vision coverage, a 401(k) with company contributions and matching, paid parental leave, tuition assistance, and generous paid time off. We are committed to investing in our people and supporting professional growth over time.

Equal Opportunity Employer

DataSpring is proud to be an equal opportunity employer and is committed to fostering a workplace where all individuals are valued, respected, and empowered.

Employment decisions at DataSpring are made without regard to race, color, religion, sex, national origin or ancestry, age, marital status, disability, protected veteran status, personal appearance, sexual orientation, gender identity or expression, familial status, family responsibilities, matriculation, political affiliation, genetic information, source of income or any other characteristic protected by law. DataSpring does not tolerate unlawful discrimination or harassment of any kind.

Applicants have rights under the Family and Medical Leave Act (FMLA), Equal Employment Opportunity (EEO), and the Employee Polygraph Protection Act (EPPA). If you need a reasonable accommodation to apply for a posted position, please contact the DataSpring People & Culture team at [email protected] or 202-517-0436.

Skills Required

  • 8+ years of experience in data engineering
  • At least 4 years designing data models that reached production at scale
  • Recent hands-on production engineering experience, including writing and shipping code
  • Experience building master data management, entity resolution, or survivorship logic across multiple data sources
  • Experience defining data modeling standards and automating their enforcement
  • Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent experience
  • Expert SQL and strong PySpark and Python skills
  • Production pipeline development and operations experience
  • Deep relational and dimensional data modeling skills, including SCD, entity resolution, and survivorship
  • Hands-on Azure Databricks experience, including Delta Lake, Unity Catalog, medallion architecture, job orchestration, and performance tuning
  • Git-based workflows and CI/CD using Azure DevOps or an equivalent platform
  • Experience with database deployment automation and gated promotion across environments
  • Experience with data quality testing and pipeline observability
  • Ability to communicate design decisions clearly and provide technical direction to engineers and vendors
  • Healthcare data experience, especially provider data, credentialing, or member eligibility data
  • Experience reviewing or directing vendor or offshore delivery teams
  • Experience with data quality platforms such as Ataccama and data modeling tools such as ER/Studio
  • Experience using AI-assisted development tools to build engineering automation
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Washington, DC
138 Employees
Year Founded: 1998

What We Do

DataSpring, powered by CAQH, is a healthcare data-management organization that connects accurate, authorized information from providers and payers. Its foundation includes millions of provider records and member eligibility data, helping organizations process claims correctly, credential providers faster, maintain accurate directories, and reduce administrative burden. Its mission is to power a more connected, efficient, trusted, and less costly healthcare ecosystem for providers, payers, and patients.

Similar Jobs

Doximity Logo Doximity

Senior Software Engineer

Healthtech • Information Technology • Mobile • Productivity • Software • Analytics • Telehealth
Easy Apply
In-Office or Remote
2 Locations
740 Employees
165K-227K Annually

Optum Logo Optum

Data Engineer

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office or Remote
Eden Prairie, MN, USA
160000 Employees
92K-164K Annually

The Aerospace Corporation Logo The Aerospace Corporation

Senior Supplier Quality Engineer - Data Analytics

Aerospace • Artificial Intelligence • Cloud • Machine Learning • Software • Cybersecurity • Defense
Remote or Hybrid
2 Locations
4600 Employees
97K-146K Annually

Liberty Mutual Insurance Logo Liberty Mutual Insurance

Analytics Engineer

Artificial Intelligence • Fintech • Insurance • Marketing Tech • Software • Analytics
Remote or Hybrid
6 Locations
40000 Employees
94K-176K Annually

Similar Companies Hiring

OneImaging Thumbnail
Healthtech
Miami, FL
62 Employees
Pomelo Care Thumbnail
Healthtech • Telehealth
New York, NY
550 Employees
Vitalize Thumbnail
Artificial Intelligence • Healthtech • Software
San Francisco, California
50 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account