The Role
Leads the design, development, and operation of Astrana’s enterprise data platform. Builds scalable Databricks, Spark, PySpark, SQL, and Delta Lake pipelines; integrates healthcare data; develops unified data models and APIs; supports Lakebase and PostgreSQL applications; and implements governance, data quality, monitoring, CI/CD, automation, and performance optimization. The role also handles incidents, disaster recovery coordination, engineering standards, code reviews, and mentoring across internal and vendor teams.
Summary Generated by Built In
We are seeking a Lead - Data Engineer to lead the hands-on design, development, and operation of key components of Astrana’s enterprise data platform, including the Constellation Lakehouse, Connect API (GraphQL) layer, and Lakebase-backed applications.
This role will build scalable data pipelines, integrate healthcare data sources, and develop unified data models that support analytics, reporting, AI and machine learning, and operational applications. The position combines advanced data engineering with responsibility for platform reliability, performance, security, and automation within an Azure Databricks environment.
As a hands-on technical lead, you will translate complex requirements into reliable production solutions, establish engineering standards, and mentor other engineers. You will collaborate with analytics, software engineering, infrastructure, security, business stakeholders, and external vendors to advance the platform and support its growing adoption.
Our Values
Our Values
- Put Patients First
- Empower Entrepreneurial Provider and Care Teams
- Operate with Integrity & Excellence
- Be Innovative
- Work As One Team
What You'll Do
- Lead the design, development, and delivery of core platform components, including the Constellation Lakehouse in Databricks, Connect API (GraphQL) layer, and Lakebase-backed applications.
- Build and maintain scalable batch and incremental data pipelines using Databricks, Apache Spark, PySpark, SQL, and Delta Lake.
- Integrate data from transactional databases, APIs, files, and external partners into governed datasets that support analytics and operational applications.
- Design and maintain unified healthcare data models spanning membership, providers, claims, encounters, clinical data, and other business domains.
- Develop reliable ingestion and transformation processes, including change data capture, schema evolution, deduplication, historical tracking, and reconciliation.
- Build and optimize data access layers and APIs in partnership with software engineering teams, supporting consistent business definitions and reliable downstream consumption.
- Develop and support Lakebase-backed applications and PostgreSQL data structures, including integration with lakehouse datasets and application services.
- Maintain and optimize Databricks workspaces, compute resources, job orchestration, and environment configurations in collaboration with infrastructure and security teams.
- Implement data governance and access controls through Unity Catalog and established organizational standards, including permissions, lineage, and auditability.
- Establish automated data quality checks, source-to-target reconciliation, freshness monitoring, and alerting to ensure datasets are accurate, complete, and usable.
- Apply DevOps practices, including Git, automated testing, CI/CD, infrastructure as code, and repeatable deployment across development, testing, and production environments.
- Tune pipelines, Spark workloads, SQL queries, and compute configurations to improve performance, scalability, and cost efficiency.
- Own the investigation and resolution of platform incidents and pipeline failures; perform root-cause analysis and implement preventive improvements.
- Partner with infrastructure teams on backup, recovery, and business continuity procedures, and maintain operational documentation and runbooks.
- Conduct code and design reviews, establish engineering standards, and mentor engineers across internal and vendor teams.
Skills, Knowledge & Expertise
- Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field.
- 5+ years of data engineering experience building and supporting enterprise data platforms and pipelines.
- 3+ years of hands-on experience with Databricks in production environments.
- Strong proficiency in Python, PySpark, SQL, and Apache Spark.
- Experience with Delta Lake and modern lakehouse architectures.
- Hands-on experience with Azure data services and cloud-based data platforms.
- Strong knowledge of data modeling, relational databases, and ETL/ELT best practices.
- Experience integrating enterprise databases and APIs into scalable data solutions.
- Experience with Git, CI/CD pipelines, and automated deployment practices.
- Proven ability to monitor, troubleshoot, and optimize data pipelines and platform performance.
- Demonstrated leadership in solution design, code reviews, mentoring, and cross-functional collaboration.
- Strong problem-solving and communication skills.
- Experience administering and optimizing Databricks environments, including Unity Catalog.
- Experience with PostgreSQL, GraphQL APIs, and operational applications leveraging lakehouse data.
- Healthcare data experience, including claims, eligibility, provider, or clinical datasets.
- Familiarity with healthcare interoperability standards such as HL7, FHIR, and X12.
- Experience with Terraform or other infrastructure-as-code tools.
- Experience with streaming data pipelines and event-driven architectures.
- Experience supporting AI, machine learning, or NLP data workloads.
- Familiarity with regulated data environments and governance best practices.
- Experience with cloud data platform migrations and modernization initiatives.
- Azure, Databricks, or AWS certifications preferred.
Job Benefits
- This position is remotely based in the U.S.
- The total compensation target pay range for this role is: $170,000 - $240,000. Actual compensation will be determined based on geographic location (current or future), experience, and other job-related factors.
Astrana Health is proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based on race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. All employment is decided based on qualifications, merit, and business need. If you require assistance in applying for open positions due to a disability, please email us at [email protected] to request an accommodation.
Additional Information:
The job description does not constitute an employment agreement between the employer and employee and is subject to change by the employer as the needs of the employer and requirements of the job change.
About
Astrana Health (NASDAQ: ASTH) is a physician-centric, technology-powered healthcare management company. We are building and operating a novel, integrated, value-based healthcare delivery platform to empower our physicians to provide the highest quality of end-to-end care for their patients in a cost-effective manner. Our mission is to combine our clinical experience, best-in-class delivery network, and technological expertise to improve patient outcomes, increase access to healthcare, and make the US healthcare system more efficient. Our platform currently empowers over 20,000 physicians to provide care for over 1.7 million patients nationwide. Our rapid growth and unique position at the intersection of all major healthcare stakeholders (payer, provider, and patient) gives us an unparalleled opportunity to combine clinical and technological expertise to improve patient outcomes, increase access to quality healthcare, and reduce the waste in the US healthcare system.
Skills Required
- Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field
- 5+ years of data engineering experience building and supporting enterprise data platforms and pipelines
- 3+ years of hands-on Databricks experience in production environments
- Strong proficiency in Python, PySpark, SQL, and Apache Spark
- Experience with Delta Lake and modern lakehouse architectures
- Hands-on experience with Azure data services and cloud-based data platforms
- Strong knowledge of data modeling, relational databases, and ETL/ELT best practices
- Experience integrating enterprise databases and APIs into scalable data solutions
- Experience with Git, CI/CD pipelines, and automated deployment practices
- Ability to monitor, troubleshoot, and optimize data pipelines and platform performance
- Leadership experience in solution design, code reviews, mentoring, and cross-functional collaboration
- Strong problem-solving and communication skills
- Experience administering and optimizing Databricks environments, including Unity Catalog
- Experience with PostgreSQL, GraphQL APIs, and operational applications leveraging lakehouse data
- Healthcare data experience involving claims, eligibility, provider, or clinical datasets
- Familiarity with HL7, FHIR, and X12 healthcare interoperability standards
- Experience with Terraform or other infrastructure-as-code tools
- Experience with streaming data pipelines and event-driven architectures
- Experience supporting AI, machine learning, or NLP data workloads
- Familiarity with regulated data environments and governance best practices
- Experience with cloud data platform migrations and modernization initiatives
- Azure, Databricks, or AWS certification
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Astrana Health is a physician-centric, technology-powered healthcare company that operates an integrated delivery platform. It enables providers to participate in value-based care arrangements, helping them deliver accessible, high-quality, and cost-effective care to patients. The company provides care coordination services to patients, primary care physicians, specialists, and health plans, leveraging proprietary technology to streamline operations and improve patient outcomes across local communities.
