Job Description - Data Engineer
About the Role
We are looking for a candidate who is passionate about building scalable data pipelines, optimizing data workflows, and ensuring high data quality across systems. Candidates should demonstrate strong technical foundations, the ability to work independently, and a willingness to collaborate in a dynamic environment.
Core Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines to support business and analytical needs.
- Work extensively with Databricks, Python, and PySpark to process large datasets.
- Build and manage DAGs using Apache Airflow for workflow orchestration.
- Collaborate with cross-functional teams to understand data requirements and translate them into efficient engineering solutions.
- Develop and optimize complex SQL queries and participate in data modeling activities for relational and cloud data warehouses.
- Work with Amazon S3 for data storage, ingestion, partitioning, and integration within broader data lake and pipeline ecosystems.
- Ensure high standards of data quality, reliability, and performance across all data processes.
- Contribute to documentation, best practices, and continuous improvement initiatives.
Core Technical Requirements
- Strong experience writing clean, efficient Python code for data manipulation, automation, scripting, and ETL workflows.
- Familiarity with widely used data libraries (e.g., pandas, numpy).
- Hands-on experience with Databricks for distributed data processing.
- Proficiency in PySpark, Delta Lake, notebooks, and building scalable pipelines.
- Apache Airflow (Required): Ability to design, implement, and maintain complex DAGs for scheduling and orchestrating workflows.
- Argo Workflows (Preferred): Experience with Kubernetes-native orchestration platforms is an added advantage.
- Advanced SQL expertise including writing complex queries, query optimization, and working with relational/cloud data warehouses.
- Experience in data modeling and performance tuning.
- Practical knowledge of S3 for ingestion, storage, data partitioning, access control, and integration as part of data lake architectures.
Experience Level
2-5 years in Data Engineering or related roles..
Preferred Personal Attributes
- Strong analytical and problem-solving skills.
- Excellent communication and collaboration abilities.
- Ability to work in a fast-paced, evolving environment.
What We Do
We’re a healthcare technology company that provides industry-leading intelligence on the healthcare provider market. Why do we do it? Because understanding provider landscapes, identifying opportunities, and reaching the right points of contact can be difficult to do in a constantly changing market. But it doesn’t have to be. Our comprehensive data platform reduces market complexity and streamlines physician and facility insights. Our experienced team is here to help your organization turn those insights into acceleration—whether it’s advancing your go-to-market strategy or closing a new deal. We make healthcare actionable and accessible for our industry partners. How do we do it? We collect proprietary research, secondary research, and third-party data and organize all of this into a searchable, user-friendly platform. Since 2011, we’ve partnered with 9 of the top 10 pharmaceutical, biotechnology, and medical device companies. In that same period, we’ve also partnered with 7 of the top 10 healthcare IT firms and over 2,500 of the top healthcare providers, healthcare staffing companies, and consulting firms.
Why Work With Us
We will never stop improving the product we’ve worked so hard to develop for our customers. We’re thinking beyond simply providing more information; we’re building a solution designed to help users derive insights so their businesses can operate at a rapid pace. We are a collaborative and high energy environment with tons of opportunity for growth.
Gallery
.jfif)
Definitive Healthcare Offices
Hybrid Workspace
Employees engage in a combination of remote and on-site work.
.jpg)
