The Data Engineer is responsible for designing, developing and maintaining scalable data pipelines and data infrastructure transforming raw data into insights.
Leveraging expertise in data engineering, data management, data operations and governance frameworks, this position will collaborate with cross-functional teams to architect scalable and efficient data pipelines, optimize data storage and retrieval systems, and ensure data quality and integrity throughout the data lifecycle.
Essential Job Statements
Leads the design and implementation of scalable data pipelines for ingesting, transforming, and loading multi-format, batch or streaming data from various sources into enterprise Data Lakehouse.
Ensures data quality and integrity through data cleansing, validation, and transformation techniques.
Automates data processing workflows using scripting languages and orchestration tools.
Builds and deploys RESTful API’s and web services for data access and data requests.
Helps and orchestrates the definition of data management strategy and roadmap for the organization.
Collaborates with data architects, data engineers, and product managers to align data solutions with business strategy.
Stays current with industry trends, vendor product offerings, and evolving data technologies to ensure the organization leverages the best modern data engineering tools.
Leads the development and maintenance of technical design documents, coding best practices and standards to ensure consistent and reliable data management processes are in place.
Responsible for maintaining legacy technologies and processes as new ones are adopted.
Mentors junior Data Engineers.
Patient Population
Not applicable to this position.
Employment Qualifications
Required Education:
Bachelor’s Degree in Business, IT, Math, Data Analytics, Engineering, or related discipline
Combination of education and experience may be considered in lieu of a degree.
Preferred Education:
Master’s degree in Business, IT, Math, Data Analytics, Engineering or related discipline
Licensure/Certification Required:
Licensure/Certification Preferred:
Minimum Qualifications
Years and Type of Required Experience
8+ years of experience in Data Engineering, Data Management or Data Warehouse/Lakehouse design and development.
5+ years of experience in Python/PySpark for building and maintaining data pipelines.
3+ years of experience with cloud based modern data platforms and services (e.g., Azure, GCP or AWS).
Other Knowledge, Skills and Abilities Required:
Expert knowledge of database systems, data modeling techniques, and SQL proficiency.
Substantial experience creating and enhancing ETL and ELT processes.
Significant experience working with JSON/YAML/XML/Parquet and other open-source data interchange and storage formats.
Experience working with big data technologies like Spark, Kafka, NoSQL DB, Splunk etc.
Significant experience with agile development processes and concepts.
Strong knowledge of DevOps practices and CI/CD processes.
Knowledge of Microservices architecture, Data Product and Data Mesh concepts.
Other Knowledge, Skills and Abilities Preferred:
Displays intellectual curiosity and integrity.
Experience in healthcare environment.
Working Conditions
Physical Requirements
Physical Demands:
Work Position: Sitting
Additional Physical Requirements/ Hazards
Physical Requirements:
Hazards:
Mental/Sensory – Emotional
Mental / Sensory: Strong Recall, Reasoning, Problem Solving, Hearing, Speak Clearly, Write Legibly, Reading, Logical Thinking
Emotional: Fast-paced environment, Able to Handle Multiple Priorities, Frequent and Intense Customer Interactions, Able to Adapt to Frequent Change
DaysEEO Employer/Disabled/Protected Veteran/41 CFR 60-1.4.
Top Skills
What We Do
We are a strong, passionate team of more than 12,500 who take pride in caring for every person who comes through our doors. We lift each other up so we can provide the very best and safest care to those who need us most. Together. Every day. With the support of our university, we make up an academic medical center committed to excellence in patient care, innovation and training tomorrow’s health care workforce. We continue to educate, research and evolve – staying on the cutting edge of care. As a community of innovators, every team member can contribute a spark of ingenuity igniting a force to discover the next cure, hold the next hand, solve the greatest challenges and create the health care of tomorrow. We foster an atmosphere of respect and welcoming for all communities. We infuse our teams with diverse talent that inspires everyone to contribute openly and freely, maximizing their impact and creating meaningful change for our patients and community. Join our team and help us create a new kind of patient experience. Join VCU Health. EEO Statement: VCU Health System strictly prohibits and does not tolerate discrimination against, or harassment of, team members, applicants, or any other covered persons because of age, race, ethnicity, religion, culture, language, physical or mental disability, socioeconomic status, sex (sexual orientation, gender identity or expression and pregnancy), protected veteran status, marital status, genetic information, or any other protected characteristics under applicable federal, state, or local law. Pay Transparency Provisions: VCU Health System complies with the Pay Transparency Provisions








