About Our Team
Our employees thrive in a culture that's fast-paced and ego-free, where innovation and collaboration are encouraged at every turn. We are an organization that provides federal agencies instant access to experienced and talented professionals who understand their unique challenges and know the most efficient ways to address them. We are continually investing in resources and talent, so we stay prepared with specialized teams in the place who are experts in creating tailored technologies. Our solutions empower Federal organizations to grow, modernize, and succeed in a rapidly evolving landscape.
We welcome diverse perspectives and seek individuals who are passionate about technology and creative problem-solving. If you enjoy learning, growing, and tackling real-world challenges, you will thrive here. Veterans and military spouses are strongly encouraged to apply and bring their unique experience to our team.
About the Role:
Our core values of People Matter, Integrity, and a Commitment to Excellence drive all that we do. By joining us, you will become a part of a fun and diverse team of talented and creative consultants who share the goal of using the latest technology to solve business challenges. We provide our clients with a dynamic mix of services and deliver focused solutions like no one else.
We are seeking talented and bright team players who are passionate about technology and want to work in a fast-paced, dynamic, and ego-free culture while applying a creative approach to problem-solving. Team members who like to grow their skill sets while solving challenging, real world business problems thrive.
We are looking for an experienced PySpark & Delta Lake Developer, who will be responsible for designing, building, and maintaining scalable ETL pipelines to process and analyze large-scale healthcare claims data. This role emphasizes building robust Delta Lake tables and ensuring ACID-compliant data lakes. The ideal candidate will focus on developing efficient PySpark scripts and leveraging Delta Lake capabilities to deliver data reliability, high performance, and seamless schema evolution within an AWS environment.
Key Responsibilities:
- Design, develop, and maintain robust ETL pipelines using PySpark and Delta Lake for large and complex healthcare data workloads.
- Implement and optimize data lake solutions using Delta Lake table formats, supporting ACID transactions, schema enforcement, and time travel.
- Write efficient, reusable, and well-documented PySpark scripts for data ingestion, transformation, cleansing, and aggregation.
- Collaborate with data engineers, architects, and data scientists to understand business and data requirements and translate them into scalable data solutions.
- Ensure data quality, consistency, lineage, and integrity across all stages of data processing.
- Troubleshoot, debug, and optimize PySpark applications and Delta Lake workflows for cost, speed, and reliability within AWS.
- Maintain detailed and up-to-date technical documentation of code, data pipelines, and standard operating procedures.
- Stay updated with the latest Delta Lake and Spark advancements, advocating for best practices in data management and analytics.
TAG: INDMJC
TAG: #LI-I4DM
Required Qualifications:
- Strong proficiency in Python and PySpark, with hands-on experience developing data pipelines.
- Advanced experience with Delta Lake and its ACID transaction and schema management features.
- Solid SQL skills for querying, joining, and optimizing data in distributed environments.
- Hands-on experience with AWS cloud data services (e.g., S3, Glue, EMR, Athena).
- Familiarity with data lake concepts, partitioning, and performance tuning.
- Excellent communication skills and a desire to continuously learn and adapt to innovative technologies.
- Familiarity with CI/CD, version control (e.g., Git), and infrastructure as code.
Preferred Qualifications:
- Experience with healthcare or claims data.
- Knowledge of data governance, security, data cataloging (AWS Glue Catalog), and compliance best practices.
- Strong ability to prioritize and execute tasks independently and within collaborative team environments.
- Previous experience working in a government or public sector setting.
Top Skills
What We Do
Ready to advance your career as an agent of change? View our available positions at i4dm.com/resourcing/careers or forward your resume to [email protected].
i4DM is a full-service information technology firm that believes in the versatility of IT. i4DM was founded in 2002 by Michael Peart and partner Ben Hannon. Forged together by Michael’s military background and Ben’s passion for technology, they created a company grounded in military values, dedicated to serving clients through innovation and strategy. With a client-first approach, the team is equipped with the necessary certifications and skill sets to serve all industries. Through market expansion, joint ventures, and new locations, i4DM has grown into an industry leader that revolutionizes the way information technology is leveraged by clients to accomplish their missions.
i4DM is passionate about empowering clients’ information technology to incite change, increase productivity, and keep them one step ahead in a dynamic market. Aiming for excellence, and delivering innovation, they go beyond the routine and create entirely customized solutions. They believe in the spirit of collaboration, exploring the line of the unknown, and pushing the boundaries of what’s possible with technology solutions
