About the Role
We are looking for a Data Pipeline Engineer with strong expertise in Databricks, AWS Glue, Python, SQL, and MongoDB to design, develop, and optimize scalable data pipelines. The ideal candidate should have experience building cloud-native ETL solutions, transforming large datasets, and delivering high-quality data for analytics and business applications.
Key Responsibilities
- Develop and maintain scalable ETL/data pipelines using Databricks and AWS Glue.
- Build data ingestion and transformation workflows for structured and semi-structured data.
- Write efficient PySpark, Python, and SQL code for large-scale data processing.
- Integrate data from multiple sources, including MongoDB and relational databases.
- Implement data quality, validation, and reconciliation checks.
- Optimize pipeline performance, reliability, and scalability.
- Collaborate with architects, analysts, and application teams to deliver data solutions.
- Support production deployments, monitoring, and troubleshooting.
Required Skills
- 5+ years of experience in Data Engineering or ETL development.
- Strong hands-on experience with Databricks and PySpark.
- Experience with AWS Glue and AWS data services.
- Proficiency in Python and SQL.
- Experience working with MongoDB.
- Knowledge of ETL, data modeling, and data warehousing concepts.
- Familiarity with Delta Lake/Lakehouse architecture is preferred.
- Experience with Git and Agile development practices.
Good to Have
- Delta Lake
- Apache Spark optimization
- AWS S3
- CI/CD pipelines
- Healthcare or Benefits domain experience
Skills Required
- 5+ years of experience in Data Engineering or ETL development
- Strong hands-on experience with Databricks and PySpark
- Experience with AWS Glue and AWS data services
- Proficiency in Python and SQL
- Experience working with MongoDB
- Knowledge of ETL, data modeling, and data warehousing concepts
- Experience with Git and Agile development practices
- Familiarity with Delta Lake/Lakehouse architecture
- Delta Lake
- Apache Spark optimization
- AWS S3
- CI/CD pipelines
- Healthcare or Benefits domain experience
What We Do
PeerIslands is an AI-native enterprise transformation partner that helps global organizations modernize by streamlining workflows and accelerating software delivery. Using its proprietary PeerAI platform, the company leverages generative AI, multi-agent orchestration, and automation. It specializes in cloud-native application modernization and data engineering, serving Fortune 100 clients in industries such as BFSI, healthcare, and telecommunications.
.png)
