Senior Data Engineer

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!

Requirement Description:

• Experienced in designing, implementation & testing of highly scalable and cost-efficient AWS based big data ETL/ELT pipeline using PySpark and SQL

• Well-experienced in data warehousing

• Well versed in working with transactional databases

Must have

• Big data engineering technologies like Spark (PySpark)

• Good knowledge of working with Python Programming

• Good knowledge of advanced SQL concepts, Optimization techniques

• Good knowledge of data warehousing, data modeling and concepts like OLAP and OLTP

• Working knowledge with SQL databases like  Postgresql, etc. is mandatory

• Good knowledge of Data Migration like bringing on premise database to cloud or one cloud to another

• Understand Architecture Requirements and ensure effective design, development, validation, and support activities.

• Understanding of data based AWS services like Glue, EMR, Lambda, DMS, RDS, S3 etc and basic AWS architecture best practices.

• Knowledge of data pipeline orchestration using Step function or Airflow

• Ability to use the AWS service APIs, AWS CLI, and SDKs to build applications

Good to have

•  Knowledge of NoSQL databases like Dynamo can be add on

• Knowledge of BI tools

• Knowledge of working with streaming pipelines