Senior Data Engineer

Posted 2 Days Ago
Be an Early Applicant
2 Locations
In-Office
Senior level
Artificial Intelligence • Big Data • Machine Learning
The Role
Design, build, and operate large-scale batch and real-time data pipelines using Spark, AWS (Glue, EMR, S3, Redshift). Implement observability, alerts, and drive technical deliverables while mentoring junior engineers.
Summary Generated by Built In

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!

Must have skills:

  • 3+ years of hands-on experience in building large-scale data pipelines.

  • Strong experience with Spark Streaming, AWS Glue, and EMR for processing both real-time and batch workloads

  • Proficiency in Pyspark/Python, including building Kafka producers for data ingestion

  • Experience working with Confluent Kafka/spark streaming for streaming ingestion from on-premise sources.

  • Solid understanding of AWS services including S3, Redshift, Glue, CloudWatch, and Secrets Manager

  • Experience working with Medallion architecture and hybrid data destinations (e.g., Redshift and on-premise Oracle)

  • Ability to implement monitoring dashboards and observability for pipelines using tools like CloudWatch or Datadog

  • Strong SQL skills for validating data accuracy and developing job-level metrics

  • Experience with building alerting mechanisms for pipeline failures and performance issues

  • Strong collaboration and communication skills

  • Proven ability to take ownership of technical deliverables, driving execution from design to deployment

  • Experience in guiding junior developers or data engineers, providing technical direction and code reviews to uphold best practices

  • AWS Data Engineering - Associate Certified.

Good to have skills : 

  • Experience with orchestration tools like Apache Airflow or AWS Step Functions

  • Exposure to Bigdata stacks like Sqoop,HDFS,Hive,Nifi

  • Exposure to Terraform for infrastructure automation

  • Familiarity with CI/CD pipelines for data workflows

If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!

Top Skills

Spark Streaming,Aws Glue,Emr,Pyspark,Python,Kafka,Confluent Kafka,S3,Redshift,Cloudwatch,Secrets Manager,Datadog,Sql,Oracle,Medallion Architecture,Apache Airflow,Aws Step Functions,Sqoop,Hdfs,Hive,Nifi,Terraform,Ci/Cd
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Marlborough, MA
3,494 Employees
Year Founded: 2013

What We Do

Quantiphi is an award-winning AI-first digital engineering company driven by the desire to solve transformational problems at the heart of business.
Quantiphi solves the toughest and complex business problems by combining deep industry experience, disciplined cloud, and data-engineering practices, and cutting-edge artificial intelligence research to achieve quantifiable business impact at unprecedented speed.

Similar Jobs

Capco Logo Capco

Senior Data Engineer

Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Hybrid
Pune, Maharashtra, IND
6000 Employees

Wolters Kluwer Logo Wolters Kluwer

Senior Data Engineer

Information Technology • Software
In-Office
Industrial Estate, Kharadi, Pune, Maharashtra, IND
18996 Employees

Autodesk Logo Autodesk

Senior Data Engineer

Big Data • Cloud • Digital Media • Machine Learning • Mobile • Software • Industrial
In-Office
Pune, Maharashtra, IND
13285 Employees
In-Office
Mumbai, Maharashtra, IND
3696 Employees

Similar Companies Hiring

Milestone Systems Thumbnail
Software • Security • Other • Big Data Analytics • Artificial Intelligence • Analytics
Lake Oswego, OR
1500 Employees
Idler Thumbnail
Artificial Intelligence
San Francisco, California
6 Employees
Bellagent Thumbnail
Artificial Intelligence • Machine Learning • Business Intelligence • Generative AI
Chicago, IL
20 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account