Senior Data Engineer - Databricks

Posted 12 Hours Ago
Be an Early Applicant
Bengaluru, Bengaluru Urban, Karnataka
Senior level
Artificial Intelligence • Big Data • Machine Learning
The Role
The Senior Data Engineer will be responsible for delivering data solutions using Databricks, constructing data pipelines with PySpark, and leveraging cloud-based technologies for data integration and management. The role involves optimizing SQL queries and implementing data integration projects, with experience in AWS and orchestration tools considered a plus.
Summary Generated by Built In

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!

Must have skills:

  • 3+ years’ of hands-on experience in delivering data solutions

  • Experience with Databricks implementations - developing data pipelines using PySpark

  • Experience in Databricks Workspaces,Notebooks, Delta Lake, APIs

  • Experience in Python, Scala

  • Experience in advanced SQL and Query optimization 

  • Experience in implementing data integration projects using ETL 

  • Experience building and deploying cloud-based solutions at scale by ingesting data from sources like DB2 

Good to have skills : 

  • Experience with AWS services - S3,Redshift,Secrets Manager

  • Experience in implementing data integration projects using ETL (preferably Qlik Replicate & QlikCompose)

  • Experience in using Airflow or Step Functions for orchestration

  • Exposure to IaC tools like Terraform and to CI/CD tools

  • Prior experience in migrating (on-prem to cloud) and processing large amounts of data

  • Experience in implementing data lake and data warehouse on cloud

  • Experience in implementing the industry's best practices

If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!

Top Skills

Python
Scala
The Company
HQ: Marlborough, MA
3,494 Employees
On-site Workplace
Year Founded: 2013

What We Do

Quantiphi is an award-winning AI-first digital engineering company driven by the desire to solve transformational problems at the heart of business.
Quantiphi solves the toughest and complex business problems by combining deep industry experience, disciplined cloud, and data-engineering practices, and cutting-edge artificial intelligence research to achieve quantifiable business impact at unprecedented speed.

Similar Jobs

Hybrid
Bengaluru, Karnataka, IND
289097 Employees

Takeda Logo Takeda

IDMC Platform Owner

Healthtech • Software • Analytics • Biotech • Pharmaceutical • Manufacturing
Hybrid
Bengaluru, Karnataka, IND
50000 Employees

Atlassian Logo Atlassian

Principle Data Engineer

Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Remote
Bengaluru, Karnataka, IND
11000 Employees

Atlassian Logo Atlassian

Senior Data Engineer

Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Remote
Bengaluru, Karnataka, IND
11000 Employees

Similar Companies Hiring

Halter Thumbnail
Software • Machine Learning • Internet of Things • Hardware • Greentech • Business Intelligence • Agriculture
Auckland City, NZ
150 Employees
MassMutual India Thumbnail
Insurance • Information Technology • Fintech • Financial Services • Big Data
Hyderabad, Telangana
InCommodities Thumbnail
Renewable Energy • Machine Learning • Information Technology • Energy • Automation • Analytics
Austin, TX
234 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account