Databricks Engineer, Data Platform & Data Science Tooling

Posted 3 Hours Ago
Be an Early Applicant
Bangalore, Bengaluru, Karnataka, IND
Hybrid
Mid level
Automotive • Hardware • Robotics • Software • Transportation • Manufacturing
Dream big. With Us. Let's create the future of mobility, together.
The Role
Design, build, and maintain Databricks-based data pipelines using PySpark and Delta Live Tables. Implement data models, optimize complex SQL and PySpark transformations, manage Unity Catalog for governance, support AI/ML downstream consumption, monitor and troubleshoot pipelines, contribute to CI/CD, and help power the semantic layer for Power BI dashboards.
Summary Generated by Built In

Job descriptions may display in multiple languages based on your language selection.

What we offer:
At Magna, you can expect an engaging and dynamic environment where you can help to develop industry-leading automotive technologies. We invest in our employees, providing them with the support and resources they need to succeed. As a member of our global team, you can expect exciting, varied responsibilities as well as a wide range of development prospects. Because we believe that your career path should be as unique as you are.
Group Summary:
Magna is more than one of the world’s largest suppliers in the automotive space. We are a mobility technology company built to innovate, with a global, entrepreneurial-minded team. With 65+ years of expertise, our ecosystem of interconnected products combined with our complete vehicle expertise uniquely positions us to advance mobility in an expanded transportation landscape.

Job Responsibilities:

Job Responsibilities – Must Have

  • Non-negotiable skill:  3+ years hands-on: PySpark, Delta Lake, Workflows, Unity Catalog, Databricks
  • Design, develop, and implement efficient and reliable data pipelines using Databricks Delta Live Tables.
  • Develop pipelines for structured and unstructured data (i.e. documents, JSON, Parquet, Excel) supporting AI and ML consumption downstream.
  • Implement and extend data models (i.e. fact/dimension tables, domain data marts) following designs defined by the Senior DE and AI team.
  • Write clean, modular, reusable PySpark and SQL transformation logic that is testable, documented, and deployable via CI/CD
  • Write and optimize complex SQL queries using Databricks SQL for data extraction, transformation, and loading.
  • Implement and manage data governance and metadata using Databricks Unity Catalog to ensure data quality and discoverability.
  • Collaborate with data scientists, analysts, and other engineers to understand data requirements and deliver appropriate data solutions.
  • Monitor, troubleshoot, and optimize existing data pipelines and data infrastructure.
  • Stay up-to-date with the latest trends and technologies in data engineering and the Databricks ecosystem.
  • Contribute to the semantic layer that powers Power BI dashboards.

Orchestration and Data Ops- Should Have

  • Build and manage Databricks Workflows: configuring task dependencies, retry policies, and failure alerting
  • Monitor & manage workspaces as Workspace admins.
  • Follow and contribute to CI/CD practices: version control, pull requests, automated testing, and deployment to Dev/QA/Prod environments using Azure DevOps or GitHub Actions.
  • Understanding of infrastructure as a service, Terraform and willingness to learn terraform for automation at infrastructure level.
  • Package and deploy reusable logic as Python libraries following team standards
  • Monitor pipeline health, investigate failures, and resolve data issues within SLA

FinOps Awareness – Should Have
 

  • Write cost-conscious PySpark avoiding unnecessary full scans, optimizing joins, using appropriate cluster types
  • Apply Delta table best practices (i.e. VACUUM, OPTIMIZE, compaction) to manage storage costs
  • Follow cluster policies defined by platform leads and flag unusual resource consumption
     

Experience/QUALIFICATIONS
 

  • 4-6+ years of overall data engineering experience
  • 2+ years of hands-on Azure Databricks experience in production environments
  • Demonstrated ability to build and deliver pipelines — not just maintain or support them
  • Experience working within a defined architecture and contributing to its improvement
  • Comfortable working with multiple data source types — relational, file-based, API
Awareness, Unity, Empowerment:

At Magna, we believe that a diverse workforce is critical to our success. That’s why we are proud to be an equal opportunity employer. We hire on the basis of experience and qualifications, and in consideration of job requirements, regardless of, in particular, color, ancestry, religion, gender, origin, sexual orientation, age, citizenship, marital status, disability or gender identity. Magna takes the privacy of your personal information seriously. We discourage you from sending applications via email or traditional mail to comply with GDPR requirements and your local Data Privacy Law.

AI-Assisted Screening Disclosure

As part of our commitment to a fair, consistent, and efficient recruitment process, we may use artificial intelligence (AI) tools to assist in the initial screening of applications submitted through our Workday system. These tools help identify qualifications and experience that align with the role requirements. Please note that AI is used solely to support our recruiters. Final decisions are always made by the hiring manager and the hiring team. Importantly, no applicant data is shared externally through these AI tools. All information remains securely within our systems and is handled in accordance with our privacy and data protection policies.


Under conditions defined by applicable law, you may have the right to request an explanation of how AI is used to support decision-making.

If you have any questions or concerns about this process, feel free to contact our Talent Attraction team.

Worker Type:
Regular / Permanent
Group:
Magna Corporate

Skills Required

  • 3+ years hands-on PySpark, Delta Lake, Databricks, Workflows, Unity Catalog
  • Design and implement data pipelines using Databricks Delta Live Tables
  • Write clean, modular, reusable PySpark and SQL transformation logic deployable via CI/CD
  • Write and optimize complex SQL queries using Databricks SQL
  • Implement and manage data governance and metadata using Databricks Unity Catalog
  • Monitor, troubleshoot, and optimize production data pipelines and infrastructure
  • 4-6+ years overall data engineering experience
  • 2+ years hands-on Azure Databricks experience in production
  • Experience working with structured and unstructured data (documents, JSON, Parquet, Excel)
  • Build and manage Databricks Workflows and workspace administration
  • Familiarity with CI/CD using Azure DevOps or GitHub Actions
  • Willingness to learn Terraform and infrastructure automation
  • Cost-conscious PySpark coding and Delta table best practices (VACUUM, OPTIMIZE, compaction)

What the Team is Saying

Doug
Dave
Sreeni
Sam
Jim

Magna International Compensation & Benefits Highlights

  • Healthcare Strength Benefits include medical, dental, vision, and pharmacy coverage alongside disability and life insurance, plus mental‑health support via an Employee & Family Assistance Program and wellness programs. Voluntary options such as pet and auto/home insurance further expand the health-related offering.
  • Retirement Support Retirement programs feature a 401(k) with company match and profit sharing, with corporate filings confirming the U.S. 401(k) plan and employer contributions. Other long‑term levers like equity programs and performance bonuses add to wealth‑building potential.
  • Wellbeing & Lifestyle Benefits Company materials highlight a total‑rewards commitment and programs such as donation‑matching and an employee disaster relief fund that provide added support. Work‑life and development elements (e.g., PTO, parental leave, flexible work, and tuition reimbursement) are presented for many roles.

Magna International Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Aurora, Ontario
171,000 Employees
Year Founded: 1957

What We Do

We are a mobility company that innovates like a start-up and thinks like a technology company. This helps us anticipate change in one of the most complex industries in the world and respond quickly. We depend on a team of 171,000 dynamic, entrepreneurial-minded employees in an environment where great ideas flourish. Our presence spans 343 manufacturing operations and 88 product development, engineering and sales centers in 29 countries. We understand that you need a career as unique as you are. Whether you want to advance your existing expertise or try something completely different, we are committed to your growth.

Why Work With Us

At Magna, our engineering team is advancing mobility for everyone and everything. Joining this team means being a part of the design, development, and manufacturing of the world’s most advanced mobility technology. Innovations that move families, shape communities, and improve lives. You can follow your passions and shape your own career path.

Gallery

Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery
Gallery

Magna International Teams

Team
Innovation
Team
Magna International’s Pipeline for Emerging Tech Talent
About our Teams

Magna International Offices

Hybrid Workspace

Employees engage in a combination of remote and on-site work.

Majority of roles are hybrid with flexibility. Please speak with our recruiting team for specific details on hybrid work.

Typical time on-site: Not Specified
HQGlobal Headquarters - Aurora, Ontario, Canada
Bangalore, Karnataka, India
Graz, Austria
Lowell, Massachusetts
Monterrey, Mexico
Munich, Germany
Pune, Maharashtra, India
Sailauf, Germany
Saltillo, Mexico
US Headquarters - Troy, Michigan
Learn more

Similar Jobs

Magna International Logo Magna International

Engineer Shift System

Automotive • Hardware • Robotics • Software • Transportation • Manufacturing
Hybrid
Bengaluru, Karnataka, IND
171000 Employees

Magna International Logo Magna International

Engineer CAE MASTA

Automotive • Hardware • Robotics • Software • Transportation • Manufacturing
Hybrid
Bengaluru, Karnataka, IND
171000 Employees

Magna International Logo Magna International

Engineer CAE Durability

Automotive • Hardware • Robotics • Software • Transportation • Manufacturing
Hybrid
Bengaluru, Karnataka, IND
171000 Employees

Magna International Logo Magna International

Analyst, Concur

Automotive • Hardware • Robotics • Software • Transportation • Manufacturing
Hybrid
Bangalore, Bengaluru, Karnataka, IND
171000 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account