Principal Software Engineer - Spark

Reposted 5 Days Ago
Be an Early Applicant
San Jose, CA, USA
In-Office
270K-320K Annually
Expert/Leader
Big Data • Software • Analytics
The Role
Lead the Cloudera Data Engineering experience using Spark and other technologies, collaborate with teams, drive technical roadmaps, and mentor engineers.
Summary Generated by Built In

Business Area:

Engineering

Seniority Level:

Director

Job Description: 

At Cloudera, we empower people to transform complex data into clear and actionable insights. With as much data under management as the hyperscalers, we're the preferred data partner for the top companies in almost every industry.  Powered by the relentless innovation of the open source community, Cloudera advances digital transformation for the world’s largest enterprises.

Cloudera Data Engineering is the next-generation cloud-native service that helps our customers run large-scale data engineering workflows made up of industry-standard big data processing frameworks like Apache Spark, Apache Airflow, Iceberg with just a few clicks, across both on-premises and public cloud environments.

Cloudera Data Engineering is a next-generation cloud-native service that enables customers to run large-scale data engineering workflows using industry-standard big data technologies such as Apache Spark, Apache Airflow, and Apache Iceberg  with just a few clicks across both on-premises and public cloud environments.

We are seeking a Principal Staff Engineer with a strong technical background in the data infrastructure space to lead the Cloudera Data Engineering experience for all customers using Cloudera Data Engineering Spark, Airflow, and Lakehouse. This high-impact IC role offers the opportunity to shape the future of Cloudera’s Data Engineering and Lakehouse  products across multiple cloud environments, impacting thousands of customers worldwide.

This role is not eligible for immigration sponsorship or relocation.

As a Principal Software Engineer you will:

  • Drive the multi-year technical roadmap and architectural vision for Cloudera Data Engineering.

  • Gain deep technical knowledge across the data services technical stack, with a focus on Spark, Airflow, Iceberg, and apply this expertise in your daily work.

  • Foster engineering excellence through technical mentorship, design reviews, and architectural guidance.

  • Collaborate with product, engineering, and cross-functional partners, leading the delivery of several large, critical features in Cloudera’s data engineering experience.

  • Work on large-scale distributed systems, ranging from hundreds to thousands of nodes in production clusters.

  • Bring passion for programming, clean coding practices, attention to detail, and a strong focus on quality.

We are excited about you if you have:

  • Relevant studies / BS or MS in Computer Science or related field

  • 10+ years of experience as a Software Engineer in the data infrastructure space

  • Strong understanding of at least one of the following languages: Java, Scala, C++, Python, GoLang. And interested to learn the languages we’re using.

  • Passionate about programming, clean coding habits, attention to detail, and focus on quality

  • Deep expertise in distributed data processing systems and cloud-native architectures.

  • Excellent communication and collaboration skills

  • Experience with containerization (Kubernetes, Docker).

  • Experience with using/developing Apache Spark/Airflow or other related technologies.

  • Experience with public cloud (AWS/Azure/GCP) and/or private cloud (OpenShift/Rancher)

  • (Most importantly) An open-minded attitude, desire to learn new things and build great products

You might also have…

  • Contributed to open-source projects.

  • Strong understanding of modern Lakehouse architectures, open table formats, and metadata/catalog services.

  • Experience with large-scale, distributed systems design and development with an understanding of scaling, performance, and scheduling.

  • Solid experience with at least one cloud service (AWS, Azure, GCP, OpenShift)

The expected base salary range for this role in California is $270 - $320k  

The salary will vary depending on your job-related skills, experience and location

What you can expect from us:

  • Generous PTO Policy 

  • Support work life balance with Unplugged Days

  • Flexible WFH Policy 

  • Mental & Physical Wellness programs 

  • Phone and Internet Reimbursement program 

  • Access to Continued Career Development 

  • Comprehensive Benefits and Competitive Packages 

  • Paid Volunteer Time

  • Employee Resource Groups

EEO/VEVRAA

#LI-AO1

#LI-HYBRID

Skills Required

  • 10+ years of experience as a Software Engineer in the data infrastructure space
  • Relevant studies / BS or MS in Computer Science or related field
  • Strong understanding of at least one of the following languages: Java, Scala, C++, Python, GoLang
  • Deep expertise in distributed data processing systems and cloud-native architectures
  • Experience with containerization (Kubernetes, Docker)
  • Experience with using/developing Apache Spark/Airflow or other related technologies
  • Experience with public and/or private cloud environments (AWS, Azure, GCP, OpenShift)

Cloudera Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Cloudera and has not been reviewed or approved by Cloudera.

  • Leave & Time Off Breadth Time off includes generous PTO and holidays plus recurring company‑wide Unplugged Days that provide regular recharge time. Volunteer time off and flexible scheduling options further expand usable leave.
  • Healthcare Strength Health coverage spans comprehensive medical, dental, and vision alongside EAP, wellness sessions, and U.S. gym reimbursement. These elements position healthcare as a strong anchor within the package.
  • Strong & Reliable Incentives Compensation often includes variable incentives and long‑term incentive programs with annual bonuses commonly offered. Sales and other revenue roles show competitive on‑target earnings when goals are met, reinforcing the incentive structure.

Cloudera Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Palo Alot, CA
3,092 Employees
Year Founded: 2008

What We Do

At Cloudera, we believe that data can make what is impossible today, possible tomorrow. We empower people to transform complex data into clear and actionable insights. Cloudera delivers an enterprise data cloud for any data, anywhere, from the Edge to AI. Powered by the relentless innovation of the open source community,

Similar Jobs

PwC Logo PwC

Connected Supply Chain, Planning - Kinaxis, Manager

Artificial Intelligence • Professional Services • Business Intelligence • Consulting • Cybersecurity • Generative AI
Hybrid
18 Locations
370000 Employees
99K-232K Annually

PwC Logo PwC

Strategy& Financial Services - AWM Consulting Manager

Artificial Intelligence • Professional Services • Business Intelligence • Consulting • Cybersecurity • Generative AI
Hybrid
14 Locations
370000 Employees
99K-232K Annually

PwC Logo PwC

Connected Supply Chain, Planning - Kinaxis, Senior Associate

Artificial Intelligence • Professional Services • Business Intelligence • Consulting • Cybersecurity • Generative AI
Hybrid
18 Locations
370000 Employees
77K-202K Annually

Cox Enterprises Logo Cox Enterprises

Communications Specialist

Artificial Intelligence • Automotive • Greentech • Information Technology • Machine Learning • Software • Cybersecurity
Remote or Hybrid
United States
50000 Employees
61K-92K Annually

Similar Companies Hiring

Hanover Park Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
42 Employees
Kepler  Thumbnail
Fintech • Software
New York, New York
6 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
60 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account