Senior Software Engineer, Resilience Engineering - DGX Cloud

Posted 3 Days Ago
Be an Early Applicant
Hiring Remotely in Santa Clara, CA, USA
In-Office or Remote
184K-357K Annually
Senior level
Artificial Intelligence • Computer Vision • Hardware • Robotics • Metaverse
The Role
Build NVIDIA DGX Cloud’s organization-wide reliability strategy and SLO program. Lead high-severity incident response, develop production code and data-platform tooling, implement chaos engineering and failure-injection practices, and improve operational standards across teams. The role requires extensive experience operating large-scale production systems, understanding complex failure modes, and influencing engineering teams through technical expertise.
Summary Generated by Built In

NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It’s a unique legacy of innovation that’s fueled by great technology—and amazing people. Today, we’re tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what’s never been done before takes vision, innovation, and the world’s best talent. As an NVIDIAN, you’ll be immersed in a diverse, supportive environment where everyone is inspired to do their best work. Come join the team and see how you can make a lasting impact on the world.


Are you passionate about building world-class reliability systems? Join NVIDIA as a Senior Software Engineer - Resilience Engineering, DGX Cloud, and be a pivotal part of a team that redefines operational excellence. Our team is at the forefront of redefining how DGX Cloud approaches reliability, making it an outstanding opportunity to develop strategies and drive innovation. We're looking for a seasoned engineer with experience in running large-scale systems and a deep understanding of operational practices.


What you'll be doing:

  • Build org-wide reliability strategy, guiding how NVIDIA matures its operational practices in a 24/7 environment.
  • Stand up a rigorous SLO program, defining and maintaining high standards across teams.
  • Lead incident response for high severity incidents, ensuring low drama and high signal resolution.
  • Build and improve production code daily, enhancing our data platform and related tooling.
  • Implement chaos engineering, failure injection, and resilience testing to elevate our team's standard practices.
  • Improve standards by setting an example with your hands-on experience and leadership.

What we need to see:

  • Deep, hands-on experience running large-scale production systems with a proven track record.
  • A detailed understanding of failure modes in large systems, including cascading dependencies and retry storms.
  • Strong software engineering skills with current, hands-on experience in Go, Python, or similar languages.
  • Proven experience in establishing and maintaining an SLO program with operational rigor.
  • Practical experience in reliability fields such as chaos engineering and failure injection.
  • The ability to influence across team boundaries through credibility and expertise.
  • 10+ years of industry experience with a Bachelor's or Master's degree, or equivalent experience operating systems at scale.

Ways to stand out from the crowd:

  • Experience within a world-class reliability function like Google SRE or Meta production engineering.
  • Expertise in operating GPU, HPC, or AI training infrastructure with outstanding failure modes.
  • A track record of measurable reliability improvements within an organization.
  • Proficiency with modern observability and operational tools like Prometheus, OpenTelemetry, Grafana, PagerDuty, and Rootly.

Widely considered to be one of the technology world’s most desirable employers, NVIDIA offers highly competitive salaries and a comprehensive benefits package. As you plan your future, see what we can offer to you and your family www.nvidiabenefits.com/ 

Your base salary will be determined based on your location, experience, and other objective job-related factors. The expected base salary ranges for this position are:
184,000 USD – 287,500 USD for Level 4; and 224,000 USD – 356,500 USD for Level 5.

You will also be eligible for equity and benefits.

Applications for this job will be accepted at least until November 16, 2026.

This posting is for an existing vacancy. 

NVIDIA uses AI tools in its recruiting processes.

NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

Skills Required

  • Deep, hands-on experience running large-scale production systems with a proven track record
  • Detailed understanding of failure modes in large systems, including cascading dependencies and retry storms
  • Strong software engineering skills with current hands-on experience in Go, Python, or similar languages
  • Experience establishing and maintaining an SLO program with operational rigor
  • Practical experience with chaos engineering and failure injection
  • Ability to influence across team boundaries through credibility and expertise
  • 10+ years of industry experience
  • Bachelor’s or Master’s degree, or equivalent experience operating systems at scale
  • Experience in a world-class reliability function such as Google SRE or Meta production engineering
  • Expertise operating GPU, HPC, or AI training infrastructure
  • Track record of measurable reliability improvements
  • Proficiency with Prometheus, OpenTelemetry, Grafana, PagerDuty, and Rootly

NVIDIA Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about NVIDIA and has not been reviewed or approved by NVIDIA.

  • Equity Value & Accessibility — Feedback suggests equity grants, stock options/RSUs, and a discounted ESPP materially boost total compensation and can deliver significant wealth-building when the stock performs well.
  • Healthcare Strength — Feedback suggests medical, dental, and vision plans with in-network preventive care at 100%, telehealth, and concierge-style support are comprehensive and valued.
  • Parental & Family Support — Feedback suggests up to 22 weeks fully paid parental leave, fertility coverage, and adoption/surrogacy reimbursement provide strong family-focused benefits.

NVIDIA Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Santa Clara, CA
21,960 Employees
Year Founded: 1993

What We Do

NVIDIA’s invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined modern computer graphics, and revolutionized parallel computing. More recently, GPU deep learning ignited modern AI — the next era of computing — with the GPU acting as the brain of computers, robots, and self-driving cars that can perceive and understand the world. Today, NVIDIA is increasingly known as “the AI computing company.”

Similar Jobs

Dynatrace Logo Dynatrace

Data Analyst

Artificial Intelligence • Big Data • Cloud • Information Technology • Software • Big Data Analytics • Automation
Remote or Hybrid
United States
5600 Employees
116K-145K Annually
Easy Apply
Remote
United States
1100 Employees
119K-138K Annually

Collectors Logo Collectors

Pricing Results Specialist

Consumer Web • eCommerce • Machine Learning • Software • Sports • Analytics
In-Office or Remote
Long Beach, CA, USA
2246 Employees
21-21 Hourly
Remote
USA
3000 Employees

Similar Companies Hiring

Unusual Machines, Inc. Thumbnail
Hardware • Robotics
Orlando, FL
300 Employees
Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
33 Employees
Vega Thumbnail
Artificial Intelligence • Automotive • Insurance • Transportation
US
65 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account