Manager, Supercomputer Site Reliability Engineering - Memphis

Posted Yesterday
Be an Early Applicant
2 Locations
In-Office
Mid level
Information Technology
The Role
Leads a multi-site supercomputer SRE organization responsible for compute uptime, cluster and facilities reliability, incident response, hardware root-cause analysis, monitoring, automation, vendor accountability, and repair velocity. Builds 24/7 operational teams, defines reliability metrics and procedures, partners with engineering and data center operations, and supports high-performance GPU infrastructure across multiple locations.
Summary Generated by Built In

SpaceX was founded under the belief that a future where humanity is out exploring the stars is fundamentally more exciting than one where we are not. Today, SpaceX is actively developing the technologies to make this possible, with the ultimate goal of making life multiplanetary and expanding consciousness to the stars.
SpaceXSI is building Super Intelligence to help humanity understand the universe. Our teams are building the full Super Intelligence stack: from our Colossus supercomputers, to Grok models, to our suite of applications, including Grok Bot, Grok Build, Grok Imagine, Grok Voice, and X. We're creating a powerful flywheel where more compute trains better models, better models power more useful products, and what we learn from people using those products shapes what we build next.

ABOUT THE ROLE:

SpaceXSI operates multi-campus, hyperscale GPU infrastructure where cluster availability directly determines training throughput, customer handover commitments, and capital efficiency of the fleet. The Manager of Supercomputer Site Reliability Engineering will be responsible for both compute and infrastructure performance, and will be responsible for significantly increasing the SRE team size. On the compute side, focus will be put on continuous site monitoring, cross-discipline incident command, deep hardware root-cause analysis (RCA), and emphasis on software systems that orchestrate repair velocity. Additional focus will be on physical infrastructure reliability as well. Understanding system margins and resiliency will be key. 

Site Reliability engineering manager owns >99% compute uptime outcome, drives unplanned facilities compute losses to 0, ensures efficient allocation of > 3GW site power, upgrade robust fault-free megapack failover systems capable of supporting all halls for >4 hours (160 MWhr per hall), drives robust hardware inventory systems and repair velocity to eliminate hardware chokepoints.

RESPONSIBILITIES:
  • Serve as the senior operational authority for the SRE organization and represent production reliability concerns in engineering and leadership discussions.
  • Lead integrated Supercompute systems engineering reliability team responsible for reactive and increasingly proactive cluster and facilities health and reliance initiatives
  • Build out the site reliability engineering sub-teams: compute hardware failure analysis, software tools, cluster monitoring, and integrated site reliability engineering

Drive operational superiority

  • Define SRE organizational structure and develop culture of 24/7 operational excellence
    • Physics limit anomaly response times and robust proactive health processes
    • Define on-call responsibilities, incident management protocols, leadership structure, and communication discipline
    • Develop training schemes for tough, competent, and disciplined monitoring and response teams
  • Establish clear operational interfaces with Datacenter Operations, engineering teams, vendors, and other service owners
    • Own vendor oversight and accountability mechanisms
    • Drive mean time to repair toward physics limits
  • Define and continuously improve operational procedures and analysis documentation mechanisms with strong emphasis on intelligent automation
    • Lead robustification of documentation workflow
    • Ensure efficient signal to noise in execution
  • Define operational targets, health indicators, alerting standards, and incident severity definitions that facilitate measurable team performance insights and accountability mechanisms

Cultivate engineering excellence

  • Drive high technical standard for hardware root cause analysis and remediation teams ensuring industry-leading hardware failure diagnostic expertise
  • Integrate with platform engineering teams to develop sophisticated hardware and system understanding to drive high operational reliability
  • Drive repeat hardware issues to a minimum and lead automation efforts to eliminate manual overhead for unavoidable remediation needs
BASIC QUALIFICATIONS:
  • Bachelor’s degree and 3+ years of experience working in a large scale operations with 2+ years leading people leaders of technical teams or 10+ years of experience working in a large scale operations with 5+ years leading people leaders of technical teams.
PREFERRED SKILLS AND EXPERIENCE:
  • Proven ability to lead large, multi-site, 24/7 operations organizations in fast-paced, high-responsibility settings.
  • Deep expertise in server hardware, cluster reliability, and data center technologies, from deployment through lifecycle management.
  • Experience supporting compute-heavy environments like AI, machine learning, or high-performance computing at scale.
  • A track record of owning uptime, Service Level Agreements, or reliability metrics for large compute clusters.
  • Experience leading site reliability engineering or equivalent reliability-focused teams, including root cause analysis and procedure ownership.
  • Strong analytical skills and the ability to explain technical concepts clearly to diverse audiences, from technicians to executive and tenant partners.
  • A history of partnering with vendors at scale, driving mean time to repair down, and scaling operations across multiple sites.
  • Familiarity with tooling and automation (e.g., Jira, Python, Bash) used to monitor cluster health and improve team efficiency.
  • Enthusiasm for SpaceXAI's mission to accelerate human discovery and unravel the universe.
  • Ability to thrive in a dynamic, mission-focused environment with on-call ownership of cluster-impacting events.
ADDITIONAL REQUIREMENTS:
  • Willingness to travel frequently to data center locations to support operations across sites.
  • Physical capability to handle data center tasks, including lifting up to 50 lbs unassisted, standing for long periods, and occasional ladder use.
  • Must be willing to work extended hours and/or weekends as needed.

SpaceXSI is an equal opportunity employer. For details on data processing, view our Recruitment Privacy Notice.

Skills Required

  • Bachelor's degree
  • 3+ years of experience working in large-scale operations and 2+ years leading people leaders of technical teams
  • Alternatively, 10+ years of experience working in large-scale operations and 5+ years leading people leaders of technical teams
  • Experience leading large, multi-site, 24/7 operations organizations
  • Deep expertise in server hardware, cluster reliability, and data center technologies
  • Experience supporting AI, machine learning, or high-performance computing environments at scale
  • Experience owning uptime, service-level agreements, or reliability metrics for large compute clusters
  • Experience leading site reliability engineering or equivalent reliability-focused teams
  • Strong analytical and technical communication skills
  • Experience partnering with vendors at scale and improving mean time to repair
  • Familiarity with Jira, Python, and Bash
  • Willingness to travel frequently to data center locations
  • Ability to lift up to 50 pounds unassisted, stand for long periods, and occasionally use ladders
  • Willingness to work extended hours and weekends as needed

xAI Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about xAI and has not been reviewed or approved by xAI.

  • Healthcare Strength — Healthcare is described as comprehensive, including medical, dental, vision, mental health support, and fertility benefits. Additional protections such as disability and life insurance, plus HSA/FSA options, are also outlined.
  • Equity Value & Accessibility — Equity is positioned as a core component of total rewards alongside high base salaries for technical roles. Pre‑IPO stock grants are highlighted as enabling participation in potential company growth.
  • Leave & Time Off Breadth — The package includes flexible or generous PTO, paid holidays, and paid sick days. Company materials also describe flexible vacation policies intended to help manage fast‑paced workloads.

xAI Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Palo Alto, CA
96 Employees

What We Do

Understand the Universe

Similar Jobs

Citizens Logo Citizens

Security Engineer

Digital Media • Fintech • Information Technology • Machine Learning • Financial Services • Cybersecurity • Automation
In-Office or Remote
2 Locations
17000 Employees
150K-190K Annually

Citizens Logo Citizens

Director Commercial Banking Marketing

Digital Media • Fintech • Information Technology • Machine Learning • Financial Services • Cybersecurity • Automation
In-Office or Remote
2 Locations
17000 Employees
155K-200K Annually

Citizens Logo Citizens

Distinguished Engineer - AI Security

Digital Media • Fintech • Information Technology • Machine Learning • Financial Services • Cybersecurity • Automation
In-Office or Remote
2 Locations
17000 Employees
175K-250K Annually

Citizens Logo Citizens

Senior Program Manager

Digital Media • Fintech • Information Technology • Machine Learning • Financial Services • Cybersecurity • Automation
In-Office or Remote
2 Locations
17000 Employees
175K-250K Annually

Similar Companies Hiring

Axle Health Thumbnail
Artificial Intelligence • Healthtech • Information Technology • Logistics
Santa Monica, CA
25 Employees
NODA AI Thumbnail
Artificial Intelligence • Information Technology • Software • Cybersecurity
Sydney, AU
54 Employees
Golden Pet Brands Thumbnail
Digital Media • eCommerce • Information Technology • Marketing Tech • Pet • Retail • Social Media
El Segundo, California
178 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account