Senior Software Engineer - Storage Control Plane

Posted 2 Days Ago
2 Locations
Remote or Hybrid
266K-395K Annually
Senior level
Software
The Role
Design and implement a vendor-agnostic storage control plane for high-performance AI clusters. Build Kubernetes CRD-based orchestration, reconciliation loops, multi-tenant isolation, capacity and placement engines, and observability for petabyte-scale storage across diverse vendors and protocols.
Summary Generated by Built In

Lambda, The Superintelligence Cloud, is a leader in AI cloud infrastructure serving tens of thousands of customers. Our customers range from AI researchers to enterprises and hyperscalers. Lambda's mission is to make compute as ubiquitous as electricity and give everyone the power of superintelligence. One person, one GPU.

If you'd like to build the world's best AI cloud, join us.

*Note: This position requires presence in our San Francisco or Bellevue office location 4 days per week; Lambda’s designated work-from-home day is currently Tuesday.


In the world of distributed AI training and inference, raw GPU and CPU horsepower is just a part of the story. High-performance networking and storage are the critical components that enable and unite these systems, making groundbreaking AI training and inference possible.

The Lambda Infrastructure Engineering organization forges the foundation of high-performance AI clusters by welding together the latest in AI storage, networking, GPU and CPU hardware.

Our expertise lies at the intersection of:

  • High-Performance Distributed Storage Solutions and Protocols: We engineer the protocols and systems that serve massive datasets at the speeds demanded by modern clustered GPUs.

  • Dynamic Networking: We design advanced networks that provide multi-tenant security and intelligent routing without compromising performance, using the latest in AI networking hardware.

  • Compute Clustering and Virtualization: We enable cutting-edge virtualization and clustering that allows AI researchers and engineers to focus on AI workloads, not AI infrastructure, unleashing the full compute bandwidth of clustered GPUs.

AI training and inference relies on petabytes of data hosted on large, high-performance storage arrays. At Lambda, the Infrastructure Storage Team’s job is to ensure that the data powering AI is fast, performant, and available across a variety of access protocols (fit for purpose).

We're looking for an experienced Senior Software Engineer to join our storage team. You'll join a team responsible for developing and implementing our next-generation storage software. This role requires expertise in distributed systems, and an in-depth understanding of file, block, and object storage protocols. You'll work on building scalable and resilient storage control plane that power our AI and machine learning infrastructure.

What You’ll Do

  • Design and build a vendor-agnostic control plane that provisions, scales, heals, and meters storage across the platforms our customers actually demand, VAST Data, WEKA, DDN, Pure, NetApp, Ceph, MinIO, and the ones that don't exist yet.

  • Define the internal abstraction layer that hides vendor-specific APIs, failure semantics, QoS knobs, and telemetry formats behind one declarative interface, so a new vendor integration is a driver, not a re-architecture.

  • Build reconciliation-loop and CRD-based orchestration (Kubernetes controllers, operators, custom schedulers) that manages capacity, tenancy, encryption domains, and placement across data centers and availability zones.

  • Own multi-tenant isolation end to end: namespace and subsystem partitioning, per-tenant QoS and rate limiting, credential and key lifecycle, blast-radius containment, noisy-neighbor detection.

  • Design the capacity and placement engine: PCIe-topology-aware, NUMA-aware, failure-domain-aware. On our platforms a drive behind the same PCIe switch as the GPU it serves beats a faster drive on a different root port, and the control plane needs to know that.

  • Instrument everything: SLI/SLO definitions, fleet-wide performance regression detection, and the observability pipeline that makes a petabyte fleet debuggable at 3 a.m.

You

  • Bachelor's or Master's degree in Computer Science or a related field.

  • 5+ years of experience in software development for storage systems.

  • Proven experience with distributed systems programming and concepts such as load balancers, data-durability, consensus algorithms, fault tolerance, and data consistency.

  • Strong programming skills in languages such as C, C++, Go, or Python.

  • Experience with Linux kernel internals and system-level programming.

  • Experience with one or more storage protocols (e.g. S3, NFS) and file systems such as Ceph, DAOS, or similar.

  • Familiarity with containerization technologies like Docker and Kubernetes and running production workloads in these environments.

  • Familiarity with CI/CD and QA practices for distributed systems development environments.

Nice to Have

  • Experience with AI/ML workloads and the unique storage challenges they present.

  • Knowledge of data center networking and high-speed interconnects (e.g., InfiniBand, RoCE).

  • Experience with performance tuning and optimization of storage systems.

  • Familiarity with hardware acceleration technologies, specifically GPUs and DPUs.

  • Production experience with VAST Data, WEKA, DDN, Pure, NetApp, or IBM Storage Scale.

  • Ceph at 100 PB+ in HPC or AI environments.

  • CXL memory pooling, computational storage, ZNS SSDs, EDSFF.

  • Published or presented at SNIA SDC, FAST, USENIX ATC, LSFMM+BPF, OCP, SC, or similar.

Salary Range Information

The annual salary range for this position has been set based on market data and other factors. However, a salary higher or lower than this range may be appropriate for a candidate whose qualifications differ meaningfully from those listed in the job description.

About Lambda

  • Founded in 2012, with 500+ employees, and growing fast

  • Our investors notably include TWG Global, US Innovative Technology Fund (USIT), Andra Capital, SGW, Andrej Karpathy, ARK Invest, Fincadia Advisors, G Squared, In-Q-Tel (IQT), KHK & Partners, NVIDIA, Pegatron, Supermicro, Wistron, Wiwynn, Gradient Ventures, Mercato Partners, SVB, 1517, and Crescent Cove

  • We have research papers accepted at top machine learning and graphics conferences, including NeurIPS, ICCV, SIGGRAPH, and TOG

  • Our values are publicly available: https://lambda.ai/careers

  • We offer generous cash & equity compensation

  • Health, dental, and vision coverage for you and your dependents

  • Wellness and commuter stipends for select roles

  • 401k Plan with 2% company match (USA employees)

  • Flexible paid time off plan that we all actually use

Equal Opportunity Employer

Lambda is an Equal Opportunity employer. Applicants are considered without regard to race, color, religion, creed, national origin, age, sex, gender, marital status, sexual orientation and identity, genetic information, veteran status, citizenship, or any other factors prohibited by local, state, or federal law.

Skills Required

  • Bachelor's or Master's degree in Computer Science or related field
  • 5+ years of experience in software development for storage systems
  • Proven experience with distributed systems concepts (consensus, fault tolerance, data durability, consistency)
  • Strong programming skills in C, C++, Go, or Python
  • Experience with Linux kernel internals and system-level programming
  • Experience with storage protocols and file systems (e.g., S3, NFS, Ceph, DAOS)
  • Familiarity with containerization and orchestration (Docker, Kubernetes) in production
  • Familiarity with CI/CD and QA practices for distributed systems
  • Experience with AI/ML workloads and AI storage challenges
  • Knowledge of data center networking and high-speed interconnects (InfiniBand, RoCE)
  • Experience with performance tuning and optimization of storage systems
  • Familiarity with hardware acceleration technologies (GPUs, DPUs)
  • Production experience with VAST Data, WEKA, DDN, Pure, NetApp, or IBM Storage Scale
  • Experience operating Ceph at very large scale (100 PB+)
  • Familiarity with CXL memory pooling, computational storage, ZNS SSDs, EDSFF
  • Published or presented at relevant industry conferences (SNIA SDC, FAST, USENIX, OCP, SC)

Lambda Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Lambda and has not been reviewed or approved by Lambda.

  • Fair & Transparent Compensation Pay is considered competitive for an AI infrastructure company, with posted ranges and observed offers indicating strong packages for senior technical roles. Compensation is often characterized as competitive or top‑shelf, aligning with market expectations.
  • Healthcare Strength Health, dental, and vision coverage are characterized as strong, with broad‑network plans noted and positive experiences highlighted. This foundation supports overall satisfaction with core insurance benefits.
  • Leave & Time Off Breadth Flexible or unlimited PTO is described as actually used, complemented by paid holidays and sick time. Generous parental leave examples further expand the time‑off offering.

Lambda Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: San Francisco, CA
106 Employees
Year Founded: 2012

What We Do

Lambda provides computation to accelerate human progress. We're a team of Deep Learning engineers building the world's best GPU workstations and servers. Our products power engineers and researchers at the forefront of human knowledge. Customers include Microsoft, MIT, Los Alamos National Lab, Disney, Tencent, Kaiser Permanente, Stanford, Harvard, Caltech, and the Department of Defense.

Similar Jobs

Lambda Logo Lambda

Senior Software Engineer

Artificial Intelligence • Cloud • Machine Learning • Infrastructure as a Service (IaaS)
Remote or Hybrid
2 Locations
750 Employees
266K-395K Annually
Remote or Hybrid
Office, Machaze, Manica, MOZ
93 Employees

Auror Logo Auror

Senior Systems Engineer

Artificial Intelligence • Big Data • Retail • Security • Social Impact • Software • Business Intelligence
Remote or Hybrid
Office, Machaze, Manica, MOZ
212 Employees
120K-150K Annually

Auror Logo Auror

Staff Software Engineer

Artificial Intelligence • Big Data • Retail • Security • Social Impact • Software • Business Intelligence
Remote or Hybrid
Office, Machaze, Manica, MOZ
212 Employees
120K-170K Annually

Similar Companies Hiring

Hanover Park Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
42 Employees
Kepler  Thumbnail
Fintech • Software
New York, New York
6 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
60 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account