ML Engineer, Inference Optimization

Posted Yesterday
San Francisco, CA, USA
In-Office
200K-320K Annually
Entry level
Artificial Intelligence • Big Data • Hardware • Machine Learning
The Role
Optimize machine-learning inference for latency, throughput, and cost. The role profiles pipelines, improves kernels, batching, quantization, compilation, serving, and hardware utilization, while building serving and evaluation infrastructure. It partners with research and product teams to make models affordable at scale and treats compute cost as a core performance metric.
Summary Generated by Built In
About Build AI

Build AI is the data hyperscaler for Physical AI. We're vertically integrated across hardware, manufacturing, logistics, collection, and model training to scale the physical labor dataset orders of magnitude faster than anyone in the world.

Job Summary

Inference is about 90% of compute spend. Economics are heavily driven by inference optimization. We’re hiring someone to make inference cheaper, faster, and good enough that we can scale the data engine and the product without the GPU bill eating the company.

Key Responsibilities
  • Own inference performance: latency, throughput, and cost per unit of work (tokens, frames, or jobs)

  • Cut the 90% compute line: kernels, batching, quantization, compilation, serving, and hardware utilization

  • Profile pipelines (Nsight, PyTorch Profiler, or equivalent), find the real bottleneck, and ship the fix

  • Work with research and product so models that are accurate are also affordable to run at scale

  • Build the serving and eval path so experiments don’t hide the inference bill

  • Measure cost as a first-class metric, not an afterthought once quality is “done”

You may be a good fit if you have (Must-have qualifications)
  • Strong ML / systems engineer with real inference optimization experience (serving, compilers, CUDA/kernels, quantization, or similar)

  • Comfortable in Python and in C++ or Rust for performance-critical paths

  • You think in dollars and tokens/frames per second, not only in accuracy tables

  • Familiarity with PyTorch (or JAX) and with profiling tools

  • Comfortable in a small research team shipping under cost pressure

Strong candidates may also have experience with (Nice-to-have qualifications)
  • CUDA, kernels, compilers (TVM, MLIR, TensorRT), or quantization in production

  • You have owned GPU/accelerator cost as a first-class metric

  • Serving stacks for video or large models

  • Understanding of memory hierarchy, data movement, and low-precision compute

Benefits
  • Competitive pay

  • Medical, dental, and vision packages with generous premium coverage

  • $500 per month credit for waiving medical benefits

  • Housing subsidy of $2k per month for those living within walking distance of the office

  • Relocation support for those moving to San Francisco (Financial District) or Shenzhen (Nanshan)

  • Various wellness benefits covering fitness, mental health, and more

  • Daily lunch and dinner in our office

  • Unlimited compute budget subject to ROI justification

  • Unlimited Codex and Claude credits

  • Travel

How we're different

Build believes in the Bitter Lesson. By taking a general approach of learning from humans, our addressable market is all physical labor.

We are a fully in-person team in San Francisco (Financial District) and Shenzhen (Nanshan), and greatly value engineering skills. We do not have boundaries between engineering and research, and we expect all of our technical staff to contribute to both and work across disciplines as needed.

Build AI is an equal opportunity employer. We review every application. If you do not meet every bullet, still apply. Questions: [email protected]

Skills Required

  • Strong machine-learning and systems engineering experience with inference optimization, including serving, compilers, CUDA/kernels, quantization, or similar
  • Proficiency in Python
  • Proficiency in C++ or Rust for performance-critical paths
  • Familiarity with PyTorch or JAX
  • Familiarity with profiling tools
  • Ability to work in a small research team and ship under cost pressure
  • Production experience with CUDA, kernels, compilers such as TVM, MLIR, or TensorRT, or quantization
  • Experience owning GPU or accelerator cost as a first-class metric
  • Experience with serving stacks for video or large models
  • Understanding of memory hierarchy, data movement, and low-precision compute
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
30 Employees
Year Founded: 2025

What We Do

Build AI is a public benefit corporation and data hyperscaler for Physical AI. It integrates hardware manufacturing, logistics, data collection, and model training to scale egocentric physical-labor datasets for researchers and labs. Its mission is to solve physical labor and unlock human potential, advancing robotics and physical superintelligence. The company operates in San Francisco and Shenzhen and develops economically useful human-data infrastructure.

Similar Jobs

Nebius Logo Nebius

Senior Machine Learning Engineer

Artificial Intelligence • Information Technology • Consulting
In-Office
Palo Alto, CA, USA
473 Employees
195K-262K Annually

Pika Logo Pika

Machine Learning Engineer

Information Technology
In-Office
Palo Alto, CA, USA
29 Employees
250K-350K Annually

Rhoda AI Logo Rhoda AI

Machine Learning Engineer

Artificial Intelligence • Computer Vision • Hardware • Robotics
In-Office
Mountain View, CA, USA
73 Employees
175K-250K Annually

Similar Companies Hiring

Revel Thumbnail
Aerospace • Hardware • Robotics • Software
Marina Del Rey, California
60 Employees
Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
30 Employees
Vega Thumbnail
Artificial Intelligence • Automotive • Insurance • Transportation
US
43 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account