Inference Software Engineer

Reposted 4 Days Ago
Be an Early Applicant
San Jose, CA, USA
In-Office
175K-275K Annually
Mid level
Artificial Intelligence • Hardware • Software
The Role
You will port models to new architectures, optimize runtime and communication layers, and develop performance profiling tools for inference operations.
Summary Generated by Built In

About Etched

Etched is building hardware for frontier intelligence. We co-design chips, racks, software, and manufacturing to deliver best-in-class throughput and latency across both prefill and decode workloads. Our first products are heavily focused on inference. Backed by hundreds of millions from top-tier investors and staffed by leading engineers, Etched is redefining the infrastructure layer for the fastest growing industry in history.

Key responsibilities

  • Support porting state-of-the-art models to our architecture. Help build programming abstractions and testing capabilities to rapidly iterate on model porting.

  • Build, enhance, and scale our runtime, including multi-node inference, intra-node execution, state management, and robust error handling.

  • Optimize routing and communication layers using our collectives.

  • Utilize performance profiling and debugging tools to identify bottlenecks and correctness issues.

You may be a good fit if you have

  • Proficiency in C++ or Rust.

  • Understanding of performance-sensitive or complex distributed software systems like Linux internals, accelerator architectures (e.g. GPUs, TPUs), Compilers, or high-speed interconnects (e.g. NVLink, InfiniBand).

  • Familiarity with PyTorch or JAX.

  • Ported applications to non-standard accelerator hardware or hardware platforms.

Strong candidates may also have experience with (Nice-to-have qualifications)

  • Developed low-latency, high-performance applications using both kernel-level and user-space networking stacks.

  • Deep understanding of distributed systems concepts, algorithms, and challenges, including consensus protocols, consistency models, and communication patterns.

  • Solid grasp of Transformer architectures, particularly Mixture-of-Experts (MoE).

  • Built applications with extensive SIMD (Single Instruction, Multiple Data) optimizations for performance-critical paths.

Benefits

  • Medical, dental, and vision packages with generous premium coverage

    • $500 per month credit for waiving medical benefits

  • Housing subsidy of $2k per month for those living within walking distance of the office

  • Relocation support for those moving to San Jose (Santana Row)

  • Various wellness benefits covering fitness, mental health, and more

  • Daily lunch + dinner in our office

  • Unlimited compute budget subject to ROI justification

How we’re different

Etched believes in the Bitter Lesson. We are the first inference-focused frontier AI system, betting early on transformer and transformer-like architectures and on increasing model sizes. Our addressable market is the entirety of inference, unlike many of our competitors.

We are a fully in-person team in San Jose (Santana Row), and greatly value engineering skills. We do not have boundaries between engineering and research, and we expect all of our technical staff to contribute to both and work across disciplines as needed.

Skills Required

  • Proficiency in Rust and/or C++
  • Familiarity with PyTorch and/or JAX
  • Knowledge of Linux internals and accelerator architectures
  • Experience porting applications to non-standard or accelerator hardware

Etched Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Etched and has not been reviewed or approved by Etched.

  • Equity Value & Accessibility Equity growth is described as strong and significant equity is part of the package. High total compensation for technical roles reinforces the equity-led upside.
  • Healthcare Strength Medical, dental, and vision coverage include generous premium support, indicating robust core healthcare. This reduces employee cost exposure for essential coverage.
  • Wellbeing & Lifestyle Benefits Daily lunch and dinner, a housing subsidy for those living near the office, relocation support, and wellness perks are highlighted. These offerings lower day-to-day living costs and support practical wellbeing.

Etched Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Pakenham
53 Employees
Year Founded: 2022

What We Do

By burning the transformer architecture into our chips, we’re creating the world’s most powerful servers for transformer inference.

Similar Jobs

xAI Logo xAI

Software Engineer

Information Technology
In-Office
Palo Alto, CA, USA
96 Employees
180K-440K Annually
In-Office
Mountain View, CA, USA
2359 Employees
170K-216K Annually

Anthropic Logo Anthropic

Senior Software Engineer

Artificial Intelligence • Natural Language Processing • Generative AI
In-Office
3 Locations
2500 Employees
320K-485K Annually

Cerebras Systems Logo Cerebras Systems

Software Engineer

Artificial Intelligence
In-Office
Sunnyvale, CA, USA
402 Employees

Similar Companies Hiring

Hanover Park Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
42 Employees
Kepler  Thumbnail
Fintech • Software
New York, New York
6 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
60 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account