Software Engineer, Training & Inference Infrastructure

Reposted 3 Days Ago
Be an Early Applicant
Redwood City, CA
In-Office
180K-250K Annually
Senior level
Artificial Intelligence • Software
The Role
Design and build the Machine Learning infrastructure, ensuring reliable GPU workloads and data curation product functionality while collaborating with research and engineering teams.
Summary Generated by Built In
About the Company

Models are what they eat. But a large portion of training compute is wasted training on data that are already learned, irrelevant, or even harmful, leading to worse models that cost more to train and deploy.

At DatologyAI, we’ve built a state of the art data curation suite to automatically curate and optimize petabytes of data to create the best possible training data for your models. Training on curated data can dramatically reduce training time and cost (7-40x faster training depending on the use case), dramatically increase model performance as if you had trained on >10x more raw data without increasing the cost of training, and allow smaller models with fewer than half the parameters to outperform larger models despite using far less compute at inference time, substantially reducing the cost of deployment. For more details, check out our recent blog posts sharing our high-level results for text models and image-text models.

We raised a total of $57.5M in two rounds, a Seed and Series A. Our investors include Felicis Ventures, Radical Ventures, Amplify Partners, Microsoft, Amazon, and AI visionaries like Geoff Hinton, Yann LeCun, Jeff Dean, and many others who deeply understand the importance and difficulty of identifying and optimizing the best possible training data for models. Our team has pioneered this frontier research area and has the deep expertise on both data research and data engineering necessary to solve this incredibly challenging problem and make data curation easy for anyone who wants to train their own model on their own data.

This role is based in Redwood City, CA. We are in office 4 days a week.

About the Role

We’re looking for an engineer with deep experience building and operating large-scale training and inference systems. You will design, implement, and maintain the infrastructure that powers both our internal ML research workflows and the high-performance inference pipelines that deliver curated data to our customers.

As one of our early hires, you will influence technical direction, partner directly with researchers and product engineers, and take ownership of systems that are central to our company’s success.

What You'll Work On
  • Architect and maintain training infrastructure that are reliable, scalable, and cost-efficient.

  • Build robust model serving infrastructure for low-latency, high-throughput inference across heterogeneous hardware.

  • Automate resource orchestration and fault recovery across GPUs, networking, OS, drivers, and cloud environments.

  • Partner with researchers to productionize new models and features quickly and safely.

  • Optimize training and inference pipelines for performance, reliability, and cost.

  • Ensure all infrastructure meets the highest bar for reliability, security, and observability.

About You
  • Have at least 5 years of professional software engineering experience.

  • Expertise in Python and experience with deep learning frameworks (PyTorch preferred)

  • Have an understanding of modern ML architectures and an intuition for how to optimize their performance, particularly for training and/or inference

  • Have familiarity with inference tooling like vLLM, SGLang, or custom model parallel systems.

  • Proven experience designing and running large-scale training or inference systems in production.

  • Have or can quickly gain familiarity with PyTorch, NVidia GPUs and the software stacks that optimize them (e.g. NCCL, CUDA), as well as HPC technologies such as InfiniBand, NVLink, AWS EFA etc.

  • Commitment to engineering excellence: strong design, testing, and operational discipline.

  • Collaborative, humble, and motivated to help the team succeed.

  • Ownership mindset: you’re comfortable learning fast and tackling problems end-to-end.

Don’t meet every single requirement? We still encourage you to apply. If you’re excited about our mission and eager to learn, we want to hear from you!

Compensation

At DatologyAI, we are dedicated to rewarding talent with highly competitive salary and significant equity. The base salary for this position ranges from $180,000 to $250,000.

  • The candidate's starting pay will be determined based on job-related skills, experience, qualifications, and interview performance.

We offer a comprehensive benefits package to support our employees' well-being and professional growth:

  • 100% covered health benefits (medical, vision, and dental).

  • 401(k) plan with a generous 4% company match.

  • Unlimited PTO policy

  • Annual $2,000 wellness stipend.

  • Annual $1,000 learning and development stipend.

  • Daily lunches and snacks are provided in our office!

  • Relocation assistance for employees moving to the Bay Area.

Top Skills

Kubernetes
Linux
Pulumi
Python
Terraform
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Redwood City, California
24 Employees
Year Founded: 2023

What We Do

DatologyAI builds tools to automatically select the best data on which to train deep learning models. Our tools leverage cutting-edge research—much of which we perform ourselves—to identify redundant, noisy, or otherwise harmful data points. The algorithms that power our tools are modality-agnostic—they’re not limited to text or images—and don’t require labels, making them ideal for realizing the next generation of large deep learning models. Our products allow customers in nearly any vertical to train better models for cheaper.

Similar Jobs

Boeing Logo Boeing

Software Engineer

Aerospace • Information Technology • Cybersecurity • Defense • Manufacturing
In-Office
Seal Beach, CA, USA
141000 Employees
5-5 Annually

Collectors Logo Collectors

Staff Accountant

Consumer Web • eCommerce • Machine Learning • Professional Services • Software • Sports • Analytics
In-Office
Santa Ana, CA, USA
2246 Employees
30-40 Hourly

Collectors Logo Collectors

Continuous Improvement Lead

Consumer Web • eCommerce • Machine Learning • Professional Services • Software • Sports • Analytics
In-Office
Santa Ana, CA, USA
2246 Employees
28-30 Hourly

Collectors Logo Collectors

Network Engineer

Consumer Web • eCommerce • Machine Learning • Professional Services • Software • Sports • Analytics
In-Office
Santa Ana, CA, USA
2246 Employees
101K-133K Annually

Similar Companies Hiring

PRIMA Thumbnail
Travel • Software • Marketing Tech • Hospitality • eCommerce
US
15 Employees
Scotch Thumbnail
Software • Retail • Payments • Fintech • eCommerce • Artificial Intelligence • Analytics
US
25 Employees
Idler Thumbnail
Artificial Intelligence
San Francisco, California
6 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account