Senior Software Engineer, Machine Learning Infrastructure

Posted Yesterday
Be an Early Applicant
San Francisco, CA, USA
In-Office
176K-220K Annually
Senior level
Edtech • Enterprise Web • HR Tech • Software
Handshake is the number one site for college students to find a job.
The Role
Build and operate shared ML/AI infrastructure: data pipelines, feature stores, training, model serving and LLM platform work. Scale inference and training (GPU, batching, autoscaling), enable evaluation and post-training workflows, and partner with AI, data science, and product teams to productionize models and improve platform reliability and developer experience.
Summary Generated by Built In

About Handshake

Handshake was founded on a simple belief that everyone deserves a path to a great career, regardless of where they went to school or who they know. Today, we power 25 million job seekers, 1 million+ employers, and 1,600 educational institutions.

In 2025, we started Handshake AI and built the fastest-growing AI data business in history. We work directly with frontier AI lab researchers to create evaluations, publish benchmarks, and push the boundary of data. We’ve grown from $0 to ~$1B run rate and pay ~$60M to over 30K individuals every month.

Why join Handshake now:

  • Shape how every career evolves in the AI economy, at global scale, with impact your friends, family and peers can see and feel

  • Partner hand-in-hand with world-class AI labs, Fortune 500 partners and the world’s top educational institutions

  • Work together with engineers, scientists, operators, and more from Palantir, Meta, Scale AI, and former YC founders

  • Build a massive, fast-growing business with billions in revenue

About Handshake AI

Human data is the core infrastructure to AI advancement. Frontier AI labs currently improve model capabilities with various data-intensive post-training techniques. We believe that data spend for AI training will increase by 3-5x in the next few years and continue for much longer as models take on new domains. Handshake AI supports all of the frontier AI labs, working on their most complex data at the largest scale.

About the Role

We’re looking for a Senior Software Engineer to join our ML Infrastructure & Platform team. This team powers both Handshake’s core career marketplace and Handshake AI by building the shared infrastructure behind our production ML and AI systems.

This is an infrastructure-heavy role for an engineer who enjoys building scalable platforms at the intersection of software engineering, machine learning, and generative AI. You’ll help teams move quickly from prototype to production while building the reliable, high-performance systems that power training, evaluation, and inference across Handshake.

What You’ll Do
  • Build and operate the shared infrastructure behind production ML and AI, including data pipelines, feature stores, training, and model serving.

  • Develop and scale our LLM platform, including provider integrations, orchestration, observability, and controls for cost, latency, and reliability.

  • Build evaluation infrastructure, including LLM eval harnesses, benchmarks, and quality measurement pipelines.

  • Support post-training workflows, including fine-tuning, reinforcement learning pipelines, and supporting data infrastructure.

  • Optimize inference infrastructure for open and fine-tuned models, including GPU serving, batching, and autoscaling.

  • Partner with AI, Data Science, and Product teams to productionize new models and establish best practices for ML infrastructure across Handshake.

  • Improve the reliability, scalability, and developer experience of our ML platform.

Desired Capabilities
  • 5+ years of production software engineering experience using Python, Go, TypeScript, or similar languages.

  • Experience building and operating cloud infrastructure on AWS, GCP, or similar platforms.

  • Strong experience with Kubernetes, Docker, Terraform, CI/CD, and operating production services.

  • Hands-on experience building ML infrastructure, including model serving, training pipelines, feature stores, embeddings, or ML observability.

  • Experience with modern data platforms such as BigQuery, Airflow, Spark, Beam/Dataflow, or streaming pipelines.

  • Practical experience building production systems with LLMs or generative AI, including orchestration, provider APIs, observability, and performance optimization.

  • Strong systems design skills, sound engineering judgment, and the ability to thrive in ambiguous, fast-moving environments.

Extra Credit
  • Experience with Ray, Anyscale, KubeRay, Ray Serve, vLLM, Triton, PyTorch, or GPU-backed inference and training.

  • Experience designing LLM evaluation frameworks, benchmarking systems, or quality regression testing.

  • Experience with Vertex AI, Bigtable, Redis, or feature platform infrastructure.

  • Experience with post-training techniques such as fine-tuning, RLHF, reinforcement learning, or reward modeling.

  • Experience building agentic systems, MCP integrations, tool use, memory systems, or voice AI applications.

Perks

Handshake delivers benefits that help you feel supported—and thrive at work and in life.

The below benefits are for full-time US employees.

🎯 Ownership: Equity in a fast-growing company

💰 Financial Wellness: 401(k) match, competitive compensation, financial coaching

🍼 Family Support: Paid parental leave, fertility benefits, parental coaching

💝 Wellbeing: Medical, dental, and vision, mental health support, $500 wellness stipend

📚 Growth: $2,000 learning stipend, ongoing development

💻 Remote & Office: Internet, commuting, and free lunch/gym in our SF office

🏝 Time Off: Flexible PTO, 15 holidays + 2 flex days

🤝 Connection: Team outings & referral bonuses

Explore our mission, values, and comprehensive US benefits at joinhandshake.com/careers.

Skills Required

  • 5+ years production software engineering experience using Python, Go, or TypeScript
  • Experience building and operating cloud infrastructure on AWS or GCP
  • Strong experience with Kubernetes, Docker, Terraform, and CI/CD operating production services
  • Hands-on experience building ML infrastructure: model serving, training pipelines, feature stores, embeddings, or ML observability
  • Experience with modern data platforms: BigQuery, Airflow, Spark, Apache Beam/Dataflow, or streaming pipelines
  • Practical experience building production systems with LLMs or generative AI, including orchestration, provider APIs, and observability
  • Experience optimizing inference infrastructure for GPU serving, batching, and autoscaling
  • Strong systems design skills and ability to thrive in ambiguous, fast-moving environments
  • Experience with Ray, Anyscale, KubeRay, Ray Serve, vLLM, Triton, or PyTorch
  • Experience designing LLM evaluation frameworks, benchmarking, or quality regression testing
  • Experience with Vertex AI, Bigtable, Redis, or feature platform infrastructure
  • Experience with post-training techniques such as fine-tuning, RLHF, or reward modeling

Handshake Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Handshake and has not been reviewed or approved by Handshake.

  • Leave & Time Off Breadth Time-off practices include flexible/unlimited PTO, companywide recharge weeks in summer and winter, plus additional volunteer and personal holiday time. Feedback suggests sabbaticals and coordinated breaks help people actually use rest time.
  • Parental & Family Support Parental leave is described as extended for primary and secondary caregivers, and family-oriented policies are highlighted. Fertility and family-support resources are referenced in public materials.
  • Healthcare Strength Core coverage spans medical, dental, and vision, with added mental-health resources and wellness programming. Feedback suggests these supports contribute meaningfully to overall wellbeing.

Handshake Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: San Francisco, CA
700 Employees
Year Founded: 2014

What We Do

Handshake is the #1 place to launch a career with no connections, experience, or luck required. The platform connects up-and-coming talent with 650,000+ employers - from Fortune 500 companies like Google, Nike, and Target to thousands of public school districts, healthcare systems, and nonprofits. Earlier this year, we announced our $200M Series F funding round. This Series F fundraise and new valuation of $3.5B will fuel Handshake’s next phase of growth and propel our mission to help more people start, restart, and jumpstart their careers.

Why Work With Us

How someone builds their career is foundational. We believe in working with the higher education community to help students build meaningful careers. We are at the nexus of universities, students and employers— and we’re able to connect the very best pieces of each side. We’re proud to create a community where students can be more successful.

Gallery

Gallery

Similar Jobs

In-Office
Sunnyvale, CA, USA
3411 Employees
153K-222K Annually

Nuro Logo Nuro

Senior Software Engineer

Artificial Intelligence • Automotive • Information Technology • Robotics
In-Office
Mountain View, CA, USA
908 Employees
194K-291K Annually

Nuro Logo Nuro

Staff Software Engineer

Artificial Intelligence • Automotive • Information Technology • Robotics
In-Office
Mountain View, CA, USA
908 Employees
194K-352K Annually
In-Office
2 Locations
2359 Employees
213K-263K Annually

Similar Companies Hiring

Hanover Park Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
42 Employees
Kepler  Thumbnail
Fintech • Software
New York, New York
6 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
60 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account