Forward Deploy AI Engineer

Reposted 10 Days Ago
San Francisco, CA, USA
In-Office
Entry level
Artificial Intelligence • Software
The Role
Embed Judgment Labs' agent behavior monitoring into customer production systems: integrate monitoring and evaluation into agent workflows, diagnose failures in live environments, guide customers on monitoring and evaluation strategy, and own multiple customer engagements end-to-end to ensure sustained adoption.
Summary Generated by Built In

We're building the infrastructure for continual learning. Agents are

We’ve raised $32M across two rounds led by Lightspeed.

The Role:

You'll own the end to end customer deployment process from understanding their pain points to building solutions end to end.

This role centers on deep technical execution and customer ownership. You will work directly with customer teams to reason about agent behavior, translate high-level goals into concrete ABM deployments, and own outcomes end-to-end across real production environments. The scope, judgment, and autonomy required in this role mirrors a training ground for what it takes to found or lead a technical company.

What You'll Do:
  • Deploy and embed Judgment Labs’ ABM platform and AI components directly into customer codebases and production AI systems

  • Work inside customer systems to integrate monitoring, evaluation, and agent-facing components into real workflows

  • Guide customers through technical decisions around agent monitoring, evaluation strategy, and integrating these capabilities into existing production systems.

  • Own multiple customer engagements end-to-end, ensuring successful integration and sustained adoption of monitoring and evaluation systems within production agent workflows.

What We're Looking For:

You identify with at least one of the following:

  • Experience deploying AI or LLM-based systems into real production environments

  • Ability to quickly learn new tools and systems, and integrate AI infrastructure into existing customer workflows and codebases

  • Ability to translate ambiguous customer goals into concrete technical solutions and evaluation strategies

  • Strong customer-facing skills, including explaining complex technical concepts clearly and building trust with both technical and non-technical stakeholders

  • Comfort owning deployments end-to-end, from initial integration through successful production adoption

  • You want to be a technical founder in the future.

Why Judgment?
  • Agents can’t work without this. Today’s agents hallucinate, drift, and break in production. We’re building the infrastructure that fixes this: the monitoring layer that makes agents self-improving.

  • We’re wired to win. We're a team of 20 but we ship like 50+ on the daily. You'll be working with olympiad medalists, debate champions, and competitive athletes who bring that same intensity to company building.

  • Fast track to founding. Our engineers interface directly with customers, ship code into their environments, and use their feedback to dictate what’s next on the roadmap. Everyone on the team is either an ex-founder or a founder-to-be.

  • We make sure our people do their best work. If you deserve a spot on the team, money will never get in the way of it. Full benefits, Equinox, and a private chef to take care of you. We sprint hard but we play hard, ask us about our Smash/Mario Kart tournaments.

    We work in person in San Francisco.

Skills Required

  • Experience deploying AI or LLM-based systems into real production environments
  • Ability to quickly learn new tools and integrate AI infrastructure into existing customer workflows and codebases
  • Ability to translate ambiguous customer goals into concrete technical solutions and evaluation strategies
  • Strong customer-facing skills, including explaining complex technical concepts clearly and building trust with technical and non-technical stakeholders
  • Comfort owning deployments end-to-end, from initial integration through successful production adoption
  • Desire to be a technical founder in the future
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: San Francisco, California
20 Employees
Year Founded: 2025

What We Do

Judgment Labs builds agent behavior monitoring (ABM) infrastructure. Judgment provides a toolkit to track and judge agent behavior in online and offline setups, enabling you to convert high-signal interaction data from production/test environments into more reliable agents.

Similar Jobs

Puntt.ai Logo Puntt.ai

Forward Deploy Engineer (Enterprise AI)

Artificial Intelligence • Marketing Tech • Automation
Hybrid
San Francisco, CA, USA
11 Employees
130K-180K Annually

ServiceNow Logo ServiceNow

Senior Pricing Manager

Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
Remote or Hybrid
Santa Clara, CA, USA
29000 Employees
155K-272K Annually

ServiceNow Logo ServiceNow

Vice President, Corporate and Product Public Relations (Americas)

Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
Remote or Hybrid
Santa Clara, CA, USA
29000 Employees
280K-475K Annually

ServiceNow Logo ServiceNow

Senior Software Engineer

Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
Remote or Hybrid
Santa Clara, CA, USA
29000 Employees
143K-243K Annually

Similar Companies Hiring

Hanover Park Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
42 Employees
Kepler  Thumbnail
Fintech • Software
New York, New York
6 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
60 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account