Technical Customer Success Manager (AI Tuning)

Reposted 8 Days Ago
Be an Early Applicant
Toronto, ON, CAN
Hybrid
Mid level
Artificial Intelligence • Software • Generative AI
The Role
The role involves ensuring the performance of AI agents in production, defining success metrics with clients, and debugging issues.
Summary Generated by Built In

HeyMilo is building AI interviewers that run real conversations with candidates. We help companies screen faster and make better hiring decisions using conversational AI.

We work closely with customers to deploy AI into real hiring workflows and ensure it delivers measurable outcomes.

The Role

We’re hiring a Forward Deployed AI Engineer to own the performance and success of AI agents in production.

This role sits at the intersection of engineering, product, and customer success. You’ll work directly with customers post-deployment to ensure their AI agents are performing at a high level and continuously improving.

What you’ll do

  • Work closely with customers and customer success to define and track success metrics

  • Monitor performance of deployed AI agents and identify gaps

  • Tune prompts, workflows, and agent behavior to improve outcomes

  • Iterate on agent logic, tool usage, and context management

  • Debug issues across prompts, integrations, and system behavior

  • Translate real-world usage into product and system improvements

What we’re looking for

  • Strong technical background with ability to code when needed

  • Deep experience with prompt engineering and LLM behavior

  • Strong understanding of agent systems (tool calling, context management, orchestration)

  • Heavy user of AI tools (Cursor, Claude, Codex, etc.)

  • Ability to debug ambiguous issues across AI and system layers

  • Strong ownership and communication skills

  • Comfortable working under pressure and tight timelines

  • Able to own outcomes, not just tasks

  • Experience implementing agentic workflows or AI agents in enterprise environments

  • Programming fundamentals + Python/JS experience are a huge plus

Bonus

  • Experience with conversational AI across voice, SMS, or chatbots

  • Experience with telephony or voice systems

  • Experience evaluating and improving production AI systems

  • Familiarity with recruiting, HR tech, or workflow automation

Why join

  • Direct ownership over how AI performs in the real world

  • Work on high-impact problems tied to customer outcomes

  • High visibility across product, engineering, and customer teams

  • Fast-paced, execution-driven environment

Skills Required

  • Strong technical background with ability to code when needed
  • Deep experience with prompt engineering and LLM behavior
  • Strong understanding of agent systems
  • Heavy user of AI tools
  • Ability to debug ambiguous issues across AI and system layers
  • Strong ownership and communication skills
  • Experience implementing agentic workflows or AI agents in enterprise environments
  • Programming fundamentals + Python/JS experience
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: New York, New York
24 Employees
Year Founded: 2023

What We Do

HeyMilo AI takes the guesswork out of the screening process and allows companies of all sizes to hire the right people for the job. HeyMilo AI is a generative AI-powered voice agent that allows companies to automate the screening process and engage candidates with an always-available voice agent. The agent can be tuned for your role and interview needs and is designed to evaluate candidates in a bias-free manner.

Similar Jobs

Motorola Solutions Logo Motorola Solutions

Data Analyst

Artificial Intelligence • Hardware • Information Technology • Security • Software • Cybersecurity • Big Data Analytics
Remote or Hybrid
Ontario, ON, CAN
23000 Employees

Capital One Logo Capital One

Product Manager

Fintech • Machine Learning • Payments • Software • Financial Services
Hybrid
Toronto, ON, CAN
55000 Employees
184K-500K Annually

Take-Two Interactive Software Logo Take-Two Interactive Software

Sr. Workday Integration & Extend Developer

Gaming • Information Technology • Mobile • Software
Hybrid
Toronto, ON, CAN
6500 Employees
96K-155K Annually

Take-Two Interactive Software Logo Take-Two Interactive Software

Data Engineer

Gaming • Information Technology • Mobile • Software
Hybrid
Toronto, ON, CAN
6500 Employees
71K-115K Annually

Similar Companies Hiring

Bellagent Thumbnail
Artificial Intelligence • Machine Learning • Business Intelligence • Generative AI
Chicago, IL
20 Employees
Kepler  Thumbnail
Fintech • Software
New York, New York
6 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York City, NY
100 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account