Applied AI Engineer — Conversational Agents (Senior/Staff)

Posted 15 Hours Ago
Be an Early Applicant
New York City, NY, USA
In-Office
225K-325K Annually
Senior level
Artificial Intelligence • Marketing Tech • Sales • Software
The Role
Improve production conversational AI agents through data analysis, experimentation, evaluation, and software engineering. Responsibilities include diagnosing failures in customer conversations, building trustworthy metrics and datasets, improving voice interactions, testing models and prompts, optimizing latency and reliability, and shipping measurable production improvements. The role requires hands-on ownership across the full agent improvement cycle, with Staff-level engineers also shaping evaluation strategy and mentoring team practices.
Summary Generated by Built In
Senior/Staff Applied AI Engineer — Conversational Agents

Revin · Agents Platform · Flatiron, New York City

$225,000–$325,000 base salary + meaningful early equity · Full-time

Four days per week in our Flatiron office, with one remote day and flexibility when life comes up.

We’re hiring at Senior and Staff levels. Final compensation depends on level, based on your demonstrated experience, scope of ownership, and impact.

About Revin

Revin builds AI-powered software for the businesses that keep our homes running: roofing, HVAC, plumbing, remodeling, and more. We help these businesses respond to customers, book appointments, and manage the customer experience across voice, SMS, and email.

Revin handles around half a million customer conversations a month. Businesses with hundreds of millions in annual revenue rely on our products for work that directly affects their customers and their bottom line.

Our customers include Vertex Service Partners, a national roofing and exterior-services platform with over $600M in revenue, and Del-Air, a $300M+ HVAC and plumbing business.

See the impact of our work: we’ve helped Del-Air book 11,000+ membership appointments, while ImproveIt uses Revin to set 34% of its booked appointments.

The role

Make our conversational AI agents more capable, responsive, and natural through rigorous experimentation. You’ll join our Agents Platform team at the intersection of software engineering and data science, using real customer conversations to understand where agents fall short and ship improvements that hold up in production.

You’ll own the full improvement cycle: identify a problem, form a hypothesis, design an experiment, implement a change, and validate its impact. Your work will span instruction following, tool use, task completion, latency, and conversational experience, with a substantial focus on voice.

You’ll improve agents through model selection, prompting, context, orchestration, and real-time voice behavior. We care most about your ability to learn from evidence and turn that learning into better customer experiences.

What you’ll do
  • Find the problems worth solving. Analyze production conversations, listen to calls, inspect traces, and work with customer-facing teams to identify failure patterns and prioritize improvements.

  • Build measurement you can trust. Define quality dimensions and representative evaluation datasets. Combine human judgment, automated graders, and production metrics, and validate that they measure what customers actually experience.

  • Develop and test hypotheses. Design reproducible experiments, compare approaches, and use offline evaluations and controlled production rollouts to understand what works, where it works, and what tradeoffs it introduces.

  • Improve the voice experience. Investigate turn-taking, interruptions, response latency, transcription errors, speech quality, and spoken naturalness. Use audio and system telemetry together to diagnose causes and validate changes.

  • Ship improvements into the agent system. Implement changes, monitor their effects in production, and take responsibility for reliability, cost, and regressions alongside quality gains.

  • Make improvement repeatable. Turn production failures and experimental findings into reusable datasets, evaluations, debugging tools, and release checks that help the team move faster.

At Staff level, you’ll also shape our evaluation and experimentation strategy across agent capabilities, identify the highest-impact opportunities, and raise the broader team’s ability to diagnose and improve agent behavior while remaining hands-on.

What you’ll bring
  • A demonstrated track record improving substantial AI agent systems deployed to external customers. You can explain the system’s scale and complexity, the problems you identified, your hypotheses and experiments, what you personally implemented, and how you established that it improved production outcomes.

  • Strong experimental and analytical judgment. You can define useful baselines, design informative comparisons, reason about noisy or confounded results, and learn from experiments that don’t support your hypothesis.

  • Strong software engineering ability. You can analyze conversational data with code and SQL, debug complex agent behavior, and turn an experimental improvement into maintainable production software.

  • Practical understanding of agent systems. You can reason about how models, instructions, context, tools, and orchestration interact, and weigh quality against latency, reliability, and cost. Specific frameworks matter less than your judgment and results.

  • Ownership and clear communication. You make progress through ambiguity, communicate findings and tradeoffs, and work closely with engineers and product to carry improvements through to customers.

  • A desire to work in an early-stage startup. You’re energized by broad responsibility, rapid iteration, and high standards. We use AI programming tools heavily and expect you to use them effectively while owning the quality of your work.

Voice experience

Hands-on experience improving production voice agents is strongly preferred. We value experience diagnosing conversational timing, speech recognition and generation, audio quality, and end-to-end latency, across any provider or stack.

We’re also open to engineers with an exceptional track record improving other customer-facing agents who can demonstrate strong judgment on voice problems and are excited to develop deeper expertise.

Nice to have
  • Experience building simulation, replay, or evaluation tools for multi-turn agents.

  • Experience developing human review workflows or calibrating automated evaluations against human judgment.

  • Experience at a strong startup or on a small team with substantial ownership.

Why join Revin

You’ll work with a large, varied set of real customer interactions and a short path from an idea to measurable impact. Improvements you ship can reach hundreds of thousands of conversations, and what you learn will help shape the direction of our agent platform.

We’re still early. You’ll join a small team of sharp, ambitious people, have substantial influence over how we build, and receive meaningful equity ownership with the opportunity to share substantially in the upside you help create.

Compensation and benefits
  • $225,000–$325,000 base salary, with final compensation dependent on level.

  • Meaningful early equity in the company you’re helping build.

  • Excellent health coverage. $0 employee premiums for individual medical coverage on most plans, plus dental and vision insurance.

  • 401(k) matching up to 4% of salary. Contribute 5% to receive the full match.

  • Unlimited PTO.

  • Commuter benefits.

  • Free lunches and dinners in the office.

  • $100/month gym membership stipend.

  • Tech equipment of your choice.

Skills Required

  • Demonstrated experience improving substantial AI agent systems deployed to external customers, including explaining scale, experiments, implementation, and production outcomes.
  • Strong experimental and analytical judgment, including defining baselines, designing comparisons, interpreting noisy results, and learning from unsuccessful experiments.
  • Strong software engineering ability, including analyzing conversational data with code and SQL, debugging complex agent behavior, and building maintainable production software.
  • Practical understanding of agent systems, including models, instructions, context, tools, orchestration, quality, latency, reliability, and cost tradeoffs.
  • Strong ownership and communication skills, with the ability to work through ambiguity and collaborate with engineering and product teams.
  • Interest in working at an early-stage startup with broad responsibility and rapid iteration.
  • Hands-on experience improving production voice agents, including conversational timing, speech recognition, speech generation, audio quality, or end-to-end latency.
  • Experience building simulation, replay, or evaluation tools for multi-turn agents.
  • Experience developing human review workflows or calibrating automated evaluations against human judgment.
  • Experience at a strong startup or on a small team with substantial ownership.
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
28 Employees
Year Founded: 2024

What We Do

Revspire is an AI-powered sales enablement platform that transforms scattered collateral into intelligent Deal Rooms, guiding buyers to the close automatically.

Similar Jobs

Legora Logo Legora

Director of Solutions Engineering

Artificial Intelligence • Legal Tech • Software
In-Office
New York City, NY, USA
1100 Employees
223K-302K Annually
Easy Apply
Remote or Hybrid
United States
2200 Employees
21-21 Annually

Magnite Logo Magnite

Product Marketing Manager

AdTech • Big Data • Digital Media • Software
Hybrid
2 Locations
950 Employees
105K-125K Annually

Hewlett Packard Enterprise Logo Hewlett Packard Enterprise

Senior Systems Engineer

Artificial Intelligence • Cloud • Information Technology • Consulting
In-Office or Remote
14 Locations
85422 Employees
152K-355K Annually

Similar Companies Hiring

Revel Thumbnail
Aerospace • Hardware • Robotics • Software
Marina Del Rey, California
70 Employees
Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
33 Employees
Vega Thumbnail
Artificial Intelligence • Automotive • Insurance • Transportation
US
65 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account