Software Engineer, Voice Evals [NYC or SF]

Posted Yesterday
2 Locations
In-Office
220K-300K Annually
Entry level
Artificial Intelligence • Enterprise Web • Healthtech • Software
The healthcare data and AI company
The Role
About Concurrence

Concurrence (formerly Amigo) is building the platform healthcare organizations use to build their own clinical AI agents. Our mission is to make excellent care accessible to every patient, not rationed by the limits of clinician time.

Underneath our platform is the Health World Model, which unifies a health system’s data into a living, event-level record of every patient’s care. On top of it, agents handle work across the patient journey, from intake and scheduling to medication management and clinical decision support.

Our agents have supported more than 10 million patient encounters and are on track to grow tenfold this year. We run natively on Databricks, integrate with every major EHR, and are backed by Madrona, General Catalyst, and Optum Ventures. We’re about 35 people, working in person in New York and San Francisco.

 
About this role

You'll own the systems that help our voice agents improve with every release. You'll connect production failures, realistic simulations and reliable scoring into a feedback loop that shows us what to improve and whether a change actually worked.

This is a hands-on engineering role with room to shape our evaluation approach. You'll work with research, runtime, product and clinical colleagues to define what good looks like, choose the failures to tackle next and verify improvements on real calls. You'll also own the release gate, with authority to block changes that fail our criteria for patient readiness.

 
What you'll do
  • Own our voice-agent improvement loop, from discovering production failures to reproducing them, evaluating fixes and measuring behavior after release

  • Design and build simulated callers and call-replay systems that exercise the production voice path with real audio

  • Test difficult conversations, including phone trees, hold queues, interruptions, conflicting clinical information and appropriate escalation

  • Build scoring systems, calibrate automated judges against human assessments and establish whether simulation results predict real-call behavior

  • Use failure analysis to help research and engineering teams prioritize improvements to models, prompts and runtime behavior

  • Define release criteria with clinical and engineering colleagues, own the gate and block changes that fail it

  • Maintain held-out evaluation sets and turn production failures into lasting regression coverage

 
What we're looking for
  • You've personally built and owned evaluation or simulation for production agents at scale. Voice and audio experience is a strong preference

  • You've built agent simulations or replay systems, implemented scoring and caught a concrete failure before release

  • Your evaluation results determined what shipped, and you led that work

  • You've taken an ambiguous agent-quality problem, decided what to measure and built systems that guided improvements and release decisions

  • You can explain what a score captures, what it misses and how you checked it against real behavior

  • You write reliable software that runs at volume and follow failures through to a verified fix

  • You use coding agents in your work and independently verify their output

  • You're low ego, direct, and can hold the line on release criteria with evidence

  • You can work on site in New York City or San Francisco

 
Nice to have
  • Experience calibrating automated judges against human labels

  • Load testing and concurrent calls using audio

  • Speech or audio machine learning

  • Experience in healthcare or another regulated environment

 
Benefits (available to Full-Time Employees)
Health & Wellness
  • Comprehensive health, dental, and vision insurance

  • Daily catered lunch and dinner

  • Mental health support and wellness coaching

  • Flexible wellness stipend for fitness, therapy, or personal growth

Growth & Development
  • Annual learning budget for courses, books, or conferences

  • Conference attendance budget for professional development

  • Annual team offsite

  • Academic collaboration opportunities

  • Unlimited PTO

Our Core Values
  1. Patients Win, We Win

    If patients aren't getting better care, we haven't earned the right to scale. Every internal decision gets pressure-tested: does this make patients' lives better? If we can't draw the line, we question why we're doing it.

  2. High Standards, High Care

    We hold a high bar for the team because patients are counting on us to get this right. But high standards only work with genuine investment in each other. You can take risks, admit mistakes, and challenge ideas. Not despite our standards, but because of them.

  3. Thoughtful Urgency

    We move fast by default, but speed without judgment is recklessness. The discipline is knowing which decisions are reversible vs. not. In healthcare AI, the companies that win will be fast everywhere they can be and careful everywhere they must be. We build the muscle to do both.

  4. Intensely Measured

    We instrument patient outcomes, provider ROI, system performance, and clinical accuracy. But data without action is surveillance. Every metric should have an owner, a threshold, and a response plan. If we're measuring something but never acting on it, we stop measuring it.

Who Builds With Us
  • Low ego: Politics and territory don't interest you. The best ideas win, regardless of who has them.

  • Direct: You say the hard thing, challenge ideas openly, and commit fully once decided.

  • High agency: You thrive on trust rather than instruction. When you see something is broken, you fix it. You don’t file tickets and wait for someone else.

  • Bar of excellence: You hold yourself to a bar most people wouldn't, and you want teammates who do the same.

  • Skeptical: You push back on rules that don’t make sense and question assumptions that haven’t earned their place.

Concurrence Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Concurrence and has not been reviewed or approved by Concurrence.

  • Healthcare Strength — Health coverage is described as comprehensive, including medical, dental, and vision insurance alongside mental health resources, FSAs, wellness coaching, and a flexible wellness stipend.
  • Equity Value & Accessibility — Equity is prominently included in offers, with company equity and an employee stock purchase plan highlighted across role listings.
  • Leave & Time Off Breadth — Time-off policies include flexible or unlimited PTO, paid sick days, holidays, and additional wellness or birthday days.

Concurrence Insights

The Company
HQ: New York, New York
35 Employees
Year Founded: 2023

What We Do

Concurrence is the data and AI platform healthcare organizations use to build their own clinical AI agents. Our platform unifies healthcare data into a living, event-level record of each patient's care and enables organizations to build agents across the patient journey, from intake and scheduling to medication management and clinical decision support. We are backed by Madrona, General Catalyst, and Optum Ventures, with a team working in person in New York City and San Francisco.

Why Work With Us

We build software that runs inside health systems and touches patient care. Engineers own problems from initial brief through production, working directly with customers and clinicians. Our team works in person in New York City and San Francisco. We value low ego, direct communication, high agency, and a high bar for craft.

Gallery

Gallery

Similar Jobs

Lowe’s Logo Lowe’s

Warehouse Part Time Days

Consumer Web • eCommerce • Information Technology • Retail • Software • Analytics • App development
Hybrid
Herkimer, NY, USA
300000 Employees
Hybrid
New York, NY, USA
289097 Employees

Tapestry - Coach and Kate Spade Logo Tapestry - Coach and Kate Spade

Acting Supervisor I

eCommerce • Fashion • Retail • Sales • Wearables • Design
Hybrid
New York, NY, USA
16000 Employees
17-28 Hourly

Tapestry - Coach and Kate Spade Logo Tapestry - Coach and Kate Spade

Allocator, Coachtopia, Special Projects/Small Departments

eCommerce • Fashion • Retail • Sales • Wearables • Design
Hybrid
New York, NY, USA
16000 Employees
66K-72K Annually

Similar Companies Hiring

Revel Thumbnail
Aerospace • Hardware • Robotics • Software
Marina Del Rey, California
70 Employees
Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
33 Employees
Vega Thumbnail
Artificial Intelligence • Automotive • Insurance • Transportation
US
65 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account