Remote, U.S.
What This Role IsWe are looking for a hands-on AI Operations Specialist to own customer AI implementations from discovery through launch and ongoing optimization.
You will configure and improve LLM-powered workflows, evaluate production behavior, troubleshoot issues, and work directly with customers and internal teams to make sure implementations work reliably in the real world.
This is a customer-facing technical IC role. You should be comfortable moving between AI behavior, implementation details, customer workflows, and production troubleshooting.
What You’ll DoOwn customer AI implementations from discovery through launch and ongoing optimization.
Translate customer workflows and requirements into clear AI behavior, implementation plans, and testing criteria.
Design and improve prompts and workflow logic for customer-facing AI agents.
Build and run evaluations, test plans, and regression checks to improve reliability and consistency.
Investigate production issues across prompts, workflows, integrations, data, APIs, and application behavior.
Determine whether problems can be resolved directly or require product or engineering support, and escalate with clear evidence and reproduction steps.
Facilitate customer testing and troubleshooting, and communicate status, risks, blockers, and required decisions clearly.
Drive open issues to resolution across customers and internal teams, including when ownership or next steps are ambiguous.
Use recurring customer and production issues to improve implementation practices and product behavior.
Hands-on experience working with LLM-powered products or workflows in production.
Experience building evaluations and improving unreliable or nondeterministic AI behavior.
Strong technical troubleshooting skills and the ability to investigate complex systems without immediately handing problems to engineering.
Experience owning customer implementations from discovery through launch and optimization, including working directly with customers throughout the process.
Experience with healthcare technology and/or conversational or voice AI is helpful but not required.
Health, dental, and vision insurance
401(k) with company match
Flexible PTO and company-paid holidays
Fully remote work within US hours
Company-provided laptop
Skills Required
- Hands-on experience working with LLM-powered products or workflows in production
- Experience building evaluations and improving unreliable or nondeterministic AI behavior
- Strong technical troubleshooting skills for investigating complex systems
- Experience owning customer implementations from discovery through launch and optimization
- Experience working directly with customers throughout implementation
- Experience with healthcare technology and/or conversational or voice AI
What We Do
Our mission is to improve the healthcare experience for patients and providers. We do this by reducing provider burden, improving the patient experience, and improving practice efficiency. SCAM ALERT: Health Note has been informed that scammers are using our name to scam individuals who believe they are applying for a career opportunity with us. These scammers can be very convincing, going so far as to steal the profile pictures from our actual employees from their LinkedIn profiles and using email addresses that contain the words “Health Note” such as “@healthnotecareers” or “@healthnotecareerdesk”. These scammers will typically ask you to send money at some point (for equipment, training, uniform) or for your personal information such as social security number or date of birth. A LEGITIMATE HEALTH NOTE RECRUITER OR HIRING MANAGER WILL NEVER ASK YOU TO SEND US MONEY. Also, you will never receive a job offer from us if we have not verbally interviewed you. For more information on recruiting scams or to report a recruiting scam, visit the website of the Federal Trade Commission at https://www.consumer.ftc.gov/articles/0243-job-scams
