Hippocratic AI is building safety-focused large language model (LLM) for the healthcare industry. Our team comprised of ex-researchers from Microsoft, Meta, Nvidia, Apple, Stanford, John Hopkins and HuggingFace are reinventing the next generation of foundation model training and alignment to create AI-powered conversational agents for real time patient-AI interactions.
As Senior Operations Manager, Evaluations, you will be the engine behind our clinical-safety evaluation platform—the cornerstone that proves our AI agents are safe, effective, and ready for real-world care. Thousands of licensed clinicians have used our app to score conversations, flag safety issues, and suggest improvements. Internal researchers rely on the resulting data to iterate fast.
Your mission: keep the app humming, the clinicians happy, and the research teams unblocked—all while driving down turnaround time and cost per evaluation. You will partner with Product, Engineering, Finance, and Clinical Safety to improve work flows and build processes to make evaluation of our AI agents as effective and seamless as possible.
Responsibilities
Run the shop: Own day-to-day operations for all clinical, safety, and user-experience evaluations—scheduling, SLAs, escalation paths, and reporting.
Build & scale teams: Recruit, and onboard a global bench of licensed nurses and doctors; design performance-management loops that keep quality high
Automate everything: Drive adoption of auto-evaluation tooling; partner with ML engineers to expand test coverage, add new rubrics, and tune pass-/fail thresholds.
Process improvement: Map current workflows, identify bottlenecks, and ship iterative fixes—reducing turnaround time, cost per eval, and defect rate quarter over quarter.
Data & dashboards: Define KPIs (e.g., evaluation cycle time, false-negative/positive rates, reviewer utilization); build real-time dashboards and monthly business reviews.
Compliance & audit readiness: Ensure every evaluation artifact (prompts, ratings, nurse notes) is stored, traceable, and audit-ready for regulators and enterprise customers.
Required Skills & Qualifications
5 + years in operations, program management, or business process ownership—ideally in health-tech, clinical trials, digital health, or AI/ML.
Demonstrated success scaling high-volume, high-accuracy processes (millions of transactions or thousands of clinical cases per month).
Metrics obsession—comfortable defining KPIs, running A/B or DOE experiments, and presenting insights to executives.
Outstanding written & verbal communication; able to translate clinical or technical nuance into clear SOPs and exec summaries.
Preferred
Experience running eval or annotation pipelines for LLMs or ML products.
Prior work in a regulated environment (healthcare, med-device, pharma, finance).
Lean Six Sigma, PMP, or similar process-improvement certification.
Clinical background (RN/PA/MD) or formal training in quality management.
Fluency with process-automation tools and data dashboards (Airtable/Asana/Jira, SQL, Looker/Mode, Python or similar).
Why Join Our Team:
Innovative Mission: We are developing a safe, healthcare-focused large language model (LLM) designed to revolutionize health outcomes on a global scale.
Visionary Leadership: Hippocratic AI was co-founded by CEO Munjal Shah, alongside a group of physicians, hospital administrators, healthcare professionals, and artificial intelligence researchers from leading institutions, including El Camino Health, Johns Hopkins, Stanford, Microsoft, Google, and NVIDIA.
Strategic Investors: We have raised a total of $278 million in funding, backed by top investors such as Andreessen Horowitz, General Catalyst, Kleiner Perkins, NVIDIA’s NVentures, Premji Invest, SV Angel, and six health systems.
World-Class Team: Our team is composed of leading experts in healthcare and artificial intelligence, ensuring our technology is safe, effective, and capable of delivering meaningful improvements to healthcare delivery and outcomes.
We value in-person teamwork and believe the best ideas happen together. Our team is expected to be in the office five days a week in Palo Alto, CA.
Why You’ll Love Working Here: At Hippocratic AI, we are revolutionizing the healthcare landscape through cutting-edge technology. We want talented individuals who thrive at the intersection of innovation and impact. You’ll work alongside some of the brightest minds in healthcare and AI to shape the future of healthcare accessibility. This is your chance to get in on the ground floor of something big, with a direct line to transforming healthcare with innovative AI solutions.
Similar Jobs
What We Do
Hippocratic AI’s mission is to develop the first safety-focused Large Language Model (LLM) for healthcare. The company believes that a safe LLM can dramatically improve healthcare accessibility and health outcomes in the world by bringing deep healthcare expertise to every human. No other technology has the potential to have this level of global impact on health.
The company was co-founded by CEO Munjal Shah, alongside a group of physicians, hospital administrators, healthcare professionals, and artificial intelligence researchers from El Camino Health, Johns Hopkins, Washington University in St. Louis, Stanford, Google, Microsoft, Meta and NVIDIA. Hippocratic AI has received a total of $137 million in funding and is backed by leading investors, including General Catalyst, Andreessen Horowitz, Premji Invest, SV Angel, NVentures (Nvidia Venture Capital), and Greycroft. For more information on Hippocratic AI: www.HippocraticAI.com.
