Own evaluation, measurement, and optimization of production LLM-powered features. Design reproducible evaluation frameworks, run experiments and analyses to identify failure modes and regressions, build metrics and dashboards, partner with engineering to productionize improvements, and mentor teammates on experimental design and measurement best practices.

Embed with clinical and operations teams to identify high-value AI opportunities; build LLM-powered features end-to-end including prompts, orchestration, Python services, evaluation suites, golden datasets, and metrics-driven experiments to measure product impact and prioritize work.

Build and ship LLM-powered production features end-to-end: prompt orchestration, tool calling, retrieval/RAG, embeddings, evaluation infrastructure, observability, safety guardrails, and integrations. Collaborate with clinical teams, prototype quickly, measure outcomes, and raise engineering standards through mentorship and technical leadership.