Staff Software Engineer

Hippocratic AI is the leading generative AI company in healthcare. We have the only system that can have safe, autonomous, clinical conversations with patients. We have trained our own LLMs as part of our Polaris constellation, resulting in a system with over 99.9% accuracy.

• Reinvent healthcare with AI that puts safety first. We’re building the world’s first healthcare-only, safety-focused LLM — a breakthrough platform designed to transform patient outcomes at a global scale. This is category creation.

• Work with the people shaping the future. Hippocratic AI was co-founded by CEO Munjal Shah and a team of physicians, hospital leaders, AI pioneers, and researchers from institutions like El Camino Health, Johns Hopkins, Washington University in St. Louis, Stanford, Google, Meta, Microsoft, and NVIDIA.

• Backed by the world’s leading healthcare and AI investors. We recently raised a $126M Series C at a $3.5B valuation, led by Avenir Growth, bringing total funding to $404M with participation from CapitalG, General Catalyst, a16z, Kleiner Perkins, Premji Invest, UHS, Cincinnati Children’s, WellSpan Health, John Doerr, Rick Klausner, and others.

• Build alongside the best in healthcare and AI. Join experts who’ve spent their careers improving care, advancing science, and building world-changing technologies — ensuring our platform is powerful, trusted, and truly transformative.

We believe the best ideas happen together. To support fast collaboration and a strong team culture, this role is expected to be in our Palo Alto office five days a week unless otherwise specified.

As a Staff Software Engineer, you will work alongside a dynamic team of engineers, applied scientists, and healthcare professionals to drive advancements in our generative AI technologies. Your focus will be on building and optimizing the core systems and services that power model inference at scale.

• Build control plane components around scheduling and capacity planning of cross-cloud distributed inference platform.

• Architect, develop, and maintain scalable backend infrastructure to support high-performance healthcare AI applications.

• Contribute to platform capabilities such as latency-aware routing, model versioning, health monitoring, and observability.

• Improve performance, autoscaling, GPU utilization, and resource efficiency in a cloud-native environment. Work across product, and ML teams to ensure the inference platform meets the scale, reliability, and latency demands of our use cases.

• Hands-on experience with tools like llm-d, SGLang and container orchestration with Kubernetes

• Mentor junior engineers, providing guidance on technical projects and fostering a culture of collaboration and innovation.

Must Have:

• Bachelor’s or Master’s degree in Computer Science, Computer Engineering, or a related field.

• 7+ years of experience in backend development with proficiency in languages like Python, Java, or Go.

• Strong understanding of SaaS, and cloud orchestration software. (Control Plane)

• In-depth knowledge of one or more of cloud platforms (AWS, GCP, or Azure) including Docker, Kubernetes, and cloud networking.

• Familiar with concepts in ML model serving and inference runtimes

• Excellent communication skills and experience working collaboratively within cross-functional teams.

Nice to Have:

• Experience integrating infrastructure with production ML workloads.

• Experience in model serving, inference, observability, and distributed infrastructure

• Previous experience in healthcare-related technologies or regulated data environments.

Join us and help build the future of safe, life-changing AI in healthcare. There’s never been a more exciting moment to make an impact.

***Be aware of recruitment scams impersonating Hippocratic AI. All recruiting communication will come from @hippocraticai.com email addresses. We will never request payment or sensitive personal information during the hiring process. If anything appears suspicious, stop engaging immediately and report the incident.