Join Tech @ Ro to build the future of healthcare, from the ground up!
At Ro, we believe that when people achieve their health goals, they can achieve their life goals. The highest-leverage way to move society forward is to give people their health, and the current healthcare system isn’t built to do that. It was built to bill, not to serve patients.
We’re building a new system. One where the patient is in control. One designed from scratch for the digital age.
At Ro, technology isn’t just a function… It's core to how we deliver care. We’ve built a vertically integrated healthcare platform that connects telehealth, diagnostics, pharmacy, and logistics into a seamless, end-to-end experience for millions of patients.
…and we’re just getting started.
As part of Tech @ Ro, you’ll work on systems that operate at scale, with an opportunity to:
- Solve complex, high-concurrency problems across a full-stack platform
- Build and ship quickly with tight feedback loops and real-world impact
- Own systems end-to-end, from architecture to production performance
- Work alongside experienced operators, technical leaders, and clinicians
- Help define how modern healthcare should be delivered
We’re a performance-driven team with a strong sense of ownership and urgency. We move fast, learn quickly, and hold a high bar for what we build, and do so with a big heart — because patients depend on it.
If you’re motivated by impact, scale, and the chance to help lead the patient revolution, come build with us.
We're hiring a Senior Applied AI Scientist to own the evaluation, measurement, and optimization of our AI systems. This role sits at the intersection of data science, applied machine learning, and product engineering. You'll design the frameworks that tell us whether our AI systems are actually working and use those insights to continuously improve them.
This is not a research role. You'll work closely with engineers and product teams to evaluate production systems, run experiments, identify failure modes, and ensure our AI products become more accurate, reliable, and cost-effective over time.
What You'll Do
- Design and own evaluation frameworks for production LLM features, including LLM-as-a-judge evaluations, regression suites, synthetic datasets, golden datasets, and human review workflows.
- Analyze production behavior to identify quality issues, hallucinations, latency bottlenecks, cost regressions, and emerging failure modes.
- Design and run experiments including prompt variations, workflow changes, retrieval improvements, and model comparisons; and quantify their impact on quality, operational metrics, and user outcomes.
- Define the metrics that matter and build dashboards that make AI performance visible across the organization.
- Partner with engineering to determine which optimizations should be productionized and how to measure ongoing success.
- Mentor teammates on experimental design, statistical rigor, evaluation methodology, and measurement best practices.
Who You Are
- 5+ years of experience in data science, applied machine learning, experimentation, or a closely related field, with at least the last year focused on applied LLMs or AI evaluation.
- Strong Python and SQL skills with experience working on production data pipelines and experimentation.
- You have experience designing reproducible evaluation frameworks rather than relying on manual spot checks or qualitative assessments.
- You have strong statistical intuition: you think in terms of distributions, confidence intervals, variance, and sample sizes rather than anecdotes.
- You’re comfortable working closely with engineers and product teams to translate experimental findings into production improvements
- Bonus: Experience with evaluation platforms (e.g. Braintrust, LangSmith, OpenAI Evals), experimentation platforms, causal inference, healthcare, or operations-heavy environments.
A note on reporting structure
This is a new function at Ro, and we're being deliberate about not over-defining it. Your manager and where you sit on the org chart will depend on the specific shape of the team we end up with. We'd rather find the right people and figure out the lines around them than pre-draw boxes and miss great candidates. If that ambiguity is a deal-breaker, this isn't the right role; if it sounds like an opportunity, we want to talk.
Skills Required
- 5+ years in data science, applied machine learning, experimentation, or closely related field with at least one year focused on applied LLMs or AI evaluation.
- Strong Python skills.
- Strong SQL skills.
- Experience working on production data pipelines and experimentation systems.
- Experience designing reproducible evaluation frameworks for production LLM features (LLM-as-judge, regression suites, synthetic/golden datasets, human review workflows).
- Strong statistical intuition (distributions, confidence intervals, variance, sample sizes) and rigorous experimental design.
- Ability to partner with engineering and product teams to translate experimental findings into production improvements.
- Mentoring teammates on experimentation, evaluation methodology, and measurement best practices.
- Experience with evaluation platforms (e.g., Braintrust, LangSmith, OpenAI Evals), experimentation platforms, causal inference, healthcare, or operations-heavy environments.
What the Team is Saying
_1.jpg)
%20(1).png)
_1.jpg)
%20(1).png)
What We Do
Ro is a direct-to-patient healthcare company with a mission of helping patients achieve their health goals by delivering the easiest, most effective care possible. Ro is the only company to offer nationwide telehealth, labs, and pharmacy services. This is enabled by Ro's vertically integrated platform that helps patients achieve their goals through a convenient, end-to-end healthcare experience spanning from diagnosis, to delivery of medication, to ongoing care. Since 2017, Ro has helped millions of patients in nearly every single county in the United States, including 98% of primary care deserts.
Why Work With Us
Ro is powering quality care at scale. The Ro Operating System (ro.OS) vertically integrates the core parts of healthcare, bringing together nationwide telehealth, lab, and pharmacy services on one platform. The result? ro.OS makes it easier for patients to access and providers to deliver high-quality care – millions of times over.
Gallery
Ro (Ro.co) Teams
Ro (Ro.co) Offices
Hybrid Workspace
Employees engage in a combination of remote and on-site work.
Ro’ers in the tri-state area join their colleagues in the NY Hub twice a week for in-person collaboration.
.png)
