ML Engineering Lead (LLM Ops)

Posted One Month Ago
Be an Early Applicant
3 Locations
Hybrid
Entry level
Healthtech
The Role
Owns the operational lifecycle of LLM, GenAI, and RAG systems, including deployment, monitoring, evaluation, observability, versioning, cost optimization, and production reliability. The role builds ML platform capabilities for regulated clinical workflows, ships agentic or RAG pipelines, establishes human-feedback evaluation loops, and partners closely with the existing MLOps team.
Summary Generated by Built In
Mission

Neko Health is a health technology company that opened its first clinic in Stockholm in 2018, founded by Daniel Ek and Hjalmar Nilsonne.

In one hour, our scan combines radiation-free imaging, heart and circulation measurements, and same-visit blood results with time to talk it through with a clinician, so people can act on what matters before symptoms appear. We design and build our own hardware, software and clinical protocols, and more than 100,000 members have already had a scan, with 75% booking and prepaying their next one before they leave.

As a team, we chase 10X, not 10%, and we’re looking for people who think the same way to help us scale what we’ve started in Sweden, the UK, and now the US. Join the clinicians, engineers and business builders making prevention a routine part of care.

Role Purpose

This role owns the operational lifecycle of Neko's LLM, GenAI and RAG based systems: deployment, monitoring, evaluation and iteration. It builds the production grade platform that lets clinical ML and GenAI workflows run reliably on proprietary sensor and device data, inside a regulated medical device QMS, across Skin, Cardio and other use cases. The role sits within the ML Engineering Area and integrates tightly with the existing MLOps team rather than as a silo.

What You'll Deliver in the First 6-12 Months

● Stand up MLflow Tracing observability, prompts, tool calls, retrievals, latency and cost, live across production LLM and agent pipelines.

● Build an evaluation suite combining built-in and custom LLM judges and scorers, with a human-feedback loop via review apps, replacing today's ad hoc review.

● Ship at least one RAG or agentic pipeline to production with prompt and application versioning through the MLflow Prompt Registry and Unity Catalog, enabling safe rollout, A/B testing and rollback.

● Produce a documented cost, latency and GPU capacity framework for choosing serving strategy: third party API versus Databricks External Models versus self-hosted.

● Integrate LLM Ops tightly with the existing MLOps team so it operates as a natural extension of the platform rather than a separate track, protecting reliability on clinical workflows.

Minimum Qualifications

● Solid MLOps fundamentals across the full lifecycle, experiment tracking, training and monitoring, demonstrated through independent ownership of complex, production grade work.

● Fluent in Python and core ML concepts, with a track record of shipping end-to-end production ML systems and platformisation initiatives.

● Practical, hands-on experience building LLM or GenAI applications: prompt engineering, RAG, agents or chains, using frameworks such as LangChain, LangGraph, or comparable orchestration tools.

● Working knowledge of PyTorch, distributed systems and ML orchestration.

● Conceptual understanding of LLM-specific MLOps trade-offs: fine-tuning versus prompting versus RAG, vector databases, embedding models, and human-feedback loops for non-deterministic outputs. Hands-on fine-tuning ownership is not required, as execution sits with a separate track.

● Genuine, demonstrable motivation for LLM, GenAI and RAG work specifically, not generic MLOps, and the ability to navigate complex systems spanning the medical domain, regulation, firmware and hardware.

Preferred Qualifications

● Experience with agentic or AI-assisted coding workflows, provided you retain full ownership and understanding of the resulting output.

● Kubernetes and Terraform, useful for infrastructure as code or self-hosting fine-tuned or open-source models outside managed serving.

● Exposure to LLM evaluation and observability practices: tracing, LLM-as-judge, guardrails and safety scorers. Databricks MLflow 3 for GenAI experience is a strong plus.

● Comfort navigating a fast-moving tools and platform ecosystem, and distilling recommendations relevant to Neko's specific context.

About titles at Neko

We use a simplified internal title framework that prioritises clarity over hierarchy, so the title on this posting may differ from your internal title. We’ll be open about the scope and level of the role throughout the process, so you know exactly what you’d be taking on.

Skills Required

  • Solid MLOps fundamentals across the full lifecycle, including experiment tracking, training, and monitoring
  • Independent ownership of complex, production-grade MLOps work
  • Fluency in Python and core machine learning concepts
  • Track record of shipping end-to-end production machine learning systems and platformization initiatives
  • Hands-on experience building LLM or GenAI applications using prompt engineering, RAG, agents, chains, or comparable orchestration tools
  • Working knowledge of PyTorch, distributed systems, and ML orchestration
  • Understanding of LLM MLOps trade-offs, vector databases, embedding models, and human-feedback loops
  • Ability to navigate systems spanning the medical domain, regulation, firmware, and hardware
  • Experience with agentic or AI-assisted coding workflows while retaining ownership of the resulting output
  • Experience with Kubernetes and Terraform
  • Exposure to LLM evaluation and observability practices, including tracing, LLM-as-judge, guardrails, and safety scorers
  • Experience with Databricks MLflow 3 for GenAI

Neko Health Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Neko Health and has not been reviewed or approved by Neko Health.

  • Healthcare Strength — Benefits materials and U.S. role listings describe comprehensive medical, dental, and vision coverage, often including mental-health support for clinical staff. This indicates a strong health-focused foundation aligned with a preventive-care mission.
  • Wellbeing & Lifestyle Benefits — Offerings include a complimentary annual Neko Health scan for employees (often extended to eligible family members), a monthly wellness allowance, and wellness rewards. These distinctive perks emphasize preventive health and everyday wellbeing.
  • Leave & Time Off Breadth — Paid time off and paid holidays are standard, with flexible time policies depending on role. Clinician postings also call out CME days in some markets.

Neko Health Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Stockholm
126 Employees
Year Founded: 2018

What We Do

Neko Health is a Swedish health-tech company co-founded in 2018 by Hjalmar Nilsonne and Daniel Ek. Neko's vision is to create a healthcare system that can help people stay healthy through preventive measures and early detection. This requires completely reimagining the patient's experience and incorporating the latest advances in sensors and AI. Neko has developed a new medical scanning technology concept to make it possible to do broad and non-invasive health data collection that is convenient and affordable for the public.

Similar Jobs

Whatnot Logo Whatnot

Strategic Partnerships Manager (German-speaking) (m/w/d)

Consumer Web • eCommerce • Mobile • Retail • Social Media • Software
In-Office
2 Locations
1200 Employees

Whatnot Logo Whatnot

Head Of Marketing

Consumer Web • eCommerce • Mobile • Retail • Social Media • Software
In-Office
2 Locations
1200 Employees

Whatnot Logo Whatnot

Head of Launch, France

Consumer Web • eCommerce • Mobile • Retail • Social Media • Software
In-Office
2 Locations
1200 Employees

Whatnot Logo Whatnot

Marketing Manager

Consumer Web • eCommerce • Mobile • Retail • Social Media • Software
In-Office
2 Locations
1200 Employees

Similar Companies Hiring

OneImaging Thumbnail
Healthtech
Miami, FL
62 Employees
Pomelo Care Thumbnail
Healthtech • Telehealth
New York, NY
550 Employees
Vitalize Thumbnail
Artificial Intelligence • Healthtech • Software
San Francisco, California
50 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account