Build your future with Sovos.
Sovos is transforming tax compliance from a business requirement into a force for growth. Trusted by more than 100,000 customers worldwide, including many of the world's leading organizations, we are leveraging AI to build smarter, more impactful customer experiences.
We're looking for an AI Ops Engineer to own the end-to-end LLMOps lifecycle for a portfolio of Ask Sovi products. You'll drive AI quality, safety, evaluation, observability, and continuous improvement, ensuring our AI-powered experiences are reliable, scalable, and production-ready.
What You'll Do- Gate new AI agents, prompts, and models before production release
- Design and run evaluation frameworks using deterministic testing and LLM-as-a-Judge methodologies
- Build and maintain golden datasets, annotation pipelines, and human-review workflows
- Monitor AI quality, hallucinations, drift, latency, cost, and safety metrics
- Convert customer feedback and production learnings into measurable quality improvements
- Drive AI adoption and quality standards across your assigned product portfolio
- 1-3 years of experience in AI Engineering, AI Operations, AI Quality Engineering, LLMOps, MLOps, Test Automation, or related disciplines
- Experience with Langfuse, LangSmith, Phoenix, or similar LLM observability/evaluation platforms
- Strong understanding of LLMOps, AI quality engineering, and evaluation frameworks
- Experience with hallucination detection, model drift monitoring, prompt regression management, and AI safety evaluations
- Hands-on experience building evaluation datasets, quality benchmarks, and feedback loops
- Familiarity with NLP concepts such as embeddings, semantic similarity, classification, and retrieval
- Strong product mindset and stakeholder management skills
- Applied AI Engineers from AI-native or AI-first organizations
- Product-minded Engineers with strong customer and business awareness
- QA/Test Automation Engineers who have transitioned into AI evaluation
- AI Quality Analysts with expertise in hallucination detection, drift monitoring, and LLM-as-a-Judge frameworks
UK Shift
Why Join Sovos?- Work on cutting-edge Generative AI products
- Global, remote-first collaboration with AI, Product, and Engineering leaders
- Bi-Weekly Meeting Free Days
- Mentoring and career development programs
- Globally recognized training opportunities
- Comprehensive health and wellness benefits
- Opportunity to shape AI experiences used by customers around the world
- CCAT & EPP Assessment
- Automated Video Interview
- Hiring Manager / Team Interview
- Final Interview
Interested in building the future of Enterprise AI with Sovos? We'd love to hear from you.
Skills Required
- 2-3 years of experience in AI Engineering, AI Operations, LLMOps, MLOps, AI Quality Engineering, Test Automation, or related disciplines
- Experience with Langfuse, LangSmith, Phoenix, or similar LLM observability/evaluation platforms
- Strong understanding of LLMOps, AI quality engineering, and evaluation frameworks
- Experience with hallucination detection, model drift monitoring, prompt regression management, and AI safety evaluations
- Hands-on experience building evaluation datasets, quality benchmarks, and feedback loops
- Familiarity with NLP concepts such as embeddings, semantic similarity, classification, and retrieval
- Strong product mindset and stakeholder management skills
Sovos Compensation & Benefits Highlights
The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about Sovos and has not been reviewed or approved by Sovos.
-
Leave & Time Off Breadth — Flexible or unlimited time off is broadly offered and stands out as a strong aspect of the package. While usage may depend on workload and manager norms, the breadth of the policy is clear.
-
Parental & Family Support — Parental leave is included, with examples of extended maternity and multi‑week leave for other parents. This support complements wider family‑oriented resources.
-
Wellbeing & Lifestyle Benefits — Wellness resources, donation matching, volunteer time off, recognition programs, and structured development/mentorship are part of a broad offering. Remote/flexible work options and community perks further bolster everyday wellbeing.
Sovos Insights
What We Do
Sovos provides businesses with the confidence needed to navigate an increasingly regulated world. Offering a modern, IT driven response to compliance, Sovos’ solutions are scalable, reliable and secure and provide a great user experience. Sovos has more than 100,000 customers, including half the Fortune 500, as well as businesses of every size operating in more than 70 countries. The company’s cloud-based solutions integrate with a wide variety of business applications and government compliance processes. Sovos has employees throughout the Americas and Europe and is owned by Hg and TA Associates.
Why Work With Us
We are dedicated to being a place where people truly enjoy being. We promote internally and offer a variety of development programs internally. Tax and the regulatory compliance space is also an incredibly stable industry! It changes a lot (because, tax!) but it is never going away. It is always something businesses will have to do!
Gallery
