About the company
Our client is a software company.
The role
Raydar is recruiting for this role on behalf of our client. Lead the measurement and improvement of AI agent quality, owning evaluation design and error analysis while building the backend systems that run and monitor agents. The role also mentors engineers and helps set the reliability direction for the AI platform.
What you'll do
- Set the strategy for measuring agent quality, including evaluation design and error-analysis practice.
- Design and build the knowledge and evaluation infrastructure that AI agents depend on.
- Lead investigations into agent failures and turn the findings into platform-level reliability fixes.
- Build and harden backend services that run, evaluate and monitor agents at scale.
- Help set the reliability and quality roadmap for the AI platform, working closely with the wider AI team.
- Mentor engineers on evaluation rigor and error analysis.
Requirements
What we're looking for
- A seasoned engineering background, with founders who have a strong track record also considered.
- 1 to 2 years of experience building end-to-end AI agents or production AI platforms, including agent loops, evaluation and error analysis.
- Backend, data platform or full-stack engineering experience.
- Hands-on history shipping backend code for AI features that end users rely on in live environments.
- Degree from a top 50 computer science program, in the US or internationally.
- Comfortable being on-site in the city roughly three days each week.
Bonus points
- Experience at a recently IPO'd technology company or an AI-native startup.
- Distributed systems experience.
Benefits
Compensation and benefits
- Base salary: $220K to $315K
- Equity
Location and work model
- San Francisco, CA, United States
- Hybrid, about 3 days in-office per week on average with no set days
- Full-time
Skills Required
- Seasoned engineering background
- 1 to 2 years of experience building end-to-end AI agents or production AI platforms, including agent loops, evaluation, and error analysis
- Backend, data platform, or full-stack engineering experience
- Hands-on experience shipping backend code for AI features used by end users in live environments
- Degree from a top 50 computer science program in the United States or internationally
- Comfortable working on-site in San Francisco approximately three days per week
- Experience at a recently IPO'd technology company or AI-native startup
- Distributed systems experience
What We Do
Raydar is a talent acquisition and business consulting firm that connects world-class and emerging talent with growing organizations. It supports companies through team development, strategic hiring, and customized growth solutions, helping clients recruit roles such as engineers, product managers, executives, legal counsel, and quantitative traders. Raydar focuses on understanding each organization’s needs, culture, and long-term goals to build high-impact teams.
.png)
