About the company
Our client is a fast-growing software company.
The role
Raydar is recruiting for this role on behalf of our client. Own machine learning projects from initial scoping through production release, building the systems that allow automated AI agents to reason, plan and be assessed reliably. The work blends research-style experimentation with hands-on engineering and a high degree of autonomy.
What you'll do
- Design and iterate on multi-agent systems that automate complex professional workflows.
- Define safe operating limits, tool usage and fallback behavior so automated agents remain dependable.
- Manage context and memory across multi-step agent runs, with clear success measures.
- Route, assess and tune models within practical limits on speed, cost and accuracy.
- Build scalable offline and online evaluation pipelines that run large numbers of experiments automatically.
- Define benchmark tasks, labeling approaches and metrics so performance can be compared over time.
- Instrument systems to catch regressions, categorize errors and drive continuous improvement.
- Shape prompt structures and instruction hierarchies, and create search and indexing pipelines that supply the right information to models.
- Convert unstructured documents into structured representations and add validation layers that keep behavior predictable.
- Write concise specs and design documents, test your work end to end, and share progress and learnings with your immediate team.
Requirements
What we're looking for
- Background at a fast-paced startup, a quantitative trading firm, or a leading technology company.
- Based in the US or Canada at the moment, with a preference for the New York area or the east coast.
- Practical experience delivering LLM-powered agent products, including performance benchmarking, coordinating multiple models and testing how reliably agents behave.
- Experience designing and running rigorous ML experiments, from forming hypotheses to building evaluation tooling and acting on measured results.
- A degree in computer science, physics, math or another technical field from a top school.
- Clear communication and the ability to break complex ideas into simple parts.
- Enthusiasm for a startup environment and for working in the office five days a week.
Bonus points
- Strong Python skills and depth with large language model systems.
- Prior work shipping ML-driven products in a fast-moving company.
- Curiosity about how AI is changing finance and the wider economy.
Benefits
Compensation and benefits
- Base salary: USD 175,000 to 350,000 per year
- Equity
- Health, dental and vision coverage
- Life insurance
- Unlimited paid time off and paid holidays
- Retirement plan
- Parental leave
- Commuter benefits
- Mental health and wellness support
Location and work model
- New York, NY, United States
- On-site, 5 days per week in office
- Full-time
Skills Required
- Background at a fast-paced startup, quantitative trading firm, or leading technology company
- Currently based in the United States or Canada
- Practical experience delivering LLM-powered agent products
- Experience with performance benchmarking, coordinating multiple models, and testing agent reliability
- Experience designing and running rigorous machine learning experiments and evaluation tooling
- Degree in computer science, physics, mathematics, or another technical field from a top school
- Clear communication skills and ability to explain complex ideas simply
- Enthusiasm for a startup environment and willingness to work in the office five days per week
- Strong Python skills and depth with large language model systems
- Prior experience shipping machine-learning-driven products in a fast-moving company
- Curiosity about AI's impact on finance and the wider economy
What We Do
Raydar is a talent acquisition and business consulting firm that connects world-class and emerging talent with growing organizations. It supports companies through team development, strategic hiring, and customized growth solutions, helping clients recruit roles such as engineers, product managers, executives, legal counsel, and quantitative traders. Raydar focuses on understanding each organization’s needs, culture, and long-term goals to build high-impact teams.
