About the company
Our client is a technology company.
The role
Raydar is recruiting for this role on behalf of our client. Design datasets, evaluation methods and measurement frameworks that shape how machine learning models are trained and assessed. Collaborate with external research partners, run fast experiments and convert findings into concrete specifications for large-scale training work.
What you'll do
- Design targeted data samples that reveal weaknesses in model behavior across a range of subject areas.
- Build and refine scoring rubrics and reward signals used in model training pipelines.
- Study how human annotators behave and run experiments to improve model capabilities.
- Develop quantitative methods for assessing dataset quality, diversity and downstream impact.
- Own the data pipelines that supply both naturally occurring and artificially generated training material.
- Work with research partners to turn their training goals into clear data and evaluation specifications.
Requirements
What we're looking for
- 1 to 4 years of experience in software or machine learning engineering.
- Strong Python skills and experience building dependable data pipelines, experimentation tooling or evaluation systems.
- Hands-on experience with language model evaluation, post-training, reward modeling or closely related ML infrastructure.
- Sound scientific judgment, including setting up well-controlled tests and drawing conclusions from imperfect results.
- Background in building or auditing training data and judging how well it supports model performance.
- Ability to learn unfamiliar domains quickly and reason across them.
Benefits
Compensation and benefits
- Base salary: USD 260,000 to 290,000 per year
- Equity
- Profit sharing
Location and work model
- San Francisco, CA, United States
- On-site
- Full-time
Skills Required
- 1 to 4 years of experience in software or machine learning engineering
- Strong Python skills
- Experience building dependable data pipelines, experimentation tooling, or evaluation systems
- Hands-on experience with language model evaluation, post-training, reward modeling, or related machine learning infrastructure
- Sound scientific judgment, including controlled testing and interpreting imperfect results
- Experience building or auditing training data and evaluating its impact on model performance
- Ability to learn unfamiliar domains quickly and reason across them
What We Do
Raydar is a talent acquisition and business consulting firm that connects world-class and emerging talent with growing organizations. It supports companies through team development, strategic hiring, and customized growth solutions, helping clients recruit roles such as engineers, product managers, executives, legal counsel, and quantitative traders. Raydar focuses on understanding each organization’s needs, culture, and long-term goals to build high-impact teams.
