Applied ML Engineer

Posted 2 Days Ago
San Francisco, CA, USA
In-Office
170K-280K Annually
Mid level
Artificial Intelligence • Machine Learning • Software • Analytics
An AI-powered understanding engine for software and engineering teams.
The Role
Build and improve ML systems powering core AI capabilities: create and maintain evaluation datasets, design experiments, train and fine-tune models (including RL approaches), analyze results to drive product decisions, participate in research, and partner with product and backend teams to integrate models into production.
Summary Generated by Built In
About Macroscope

Macroscope aims to be the source of truth of what's happening for any company that builds software. Our mission is to give leaders clarity and engineers time.

We help leaders understand how their products and codebases are evolving—what’s changing, who’s working on what, and where progress is happening—grounded in the ultimate source of truth: the code.

Macroscope is founded by former entrepreneurs who have started and sold multiple companies, and operated as product/engineering executives at public tech companies. We're fortunate to be supported by the best VC firms and angels in the business, including Lightspeed Venture Partners, Thrive Capital, Google Ventures, and Adverb.

About the role

We're looking for an Applied ML Engineer to help build and improve the machine learning systems that power Macroscope's core AI capabilities. Your primary focus will be on improving model quality through high-quality evaluation datasets, rigorous experimentation, and model training. You'll work closely with our founders and engineering team to determine which models perform best, why they perform the way they do, and how we can continuously improve them.

This is a highly collaborative role where you'll own large parts of the model development lifecycle—from building and maintaining evaluation datasets, to designing experiments, training and fine-tuning models (including reinforcement learning where appropriate), and analyzing results to drive product decisions. You'll also be expected to participate in the research that can push the boundaries of what we are able to do and stay up to date on the latest RL training techniques.

You'll also partner closely with our product and backend engineering teams to integrate new models into production and help shape the future of AI-powered software engineering.

Our technology stack: Typescript/React (front-end), Golang (backend), Temporal, Google Cloud (GCP), Postgres, Terraform, custom-built AST "code walkers" in various languages (Golang, Typescript, Swift, Python, Rust)

Qualifications
  • 3+ years of experience in applied machine learning, AI, or related engineering roles.

  • Experience building, training, fine-tuning, or evaluating modern ML models in production or research environments.

  • Experience with reinforcement learning or reinforcement learning for LLMs (RLHF, RLAIF, GRPO, PPO, DPO, or similar techniques). Any practical experience is valuable.

  • Strong skills in dataset creation, curation, and evaluation, including designing benchmarks, labeling strategies, and evaluation methodologies.

  • Experience designing and running rigorous experiments, analyzing results, and using data to drive model improvements.

  • Familiarity with LLMs, reasoning models, and the rapidly evolving open-source model ecosystem.

  • Strong software engineering skills and experience building reliable ML pipelines and tooling.

  • Comfortable working in a fast-paced, high-agency startup environment where priorities evolve quickly and everyone helps define what to build next.

  • Experience in Golang (the primary language of our backend systems) is a plus, but not required.

  • Bonus: Experience with large-scale distributed training, preference optimization, synthetic data generation, evaluation frameworks, GCP infrastructure, Temporal, or building internal ML tooling.

The base pay for the Senior Applied ML Engineer position ranges from $170k/year to $280k/year. Pay is based on several factors and may vary depending on job-related knowledge, skills, and experience.

About you
  • You are extremely high agency. We are a small startup and we intend to keep an extremely flat organizational structure for as long as possible. Instead of relying on people managers, product managers and heavy processes, we rely on exceptionally talented individuals with high agency to be self-motivated towards contributing to our mission.

  • You want to work at an early stage start-up. The default state of any startup is failure. The only way to overcome the daunting odds of making a startup venture successful is for a densely packed group of insanely hard working and talented people to work together to building something useful to and loved by customers. If you're not willing to work extremely hard on something high risk, this startup isn't for you.

  • You act like an owner. You put immense care and craft into what you build because you take responsibility for all parts of the product. You don't walk past broken windows.

  • You care about what we're building. Life's too short to work on something you're not passionate about. We are a small group of ambitious people who want to build something insanely great that we want to use, and that we think every company will want to use. If our mission and product doesn't resonate with you, we understand and would encourage you to find something that does.

Skills Required

  • 3+ years of experience in applied machine learning, AI, or related engineering roles
  • Experience building, training, fine-tuning, or evaluating modern ML models in production or research environments
  • Experience with reinforcement learning or reinforcement learning for LLMs (RLHF, RLAIF, GRPO, PPO, DPO, or similar)
  • Strong skills in dataset creation, curation, and evaluation, including designing benchmarks, labeling strategies, and evaluation methodologies
  • Experience designing and running rigorous experiments, analyzing results, and using data to drive model improvements
  • Familiarity with LLMs, reasoning models, and the open-source model ecosystem
  • Strong software engineering skills and experience building reliable ML pipelines and tooling
  • Comfortable working in a fast-paced, high-agency startup environment
  • Experience in Golang
  • Experience with large-scale distributed training
  • Experience with preference optimization or synthetic data generation
  • Experience with evaluation frameworks, GCP infrastructure, Temporal, or building internal ML tooling
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
24 Employees
Year Founded: 2023

What We Do

Macroscope is an AI-powered developer platform that helps engineering teams understand and manage complex codebases through intelligent code analysis. The platform summarizes engineering activity, detects bugs, and answers natural-language questions about code to improve productivity.

Why Work With Us

We are a small startup and we intend to keep an extremely flat organizational structure for as long as possible. Instead of relying on people managers, product managers and heavy processes, we rely on exceptionally talented individuals with high agency to be self-motivated towards contributing to our mission.

Gallery

Gallery

Similar Jobs

Hybrid
Palo Alto, CA, USA
289097 Employees

Block Logo Block

Senior Machine Learning Engineer

Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
In-Office or Remote
8 Locations
12000 Employees
195K-343K Annually

Databricks Logo Databricks

Machine Learning Engineer

Big Data • Machine Learning • Software • Analytics • Big Data Analytics
In-Office
San Francisco, CA, USA
2200 Employees
166K-210K Annually
In-Office or Remote
San Francisco, CA, USA
189K-390K Annually

Similar Companies Hiring

Hanover Park Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
42 Employees
Kepler  Thumbnail
Fintech • Software
New York, New York
6 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
60 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account