Staff ML Engineer

A code change lands. It touches five percent of the codebase. The test suite doesn't care — it runs everything, burning time and compute testing the ninety-five percent that didn't change.

Every engineering team knows this is wasteful. They do it anyway, because the alternative — knowing which tests actually matter for a given change — has been too hard to get right.

We're building the team that gets it right. This role sits at the centre of it.

Test Engine already ingests billions of test runs. We can see test suites, the code behind them, and the relationships between the two at a scale very few people ever get to work with.

The next step is the hard one: for a given change, predict the slice of tests most likely to fail — and run only those. Get it right and teams stop re-testing what hasn't changed, and spend that time where it counts: like fixing the two percent of tests most likely to break.

It's a genuinely difficult ML problem — sparse signal, cold-start on new repos, generalising across languages and frameworks, and latency tight enough to sit in the critical path. It's also close to a blank page. There's no ML org above you setting the direction — you'd set it. And you won't be setting it alone: we've just hired another ML engineer, so there's someone to think out loud with from day one.

Machine learning in Test Engine, end-to-end — the strategy, the architecture, and the models running in production.

That means shaping the whole path: pulling features out of code changes and test history, training and evaluating models, building the serving layer that keeps predictions fast, and closing the loop so the system keeps improving. You'd make the trade-offs that matter — accuracy versus latency, what happens when confidence is low — and build the platform underneath so the next model into production is quick and repeatable, not a one-off.

You've taken ML models the whole way — from rough idea to something running reliably in production, monitored and retrained, owned rather than handed off.

Two things matter more than any specific tool:

• You've built ML that generalised. Not one clever model — a repeatable approach that worked across more than one use case.
• You're comfortable where the signal is noisy. Classification, ranking, prediction — problems where the data doesn't hand you the answer.

Day to day you'll live in Python and SQL, on AWS, with containerised workloads and data-at-scale tooling (Spark, Flink, or similar). Experience with code analysis, CI/CD systems, or ranking problems is a real head start — a bonus, not a bar.

The one thing we won't budge on: you've shipped and owned ML in production. Prototyped and handed off doesn't count here.

You're likely a strong fit if you:

• Get energised by a blank page and want to be the one who fills it
• Care more about models working in production than papers about models
• Do your best work async, with deep focus and real autonomy

This probably isn't the right role if you:

• Want an established ML org around you for direction and review
• Prefer research and experimentation over shipping and operating
• Need close scaffolding — flat and high-autonomy means less of it

We'd rather you know that now than three interviews in.

• Frontier work. CI/CD is becoming the next bottleneck in the AI era, and Buildkite is built for that moment.
• Real scale. The world's leading engineering teams ship software to over a billion daily users through Buildkite. Your models sit in their critical path.
• Ownership. ~150 people, flat structure, and you're the most senior ML person here — influence you don't get where the ML org is three layers deep.
• Remote, properly. We've worked this way since 2013 — async, built for deep focus, with genuine overlap across ANZ and US-Pacific.

Every application gets a response. If this is the problem you've been wanting to get your hands on, apply now, or reach out with questions first.