Epoch AI is looking for a Software Engineer, Cybersecurity Benchmarks who will help us measure how capable frontier AI models are at cybersecurity, and keep those measurements safe to run. The role will involve building and owning cybersecurity benchmarks, securing our evaluation environments against AI escape, and using threat modelling to shape our evaluation strategy.
About The Role
We are looking for a Research Engineer to help us build trustworthy, public evaluations of AI cybersecurity capabilities as part of our AI Benchmarking Hub. You will work closely with the benchmarking team, researchers, analysts and other engineers to design benchmarks, run them on our infrastructure, and make sure the results are accurate and feed into our research.
Your day-to-day activities will include building and maintaining cybersecurity benchmarks (primarily in the Inspect library), hardening the sandboxes they run in, and keeping our benchmarks running so we can evaluate new model releases quickly. An ideal candidate for this role might have a background in red teaming and sandboxing software, alongside strong software engineering experience and hands-on familiarity with AI coding harnesses. Your work will be key to giving researchers, developers and policymakers an independent, rigorous picture of what AI can do in cybersecurity.
Practicalities
Location: Fully remote. We are able to hire in many countries.
Work Hours: Full-time, flexible. We prefer candidates who can overlap with UTC–8 (Pacific Time) and UTC (Greenwich Mean Time).
Travel Requirement: None required. We prefer candidates who can travel to our three retreats per year.
Compensation: $150,000 to $350,000
Application Deadline: Applications are reviewed on a rolling basis
Start Date: As soon as selected candidate is available
Reports to: CTO
What You’ll Own
Design, build and maintain cybersecurity benchmarks for AI within our evaluation infrastructure, primarily using the Inspect library
Pitch and prototype your own ideas for new benchmarks and experiments
Keep the suite running so we can evaluate new model releases quickly and painlessly
Secure evaluation environmentsSandbox and harden our evaluation environments so highly capable AI models cannot escape them
Stay current on how AI agents and coding harnesses interact with their environments
Evaluation strategy and threat modellingUse threat modelling to inform our benchmarking approach and decide what we measure
Provide strategic input on our AI cybersecurity evaluation strategy
Infrastructure and collaborationIntegrate with AI providers and set up benchmarks to run on our infrastructure
Work with researchers, analysts and other engineers to ensure evaluation data and outputs are accurate, insightful, and integrated into our research products and publications
About You
Strong software engineering experience, including several years building and maintaining complex systems and regularly contributing robust, maintainable code
Experience sandboxing software against highly capable adversaries
Red team experience ethically hacking software
Familiarity with agentic AI use and coding harnesses
Ideas and creativity: you can generate your own ideas for new benchmarks and experiments
Motivation from Epoch AI’s mission to provide rigorous, independent insight into key trends in AI, including delivering public, trustworthy evaluations of AI capabilities
Nice-to-havesExperience benchmarking AIs and running evaluations
Experience setting up CTF infrastructure
Experience using modern AI for red teaming or ethically hacking
Knowledge of the inner workings of AI coding harnesses and tool calls
Familiarity with inspect_ai
A solid grasp of current AI trends
If you don’t tick all these boxes but think you would be a great fit, please consider applying anyway!
Additional Information
Please submit all of your application materials in English and note that we require professional level English proficiency.
Please do not include a cover letter, photograph, or headshot of yourself, or any personal information that is not relevant to the role for which you're applying (including marital status, age, identity traits, etc.).
Epoch is committed to building an inclusive, equitable, and supportive community for you to thrive and do your best work. We’re committed to finding the best people for our team, so please don’t hesitate to apply for a role regardless of your age, gender identity/expression, political identity, personal preferences, physical abilities, veteran status, neurodiversity or any other background. Please email [email protected] if you have any questions about this role, accessibility requests, or if you want to request an extension to the application deadline. However, we will not review applications submitted to this email address; please submit your application through the link on this page.
Skills Required
- Several years of strong software engineering experience building and maintaining complex systems
- Experience sandboxing software against highly capable adversaries
- Ethical software hacking or cybersecurity red team experience
- Familiarity with agentic AI and coding harnesses
- Ability to generate ideas for new benchmarks and experiments
- Motivation to provide rigorous, independent evaluations of AI capabilities
- Professional-level English proficiency
- Experience benchmarking AI systems and running evaluations
- Experience setting up CTF infrastructure
- Experience using modern AI for red teaming or ethical hacking
- Knowledge of AI coding harnesses and tool calls
- Familiarity with inspect_ai
- Knowledge of current AI trends
What We Do
Epoch AI is a multidisciplinary research institute investigating the trajectory of artificial intelligence. Its mission is to understand the driving forces behind AI's development and forecast its ramifications on the economy and society. The organization emphasizes making its research accessible through reports, models, and visualizations to ground discussions of AI on a solid empirical footing and foster a rigorous scientific environment.
.png){kind=logo}
