Software Engineers: Create and Validate AI Coding Tasks

We're hiring software engineers for an ongoing remote project focused on building robust evaluations for AI coding agents. Your expertise ensures these models can handle realistic, complex problems in a terminal environment. This work directly feeds into improving the accuracy and reliability of automated engineering tools.

You will design realistic, self-contained software-engineering problems from scratch. The work involves implementing features, debugging software, configuring Docker environments, and writing automated tests. You will also create reference solutions and verify that each task can be graded reliably by our systems. All work happens remotely via your own machine using standard command-line tools and GitHub.

This opportunity is best suited for experienced software engineers who are comfortable navigating unfamiliar codebases. We welcome backend developers, full-stack engineers, DevOps specialists, and systems engineers. Candidates must be highly proficient with command-line tools and writing clean, deterministic tests.

• Design self-contained software-engineering problems for AI agents to solve.

• Build and configure development environments using Docker.

• Write automated, deterministic tests to reliably grade AI performance.

• Create accurate reference solutions for complex coding challenges.

• Professional experience working as a software engineer.

• Comfortable navigating unfamiliar codebases independently.

• Strong proficiency with GitHub and command-line tools.

• Willingness to complete a one-time skills assessment before starting.

$175 per task

Start your paid interview now

Terac is building the world's largest pool of vetted human experts for AI. Researchers, AI labs, and product teams use Terac to recruit, screen, and pay study participants across industries, languages, and skill sets.

Learn more at terac.com or on YouTube at @jointerac.