CodePath.org Leadership & Management

What’s the leadership like on the AI and machine learning team at CodePath.org?

CodePath's AI and machine learning team is relatively small, consisting of four team members. The director leading the team still writes code and contributes to analyses.

Leadership on the team is built around responsibility and ownership. The team sets goals each quarter, and every member of the team owns and leads a set of them, with the trust to plan that work and deliver it without asking permission along the way. There’s a weekly meeting to check status and clear roadblocks, and this the main avenue for director oversight, along with weekly one-on-ones with team members.

The operating belief is that the job is to hire A-players and let them do their best work, so direction gets set at the level of goals rather than tasks. Experimentation is encouraged, as long as it’s pointed at one of those goals. CodePath is a nonprofit with a startup culture, and pivoting quickly when something isn’t working matters for a team this size.

Where leadership does step in is on workload. There are a lot of balls in the air and several workstreams running at once, so people are encouraged to work in discrete blocks of time and limit context switching.

How does leadership help your team decide where to focus AI or ML efforts?

Each quarter, the team director works with other members to set goals that they will be accountable for. They cover two kinds of work: serving the rest of the organization with internal reports, dashboards and the data infrastructure underneath them, and building CodePath’s evidence base through research partnerships with outside evaluators.

Where machine learning fits gets decided in that same conversation, rather than on a separate track. Once a goal is set, the next question is how the deliverable actually gets built, and whether a model is the right way to build it.

The clearest case is classification. Large language models sort student outcomes data at a scale nobody could review by hand, and they apply the same rules on every run, which manual classification never managed.

We generally set two hard limits on where AI can be used. It can be a second set of eyes reviewing code changes but can never be the primary “reviewer” of any code, especially code that was generated by AI agents. And any analysis that could be visible inside or outside the organization doesn’t get signed off without a person first reviewing it.

What should candidates know about how your team balances strategic direction with experimentation?

The quarterly goals provide the strategic direction for the team, but staff have a lot of latitude in how they get there. We don’t decide in advance exactly which tools or methods will be used to achieve a goal. If there’s a better way to do something, people are expected to try it.

The important part is knowing when to stop. We don’t want experimentation to become work in its own right, so it needs to be connected to a specific question or deliverable. If an approach isn’t working, the expectation is to move on rather than keep investing in it because we’ve already put time into it.

That can mean changing course several times during a quarter. The team is small, the work is varied, and there are usually several reasonable ways to solve a problem. Staff are expected to have substantial freedom over how they approach their work, but also to make decisions about what is worth pursuing and what isn’t.