Senior Product Manager - Tech, Experimentation & Failure

COMPANY: STEVEN.COM

REPORTING TO: CITO

LOCATION: LONDON

Steven.com is building the operating system for the creator economy, forecast to pass a trillion dollars by the early 2030's. This industry is currently held back by fragmented tools and a lack of professional infrastructure. Steven.com is the unlock. We are the end-to-end Operating System designed to scale what is irreplaceably human.

We have built proprietary technology and obsessed teams to identify and scale the highest-potential creators across our core pillars:

• Creator Media: Amplifying reach, influence, and trust

• Creator Community: Transforming audiences into connected tribes

• Creator Products: Providing creators with the infrastructure to build and back aligned products and ventures

• Powered by Creator Tech & Intelligence: A proprietary data and technology suite that fuels smarter decisions and drives innovation across the entire flywheel.

Our Experimentation & Failure team has one of the most strategically important and unusual mandates at Steven.com: increase the rate of failure. As Steven Bartlett puts it: "the path to the correct answer is out-failing your competition." This isn't a growth team or an optimisation function. It's the team that exists to make sure we learn faster than anyone else.

Lead the Experimentation & Failure team, reporting to the CITO. You'll out-experiment and out-fail the competition - running high-velocity, rigorous experiments across every show, creator, piece of content, and commercial bet at Steven.com, while building a team and a culture that treats deliberate failure as the primary learning mechanism. This is deeply hands-on: you'll be setting hypotheses, isolating variables, checking statistical power, reading results, and moving to the next test - not directing from a distance.

• Own and drive the experimentation agenda across every show, creator, and IP property in the FlightStory portfolio - from podcast topic selection and episode structure through to thumbnail design and social tile copy. No detail too small to test.

• Lead hypothesis formation for every experiment, with clear success, failure, and inconclusive criteria defined in advance - and enforce single-variable discipline across the board.

• Ensure every experiment is adequately powered before launch: sample sizes calculated, measurement windows defined, results interpretable by design.

• Systematically increase experimentation velocity and build the intake process that makes high-volume testing the default across FlightStory and Steven.com.

• Build institutional memory - a searchable, structured record of every experiment run, what was learned, and what was decided - as a compounding organisational asset.

• Partner with the VP of Engineering & Applied AI to apply AI tooling to experiment design, analysis, and reporting, and to ensure infrastructure supports testing at this velocity.

• Deep, first-principles command of experimentation mathematics: statistical significance, power, sample size calculation, p-values, confidence intervals, Type I/II errors, and the difference between statistical and practical significance.

• Genuine mastery of the scientific method applied to product and content - hypothesis formation, single-variable isolation, measurement design, and result interpretation.

• A track record of building experimentation culture, not just running tests - creating an environment where the whole team experiments and failure is rewarded.

• Comfortable querying data and working shoulder-to-shoulder with engineers and data scientists at implementation depth.

• Strong written and verbal communication - able to write a hypothesis an engineer respects and explain a result a producer will act on.

• Experience operating at pace, in high-volume testing environments where speed of learning is the competitive edge.

• You think in hypotheses, not features.

• You isolate one variable at a time, and understand viscerally why changing five things at once makes a result meaningless.

• You give a null or inconclusive result the same intellectual respect as a win - you know how to extract the signal either way.

• You're obsessive about measurement: an experiment that can't be measured is just a change, not an experiment.

• You're deeply sceptical of your own results, and design experiments to prove yourself wrong rather than confirm what you already believe.

• You move fast, expect others to move fast, and don't wait for perfect conditions to run a test.

• You believe failure is feedback, feedback is knowledge, and knowledge is power - and you build systems to generate that knowledge at the highest possible rate.

• Demonstrable experience leading (not just participating in) product or content experimentation programs at a technology, media, or creator-economy company - owning the methodology, volume, and culture.

• Strong advantage: experience with algorithmic platforms and how to design experiments against platform-specific metrics; podcasting, video, social, or creator-economy background; familiarity with YouTube/Spotify/social analytics (CTR, retention, watch time, audience behaviour).

• Nice to have: experience building an experimentation platform from scratch; familiarity with causal inference beyond standard A/B testing (holdouts, quasi-experiments, diff-in-diff); experience experimenting on AI/ML systems or prompt variations in production; a background in statistics, maths, CS, economics, or a natural science; exposure to early-stage environments where you had to build the experimentation infrastructure yourself.