Head of Trust & Safety

Posted Yesterday
Be an Early Applicant
Hiring Remotely in United States
Remote
Expert/Leader
Artificial Intelligence • Edtech
The Role
Leads trust and safety for AI products serving teachers and students. Owns model guardrails, evaluations, red-teaming, monitoring, incident response, student-facing safety, district trust documentation, and regulatory alignment. Builds safety infrastructure into product development, manages incidents and postmortems, partners with Legal and Engineering, and leads a growing Trust & Safety team.
Summary Generated by Built In

RESPONSIBILITIES: Own the safety of AI output to teachers and students: policy, model-level guardrails, evals, red-teaming, and incident response. Own the trust posture with districts: safety documentation, RFP and school-board responses, and alignment with state AI-in-education guidance. Develop safety evaluations and monitoring as integral components of product infrastructure; ensure that each new tool includes safety acceptance criteria established during the scoping phase, rather than at launch. Own student-facing safety specifically: age-appropriate content, self-harm and crisis handling, and escalation paths to educators. Run the safety incident process end to end and own the postmortem loop back into product and engineering. Partner with Legal on COPPA, FERPA, state privacy laws, and emerging AI regulation. T&S owns the operational reality; Legal owns the legal position. Build and lead a small team (1 to 3 to start), scaling the function through 2027. QUALIFICATIONS / COMPETENCIES / SKILLS. Must Haves: Has owned trust & safety or responsible AI for a consumer or youth-facing product at scale: real incidents, real policy decisions, real regulators. Hands-on with LLM safety: evals, classifiers, prompt/system-level guardrails, red-teaming. Can read an eval result and say what to change. Has made safety a velocity enabler, not friction, by building guardrails as infrastructure so teams ship faster with confidence. Makes judgment calls under ambiguity and owns them publicly, to districts and to the exec team. Strong writer: safety docs, district-facing explainers, incident communications. Operates hypothesis-first and escalates one-way doors fast rather than sitting on them. Nice to Haves: K-12 or youth-platform experience (GoGuardian, Securly, Roblox, Discord, YouTube Kids, Epic Games). Familiarity with state AI-in-education frameworks and district procurement questionnaires. Prior work with child-safety NGOs or standards bodies (Common Sense Media, NCMEC, Thorn, ISTE). Data/ML fluency to partner directly with engineering on classifiers and eval pipelines. Comfortable being the external face of safety at conferences, with press, and in front of district boards. REQUIRED EXPERIENCE: Proven ownership of trust & safety or responsible AI at a consumer or youth-facing product at scale. Hands-on experience with LLM safety tooling: evals, classifiers, and red-teaming. Experience handling live incidents involving minors or vulnerable users. Track record of building safety as product infrastructure, not a gate. Demonstrated ability to communicate safety tradeoffs clearly to executive and district-level audiences. TECH STACK: LLM layer: multi-provider (OpenAI / Anthropic / Google via GCP Vertex); moderation APIs and custom classifiers. Eval and red-team tooling (to be built out); Datadog for output observability and alerting. Product stack context: TypeScript / Next.js on Vercel, Supabase (PostgreSQL) backend. Incident and knowledge tooling: Linear, Slack workflows, Notion.

Skills Required

  • Proven ownership of trust and safety or responsible AI for a consumer or youth-facing product at scale
  • Hands-on experience with LLM safety tooling, including evaluations, classifiers, guardrails, and red-teaming
  • Experience handling live safety incidents involving minors or vulnerable users
  • Experience building safety capabilities as product infrastructure rather than as a launch gate
  • Ability to communicate safety tradeoffs clearly to executives and district-level audiences
  • Strong writing skills for safety documentation, explainers, and incident communications
  • K-12 or youth-platform experience
  • Familiarity with state AI-in-education frameworks and district procurement questionnaires
  • Experience with child-safety NGOs or standards bodies
  • Data and machine learning fluency to partner with engineering on classifiers and evaluation pipelines
  • Comfort serving as an external safety representative at conferences, with press, and before district boards

MagicSchool AI Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about MagicSchool AI and has not been reviewed or approved by MagicSchool AI.

  • Healthcare Strength — Health coverage is described as including medical, dental, and vision with low-to-no premiums and quick eligibility, indicating robust core benefits.
  • Retirement Support — Retirement support includes a defined 401(k) match alongside equity, presenting a balanced total‑rewards mix for long‑term value.
  • Leave & Time Off Breadth — Paid time off is framed as unlimited, offering broad flexibility for rest and personal time.

MagicSchool AI Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Boulder, CO
95 Employees
Year Founded: 2023

What We Do

MagicSchool is the leading generative AI platform in education, boasting over 2 million educators on the platform in less than a year. It fights educator burnout by providing them AI assistance and creates responsible and safe AI experiences for students to prepare them for the future of work.

Similar Jobs

Shield AI Logo Shield AI

Capture Manager

Aerospace • Artificial Intelligence • Machine Learning • Robotics • Software • Defense Technology
Remote
USA
280K-420K Annually

Headway Logo Headway

Senior Engineering Manager

Consumer Web • Healthtech • Professional Services • Social Impact • Software
Remote
USA
819 Employees
265K-332K Annually

Headway Logo Headway

Vendor Operations Lead, CX

Consumer Web • Healthtech • Professional Services • Social Impact • Software
Remote
USA
819 Employees
95K-140K Annually

General Motors Logo General Motors

Field Warranty Manager -Phoenix, AZ

Automotive • Big Data • Information Technology • Robotics • Software • Transportation • Manufacturing
Remote or Hybrid
United States
165000 Employees

Similar Companies Hiring

Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
60 Employees
Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
30 Employees
Vega Thumbnail
Artificial Intelligence • Automotive • Insurance • Transportation
US
65 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account