Content Strategy Manager, Assessments

Where we work

Udemy is a global company headquartered in San Francisco, with additional U.S. offices in Denver and Austin, and international hubs in Australia, India, Ireland, Mexico, and Türkiye. This is an in-office position, requiring three days a week in the office (Tuesday, Wednesday, Thursday) and flexibility on Mondays and Fridays.

About this role

Udemy is seeking a dynamic Assessment Content Manager to join our growing assessment team as part of the Instructor & Content Strategy department. This role will help manage day-to-day assessment content operations, leveraging automations and AI-driven processes to enhance the development and maintenance of our assessment content at scale. For this role, you will leverage your extensive foundational knowledge of assessments to ensure the smooth operation and scalability of our assessment content while contributing to assessment innovations in support of multiple products.

As an Assessment Content Manager, you will play a pivotal role in fostering cross-functional collaboration with our Psychometric, Product, Learning Design, and Engineering teams to further establish and optimize our end-to-end assessment operations. You will contribute to and manage key aspects of our assessment content program including assessment creation and maintenance, vendor and instructor management, and assessment model development.

What you’ll be doing

• Oversee and manage the end-to-end scaled creation and maintenance of assessment content

• Drive the transition to scalable, AI-driven assessment content operations

• Manage relationships with vendors, instructors, and subject matter experts (SMEs) for project execution, exploration, and feedback.

• Collaborate with cross-functional teams to support strategic initiatives, including content strategy development and the development of new assessment models

What you’ll have

• 3+ years of hands-on industry experience in an assessment development role

• You have experience in managing end-to-end assessment content operations, including development and maintenance

• You have a strong understanding of assessment fundamentals (validity, reliability, fairness) and diverse assessment models (formative, summative, diagnostic)

• You have experience with learning and assessment product design 

• You excel at analyzing results and metrics to inform data-driven decisions

• You thrive in dynamic environments and demonstrate flexibility and adaptability in response to evolving business needs

• You are highly organized with excellent project management and communication skills capable of managing multiple tasks independently

Bonus points for:

• +5 years of relevant industry experience such as leading assessment development projects,  managing assessment content operations, and/or leading assessment product teams

• Educational background in assessment, learning, and/or instructional design fields

• Experience with learning and assessment product design and development

• Experience with e-learning content and assessment management systems, platforms, and tooling.

• Strong fundamentals using generative AI in assessment contexts (e.g., prompt engineering) as well as familiarity with AI-tooling (e.g., ChatGPT)

• Familiarity with technical skills space (computer science, AI/ML, data science, etc.)