Machine Learning Engineer - Moderation

Posted Yesterday
Be an Early Applicant
San Francisco, CA, USA
In-Office
220K-280K Annually
Senior level
Artificial Intelligence • Machine Learning • Generative AI
The Role
Develop and deploy multimodal machine learning systems that detect harmful and illegal content. Integrate moderation models into large-scale production pipelines, optimize accuracy, cost, latency, and scalability, and respond to customer feedback. Collaborate with infrastructure, legal, policy, product, and go-to-market teams to establish effective moderation policies. Responsibilities span ML research, experimentation, evaluation, deployment, monitoring, and addressing adversarial evasion across text, image, video, and audio modalities.
Summary Generated by Built In
About Black Forest Labs

We’re the team building the foundational layer for visual intelligence: multimodal models trained jointly on images, video, audio, and action to see, generate, and understand the world.

Our founding team pioneered latent diffusion and Stable Diffusion, the breakthroughs that made generative AI accessible to millions. With FLUX, we're pushing into generative media at scale, machines that act in the physical world, and emerging applications in computer use.

Our FLUX models top public benchmarks and run in production in some of the world's biggest platforms. We're a small, fast growing research lab competing with the largest companies in the world. From our labs in Freiburg to San Francisco, we're building what comes next.

 
Why This Role

Black Forest Labs’ products arm individuals with great creative power to produce novel media across a range of modalities. The quality of our models inevitably also gives bad actors the opportunity to create harmful, copyright-infringing, or illegal content.

In this role you will develop and improve our moderation system: a multilayered stack of detection models. Succeeding in this role requires creativity and familiarity with a large range of ML techniques. An effective solution that balances accuracy, cost, latency, and scalability will vary case-by-case, from classical ML fine-tuning, to RAG-based agents, to simple, lightweight semantic embeddings.

This is a Senior / Staff IC role. Our ideal candidate is able to work across the full stack of ML engineering, collaborate cross-functionally with many teams, including legal, policy, go-to market, and infra, and has a real interest in the domains of AI safety and integrity.

 
What You’ll Work On
  • Develop ML models and systems to detect harmful and/or illegal content across a variety of classes and modalities

  • Collaborate with our core infrastructure team to integrate these models into a large-scale production serving pipeline

  • Handle customer requests and feedback on our moderation behavior, and make quick adjustments in response

  • Collaborate with our legal, policy, and product teams to design moderation policies that are technically feasible and effective while preserving a positive user experience

What We’re Looking For
  • Experience deploying VLMs at scale for the classification of multimodal content

  • Experience building AI systems around state-of-the-art open-source VLMs

  • Experience tuning models for classification tasks, including building eval sets, reasoning about precision/recall at threshold, and monitoring for drift

  • Comfort with the full stack of ML engineering: from research and experimentation to large-scale deployment

Nice to have:
  • Comfort with the adversarial nature of generative AI moderation: prompt obfuscation, jailbreaks, output laundering, and evasion generally

  • Experience in moderation/safety space, preferably across a range of modalities (text, image, video, audio)

  • Experience optimizing VLM inference latency at scale

  • Experience with non-VLM solutions for safety classification (e.g., classical ML)

  • Domain knowledge and experience with content copyright and/or child safety moderation systems

 
How We Work Together

We’re a distributed team with real offices that people actually use. This role is based in San Francisco, and you’ll join us in the office at least 2 days a week, or one full week every other week.

Everything we do is grounded in four values:

  • Obsessed. We are a frontier research lab. The science has to be right, the understanding deep, the product beautiful.

  • Low Ego. The work speaks. The best idea wins, no matter who said it. Credit is shared. Nobody is above any task.

  • Bold. We take the ambitious bet. We ship, we do not wait for conditions to be perfect.

  • Kind. People over politics. We treat each other with genuine warmth. Agency without empathy creates chaos.

If this sounds like work you’d enjoy, we’d love to hear from you.

Base Annual Salary:

US $220,000 - $280,000 USD + Equity

 

Note: We may use AI-assisted tools to help organize applications. All hiring decisions are always made by our team.

Skills Required

  • Experience deploying vision-language models at scale for multimodal content classification
  • Experience building AI systems around state-of-the-art open-source vision-language models
  • Experience tuning models for classification, including building evaluation sets, analyzing precision and recall thresholds, and monitoring model drift
  • Experience across the full machine learning engineering stack, from research and experimentation through large-scale deployment
  • Experience with adversarial generative AI moderation, including prompt obfuscation, jailbreaks, output laundering, or evasion
  • Experience in moderation or AI safety across text, image, video, or audio modalities
  • Experience optimizing vision-language model inference latency at scale
  • Experience with non-vision-language-model approaches for safety classification, such as classical machine learning
  • Domain knowledge of content copyright or child safety moderation systems
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company

What We Do

Black Forest Labs is a frontier AI research laboratory focused on building state-of-the-art generative deep-learning models for media, including images and videos. Its mission is to advance creativity, efficiency, and diversity in generative media while developing visual intelligence—systems that understand, reason, and act in the world. The company created FLUX, its image-generation model suite, and aims to establish an industry standard for generative media.

Similar Jobs

Pfizer Logo Pfizer

Director, Solution Engineer, Data Storage and Registration

Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Remote or Hybrid
13 Locations
121990 Employees
147K-272K Annually

Pfizer Logo Pfizer

Senior Computational Chemist

Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Hybrid
La Jolla, CA, USA
121990 Employees
106K-177K Annually

Pfizer Logo Pfizer

Sr. Manager, Solution Engineer, Visualization, Analytics & Collaboration

Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Remote or Hybrid
13 Locations
121990 Employees
112K-207K Annually

Capital One Logo Capital One

Machine Learning Engineer

Fintech • Machine Learning • Payments • Software • Financial Services
Hybrid
4 Locations
55000 Employees
269K-335K Annually

Similar Companies Hiring

Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
85 Employees
Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
33 Employees
Vega Thumbnail
Artificial Intelligence • Automotive • Insurance • Transportation
US
65 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account