Multimodal VLM/LLM Researcher

Join Black Forest Labs, the pioneering team behind Stable Diffusion, Stable Video Diffusion, and FLUX.1, as we push the boundaries of generative AI. We're seeking an exceptional researcher to run cutting-edge projects in multimodal vision-language and large language models.

Key Responsibilities

Model Development & Training

• Run the development and training of state-of-the-art multimodal vision-language models within the FLUX technology stack
• Design and implement specialized fine-tuning strategies for VLMs to address specific use cases and performance requirements
• Develop and optimize LLM implementations for prompt enhancement, content moderation, and novel applications

Research & Innovation

• Drive innovation by integrating VLM/LLM capabilities into our media generation pipeline
• Conduct research to creatively combine vision and language models for enhanced generative capabilities
• Maintain cutting-edge knowledge of the latest developments in multimodal AI and LLM research
• Evaluate emerging models and architectures for potential integration into our technology stack

Technical Leadership

• Collaborate with cross-functional teams to implement and deploy models at scale
• Contribute to architectural decisions and technical roadmap planning
• Document and share research findings with the broader team

Required Qualifications

• Demonstrated expertise in training and fine-tuning large-scale vision-language models
• Strong publication record or practical experience with relevant projects in multimodal AI research
• Proficiency in PyTorch or similar deep learning frameworks
• Experience with distributed training systems and large-scale model optimization
• Track record of implementing and scaling AI models in production environments

Nice to have

• Experience with diffusion models and generative AI architectures alongside autoregressive modelling 
• Background in computer vision 
• Contributions to open-source AI projects
• Experience working in fast-paced startup environments
• Strong software engineering practices and system design skills
• Experience in open-source VLM inference frameworks such as vLLM

What We Offer

• Opportunity to work with the strong technical team at Black Forest Labs
• Access to state-of-the-art computing resources
• Collaborative, research-focused environment
• Competitive compensation package
• Flexible work arrangements
• Chance to shape the future of generative AI

We're looking for self-motivated individuals who are passionate about advancing the field of AI and can thrive in a fast-paced, research-driven environment. If you're excited about pushing the boundaries of what's possible in generative AI, we want to hear from you.