Research Engineer, Human Understanding

We are seeking a highly motivated Research Engineer (L5) with a strong background in multi-modal modelling for humans and a focus on speech & audio/visual to join the effort within Google DeepMind's Frontier AI unit. This role is pivotal in developing foundational multimodal AI capabilities to understand, generate, and protect human likeness. As a key contributor, you will design and implement cutting-edge models and frameworks, pushing the boundaries of AI to enable foundational capabilities for human-centric understanding and generation. This is a unique opportunity to contribute to impactful research and advance Google DeepMind's mission towards Artificial General Intelligence (AGI).

Artificial Intelligence could be one of humanity’s most useful inventions. At Google DeepMind, we’re a team of scientists, engineers, machine learning experts and more, working together to advance the state of the art in artificial intelligence and ultimately achieve Artificial General Intelligence. We use our technologies for widespread public benefit and scientific discovery, and collaborate with others on critical challenges, ensuring safety and ethics are the highest priority.

The effort is a part of Google DeepMind's Frontier AI unit. The team aims to build holistic representation encompassing a full spectrum of human understanding. We develop systems to provide perception skills critical for person-centric applications, which is crucial for enabling AI to interact naturally & seamlessly, depict humans accurately &  responsibly in generative AI, and build trustworthy & resilient systems that can detect and prevent misuse like deepfakes and impersonation.

You will drive outcomes for critical technical components aimed at advancing our capabilities in multimodal human understanding. You will play a critical role in developing and deploying models that can provide accurate human understanding across multiple modalities (e.g., visual appearance, voice, dynamics, etc), while also building robust defenses against sophisticated AI-driven manipulation and impersonation.

This role involves tackling complex, ambiguous problems with no obvious "best" solution, requiring independent judgment and a proactive approach to exploring multiple technical avenues. You will be instrumental in shaping the technical direction for core components of the effort. Your contribution will lead to key breakthrough and impactful landings within GDM and across Google products, ensuring our technologies are both groundbreaking and responsibly deployed.

• Advance multimodal human representations & understanding : Research and implement novel models and other multimodal techniques for a more holistic understanding of humans across visual, audio, and textual data.
• Conduct applied research: Conduct experimental research cycles from hypothesis to deployment.
• Drive technical projects: Take ownership of substantial technical projects within the effort, from ideation and design to implementation and evaluation, often involving cross-functional collaboration.
• Contribute to Infrastructure: Inform and contribute to the development of scalable and efficient research infrastructure for multimodal human understanding models and datasets.
• Design and execute strategies for tuning and adapting VLMs and other foundation models for specific tasks

In order to set you up for success as a Research Engineer at Google DeepMind, we look for the following skills and experience:

Requirements:

• PhD degree in Computer Science, Machine Learning, or a related technical field with 3+ years of relevant experience.
• Experience in developing machine learning models, such as audio & speech-visual models.
• Experience in working with and tuning large-scale vision language models.
• Strong programming skills in Python and experience with at least one major deep learning framework (e.g., JAX)
• Experience conducting independent research and development, including experimental design, implementation, and analysis.

In addition, the following would be an advantage:

• Experience with Generative AI techniques and architectures.
• Familiarity with Reinforcement Learning or alignment methods.
• A track record of publications in top-tier AI/ML conferences (e.g., NeurIPS, ICML, ICLR, CVPR, ICCV, ECCV).
• Experience with multimodal learning, integrating information from different data types (e.g., vision, audio, text).
• Understanding of privacy-preserving machine learning or responsible AI practices.

The US base salary range for this full-time position is between 174,000 USD - 252,000 USD + bonus + equity + benefits. Your recruiter can share more about the specific salary range for your targeted location during the hiring process.

Note: In the event your application is successful and an offer of employment is made to you, any offer of employment will be conditional on the results of a background check, performed by a third party acting on our behalf. For more information on how we handle your data, please see our Applicant and Candidate Privacy Policy.

At Google DeepMind, we value diversity of experience, knowledge, backgrounds and perspectives and harness these qualities to create extraordinary impact. We are committed to equal employment opportunities regardless of sex, race, religion or belief, ethnic or national origin, disability, age, citizenship, marital, domestic or civil partnership status, sexual orientation, gender identity, pregnancy, or related condition (including breastfeeding) or any other basis as protected by applicable law. If you have a disability or additional need that requires accommodation, please do not hesitate to let us know.