Senior Research Engineer, Computer Vision (LFV/WFM)

Posted 2 Days Ago
Be an Early Applicant
Los Altos, CA, USA
Hybrid
180K-259K Annually
Senior level
Artificial Intelligence • Computer Vision • Machine Learning • Natural Language Processing • Software • Generative AI • Renewable Energy
The Role
Develop and deploy world foundation models for autonomous driving and embodied AI. Responsibilities include implementing research architectures, building scalable multimodal data pipelines, optimizing distributed multi-GPU and multi-node training, developing experiment and evaluation tools, and translating research prototypes into maintainable production-grade systems. The role collaborates with research teams, Toyota affiliates, universities, and multiple TRI divisions while supporting datasets, model evaluation, infrastructure, and open-source contributions.
Summary Generated by Built In

At Toyota Research Institute (TRI), we envision a future where technology helps humanity thrive. Our world-class teams work across disciplines and collaborate with leading universities to advance research in AI, human happiness, robotics, driving, and materials science, helping shape the future of mobility, amplify human ability, and make our lives safer and more sustainable.


The Learning From Videos (LFV) team develops world foundation models that leverage large-scale multi-modal data (RGB, depth, flow, semantics, actions, tactile, audio, etc.) from multiple domains to power downstream embodied AI tasks. Our topics of interest include Video Generation, World Models, 4D Reconstruction, Multi-Modal Models, Multi-View Geometry, Data Augmentation, and Video-Language-Action models, with a primary focus on embodied applications such as robotics and driving. We are making progress on some of the hardest scientific challenges around spatio-temporal reasoning, and how it can lead to the deployment of autonomous agents in real-world unstructured environments.


Our team is looking for a Research Engineer to help develop and deploy our world foundation models (WFMs) toward their key milestones in the autonomous driving domain. As our WFMs scale in both capability and ambition, we need a strong engineer who can bridge the gap between research ideas and production-grade systems. This is not a traditional software engineering role; you will work directly alongside research scientists, understand the research deeply enough to make independent technical decisions, and play a key role in enabling the deployment of key research breakthroughs into close-to-production environments.


As a Research Engineer, you will be responsible for supporting and optimizing large-scale distributed training of diffusion and transformer models; maintaining the infrastructure that ingests, unifies, and serves heterogeneous multi-modal datasets at scale; and developing tools and pipelines that accelerate the research-to-results cycle. You will work closely with researchers to prototype new ideas, run experiments, and help ship our most successful models toward real-world applications with real-world impact.

This role requires close collaboration with multiple TRI divisions (Robotics, Automated Driving, Human-Interactive Driving, etc.) as well external Toyota and University partners, and the ability to reconcile and prioritize possibly competing requirements in a fast-paced combination of research and production environments.

Responsibilities

  • Collaborate directly with research scientists to implement, iterate on, and evaluate new architectures, objectives, datasets, and training strategies. Translate research prototypes into clean, maintainable, reusable code that will be shared across multiple TRI teams and the broader Toyota ecosystem.

  • Build and maintain scalable pipelines for ingesting, converting, validating, and serving heterogeneous datasets (multi-view, multi-modal, multi-embodiment, etc.), across robotics and autonomous driving, into unified training-ready formats. Track and integrate new public and internal datasets as they become available.

  • Support and optimize large-scale distributed training of world foundation models on multi-GPU and multi-node clusters. Manage experiment workflows, profiling, debugging, and hyperparameter sweeps to ensure optimal performance in a timely manner.

  • Develop tools for dataset inspection, experiment tracking, model evaluation, GPU resource management, and visualization. Automate repetitive workflows to improve team velocity.

  • Work with other TRI teams and Toyota affiliates to set up shared pipelines, onboard their data, and support joint training and evaluation efforts.

  • Produce maintainable, well-documented code. Contribute to internal tooling and open-source releases to the scientific community.

Qualifications

  • Master’s or PhD in Computer Science, Electrical Engineering, Machine Learning, or a related field, with a minimum of 3 years of relevant experience and strong software engineering skills.

  • Deep proficiency in Python, PyTorch, and the Unix/Linux toolchain. Comfort working in terminal-heavy, SSH-based workflows on shared GPU clusters.

  • Hands-on experience with large-scale deep learning training, including distributed training (DDP, FSDP, DeepSpeed, or similar), GPU profiling, and debugging training failures at scale.

  • Experience building data pipelines for heterogeneous or multi-modal datasets (images, video, depth, point clouds, actions, etc).

  • Experience with video diffusion models, 3D/4D reconstruction, and multi-view geometry.

  • You are proactive, self-directed, and comfortable operating with ambiguity in a research-driven environment that spans multiple divisions.

  • You are a reliable teammate who communicates clearly and takes ownership of problems end-to-end.

Bonus Qualifications

  • Experience with cloud training infrastructure (AWS SageMaker, EC2) and containerized workflows (Docker, Kubernetes).
  • Familiarity with standard data formats and collection pipelines (ROS, MCAP, HDF5, etc.) as well as simulation environments.

  • Proficiency with modern AI-assisted development tools (e.g., Copilot, Cursor, Claude Code) for accelerating engineering workflows.

  • Track record of contributions to open-source projects or publications at top venues (CVPR, ICLR, NeurIPS, RSS, ICRA, etc.) is a plus but not required.

Please include links to any relevant open-source contributions or technical project write-ups with your application.
 
The pay range for this position at commencement of employment is expected to be between $180,000 and $258,750/year for California-based roles. Base pay offered will depend on multiple individualized factors, including, but not limited to, a candidate's experience, skills, job-related knowledge, and market location. TRI offers a generous benefits package including medical, dental, and vision insurance, 401(k) eligibility, paid time off benefits (including vacation, sick time, and parental leave), and an annual cash bonus structure. Additional details regarding these benefit plans will be provided if an employee receives an offer of employment.

Please reference this Candidate Privacy Notice to inform you of the categories of personal information that we collect from individuals who inquire about and/or apply to work for Toyota Research Institute, Inc. or its subsidiaries, including Toyota A.I. Ventures GP, L.P., and the purposes for which we use such personal information.
 
TRI is fueled by a diverse and inclusive community of people with unique backgrounds, education and life experiences. We are dedicated to fostering an innovative and collaborative environment by living the values that are an essential part of our culture. We believe diversity makes us stronger and are proud to provide Equal Employment Opportunity for all, without regard to an applicant’s race, color, creed, gender, gender identity or expression, sexual orientation, national origin, age, physical or mental disability, medical condition, religion, marital status, genetic information, veteran status, or any other status protected under federal, state or local laws.
 
It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability. Pursuant to the San Francisco Fair Chance Ordinance, we will consider qualified applicants with arrest and conviction records for employment.

Skills Required

  • Master's or PhD in Computer Science, Electrical Engineering, Machine Learning, or a related field
  • Minimum of 3 years of relevant experience
  • Strong software engineering skills
  • Deep proficiency in Python, PyTorch, and the Unix/Linux toolchain
  • Experience with large-scale deep learning training and distributed training such as DDP, FSDP, or DeepSpeed
  • Experience with GPU profiling and debugging training failures at scale
  • Experience building pipelines for heterogeneous or multimodal datasets including images, video, depth, point clouds, or actions
  • Experience with video diffusion models, 3D/4D reconstruction, and multi-view geometry
  • Proactive, self-directed, and comfortable operating in ambiguous research environments
  • Reliable teamwork, clear communication, and end-to-end ownership
  • Experience with AWS SageMaker or EC2 and containerized workflows using Docker or Kubernetes
  • Familiarity with ROS, MCAP, HDF5, and simulation environments
  • Proficiency with AI-assisted development tools such as Copilot, Cursor, or Claude Code
  • Open-source contributions or publications at venues such as CVPR, ICLR, NeurIPS, RSS, or ICRA
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Los Altos, CA
250 Employees
Year Founded: 2016

What We Do

Toyota Research Institute (TRI) envisions a future where Toyota products, enabled by TRI technology, dramatically improve quality of life for individuals and society. To achieve its Vision, TRI’s Mission is to create new tools and capabilities focused on improving the human condition through research in Energy & Materials, Human-centered AI, Human Interactive Driving, and Robotics. We’re on a mission to improve the quality of human life. To lead this transformative shift, we are looking for the world's best talent -- people who enjoy solving tough problems while having fun doing it.

Why Work With Us

TRI is fueled by a diverse and inclusive community of people with unique backgrounds, education and life experiences. We are dedicated to fostering an innovative and collaborative environment by living the values that are an essential part of our culture.

Gallery

Gallery

Similar Jobs

Liberty Mutual Insurance Logo Liberty Mutual Insurance

Associate Claims Adjuster, Workers Compensation

Artificial Intelligence • Fintech • Insurance • Marketing Tech • Software • Analytics
Hybrid
2 Locations
40000 Employees
54K-94K Annually

Shield  Logo Shield

Operations Manager

Aerospace • Artificial Intelligence • Machine Learning • Robotics • Software • Defense Technology
Hybrid
San Mateo, CA, USA
153K-230K Annually

Product.ai Logo Product.ai

Design Engineer

Artificial Intelligence • Big Data • Consumer Web • eCommerce
In-Office
Metropolitan, CA, USA
25 Employees
155K-180K Annually

Rocket Logo Rocket

Senior Marketing Manager

Fintech • Real Estate • Sales • Financial Services
In-Office
Los Angeles, CA, USA
14200 Employees
77K-162K Annually

Similar Companies Hiring

Revel Thumbnail
Aerospace • Hardware • Robotics • Software
Marina Del Rey, California
70 Employees
Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
33 Employees
Vega Thumbnail
Artificial Intelligence • Automotive • Insurance • Transportation
US
65 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account