Research Scientist - Multi-modal LLMs

Job Posted 10 Days Ago Posted 10 Days Ago
Be an Early Applicant
London, Greater London, England
Hybrid
Mid level
Artificial Intelligence • Computer Vision • Machine Learning • Software
The Role
The Research Scientist will focus on building and fine-tuning multi-modal LLMs to enhance data interaction and analytics for users. Responsibilities include developing RAG systems, personalizing models, and following machine learning research to improve processes. This role offers significant growth potential, with a possibility of leading a team in the future. The aim is to create a powerful AI-powered data development platform.
Summary Generated by Built In

About Us


At Encord, we're building the AI infrastructure of the future. One of the biggest challenges AI companies face today is data quality. The success of any AI application relies heavily on the quality of its training data, yet for most teams, this crucial step is both the most costly and time-consuming. We’re here to change that.

As former computer scientists, physicists, and quants, we’ve experienced firsthand how a lack of tools to prepare quality training data impedes progress in building AI. We believe AI is at a stage similar to the early days of computing or the internet—where the potential is clear, but the surrounding tools and processes are still catching up. That's why we started Encord.


We are a talented and ambitious team of 60, working at the cutting edge of computer vision and deep learning. Backed by $30M in Series B funding from top investors like CRV and Y Combinator, we’re one of the fastest-growing companies in our space. Our platform is consistently rated the best by our customers, and we have big plans ahead. We’re looking for a Research Scientist to help our customers get the right data faster, easier, and cheaper.


The Role


As a Research Scientist focusing on multi-modal LLMs, you'll be allowing all the data, metadata, and embeddings that live in our system to be explored, used, and analyzed in ways no one thought possible. Although starting narrow with “smaller” multi-modal problems like, e.g., improving similarity searches via metadata, we have high ambitions for this role. You'll progressively work on harder problems that will improve user experience, surface the right (personalized) analytics to every customer, and put our users in the driver's seat of a data development platform that can do things much beyond today’s standards. Tasks can be i) fine-tuning models to understand how our platform is used by customers, ii) employing LLM reasoning to assist customers in their data analysis tasks, and iii) Building tools for customers to interface naturally with our platform. All to put the power in the hands of anyone using Encord.


You'll follow the latest research and accelerate state-of-the-art technologies to enrich customers’ data journeys. This role offers a great growth opportunity, with the potential to lead a bigger team of scientists over time in our efforts to build the ultimate data development platform

What you will be doing:

  • Building, fine-tuning, and experimenting with multi-modal LLMs to surface potential actions and analytical conclusions in a data-driven manner.
  • Developing scalable and novel ways to personalize LLMs based on information from our data development platform.
  • Build sophisticated RAG systems on other types of data than the usual text documents.
  • Follow the latest machine learning research to identify and apply new methods that improve our processes or the user experience.
  • Ensure our customers have the world’s most powerful AI-powered data development platform.

Skills for the job:

  • A PhD or similarly strong academic background in machine learning, with 2+ years of hands-on experience in with LLM fine-tuning, RAG systems, and prompt engineering.
  • Proficiency with frameworks like PyTorch, Tensorflow, JAX, Pandas, and OpenCV.
  • A solid understanding of transformer models and their common variants, loss functions, and pitfalls.
  • A quick learner with a structured, organized approach to problem-solving.
  • Excellent communication skills with an ability to uncover use cases and solve problems efficiently.
  • Ambitious and self-motivated, with a proven track record of top performance in academic or professional settings.

Bonus skills:

  • Experience working with data in the order of millions.
  • Familiarity with using (and adapting) models like LLaMa and LLaVa.
  • Experience with image-to-text embedding models like CLIP and SigLIP.
  • Familiarity with cloud-based model training and inference.

  • We encourage people from all backgrounds, cultures and skill levels to apply. It is okay to not meet all requirements listed as we are looking for individuals who are passionate, eager to learn.

What We Offer

- Competitive salary, commission, and equity in a high-growth business.

- A collaborative, in-person culture with most of the team working in the office 3+ days a week (engineers typically work on-site Wednesdays).

- 25 days annual leave + public holidays.

- An annual learning and development budget to help you grow your skills.

- Company lunches twice a week and regular socials, including bi-annual off-sites.


At Encord, you’ll have the unique opportunity to be part of a fast-growing startup with a clear mission and vision. You’ll work on real-world AI use cases across a variety of industry verticals and get hands-on experience with cutting-edge computer vision and deep learning technologies. This is a role where you'll grow quickly, take ownership of projects, and help shape the future of our company.

Top Skills

Jax
PyTorch
TensorFlow
The Company
London
37 Employees
On-site Workplace
Year Founded: 2020

What We Do

Machine learning and data operations teams of all sizes use Encord's collaborative applications, automation features, and APIs to train & validate models and to annotate, manage & evaluate training data.

Encord was recognized by CBInsights as one of Top 100 most innovative AI companies in 2022.

We're hiring across product, design, and engineering - check out https://encord.com/careers

Similar Jobs

McDonald’s Global Technology Logo McDonald’s Global Technology

Director Data and Back Office Enablement

eCommerce • Food • Information Technology • Mobile • Cybersecurity • App development • Big Data Analytics
Hybrid
London, Greater London, England, GBR
1400 Employees

ZS Logo ZS

Associate Principal, Clinical Data & Technology

Artificial Intelligence • Healthtech • Professional Services • Analytics • Consulting
Hybrid
London, Greater London, England, GBR
13000 Employees
Hybrid
London, Greater London, England, GBR
289097 Employees

ZS Logo ZS

Manager - Data Strategy and Data Office (Europe) - Data Strategy & Partnerships

Artificial Intelligence • Healthtech • Professional Services • Analytics • Consulting
Hybrid
4 Locations
13000 Employees

Similar Companies Hiring

Stepful Thumbnail
Software • Healthtech • Edtech • Artificial Intelligence
New York, New York
60 Employees
HERE Technologies Thumbnail
Software • Logistics • Internet of Things • Information Technology • Computer Vision • Automotive • Artificial Intelligence
Amsterdam, NL
6000 Employees
True Anomaly Thumbnail
Software • Machine Learning • Hardware • Defense • Artificial Intelligence • Aerospace
Colorado Springs, CO
131 Employees
Not Eligible
Save
By clicking Apply you agree to share your profile information with the hiring company.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account