LanceDB is a developer-friendly, open-source database for multimodal AI. From hyper-scalable vector search to advanced retrieval for RAG, from streaming training data to interactive exploration of large-scale AI datasets, LanceDB is the best foundation for your AI application, and powers some of the most groundbreaking applications and challenging requirements today.
About the roleJoin our world-class team as an experienced Open Source Engineer, where you'll drive the evolution of high-performance multimodal databases. You'll leverage your expertise in Java/Scala and Rust to expand the reach of Lance and LanceDB within the broader Data Infrastructure ecosystem, contributing to cutting-edge open source projects.
As an Open Source Engineer at LanceDB, your responsibilities will include:
Driving OSS community effort to integrate Lance format into Spark, Hive Metadata Store, Presto, Trino, Ray and other Data Infrastructure systems.
Helping promote Lance format in Big Data conferences and meetups.
Designing and maintaining efficient distributed Lance dataset operations.
Designing efficient indices to power predicate push down in Spark, Ray or Trino.
Work on table format, data encodings and various aspect of the Lance format in Rust.
Operating on in-house data processing infrastructure.
You have at least five years of experience building high-performance databases, big data systems, or web-scale data services.
Experience with internals of Open Source Big Data or AI training systems, such as Hadoop, Spark, Flink, Ray, Iceberg, Delta-lake, Hudi, Clickhouse, Trino, Presto, PyTorch or JAX.
Hands-on experience with high-performance computing in Java or Scala.
You like working with a small, high-caliber team with a lot of autonomy and drive, and you can iterate fast
You are an open-source veteran, committer or PMC of large Open Source systems in the Apache community.
You fearlessly challenge the status quo and dismiss mediocre engineering as unacceptable.
You have a proven record driving large features in Apache projects.
You are familiar with Java, Rust, C++, Apache Arrow, Apache DataFusion, Apache Parquet, Apache Iceberg, and Delta Lake.
LanceDB was created by experts with decades of experience building tools for data science and machine learning. From co-authors of pandas to Apache PMC of HDFS, Arrow and Delta, the LanceDB team has created open source tools used by millions world-wide.
Top Skills
What We Do
LanceDB is a developer-friendly, open source database for multimodal AI. From hyper scalable vector search to advanced retrieval for RAG, from streaming training data to interactive exploration of large scale AI datasets, LanceDB is the best foundation for your AI application.
