We are hiring for a Machine Learning Engineer / Research Scientist – Evals to join our AI team. As an early member of our Answer Quality group, you will play a pivotal role in developing and refining the evaluation methodologies that measure the quality, safety, and impact of our AI models. Your work will be central to ensuring that our search engine delivers high-quality answers that drive user engagement and reduce churn.
Responsibilities- Design and Build Evaluation Systems, ensuring our evaluation metrics are reliable, scalable, and actionable. Design, implement, and analyze experiments (such as A/B tests) to validate hypotheses regarding answer quality.
- Integration with Product and Data Teams: Collaborate closely with our PM and AI teams to define common evaluation standards and integrate eval results into our product feedback loop.
- Research and Innovation: Conduct research into state-of-the-art evaluation methods and contribute to the long-term vision for a centralized, scalable evaluation platform.
- Drive Answer Quality via Culture and Tooling: As Perplexity ships new products, build the tools and establish organizational norms to drive answer quality up for each of the new products (Deep Research, Pro Search, Vertical Search, Enterprise, etc.)
- Have 4+ years of experience as an ML Engineer or Research Scientist, with a strong background in designing and running experiments in a fast-growing technology environment
- Solid understanding of Bayesian statistics, and various tools and methodologies for evaluating uncertainty in a way that is specific to the given product being shipped
- Possess deep knowledge in machine learning, including evaluation methods, statistical testing, and algorithm development
- Proficiency in Python and have hands-on experience with ML frameworks (e.g., TensorFlow, PyTorch) and data processing tools
Bonuses:
- Prior experience in evaluating large language models (LLMs) or building evaluation systems for consumer search products
- Exposure to research methodologies in ranking, retrieval, or natural language processing
- Experience in a startup or early-stage company environment
The cash compensation range for this role is $200,000 - $280,000.
At Perplexity, we've experienced tremendous growth and adoption since publicly launching the world's first fully functional conversational answer engine in 2022. We've grown from answering 2.5 million questions per day at the start of 2024 to around 20 million daily queries in December 2024. We also offer Perplexity Enterprise Pro, which counts leading companies like Nvidia, the Cleveland Cavaliers, Bridgewater, and Zoom as customers.
To support our rapid expansion, we've raised significant funding from some of the most respected technology investors. Our investor base includes IVP, NEA, Jeff Bezos, NVIDIA, Databricks, Bessemer Venture Partners, Elad Gil, Nat Friedman, Daniel Gross, Naval Ravikant, Tobi Lutke, and many other visionary individuals. In 2024, our employee base grew nearly 300%, and we're just getting started.Similar Jobs
What We Do
The most powerful answer engine. Powering curiosity with answers backed by up-to-date sources. This is where knowledge begins.
