Principal Product Engineer

Reposted 2 Days Ago
Be an Early Applicant
29 Locations
In-Office or Remote
Mid level
Machine Learning
The Role
The AI Engineer - Optimizer will research and develop optimization techniques for large language models, focusing on improving performance and efficiency through automatic prompt optimization and collaboration with ML teams.
Summary Generated by Built In
About Comet

Comet is building the development platform for teams who want to ship robust, reliable, and responsible AI applications. Opik, our open source LLM evaluation framework, has quickly become one of the most popular tools in the space. Our experiment management platform is used by data scientists at companies like Uber, Netflix, and Etsy. Tens of thousands of researchers, engineers, academics, and hobbyists use Comet every day to build the future of AI. 

Working at Comet will give you access to the most exciting work being done in all areas of machine learning. Some of the top researchers and companies working on self-driving cars, drug discovery, particle research, diffusion models, and LLMs use Comet every day. Your work has the potential to accelerate the development of some of the most impactful technology in the world, and you will be doing it alongside a team of passionate, caring individuals. If that sounds exciting, Comet is the right place for you.

Comet is backed by more than $63 million in venture capital funding and powers some of the best machine-learning teams in the world, including Netflix, Uber, Etsy, and Mobileye. We are a remote-first company with offices in New York City (USA) and Tel Aviv (Israel).

Comet is seeking a Principal Product Engineer to join our team and help build the next generation of Opik AI products!  At the heart of this role is the Opik Agent Optimizer, a toolkit designed to automatically enhance the performance and efficiency of Large Language Model (LLM) applications. Instead of manually tweaking prompts and running evaluations, our optimizer enables automated, intelligent prompt optimization, helping developers build better, faster, and more reliable AI applications. Ready to shape the future of AI optimization? Join us in building the most advanced agent optimization tools in the market!

This position will be located in Europe/USA (Remote) or Israel (Hybrid).

Responsibilities:
  • Build and ship AI-first product features with a focus on automatic prompt optimization and efficiency gains
  • Conduct user research and interviews to uncover pain points and translate insights into impactful product features
  • Perform market research to identify emerging trends, tools, and workflows in the AI ecosystem
  • Collaborate with ML engineers and researchers to integrate state-of-the-art optimization and evaluation methodologies
  • Develop user-facing APIs, dashboards, and tools that simplify and accelerate optimization workflows
  • Share technical learnings through blogs, open-source contributions, and engagement with the developer community
Requirements:
  • Proven experience building products and tools that developers love to use
  • 5+ years in AI/ML engineering, with a focus on optimization, model training, or applied research
  • Hands-on experience with LLM evaluation pipelines, hyperparameter tuning or AutoML frameworks
  • Ability to design experiments, analyze results, and turn insights into actionable improvements
  • Deep understanding of large-scale models (GPT-5, Claude, Gemini, etc.), their limitations, and optimization opportunities
The ideal candidate is:
  • A strong communicator with excellent collaboration skills in a fast-paced, distributed environment
  • Product-minded, combines technical depth with a passion for improving developer experience
  • Comfortable with ambiguity and has the ability to solve complex technical and product challenges
  • A hands-on builder who ships early, learns from real-world usage, and iterates quickly
  • Holds a high bar for quality, simplicity, and user impact, taking pride in building elegant solutions
  • Curious, self-directed, and excited to contribute to open-source and engage with the AI community
Nice to Have:
  • Open Source: Contributions to ML/AI open-source projects
  • Optimization Background: Familiarity with Bayesian optimization, reinforcement learning, or meta-learning techniques
  • LLM Evaluation expertise: Knowledge of AI evaluation metrics, prompt engineering, and “LLM-as-a-Judge” systems
  • Community Impact: Track record of sharing research/insights via blogs, publications, or talks
  • Experience with agent optimization, including dynamic prompt adjustment and evaluation at scale

Bonus points:

  • Former founder
Why Join Comet?
  • Play a key role in shaping the future of ML Ops and LLM Ops in one of the most exciting domains today
  • Work with a talented and passionate team on cutting-edge technology
  • Competitive salary, benefits, and opportunities for career growth
  • A dynamic and inclusive work environment that values innovation and creativity and promotes personal development and growth
  • This role will be located in Europe/ USA (Remote) or Israel (Hybrid), working with a global team (large presence in the US, Tel Aviv and Europe), some flexibility with work hours is required

Comet is an equal-opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees without regard to race, religion, color, sex, gender identity, gender expression, sexual orientation, national origin, ancestry, citizenship status, uniform service member status, marital status, pregnancy, age, medical condition, physical or mental disability, genetic information/characteristics, and any other characteristic protected by State or Federal law.

Top Skills

Automl
Bayesian Optimization
Hyperparameter Tuning
Llm Evaluation
Machine Learning Frameworks
Python
Reinforcement Learning
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: New York, NY
87 Employees
Year Founded: 2017

What We Do

Comet is a meta machine learning platform designed to help AI practitioners and teams build reliable machine learning models for real-world applications by streamlining and connecting the machine learning model lifecycle. By leveraging Comet, users can employ machine learning experiment tracking to track, compare, explain and reproduce their models. Backed by thousands of users and multiple Fortune 100 companies, Comet provides insights and data to build better, more accurate AI models while improving productivity, collaboration and visibility across teams.

Similar Jobs

MacPaw Logo MacPaw

Head of Growth

Information Technology • Security • Software • Cybersecurity • App development • Data Privacy
Remote or Hybrid
28 Locations

MacPaw Logo MacPaw

Growth Manager for CleanMyMac

Information Technology • Security • Software • Cybersecurity • App development • Data Privacy
Remote or Hybrid
28 Locations

GitLab Logo GitLab

Product Analyst

Cloud • Security • Software • Cybersecurity • Automation
Easy Apply
Remote
31 Locations
115K-246K Annually

GitLab Logo GitLab

Product Analyst

Cloud • Security • Software • Cybersecurity • Automation
Easy Apply
Remote
31 Locations
87K-186K Annually

Similar Companies Hiring

Caliola Engineering Thumbnail
Software • Machine Learning • Hardware • Defense • Data Privacy • App development • Aerospace
Colorado Springs, CO
59 Employees
Yooz Thumbnail
Software • Machine Learning • Fintech • Financial Services • Cloud • Automation • Artificial Intelligence
Aimargues, FR
470 Employees
Credal.ai Thumbnail
Software • Security • Productivity • Machine Learning • Artificial Intelligence
Brooklyn, NY

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account