Senior AI Engineer, Agents (Python) (f/m/d)

Posted 2 Days Ago
Be an Early Applicant
Hiring Remotely in Office, Machaze, Manica, MOZ
Remote
Senior level
Artificial Intelligence • Productivity • Sales • Software
The Role
Own the quality of Bliro’s chat and voice AI agents by improving prompts, context, model selection, evaluation harnesses, and benchmarking. Review real conversations, identify failure modes, build test sets and metrics, measure quality, latency, and cost, and establish a repeatable process for continuous agent improvement. The role requires strong Python skills, experience shipping LLM agents, statistical evaluation expertise, and the ability to translate conversations into actionable fixes.
Summary Generated by Built In

Bliro:

  • The AI assistant for field sales

  • Takes desk work off sales reps' plates entirely, with more than one million customer touchpoints documented

  • Trusted by lighthouses of the German Mittelstand like igus, Lanxess and Brillux

  • Backed by 468 Capital, Lea Partners, Rockstart and more, with over €3M raised

  • See Bliro in action: https://bliro.io

Why We're Hiring This Role:

  • Our chat agent and our voice to voice phone agent process and document thousands of customer interactions a week for field sales teams. Every answer has to be right, fast and useful on the road.

  • Our backend engineers build the platform. We need a senior AI engineer who owns agent quality: what the agents say, how well they say it, and how we prove they got better.

  • You'll own prompting and context, model benchmarking for our use cases, evals, and the agent harness behind both agents. You join a small team of high performing specialists who own the core of our agents. This role is about orchestrating and evaluating models, not training them.

What You'll Do:

  • 30 Days: Map Bliro's chat and voice agents end to end. Partner with the backend engineers, review real user conversations, identify the biggest failure modes, and ship an initial improvement.

  • 60 Days: Own agent quality measurement. Build eval sets and metrics for our key use cases, benchmark models on quality, latency and cost, and ship a change with a measurable improvement against the baseline.

  • 90 Days: Establish a repeatable model for improving agent quality. Run evals on every prompt, model and harness change, turn real conversations into new test cases automatically, and set the longer term roadmap for agent quality.

Who You Are:

  • Evidence of ownership and shipping LLM agents to real users and proving with data that they got better. You can demonstrate excellence and pursued a technical major at one of the major technical universities like TUM, ETH, KIT, RWTH, EPFL, etc.

  • Strong Python and the ability to debug agent behavior across prompts, tools, context and model choice.

  • Hands on experience building eval pipelines, LLM judges and model benchmarks, with enough statistics to tell a real gain from noise.

  • A habit of reading raw conversations and turning what you find into test cases, fixes and clear priorities.

  • Deep curiosity and the ability to gather context quickly, form a clear point of view, and keep pace as models and the product evolve.

  • A low ego teammate who communicates clearly and owns outcomes. Familiarity with AI coding tools is expected; experience with the latest agent frameworks, realtime voice agents and multilingual agents is a plus.

  • This is a full time role. We prefer hybrid in Munich but are open to remote.

How We Work:

  • Ambition: We get better every day by being curious and bold.

  • Continuous improvement: Move fast and break things, but never the same thing twice.

  • It's our problem: We operate as one team. We share credit, own mistakes together, and support each other when things get hard.

  • Why not today? We value urgency and momentum. The fastest path to customer value usually wins.

  • Build something worthy of love: Craft matters. We aim to build products customers genuinely love.

Why Bliro:

  • Generational impact: Build the human interface for millions of field sellers

  • Ownership culture: Many of us are former founders

  • Forward thinkers: We are the world leader in privacy preserving transcription, we know how boundaries can be overcome

What We Offer:

  • Real stake: Strong base salary plus meaningful equity ownership

  • Time off: 30 days of PTO, so take what you need

  • Hybrid setup: Work from our office in the heart of Munich, right next to the Frauenkirche, or from home. We value outcomes, not attendance

  • More: Free meals and drinks, Deutschlandticket, best in class equipment, and Wellpass

Skills Required

  • Evidence of owning and shipping LLM agents to real users
  • Demonstrated ability to prove agent improvements with data
  • Technical major from a major technical university
  • Strong Python programming skills
  • Ability to debug agent behavior across prompts, tools, context, and model choice
  • Hands-on experience building evaluation pipelines, LLM judges, and model benchmarks
  • Sufficient statistics knowledge to distinguish real gains from noise
  • Ability to analyze raw conversations and turn findings into test cases and fixes
  • Clear communication, low-ego collaboration, and ownership of outcomes
  • Familiarity with AI coding tools
  • Experience with the latest agent frameworks, realtime voice agents, or multilingual agents
  • Availability for a full-time role
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
23 Employees
Year Founded: 2022

What We Do

Bliro is an AI Sales Assistant that automates administrative tasks for sales teams, including meeting preparation, CRM updates, and follow-ups, allowing them to focus on customers.

Similar Jobs

Compa Logo Compa

Vice President Of Finance

Artificial Intelligence • HR Tech • Software • Business Intelligence
Remote or Hybrid
Office, Machaze, Manica, MOZ
75 Employees
200K-250K Annually

Mondelēz International Logo Mondelēz International

Senior Product Manager

Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
Remote or Hybrid
3 Locations
90000 Employees

Mondelēz International Logo Mondelēz International

o9 Data & Integration Lead

Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
Remote or Hybrid
2 Locations
90000 Employees

Compa Logo Compa

Enterprise Account Executive

Artificial Intelligence • HR Tech • Software • Business Intelligence
Remote or Hybrid
3 Locations
75 Employees
200K-250K Annually

Similar Companies Hiring

Revel Thumbnail
Aerospace • Hardware • Robotics • Software
Marina Del Rey, California
60 Employees
Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
30 Employees
Vega Thumbnail
Artificial Intelligence • Automotive • Insurance • Transportation
US
43 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account