Legal AI Annotator

Posted 5 Days Ago
Be an Early Applicant
Milan, ITA
Hybrid
20-20 Hourly
Entry level
Artificial Intelligence • Legal Tech • Software
The Role
Evaluate AI-generated legal answers, define accuracy and quality criteria, build and review legal evaluation benchmarks, and compare responses from different AI models. The role requires independent remote work, strong legal judgment, excellent Italian, admission to the Italian Bar, and an active Italian VAT number. Familiarity with legal technology, generative AI, or AI evaluation methods is advantageous but not required.
Summary Generated by Built In

At Lexroom, we set the bar for legal AI – clear, fast, and built for trust.

In just a few years, what began as an idea has become a reliable platform adopted by thousands of legal professionals across Europe.

Our mission is clear: make the law faster, clearer, and more reliable, giving lawyers, law firms, and companies the time and confidence to focus on what truly matters. We're scaling rapidly, backed by top-tier investors (having recently closed our Series B) and a world-class team dedicated to rigor, clarity, and impact.

Today, Lexroom is a trusted partner, shaping the future of legal AI with discipline and purpose.

Why Lexroom?

Shape the Standard: Be part of the team setting the benchmark for legal AI in Europe.

Work with the Best: Collaborate with exceptional colleagues in a culture that values competence, curiosity, and ambition.

Deliver Meaningful Results: Your legal judgment will directly shape the quality of answers thousands of professionals rely on every day.

At Lexroom, we leave buzzwords behind. Here, you'll build something that lasts: a trusted platform, a strong reputation, a new standard for the legal world.

The Role & Why Now

The quality of our legal answers is the core of Lexroom's services. To measure and continuously improve it, we're looking for lawyers who can put their expertise to work evaluating AI systems.

You'll work independently and remotely on dedicated platforms, following Lexroom's guidelines, in direct contact with our People team.

Key Responsibilities

The scope of these activities may expand or change over time.

Legal AI Benchmarks: You'll evaluate AI-generated answers to legal questions. You'll also define the positive and negative criteria used to judge whether an answer is correct. Finally, you'll build and review the evaluation sets we use to measure the quality of Lexroom's services.

Arena (Comparative Evaluation): You'll compare alternative answers from different models to the same legal question. For each comparison, you'll pick the preferred answer and explain your choice against predefined criteria: legal accuracy, completeness, use of sources, and clarity. Your evaluations directly guide product development.

Requirements

Bar Admission: You must be a qualified lawyer, already admitted to the Italian Bar (iscrizione all'Albo).

VAT Number: You need an active Italian VAT number (Partita IVA), as the collaboration is invoiced.

Legal Expertise: You should have solid experience in at least one area of law, to be specified in your CV. Examples include civil, criminal, administrative, banking & insurance, tax, corporate, employment, family, consumer, and insolvency law.

Language: Excellent command of Italian is required.

Nice to Have

These are a plus, but not required:

  • familiarity with legal tech or generative AI tools in legal work;

  • experience evaluating AI systems, such as benchmarks, A/B tests, or model comparisons.

Contract & Compensation

Contract: Service agreement as an independent contractor with a VAT number.

Compensation: €20/hour, invoiced monthly based on the hours actually worked and reported.

Work Setup: Fully remote, with one in-person meeting per month for alignment and goal-setting.

Hiring Process

The process is a single 30-minute introductory interview.

At Lexroom, ambition meets discipline and purpose. We're not just building a product—we're setting a new standard for the legal world. If you're ready to contribute to something lasting and meaningful, we'd love to hear from you.

Skills Required

  • Qualified lawyer admitted to the Italian Bar (iscrizione all'Albo)
  • Active Italian VAT number (Partita IVA)
  • Solid legal experience in at least one specified area of law
  • Excellent command of Italian
  • Familiarity with legal technology or generative AI tools in legal work
  • Experience evaluating AI systems, including benchmarks, A/B tests, or model comparisons
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Milan, Lombardy
80 Employees

What We Do

Legal professionals spend up to 70% of their time on repetitive, low-value tasks. Lexroom provides them with the superpowers to change that. Our AI-powered platform enables lawyers to transform hours of time-consuming legal research into seconds, generating high-quality drafts and practical answers backed by trusted, institutional legal sources. Designed in collaboration with top-tier law firms (GOP, Ichino Brugnatelli, GPBL), Lexroom.ai seamlessly integrates into the daily workflow of lawyers, helping them focus on strategic, value-adding activities. Available today for Civil Law, Commercial Law, Privacy, Banking, Labour Law, Tax, Public, Criminal, Family, IP Law and more. Lexroom is redefining the way lawyers work—one regulation at a time.

Similar Jobs

Pfizer Logo Pfizer

Principal Engineer

Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Remote or Hybrid
31 Locations
121990 Employees
177K-294K Annually

Circle (circle.so) Logo Circle (circle.so)

Lead Engineer, AI Platform

Artificial Intelligence • Consumer Web • Digital Media • Information Technology • Social Impact • Software
In-Office or Remote
43 Locations
250 Employees

Pfizer Logo Pfizer

Medical Information Specialist - Italian Speaker (Secondment / Temporary 6M)

Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
In-Office
3 Locations
121990 Employees

Legora Logo Legora

Engagement Manager

Artificial Intelligence • Legal Tech • Software
In-Office
Milan, ITA
900 Employees

Similar Companies Hiring

Revel Thumbnail
Aerospace • Hardware • Robotics • Software
Marina Del Rey, California
70 Employees
Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
30 Employees
Vega Thumbnail
Artificial Intelligence • Automotive • Insurance • Transportation
US
65 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account