Applied Legal Benchmark Specialist

Posted One Month Ago
Be an Early Applicant
Hiring Remotely in United States
Remote
83-105 Hourly
Senior level
Artificial Intelligence • HR Tech • Professional Services • Software
The Role
Author and verify rigorous multiple-choice legal assessment items for AI benchmarks across multiple legal specialties. Create clear, self-contained questions with one correct answer and nine plausible distractors, classify difficulty, provide structured solution explanations and references, and review/edit questions for legal accuracy and clarity. Work remotely, asynchronously, as an independent contractor, committing 10+ hours weekly.
Summary Generated by Built In

This role is for one of our clients

Compensation: $83-$105 per hour

We are seeking experienced legal professionals to author and review high-quality academic assessment content for an AI research initiative. In this role, you will contribute your legal expertise to the development of rigorous evaluation benchmarks designed to assess how effectively AI systems understand and reason about complex legal concepts.

You may contribute through one of two task types:

Question Authoring: Create original, challenging multiple-choice questions within your area of legal expertise, assess their difficulty, and submit them for quality review.

Question Verification: Review existing questions for legal accuracy, clarity, completeness, and rigor. Make necessary edits, evaluate difficulty, and document the reasoning behind any changes.


RequirementsLaw Domains Covered

Opportunities span a broad range of legal specialties, including:

  • Intellectual Property, Privacy & Technology
  • Regulatory & Government Affairs
  • Securities & Capital Markets
  • Financial Regulation & Compliance
  • Private Equity
  • M&A & Transaction Structuring
  • Antitrust, Merger Control & Competition
  • Healthcare, Life Sciences & Pharmaceuticals
  • Environmental, Energy & ESG/Climate Law
Key Responsibilities
  • Develop original legal questions that assess deep conceptual understanding, analytical ability, and legal reasoning rather than surface-level recall.
  • Ensure every question is clear, unambiguous, self-contained, and sufficiently detailed to support a definitive answer.
  • Classify questions by difficulty level: Medium (introductory undergraduate), Hard (advanced undergraduate), or Expert (postgraduate level and above).
  • Provide one correct answer along with nine plausible but subtly incorrect alternatives designed to challenge advanced legal reasoning.
  • Develop concise, logically structured solution explanations that demonstrate the appropriate legal reasoning process.
  • Provide 1–5 credible academic or professional references for each question, drawing from reputable legal publications, academic sources, case law, statutes, and legal repositories.
  • For verification assignments, identify issues involving legal accuracy, clarity, completeness, precision, or solvability and clearly explain any recommended revisions.
  • Apply consistent quality standards across questions and contribute to the development of reliable, gold-standard legal evaluation benchmarks.
Ideal Qualifications
  • JD, LLM, SJD, or doctoral-level qualification in Law or a closely related discipline.
  • Master's-level candidates may be considered where they demonstrate exceptional expertise in a specialized legal domain.
  • Strong command of legal reasoning, statutory and regulatory interpretation, case analysis, and legal research.
  • Deep expertise in at least one of the covered practice areas.
  • Experience with legal education, bar examination content development, academic assessment, judicial clerkships, legal research, or legal writing is highly valued.
  • Excellent written English with the ability to communicate complex legal concepts clearly, precisely, and concisely.
  • Strong attention to detail and the ability to identify subtle legal inaccuracies or ambiguities.
  • Prior experience with AI evaluation, training data, benchmarking, or human-in-the-loop research is a plus.
Opportunity Details
  • Commitment: 10+ hours per week
  • Work Arrangement: Fully remote
  • Work Style: Asynchronous and flexible
  • Engagement: Project-based opportunities with potential for continued participation based on performance and project requirements
Application Process

Interested candidates can submit a resume or relevant professional background highlighting their legal specialization and experience. Selected applicants may be asked to complete a short sample assessment or provide additional information about their subject-matter expertise.

Equal Opportunity

We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.

Contract & Payment Terms
  • You will be engaged as an independent contractor.
  • The role is fully remote and can be completed on a flexible schedule.
  • Projects may be extended, shortened, or concluded based on project requirements and performance.
  • Work will not require access to confidential or proprietary information belonging to any current or former employer, client, or institution.
  • Payments are made weekly based on services rendered through available payment methods.
  • H-1B and STEM OPT candidates are not eligible for this opportunity at this time.

Skills Required

  • JD, LLM, SJD, or doctoral-level qualification in Law or a closely related discipline
  • Master's-level candidates may be considered with exceptional specialized expertise
  • Strong command of legal reasoning, statutory and regulatory interpretation, case analysis, and legal research
  • Deep expertise in at least one covered practice area (e.g., IP, Privacy, Securities, M&A, Antitrust, Healthcare, Environmental, Financial Regulation)
  • Experience with legal education, bar examination content development, academic assessment, judicial clerkships, legal research, or legal writing
  • Excellent written English with ability to communicate complex legal concepts clearly and concisely
  • Strong attention to detail and ability to identify subtle legal inaccuracies or ambiguities
  • Prior experience with AI evaluation, training data, benchmarking, or human-in-the-loop research
  • Ability to commit 10+ hours per week and work fully remotely on an asynchronous, flexible schedule
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
Year Founded: 2021

What We Do

Weekday is an AI-powered recruitment platform that helps startups hire top-tier engineering and product talent. By leveraging a massive database of white-collar professionals and advanced outreach tools, the company streamlines the hiring process through automated sourcing, AI-driven resume screening, and white-glove contingency services. Their mission is to modernize recruitment by enabling companies to discover and engage passive candidates efficiently, ensuring high-quality hires for critical roles.

Similar Jobs

Capital One Logo Capital One

Artificial Intelligence Engineer

Fintech • Machine Learning • Payments • Software • Financial Services
Remote or Hybrid
5 Locations
55000 Employees
286K-392K Annually

Optum Logo Optum

Director, Executive and Enterprise Storytelling - Remote

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office or Remote
Eden Prairie, MN, USA
160000 Employees
135K-231K Annually

Optum Logo Optum

LVN or LPN, Case Manager - Remote

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
In-Office or Remote
Cypress, CA, USA
160000 Employees
20-36 Hourly

Optum Logo Optum

Account Manager

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Remote
Pennsylvania, USA
160000 Employees
20-36 Hourly

Similar Companies Hiring

Revel Thumbnail
Aerospace • Hardware • Robotics • Software
Marina Del Rey, California
60 Employees
Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
30 Employees
Vega Thumbnail
Artificial Intelligence • Automotive • Insurance • Transportation
US
43 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account