AI Coding Evaluator

Posted One Month Ago
12 Locations
Remote
Entry level
Artificial Intelligence • Edtech • Machine Learning • Professional Services
The Role
Evaluate AI-generated customer service and technical responses for clarity, usefulness, natural language, technical accuracy, JSON validity, formatting, and guideline adherence. Provide clear feedback explaining evaluation decisions. The role requires multilingual fluency, coding knowledge, strong written communication, attention to detail, and consistent judgment. Experience with AI evaluation, quality assurance, APIs, structured data, customer service, software development, or technical writing is preferred.
Summary Generated by Built In

Help shape the future of AI by evaluating the technical accuracy, language quality, and usefulness of AI-generated responses.

  • Location: Remote

  • Type: Independent contractor / Part-time

  • Project duration: up to 3 months

About the role

At Anyone AI, we create high-quality training and evaluation data for frontier AI models.

We’re looking for multilingual experts with coding knowledge and a solid understanding of JSON to evaluate AI-generated technical content and customer service responses.

In this role, you’ll assess whether outputs are clear, helpful, natural, technically accurate, and aligned with project guidelines. Your evaluations will help improve both the customer experience and the technical reliability of AI systems.

What you’ll do
  • Review AI-generated customer service responses and technical content.

  • Evaluate responses for clarity, helpfulness, natural language, and overall quality.

  • Assess technical correctness using your coding knowledge.

  • Read, validate, and troubleshoot JSON, identifying structural and formatting issues.

  • Check outputs for formatting accuracy and adherence to project guidelines.

  • Provide clear, well-reasoned feedback explaining your evaluations.

What we’re looking for
  • Fluency in one or more of the supported language varieties listed below.

  • A solid understanding of JSON, including syntax, structure, validation, and troubleshooting.

  • Coding knowledge sufficient to assess technical content and identify errors.

  • Strong judgment when evaluating customer service responses.

  • Excellent attention to detail and strong written communication skills.

  • Ability to follow detailed evaluation guidelines consistently.

Supported languages/locales
  • Dutch — Netherlands

  • Welsh — United Kingdom

  • French — Canada

  • German — Germany

  • Italian — Italy

  • Portuguese — Brazil

  • Greek — Greece

Nice to have
  • Previous experience in AI evaluation, quality assurance, or technical support.

  • Experience working with APIs and structured data.

  • A background in customer service, software development, or technical writing.

Voice response instructions

When completing the voice response, speak naturally and spontaneously, as if responding in real time. Do not pre-plan, write, or script your answer—we are evaluating natural voice cadence.

Responses may be up to one minute long. A few sentences are sufficient if that feels natural.

Skills Required

  • Fluency in at least one supported language variety: Dutch, Welsh, Canadian French, German, Italian, Brazilian Portuguese, or Greek
  • Solid understanding of JSON syntax, structure, validation, and troubleshooting
  • Coding knowledge sufficient to assess technical content and identify errors
  • Strong judgment when evaluating customer service responses
  • Excellent attention to detail and written communication skills
  • Ability to consistently follow detailed evaluation guidelines
  • Previous experience in AI evaluation, quality assurance, or technical support
  • Experience working with APIs and structured data
  • Background in customer service, software development, or technical writing
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
Year Founded: 2022

What We Do

Anyone AI is an edtech startup dedicated to bridging the AI talent gap by investing in software developers from Latin America. The company provides intensive, hands-on training programs in Machine Learning and Artificial Intelligence, led by industry experts. By combining technical skill development with employability support, Anyone AI prepares professionals for global career opportunities, helping them transition into high-impact roles within the rapidly evolving AI and technology sectors.

Similar Jobs

Luxury Presence Logo Luxury Presence

Manager, Production Quality

Marketing Tech • Real Estate • Software • PropTech • SEO
Easy Apply
Remote or Hybrid
12 Locations
500 Employees

Luxury Presence Logo Luxury Presence

Tech Lead Manager, Mid Market & Celebrity - LATAM (Remote)

Marketing Tech • Real Estate • Software • PropTech • SEO
Easy Apply
Remote or Hybrid
12 Locations
500 Employees

Circle (circle.so) Logo Circle (circle.so)

Operations Coordinator

Artificial Intelligence • Consumer Web • Digital Media • Information Technology • Social Impact • Software
Remote
32 Locations
250 Employees

Circle (circle.so) Logo Circle (circle.so)

IT Coordinator

Artificial Intelligence • Consumer Web • Digital Media • Information Technology • Social Impact • Software
Remote
40 Locations
250 Employees

Similar Companies Hiring

Blee Thumbnail
Artificial Intelligence • Marketing Tech • Software
New York, New York
33 Employees
Vega Thumbnail
Artificial Intelligence • Automotive • Insurance • Transportation
US
65 Employees
VCA Animal Hospitals Thumbnail
Pet • Professional Services
Los Angeles, CA
35000 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account