Expert Rater

Posted Yesterday
Be an Early Applicant
17 Locations
In-Office or Remote
Junior
Machine Learning • Natural Language Processing
The Role
Rate and evaluate online advertisements against client guidelines for quality, relevance, and compliance. Identify ambiguities, refine rating standards, curate benchmark datasets for AI training, analyze rating trends, collaborate with quality and AI teams, and mentor new raters. The role requires strong analytical judgment, research skills, digital advertising knowledge, target-language fluency, excellent English, and experience in AI relevance rating, data labeling, linguistics, marketing, content, QA, or related fields.
Summary Generated by Built In

If you have a Candidate Login already, but have forgotten your password please use the steps to reset your password. If you have forgotten your email login, please contact [email protected] subject Workday Candidate Login

When creating your Workday account and entering personal information like name, address, please do not use ALL CAPS.

Thank you!

NOTICE:  For Privacy Policy please review here

Job Responsibilities:

Welo Data is an award-winning localization and AI data company. We run some of the world’s largest AI model relevance rating programs. This will play a crucial role in analyzing and improving AI-driven online advertising platforms.

This role will help establish benchmarks and guidelines for data used to train AI models, particularly in areas lacking defined standards. That means you will be shaping the future of web-based advertising and AI models.

MAIN DUTIES

The following is a non-exhaustive list of responsibilities and areas of ownership of an Expert Rater

  • Rate Client-provided ads following pre-defined guidelines
  • Evaluate advertisements based on specific quality, relevance, and compliance criteria provided by the client.
  • Ensure ratings are objective, consistent, and aligned with the predefined rating standards.
  • Identify any guideline ambiguities and apply critical thinking when assessing borderline cases.
  • Maintain a high level of accuracy and efficiency, meeting daily/weekly rating quotas.
  • Review and refine rating guidelines, identifying areas for improvement and recommending updates to enhance accuracy, consistency, and overall rating quality.
  • Curate high-quality datasets ("golden sets") by selecting and validating top-tier rated ads, which can serve as benchmarks for AI model training and human rater calibration.
  • Collaborate with cross-functional teams (e.g., quality assurance, data analysts, and AI training teams) to ensure rating processes align with evolving industry and client needs.
  • Analyse trends and patterns in ad ratings, providing insights that contribute to guideline optimizations and overall rating effectiveness.
  • Assist in training and mentoring new raters, ensuring they understand guidelines and best practices for accurate evaluations.

REQUIREMENTS

Experience

  • Have experience in AI model relevance, such as data labeling, linguistic annotation, relevance rating, etc.
  • Are analytical and data-driven with a proven track-record of data-based decision making  
  • Have a strong interest and understanding of digital marketing and advertising, especially as it relates to artificial intelligence  
  • Can effectively summarize complex information, questions, and solutions for various audiences Have experience researching and gathering insights, and understanding complex problems across a variety of topics  
  • Can defining, explain, and defend their points of view in group discussions 
  • Are comfortable reviewing and discussing a variety of topics – you do not need to have expertise on the topics, but willingness to research and understand to support consensus decisions is vital 
  • Are comfortable with ambiguity in guidelines, workflows, and processes, and are motivated to help establish those systems 

Requirements

  • Native level fluency in the target language
  • Excellent understanding of both written & spoken English
  • A 4 year university degree and/or or at least 2 years work experience in one or more of the following: AI-based relevance rating, marketing/advertising, linguistics, content creation, writing, tech-related quality assurance, data labeling, or linguistic annotation.

Additional Job Details:

Skills Required

  • Native-level fluency in the target language
  • Excellent understanding of written and spoken English
  • A four-year university degree and/or at least two years of work experience
  • Experience in AI-based relevance rating, marketing or advertising, linguistics, content creation, writing, tech-related quality assurance, data labeling, or linguistic annotation
  • Experience with AI model relevance, data labeling, linguistic annotation, or relevance rating
  • Analytical and data-driven decision-making skills
  • Strong interest in and understanding of digital marketing and advertising, particularly in relation to artificial intelligence
  • Ability to summarize complex information, questions, and solutions for varied audiences
  • Experience researching and gathering insights across varied topics
  • Ability to define, explain, and defend viewpoints in group discussions
  • Comfort reviewing and discussing diverse topics and researching unfamiliar subjects
  • Comfort working with ambiguity in guidelines, workflows, and processes

WeLocalize Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about WeLocalize and has not been reviewed or approved by WeLocalize.

  • Wellbeing & Lifestyle Benefits — Flexible, fully remote scheduling is often highlighted as making earnings more workable for supplemental income. Feedback suggests that the ability to choose hours and work from home offsets some concerns about lower or variable pay.

WeLocalize Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: New York, NY
2,331 Employees
Year Founded: 1997

What We Do

Welocalize accelerates the global business journey by enabling brands and companies to reach, engage, and grow international audiences. Welocalize delivers multilingual content transformation services in translation, localization, and adaptation for over 250 languages with a growing network of over 250,000 in-country linguistic resources. Driving innovation in language services, Welocalize delivers high-quality training data solutions for NLP-enabled machine learning by blending technology and human intelligence to collect, annotate, and evaluate all content types. Our people work across offices in North America, Europe, and Asia serving our global clients in the markets that matter to them. • Global team of 2,100+ • Offices in North America, Europe and Asia • Quality Certifications: ISO 9001:2015, ISO/IEC 27001:2013, ISO 17100:2015, ISO 13485:2016, ISO 18587:2017 • Accredited professional translators and interpreters for 250+ languages www.welocalize.com

Similar Jobs

Micron Technology Logo Micron Technology

Chief Technology Officer

Artificial Intelligence • Hardware • Information Technology • Machine Learning
Remote
Batu Kawan, Selatan, Pulau Pinang, MYS
45000 Employees

Micron Technology Logo Micron Technology

Process Engineer

Artificial Intelligence • Hardware • Information Technology • Machine Learning
Remote
Batu Kawan, Selatan, Pulau Pinang, MYS
45000 Employees

Micron Technology Logo Micron Technology

Accountant

Artificial Intelligence • Hardware • Information Technology • Machine Learning
Remote
Batu Kawan, Selatan, Pulau Pinang, MYS
45000 Employees

Mondelēz International Logo Mondelēz International

Analyst, Analytics - 6 months contract

Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
Remote or Hybrid
4 Locations
90000 Employees

Similar Companies Hiring

Learneo Thumbnail
Software • Machine Learning • Edtech • Artificial Intelligence
NL
397 Employees
Allen Control Systems Thumbnail
Artificial Intelligence • Computer Vision • Machine Learning • Robotics • Defense • Manufacturing
Austin , TX
450 Employees
Blissway Thumbnail
Computer Vision • Fintech • Hardware • Internet of Things • Machine Learning • Software • Transportation
Denver, CO
24 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account