The Role
Evaluate AI search responses for realistic failures involving relevance, source quality, freshness, language, completeness, and regional applicability. Develop search queries, test multiple scenarios, document model shortcomings, and write clear instructions and evaluation criteria. The role requires Chinese language expertise, strong research and source-verification skills, critical thinking, attention to detail, problem-solving, clear writing, and deep familiarity with local Australian cultural and regulatory context.
Summary Generated by Built In
Overview
Are you passionate about languages, writing, and quality evaluation? We are launching an exciting language evaluation project and are looking for Chinese-speaking experts to join our team.
What you will do:
Identify realistic search-related failures where an AI model provides inadequate responses. Create search queries, test them, document what went wrong, and write instructions and evaluation criteria for correct answers.
Common issues: local relevance, source quality, outdated information, language mismatches, incomplete answers, and region-specific inapplicability. Testing multiple queries before finding failures is expected.
Project Details:
- Start Date: ASAP
- Duration: Ongoing Project.
- Commitment: 20 hours per week.
- Job Type: Freelance contract.
- Location: Remote, Australia
- Rate: $40 per hour.
Requirements:
- Research skills — ability to find and verify information from multiple sources; comfort navigating different websites, databases, and documentation
- Critical thinking — Can spot contradictions between sources, identify outdated information, and recognize when something doesn't apply to a specific locale/context
- Attention to detail — Can catch nuances in language, local variations (prices, regulations, requirements), and edge cases
- Problem-solving mindset — Comfortable iterating and testing when things don't work; doesn't get frustrated by trial- and-error
- Domain knowledge — Ideally someone with background in research, quality assurance, compliance, or similar work that requires precision and source verification
- Writing clarity — Can articulate instructions and rubrics clearly so others understand the logic behind their reasoning
Locale knowledge — Deep familiarity with their specific locale (local regulations, business practices, cultural nuances) is valuable for spotting region-specific failures
Skills Required
- Chinese-speaking expertise
- Ability to find and verify information from multiple sources
- Comfort navigating websites, databases, and documentation
- Critical thinking and ability to identify contradictions, outdated information, and locale-specific limitations
- Strong attention to detail, including language nuances, local variations, and edge cases
- Problem-solving mindset and comfort with iterative testing and trial-and-error
- Background in research, quality assurance, compliance, or similarly precision-focused work
- Ability to write clear instructions and evaluation rubrics
- Deep familiarity with the relevant locale, including regulations, business practices, and cultural nuances
- Availability for 20 hours per week
- Location in Australia
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
