Role Overview:
As a Personal Account Rater, you will participate in a personalization evaluation project focused on assessing the quality of AI-generated responses using your primary personal Gmail account (@gmail.com). You will evaluate how effectively the system leverages your email history to deliver relevant and personalized responses across English and your native language. This role requires strong analytical skills, attention to detail, and the ability to evaluate prompts and responses in English.
Key Qualifications
Language Proficiency: Native speaker of one of the following languages:German (DE), Korean (KO)
Location: Must currently reside outside the United States.
Personal Account: Must have an active personal Gmail account (@gmail.com) that you use as your primary email account.
Mixed-Language Inbox: Your inbox must contain a meaningful mix of English and your target language, with each language representing at least 30% of your total inbox content. (Mandatory requirement)
Gmail Language: Gmail display language may be set to either English or your target language.
English Proficiency: Must be comfortable reading, understanding, and evaluating prompts and AI-generated responses in English.
Email History: Must have a well-established inbox with approximately 1,000+ emails and regular account activity.
Consent: Willing to use your primary personal Gmail account for the evaluation and complete the required consent process.
Exceptional Analytical Thinking: Ability to assess AI responses for relevance, personalization quality, and overall helpfulness.
Attention to Detail: Strong ability to identify subtle issues and inconsistencies in AI-generated responses.
Communication: Excellent written communication skills for providing clear and structured evaluation feedback.
Independence: Self-motivated and able to work independently in a remote environment.
Technical Setup: Desktop or laptop with a reliable internet connection.
Description:
In this role, you will be responsible for evaluating the quality of personalized AI interactions using your personal Gmail account. Your day-to-day responsibilities will include:
Evaluating AI-generated responses based on your personal email history and account activity.
Assessing whether responses are relevant, accurate, helpful, and appropriately personalized.
Reviewing prompts and AI responses in English while considering personalization across English and your native language.
Identifying incorrect personalization, unsupported assumptions, or irrelevant recommendations.
Comparing multiple AI responses to determine which provides the better overall user experience.
Providing clear, detailed, and objective evaluation feedback in accordance with project guidelines.
Maintaining data privacy and following all project confidentiality requirements throughout the evaluation process.
Education & Experience
Bachelor's degree or equivalent practical experience in any field.
Experience in AI evaluation, data annotation, content review, quality assurance, or a related analytical role is preferred but not required.
Consent:
Must be willing to use their primary personal account and sign the Required Consent form. https://docs.google.com/document/d/1DJqKicTlfjhLFFKXEW_rEuflBZlXZPG2pdutU6AQdsI/edit?tab=t.0
Availability:
Full-time, at least 4 hours per day and up to 40 hours per week with 4 hours of overlap with PST.
Location:
Greenlit (except USA)
Contract duration:
16 weeks
Skills Required
- Native speaker of German
- Must currently reside outside the United States (Greenlit except USA)
- Active personal Gmail account (@gmail.com) used as primary email
- Inbox with a meaningful mix of English and target language (each >= 30% of inbox)
- Gmail display language may be English or target language
- Comfortable reading, understanding, and evaluating prompts and AI-generated responses in English
- Well-established inbox with approximately 1,000+ emails and regular account activity
- Willing to use primary personal Gmail account for evaluation and sign required consent form
- Desktop or laptop with reliable internet connection
- Availability: full-time, at least 4 hours per day and up to 40 hours per week with 4 hours overlap with PST
- Bachelor's degree or equivalent practical experience
- Experience in AI evaluation, data annotation, content review, or QA
- Strong analytical thinking, attention to detail, clear written communication, and ability to work independently
What We Do
Careerflow.ai is an AI-powered career management platform and 'career copilot' dedicated to helping job seekers land their dream jobs. The company provides a comprehensive end-to-end toolkit featuring an AI resume builder, LinkedIn profile optimizer, and job tracking tools. By streamlining the application process and optimizing professional profiles, Careerflow helps users navigate the competitive job market and get hired at top tech and startup companies faster.
