Generative AI Data Analyst - USA (Remote)

Key Responsibilities

• Creatively writing prompts and responses to a variety of diverse topics
• Perform LLM annotation and evaluation tasks (ranking, scoring, labeling, tagging)
• Evaluate model outputs for accuracy, relevance, and instruction-following
• Identify and document issues such as hallucinations and inconsistencies
• Participating in and/or supporting labeling workflows, including hands-on annotation and collaboration with internal or external teams
• Training teams on best practices for creating Large Language Models/Data sets

Requirements

• Hands-on experience performing data annotation or evaluation tasks (e.g., labeling, ranking, scoring, or tagging LLM outputs)
• Native or near-native English with excellent writing skills
• Strong attention to detail and ability to follow guidelines consistently
• Self-driven, motivated and enthusiastic to work on state-of-art machine learning tools
• 4 year Accredited College degree or equivalent experience

Ways to stand out from the crowd:

• College Degree or experience in Linguistics, English Literature, Creative Writing, Journalism, and domain knowledge (Law/Medical/Math/Coding/etc.)
• Experience working in annotation platforms or structured labeling environments is a plus
• Deep understanding of Large Language Models/RLHF
• Experience in labeling/tagging of frames/tasks/prompts to prepare for DNN
• QA/testing experience