This description is a summary of our understanding of the job description. Click on 'Apply' button to find out more.

Role Description

This role supports collaborations with leading AI teams focused on improving how conversational AI models respond to real-world user questions across a wide range of topics.

• Evaluate AI-generated responses to determine how effectively they answer user queries
• Conduct fact-checking using trusted public sources and external tools
• Annotate responses by identifying strengths, weaknesses, and factual inaccuracies
• Assess reasoning quality, tone, clarity, and completeness of responses
• Ensure model responses align with expected conversational behavior and system guidelines
• Apply consistent evaluation standards using structured taxonomies and benchmarks

Qualifications

• Bachelor's degree or higher
• Native or C2-level proficiency in Chinese (Mandarin)
• Fluent English
• Experience working with large language models (LLMs)
• Excellent writing and feedback articulation skills
• Strong analytical and critical thinking ability
• High attention to detail and ability to detect subtle reasoning issues
• Comfort working across diverse topics and domains

Requirements

• Experience with RLHF, model evaluation, or AI data annotation
• Experience producing high-quality written content
• Experience comparing multiple outputs and making qualitative judgments
• Familiarity with evaluation rubrics or structured scoring frameworks

Benefits

• Contribute directly to improving how AI systems interact with real users
• Work with cutting-edge AI labs and research teams
• Flexible remote work with competitive compensation
• Play a direct role in shaping next-generation AI systems

Contract Details

• Independent contractor role
• Fully remote with flexible scheduling
• Compensation: $36.16/hour
• Weekly payments via Stripe or Wise
• Projects may extend depending on performance and project needs

Location Requirements

• Taiwan
• Malaysia
• United States