Labeler / Annotator – AI Response Evaluation… at Blueprint · Remote

Summary: Blueprint Technologies is a technology solutions firm headquartered in Bellevue, Washington, and they are seeking a detail-oriented Labeler / Annotator to evaluate responses generated by AI systems in Korean. This role focuses on the evaluation and analysis of AI outputs, requiring strong judgment and attention to detail to improve AI communication in Korean. Responsibilities: - Perform side-by-side (SBS) comparisons of AI-generated responses - Evaluate outputs based on: - Accuracy - Relevance - Clarity - Instruction-following - Identify nuances in tone, meaning, and cultural context across Korean - Apply detailed, scenario-specific annotation guidelines - Maintain consistency and high-quality evaluations Required Qualifications: - Native or professional fluency in Korean - Strong English reading comprehension (required for guidelines) - Prior experience with side-by-side (SBS) labeling or annotation - Excellent analytical thinking and attention to detail - Ability to quickly understand and apply structured evaluation frameworks Preferred Qualifications: - Background in linguistics, translation, or localization - Experience in AI evaluation - Experience in data annotation - Experience in search relevance or content quality analysis Required Skills: Side-by-side labeling, Annotation, Structured evaluation frameworks, Analytical thinking, Korean language fluency, English reading comprehension Benefits: Statutory benefits (e.g., paid vacation and aguinaldo / Sueldo Anual Complementario (SAC)), Social security contributions and healthcare coverage (Obra Social), Additional benefits aligned with local market practices