Labeler / Annotator – AI Response Evaluation… at Blueprint · Remote

Summary: Blueprint Technologies is a technology solutions firm headquartered in Bellevue, Washington, focused on solving complex problems through innovative technology. They are seeking a detail-oriented Labeler / Annotator to evaluate AI-generated responses in German, playing a key role in improving AI communication and understanding. Responsibilities: - Perform side-by-side (SBS) comparisons of AI-generated responses - Evaluate outputs based on: - Accuracy - Relevance - Clarity - Instruction-following - Identify nuances in tone, meaning, and cultural context across German - Apply detailed, scenario-specific annotation guidelines - Maintain consistency and high-quality evaluations Required Qualifications: - Native or professional fluency in German - Strong English reading comprehension (required for guidelines) - Prior experience with side-by-side (SBS) labeling or annotation - Excellent analytical thinking and attention to detail - Ability to quickly understand and apply structured evaluation frameworks Preferred Qualifications: - Background in linguistics, translation, or localization - Experience in AI evaluation - Experience in data annotation - Experience in search relevance or content quality analysis Required Skills: Side-by-Side Labeling, Annotation, Structured Evaluation Frameworks, Analytical Thinking, German Language Fluency, English Reading Comprehension Benefits: Statutory benefits (e.g., paid vacation and aguinaldo / Sueldo Anual Complementario (SAC)), Social security contributions and healthcare coverage (Obra Social), Additional benefits aligned with local market practices