Japanese Culture Expert – Search Quality Evaluation

About this task

Responsibilities:
- Identify realistic search-related failures where AI responses are inadequate by creating and testing search queries.
- Document issues and write instructions and evaluation criteria for correct answers.
- Assess local relevance, source quality, outdated information, language mismatches, incomplete answers, and region-specific applicability.
- Research and verify information across websites, databases, and documentation; iterate through tests as needed.

Requirements:
- Deep familiarity with Japanese culture, local practices, and context.
- Strong research, source verification, critical thinking, and attention-to-detail skills.
- Ability to identify contradictions, outdated information, and locale-specific issues.
- Clear writing skills for explaining evaluation logic and criteria.
- Freelance contract; approximately 20 hours per week. Ongoing project, starting ASAP.