Annotate and evaluate multilingual prompt-response data for language models, ensuring translation accuracy, fluency, and cultural context. Follow guidelines and collaborate with a team to maintain high-quality standards for AI safety.
Join the CrowdGen team as an Independent Contractor for Project Babel. We are seeking detail-oriented and skilled annotators to support the evaluation and improvement of multilingual prompt-response data used in the training and assessment of large language models (LLMs). Your role will directly contribute to ensuring translations are accurate, contextually appropriate, and aligned with AI safety and ethical standards.
Key Responsibilities:
- Evaluate Translations: Review English prompts and responses alongside their translated counterparts. Assess quality across multiple linguistic and contextual dimensions.
- Rate Quality: Use structured criteria (fluency, grammar, translation accuracy, relevance, and correctness) to score prompt and response translations.
- Apply Judgement: Identify issues in translations and suggest post-edits when ratings fall below threshold (≤2), ensuring that corrections maintain the original meaning, fluency, and cultural appropriateness.
- Adhere to Evaluation Protocol: Follow defined annotation guidelines and metric definitions. Ensure all work aligns with project standards for linguistic quality and AI safety goals.
- Collaborative Accuracy: Work individually or as part of a group of annotators (three per task for Priority 0 languages; one for others). Engage with adjudication rules (majority consensus or flagging) to resolve rating discrepancies.
- Structured Submission: Complete annotations using a provided CSV template, ensuring all required fields are accurately filled.
Qualifications:
- Language Proficiency: Native or near-native fluency in both English and Chinese.
- Language Aware: Strong understanding of grammar, syntax, and cultural nuance in both source and target languages.
- Attention to Detail: Ability to follow complex guidelines and apply consistent evaluation standards.
Experience:
- Prior work in translation, linguistic annotation, localization QA, or similar roles is highly desirable.
- Comfortable working with structured file formats (e.g., CSV), spreadsheets, and task management tools.
- Background in linguistics, computational linguistics, translation studies, or a related field highly desired.
- Familiarity with multilingual NLP tasks or LLMs.
- Ability to work independently and meet deadlines reliably.
This role is a project-based opportunity with CrowdGen, where you will join the CrowdGen Community as an Independent Contractor. If selected, you will receive an email from CrowdGen regarding the creation of an account using your application email address. You will need to log in to this account and reset the password, complete the setup requirements, and proceed with your application for this project-based role.
Why This Work Matters:
High-quality annotations are essential to building safer, more natural, and more inclusive AI systems. Your work ensures that multilingual outputs retain meaning, accuracy, and relevance across languages and cultural contexts—directly shaping how language models interact with a global user base.
Top Skills
Csv
Spreadsheets
Task Management Tools
Similar Jobs
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
As an Enterprise Account Executive, you will drive new business sales in a SaaS model, manage relationships with C-suite executives, and strategize account management for BFSI clients.
Top Skills:
AISaaS
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
The role involves generating new business sales revenue through account and territory planning, developing relationships with C-suite clients, and promoting customer success within a team environment.
Top Skills:
AISaaS
Software
The Expert Support Engineer troubleshoots and resolves complex technical issues for corporate customers while mentoring junior team members and collaborating with multiple internal departments.
Top Skills:
Business ApplicationsCiscoJuniperKvmLinux/UnixMicrosoft Hyper-VNetworkingVirtualizationVmware EsxiWindows Server
What you need to know about the Pune Tech Scene
Once a far-out concept, AI is now a tangible force reshaping industries and economies worldwide. While its adoption will automate some roles, AI has created more jobs than it has displaced, with an expected 97 million new roles to be created in the coming years. This is especially true in cities like Pune, which is emerging as a hub for companies eager to leverage this technology to develop solutions that simplify and improve lives in sectors such as education, healthcare, finance, e-commerce and more.
