Search
Go
English-Speaking AI Data Evaluation Specialist | Large Language Models
Summary by AIEngineer.careers
English-Speaking AI Data Evaluation Specialist | Large Language Models is a LLM Evaluation Engineer role (full-time). with Workster Jobs. in LISBOA, PT. Imported listing (source: worksterjobs.teamtailor.com). Apply on the employer's site (worksterjobs.teamtailor.com).
Imported listing (source: worksterjobs.teamtailor.com) · Apply on worksterjobs.teamtailor.com
LISBOA · PT
—
Full-time
MID
The sections below reproduce the third-party job description for reference. AIEngineer.careers does not write or control this text.
Imported job description
Sourced from worksterjobs.teamtailor.com
This role focuses on evaluating and improving Large Language Model outputs by reviewing AI-generated responses for quality, safety, and factual accuracy. You will apply detailed annotation guidelines to label model outputs, identify edge cases, and provide feedback that directly contributes to training next-generation generative AI systems. The position offers hands-on experience with cutting-edge LLM projects in a collaborative office environment in Lisbon.
Responsibilities
- Review and evaluate AI-generated responses against quality and safety guidelines
- Label and annotate language model outputs with high accuracy and consistency
- Identify inconsistencies, edge cases, and potential model issues for escalation
- Analyze structured datasets and AI content to support ongoing model improvement
- Maintain strict confidentiality and adhere to project security standards
Requirements
- Native-level fluency in English (B2 or above)
- Strong understanding of Generative AI, LLMs, and AI evaluation methodologies
- Experience in AI data annotation, RLHF, SFT, or similar analytical roles
- Excellent analytical thinking, attention to detail, and reading comprehension
- Ability to interpret complex guidelines and work efficiently in a fast-paced environment
Workster Jobs
Lisboa · PT · 24+ employees
Workster is an international HR consultancy and recruitment agency that connects candidates with opportunities across Europe. It focuses on matching international companies with talent across various sectors including IT, customer support, and banking.
