AI Research engineer at Cerebro (Closed)

AI Research engineer

SAN FRANCISCO · US

$250K–$280K

Full-time

About the Role

AI Researcher – Data, Evaluation & Alignment at Cerebro. You will work on datasets, evaluation systems, and feedback loops that improve frontier language models. This role involves designing novel datasets, developing evaluation frameworks, refining reward signals for RLHF, and collaborating with leading AI research teams.

Responsibilities

Design novel datasets that reveal meaningful model failure modes across domains like reasoning, coding, finance, and enterprise workflows.
Develop evaluation frameworks that go beyond static benchmarks and provide actionable insights into model capabilities.
Create and refine reward signals, rubrics, and measurement systems for RLHF, RLAIF, and related post-training methods.
Run experiments to understand how data quality, structure, and selection influence model behaviour.
Build quantitative approaches for measuring dataset quality, diversity, robustness, and downstream impact.
Investigate annotator behaviour and human-feedback dynamics to improve training signal quality.
Partner closely with frontier AI research teams to translate high-level research goals into concrete datasets and evaluation protocols.

Requirements

Deep interest in understanding how data drives model behaviour.
Familiarity with LLM training, RLHF, reward modelling, evaluation methodologies, or alignment research.

Cerebro

Santa Monica · US · 10+ employees

Cerebro is a technology company offering project management and collaborative software designed for creative industries such as marketing, VFX, animation, and architecture. It provides tools for task management, visual content collaboration, and project progress tracking.