Search
Go
Data Scientist, Agent
Summary by AIEngineer.careers
Data Scientist, Agent is a LLM Evaluation Engineer role (full-time). with Lovable. in STOCKHOLM, SE. Imported listing (source: aihiringboard.com). Apply on the employer's site (aihiringboard.com).
Imported listing (source: aihiringboard.com) · Apply on aihiringboard.com
STOCKHOLM · SE
—
Full-time
MID
The sections below reproduce the third-party job description for reference. AIEngineer.careers does not write or control this text.
Imported job description
Sourced from aihiringboard.com
Lovable is seeking a Data Scientist to own the measurement and improvement of their AI agent. You will build evaluation systems, design experiments, and analyze telemetry to drive agent quality metrics like success rates and error reduction. This role bridges data science and product engineering, focusing on LLM observability and continuous feedback loops.
Responsibilities
- Define and own agent quality metrics (success, completion, error rates)
- Build eval systems and experiment frameworks for A/B testing agent changes
- Analyze agent traces and telemetry to provide concrete fixes to engineering teams
- Develop tooling for continuous evaluation as agents evolve
- Establish standards for judging agent behavior in ambiguous scenarios
Requirements
- Strong proficiency in SQL and Python
- Experience with applied statistics and experimental design (A/B testing)
- Interest or experience in LLM evaluation and observability tools
- Ability to work closely with engineers in a fast-paced startup environment
- Instinct for defining 'good' agent behavior without clean answer keys
Nice to Have
Lovable
Stockholm · SE · 539+ employees
Lovable is a Swedish technology company that provides an AI-powered platform for building full-stack web applications and websites through natural language prompts, a practice often referred to as 'vibe coding.' Founded in 2023, the platform enables users to generate production-grade software, including frontend, backend, and database components, without requiring deep coding expertise.
