Search
Go
Senior Software Engineer - Testing AI Coding Agents
REMOTE · WORLDWIDE
$106K+
Part-time
SENIOR
About the Role
Mindrift connects specialists with project-based AI opportunities. This role involves creating challenging tasks and evaluation criteria to test AI coding agents. You will build realistic developer environments, design tasks, write tests, and iterate based on feedback.
Responsibilities
- Build realistic developer environments with codebase, infrastructure, and context
- Design tasks from intermediate states and define evaluation criteria
- Write tests that verify agent solutions and accept all valid approaches
- Iterate on tasks based on QA feedback
Requirements
- 5+ years in software development
- Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis
- Experience writing tests (functional, integration)
- English proficiency B2+
Tech Stack
PythonReactTypeScriptFastAPIPostgreSQLRedisDockerJavaScript
Mindrift
Amsterdam · NL · 1038+ employees
Mindrift is an expert-sourcing platform built by Toloka, a company that provides data services for AI. It connects domain experts with AI training projects to help develop safer, more accurate, and more helpful generative AI models.
