Search
Go
AI Benchmark Engineer | Native Language Specialist - German (Switzerland) - Remote
Summary by AIEngineer.careers
AI Benchmark Engineer | Native Language Specialist - German (Switzerland) - Remote is a LLM Evaluation Engineer role (contract). with LILT (Production). in REMOTE, WORLDWIDE. Imported listing (source: arbeitnow.ch). Apply on the employer's site (arbeitnow.ch).
Imported listing (source: arbeitnow.ch) · Apply on arbeitnow.ch
REMOTE · WORLDWIDE
—
Contract
MID
The sections below reproduce the third-party job description for reference. AIEngineer.careers does not write or control this text.
Imported job description
Sourced from arbeitnow.ch
LILT is seeking a native German-speaking software engineer to design and validate rigorous benchmarks for multilingual large language models. The role focuses on creating high-signal tasks that test model robustness in non-English terminal workflows, encoding edge cases, and locale-specific conventions. You will build reference implementations and verifier scripts to evaluate coding agents against various model tiers.
Responsibilities
- Design and create realistic task environments using datasets and files in native German.
- Develop deterministic verifier scripts and reference implementations for benchmark tasks.
- Calibrate task difficulty and analyze execution logs across different model tiers.
- Participate in multi-layer quality control processes to ensure benchmark integrity.
Requirements
- 1+ years of industry experience in software or prompt engineering.
- Native or near-native fluency in German with deep understanding of grammar and phrasing.
- Strong proficiency in Python, shell scripting, and data processing.
- Experience with Terminal/CLI-based development workflows and coding agents.
- High English proficiency.
Nice to Have
LILT (Production)
Palo Alto · US · 782+ employees
LILT is an enterprise AI translation and multilingual-content platform founded in 2015 by researchers associated with Google Translate. It combines artificial intelligence, translation workflows, and human expertise to help businesses and governments translate content and create multilingual datasets securely at scale.
