Research Engineer (Evals)

White Circle is seeking a Research Engineer to build and maintain internal benchmarks for AI safety, covering content and agentic guardrails. The role involves working on research projects studying agent behaviors in the wild, and developing evals for new features. You'll own the benchmark suite and collaborate with product and research teams.

• Own and maintain internal benchmark suite covering single/multi-turn content guardrails and agentic safety.
• Build benchmarks that distinguish specific model capabilities.
• Work with product team to build evals for flagship models.
• Build benchmarks for new research features.
• Adapt evals to new verticals and product data.
• Work on research projects studying agentic and LLM failure modes.

• Have built an LLM benchmark from scratch that distinguished specific model capabilities.
• Have built synthetic data for post-training textual or multimodal models.
• Can reproduce a published benchmark result and identify methodology weaknesses.
• Write production-quality Python code.
• Can write efficient LLM inference setups (parallel calls, retries, rate-limit handling).
• Fluent with frontier models and coding agents daily.