Research Engineer

Sourced from career.io

Gauntlet is hiring an AI Research Engineer to build systematic evaluation frameworks for AI agents used across research, engineering, and business workflows. You'll design experiments comparing models, harnesses, and tooling, investigate agent failures, and ship improvements to production agents like their in-house generalist agent Aria. The role blends applied research with hands-on engineering at the intersection of AI, quantitative research, and DeFi.

• Build evaluation frameworks for AI agents, including datasets, sandbox environments, and metrics for real-world tasks
• Design experiments to compare models, agent harnesses, tools, and execution environments
• Turn qualitative feedback on agent behavior into concrete research questions and measurable outcomes
• Investigate agent failures and determine root causes across model, harness, tooling, or prompting
• Build and improve agent harnesses, internal tools, and integrations including Aria
• Develop shared knowledge and reusable skills across agents
• Work with Infrastructure and Security to ensure agents operate reliably and safely in production

• Hands-on experience building, evaluating, or improving LLM models or agents
• Strong Python skills and solid software engineering fundamentals
• Experience with applied research, machine learning, or data science including experiment design
• Understanding of LLM agents beyond prompting: tool use, harnesses, context management, failure modes
• Ability to break down open-ended problems into testable questions and work independently
• Experience building tools or systems used by others
• Comfort collaborating across research and engineering teams

• Experience building LLM evaluation pipelines, benchmarks, or model testing infrastructure
• Contributions to open-source agent frameworks, harnesses, or evaluation datasets
• Experience with agent frameworks such as Hermes or OpenClaw
• Applied ML research, model training, fine-tuning, or reinforcement learning
• Advanced degree or postgraduate research in CS, math, statistics, or ML
• Experience in financial markets, quantitative research, DeFi, or risk modeling
• Familiarity with AI security considerations such as access controls and prompt injection