Senior Research Engineer (Agentic Behavior)

JetBrains is hiring a Senior Research Engineer to focus on agentic behavior in AI coding agents. The role involves building evaluation and analysis pipelines for LLMs and agent systems, conducting data-driven research, and improving agent performance. You will work on post-training methods and experiment with agentic frameworks to enhance developer tools.

• Design and implement evaluation pipelines for LLM-based coding agents
• Analyze agent behavior traces and identify failure modes
• Develop data pipelines and statistical analyses for experimental results
• Contribute to training data and reward model pipelines for post-training
• Collaborate with product teams to translate insights into product improvements

• Hands-on experience building evaluation or analysis pipelines for LLMs or AI coding agents
• Strong Python engineering skills (3+ years) with clean, maintainable code
• Experience with large-scale data analysis (SQL, Athena, data pipelines, statistical analysis)
• Ability to own projects end-to-end from problem identification to solution
• Product-aware mindset and understanding of developer workflows
• Familiarity with Kotlin or strong willingness to learn