Research Engineer, LangSmith Engine - LangChain

Sourced from underprompt.com

LangChain is looking for an experienced Research Engineer to join the LangSmith Engine team, which builds a proactive agent engineer that analyzes production traces and improves AI agents. In this role, you will build benchmarks and evaluations, run experiments across models, prompting, and agent strategies, and turn successful ideas into production improvements. You will work on a production system, balancing quality with cost, latency, reliability, and scalability.

• Build and maintain benchmarks and evaluations that measure the quality and efficiency of Engine agents on real-world tasks.
• Design and run experiments to improve agent performance across models, prompting, context, tools, orchestration, and agent strategies.
• Explore and implement post-training and fine-tuning techniques to improve agent capabilities, quality, or cost.
• Turn successful experiments into production improvements, working with engineers and researchers to measure impact and prevent regressions.
• Help define the ML roadmap and technical direction for improving Engine agents and mentor other engineers.

• 4+ years of experience in ML/AI research or a closely related field.
• Master's or PhD in a relevant scientific field.
• Hands-on experience with LLMs and AI agents, including analyzing model behavior and improving real-world performance.