AI Research Engineer – Datadog AI Research (DAIR)

Datadog AI Research (DAIR) seeks an AI Research Engineer to collaborate with research scientists on building systems for observability foundation models, SRE autonomous agents, and production code repair agents. The role involves developing training and evaluation pipelines, implementing models at scale, and integrating AI capabilities into Datadog's product ecosystem.

• Build and operate datasets, training and evaluation pipelines, benchmarks, and internal tooling
• Implement models, run experiments at scale, and profile for reliability, performance, and cost
• Orchestrate distributed training and distributed RL with Ray, including scheduling, scaling, and failure recovery
• Make the research stack observable, reproducible, and easier to use
• Establish rigorous automated benchmarks and regression tests for forecasting, anomaly detection, multi-modal analysis, agents, and code repair tasks
• Collaborate with Research Scientists, Product, and Engineering to integrate advanced AI capabilities into Datadog’s product ecosystem and to harden prototypes into reliable services
• Contribute high-quality code, documentation, and open-source artifacts

• Strong software engineering skills with experience in observability, SRE, or security
• Depth in distributed computing and ML systems for training and inference at scale