Senior Machine Learning Engineer - Applied Scientist

Sourced from scoutify.com

Microsoft seeks a Senior Machine Learning Engineer / Applied Scientist to design, evaluate, and ship next-generation AI-powered product experiences. The role blends applied science depth with hands-on engineering, product judgment, and end-to-end ownership from prototyping through launch. You will lead applied science strategy and build LLM evaluation frameworks for agentic, retrieval-augmented, and multimodal AI systems.

• Lead applied science strategy for AI-powered product areas from opportunity identification through launch and continuous improvement
• Design and operationalize LLM evaluation frameworks across the product development lifecycle
• Build evaluation methodologies for LLM-based, agentic, retrieval-augmented, and multimodal AI systems
• Develop offline experimentation and evaluation pipelines to compare models, prompts, retrieval strategies, and tool use
• Partner with engineering and PM leadership to align science priorities with product strategy
• Establish quality bars and decision-making frameworks for AI launches, including scorecards and regression testing
• Mentor scientists and engineers and influence cross-team best practices

• Bachelor's Degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 4+ years related experience (or Master's + 3 years, or Doctorate + 1 year, or equivalent)
• Ability to meet Microsoft, customer and/or government security screening requirements, including Microsoft Cloud Background Check
• Hands-on experience with large language models, including prompt engineering, RAG, agents, and tool use
• Deep understanding of LLM evaluation throughout the development lifecycle
• Experience designing offline experiments and interpreting results
• Solid software engineering skills and experience building production-quality AI systems

• PhD in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 6+ years related experience
• Significant experience building, evaluating, and shipping AI/ML-powered products or services
• Experience evaluating multimodal AI systems
• Solid understanding of product success metrics and translating business goals into measurable objectives