Sr. Software Development Engineer, Inference Team - AWS Neuron - Seattle

AWS Neuron is the software stack for AWS Inferentia and Trainium accelerators for cloud-scale ML. As a senior engineer on the Machine Learning Inference Applications team, you will lead development and optimization of open-source inference frameworks like vLLM and SGLang to deliver high-performance LLM serving on AWS Neuron. You will collaborate with model, compiler, runtime, and performance engineers to ensure end-to-end performance, scalability, and production readiness.

• Lead customization and optimization of open-source inference frameworks (vLLM, SGLang) on AWS Neuron, including core framework logic like scheduling and model execution
• Apply state-of-the-art techniques in kernel development, parallel computation, distributed KV cache, and speculative decoding to deliver best-in-class LLM serving performance
• Influence the technical roadmap by evaluating emerging inference research and translating it into production-ready features
• Provide design leadership and mentorship while collaborating across model development, compiler, runtime, and performance engineering teams

• 5+ years of full software development life cycle experience, including coding standards, code reviews, source control management, build processes, testing, and operations
• Bachelor's degree in computer science or equivalent
• 5+ years of programming in a modern language such as Java, C++, or C#, including object-oriented design