Staff / Principal Machine Learning Engineer, Serving - USA

Inworld AI is hiring a Staff/Principal Machine Learning Engineer to optimize real-time multimodal inference at scale. This role focuses on inference optimization, model acceleration, and high-performance systems. The position is fully remote and open to candidates in California, USA.

• Optimize inference performance and serving frameworks like vLLM or TRT-LLM.
• Implement model acceleration techniques including quantization, distillation, caching, and speculative decoding.
• Own the full cycle from model research to production deployment.
• Ensure reliable, low-latency serving at scale with Kubernetes and Ray.

• Deep understanding of modern serving frameworks (vLLM, TRT-LLM).
• Hands-on experience with quantization, distillation, caching, continuous batching, paged attention, and speculative decoding.
• Proficiency in C++, CUDA, Rust, or highly optimized Python.
• Experience with distributed systems, Kubernetes, Ray, and multi-GPU/multi-node inference.
• Public work such as open-source contributions or deep-dive technical write-ups.
• PhD in CS, Physics, Math, or equivalent practical experience.