Senior ML Engineer (Inference Serving)

Join a stealth AI hardware startup building a custom AI SoC and inference serving stack. As a Senior ML Engineer, you will architect high-performance multi-node inference stacks and implement optimizations in frameworks like vLLM and PyTorch. You will work on advanced cluster scheduling and contribute to open-source AI infrastructure.

• Serve as a core contributor on a small, senior team building state-of-the-art inference serving and cluster scheduling capabilities for a custom AI SoC.
• Architect high-performance multi-node inference stacks, designing and tuning throughput and latency from the ground up.
• Implement advanced optimisation strategies across TP/PP/EP hybrids, continuous batching, and KV cache management.
• Drive performance improvements inside leading inference frameworks including vLLM, SGLang, and PyTorch.
• Develop advanced cluster scheduling algorithms for large-scale open-source models.
• Engage with the open-source community, upstreaming optimisations and influencing AI infrastructure projects.
• Apply best practices in performance benchmarking, testing, and debugging for a production-grade stack on novel silicon.

• Strong Python, C++ and PyTorch engineering fundamentals with a track record of shipping high-quality software.
• 1+ years as an active developer on LLM inference serving frameworks such as vLLM or SGLang.