Search
Go
Staff / Principal Machine Learning Engineer, Serving - USA
Summary by AIEngineer.careers
Staff / Principal Machine Learning Engineer, Serving - USA is a MLOps Engineer role (full-time). with Inworld AI. in MOUNTAIN VIEW, US. Compensation shown: $270K–$500K. Imported listing (source: dice.com). Apply on the employer's site (dice.com).
Imported listing (source: dice.com) · Apply on dice.com
MOUNTAIN VIEW · US
$270K–$500K
Full-time
STAFF
The sections below reproduce the third-party job description for reference. AIEngineer.careers does not write or control this text.
Imported job description
Sourced from dice.com
Inworld AI is seeking a Staff/Principal Machine Learning Engineer to lead high-performance inference serving for real-time, consumer-facing AI applications. You will optimize large-scale LLM and speech model deployments using modern frameworks like vLLM and TRT-LLM, ensuring sub-second latency and high throughput. This role requires deep expertise in systems programming, GPU acceleration, and distributed scaling to support trillions of tokens monthly.
Responsibilities
- Optimize inference performance using techniques like quantization, distillation, continuous batching, and speculative decoding.
- Build and maintain high-performance serving infrastructure on NVIDIA GPUs using C++, CUDA, Rust, or optimized Python.
- Scale distributed systems to handle thousands of concurrent connections using Kubernetes and Ray.
- Take full-cycle ownership of models from research containerization to production reliability.
- Profile code and architecture to maximize throughput and minimize latency.
Requirements
- Deep understanding of modern serving frameworks (vLLM, TRT-LLM).
- Proficiency in C++, CUDA, Rust, or highly optimized Python with strong systems programming skills.
- Experience with distributed systems, multi-GPU/multi-node inference, and load balancing.
Inworld AI
Mountain View · US · 85+ employees
Inworld AI is a product-oriented realtime AI research lab and model provider focused on voice AI for natural, human-like conversation. Its platform supports interactive voice agents and conversational applications across areas such as entertainment, education, customer support, and enterprise use.
