AI Inference Platform Engineer

Sourced from thejobsmap.com

DRW, a diversified proprietary trading firm headquartered in Chicago, is hiring an AI Inference Platform Engineer to build, operate, and optimize the systems serving its LLM, vision, multimodal, and embedding models. You will own the firmwide AI inference platform end-to-end, from day-0 model onboarding through reliable production serving, working across inference runtimes, distributed systems, and GPU platform engineering. The role focuses on maximizing latency, throughput, utilization, reliability, and cost efficiency across the inference fleet on modern NVIDIA GPU architectures.

• Optimize LLM inference performance across modern NVIDIA GPU architectures and inference runtimes
• Build end-to-end performance profiling and observability from individual GPU kernels to multi-node inference systems
• Design and optimize KV cache and distributed inference architectures, including caching, routing, memory tiering, and prefill/decode strategies
• Own day-0 model onboarding, determining runtime, precision, sharding, memory, batching, cache policy, and serving configuration
• Measure and monitor quality equivalence across serving configurations, including KV cache quantization, speculative decoding, precision choices, and model routing
• Manage the production serving lifecycle of models, including versioning, staging, canarying, promotion, rollback, and retirement
• Partner with SRE and platform teams to automate model deployment, observability, and reliable operation across environments