AI Engineer & Researcher - Inference

PALO ALTO · US

$180K–$440K

Full-time

About the Role

xAI is seeking an AI Engineer & Researcher to optimize inference latency and throughput for large-scale models. This role involves building reliable production serving systems and accelerating research on test-time compute scaling. The team leads development of the open-source inference engine SGLang.

Responsibilities

Optimizing latency and throughput of model inference. Building reliable production serving systems to serve millions of users. Accelerating research on scaling test-time compute.

Requirements

Experience with system optimizations for model serving (batching, caching, load balancing, model parallelism). Low-level optimizations such as GPU kernels and code generation. Algorithmic optimizations like quantization, distillation, speculative decoding. Large-scale, high concurrent production serving. Testing, benchmarking, and reliability of inference services.

Tech Stack

PythonPyTorchJAXKubernetesCUDANCCLRustTritonSGLang

xAI

Palo Alto · US · 2091+ employees

xAI is an artificial intelligence company, now a subsidiary of SpaceX, focused on advancing scientific discovery and understanding the universe. It develops large language models like Grok and provides AI-powered conversational assistants.