Software Engineer, Inference (AI Data Engineering)

SpaceX is looking for a Software Engineer to design and optimize large-scale AI inference serving systems end-to-end, from distributed infrastructure to low-level GPU optimizations. The role involves building reliable, high-throughput inference platforms that serve internal AI models, with ownership of request routing, rate limiting, and inference engine acceleration. This is an on-site role in Palo Alto, supporting mission-critical applications and training workloads.

• Develop highly reliable, high-throughput inference systems. Architect scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache, continuous batching). Optimize latency and throughput with GPU kernel work, quantization, and speculative decoding. Build high-concurrency serving systems with low tail latency and observability. Own components such as request routing, SDK development, and rate limiting. Benchmark and accelerate inference engines (SGLang, vLLM, TensorRT-LLM). Create CI/CD infrastructure for endpoint deployment and engine updates.

• Bachelor's degree in computer science, engineering, math, or scientific discipline; OR 2+ years of professional experience building software. Experience designing, implementing, and maintaining reliable, horizontally scalable distributed systems. 1+ years of experience in full stack or backend development with production systems. 1+ years of experience with Rust or C++.