Search
Go
This job is no longer accepting applications
This listing was closed on Jul 18, 2026.
Browse Open JobsLLM Inference Engineer
Summary by AIEngineer.careers
LLM Inference Engineer is a AI Engineer role (full-time). with Hippocratic AI. in PALO ALTO, US. Imported listing (source: linkedin.com). Apply on the employer's site (linkedin.com).
Imported listing (source: linkedin.com) · Apply on linkedin.com
PALO ALTO · US
—
Full-time
SENIOR
The sections below reproduce the third-party job description for reference. AIEngineer.careers does not write or control this text.
Imported job description
Sourced from linkedin.com
Hippocratic AI is seeking an LLM Inference Engineer to optimize large language model serving infrastructure. The role involves designing multi-node serving architectures, applying quantization techniques, and implementing speculative decoding. Candidates need expertise in Python, C++, CUDA, and GPU optimization.
Responsibilities
- Design and implement multi-node serving architectures for distributed LLM inference; Optimize multi-LoRA serving systems; Apply advanced quantization techniques (FP4/FP6); Implement speculative decoding and other latency optimization strategies; Develop disaggregated serving solutions with optimized caching strategies; Continuously benchmark and improve system performance.
Requirements
- Experience optimizing LLM inference systems at scale; Proven expertise with distributed serving architectures; Hands-on experience implementing quantization techniques; Strong understanding of modern inference optimization methods including speculative decoding; Proficiency in Python and C++; Experience with CUDA programming and GPU optimization.
Nice to Have
- Contributions to open-source inference frameworks such as vLLM, SGLang, or TensorRT-LLM; Experience with custom CUDA kernels; Track record of deploying inference systems in production environments; Deep understanding of performance optimization systems.
Hippocratic AI
Palo Alto · US · 310+ employees
Hippocratic AI is a healthcare technology company that develops safety-focused large language models for clinical applications. It aims to improve healthcare accessibility and outcomes by providing AI agents that assist medical professionals and healthcare systems.
