Senior Machine Learning Engineer, LLM Inference Optimization
Nebius
LLM Inference Engineer (Mid, Sr, Staff) is a AI Inference Engineer role (full-time). with Hippocratic AI. in MENLO PARK, CA (ONSITE), US. Imported listing (source: mikeishiring.com). Apply on the employer's site (mikeishiring.com).
Imported listing (source: mikeishiring.com) · Apply on mikeishiring.com
The sections below reproduce the third-party job description for reference. AIEngineer.careers does not write or control this text.
Sourced from mikeishiring.com
Hippocratic AI is hiring an LLM Inference Engineer to own the serving infrastructure behind its healthcare-focused LLM platform. You will optimize distributed inference to deliver sub-100ms responses at scale, improving latency, throughput, and cost per inference. The role spans multi-node serving architectures, quantization, speculative decoding, and disaggregated serving across GPU types. This is an on-site role in Menlo Park, CA.

Palo Alto · US · 150+ employees
Hippocratic AI is a healthcare technology company building a safety-focused large language model (LLM) designed specifically for healthcare applications. Its AI agents are designed to assist health systems, payors, and pharmaceutical organizations with non-diagnostic clinical tasks.
Get a weekly email when new AI engineering jobs match your filters. Free — confirm your address to start.
Nebius
Apple
Advanced Micro Devices, Inc.
Netpreme
SambaNova Systems
Capgemini