Search
Go
LLM Inference Systems Engineer
Summary by AIEngineer.careers
LLM Inference Systems Engineer is a AI Infrastructure Engineer role (full-time). with Hippocratic AI. in MENLO PARK, US. Imported listing (source: jobs.a16z.com). Apply on the employer's site (jobs.a16z.com).
Imported listing (source: jobs.a16z.com) · Apply on jobs.a16z.com
MENLO PARK · US
—
Full-time
MID
The sections below reproduce the third-party job description for reference. AIEngineer.careers does not write or control this text.
Imported job description
Sourced from jobs.a16z.com
Hippocratic AI is hiring an LLM Inference Systems Engineer to build distributed serving and high-performance networking layers for large language models. The role focuses on multi-node execution, KV-cache transfer, and observability. This is an on-site position in Menlo Park, CA, requiring five days a week in the office.
Responsibilities
- Build distributed serving infrastructure for large language models
- Develop high-performance networking layers for multi-node execution
- Optimize KV-cache transfer and management
- Implement observability and monitoring for inference systems
Requirements
- Experience with distributed systems and high-performance networking
- Knowledge of large language model inference and serving
- Familiarity with KV-cache optimization techniques
- Ability to work on-site in Menlo Park, CA five days a week
Nice to Have
- Experience with multi-node execution frameworks
- Background in observability tools for ML systems
Tech Stack
Hippocratic AI
Palo Alto · US · 310+ employees
Hippocratic AI is a healthcare technology company that develops safety-focused large language models for clinical applications. It aims to improve healthcare accessibility and outcomes by providing AI agents that assist medical professionals and healthcare systems.
