Summary by AIEngineer.careers
AI Infrastructure Engineer is a AI Infrastructure Engineer role (full-time). with Netpreme. in SANTA CLARA, US. Imported listing (source: careers4graduates.com). Apply on the employer's site (careers4graduates.com).
Imported listing (source: careers4graduates.com) · Apply on careers4graduates.com
How our listings work
The sections below reproduce the third-party job description for reference. AIEngineer.careers does not write or control this text.
Imported job description
Sourced from careers4graduates.com
Netpreme is building full-stack memory solutions, from a proprietary memory management library to X-Mem Memory Processing Unit silicon, to break the AI inference memory wall. As an AI Infrastructure Engineer, you'll own the LLM inference-serving stack end to end, tuning engines like vLLM, SGLang, and TensorRT-LLM and architecting distributed serving with NVIDIA Dynamo. You'll work directly with hardware and systems engineers to benchmark and optimize new models from research to production. This is an onsite role in Santa Clara, CA or Boston, MA, with relocation assistance available.
Responsibilities
- Deploy and operate large language and multimodal models in production using vLLM, SGLang, and TensorRT-LLM
- Build and maintain distributed inference systems with NVIDIA Dynamo, Kubernetes, Helm, and Ansible
- Tune parallelism strategies (TP/EP/DP/PP), batching, KV caching, routing, and scheduling to meet latency, throughput, and SLA targets
- Build reproducible benchmarks for latency (TTFT, TPOT, E2E), throughput, concurrency scaling, and system efficiency
- Enable tool calling, agentic and coding workflows, RAG, and long-context applications
- Evaluate new models, frameworks, and optimizations and partner with hardware, model, and systems engineers to ship improvements
Requirements
- BS, MS, or PhD in Computer Science, Computer Engineering, or related field, or equivalent practical experience
- 2+ years of engineering experience in LLM inference, ML systems, GPU systems, or related infrastructure
- Hands-on experience deploying and optimizing at least one LLM inference engine (vLLM, SGLang, or TensorRT-LLM)
- Solid understanding of LLM inference fundamentals: prefill/decode, batching, KV caching, and latency/throughput tradeoffs
- Strong Python and Linux skills, plus hands-on experience with Kubernetes and containerized deployments
- Comfortable benchmarking and troubleshooting systems with clear communication in a small team
Nice to Have
- Experience with Helm, Ansible, or similar deployment and configuration tools
- Experience with distributed inference systems like NVIDIA Dynamo, including parallelism strategies and prefill/decode disaggregation
- Experience with advanced inference optimizations such as prefix caching, speculative decoding, and quantization
- Experience analyzing LLM architectures and tuning deployments across hardware platforms to meet SLAs
- Contributions to vLLM, SGLang, TensorRT-LLM, or related open-source projects
- Experience owning production infrastructure, ideally at an early-stage startup
Benefits
Equity / stock optionsHealth insurancedental insurancevision insurance401k matchlife insurancedisability insuranceptorelocation assistanceVisa sponsorshiplunch stipendfitness stipend
Tech Stack
AnsibleHelmKubernetesKV cachingLinuxNVIDIA DynamoPythonQuantizationRAGSGLangspeculative decodingTensorRT-LLMvLLM

Cambridge · US · 20+ employees
Netpreme is a Cambridge, Massachusetts startup founded in 2024 that builds AI-native memory systems, from silicon to tokens. It develops the world's first Memory Processing Unit (MPU) chip and X-Mem networked memory tiering technology that expands GPU/accelerator memory capacity for AI training, inference, and agentic AI workloads.