Skip to main content

AI Infrastructure Engineer

Summary by AIEngineer.careers

AI Infrastructure Engineer is a AI Infrastructure Engineer role (full-time). with Netpreme. in SANTA CLARA, US. Imported listing (source: careers4graduates.com). Apply on the employer's site (careers4graduates.com).

Imported listing (source: careers4graduates.com) · Apply on careers4graduates.com

How our listings work

SANTA CLARA · US
—
Full-time
MID
Apply Now

More AI jobs in United States

The sections below reproduce the third-party job description for reference. AIEngineer.careers does not write or control this text.

Imported job description

Sourced from careers4graduates.com

Netpreme is building full-stack memory solutions, from a proprietary memory management library to X-Mem Memory Processing Unit silicon, to break the AI inference memory wall. As an AI Infrastructure Engineer, you'll own the LLM inference-serving stack end to end, tuning engines like vLLM, SGLang, and TensorRT-LLM and architecting distributed serving with NVIDIA Dynamo. You'll work directly with hardware and systems engineers to benchmark and optimize new models from research to production. This is an onsite role in Santa Clara, CA or Boston, MA, with relocation assistance available.

Responsibilities

  • Deploy and operate large language and multimodal models in production using vLLM, SGLang, and TensorRT-LLM
  • Build and maintain distributed inference systems with NVIDIA Dynamo, Kubernetes, Helm, and Ansible
  • Tune parallelism strategies (TP/EP/DP/PP), batching, KV caching, routing, and scheduling to meet latency, throughput, and SLA targets
  • Build reproducible benchmarks for latency (TTFT, TPOT, E2E), throughput, concurrency scaling, and system efficiency
  • Enable tool calling, agentic and coding workflows, RAG, and long-context applications
  • Evaluate new models, frameworks, and optimizations and partner with hardware, model, and systems engineers to ship improvements

Requirements

  • BS, MS, or PhD in Computer Science, Computer Engineering, or related field, or equivalent practical experience
  • 2+ years of engineering experience in LLM inference, ML systems, GPU systems, or related infrastructure
  • Hands-on experience deploying and optimizing at least one LLM inference engine (vLLM, SGLang, or TensorRT-LLM)
  • Solid understanding of LLM inference fundamentals: prefill/decode, batching, KV caching, and latency/throughput tradeoffs
  • Strong Python and Linux skills, plus hands-on experience with Kubernetes and containerized deployments
  • Comfortable benchmarking and troubleshooting systems with clear communication in a small team

Nice to Have

  • Experience with Helm, Ansible, or similar deployment and configuration tools
  • Experience with distributed inference systems like NVIDIA Dynamo, including parallelism strategies and prefill/decode disaggregation
  • Experience with advanced inference optimizations such as prefix caching, speculative decoding, and quantization
  • Experience analyzing LLM architectures and tuning deployments across hardware platforms to meet SLAs
  • Contributions to vLLM, SGLang, TensorRT-LLM, or related open-source projects
  • Experience owning production infrastructure, ideally at an early-stage startup

Benefits

Equity / stock optionsHealth insurancedental insurancevision insurance401k matchlife insurancedisability insuranceptorelocation assistanceVisa sponsorshiplunch stipendfitness stipend

Tech Stack

AnsibleHelmKubernetesKV cachingLinuxNVIDIA DynamoPythonQuantizationRAGSGLangspeculative decodingTensorRT-LLMvLLM
Netpreme

Netpreme

Cambridge · US · 20+ employees

Netpreme is a Cambridge, Massachusetts startup founded in 2024 that builds AI-native memory systems, from silicon to tokens. It develops the world's first Memory Processing Unit (MPU) chip and X-Mem networked memory tiering technology that expands GPU/accelerator memory capacity for AI training, inference, and agentic AI workloads.

Weekly job alert

Get a weekly email when new AI engineering jobs match your filters. Free — confirm your address to start.

Related Jobs