Search
Go
Senior Machine Learning Engineer, LLM Inference Optimization
Summary by AIEngineer.careers
Senior Machine Learning Engineer, LLM Inference Optimization is a ML Engineer role (full-time). with Nebius. in BERLIN, DE. Imported listing (source: databerlin.net). Apply on the employer's site (databerlin.net).
Imported listing (source: databerlin.net) · Apply on databerlin.net
BERLIN · DE
—
Full-time
SENIOR
The sections below reproduce the third-party job description for reference. AIEngineer.careers does not write or control this text.
Imported job description
Sourced from databerlin.net
Nebius is seeking a Senior Machine Learning Engineer to optimize LLM and VLM inference performance for their cloud platform. The role focuses on reducing latency, improving throughput, and lowering cost per token through advanced model compression and serving architecture optimizations. You will work closely with kernel and platform engineers to deploy and benchmark inference engines like vLLM and TensorRT-LLM.
Responsibilities
- Own optimization work for specific model families and customer endpoints
- Deploy, configure, and extend inference engines such as vLLM, SGLang, and TensorRT-LLM
- Build and productionize model-compression workflows including quantization and distillation
- Implement speculative decoding, KV-cache optimization, and continuous batching
- Build reproducible benchmark harnesses for latency, throughput, and GPU memory usage
- Partner with GPU kernel engineers to diagnose bottlenecks across the stack
Requirements
- Strong Python and PyTorch engineering skills
- Hands-on experience deploying or optimizing LLM/VLM inference systems
- Practical knowledge of modern inference stacks (vLLM, SGLang, TensorRT-LLM, etc.)
- Deep understanding of transformer inference bottlenecks (KV cache, attention, memory bandwidth)
Nebius
Amsterdam · NL · 1590+ employees
Nebius is an AI cloud company providing full-stack infrastructure for AI developers and enterprises. Its platform supports the complete AI journey, including data storage, model training, tuning, and production runtime deployment.
