MLOps Engineer - AI Systems

Sourced from rightjobsnow.com

Mercor is hiring a contract MLOps Engineer focused on LLM performance, covering kernels, profiling, debugging, and inference serving. The role involves designing challenging ML systems tasks, writing reference solutions, and evaluating frontier model outputs on accelerator code. You will develop rubrics and provide expert technical feedback to help research and engineering teams improve model performance on ML systems, compilers, and training infrastructure.

• Design challenging tasks across kernels, profiling, debugging, and inference serving
• Write accurate, well-structured reference solutions exposing where frontier models fail on accelerator code
• Evaluate ML systems tasks and solutions
• Provide clear written technical feedback that stands up to expert review
• Develop guidelines and rubrics covering kernel optimization, profiler interpretation, distributed systems reasoning, and serving throughput/latency trade-offs
• Collaborate with subject matter experts to keep training data consistent and accurate

• 2+ years of hands-on professional experience in ML systems, ML infrastructure, model serving, or accelerator performance engineering
• Experience writing or optimizing custom kernels (Pallas, Triton, CUDA, CUTLASS)
• Performance profiling and trace analysis (JAX/XLA profiler, Kineto, torch.profiler, Nsight)
• Debugging distributed or accelerator-bound workloads
• Serving LLMs at scale (vLLM, SGLang, TensorRT-LLM, JetStream, KV cache, paged attention, continuous batching)
• Production experience with JAX and/or PyTorch
• Experience with modern accelerators (TPU v4–v6e, H100, B200, Trainium, MI300) and reasoning about throughput, latency, and memory trade-offs
• Strong written communication skills
• Availability for at least 40 hours/week during weekdays

• Framework-level depth: custom operators, sharding and distributed training (pjit/shard_map, FSDP, Megatron, MaxText)
• Compiler and graph-level work (XLA/StableHLO, torch.compile, MLIR)
• Demonstrable career progression