Senior LLMOps / MLOps Engineer

Nityo Infotech is seeking a Senior LLMOps/MLOps Engineer for an onsite contract role in Santa Clara, CA. The role focuses on deploying and optimizing open-source LLMs, building high-performance inference platforms using vLLM, SGLang, TGI, Triton, and Ray Serve, and driving GPU utilization, latency, throughput, and cost optimization. The ideal candidate has 5–7 years of MLOps/LLMOps experience, strong Python skills, and expertise in Kubernetes, Docker, Azure ML, Databricks, and MLflow.

• Design, build, troubleshoot, and optimize production AI systems
• Deploy and optimize open-source LLMs at scale
• Build high-performance inference platforms using vLLM, SGLang, TGI, Triton, Ray Serve, Azure ML, or Databricks Model Serving
• Drive improvements in GPU utilization, latency, throughput, and cost
• Deploy, scale, and monitor production-grade GenAI/LLM applications

• 5–7 years of experience in MLOps, LLMOps, or AI/ML Platform Engineering
• Strong proficiency in Python and software engineering best practices
• Experience with open-source LLMs such as Llama, Mistral, Gemma, or Qwen
• Strong expertise in LLM inferencing and model hosting using vLLM, SGLang, TGI, Triton, Ray Serve, Azure ML, or Databricks Model Serving
• Experience with Kubernetes, Docker, Azure ML, Databricks, and MLflow
• Good understanding of RAG, vector databases, GPU optimization, quantization, KV cache, PagedAttention, and dynamic batching