Skip to main content

Senior AI Platform Engineer (m/f/d)

Summary by AIEngineer.careers

Senior AI Platform Engineer (m/f/d) is a AI Platform Engineer role (full-time). with evoila Frankfurt GmbH. in REMOTE, WORLDWIDE. Imported listing (source: remoshift.com). Apply on the employer's site (remoshift.com).

Imported listing (source: remoshift.com) · Apply on remoshift.com

How our listings work

REMOTE · WORLDWIDE
—
Full-time
SENIOR
Apply Now

More AI jobs in Germany · Remote AI jobs

The sections below reproduce the third-party job description for reference. AIEngineer.careers does not write or control this text.

Imported job description

Sourced from remoshift.com

As a Senior AI Platform Engineer at evoila, you design, build, and operate highly available AI/ML platforms for clients, primarily on Kubernetes across on-premises, hybrid, and cloud environments. You architect scalable GPU infrastructure, run model-serving and inference services for classical ML and LLMs, and establish MLOps/LLMOps workflows for production GenAI. The role combines hands-on platform engineering with client advisory on AI governance, security, and best practices.

Responsibilities

  • Plan, build, and operate highly available AI/ML platforms and services on Kubernetes across on-prem, hybrid, and cloud environments
  • Design and implement scalable GPU infrastructure within Kubernetes clusters, including GPU scheduling and sharing (Kueue, KAI-Scheduler, Run:ai, NVIDIA MIG/Time-Slicing)
  • Build and operate model-serving and inference services (vLLM, NVIDIA Triton, KServe, NVIDIA NIM, Ray) for ML models and LLMs
  • Develop and run MLOps/LLMOps workflows for production GenAI services, including guardrails, evaluation, regression testing, and cost/quality monitoring
  • Provide fine-tuning and re-training workflows (e.g., LoRA, continuous re-training) on the platform
  • Monitor, troubleshoot, and performance-tune AI platforms (GPU utilization, inference latency, throughput)
  • Advise clients on best practices, AI governance, data security, and privacy for AI platforms on Kubernetes
  • Collaborate closely with data science, data platform, and development teams

Requirements

  • At least 3 years operating and optimizing platforms and services at scale in Kubernetes (on-prem or cloud)
  • Solid knowledge of architecting and operating scalable solutions with Kubernetes, ideally including GPU workloads
  • Programming skills in Python, ideally supplemented by Golang or Java
  • Experience with at least one model-serving framework (vLLM, NVIDIA Triton, KServe) and MLOps tools (MLflow, Kubeflow, ClearML)
  • Experience with Infrastructure-as-Code tools such as Ansible and Terraform, plus CI/CD and GitOps tools (Argo CD, Flux)
  • Excellent communication skills in German and English
  • For the senior role: at least 5 years of relevant platform engineering experience, architecture ownership, technical leadership in client projects, and mentoring

Nice to Have

  • Experience with cloud AI services (AWS SageMaker/Bedrock, Azure ML/Azure OpenAI/Azure AI Foundry) and integration with Kubernetes workloads
  • Knowledge of LLM-based architectures such as RAG pipelines, vector databases, embedding services, agentic AI, and Model Context Protocol (MCP)
  • Understanding of LLM serving fundamentals (rate limiting, token streaming, load balancing) and LLM concepts like reasoning, tool calling, and prompt templates
  • Experience with llm-d or Kubernetes Inference Gateway
  • Knowledge of securing AI platforms (TLS, RBAC, Network Policies) and AI governance (e.g., EU AI Act)
  • Certifications such as CKA/CKAD or NVIDIA certifications
  • Willingness to travel occasionally within the DACH region

Benefits

retirement planteam eventsprofessional developmentcorporate benefitsflexible hoursmodern equipmentremote work

Tech Stack

AnsibleArgo CDAWS BedrockAWS SageMakerAzure MLAzure OpenAIClearMLFluxGolangGPU InfrastructureJavaKServeKubeflowKubernetesKueueLLMOpsLoRAMLflowMLOpsNVIDIA MIGNVIDIA NIMNVIDIA TritonPythonRAGRayRun:aiTerraformvLLM
EF

Weekly job alert

Get a weekly email when new AI engineering jobs match your filters. Free — confirm your address to start.

Related Jobs