Scala Senior Machine Learning Infrastructure Engineer, ML Enablement

Sourced from etsy.wd5.myworkdayjobs.com

Etsy is hiring a Senior Machine Learning Infrastructure Engineer to build and operate its ML orchestration platform, the single entry point for all ML inference requests across search, recommendations, ads, trust and safety, and generative AI. The role focuses on high-scale, latency-sensitive, highly available systems and self-service tooling for Applied Scientists and Software Engineers. You will join the ML Enablement platform team, working closely with model serving, LLM hosting, and data teams.

• Design, build, and operate services that route, orchestrate, and execute online ML inference at scale
• Build orchestration capabilities such as multi-model request graphs, feature retrieval, caching, batching, fallbacks, and graceful degradation
• Enable safe model rollout and evaluation with shadow traffic, canaries, A/B experiments, and online/offline evaluation
• Support predictive and generative LLM workloads, including streaming, long-running requests, and GPU-backed serving
• Build self-service APIs, SDKs, and tooling so scientists and engineers can ship models without platform expertise
• Define SLOs and capacity plans; lead monitoring, alerting, incident response, and postmortems
• Mentor teammates and contribute to design and code reviews

• 5+ years of software engineering experience building and operating distributed, low-latency production services
• Experience with Scala or another functional programming language and functional patterns (immutability, composition, effect/concurrency libraries)
• Solid understanding of production ML inference: model serving, feature retrieval, batching, latency/throughput trade-offs
• Hands-on experience operating services on Kubernetes, including scaling, resource management, and debugging
• Experience designing service APIs and request-routing or orchestration systems (e.g., gRPC, queues)
• Strong operational skills: metrics, tracing, alerting, on-call ownership, and performance profiling
• Product mindset and strong communication skills

• Experience with cloud infrastructure, ideally GCP
• Experience with model serving frameworks and LLM inference infrastructure (e.g., vLLM, Triton, KServe)
• Experience with experimentation platforms, feature stores, or online evaluation systems
• Experience with GPU capacity management and cost optimization
• Experience with Scala libraries such as Cats Effect, ZIO, Akka/Pekko, or http4s