Senior AI/ML Engineer — LLM & Agent Stack (Customer Facing)

Sourced from nogigiddy.com

TrueFoundry is hiring a Senior AI/ML Engineer to design and own core components of its AI Gateway control plane, enabling enterprise customers to run production agentic AI safely and efficiently. The role focuses on agent orchestration, model routing, observability, and policy enforcement, with direct engagement with strategic enterprise customers. You will integrate upstream tooling such as LangGraph, LangChain, vector stores, and specialized LLM runtimes.

• Architect and implement scalable agent orchestration patterns (graph-based executors, state management, multi-agent coordination) for production workloads
• Own critical integrations: model adapters, LLM gateway hooks, vector DBs, tools and external APIs, and LLMOps flows
• Build and improve tracing, benchmarking, and observability for LLMs and agents, including token/cost accounting, latency p95, throughput, and correctness checks
• Drive design for safety and guardrails: moderation hooks, human-in-the-loop checkpoints, replayable audit trails, and policy enforcement
• Mentor junior engineers, run design reviews, and improve engineering practices including testing, CI/CD, and chaos testing for agents
• Work directly with strategic customers to prototype complex agentic solutions and translate them into product features

• 5–10 years of software engineering experience with substantial work on distributed systems, infrastructure, or ML platforms
• Deep practical experience integrating and deploying LLMs in production (RAG, retrieval, embeddings pipelines)
• Hands-on experience with agent orchestration frameworks such as LangGraph, LangChain, or custom agent runtimes, and stateful workflow design
• Strong systems knowledge: Kubernetes, container orchestration, service meshes, and performance tuning
• Proven track record building observability, cost controls, and policy enforcement for production services

• Experience building or contributing to open-source LLM orchestration tools (LangGraph, LangChain, or similar)
• Familiarity with enterprise constraints: on-prem/cloud hybrid deployments, data residency, and compliance requirements
• Background in security, privacy, or model governance for LLMs
• Demonstrated leadership in cross-functional projects and direct customer engagement