LLM Ops Lead - AI Quality, Governance & Observability

Sourced from emprego24.pt

We are seeking a Senior LLM Ops Lead to scale our Generative AI initiatives with a strong focus on quality, safety, and observability. This role involves defining AI quality standards, managing evaluation frameworks, and overseeing governance and incident response protocols. The ideal candidate will drive continuous improvement in model performance and reliability.

• Own and define AI quality standards and evaluation frameworks
• Manage release gates for model deployments
• Oversee prompt and model versioning strategies
• Lead governance initiatives and safety programs
• Manage AI incident response and mitigation

• Experience scaling Generative AI systems
• Strong understanding of AI quality assurance and observability
• Knowledge of LLM governance and safety protocols
• Ability to design and implement evaluation frameworks

• Experience with automated testing for LLMs
• Background in AI risk management