Applied AI Engineer, Operations Automation

Sourced from cvkeskus.ee

Bolt is seeking a Senior Applied AI Engineer to join the Growth team in Tallinn, focusing on building operations automation systems for its global mobility platform. The role involves applying machine learning and LLMs to solve complex challenges in customer support, document verification, and fraud detection. You will redesign operational processes, build evaluation suites for AI agents, and optimize models through fine-tuning and post-training techniques.

• Build and operate systems that call LLMs for operational decision-making.
• Redesign business processes by identifying steps suitable for software, agents, or human checkpoints.
• Design and calibrate LLM judges against human labels to ensure quality and prevent reward hacking.
• Run A/B tests and manage rollout gates for new AI features.
• Implement fine-tuning and post-training methods (e.g., DPO, LoRA) to improve model performance.

• Experience shipping and operating systems that integrate LLMs.
• Proven track record of building evaluation suites for LLM or agent systems.
• Ability to analyze agent traces to identify and fix decision errors.
• Strong understanding of when to use rules, classic ML, humans, or LLMs based on cost and quality trade-offs.