Project Lion - Senior Prompt Engineer - Italy (Remote, Part-Time)

This role is part of Project Lion, focused on migrating templates to LLM autoraters using prompt engineering techniques. The Senior Prompt Engineer will use internal APG/APO tools, manually refine prompts for complex edge cases, and evaluate model performance against gold data. The position is remote, part-time, and requires native Italian.

• Utilize Automatic Prompt Generation (APG) tools to create baseline prompts for complex parent-child template clusters.
• Run and supervise Automated Prompt Optimization (APO) tools, review outputs, and flag deadlocks or plateaus.
• Manually draft, test, and refine prompts to navigate complex template architectures and handle edge cases.
• Monitor shadowbot runs to ensure sufficient disagreements between human and LLM ratings are registered and tracked.
• Run prompt versions against gold data to measure autorater quality, calculating F1, precision, and recall.
• Draft technical launch readiness justifications (Launch Certification Documentation).

• Native fluency in Italian and fluent English.
• Based in Italy.
• Bachelor's, Master's, or Doctorate in Computer Science, Data Science, Computational Linguistics, HCI, Cognitive Science, or related field.
• At least 4 years of experience as a Prompt Engineer.