Senior Data Scientist - ML Operations & Analytics

Filevine

PRAHA · CZ

—

Full-time

SENIOR

Apply Now

About the Role

Filevine, a Legal AI company, is seeking a Senior Data Scientist for ML Operations & Analytics to build a critical analytics layer for its AI systems. This role involves leading LLM evaluation initiatives, creating dashboards, and collaborating with ML teams to ensure operational excellence. Candidates need 3+ years in data science or ML operations, experience with LLM evaluation, and strong Python/SQL skills.

Responsibilities

Build ML operations analytics tracking pipeline health, costs, latency, and quality metrics across all services; Lead production LLM evaluation initiatives, designing custom metrics, quality monitoring and LLM observability in general; Drive development efforts in collaboration with different ML teams based on analytics outputs; Collaborate with our data annotation team on prompt engineering and evaluation dataset creation; Help with curating the Filevine dataset; Create actionable dashboards and insights for ML teams and leadership; Help define SLAs, cost optimization and failure prevention strategies for our ML pipelines.

Requirements

3+ years in data science, ML operations, or analytics engineering; Experience with LLM evaluation frameworks and quality metrics design; Strong Python, SQL, and data visualization skills; Knowledge of prompt optimization techniques; Deep understanding of latest trends in LLMs and their production deployments; Excellent communication skills in English to translate technical metrics into business insights; Collaborative mindset with ability to work across machine learning teams.