Search
Go
This job is no longer accepting applications
This listing was closed on Aug 12, 2026.
Browse Open JobsPrincipal Data Scientist - Agent Builder
Summary by AIEngineer.careers
Principal Data Scientist - Agent Builder is a Data Scientist role (full-time). with Elastic. in REMOTE, WORLDWIDE. Compensation shown: $87K–$137K (est.). Imported listing (source: jobs-radar.com). Apply on the employer's site (jobs-radar.com).
Imported listing (source: jobs-radar.com) · Apply on jobs-radar.com
REMOTE · WORLDWIDE
$87K–$137K (est.)
Full-time
PRINCIPAL
The sections below reproduce the third-party job description for reference. AIEngineer.careers does not write or control this text.
Imported job description
Sourced from jobs-radar.com
Elastic is building a conversational and agentic platform for searching data. The Principal Data Scientist will define evaluation strategies for RAG and agent systems, influence product direction, and mentor teams. This is an applied leadership role working on cutting-edge AI/ML problems.
Responsibilities
- Define evaluation strategy for conversational/agentic search, including offline/online evaluation, golden datasets, LLM-as-judge, A/B testing.
- Lead quality metrics and decision frameworks for RAG, agents, model selection, routing.
- Improve retrieval and re-ranking approaches.
- Partner with engineering to productionize evaluation pipelines.
- Influence roadmap and mentor data scientists.
Requirements
- 8+ years of applied DS/ML experience with deep expertise in IR, NLP, ranking, semantic search, RAG or LLM products.
- Strong track record in evaluation for production AI/ML systems.
- Experience influencing product strategy through data.
- Hands-on with Python, PyTorch, Transformers, Pandas.
- Understanding of retrieval systems and metrics (nDCG, MRR, Recall@k).
- Experience with Elasticsearch or similar distributed data systems.
Elastic
Amsterdam (registered) and San Francisco (operational) · NL · 13556+ employees
Elastic is a Dutch-American software company that provides a Search AI platform for enterprise search, observability, and cybersecurity. It is known for its open-source foundation, including Elasticsearch, and helps organizations find answers in real-time from their data at scale.
