Senior Data Search Engineer (Data Scientist)

Sourced from cz.linkedin.com

IDC is building the next generation of AI-powered intelligence platforms. The Senior Data Search Engineer will automate ETL pipelines, build embedding pipelines for semantic search, and develop document processing infrastructure that powers customer uploads. This role ensures accurate retrieval for AI agent queries.

• Automate and extend ETL pipelines that ingest, transform, and deliver IDC's research content to the platform. Build embedding pipelines that power semantic search across the platform's content library. Develop document processing and extraction capabilities for customer uploads, including text extraction, chunking, and citation anchor generation. Improve search relevance through indexing strategies, ranking tuning, and retrieval optimization. Build data infrastructure that supports AI agent queries with accurate, well-structured results. Monitor pipeline performance and data quality, identifying and resolving issues before they affect users.

• 3+ years of experience in data engineering, search engineering, or a related field. Experience building and maintaining ETL pipelines and working with data warehouses. Familiarity with semantic search concepts and embedding-based retrieval. Proficiency in Python and SQL. Experience with document processing, text extraction, or content transformation. A strong data quality mindset.

• Experience with Snowflake or Snowflake Cortex. Hands-on experience with vector databases. Background in information retrieval or natural language processing. Experience implementing access control at the data layer. Familiarity with data quality monitoring and pipeline observability tools.