Software Engineer, Pretraining Data

Sourced from recentlypostedjobs.com

Build the data systems behind frontier model pretraining, spanning large-scale web crawling, data platform, and pipeline infrastructure. Turn raw web, code, and multimodal dumps into training-ready datasets while making researcher iteration fast, reliable, and observable. Own data quality as both a systems and research problem, shipping models that classify, rank, and filter data at extreme throughput.

• Build the platform that turns raw web, code, multimodal, and acquired data into training-ready datasets for frontier pretraining runs
• Own pipelines, orchestration, and tooling that make pretraining data iteration fast, reliable, observable, and reproducible at scale
• Build high-throughput, fully telemetered data pipelines with end-to-end traceability
• Train and ship models that classify, rank, filter, clean, and identify data at extreme throughput
• Design and run scaling-ladder experiments on data mixture, repeatability, and quality depth
• Build and scale web crawling systems that discover, schedule, fetch, and parse high-quality documents
• Debug and harden crawl infrastructure for availability, recovery, and ingestion lag

• Minimum of 3 years of software engineering experience
• Experience building distributed systems and/or large-scale data infrastructure

• High-slope engineer with staff-level ownership within a few years or relevant domain experience
• Able to architect and ship end-to-end with high ownership and debug complex systems independently
• Strong intuitions about large-scale distributed systems
• Excited to learn how pretraining data shapes model quality