- Berlin, Germany +1 more
- Full-time · Senior
- 5+ years
Participate in the development, curation, and scaling of the core data asset pipeline, transforming raw web-scale data into high-signal datasets used to train the Scientist AI.
Locations (2)
- Berlin, Germany
- Montreal, Quebec, Canada
Responsibilities
- Partner with the Research team to define, build, automate, scale, and manage data pipelines that transform raw web-scale data into training datasets for the Scientist AI.
- Build and maintain data processing pipelines, including deduplication, model-based quality scoring, heuristic filtering, toxicity removal, PII scrubbing, metadata extraction, and proprietary data transformations, with full dataset versioning and provenance tracking, optimizing for throughput and cost at scale.
- Ensure all ingested data meets compliance requirements, internal Data Governance policies, and legal obligations.
- Develop and refine the scoring and filtering toolchain: heuristics, LLM-as-a-judge evaluators, ML classifiers, metadata extraction modules, and human-in-the-loop review workflows required for data processing and quality assurance.
.png?1748917260)