Design, architect, and operate a distributed storage system that feeds model training and evaluation at petabyte scale, leading deployment across multiple Kubernetes clusters and collaborating with research teams to define data access patterns.
Locations (8)
New York, New York, United States
Toronto, Ontario, Canada
London, England, United Kingdom
San Francisco, California, United States
Montreal, Quebec, Canada
Paris, Île-de-France, France
Show all 8 locations
Berlin, Germany
Seoul, Seoul, South Korea
Responsibilities
Design, architect, and operate the distributed storage system that feeds model training and evaluation, driving the technical vision from concept through production.
Lead and own the deployment of this system across multiple Kubernetes clusters at petabyte scale, setting standards and best practices for the organization.
Collaborate with researchers and training‑infra teams to define how jobs read and write data, translating those patterns into throughput, latency, and durability requirements that shape the platform's roadmap.
Navigate the networking, I/O, and consistency challenges of moving large datasets and checkpoints across regions and backends, with GPU idle time and time‑to‑insight as the key metrics of success — and drive the solutions that improve them.