- Shanghai, China
- On-site
- Full-time · Senior
- CN¥341K–CN¥1M per year
Design and evolve the large-scale offline ML platform, focusing on building reliable infrastructure for generating training datasets, orchestrating ML workflows, and enabling efficient, distributed model training at scale.
Compensation
Base Salary Range: CNY ¥341,000- ¥1,247,000 gross
Responsibilities
- Design and operate large-scale data pipelines that generate training datasets used for machine learning training and experimentation
- Develop infrastructure that supports distributed training workflows using technologies such as Pytorch, Ray Data, and Ray Train, etc.
- Integrate ML pipelines with workflow orchestration systems (e.g., Flyte, Airflow, or similar) to enable reliable multi-stage training workflows
- Improve reproducibility and observability of ML pipelines through dataset validation, monitoring, and automated testing
- Optimize performance and resource utilization across distributed compute systems used for data processing and model training
Requirements
- Nice to have: Pytorch
- Nice to have: Ray Data
- Nice to have: Ray Train
- Nice to have: Flyte
- Nice to have: Airflow