- Livermore, California, United States
- On-site
- Contract · Associate
- $143K–$143K per year
Postdoctoral Researcher to perform research in data provenance, multi-layer tracing, and I/O characterization for large-scale scientific and AI workflows on high-performance computing (HPC) systems.
Compensation
$143,328 Annually
Responsibilities
- Research and develop low-overhead, multi-layer provenance and I/O capture techniques for HPC workflows spanning the application, runtime, I/O library, file system, and operating system layers.
- Research and develop methods to integrate capture with HPC resource managers (e.g., Flux, Slurm) so that provenance is collected automatically without requiring changes to user workflows.
- Design and implement scalable registries that assign stable identities to datasets, model checkpoints, and workflow stages and reconcile the names used for them across different tools.
- Research and develop techniques to normalize, compress, and temporally align high-volume heterogeneous trace and event streams into a common schema.
- Characterize and minimize the performance overhead of tracing and provenance capture on large-scale deep learning and scientific workloads.
Requirements
- PhD in Computer Science
- PhD in Computer Engineering
- Nice to have: Data provenance
- Nice to have: Lineage tracking
- Nice to have: Reproducibility of scientific or machine learning workflows