- San Francisco
Data Scientist focused on building, evaluating, and improving mitigation systems to prevent catastrophic harms from AI models.
Responsibilities
- Evaluate and improve mitigation systems, including classifiers and detection pipelines across domains
- Diagnose false positives and false negatives with deep error analysis, root cause investigation, and clear recommendations for mitigation adjustments
- Build monitoring and measurement frameworks to track mitigation effectiveness over time and across user segments and use cases
- Identify trends in over-blocking vs. under-blocking, quantify customer impact, and propose prioritized interventions
- Develop insights from customer feedback, complaints, and usage patterns to detect shifts in adversarial behavior and system failure modes
Requirements
- Nice to have: Cybersecurity data science