- New York City, New York, United States
- Full-time · Senior
- $164K–$235K per year
As a Research Scientist on the Agent Flywheel team, you will build the context layer for recursive self-improvement and autofix loops for agents: turning raw production session traces from Databricks’ data agents into online signals agents can optimize against.
Compensation
$164,000—$235,000 USD
Responsibilities
- Turn raw production session traces from Genie Code and Genie One into online signals agents can optimize against.
- Train and calibrate LLM judges so model feedback becomes reliable, repeatable signal, including calibration against expert human ratings.
- Convert multi-turn agent trajectories into datasets and reward signals for post-training, closing the loop between production signals, evaluation, and agent improvement.
- Design benchmarks and RL environments from real customer workloads that probe long-horizon, tool-using agent behavior.
- Advance evaluation research: online evals on live traffic, automated raters, verifiable rewards, and measurement that keeps pace with agents that act.
Requirements
- Nice to have: Agent systems
- Nice to have: Coding agents
- Nice to have: Long-horizon tool-using agents
- Nice to have: Building evaluation environments
- Nice to have: Learning from feedback
