As a Research Scientist on the Agent Flywheel team, you will build the context layer for recursive self-improvement and autofix loops for agents: turning raw production session traces from Databricks’ data agents into online signals agents can optimize against.
Compensation
$164,000—$235,000 USD
Responsibilities
Turn raw production session traces from Genie Code and Genie One into online signals agents can optimize against.
Train and calibrate LLM judges so model feedback becomes reliable, repeatable signal, including calibration against expert human ratings.
Convert multi-turn agent trajectories into datasets and reward signals for post-training, closing the loop between production signals, evaluation, and agent improvement.
Design benchmarks and RL environments from real customer workloads that probe long-horizon, tool-using agent behavior.
Advance evaluation research: online evals on live traffic, automated raters, verifiable rewards, and measurement that keeps pace with agents that act.
Lead architecture and delivery of scalable merchant onboarding and activation systems, partner with product, design integrations, build agent-enabled experiences, use AI to accelerate development, improve AI harness, raise technical bar through direction and coaching.
Locations (4)
San Francisco, California, United States
Los Angeles, California, United States
Seattle, Washington, United States
New York, New York, United States
Compensation
$208,454—$364,795 USD
Responsibilities
Lead the architecture and delivery of scalable, MCP first systems for merchant onboarding, activation and growth experiences.
Partner with Product to shape a growth roadmap, identify high value opportunities and convert the strongest concepts into production experiences.
Design integrations that apply high quality merchant signals and personalized recommendations across Pinterest surfaces and approved external agent ecosystems.
Build agent enabled experiences that improve merchant readiness and campaign creation, with clear measurement of their contribution to business impact.
Use AI to accelerate prototyping, analysis and iteration while applying judgment and verification to ensure quality, reliability and user trust.
Requirements
Bachelor's degree in computer science or equivalent work experience
