Full-Stack Engineer, Agent Evaluation & Reliability
True Talent Solutions Partners · Posted Nov 15, 2026
- San Francisco, California, United States
- On-site
- Full-time
- $120K–$180K per year
- 2+ years
- Visa sponsorship
Join an AI team building infrastructure for agent evaluation and continuous improvement.
Compensation
Base salary: $120,000 to $180,000 annually.
Responsibilities
- Build backend systems and data models for evaluation datasets, experiment runs, agent traces, knowledge versions, findings, and customer-defined quality criteria.
- Develop and support enterprise integrations, including authentication, OAuth, webhooks, incremental syncs, backfills, rate limits, retries, and monitoring.
- Own features from technical design and implementation through deployment, monitoring, customer feedback, and iteration.
- Work with customers to understand their systems, diagnose issues, and turn specific needs into reusable product capabilities.
- Contribute across the stack, building product experiences that make underlying systems useful.
Requirements
- Nice to have: agent infrastructure
- Nice to have: LLM evaluation
- Nice to have: tracing
- Nice to have: agent memory systems