Full-Stack Engineer, Agent Evaluation & Reliability
Join an AI team building infrastructure for agent evaluation and continuous improvement.
- San Francisco, California, United States
- Full-time
- On-site
- $120K–$180K per year
- Posted Nov 15, 2026
INDUSTRYSoftware & InternetAI & Machine Learning
Responsibilities
- Build backend systems and data models for evaluation datasets, experiment runs, agent traces, knowledge versions, findings, and customer-defined quality criteria.
- Develop and support enterprise integrations, including authentication, OAuth, webhooks, incremental syncs, backfills, rate limits, retries, and monitoring.
- Own features from technical design and implementation through deployment, monitoring, customer feedback, and iteration.
- Work with customers to understand their systems, diagnose issues, and turn specific needs into reusable product capabilities.
- Contribute across the stack, building product experiences that make underlying systems useful.
Requirements
- Experience in a Machine Learning
Nice to have: agent infrastructure, LLM evaluation, tracing, agent memory systems
Skills
TypeScriptNode.jsBunPostgreSQLjob queuesOAuthwebhooksincremental syncsbackfillsrate limitsmachine learning engineeringmodel development
More jobs nearby
Full description
True Talent Solutions Partners
Company details are for members
Sign up for a free job alert to see True Talent Solutions Partners's profile, website and LinkedIn, and the same for every employer on Referso Jobs.