Referso Jobs

Lead the design and delivery of agentic AI applications and the software that brings them into everyday use, combining applied AI, software development and architectural judgement.

Responsibilities

Design and build AI agents that can plan tasks, use tools and adapt to results
Write and review production code
Shape application architecture
Take AI applications from early prototypes through to production and ongoing support
Establish automated tests, agent evaluations and controlled release processes

Requirements

Nice to have: Azure
Nice to have: Databricks
Nice to have: TypeScript
Nice to have: JavaScript
Nice to have: Agent SDKs

Skills

Python Software Architecture Distributed Systems API Development Automated Testing Code Review SQL

Research Intern - Next-Generation AI Agents

Microsoft · Posted Nov 9, 2026

Open

Redmond, Washington, United States
Internship · Intern
$7K–$15K per month
1+ years

Apply with RefersoAI Apply on careers page

Research Interns put inquiry and theory into practice, collaborating with researchers and contributing to AI agent development.

Compensation

The base pay range for this internship is USD $6,810 - $13,480 per month. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $8,920 - $14,610 per month.

Responsibilities

Conduct research and author peer‑reviewed publications
Build and prototype AI agent systems
Evaluate and improve agent performance
Collaborate across research and product teams
Present findings and contribute to community life

Requirements

Doctorate Degree program in Computer Science, Software Engineering, Electrical Engineering, or related STEM field
Nice to have: At least 1 year of research experience
Nice to have: Publications in top venues such as NeurIPS, ICML, ICLR, ACL, EMNLP, NAACL, COLM, AAAI, IJCAI, KDD, or SIGIR

Skills

Machine learning NLP LLMs AI agents Agent memory

AI Engineer – Agent Evaluation Specialist

TMS LLC · Posted Nov 5, 2026

Open

Los Angeles, California, United States
Hybrid
Contract
14+ years

Apply with RefersoAI Apply on careers page

AI Engineer focused on evaluating LLM-powered agents, designing and running evaluation frameworks, and developing golden datasets.

Responsibilities

Design, implement, and run LLM-based evaluation frameworks to assess agent response quality, accuracy, relevance, and reliability.
Collaborate with Business Analysts and stakeholders to create, validate, and maintain golden datasets for agent evaluation benchmarks.
Instrument AI agents using Langfuse, including tracing execution, capturing evaluation scores, monitoring performance, and troubleshooting failures.
Evaluate agent behavior, tool usage, reasoning workflows, and response quality to ensure high standards.
Apply expertise in Agentic AI, AWS Bedrock AgentCore, Claude Agent SDK, Langfuse, and AI Observability & Evaluation.

Skills

LLM-based evaluation frameworks Golden dataset development Langfuse instrumentation Agent evaluation AWS Bedrock AgentCore Claude Agent SDK

Agentic AI Lead / Architect (Pharma) – Remote

Agentic AI Lead / Architect (Pharma) – Remote

Locations (5)

Responsibilities

Skills

AI /LLM Application Engineer

Responsibilities

Skills

Staff AI Engineer

Responsibilities

Requirements

Skills

Research Intern - Next-Generation AI Agents

Compensation

Responsibilities

Requirements

Skills

AI Engineer – Agent Evaluation Specialist

Responsibilities

Skills

Benefits

Agent Evaluation jobs

Locations (5)

Responsibilities

Skills

Responsibilities

Skills

Responsibilities

Requirements

Skills

Compensation

Responsibilities

Requirements

Skills

Responsibilities

Skills

Benefits