Referso Jobs

Member of Technical Staff, Reinforcement Learning

True Talent Solutions Partners · Posted Nov 9, 2026

Open

Remote, Canada +1 more
Remote
Full-time · Mid
$250K–$400K per year

Member of Technical Staff, Reinforcement Learning

True Talent Solutions Partners · Posted Nov 9, 2026

Open

Remote, Canada +1 more
Remote
Full-time · Mid
$250K–$400K per year

AI Engineer – Agent Evaluation Specialist

TMS LLC · Posted Nov 5, 2026

Open

Los Angeles, California, United States
Hybrid
Contract
14+ years

Apply with RefersoAI Apply on careers page

AI Engineer focused on evaluating LLM-powered agents, designing and running evaluation frameworks, and developing golden datasets.

Responsibilities

Design, implement, and run LLM-based evaluation frameworks to assess agent response quality, accuracy, relevance, and reliability.
Collaborate with Business Analysts and stakeholders to create, validate, and maintain golden datasets for agent evaluation benchmarks.
Instrument AI agents using Langfuse, including tracing execution, capturing evaluation scores, monitoring performance, and troubleshooting failures.
Evaluate agent behavior, tool usage, reasoning workflows, and response quality to ensure high standards.
Apply expertise in Agentic AI, AWS Bedrock AgentCore, Claude Agent SDK, Langfuse, and AI Observability & Evaluation.

Skills

LLM-based evaluation frameworks Golden dataset development Langfuse instrumentation Agent evaluation AWS Bedrock AgentCore Claude Agent SDK

Founding AI Platform Engineer (MLOps / Backend)

Gamingtec · Posted Mar 23, 2026

Open

Apply with RefersoAI Apply on careers page

Founding AI Platform Engineer to own systems for ML and GenAI products

Responsibilities

Build and maintain infrastructure and tooling for training, evaluation, deployment, and monitoring of ML models and GenAI services
Own production services, APIs, and pipelines powering recommendations, agent workflows, and customer-facing integrations
Improve CI/CD, testing, release workflows, rollback processes, and environment management
Establish observability across service health, model behaviour, agent quality, latency, cost, and failure modes
Build reproducibility and lifecycle practices for models, prompts, datasets, configurations, and releases

Skills

Python backend services cloud infrastructure CI/CD testing observability automation MLOps workflows model training evaluation deployment monitoring experimentation platforms event pipelines analytics instrumentation feature delivery platforms

See full job description

Senior AI Platform Engineer

Accenture Infrastructure and Capital Projects, LLC · Posted Nov 3, 2026

Open

Lisbon, Portugal, Portugal +2 more
Hybrid
Senior
8+ years

Apply with RefersoAI Apply on careers page

Design, build, and scale an AI-driven engineering platform to improve developer productivity across the SDLC.

Locations (3)

Lisbon, Portugal, Portugal
Braga, Portugal, Portugal
Coimbra, Portugal, Portugal

Responsibilities

Design and implement agentic development workflows, tools, and automation across the SDLC
Develop and integrate production-ready AI Agents using orchestration, tool use, MCP, and context engineering
Build evaluation frameworks, guardrails, automated quality checks, and monitoring mechanisms to ensure reliable agent behavior
Design and optimize CI/CD pipelines, infrastructure automation, and developer tooling
Integrate AI-powered workflows with enterprise engineering platforms, including Jira, source control systems, and CI/CD platforms

Member of Technical Staff, Reinforcement Learning

Member of Technical Staff, Reinforcement Learning

Locations (2)

Compensation

Responsibilities

Requirements

Skills

AI Engineer – Agent Evaluation Specialist

Responsibilities

Skills

Founding AI Platform Engineer (MLOps / Backend)

Responsibilities

Skills

Senior AI Platform Engineer

Locations (3)

Responsibilities

Requirements

Skills

Agent evaluation jobs

Locations (2)

Compensation

Responsibilities

Requirements

Skills

Responsibilities

Skills

Responsibilities

Skills

Locations (3)

Responsibilities

Requirements

Skills