Designs, develops and deploys the evaluation and trust capabilities of Equinix's GenAI platform, building features for agent evaluation, trust controls, and root‑cause analysis.
Compensation
The targeted pay range for this position in the following location is / locations are:
Canada - Toronto Office TRO : 102,000 - 152,000 CAD / Annual
Responsibilities
Build platform features for evaluation: dataset creation and versioning, golden datasets, synthetic test data and reusable metric templates
Build scoring for agents: code‑based checks, LLM‑as‑judge, tool‑call accuracy and grounding checks
Build human‑review features that compare judge scores against human reviewers, and recalibrate when agreement is low
Run evaluations at scale: before release, on every change to a prompt, model or tool, and on agents in production
Build evaluation playbooks so agent builders can evaluate their agents without specialist help
Requirements
Bachelor's degree in computer science, engineering, data science, mathematics or a related field
Nice to have: evaluation or observability features used by many teams or at scale
Nice to have: LLM evaluation and observability tools
Nice to have: OpenTelemetry
Nice to have: red‑teaming or adversarial testing of LLM systems
Play a pivotal role in measuring the safety of Zoox’s autonomous driving technology, owning and scaling the pipeline behind behavioral estimates.
Responsibilities
Build and maintain the Behavior Evaluation data pipelines (PySpark/zspark medallion architecture on Databricks, EMR Serverless and Airflow) that produce weekly per-geofence estimates
Turn the statistical method (sampling, triage weights, mileage sharing, projections) into tested, reproducible code with checks that fail loudly when the data is bad
Build tools that cut the manual work in each weekly run
Work with cross functional stakeholders so the methodology is applied consistently in code and in clearance reports
Help adapt to new release methodologies and new geofences
Requirements
Nice to have: Autonomous-vehicle simulation
Nice to have: Safety metrics
Nice to have: Triage workflows
Nice to have: ML model production in data pipeline
Design and propose fraud models and rules end to end, build LLM pipelines, automate workflows with AI agents, and measure impact with statistical rigor.
Compensation
Competitive salary based on performance and experience
Chance of earning Klar stock options
30 days of Christmas bonus
Responsibilities
Design and propose fraud models and rules end to end
Detect and size new fraud patterns before they scale
Measure with statistical rigor and understand how each change affects fraud risk and the customer experience
Build LLM pipelines that turn unstructured text and documents into validated information for our models and monitoring
Automate workflows with AI agents, including validation, monitoring, prompt engineering, RAG systems, and A/B experimentation
Requirements
Bachelor's degree in a quantitative field (Data Science, Mathematics, Actuarial Science, Physics, Engineering, Economics, or related fields)
Provide database and analytical support for multiple initiatives, projects, and programs. Design, implement, and deliver reports, exception reports, measurements, and analyses to support customer-focused, data-driven analysis.
Responsibilities
Perform data analyses using appropriate statistical methods and graphical display standards
Retrieve and analyze data and present reports to management for use in supporting operations and business objectives
Provide recommendations for improvement and written documentation that relates to the analysis of the data
Develop in-depth knowledge of customer information systems and provide training and support to users
Analyze, summarize and present information and data for discussion and decision making
Requirements
Bachelor’s degree or equivalent combination of education and experience
Lead and conduct advanced modeling, simulation, and analysis for Joint Integrated Air and Missile Defense Organization (JIAMDO) and Department of War (DoW) customers.
Locations (2)
McLean, Virginia, United States
Huntsville, Alabama, United States
Responsibilities
Support modeling, simulation, and analysis of Blue and Red weapons systems
Use simulation tools, MATLAB, and statistical methods to analyze system trade-offs
Develop system performance metrics
Engage with customers to define requirements and set expectations
Lead identification of critical model elements
Requirements
Master's degree in engineering (aerospace, electrical, mechanical, or systems engineering)
Provide enterprise-level technical support to Amplitude customers, diagnosing and resolving issues, and advocating for customers across product and engineering teams.
Compensation
Based legislation in New York City, the following details are for individuals who will work for Amplitude in New York City: New York City salary range: $104,000 - $157,000 base salary (inclusive of bonus or commission)
Responsibilities
Answer and/or escalate all support tickets via email and chat
Drive resolution by digging in and escalating challenging tickets
Ensure that all tickets meet our service level agreements
Provide customers a great experience using Amplitude and working with our team
Create and update non-technical and technical documentation as the product evolves
Participate in scientific data reviews and contribute to analyses supporting study conclusions, publication activities, clinical evaluation activities, and evidence-generation initiatives.
Contribute to development, review, and maintenance of study-related documents including clinical study protocols, protocol amendments, case report forms, investigator brochures, informed consent forms, study manuals, clinical study reports, and associated scientific documentation.
Support the planning, execution, analysis, and reporting of prospective clinical studies, registries, observational studies, post-market studies, and other evidence-generation activities.
Conduct literature reviews, scientific assessments, competitive analyses, and evidence evaluations to support protocol development, endpoint selection, publication activities, clinical strategy discussions, and evidence-generation planning.
Collaborate with clinical operations and CRO partners to support study execution activities, enrollment monitoring, site communications, protocol compliance activities, issue identification, and study-risk mitigation efforts.
Requirements
Bachelor's degree in Life Sciences, Biological Sciences, Biomedical Sciences, Biomedical Engineering, Bioengineering, Public Health, Epidemiology, Nursing, Medicine, Pharmacy, or related scientific discipline.
Build and maintain the answer key, audit and calibrate reviewers, analyze quality data, recommend fixes, and report on quality metrics for Roblox’s content moderation operations.
Senior Software Engineer - Autonomy Safety MetricNewZoox · Foster City, US
Data & AI Scientist – FraudKlar Technologies SA de CV SFP · Mexico City, Mexico
M & S Operations Research Analyst, SeniorSimTech (Simulation Technologies, Inc.) · McLean, US+1 more
Research Engineering Manager, STEMTuring · US
Technical Support EngineerAmplitude · New York City, US
Machine Learning ScientistGogolook · Taipei, Taiwan
Quality AnalystRoblox · Gurugram, India