Senior Software Engineer to help build the core libraries, evaluation systems, and reusable harness capabilities that make AI agents safer, faster, and easier for developers to trust.
Locations (2)
United States, United States
Canada, Canada
Compensation
The expected base salary ranges for this position are: 184,000 USD – 287,500 USD for Level 4; and 224,000 USD – 356,500 USD for Level 5.
Responsibilities
Define and evolve the technical approach for evaluating and improving open agent harnesses across frameworks, models, and benchmark environments.
Build and operate scalable, reproducible evaluation systems spanning CI, scheduled compute, sandboxed execution, artifact provenance, trace capture, and analysis.
Diagnose agent failures from traces, logs, tool results, and benchmark artifacts, separating harness, evaluator, environment, inference, and model causes.
Design controlled experiments and promotion criteria that distinguish genuine harness improvements from noise, benchmark artifacts, or model-specific gains.
Ship focused harness and runtime improvements, including upstream open-source contributions, detectors, regression tests, and maintainable documentation.
Requirements
Bachelor’s or Master’s degree in Computer Science, Artificial Intelligence, Applied Math, or a related field, or equivalent experience
Nice to have: Meaningful contributions to open-source coding agents, harnesses, agent runtimes, evaluation frameworks, developer tools, or observability projects.
Nice to have: Published research, technical writing, patents, or substantive open-source work in autonomous software engineering, agent evaluation, reliability, tool use, or inference efficiency.
Nice to have: Experience with SWE-bench, Terminal-Bench, AgentBench, or comparable public agent benchmarks.
Nice to have: Shipped improvements to context management, tool interfaces, retries, sandboxed repository execution, long-running agent loops, or evaluation environments.