As a Software Engineer, Cybersecurity Benchmarks, you will play a key role in evaluating the cybersecurity capabilities of advanced AI models through rigorous, reliable, and publicly accessible benchmarks. You will design, develop, and maintain evaluation frameworks while ensuring that testing environments remain secure against increasingly capable AI systems.
Compensation
Annual Salary: Competitive compensation ranging from $150,000 to $350,000, depending on experience and qualifications.
Responsibilities
Design, develop, and maintain cybersecurity benchmarks for evaluating AI capabilities within the organization's evaluation infrastructure, primarily using the Inspect library.
Develop and prototype new benchmark concepts, experiments, and evaluation methodologies to measure emerging AI cybersecurity capabilities.
Maintain and improve existing benchmark suites, ensuring evaluations can be executed reliably and efficiently whenever new AI models are released.
Design, implement, and strengthen secure sandboxing environments to prevent highly capable AI models from escaping their designated evaluation environments.
Monitor emerging developments in AI agents, coding harnesses, and their interactions with software environments to anticipate potential security vulnerabilities.
Requirements
Nice to have: Benchmarking AI systems
Nice to have: Capture the Flag (CTF) infrastructure
Partner with Support, Engineering, and other cross-functional leadership to define and drive the roadmap for technical support innovation and agility across the AI/ML product portfolio.
Identify customer issues, advocate for their needs with internal teams, including product and engineering teams, to find ways to deliver support and products.
Resolve technical troubleshooting and mitigating complex systemic problems faced by our customers.
Design the future state of technical support and customer experience by defining and advocating new standards for support processes, telemetry, tooling, and operational readiness.
Oversee support launch readiness and excellence for new AI/ML products and identify targeted opportunities to transform the support experience.
Requirements
Bachelor's degree in Science, Technology, Engineering, Mathematics, or equivalent practical experience
Nice to have: lead projects or initiatives from design to delivery
Nice to have: analyze and influence systems architecture and design patterns
Nice to have: grow in changing environments and requirements
Nice to have: lead design and implementation of AI-based solutions
Lead multiple engineering teams developing and deploying an enterprise LLM-powered platform that supports analysts, engineers, and operators across a variety of mission domains.
Compensation
Salary range: $205,000 - $280,000
Responsibilities
Solicit requirements and architect an enterprise agentic platform designed to power intelligent, autonomous workflows
Work directly with stakeholders, engineers, and cyber analysts to understand operational workflows and identify high-impact automation opportunities
Evaluate AI models, deployment architectures, and infrastructure options to determine the optimal balance of capability, performance, and cost
Advise customers on AI investment strategies and resource allocation to maximize mission impact and operational effectiveness
Stay current on emerging AI technologies, industry trends, and best practices, advocating for the adoption of innovative capabilities where appropriate
Lead and contribute to research programmes that shape our understanding of how frontier AI systems influence people, design and execute rigorous studies, publish work, and help set the research agenda for the team.
Annual salary is benchmarked to role scope and relevant experience. Most offers land between £65,000 and £145,000 made up of a base salary plus a technical allowance (take-home salary = base + technical allowance).
Responsibilities
Design and execute rigorous human-AI interaction studies
Publish research findings in academic venues
Lead author and co-author academic publications
Engage with government officials, regulators, and the research community
Develop datasets to monitor real-world risk exposure
Requirements
PhD (or equivalent years of research experience) in a relevant field
Nice to have: Published work related to societal impacts of AI
Nice to have: Experience fine-tuning language models
Nice to have: Experience with reinforcement learning
Nice to have: Experience with large-scale analysis of multi-turn conversations