Software Engineer - Python/Typescript
Experienced, hands-on software engineers to help evaluate and improve AI coding models by reviewing generated code and agent behavior, determining technical correctness, and creating evaluation signals and feedback.
- Remote, United States +5 more
- Contract
- Remote
- Posted Nov 17, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise SoftwareAI & Machine LearningDeveloper Tools & Infrastructure
Locations (6)
- New York, New York, United States
- San Francisco, California, United States
- Seattle, Washington, United States
- North America
- Latin America
- India
Responsibilities
- Evaluate AI-generated code and solutions across real-world software repositories
- Review agent behavior, tool usage, and code changes for correctness and quality
- Identify technical errors, weak approaches, and recurring model failure modes
- Compare model outputs and explain why one solution is better than another
- Create and refine rubrics and evaluation criteria for coding tasks
- Produce high-quality evaluation and preference data used to improve coding models
- Build and maintain pipelines and infrastructure supporting data generation, collection, and evaluation workflows
- Synthesize findings from data work into clear write-ups, updates, and recommendations for the team
- Collaborate closely with researchers and engineers to translate qualitative judgment into scalable processes
- Share clear, actionable findings with AI researchers and engineers
Requirements
- Experience in a real-world codebases
Nice to have: LLM evaluation, coding agents, RLHF, preference data, rubric design, post-training
Skills
PythonTypeScriptJavaScriptGoCode-review skillsTechnical judgmentWritten communicationLLMsAI coding tools
More jobs nearby
Full description
Turing
Company details are for members
Sign up for a free job alert to see Turing's profile, website and LinkedIn, and the same for every employer on Referso Jobs.