Content Engineer
Content Engineer
- Menlo Park, California, United States +1 more
- On-site
- Posted Nov 14, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise Software
Locations (2)
- Menlo Park, California, United States
- New York, New York, United States
Responsibilities
- Define and apply quality criteria for assigned AI product capabilities
- Build and maintain guidelines, golden response sets, and benchmarks across features and surfaces
- Develop failure-mode taxonomies that turn user-facing quality issues into structured, prioritized findings for technical partners
- Design and run human evaluation workstreams, including rubric development, annotator guidance, calibration, and analysis of results
- Partner with research science to support automated evaluation and validate automated quality signals against human ratings
- Track quality trends and regressions across model versions and product capabilities
- Run structured internal testing programs, design test plans targeting specific failure modes, triage results, and deliver prioritized recommendations to product and engineering
- Craft and refine system prompts and agentic behavior in partnership with technical teams to support product goals and model outcomes
- Partner with engineering, research science, product, policy, and design teams on quality standards and priorities for assigned product areas
- Communicate evaluation findings clearly, align partners on next steps, and independently deliver workstreams through influence and collaboration
- Use AI-native tools to build scalable, repeatable processes for evaluation design, data analysis, system prompting, and reporting
- Experiment with emerging tools, validate their outputs, and document workflows that improve quality and reduce manual work
Requirements
- Bachelor's degree in a directly related field
Skills
human evaluation workflowsrubric or guideline developmentgolden set constructioncalibrationqualitative content or user-experience feedback translationstructured testing programsanalysis of resultseditorial and content-quality decisionsprompt engineeringagentic workflowsAI-native toolingLLM-based development toolsannotation platformsprototyping environmentsresponsible AI practicesbias mitigationquality and accuracy reviewsLLM-as-judge developmentgenerative AI trainingmodel evaluationhuman-annotated or synthetic datasetsLLM-based development toolsannotation platformsprototyping environments
More jobs nearby
Full description
MetaCompany details are for members
Sign up for a free job alert to see Meta's profile, website and LinkedIn, and the same for every employer on Referso Jobs.