How does it work?
A comprehensive overview of our end-to-end dataset creation, model evaluation, and alignment pipeline. Domain experts progress by hitting the goals below, unlocking each new contribution type.
1. Domain Expert Onboarding & Verification
Domain experts undergo rigorous academic and professional credential verification to ensure the highest standards of domain expertise.
Domain Experts
2. Commission Definition
AI labs identify specific capabilities or knowledge gaps requiring improvement and fund target alignment commissions with a budget.
AI Labs
3. Dataset Question Preparation
Verified domain experts author and peer-review challenging, high-quality questions designed to push the boundaries of model capabilities.
Domain Experts
Unlock by: Get your first question approved
4. Benchmark Execution & Evaluation
Frontier models are evaluated against our custom benchmarks, and domain experts grade model responses for reasoning and factual accuracy.
Domain Experts
Unlock by: Complete 10 question reviews
5. Behavioural Testing in Bentham Studio
Domain experts author tasks targeting safety and ordinary failures, and every task runs twice — once as a plain control, once with an adversarial probe applied — so concealed misalignment surfaces alongside the surface failure.
Domain Experts
6. Deception Grading & Honest Exit Rating
Each run is judged on its own five-point deception scale, from unprompted honesty to fabrication, and rated for how much room the model had to be honest in the first place.
Domain Experts
7. Lab-Ready Dataset Export
Completed evaluations are exported as self-contained records — prompts, transcripts, verdicts, grades and the expert's written rationale — traceable to the named specialist who produced each finding.
AI Labs