Define the mission and the denominator
An entire institution, a department, or a single project can take part. Identify an accountable owner, the observation period, core tasks, acceptance criteria, and sources for the task ledger. The pilot tool requires every task due in that period. Fix the value streams and workload weights in advance. For a research project, acceptance might mean successful independent reproduction. For a public service, it might mean resolving a case and passing a quality check.
End-to-end coverage = total weight of due tasks that AI completed end to end and that passed operational acceptance ÷ total weight of all tasks due in the observation period.
Give comparable tasks equal weight. Where complexity varies, use baseline labor hours set in advance and disclose each value stream’s task count and unweighted result. Include failed, overdue, unknown-outcome, and human-takeover tasks in the denominator. Record mandatory authorization or final sign-off separately. Count it as human intervention when a person redoes a substantive judgment, revises the deliverable, or coordinates intermediate steps.
The full ledger describes this observation period. Sampling is a planned extension. If only a sample is available, fix the sampling frame, strata, and random seed in advance. Report inclusion rates and omission risks by stratum. For comparable, independent, equally weighted random observations, report a binomial proportion interval. Account for clustering when several tasks share a client or project. Defer promotion when the interval crosses the relevant threshold. Statistical inference from samples needs separate analysis.
Evidence for the two thresholds
G2 · Operational acceptance
Most core mission work must have a record of end-to-end delivery. The pilot defines “most” as more than half by weight and requires records for every declared core value stream. Fix the scope, weights, and acceptance criteria before seeing results. Reviewers trace the start of each task, changes in who performed it, human interventions, and the final operational state. Report human review time, total cost, and quality incidents so readers can understand the cost of reaching a given level of coverage.
G4 · Improvements that carry forward
G4 builds on the complete projects and process improvements required at G3. The pilot requires two improvement cycles addressing different problems, each with distinct held-out tasks. Trace how AI identified the problem, developed a change, implemented it within its authority, and retested it on subsequent independent tasks. Keep the previous version, unsuccessful alternatives from the same period, a prespecified minimum gain and quality floor, and evidence that a change could be rejected, stopped, or rolled back. Compare both methods on the same new cases. Document changes in model, budget, and task difficulty. If a supplier upgrade and organizational learning could both explain a gain, disclose the uncertainty.
For example, a nonprofit delivery project’s AI finds that missed deliveries cluster around last-minute address changes and proposes a new verification step. The next shift replays both versions against newly due orders, comparing missed deliveries and on-time arrivals. The results show whether the improvement carries forward.
G5 · Research challenge
Test whether the improvement mechanism itself has become better and whether, in a new task domain, it can raise the success rate of improvements using comparable resources. G5 remains a research target.
Already operating without a task ledger?
An agent can reconstruct tasks from requests, work records and receipts, including stalled and human-led work. Each task has four stages: preparation, execution, checking and delivery. The report estimates the AI share as a range and shows how task weights can change it. A lower bound above 50% supports a retrospective G1 estimate. This pilot issues no retrospective badge.
Download the evaluator and worked example
Case 01 · Launch team · How GO Benchmark assessed its own launch
Make agent judgments open to challenge
- Evidence collection agent
- Use a common template for records within the authorized scope. Preserve evidence references, unexplained records, version information, and human interventions. Document gaps and contradictions as found.
- Reproducible scoring
- Classify evidence separately from calculating the score. The same inputs and rule version should yield the same result. File digests reveal changes to content. Record providers and publishers are responsible for authenticity.
- Neutral review
- For deliverables requiring judgment, conceal supplier, model, and submitter identities where practical. Vary presentation order and have reviewers score independently against a fixed rubric. Record model versions, prompts, reasoning, and human review. Where reviewers materially disagree, publish the point of disagreement and retain the more conservative conclusion.
- Publication and corrections
- Publish the self-assessment badge with a report identifying the unit, scope, evidence period, and method version. Publish redacted evidence summaries and retain the originals within the organization. Issue a correction linked to the prior version if a sampling or classification error emerges. Third parties can state which evidence they independently reviewed.
Sources behind the method
GDPval uses real professional deliverables and blinded peer review. GO adapts its task design and also requires the organization’s actual acceptance result. GDPval overview
τ-bench checks the final database state after an interaction and evaluates reliability across repeated trials. GO verifies delivery using the operational system’s final state and reports repeated replays separately from production observations. Official τ-bench repository
METR characterizes task length by how long people need to complete comparable work. GO recommends reporting baseline labor hours, elapsed time, and takeover frequency to help interpret project difficulty. METR Time Horizons
A review of SWE-bench Verified identified grading problems in some tests and potential prior exposure to public tasks. GO recommends holding out future cases, correcting scoring errors, and preserving the revision history. SWE-bench Verified review
The Darwin Gödel Machine evaluated self-modifications on downstream tasks and documented an agent changing a detection mechanism. For formal verification, GO requires evaluation evidence independent of the system under assessment. Darwin Gödel Machine
Calibrating the next version
Preregister pilots in business, nonprofit, research, or government projects. Have two independent reviewers rate the same de-identified evidence. Report sample sizes by level, a confusion matrix, weighted kappa, and its uncertainty. Compare agent judgments with operational owners’ checks and examine where errors cluster by task or supplier.
Use subsequent deliveries to test whether ratings predict completion and human takeover on new tasks. Vary weights, observation periods, and thresholds separately to identify which conclusions change. Use calibration results to decide whether to revise the next version’s rules. Label pilot results and published conclusions with the corresponding version of the standard.