Current scoring: A subject receives 1 when the predicted label matches the reference, and 0 otherwise.
Current subjects and volunteer comparison methods
All 100 current campaign subjects are shown. GPT-6 Astra received the campaign tutorial with low reasoning and the optional field guide off. These are direct API responses from one run on 2026-10-01.
Each individual volunteer vote is scored directly against the frozen benchmark target label with the campaign's exact-match scorer. A subject's baseline is the mean over its volunteer items, and the campaign baseline is the subject-balanced mean of the per-subject means, matching how the model's campaign score averages one score per subject. Votes with excluded values are omitted before scoring: Image processing error.
# Volunteer Baseline Methodology — Infection Inspection ## Method Direct target scoring (`direct_target`). Each volunteer `Sensitive` or `Resistant` classification is scored against the subject's frozen laboratory `expected_response` with the same exact-match scorer used for the model. `Image processing error` classifications are excluded before scoring. This matches the Infection Inspection publication's volunteer-accuracy analysis and keeps the human comparison on the same binary phenotype decision required from the model. ## Why direct The target labels come from the strain's known antibiotic resistance and treatment concentration, not from the selected volunteers. Direct scoring is therefore non-circular; leave-one-out consensus is unnecessary. ## Aggregation For each subject, average the scored volunteer classifications. Then average the 100 subject scores so every subject has the same weight as it does in the model benchmark, regardless of its exact vote count. ## Selection distinction The initial 100-subject draw required an image-error vote fraction below `50%`. After expert review, the 95 approved subjects were frozen and five rejected images were replaced within the same Resistant/Sensitive target strata without applying an image-error vote threshold to the replacements. Volunteer phenotype agreement does not determine labels, difficulty, or selection within either target stratum.
Subject {{ s.id }}
How volunteers voted
{{ g.label }}
How this subject was scored
Structured model response
{{ s.raw }}