Rubin Alert Explorers

All 100 current campaign subjects are shown. GPT-6 Astra received the campaign tutorial with low reasoning and the optional field guide off. These are direct API responses from one run on 2026-10-01.

Each individual volunteer vote is scored directly against the frozen benchmark target label with the campaign's exact-match scorer. A subject's baseline is the mean over its volunteer items, and the campaign baseline is the subject-balanced mean of the per-subject means, matching how the model's campaign score averages one score per subject.

# Volunteer Baseline Methodology — Rubin Alert Explorers

## Method

Direct scoring (`direct_target`): each volunteer real-vs-artefact vote is scored
against the frozen expert label with the same exact-match scorer used for the
model.

## Why direct

The active benchmark targets come from the independent expert review rather
than from the volunteer votes being scored. Leave-one-out consensus is therefore
unnecessary and would answer a different question.

## Data representation

The blinded-review provenance freezes the valid vote count for each answer on
each selected subject. `volunteer_labels/` expands those counts into individual
binary vote items for baseline scoring and explicitly records that reconstruction.
For a direct exact-match baseline, vote order and volunteer identity do not
affect the result.

The cohort is balanced at 50 expert-labelled Real and 50 expert-labelled
Artefact subjects.