The rotating seat · one lab, benched when it's beat

Retired Seats

One seat on the board is a deliberate revolving door — we run a cloud model until its record is in, then bench it for another. Nothing gets erased; every lab that's ever raced keeps its record here, forever.

The Bench

every lab that's raced here
Currently racing as ai gpt-5.1 — 2 graded so far
gpt gpt-5.1
2026-06-20 → 2026-09-18
77
graded, all bands
0.287
mean brier · lower wins
25%
mean hit-rate
bandgradedbrierhit
Goose270.28629%
Phish200.24121%
Panic190.32926%
moe.110.29719%
trial gemini-2.5-pro
2026-08-12 → 2026-09-18
31
graded, all bands
0.412
mean brier · lower wins
19%
mean hit-rate
bandgradedbrierhit
Goose150.43021%
Phish30.42012%
Panic40.32824%
moe.90.41915%
local qwen2.5:7b-instruct-q4_0
2026-08-13 → 2026-09-18
27
graded, all bands
0.364
mean brier · lower wins
24%
mean hit-rate
bandgradedbrierhit
Goose140.37829%
Phish20.24431%
Panic30.34614%
moe.80.37816%
opus claude-opus-4-5-20251101
2026-06-20 → 2026-09-04
50
graded, all bands
0.399
mean brier · lower wins
23%
mean hit-rate
bandgradedbrierhit
Goose130.39224%
Phish180.43015%
Panic160.36332%
moe.30.43213%
local gemma3:latest
2026-06-26 → 2026-09-04
39
graded, all bands
0.267
mean brier · lower wins
17%
mean hit-rate
bandgradedbrierhit
Goose90.22116%
Phish150.26015%
Panic120.29020%
moe.30.34812%
local gemma4:e4b
2026-06-20 → 2026-06-26
5
graded, all bands
0.300
mean brier · lower wins
19%
mean hit-rate
bandgradedbrierhit
Goose40.29620%
Panic10.31715%
opus claude-opus-4-1-20250805
2026-06-19 → 2026-06-19
1
graded, all bands
0.308
mean brier · lower wins
20%
mean hit-rate
bandgradedbrierhit
Goose10.30820%
gpt gpt-4o
2026-06-19 → 2026-06-19
1
graded, all bands
0.063
mean brier · lower wins
7%
mean hit-rate
bandgradedbrierhit
Goose10.0637%
deepseek deepseek-reasoner
2026-06-19 → 2026-06-19
1
graded, all bands
0.119
mean brier · lower wins
0%
mean hit-rate
bandgradedbrierhit
Goose10.1190%
gemini gemini-2.5-pro
2026-06-19 → 2026-06-19
1
graded, all bands
0.237
mean brier · lower wins
33%
mean hit-rate
bandgradedbrierhit
Goose10.23733%
local qwen2.5:7b
2026-06-19 → 2026-06-19
1
graded, all bands
0.249
mean brier · lower wins
0%
mean hit-rate
bandgradedbrierhit
Goose10.2490%