Experiment register · September 13–24, 2026
Ten beats: what changed, what moved, and what we had to stop claiming. Each result belongs to its own test, not one continuous performance curve.
12 seconds per beat. Pause to read, or choose a date.
The call dispatches internally. Naming the record on the surface removes a routing decision the agent had been getting wrong.
Zero mis-routes across 1,631 dispatches. Still the largest single improvement in this record.
Separately, four question-blind rules each scored perfectly. The corpus was rebuilt around records that exist but lack the requested field.
Refusing everything had scored 0.667 against the attached arm’s 0.646. Repetitions and the corpus rebuild repaired different weaknesses in the instrument.
Each requested item must be present, receipted, correct and sourced. Refusing everything cannot earn completeness.
The cases and reference answers are tied to a named world build. This introduces a test; it is not a measured harness gain.
Models fail at different gates. A critic reading the progress ledger lowers completeness on both clean model pairs.
The ladder targeted eight models. Two model runs were excluded from comparison for faults; two additional models could not run because the provider rejected temperature zero.
A vote chooses between two models’ answers. It cannot reliably identify when to switch away from the stronger model.
1,400 observations, 11,200 verdicts. No threshold from one to eight reaches the stronger model alone.
Check the figure against its payload deterministically, then ask one typed question. Discrimination improves at one eighth the calls and one fourteenth the tokens.
A better critic score and lower cost did not establish better answer selection. This is not evidence that the critic learned nothing.
A delivery tool blocks figures that no payload served. Sometimes the model deletes a correct item instead of fetching its evidence.
The block-fetch-redeliver loop closed 30 times in 31. That local success did not preserve completeness.
Same model, cases and world package; different runtime tool listing. The surface must be a registered variable.
Do not carry a comparison across binaries. A shared data package does not establish a shared experimental configuration.
Calibrate acceptance bands from saved runs. Separately, rescoring the same answers moves correctness about five points after a number-matcher change.
Leakage differences in the record sit inside the estimated noise. These bands are provisional: resampling saved runs does not replace fresh, independent runs of an unchanged configuration.
Hide ten tools that no prior measured run called. The data stays the same. The gains sit in the two failing question families.
Gemini 2.5 Flash, 280 questions, five repetitions. The control lost 10% of rows to runtime failures; excluding lost rows left the arrival gain almost unchanged. This is a separate comparison from the front door.
The front door and deletion. Both simplify the route to evidence.
The ledger critic and delivery gate cost completeness. The vote costs selection accuracy.
A rebuilt corpus, five repetitions, and a calibrated noise band.
The runtime surface moved with fixed data. The scorer moved with fixed answers.
A check has to improve the work, not just the quantity it checks.
These findings overlap the ten beats; they are not ten independent interventions. The sharper critic improved discrimination and efficiency, but no routing gain was established.