Log in
benchmarksPublic

Commit 5bd54e7b

· Browse files at this commit

5bd54e7b7233

Merged on mrmrs-1b7902
mrmrs · 1 month ago · 11 files +1 added ~10 modified
M AGENTS.md
@@ -71,9 +71,11 @@
7171 scenario/operation cell on its own interleaved A/A floor (`oak_installed_aa`),
7272 a bootstrap CI and Benjamini-Hochberg across the family, then re-measures
7373 screened cells (`--confirm-runs 20`, within `--confirm-budget-s`) and decides
74- them on fresh data (ADR-0011). Every no-change cell prints "no change
75- detected >= X%": that X is the effect you could have seen. Chasing a smaller
76- win? Raise `--runs`/`--confirm-runs` until X is below it, or the claim is noise.
74+ them on fresh data (ADR-0011). Every no-change cell prints "no change >= X%
75+ at 80% power": a true change of X% or more would have been detected at least
76+ 80% of the time (calibrated on real A/A rows). Smaller changes may pass
77+ undetected. Chasing a smaller win? Raise `--runs`/`--confirm-runs` until X is
78+ below it, or the claim is noise.
7779 - **Defaults are `--runs 5`; fewer, or `--skip-noise-floor`, is UNMEASURED
7880 (exit 2), never PASS.** An unconfirmed Oak-baseline regression is also
7981 UNMEASURED with the runs needed to confirm it. A Git-guardrail latency breach