benchmarksPublic
benchmarksPublic
Commit fcd8a785
M
docs/qa-follow-up-2026-09-11.md
@@ -1,5 +1,11 @@
11
# Benchmark QA follow-up — 2026-09-1122
3+
Historical snapshot: review rejected `2735d7f5558e`. In particular, the claim4+
below that non-trusted workflows retained other correctness publication checks5+
was wrong; failed workflow summaries could publish. Cache validation also6+
admitted invalid trials. See [the next correction](qa-invariants-2026-09-12.md)7+
for the replacement invariants and new producer-level evidence.8+
39
This review corrects defects found in `mrmrs-eec7ec` at `2310099b696d`.410
Main remains `b297e8a25d08`. The branch stays open for independent re-review;511
this document is not a merge receipt.
A
docs/qa-invariants-2026-09-12.md
Loading diff…
M
scripts/agent_metrics_schema.py
Loading diff…
M
scripts/agent_workflow.py
Loading diff…
M
scripts/bench.py
Loading diff…
M
scripts/crosscheck_hyperfine.py
Loading diff…
M
scripts/devloop.py
Loading diff…
M
scripts/mount_probe.py
Loading diff…
M
scripts/oakbench/agent_campaign.py
Loading diff…
M
scripts/oakbench/campaign_report.py
Loading diff…
M
scripts/oakbench/environment.py
Loading diff…
M
scripts/oakbench/output_semantics.py
Loading diff…
M
scripts/oakbench/perf_loop.py
Loading diff…
M
scripts/oakbench/public_output.py
Loading diff…
M
scripts/oakbench/reporting.py
Loading diff…
M
scripts/oakbench/workflow_oracles.py
Loading diff…
M
scripts/workflow_ab.py
Loading diff…
M
tests/test_agent_campaign.py
Loading diff…
M
tests/test_agent_campaign_cli.py
Loading diff…
M
tests/test_agent_execution_limits.py
Loading diff…
M
tests/test_campaign_report.py
Loading diff…
M
tests/test_devloop.py
Loading diff…
A
tests/test_measurement_environment_paths.py
Loading diff…
M
tests/test_output_exact.py
Loading diff…
M
tests/test_public_output.py
Loading diff…
M
tests/test_workflow_oracles.py
Loading diff…