benchmarksPublic
benchmarksPublic
mrmrs-eec7ec
mrmrs-eec7ec
Closed into main
4 commits
·
53 files
+23 added ~30 modified
·Seen by zach 1 month ago
M
AGENTS.md
@@ -61,8 +61,8 @@
6161
the latest results. Pick a target from the top; don't re-derive priorities6262
from raw JSONL.6363
2. Edit Oak in `../oak`, build it (`cargo build --release` there).64-
3. `python3 scripts/devloop.py` — one command, one verdict: PASS or REGRESSED65-
for your changeset vs the Oak baseline.64+
3. `python3 scripts/devloop.py` — one command, one verdict: PASS, REGRESSED,65+
or UNMEASURED for your changeset vs the Oak baseline.6666
4. Only claim a win that devloop's printed detection limit supports.6767
6868
Claim discipline (non-negotiable):
M
CONTEXT.md
Loading diff…
M
README.md
Loading diff…
A
config/agent-campaign-smoke.json
Loading diff…
M
config/command_semantics.json
Loading diff…
A
docs/adr/0010-agent-campaigns.md
Loading diff…
A
docs/agent-campaign-plan.md
Loading diff…
M
docs/agent-workflow-benchmarks.md
Loading diff…
M
docs/benchmark-workstation.md
Loading diff…
A
docs/gap-closure-2026-09-10.md
Loading diff…
A
docs/output-correctness-oracles.md
Loading diff…
A
docs/qa-convergence-2026-09-10.md
Loading diff…
A
docs/qa-follow-up-2026-09-11.md
Loading diff…
A
docs/qa-invariants-2026-09-12.md
Loading diff…
A
scripts/agent_campaign.py
Loading diff…
M
scripts/agent_metrics_schema.py
Loading diff…
M
scripts/agent_workflow.py
Loading diff…
M
scripts/bench.py
Loading diff…
M
scripts/crosscheck_hyperfine.py
Loading diff…
M
scripts/devloop.py
Loading diff…
M
scripts/mount_probe.py
Loading diff…
A
scripts/oakbench/agent_campaign.py
Loading diff…
A
scripts/oakbench/agent_environment.py
Loading diff…
A
scripts/oakbench/agent_execution.py
Loading diff…
M
scripts/oakbench/baseline_book.py
Loading diff…
A
scripts/oakbench/campaign_report.py
Loading diff…
M
scripts/oakbench/command_semantics.py
Loading diff…
M
scripts/oakbench/conformance.py
Loading diff…
M
scripts/oakbench/environment.py
Loading diff…
M
scripts/oakbench/execution.py
Loading diff…
M
scripts/oakbench/output_semantics.py
Loading diff…
M
scripts/oakbench/perf_loop.py
Loading diff…
A
scripts/oakbench/public_output.py
Loading diff…
M
scripts/oakbench/reporting.py
Loading diff…
M
scripts/oakbench/runlock.py
Loading diff…
M
scripts/oakbench/scorecard.py
Loading diff…
A
scripts/oakbench/workflow_oracles.py
Loading diff…
M
scripts/publish_gate.py
Loading diff…
M
scripts/workflow_ab.py
Loading diff…
A
tests/test_agent_campaign.py
Loading diff…
A
tests/test_agent_campaign_cli.py
Loading diff…
A
tests/test_agent_execution_limits.py
Loading diff…
M
tests/test_baseline_book.py
Loading diff…
A
tests/test_campaign_report.py
Loading diff…
M
tests/test_claim_eligibility.py
Loading diff…
M
tests/test_devloop.py
Loading diff…
M
tests/test_environment.py
Loading diff…
A
tests/test_measurement_environment_paths.py
Loading diff…
A
tests/test_output_exact.py
Loading diff…
A
tests/test_public_output.py
Loading diff…
M
tests/test_runlock.py
Loading diff…
M
tests/test_scorecard.py
Loading diff…
A
tests/test_workflow_oracles.py
Loading diff…