benchmarksPublic
benchmarksPublic
mrmrs-7776e1
mrmrs-7776e1
Closed into main
3 commits
·
26 files
+13 added ~13 modified
M
README.md
@@ -127,6 +127,13 @@
127127
`docs/benchmark-coverage.md`, `docs/publish-checklist.md`, and128128
`docs/statistical-methodology.md`.129129
130+
The [September 2026 quality review](docs/review-2026-09-09.md) records current131+
testing strategies, major evidence gaps, and independently authored behavioral132+
ports from upstream VCS test suites. Start local development with the133+
[workstation guide](docs/benchmark-workstation.md). Human usability remains134+
unmeasured; the [proposed participant protocol](docs/human-vcs-study-protocol.md)135+
defines the separate study needed to evaluate it.136+
130137
## In Development And Roadmap131138
132139
The benchmark suite is also a product map for Oak. Current gaps are intentional
A
docs/adr/0009-baseline-comparison-eligibility.md
Loading diff…
A
docs/benchmark-workstation.md
Loading diff…
A
docs/human-vcs-study-protocol.md
Loading diff…
A
docs/review-2026-09-09-methodology.md
Loading diff…
A
docs/review-2026-09-09-oracles.md
Loading diff…
A
docs/review-2026-09-09.md
Loading diff…
M
docs/statistical-methodology.md
Loading diff…
A
docs/vcs-conformance.md
Loading diff…
M
scripts/agent_metrics_schema.py
Loading diff…
M
scripts/agent_workflow.py
Loading diff…
M
scripts/benchmark_stats.py
Loading diff…
M
scripts/oakbench/baseline_book.py
Loading diff…
A
scripts/oakbench/conformance.py
Loading diff…
A
scripts/oakbench/populations.py
Loading diff…
M
scripts/oakbench/scorecard.py
Loading diff…
M
scripts/oakbench/stats.py
Loading diff…
M
scripts/scorecard.py
Loading diff…
A
scripts/vcs_conformance.py
Loading diff…
M
tests/test_baseline_book.py
Loading diff…
M
tests/test_baseline_campaign.py
Loading diff…
A
tests/test_claim_eligibility.py
Loading diff…
A
tests/test_population_reporting.py
Loading diff…
M
tests/test_schema_and_oracle.py
Loading diff…
M
tests/test_scorecard.py
Loading diff…
A
tests/test_vcs_conformance.py
Loading diff…