benchmarksPublic
benchmarksPublic
Commit b4b51d4d
M
README.md
@@ -331,8 +331,38 @@
331331
--oak-local-bin /path/to/current-oak \332332
--scenarios interrupted_publication_after_request_v1 \333333
--runs 10 --randomize-subject-order334+
335+
# Legacy selected-branch clone materialization. Run separate A/A and A/B336+
# configs with the same immutable Serve binary and at least 10 randomized337+
# pairs. Its fixed fixture has 192 regular 16 KiB files (eight executable),338+
# one empty file, and one selected-branch-only file: 194 paths total. Fresh339+
# destinations do not imply cold OS cache.340+
OAK_BENCH_LOCK_PATH=/Users/mrmrs/o/oak-bench-measurement.lock \341+
python3 scripts/task_loop.py \342+
--config /abs/path/subjects-ab.toml \343+
--subjects oak_baseline,oak_candidate \344+
--oak-serve-bin /abs/immutable/fixed-oak-serve \345+
--oak-serve-source-head 64_HEX_HEAD \346+
--materialization-baseline-subject oak_baseline \347+
--scenarios legacy_selected_clone_materialization_v1 \348+
--runs 10 --randomize-subject-order --random-seed 46802 \349+
--claim-materialization-timing \350+
--workdir /abs/worker-owned/materialization-ab/work \351+
--results /abs/worker-owned/materialization-ab/results334352
```335353
354+
Add `--oak-serve-source /abs/read-only/exact-head-checkout` when that immutable355+
checkout is retained; otherwise rows explicitly record that only the supplied356+
source head and binary hash were available for provenance.357+
358+
The scenario records raw proxy byte/connection totals, but its correctness gate359+
uses normalized pull phases, response/object/chunk counters, and connection360+
count; it does not pretend variable HTTP framing is product work. Clone, first361+
post-clone status, and first no-op commit are separate rows. Timing is marked362+
claimable only when `--claim-materialization-timing` is explicit, the shared363+
lock is held, order is randomized, and at least ten runs are present; operators364+
must still coordinate the documented quiet-host window.365+
336366
These cells are different evidence classes. Local Git wall time is not a337367
network comparison, and the Oak local-server row is loopback-only. Its second338368
checkout is fresh and does not claim shared-object or OS-cache warmth. The@@ -344,6 +374,11 @@
344374
and stay explicit skips rather than synthetic passes. Use345375
`parallel_contention.py --modes workspace_per_task` for concurrent isolated346376
tasks instead of duplicating its worker implementation here.377+
The materialization cell is Oak-before versus Oak-after only. Its setup probe378+
must first observe the exact legacy two-pull/two-pass route; one fixed Serve379+
and one repo are then shared by all subjects. Receipt phases, proxy byte counts,380+
HEAD, history, file bytes, and modes are correctness gates. Establish the A/A381+
noise floor before interpreting A/B timing.347382
348383
Mount and acquisition probes:349384
M
scripts/task_loop.py
Loading diff…
M
tests/test_task_loop.py
Loading diff…