Benchmark workstation

# Benchmark workstation

Use an isolated `worker-mrmrs-*` checkout, never a canonical repository.

The harness requires Python 3.9+ and the subjects under test; its core has no

required third-party Python packages. Tests using local HTTP fixtures require

permission to bind loopback sockets. They do not require production remotes.

## Prepared workstation for the 2026-09-09 review

The integration checkout is:

```

/Users/mrmrs/o/worker-mrmrs-20260909T080918-benchmark-quality/bench

```

Its parent directory contains uncommitted workstation artifacts:

- `subjects/oak`: copied Oak 0.103.0 binary.

- `subjects/jj`: copied Jujutsu 0.42.0 binary for subsequent subject work.

- `subjects.toml`: Git and copied Oak configuration for the core harness.

- `workstation.json`: source paths, executable paths, versions and SHA256s.

- `workstation-env.sh`: worker/config paths and the shared measurement lock.

- `baseline-tests-unrestricted.log`: original 903-test passing baseline,

  with three skips. The first sandboxed run's 18 socket errors are retained

  separately in `baseline-tests.log`.

- `final-tests.log`: final 921-test passing integration run, with three skips.

- `conformance-final.jsonl`: Git five passes, Oak four passes and one retained

  tab-filename failure. This is diagnostic correctness data, not timing rows.

Git is Apple Git 2.50.1 at `/usr/bin/git`; its executable identity is recorded,

but its system installation/helpers are not made immutable by copying a

launcher. Recheck provenance after OS/tool updates. Copied binaries likewise

still depend on host libraries and OS services. This setup is a developer

workstation, not a certified dedicated performance runner.

## Local validation

From the checkout above:

```bash

. ../workstation-env.sh

pwd

oak status --json

python3 -m unittest discover -s tests

# Small diagnostic smoke only; no public speedup claims from one repetition.

python3 scripts/bench.py --profile smoke --runs 1 \

  --config "$BENCH_SUBJECTS" --subjects git,oak_installed \

  --skip-remote --randomize-subject-order \

  --workdir workdirs/workstation-smoke --results results/workstation-smoke

```

Run measurements serially after builds and test activity have stopped. Keep

the shared lock enabled; do not stop another worker's campaign. Raw rows and

run artifacts stay ignored or outside the checkout. A smoke failure is a

development signal, not evidence to discard or relabel as a skip.

## Expanding to a performance campaign

Clone Oak source into another required worker directory before building a

candidate. `scripts/prepare-subjects.sh` invokes Cargo and writes its target

directory; therefore its `OAK_REPO` must be an isolated worker source checkout.

Never point it at a canonical checkout under the assumption it is read-only.

Use pinned baseline/candidate binaries and source provenance, equal semantic

tracks, matched transport, seeded randomized/interleaved repetitions, recorded

cache evidence and runner state, and a separate A/A noise campaign. Choose

sample size for the effect and uncertainty needed; a minimum count alone is

not sufficient precision. See `runner-discipline.md`,

`statistical-methodology.md`, and the methodology review for current gaps.

Source `config/bench-env.sh` only for deliberately configured remote-backed

lanes. It supplies existing named disposable remotes, not per-run isolation.

Allocate and track run-owned repositories before destructive lifecycle or

contention campaigns. Missing credentials, FUSE, Linux network privileges,

LFS, real-agent binaries/models, and hosted CI are capability gaps, not zero

cost or successful measurements. No such campaign was commissioned merely by

preparing this local workstation.

For human usability, implement and pilot the separate

[`human-vcs-study-protocol.md`](human-vcs-study-protocol.md); neither a scripted

workflow nor a mock-agent row substitutes for a participant observation.