mrmrs-fad256

Open into main

2 commits · 6 files +4 added ~2 modified · 6 behind main ·Seen by zach 3 days ago

Description

task_loop: shallow_worker_branch_refresh_v1 (fresh shallow worker -> task branch -> first parent refresh, Oak vs Git, HTTP request counts)

Problem

On hosted oak.space, a fresh oak clone --shallow worker spends ~180 s on its first parent refresh (switch -c, fetch, agent state --refresh).
The cause is the client walking every main ancestor with one sequential POST /commits/info per commit.
This branch adds loopback, reproducible evidence comparing Oak with competent Git on the same task, judged by an independent final-state oracle.

Scenario contract (new immutable name, ADR-0005)

Untimed setup, per COMMITS:SEED fixture:
- Build a deterministic merge-history Git repo.
- Import it with the fixed --oak-serve-bin into an owned loopback oak serve, then push.
- Serve a bare copy with GitHTTPServer.
- Put both servers behind RequestCountingProxy.
Oak cell shallow.*: clone --shallow --allow-legacy-scope, then switch -c task, then fetch. Each step is timed separately.
- A separate shallow.branch_create_repeat_local row runs a second switch -c in the refreshed worker to isolate local switch cost.
Oak cell shallow_pull.*: a fresh clone --shallow, then switch -c task, then pull (branch with no new work, main unchanged). Comparing it with the shallow.* cell shows whether first-refresh ancestry work lands in switch, fetch, or pull.
Git cell shallow.*: clone --depth 1, then switch -c task, then fetch.
Git cell warm_worktree.*: worktree add -b task … origin/main, then fetch, starting from an untimed warm full clone. That is a different starting state and is labeled as such.
Per-step fields: elapsed_ms, returncode, http_request_count, http_commits_info_request_count, http_requests_by_method, network_wire_bytes/connections, injected_request_delay_ms, oracle_passed.
Unobserved values are null, never 0 (ADR-0002); an unframable stream gives null counts.
Oracle:
- The task branch head must equal the server's main. For Oak this is read from the branch API; for Git, from the bare repo's rev-parse.
- The worktree tree hash must equal the fixture model's tree.
- Oak state is read from .oak/oak.db query-only (current_branch, branch_heads, commit rows, ancestry gaps), not from CLI output.
Every row carries serve_depth_behavior (observed from stored commit rows after clone) and hosted_equivalent: false.
Subjects and serve binary are parameters. The run holds measurement_lock("task_loop").

Fixture seeds

merge-history-v1-c1000-s1: git head ed3b72471b73274cdf80f542e7b5835faf6badc2, 66 merges (6.6%), 374 files.
merge-history-v1-c2000-s2 (held out): git head 4ebc5f93e4a0d170f28d847df7e76c7b573166e1, 112 merges (5.6%).

Files

scripts/oakbench/merge_history_fixture.py
scripts/oakbench/httpcount.py
scripts/oakbench/shallow_refresh.py
scripts/task_loop.py (new campaign plus --shallow-refresh-fixtures and --injected-request-delay-ms)
tests/test_shallow_refresh.py (24 tests)
README.md

Limitations

Oak 0.103.0 oak serve ignores pull depth (serve.rs pull handler). On current Serve, --shallow stores full history and the ancestor backfill does not occur (serve_depth_ignored). Rows against such a Serve are a fidelity baseline, not the hosted result. The real before/after needs the depth-honoring Serve build (a separate oak/oak lane).
Local Serve lacks the bounded integrity preflight, so clone needs --allow-legacy-scope.
Loopback only. OS cache state is warm or unknown. The injected delay is a fixed per-request sleep in the proxy, not netem.
The Git warm_worktree cell starts from a pre-existing full clone, so it is not the same starting state as a fresh shallow worker.
Raw results are kept outside the repo (ADR-0003).

Commits 2

Files 6

scripts

oakbench

A httpcount.py A merge_history_fixture.py A shallow_refresh.py

M task_loop.py

tests

A test_shallow_refresh.py

M README.md

@@ -383,6 +383,43 @@

383383 --results /abs/worker-owned/materialization-ab/results

384384 ```

385385

386+Fresh shallow worker -> named task branch at current `main` -> first parent

387+refresh, Oak versus Git, with per-step HTTP request counts:

388+

389+```bash

390+python3 scripts/task_loop.py \

391+ --config /abs/path/subjects.toml --subjects git,oak_baseline,oak_candidate \

392+ --oak-serve-bin /abs/immutable/fixed-oak-serve \

393+ --scenarios shallow_worker_branch_refresh_v1 \

394+ --shallow-refresh-fixtures 1000:1,2000:2 \

395+ --runs 7 --randomize-subject-order --injected-request-delay-ms 0 \

396+ --workdir /abs/worker-owned/shallow-refresh/work \

397+ --results /abs/worker-owned/shallow-refresh/results

398+```

399+

400+Each `COMMITS:SEED` fixture is a deterministic merge-history Git repo

401+(`oakbench/merge_history_fixture.py`, ~5-7% two-parent merges). During untimed

402+setup it is imported into an owned loopback `oak serve` and a bare Git repo on

403+`GitHTTPServer`. Both sit behind `oakbench/httpcount.py` proxies that frame

404+HTTP/1.x requests: rows carry `http_request_count`,

405+`http_commits_info_request_count` and wire bytes, and all are null when the

406+stream could not be framed. The Oak cell times `clone --shallow` ->

407+`switch -c task` -> `fetch`. It then records a separate

408+`shallow.branch_create_repeat_local` row, which isolates local switch cost in

409+the already-refreshed worker. A second Oak cell, `shallow_pull.*`, repeats

410+the fresh clone and `switch -c task`, then runs `pull` instead of `fetch`.

411+Together the two cells show which command pays for the first-refresh

412+ancestry work. The Git cells are `clone --depth 1` ->

413+`switch -c` -> `fetch`, plus `warm_worktree.*`, which starts from an untimed

414+warm full clone. That is a different starting state and is labeled as such.

415+The oracle reads `.oak/oak.db` query-only (or `git rev-parse`), requires the

416+task branch head to equal the server's `main`, and requires the worktree to

417+hash to the fixture model's tree. Rows record `serve_depth_behavior`: a Serve

418+that ignores `--shallow` depth yields `serve_depth_ignored`, and such rows are

419+a fidelity baseline, not hosted-equivalent evidence (`hosted_equivalent` is

420+always false on loopback). `--injected-request-delay-ms` is a fixed per-request

421+sleep in the proxy, not a shaped network.

422+

386423 Add `--oak-serve-source /abs/read-only/exact-head-checkout` when that immutable

387424 checkout is retained; otherwise rows explicitly record that only the supplied

388425 source head and binary hash were available for provenance.