Oak Mount Lazy-Hydration Benchmarks

# Oak Mount Lazy-Hydration Benchmarks

This lane is for Oak's flagship mount workflow: a task directory appears quickly,

files hydrate on demand, and agents can start useful work before a full checkout

exists locally.

The default execution is a dry probe. It records Oak binary capabilities and

skip rows, but it does not need a remote and does not push anything:

```bash

python3 scripts/mount_probe.py

```

To run real mount timings, use a throwaway Oak repo:

```bash

OAK_BENCH_MOUNT_REPO=owner/repo python3 scripts/mount_probe.py \

  --spec scenarios/mount.yaml \

  --dest "${TMPDIR:-/tmp}/oak-mount-probe" \

  --results results

```

`push_virtual_branch` is disabled until `remote.safe_push: true` is set in

`scenarios/mount.yaml`. Keep it pointed at a disposable benchmark repo because

that scenario intentionally publishes virtual branches.

## Scenario Coverage

| Scenario | What it proves |

| --- | --- |

| `capability_probe` | Whether the chosen Oak binary exists and exposes `oak mount`. |

| `cold_mount_startup` | Time from command start until a mount is available on a cold runner. |

| `warm_mount_startup` | Startup after Oak metadata or object caches have already been touched. |

| `first_use_read_edit` | Time to first `ls`, first targeted read, and first write. |

| `status_diff_commit_inside_mount` | Agent edit loop inside the mounted virtual branch. |

| `push_virtual_branch` | End-to-end cost of publishing a mounted task branch. |

| `teardown_cycle` | Cost and correctness of `oak mount end`. |

| `interrupted_recovery` | Behavior with a leftover partial destination from an interrupted mount. |

| `offline_after_partial_hydration` | Dedicated-runner test for already-hydrated content after network isolation. |

| `parallel_mounts` | Contention and correctness when multiple task mounts start concurrently. |

| `sparse_task_paths` | Hydration behavior when an agent touches only a few task paths. |

| `huge_file_partial_read` | Whether a small prefix read of a huge file avoids full-file hydration. |

## Metrics

Every row is JSONL-compatible with `scripts/bench.py` style fields:

`bench_id`, `profile`, `scenario`, `operation`, `run`, `subject`,

`elapsed_ms`, `returncode`, `command`, host metadata, and source spec metadata.

Mount-specific fields include:

- `since_scenario_start_ms`: cumulative time within the scenario.

- `time_to_first_useful_work_ms`: set on the first useful filesystem operation.

- `bytes_hydrated` and `bytes_downloaded`: parsed from Oak output when Oak

  exposes those counters; otherwise omitted.

- `visible_tree_bytes`, `allocated_tree_bytes`, `disk_entry_count`: bounded disk

  usage measured as its own explicit `disk.usage` operation so it does not hide

  inside startup or first-read timings.

- `retry_count`, `attempts`, `returncode`, `stderr`: command failure and retry

  signals.

- `checks`: correctness checks such as target file existence, bytes read,

  edit round-trip, status success, and teardown path release.

Skip rows use `returncode: 77`, `skipped: true`, and `skip_reason`. They are

deliberately shaped like benchmark rows so dashboards can count missing coverage

without treating default dry runs as regressions.

## Lifecycle Operations

The `task_lifecycle_loop` scenario measures the steady-state space lifecycle an

agent pays per task, not just the first mount:

- `oak.desc`: set the branch description (`defaults.desc_message`).

- `loop.push_desc`: `defaults.loop_iterations` repetitions of

  edit → commit → desc → push (push skips per-iteration when

  `remote.safe_push` is false). Per-iteration rows keep the trend

  (`loop.edit`, `loop.commit`, `loop.desc`, `loop.push`); the

  `loop.push_desc.total` row carries `per_iteration_ms` and the loop-level

  cost — lifecycle overhead lives in the repetition, not the first call.

- `oak.finish`: writes `defaults.desc_message` to a per-run description file

  and runs `oak finish --desc-file <file>`. The command is capability-gated per

  binary (untimed `--help` probe); binaries that predate the command emit a

  `finish_not_supported_by_binary` skip row.

- `space.clean`: runs in the space directory; `checks.mount_released_by_clean`

  records whether the clean tore down the (committed and pushed) mount, which

  is the lifecycle property under test.

## Cold, Warm, and Offline Runs

The probe does not delete Oak's global caches. A cold run means "first mount in

this benchmark process" unless the runner provides cache isolation. For stronger

cold-cache numbers, run on fresh ephemeral machines or add an Oak-supported

cache override to the environment.

The offline scenario is specified but skipped by the local probe because safely

disabling network access is runner-specific. In CI, wrap the probe with a

network namespace, firewall rule, or VM-level network toggle, then run the

offline check after one targeted file has been hydrated.

## Benchmark Repo Shape

Use a disposable repo with:

- A normal root file such as `README.md` for first targeted reads.

- Several nested docs/source paths for sparse task probes.

- At least one large binary or generated fixture path for

  `huge_file_partial_read`; configure it as `defaults.huge_file_path`.

- Enough history or object size that a full clone would be visibly expensive.

The lane should answer the product question directly: how long until an agent

can inspect and edit the one file it needs, and how much local disk/network work

did Oak avoid on the way there.