Tuned Git Baselines

Stock Git is not enough for public claims. The benchmark suite should compare Oak against credible Git modes where those modes apply.

The machine-readable source of truth is config/git_modes.json.

Modes

Mode	Purpose	Applies To	Oak Comparison
`git_stock`	Default installed Git.	All scenarios.	`oak_default`, `oak_virtual_branch`.
`git_untracked_cache`	Speed up status in large trees with many untracked paths.	Large-tree, ignored-heavy, wide-dirty-tree scenarios.	`oak_default`.
`git_split_index`	Reduce index write cost in large working trees.	Large-tree and wide-dirty-tree scenarios.	`oak_default`.
`git_fsmonitor`	Avoid full filesystem scans when platform support exists.	Large working-tree scenarios.	`oak_default`.
`git_sparse_checkout`	Materialize only task-relevant paths.	Sparse task and large-tree scenarios.	`oak_space_mount`, `oak_mount_per_task`.
`git_worktree`	Isolate per-task branches.	Parallel-agent and branch-per-task scenarios.	`oak_space_mount`, `oak_virtual_branch`.
`git_lfs`	Avoid storing large binary payloads directly in Git objects.	Binary-heavy scenarios when LFS is installed.	`oak_default`, `oak_space_mount`.

Fairness Rules

Report tuned modes separately from stock Git.
Mark unsupported modes explicitly; do not silently drop them.
Do not use sparse checkout as the full-tree baseline.
Do not compare Git LFS to normal Oak storage unless the scenario is explicitly about external binary-storage workflows.
Include setup costs only when the scenario is about task startup or isolation.

Implementation Notes

Each mode needs:

setup commands
teardown commands if any
compatibility checks
fixture applicability
exact command mapping for timed operations
correctness checks proving the mode saw the intended paths

The public claim should be “Oak beats stock Git” or “Oak beats tuned Git mode X” depending on which rows actually ran.

scripts/bench.py --git-modes lfs is wired for the binary fixture scenarios named in git_lfs.applies_to (this file's table is prose; the JSON list is enforced). The mode runs git lfs install --local and git lfs track "*.bin" after repo.init and before snapshot.initial, recorded as mode.setup.lfs_install and mode.setup.lfs_track operation rows (mirroring mode.setup.fsmonitor), so the generated binary files are tracked through LFS before the first commit and the setup cost stays visible. Scenarios outside applies_to and hosts without Git LFS emit scenario.mode_skipped rows, and a fixture file the track patterns miss fails setup closed with a mode.setup.lfs_track_coverage skip row (lfs_track_incomplete:<ext>) — an untracked comparator is an unfair comparator. All of these are unmeasured, not failures or hidden stock Git measurements.

# Tuned Git Baselines

Stock Git is not enough for public claims. The benchmark suite should compare
Oak against credible Git modes where those modes apply.

The machine-readable source of truth is `config/git_modes.json`.

## Modes

| Mode | Purpose | Applies To | Oak Comparison |
| --- | --- | --- | --- |
| `git_stock` | Default installed Git. | All scenarios. | `oak_default`, `oak_virtual_branch`. |
| `git_untracked_cache` | Speed up status in large trees with many untracked paths. | Large-tree, ignored-heavy, wide-dirty-tree scenarios. | `oak_default`. |
| `git_split_index` | Reduce index write cost in large working trees. | Large-tree and wide-dirty-tree scenarios. | `oak_default`. |
| `git_fsmonitor` | Avoid full filesystem scans when platform support exists. | Large working-tree scenarios. | `oak_default`. |
| `git_sparse_checkout` | Materialize only task-relevant paths. | Sparse task and large-tree scenarios. | `oak_space_mount`, `oak_mount_per_task`. |
| `git_worktree` | Isolate per-task branches. | Parallel-agent and branch-per-task scenarios. | `oak_space_mount`, `oak_virtual_branch`. |
| `git_lfs` | Avoid storing large binary payloads directly in Git objects. | Binary-heavy scenarios when LFS is installed. | `oak_default`, `oak_space_mount`. |

## Fairness Rules

- Report tuned modes separately from stock Git.
- Mark unsupported modes explicitly; do not silently drop them.
- Do not use sparse checkout as the full-tree baseline.
- Do not compare Git LFS to normal Oak storage unless the scenario is explicitly
  about external binary-storage workflows.
- Include setup costs only when the scenario is about task startup or isolation.

## Implementation Notes

Each mode needs:

- setup commands
- teardown commands if any
- compatibility checks
- fixture applicability
- exact command mapping for timed operations
- correctness checks proving the mode saw the intended paths

The public claim should be “Oak beats stock Git” or “Oak beats tuned Git mode X”
depending on which rows actually ran.

`scripts/bench.py --git-modes lfs` is wired for the binary fixture scenarios
named in `git_lfs.applies_to` (this file's table is prose; the JSON list is
enforced). The mode runs `git lfs install --local` and `git lfs track "*.bin"`
after `repo.init` and before `snapshot.initial`, recorded as
`mode.setup.lfs_install` and `mode.setup.lfs_track` operation rows (mirroring
`mode.setup.fsmonitor`), so the generated binary files are tracked through LFS
before the first commit and the setup cost stays visible. Scenarios outside
`applies_to` and hosts without Git LFS emit `scenario.mode_skipped` rows, and a
fixture file the track patterns miss fails setup closed with a
`mode.setup.lfs_track_coverage` skip row (`lfs_track_incomplete:<ext>`) — an
untracked comparator is an unfair comparator. All of these are unmeasured, not
failures or hidden stock Git measurements.

About

Reversible clean-root candidate rebuilt from oak/benchmarks main@009dead264ca; original repository retained.

Download

RunnersDetails

No runs on main yet.

Open branchesHistory

mrmrs-516758agent_loop_hosted_v1: hosted agent-loop gap map lane (Oak vs GitHub vs ideal)checking…6 behind1mo ago

mrmrs-b26e28Add stale_branch_land_v1: baseline for landing branches that are behind mainchecking…6 behind1mo ago

mrmrs-fad256task_loop: shallow_worker_branch_refresh_v1 (fresh shallow worker -> task branch -> first parent refresh, Oak vs Git, HTTP request counts)checking…6 behind1mo ago

oakmark-throughput-benchmark-transplantAdd oakmark: per-repo push/clone throughput benchmark for oak (ForgeMark analogue)checking…11 behind3mo ago

All branches →

Activity12 commits to main

May 2026now

1	`# Tuned Git Baselines`
2
3	`Stock Git is not enough for public claims. The benchmark suite should compare`
4	`Oak against credible Git modes where those modes apply.`
5
6	The machine-readable source of truth is `config/git_modes.json`.
7
8	`## Modes`
9
10	`\| Mode \| Purpose \| Applies To \| Oak Comparison \|`
11	`\| --- \| --- \| --- \| --- \|`
12	\| `git_stock` \| Default installed Git. \| All scenarios. \| `oak_default`, `oak_virtual_branch`. \|
13	\| `git_untracked_cache` \| Speed up status in large trees with many untracked paths. \| Large-tree, ignored-heavy, wide-dirty-tree scenarios. \| `oak_default`. \|
14	\| `git_split_index` \| Reduce index write cost in large working trees. \| Large-tree and wide-dirty-tree scenarios. \| `oak_default`. \|
15	\| `git_fsmonitor` \| Avoid full filesystem scans when platform support exists. \| Large working-tree scenarios. \| `oak_default`. \|
16	\| `git_sparse_checkout` \| Materialize only task-relevant paths. \| Sparse task and large-tree scenarios. \| `oak_space_mount`, `oak_mount_per_task`. \|
17	\| `git_worktree` \| Isolate per-task branches. \| Parallel-agent and branch-per-task scenarios. \| `oak_space_mount`, `oak_virtual_branch`. \|
18	\| `git_lfs` \| Avoid storing large binary payloads directly in Git objects. \| Binary-heavy scenarios when LFS is installed. \| `oak_default`, `oak_space_mount`. \|
19
20	`## Fairness Rules`
21
22	`- Report tuned modes separately from stock Git.`
23	`- Mark unsupported modes explicitly; do not silently drop them.`
24	`- Do not use sparse checkout as the full-tree baseline.`
25	`- Do not compare Git LFS to normal Oak storage unless the scenario is explicitly`
26	`about external binary-storage workflows.`
27	`- Include setup costs only when the scenario is about task startup or isolation.`
28
29	`## Implementation Notes`
30
31	`Each mode needs:`
32
33	`- setup commands`
34	`- teardown commands if any`
35	`- compatibility checks`
36	`- fixture applicability`
37	`- exact command mapping for timed operations`
38	`- correctness checks proving the mode saw the intended paths`
39
40	`The public claim should be “Oak beats stock Git” or “Oak beats tuned Git mode X”`
41	`depending on which rows actually ran.`
42
43	`scripts/bench.py --git-modes lfs` is wired for the binary fixture scenarios
44	named in `git_lfs.applies_to` (this file's table is prose; the JSON list is
45	enforced). The mode runs `git lfs install --local` and `git lfs track "*.bin"`
46	after `repo.init` and before `snapshot.initial`, recorded as
47	`mode.setup.lfs_install` and `mode.setup.lfs_track` operation rows (mirroring
48	`mode.setup.fsmonitor`), so the generated binary files are tracked through LFS
49	`before the first commit and the setup cost stays visible. Scenarios outside`
50	`applies_to` and hosts without Git LFS emit `scenario.mode_skipped` rows, and a
51	`fixture file the track patterns miss fails setup closed with a`
52	`mode.setup.lfs_track_coverage` skip row (`lfs_track_incomplete:<ext>`) — an
53	`untracked comparator is an unfair comparator. All of these are unmeasured, not`
54	`failures or hidden stock Git measurements.`