Adding a Subject: One JSON, One TOML Entry, Golden Samples

# Adding a Subject: One JSON, One TOML Entry, Golden Samples

Any VCS can join the benchmark. The structural recipe is:

1. **One plugin JSON** — `config/subject_kinds/<kind>.json`

2. **One subjects entry** — `[subjects.<name>]` in `config/subjects.toml` with `kind = "<kind>"`

3. **Golden output samples** — real command output captured under `tests/fixtures/` so parsing claims are testable

Zero lane-code changes. Lanes read commands per kind from data; the plugin

layer is a parallel read of `config/command_semantics.json`, proven

measurement-identical by `oakbench.subject_kinds.parity_report` (see "The

parity proof" below). Capability gaps are declared as `null` commands and

become skip rows (`capability_gap:<kind>:<operation>`) — recorded, never

hidden, never silently substituted (ADR-0002 null honesty).

## The plugin JSON

```json

{

  "schema_version": 1,

  "kind": "jj",

  "commands": {

    "init": ["git", "init"],

    "stage": null,

    "commit": ["commit", "-m", "{message}"],

    "branch_create": null,

    "status": {"agent-default": ["st"], "core-equivalent": null},

    "diff": {"agent-default": ["diff"], "core-equivalent": ["diff", "--git"]}

  },

  "snapshot_call_semantics": {"stage_required": false, "calls_per_snapshot": 1},

  "lock_hint_patterns": [],

  "integrity_command": null,

  "workspace_model": "working_copy_autosnapshot",

  "server_error_patterns": [],

  "remote_env_var": null

}

```

Field rules:

- `commands`: argv template per operation (binary path is prepended by the

  harness; `{message}` substituted at run time). `status`/`diff` are keyed by

  track (`agent-default`, `core-equivalent`). `null` = honest capability gap.

  Be conservative: anything unverified is `null` until golden samples land.

- `snapshot_call_semantics`: how many recorded calls one snapshot costs

  (git: stage+commit = 2; oak and jj: 1). Must agree with `stage`.

- `integrity_command`: list of argv lists for the post-storm sanity pass

  (git: `[["fsck", "--no-progress"]]`), or `null` when no equivalent exists.

- `workspace_model`: e.g. `working_copy`, `virtual_branch`,

  `working_copy_autosnapshot` — drives mode comparability, not commands.

- `remote_env_var`: the env var holding the disposable remote, or `null`

  until the kind has a network lane.

## The worked example: jujutsu (jj)

`config/subject_kinds/jj.json` is the pilot. jj auto-snapshots the working

copy, so `stage` is `null` and a snapshot is one call (`jj commit -m ...`);

`branch_create` is `null` because jj's anonymous-change + bookmark model has

no verified equivalent of create-and-switch; `status` core-equivalent is

`null` until golden samples prove a fair porcelain-level row. Enabling it is

one `[subjects.jj]` entry in `config/subjects.toml` (`kind = "jj"`,

`bin = "jj"`) plus captured golden outputs.

## The parity proof (zero lane-code changes claim)

Existing lanes still load `config/command_semantics.json` byte-for-byte

unchanged; the plugin JSONs for git and oak are extracted from it, and the

extraction is proven, not asserted:

```python

from oakbench.subject_kinds import parity_report

assert parity_report("git") == []   # empty = measurement-identical

assert parity_report("oak") == []

```

`tests/test_subject_kinds.py` pins this. If you edit a command in either

place, the parity test fails until both agree (and a command change is a

`command_semantics.json` version bump per ADR-0005).