Evaluation
Results
Each row is a constructed diff: three to five real commits squashed into one tangled change, with the original commits kept as ground truth. B1 asks Bob in Agent mode, in one prompt, to split it into stacked branches. Cleave runs the ✂ Cleave mode, where Bob can only move hunks. Both runs are checked by the same five checks.
Every row links to its public proof. Nothing here is typed in by hand: rows appear when a run is pushed.
On 3 diffs with both runs, Cleave produced a valid stack on 3 and B1 on 0. Zero foreign lines: Cleave on 3, B1 on 3.
| Diff | Run | Valid stack | Green layers | Foreign lines | Largest layer | Bobcoins | Proof |
|---|---|---|---|---|---|---|---|
| click-completionspallets/click · 1,055 lines | B1 | Invalid | 2/3 | 0 | 716 | 2.20 | Proof |
| Cleave | Valid | 2/2 | 0 | 895 | 1.60 | ||
| click-deprecated-paramspallets/click · 472 lines | B1 | Invalid | 1/2 | 0 | 298 | 1.90 | Proof |
| Cleave | Valid | 3/3 | 0 | 220 | 1.40 | ||
| galaxium-lint-passedish-github/galaxium-travels · 218 lines | B1 | Invalid | 0/3 | 0 | 191 | 1.70 | Proof |
| Cleave | Valid | 2/2 | 0 | 207 | 1.20 |
- Valid stack.
- Coverage, order, fidelity, shippability and partition all pass.
- Green layers.
- Layers whose check command passes on their own.
- Foreign lines.
- Lines in the stack that aren't in the original diff. Cleave's target is 0.
- Largest layer.
- Added plus removed lines in the biggest layer.
- Bobcoins.
- Reported by Bob for the run.