Evidence / All results

Every check, benchmark and baseline

This page is the repository's own record, unedited: the results ledger, then each milestone's status and the pass and fail lines of every work package's gate. It lists 380 rows in 96 tables, failures and unrun rows included. For the short version, see the overview.

Built 2026-11-15 from tests/RESULTS.md and build/BUILD-PLAN.md. Paths in the tables are files in the Graftyard repository. Evidence tags: measured was run; doc comes from vendor documentation; inference and guess are reasoned, not run.

tests/RESULTS.md

Results ledger, Milestones 0 and 1

This file lists every run that counts toward a gate row or a decision in Milestones 0 and 1. It covers runs on Cloudflare, in the Raft simulator, and through the Oracle relay. It gives each run's gate row, place, dates, load, faults, checks, and result, and cites the results file and section the numbers come from. Smoke runs and discarded runs appear only in "Smoke, local, and discarded runs", labelled as such. Every figure is measured unless a row says otherwise.

Raw data paths (raw/) are git-ignored and exist only on the machine that ran the test. Results files cite them. Where this file and a results file's prose disagree, this file takes the raw value, and "Prose and raw data that disagree" lists the difference.

Totals

Each total has one definition. Totals add only counts of the same thing from different runs, never the counts of different checkers on one history.

TotalValueDefinition
Operations checked for linearizability on Cloudflare12,304,207Operations in the histories of the seven chaos runs in "Linearizability and chaos runs on Cloudflare". Each history counts once, though most were checked two or three ways (recorded clocks, widened 100 ms, widened by the measured bound). By verdict: 9,089,215 Ok (cf-long-1, cf-48h widened 100 ms, wp4-cf-1, wp4-cf-5, lin24b). 475,301 Illegal from a real bug (cf-long-2). 2,739,691 not passed (lin24: Illegal on the recorded clocks, Ok widened, early ack not ruled out). Excludes the smoke runs, BF-13's 3,680 operations, and every simulator history
Hours of chaos on Cloudflare104.6 hWall-clock length of those seven runs: 2 + 4 + 48 + 0.52 + 2.08 + 24 + 24. The windows include stretches without chaos: four of 68 to 115 min in cf-48h, three driver gaps in cf-long-2, a fault plan overwritten for part of cf-long-1, and call faults on only 39 of 125 min in wp4-cf-5
Member resets injected16,258/reset steps by the chaos driver in those seven runs: 243 + 138 + 5,729 + 82 + 343 + 5,547 + 4,176. cf-long-2's 138 comes from prose, because its event log was overwritten
Leadership transfer attempts injected10,680Transfer steps the chaos driver sent in those seven runs: 186 + 77 + 3,607 + 62 + 253 + 3,743 + 2,752, including planned abandons and attempts answered "not leader" or refused by a quarantined target. Transfers that moved the leader are not totalled, because M0's driver and M1's report classify outcomes differently (see each row)
Acknowledged writes audited against the members' logs3,588,424, 0 lostcf-48h 1,814,873, lin24 869,257, and lin24b 904,294. Each audit reads all three members' stored logs after the run
Leader kills after quorum and before the reply12,000 landedBF-12 on Durable Objects: 10,000 (bf12-full), 1,000 (bf12-p3), and 1,000 on Push through the git edge (p2). Every one returned its original outcome
SSH git sessions31,482, 31,362 succeededB0-24 (2,637), B0-24a r4 (28,347), and the Oracle relay run r1 (249 through the relay plus 249 control sessions through cloudflared). Excludes three stopped 24-hour attempts (1,144 sessions)
Results of git's own t/ tests on Cloudflare2,667Rows in results.tsv of the no-fault baseline (1,067), the three BM-1 faulted runs (1,558), and the t5541 recheck (42), skips included. Per version below
clients.sh runs12 runs, 348 of 348 rows29 rows each, on git 2.39.5, 2.51.0, and 2.56.0, with and without a site failing
Tests in the workspace259 Rust, 14 Gocargo test --workspace at c0c6fb9: 259 passed, 0 failed, 2 ignored (long_sweep, long_liveness). Go: tests/m0/porcupine 11, containers/git/server 3
Simulator long sweep40,191,804 steps, 1,000,856 acked writesOne run of long_sweep at 2,000 seeds × 10,000 steps, HARSH plus GRAY. Every acked write passed the in-simulator ack check. Earlier sweeps are listed, not added
Planted bugs caught84 of 85Code mutants planted one at a time against a test suite. Planted bad histories fed to a checker are counted apart

Linearizability and chaos runs on Cloudflare

Every run used the same fault plan unless its row says otherwise. 5% of member-to-member calls fail from each site, and 5% of AppendEntries replies are dropped. A chaos loop resets a member or transfers leadership every 8 to 16 s, and some transfers are abandoned on purpose. Load clients are pinned in ATL, LHR, and SIN unless the row names the orb (LAX). Porcupine is tests/m0/porcupine, run per key with segmented mode (-seg N).

scripts/results_lin.py <run>.report.json prints the lin24 row's first seven columns from the run's report and event log. Use it for every later lin run.

RunGateWhere and whenDurationOperationsFaultsChecksResultSource
cf-long-1B0-11Cloudflare, graft-raft-*, 2026-10-28T08:42Z to 10:42Z2 h270,040, 0 failed; p50 218 ms, p99 1,033 ms243 resets. 186 transfer attempts: 81 transferred, 46 "not leader", 5 abandoned with the reply lost, and 54 planned abandons (17 of them answered "not leader"). The B0-06 and B0-03 runs overwrote the fault plan for parts of 09:34Z to 10:25Z, and one deploy reset every member at 09:33Z (raft-chaos-cf-long-1.started)Porcupine on the whole history, rechecked with unique write namesOktests/m0/results/m0-wp2.md, "B0-11: the 4-hour run found a linearizability violation"; raw/raft-chaos-cf-long-1.json
cf-long-2B0-11Cloudflare, graft-raft-*, 2026-10-28T13:00Z to 17:00Z4 h475,301138 resets and 77 transfers (prose only). No resets or transfers in three driver gaps, and one deploy reset every member at 14:44Z (raft-chaos-cf-long-2.started)Porcupine; members' stored logs read back with /dump; deterministic repro (run/raft-ticket.ts, 3 of 3)Illegal. A real bug: a Ticket of (term, seq) was reused after a leader reset, so outcome answered an old ticket with another command's result. Fixed in the design and the prototype by adding boottests/m0/results/m0-wp2.md, "B0-11: the 4-hour run found a linearizability violation"; raw/raft-chaos-cf-long-2.json
cf-48hB0-11Cloudflare, graft-raft-*, 2026-10-28T18:50Z to 2026-10-30T18:50Z48 h5,742,736, 12 with no reply; p50 284 ms, p99 1,211 ms5,729 resets (A 2,323, B 2,292, C 1,114). 3,607 transfer attempts, 1,265 "not leader", so 2,342 from a leader. Of those, 1,431 transferred, 779 abandoned, and 132 transferred with the reply lost. No per-isolate counters. Coverage on site C is thin (F36)Porcupine -seg 20000 per key. Lost-write audit of all three logs. Single-clock early-ack testRecorded clocks Illegal on all 6 keys, from 72 cross-colo reads 27 to 94 ms stale, all consistent with clock skew. Widened 100 ms Ok on all 6 keys. 0 of 1,814,873 acked writes lost; 984,514 identical entries on 3 members. Early-ack test 0 of 72 and 0 of 5,934 controls. Pass with the 100 ms allowance as an assumption, not a measured bound. Rechecked with the fixed segmented checker: unchangedtests/m0/results/m0-wp2.md, "B0-11: the 48-hour run on the fixed Ticket code"; tests/m1/results/porcupine-seg.md; raw/raft-chaos-cf-48h.events.jsonl, raw/cf-48h-analysis/lost-acked.json
wp4-cf-1B0-11 (fault and transfer rules on the M0-WP4 code)Cloudflare, graft-wp4-*, 2026-10-30T00:09Z to 00:40Z, 8 clients from the orb31 min41,567 (4 failed at the orb's own fetch); p50 287 ms, p99 1,090 ms82 resets. 62 transfer attempts: 29 transferred, 2 with the reply lost, 13 planned abandons, 18 "not leader"PorcupineOk. 16 operations over 9 s in two stall episodes, cause not establishedtests/m0/results/m0-wp4.md, "Chaos run on graft-wp4"; raw/raft-chaos-wp4-cf-1.json
wp4-cf-5B0-11 and B0-06 rules (STEP_DOWN count 6, quarantined target)Cloudflare, graft-wp4-*, 2026-10-30T07:25Z to 09:30Z, 8 clients from the orb2 h 5 min184,349 (3 failed at the orb's own fetch); p50 246 ms, p99 1,003 ms343 resets. 253 transfer attempts: 119 transferred, 1 with the reply lost, 51 planned abandons, 82 "not leader". 14 slow-storage periods. Call faults on for 39 of 125 minPorcupine; STEP_DOWN trips from 22,979 member log linesOk, 0 operations over 9 s. 23 confirm trips, all on a slowed sitetests/m0/results/m0-wp4.md, "Two-hour chaos run on the final code"; raw/raft-chaos-wp4-cf-5.json
lin24BM-1, 24 hCloudflare, graft-m1-lin-*, repo:915635, 2026-11-01T18:27:14.621Z to 2026-11-02T18:27:14.621Z24 h2,739,691, 0 with no reply; p50 326 ms, p99 1,009 ms. 1,096,016 gets, 821,524 puts, 47,733 CAS ok, 774,418 CAS refused5,547 resets (3,676 of a leader), 3,743 transfer attempts (1,131 planned to be abandoned, 73 accepted by the leader, 3,663 refused by the target), 77,653 failed calls, 72,130 dropped AppendEntries replies, counters from 88 isolatesPorcupine -seg 2000 per key, recorded clocks Illegal on k2, k4, k5. Widened 100 ms Ok on all 6 keys. Widened 275 ms (the measured bound, halved) Ok on all 6 keys. No member-side ack evidence. Lost-write audit of all three logsNot passed. The 5 Illegal segments read writes whose history rows were lost in 5 of 12,518 client chunks, and are Ok with the rows restored. 0 of 869,257 acked writes lost, 0 of 774,418 refused CAS in the log, three identical logs of 479,965 entries. An early ack is not ruled out: one ordering needs 93 ms of ATL to LHR skew against a measured 49 ms, and 17 writes were acked faster than a quorum could commit them on their own client's clock (F44)tests/m1/results/m1-wp2.md, "The 24-hour run's result, 2026-11-02: lin24 does not pass"; tests/m1/lin/raw/lin24.report.json, lin24.events.jsonl, lin24.jsonl
lin24bBM-1, 24 hCloudflare, graft-m1-lin-*, repo:934783, 2026-11-03T00:55:07.302Z to 2026-11-04T00:55:07.302Z24 h2,850,523, 0 with no reply; p50 314 ms, p99 952 ms. 1,141,028 gets, 854,483 puts, 49,811 CAS ok, 805,201 CAS refused4,176 resets (2,795 of a leader), 2,752 transfer attempts (855 planned to be abandoned, 56 accepted by the leader, 2,690 refused by the target), 83,209 failed calls, 77,604 dropped AppendEntries replies, counters from 97 isolatesPorcupine -seg 2000 per key, recorded clocks Illegal on k0, k1, k2, k3, k4. Widened 100 ms Ok on all 6 keys. Widened 153 ms (the measured bound, halved) Ok on all 6 keys. Ack check Ok on 904,294 of 904,294 acknowledged writes, 0 violations. Lost-write audit of all three logs. Single-clock early-ack testPass. Of the 20 recorded-clock Illegal segments, 18 read or hold writes whose history rows were lost in 15 of 12,535 client chunks, and are Ok with the rows restored. 2 are cross-colo ties, Ok with 1 ms (LHR–SIN) and 4 ms (ATL–LHR) of skew against that minute's measured 92 ms and 63 ms. None is same-colo. 0 of 904,294 acked writes lost, 0 of 805,201 refused CAS in the log, three identical logs of 513,232 entries, no request id twice. Early-ack test 0. F44 answered: a client-clock artifact, since 59 writes acked faster than their own quorum proof's round trip on the client's clock all pass the member checktests/m1/results/m1-wp2.md, "The 24-hour rerun's result, 2026-11-04: lin24b passes"; tests/m1/lin/raw/lin24b.report.json, lin24b.events.jsonl, lin24b.jsonl

Other gate runs on Cloudflare, Milestone 1

GateRunsLoad and faultsCheckResultSource
BF-12bf12-full, 2026-11-01 02:09Z to 02:44Z, 10 groups10,001 leader kills sent, 10,000 landed: after_quorum 4,800, after_quorum_unsynced 4,800, each plus 200 with the object held down 12 sRe-send under the same RequestId; audit of all 30 membersPass. 10,000 of 10,000 kept their outcome, 0 double applies, 76 answered by another leadertests/m1/results/m1-wp2.md, "BF-12: leader killed after quorum and before the reply"; tests/m1/wp2/raw/bf12-full.bf12-audit.json
BF-12bf12-p3 regression, 2026-11-011,000 kills, 1,000 landedSame auditPass, 0 changed, 0 double appliestests/m1/results/m1-wp2.md, "BF-12 and BF-13 regression"; tests/m1/wp2/raw/bf12-p3.bf12-audit.json
BF-12 on Pushp2, 2026-11-01, 10 repositories1,000 kills through the git edge's commit_hedgedRe-send outcome and per-ref result; audit of 30 membersPass, 1,000 of 1,000, 0 double appliestests/m1/results/m1-wp6.md, "BF-12 on Push commands"; tests/m1/integ/raw/p2.bf12-report.json
BF-13bf13-full (400 rounds), bf13-p3 (400), bf13-p3e with early AppendEntries (120)Leader killed before commit on an entry with a Stale refusal caused by an earlier command in the same entryPorcupine on 1,600, 1,600, and 480 operations; a planted refusal made the bf13-full and bf13-p3 histories IllegalPass. 0 refusals returned from uncommitted entries, Porcupine Ok on all threetests/m1/results/m1-wp2.md, "BF-13" and "BF-12 and BF-13 regression"; tests/m1/wp2/raw/bf13-*.bf13-report.json
BF-14r1 11:39Z and r2 13:03Z, 2026-11-02, hosted edge1,200 commands each (600 Push, 600 PutAccount), every first receipt missed: half held 2.6 s, half dropped (test-faults)Edge trace against every member's arrival rowsPass. 2,400 of 2,400 decided ok, 0 forwardedtests/m1/results/m1-wp8.md, "BF-14: receipts that miss the deadline"; tests/m1/wp8/raw/r1.bf14-report.json, r2.bf14-report.json
BF-10g1, 2026-11-011,000 chain creations from two edges at once (13.9 min), then 100 rounds during an owner move, plus a 20-round control without the generation claim/census of every memberPass. 1,000 of 1,000 and 100 of 100 with one group. The control made two groups in 20 of 20, which is a design problem foundtests/m1/results/m1-wp7.md, "BF-10: one group per chain, and no read or vote without a config"
BF-11g2, 2026-11-01, 30 chains1,500 cold-cache requests across a member retry, a ReplaceMember stand-in, and an owner moveEvery request found the groupPass, 1,500 of 1,500tests/m1/results/m1-wp7.md, "BF-11"
BF-15g1, then rerun i1 on the integration site script1,000 commands queued behind a token revocation, and 1,000 behind an account suspension, per runEvery command decided after the changePass, 1,000 of 1,000 refused 401 in each case, in both runstests/m1/results/m1-wp7.md, "BF-15" and "BF-15 rerun on this pass's site script"; tests/m1/wp7/raw/i1.bf15.jsonl
BF-16Stand-in group, then the real groups, 2026-11-01N+1 edge with an unenabled format; forced unknown and known-new formatsMember receive counters; refs and entries unchangedPass on bothtests/m1/results/m1-wp6.md, "BF-16 on Cloudflare" and "BF-16 on the real groups"
BF-17Stand-in group (git 2.56.0 and 2.39.5), then the real groupsAtomic push of 10,000 tags with a 1 MiB pack; 10,001 tags; 12,000 non-atomic creates over the byte limitEntry size, refs, clone, fsckPass, 12 of 12 rows on the real groupstests/m1/results/m1-wp6.md, "BF-17 on Cloudflare" and "BF-17 on the real groups"
BF-18Bed with a stubbed chain leader: networks 5000001 to 5000003 and the LRU rerun 5000020. Hosted with a real ChainLog: run 1 and run 2, 2026-11-02Archive a range, kill both shards, rehydrate, a second shard behind its cursorgit fsck --strict, 9 clones per run, a fetchBed: 4 runs, 0 failures. Hosted run 1 failed at rehydrate: a thin stored pack in the promoted range, now a design rule. Hosted run 2 Pass: fsck clean, 9 of 9 clonestests/m1/results/m1-wp5.md, "BF-18 on Cloudflare Containers", "BF-18 rerun after the LRU", "Run 1" and "Run 2"
BF-19r10 and r11 on real buckets with a stubbed request table; r1 on the hosted cron1,000 failed placements plus 1,000 commits with the marker write killed per run (r10 and r11 reached 2,150 of each with retried batches)HEAD of every pack and marker in all three buckets after each sweep; injected sweeper clockPass. 0 committed packs deleted, every missing marker repaired, every failed placement deleted after 24 h and none beforetests/m1/results/m1-wp4.md, "BF-19 on real buckets" and "BF-19 on the hosted cron"; tests/m1/wp8/raw/r1.bf19-report.json
B0-01 on the Rust hostb1 to b6 (first pass) and d1 to d6 (second pass), 60 s each6,000 commands per run, 0 failedCommit p99 on the leader, against M0 controls in the same hourFirst pass p99 645 to 791 ms in all 6. Second pass p99 414 to 807 ms: 2 of 6 pass 600 ms, 4 between the lines, none over 1 s. On the current host (m0-carried, graft-m3c, 2026-11-11): 6 of 6 at p99 244 to 377 ms, Pass (tests/m0/results/carried.md)tests/m1/results/m1-wp2.md, "Commit latency, B0-01 style" and "B0-01 on the new host"
F43 soakf43-soak, 2026-11-02 19:28Z to 20:21Z, 4 repositories14 cycles: 4 total faults on a follower site, 4 gray faults (30%) on the leader's site, 3 follower resets, 3 without chaosPush results; follower last against the leader 60 s after each fault374 of 374 pushes ok. Found a different defect, F45: restore ran out of memory on a log of large entries, since fixedtests/m1/results/m1-f43.md, "The soak"; tests/m1/results/raw/f43-20261102/soak/summary.txt

Git conformance on Cloudflare (BM-1, git half)

BM-1's fault is B0-03's simulated region loss: every call to and from site 0's member class fails from 120 s into each t/ run, or 60 s into each clients.sh run, to the end. The gray rerun fails each such call with chance 0.3 instead.

git's own smart-HTTP tests, 16 scripts from t/t55xx to t/t58xx, counted from each run's results.tsv:

RungitFaultTestsPassFailn/a (hosted)SkipBaseline passes lost
conformance-202611012.51.0none53541315998baseline
conformance-202611012.56.0none53241215978baseline
t-251, 2026-11-02 04:03Z to 04:49Z2.51.0site 0 failing535415149880
t-256, 05:51Z to 06:33Z2.56.0site 0 failing532412149881 (t5549 #2, a test that sets a server-side receive.hiderefs)
t-239, 06:33Z to 07:15Z2.39.5site 0 failing49137716908no 2.39.5 baseline. Against t-251, only t5541 #20 and t5549 #2 pass on 2.51.0 and not here, both from fixtures the host cannot reproduce
t5541-nofault, after the t5541 #20 fix2.56.0none4230363#20 now passes
RunsFaultResult
Stand-in group: 2.39.5, 2.51.0, 2.56.0none29 of 29 each
Real groups: 2.39.5, 2.51.0, 2.56.0none29 of 29 each
clients-239, clients-239-homed, clients-251, clients-251-homedsite 0 failing29 of 29 each
clients-239-gray, clients-251-gray, 2026-11-02 13:10Z to 13:22Zsite 0's calls failing 30%29 of 29 each. Every leader move was an election, and no STEP_DOWN fired on any member

SSH git sessions

Each session is one git clone, fetch, or push over SSH through a Cloudflare Tunnel to cloudflared in three containers. Two workers ran one session every 6 s each.

RunGateWhere and whenSessionsSucceededFaultsPushesResultSource
b0-24B0-24Cloudflare, Pro zone, 2026-10-30T23:58:59Z to 2026-10-31T02:10:49Z (2.2 h)2,6372,634 (99.886%)13 container kills878 acked, all in the group logBetween the lines (pass 99.9%, fail 99%)tests/m0/results/m0-wp7.md, "Results (run b0-24 ...)"; tests/m0/results/raw/wp7/b0-24/summary.json
b0-24a-r4B0-24aCloudflare, Pro zone, 2026-10-31T07:36:12Z to 2026-11-01T07:36:07Z (24.0 h)28,34728,230 (99.587%)143 container kills, one every 10 min9,418 acked, all 9,418 in the group logBetween the lines. 85 of 117 failures within 2 min of a kill, the rest Tunnel-edge drops. Carried to M6-WP2tests/m0/results/m0-wp7.md, "The run of record: b0-24a-r4"; tests/m0/results/raw/wp7/b0-24a-r4/summary.json
r1none (the plain-SSH relay the operator allowed until go-live)Oracle relay to the same Tunnel, 2026-11-03T00:54:33Z to 01:19:25Z (24.9 min)249 relay + 249 cloudflared control249 and 249One forwarder restart: sessions open at the stop were cut, new ones worked within about 0.4 s166 acked, all in the group log0 failures, which cannot tell this path from M0's 99.86% floor at this sample sizetests/m1/results/oci-ssh.md, "Measurements (run r1 ...)"; tests/m1/results/raw/oci-ssh-20261103/r1/summary.json

Other Milestone 0 fault and load rows on Cloudflare

These rows ran on the M0 prototype. Verdicts are the ones build/BUILD-PLAN.md, "Milestone 0 status", carries, or the results file's where that table has no row.

RowRunsLoad and faultsResultSource
B0-0123 one-minute runs, one entry in flight81 to 100 commands/s, 1 KBp99 399 to 690 ms, median 491 ms. Pass in 19 of 23 at the restated 600 ms line, none over 1 stests/m0/results/m0-wp2.md, first table
B0-023 reruns at 300 commands/s offeredGroup commit, one entry in flight159 to 280 commands/s carried at p99 750 to 1,288 ms. Between the lines, carried forward. Rerun on the Rust host (2026-11-11): 6 of 8 runs at 308/s carried at p99 286 to 509 ms, 2 between. Second rerun with the STEP_DOWN fix (2026-11-12): 12 of 16 at 276 to 579 ms, 4 between (678 to 790 ms), every miss on a group with a near follower over 120 ms. Between on every-run reading, F68. Third rerun on groups placed within NEAR_RTT (2026-11-12): 15 of 16 at 249 to 540 ms (8 of 8 at 300 offered, 7 of 8 at 330), 0 failed, and 1 between at 614 ms from one leader stall. Between. Fourth rerun with streamed member-call bodies (m0-carried-4, 2026-11-12): 11 of 16 at 342 to 588 ms, 5 between (614 to 738 ms), then 3 of 4; every miss follows a slow leader storage confirm (p99 132 to 399 ms) or one follower stall. Between. Fifth rerun, a same-window A/B of streamed and array bodies (m0-carried-5): 5 of 8 at 330 with each (361 to 911 ms), so the stream is not the cause; every miss is an AppendEntries held until the leader's own write confirms. Between, carried forwardsame; tests/m0/results/carried.md; tests/m0/results/carried-2.md; tests/m0/results/carried-3.md
B0-0326 runs (13 failing-calls, 13 blackholed) plus 6 reruns on the final codeOne site's member class failing or blackholed, about 100 ops/sRTO 11.0 to 12.9 s, 0 failed operations. Pass at 13 ssame, and "B0-03"
B0-06Three modes, 2 min each, plus reruns30% of one site's calls failing; 30% of its storage writes failing; only the edge's calls to it failingEvery group moved within 20 s. Pass, 2 minutes per mode, not an hoursame, and "B0-06 and STEP_DOWN"
B0-183 runs16 learners at 1,000 reads/s each16,000 reads/s, 0 failed, learner p99 683 to 743 ms. Pass at the restated 1 ssame
B0-259 runs with the STEP_DOWN hold, 300 pushes each22% of member calls and storage writes failing92.3% to 98.7% within three attempts, median 95.0%. Pass at the restated 92%. 17 earlier runs on older code gave 79.3% to 98.3%same, and "B0-25"; tests/m0/results/raw/b0-25-*.json
B0-264 runsEvery member of 4 or 8 groups reset inside 60 s under 100 writes/sEvery group committed within 1.2 s of its last reset. Passsame
B0-04, B0-05, B0-08, B0-09, B0-10, B0-27Site days on 300 or 1,000 groups100 writes/s; follower regressions, verify refusals, drain guardPass. 0 failed writes in every site daytests/m0/results/m0-wp4.md, first table
B0-07Runs 11 to 16 on the drain-alarm codeGray site C during B's site day179 of 182 groups moved within 20 s as written, 181 of 182 under the 50 s form. Carried. Rerun on the Rust host (2026-11-12) with test-faults stand-ins for graft-ops's drain and SiteHealth: 235 of 235 within 20 s over 3 days, max 17.2 s, 0 failed writes. Pass on the member path; M5-WP4 reruns it with the real draintests/m0/results/m0-wp4.md, "B0-07 on the final code"; tests/m0/results/carried.md, B0-07
B0-228 rehydrations5.38 GB network from R2 into a new containerAt most 118.1 s, median 67.9 s. Pass, thin margintests/m0/results/m0-wp6.md, "B0-22"
B0-238 runs33 CI clones/s, tip moving 10 times a minutePass with a push hook (100% from R2, p99 406 ms) or clones 5 to 60 s after a push (97.5 to 97.8%). Up to 44% refused at the instant of a push without the hooktests/m0/results/m0-wp6.md, "B0-23"
B0-29 to B0-35Eight passes of the agent-fleet replay500 agents, journey 9B0-29 passes in 5 of 8 runs. B0-31, B0-32, and B0-33 pass on the final rules. B0-30 passes at the margin (accelerated), and B0-35 passes. B0-34 is a partial pass at reduced scaletests/m0/results/m0-wp9.md, "Verdicts", "Second-pass summary", and "Eighth-pass verdicts"; build/BUILD-PLAN.md, "Milestone 0 status"
L0-12, LF-12L0-12 in the first and third to seventh passes, and LF-12 runs r2 to r37A1 replay with leases, held feed polls, and leader killsL0-12's feed clause passes for all five actor kinds in the sixth pass. LF-12 is between the lines: 1 pass and 3 between in 4 runs on the final rulestests/m0/results/m0-wp12.md, "Sixth-pass summary" and "Tenth-pass summary"

Simulator

crates/raft/tests/sim.rs runs three members under one seeded generator per run. HARSH drops 8% of messages, duplicates 4%, delays each 5 to 1,500 ms, crashes members, cuts sites off, and skews clocks. GRAY adds one gray site at a time: failing calls, 1 s storage confirms, or failing writes. After every message and step the simulator checks one leader per term, identical committed entries, two disks per committed index, every Acked write in every later leader's log, no RequestId at two indexes, BF-14's no-forward rule, and the ack evidence of every acked write (ack_check).

The long sweep, run on 2026-11-03 at c0c6fb9 with RAFT_SEEDS=2000 RAFT_STEPS=10000 in release mode. It ran twice with identical counts, in 344 s and 347 s:

HalfStepsOperationsAcked writesFaultsPorcupine on the exported historyResult
HARSH20,109,261160,066 (49,629 acked, 18,649 refused, 60,710 reads, 31,078 no reply)49,629, all with valid evidence226,083 crashes (97,866 mid-input), 31,230 partitions, 696,535 of 2,215,368 messages dropped, 3,524 leadersOk on 156,723 operationsPass, 0 forwarded
GRAY20,082,5432,226,291 (951,227 acked, 384,638 refused, 890,206 reads, 220 no reply)951,227, all with valid evidence17,108 gray faults, 24,327 write failures, 15,329 partitions, 1,534 STEP_DOWN transfers, 12,727 leadersOk on 2,226,280 operationsPass, 0 forwarded

Unit and property tests

cargo test --workspace at c0c6fb9, 2026-11-03: 259 passed, 0 failed, 2 ignored, in 25 test binaries.

CrateTests
graftyard (cli)20
graftyard-fold9
graftyard-git87
graftyard-identity39
graftyard-lookup1
graftyard-raft76, plus long_sweep and long_liveness ignored by default
graftyard-release13
graftyard-site3
graftyard-types11
graftyard-edge, graft-m1-wp4, graft-m1-wp5, graft-m1-wp6-bed0
Total259

Planted bugs

A planted bug is a code mutant applied alone, run against a test suite, and then reverted.

WherePlantedCaughtSource
crates/raft, M1-WP2 first pass88tests/m1/results/m1-wp2.md, "The tests catch real bugs"
crates/raft, M1-WP2 second pass43. Bug D, a resumed led_term sending its Noops before they are durable, was not caughttests/m1/results/m1-wp2.md, "The simulator"
crates/raft, Member::record11tests/m1/results/m1-wp2.md, "Gate-row pass, 2026-11-02"
crates/raft, the ack mutants A and B22, by the ack check under GRAY (54 of 3,873 acked writes flagged)tests/m1/results/m1-wp2.md, "The planted mutant"
crates/raft, M1-WP31717tests/m1/results/m1-wp3.md, "The tests catch real bugs"
crates/git, M1-WP4 pack store and sweeper, plus M1-WP5's alarm2525, after one plant was rewritten to compile and one miss led to a new assertiontests/m1/results/m1-wp4.md, "Adversarial unit tests and planted bugs"
M1-WP5: the shard server's LRU (2) and sweep_all (2)44tests/m1/results/m1-wp5.md, "The LRU of bare repositories" and "The sweeper's cron entry"
crates/git, M1-WP666tests/m1/results/m1-wp6.md, "Planted bugs"
crates/identity, M1-WP71818tests/m1/results/m1-wp7.md, "Planted bugs"
Total8584

Smoke, local, and discarded runs

None of these runs counts toward a total above.

KindRunsCounts
Smoke runs on Cloudflare, Porcupine Okcf1 (150 s), cf-remote-smoke (180 s), cf-remote-2 (300 s), cf-boot-smoke (10 min), lin-smoke1 (9 min), lin24b-smoke (30 min)112,985 operations: 2,729, 6,658, 11,188, 19,030, 16,243, and 57,137. lin24b-smoke's ack check was Ok on 18,225 of 18,225 acked writes
Local chaos runs on wrangler32 raft-chaos-local* runs744,258 operations, all Ok. Local workerd, not Cloudflare
Discarded as set up wrongwp4-cf-2, wp4-cf-3, wp4-cf-4; conf-run2 (stale shard containers)Not checked or not kept
Stopped 24-hour SSH attempts, each on a defect since fixedb0-24a-24h, b0-24a-r2, b0-24a-r31,144 sessions: 341 (338 ok), 536 (533 ok), and 267 (263 ok)
Stopped t/ runt-256-firstFound fix 4 in m1-wp8.md, then rerun as t-256

build/BUILD-PLAN.md

Milestone 0: measure the platform and the protocol

Milestone 0 status (2026-10-31)

Every package ran on prototype code in the one Milestone 0 account, M0-WP6 and M0-WP7 on its one Pro zone, graftyard.dev. Results are in tests/m0/results/, one file per package, and each fix the runs found went into the design first (decisions/DECISIONS.md, "Milestone 0"). No row hit its Fail value. Milestone 0 is closed for starting M1 (FRAME.md, "Operator decision, 2026-10-31, Pro only and no GHES"; decisions/DECISIONS.md, "Decisions for a Pro-only build with no GHES, 2026-10-31", "M1 starts with the between rows carried forward"): the rows below carry forward to the owner named, with no gate number changed, and a Fail value on any of those reruns still sends the design back. Rows that need a plan, an instance, or accounts the operator does not have stay not run, owned by whoever first has them, and gate no M1 work. Rows still short of their Pass value, each with its owner:

RowStateWhat closes it
B0-11Passes: 0 violations on 5,742,736 operations and 0 lost acknowledged writes in 48 hours, with a 100 ms clock allowance across client colos that the run did not measure; fault coverage on site C is thin (F36). Rechecked with the fixed segmented checker, the verdict is unchanged at 0 ms and 100 ms: none of its 4 no-reply writes was ever observed, so dropping them, as M0's check did, changed nothing here, though it is not "only stricter" in general (tests/m1/results/porcupine-seg.md). Its 24-hour rerun on the M1 code, lin24, had fault counters in all 88 isolates and recorded clocks, and did not pass, because an early ack was not ruled out; the second rerun, lin24b, passed with member-side ack evidence, which answered that as a client-clock artifact ("Milestone 1 status", BM-1; decisions/DECISIONS.md, "F44 answered: the fast acks were a client-clock artifact")Done (lin24b)
B0-01Passes on the Rust host (m0-carried, graft-m3c, 2026-11-11): 6 of 6 one-minute runs at 100 commands/s, commit p99 244 to 377 ms, 0 failed; M1-WP2's host had passed 2 of 6 (tests/m0/results/carried.md)Done
B0-02Passed on m0-carried-8 (graft-m3d, standard-4 shards, 2026-11-13, tests/m0/results/carried-8.md): 8 of 8 at 300 and 8 of 8 at 330, commit p99 249 to 579 ms, on main 84d29c2 (F67 rules 10 to 13) plus an AppendEntries pipeline (PIPELINE 4, stopped only on a lossy link), the LOOP_LAG step-down, ENTRY_SOFT at 100 KB and APPEND_BYTES_MAX at 100 KB. Heavy turns stayed at the 60 KB tree's level (28 /commit turns of 80 ms or more over 16 runs against 24; CPU p999 23 to 49 ms), and confirm p99 was 108 ms at 80 to 100 KB and 131 ms at 100 to 130 KB. Long sweep (live half 4,000 seeds at an absolute 30 s), Porcupine, long_liveness, plants 15 of 16 and the full checks pass. Before the two size changes the same code passed 12 of 16. Open: one run's LOOP_LAG trip moved the leader to weur, where the apac voter's link was lossy at these call sizes; and LM-1's B0-29 A/B on this tree before mergeMerge after LM-1's journey 9 A/B
B0-07Passes on the Rust host (m0-carried, graft-m3c, 2026-11-12), with test-faults stand-ins for graft-ops's drain and SiteHealth, which the hosted code does not have yet: 3 days, C detected 3.6 to 3.7 s after the fault, 235 of 235 groups led on C moved within 20 s (max 17.2 s), 0 failed writes, never two sites drained (tests/m0/results/carried.md). M0's one group at 20.2 s stays unexplained for the prototypeDone for the member path; M5-WP4 reruns it with the real graft-ops drain
B0-20, B0-21Ran on the Pro zone graftyard.dev: the 100 MiB cap holds for sized and chunked bodies, and git over it sees a bare 413, as the 413 decision accepts. Business and Enterprise not run; their caps (200 MB, 5 GB) stay docThe first install on that plan
B0-24aBetween pass and fail: 99.587% of 28,347 sessions over 24 h with 143 kills; 85 of 117 failures within 2 min of a kill, the rest Tunnel-edge drops (background near 99.89%); 0 acknowledged pushes lost. The option stays built, off by default. Gates only M6-WP2's SSH option, not M1M6-WP2's rerun on real code
B0-24bEnterprise Spectrum: not run (Spectrum refuses a Tunnel origin on Pro); the ingress is designed and not builtThe first Enterprise install
B0-29Passes in 5 of 8 runs; misses at 602 and 625 ms track placement (first pass, r2 and the sd3r1 variant) and 809 ms tracks storage stalls on the leader (fifth pass, p5r1)LM rerun on real code
B0-34Partial pass at reduced scale, with the composed-miss rules now in the designA full-scale run with the warm pool
L0-06, L0-11, L0-04Partial: predictor without other git work; byte diff without a GHES oracle, which becomes the pinned ghes-3.22 descriptions and github.com inference; agent CI without an Actions engineLM rows that rerun them on real code (LM-4, LM-1, LM-3)
L0-07Vendor cells not run: they need vendor accounts and a public host; the documented-behavior drivers are the evidence (Codex's documented permissions fail the check cells). Does not block M1; no LM gate names itThe operator's vendor accounts
L0-08The ghr_ cell needs a GHES 3.22 instance, and none exists. AM2-WP1 builds A4's fallback, so the cell becomes the held-token refresh and A4 gates nothingLM-2's rerun of L0-08 on real code (AM2-WP1)
LF-12Passes on real code after am1-fix (lf1-6, AM1-WP6 status): leader commit p99 474 ms placed and 710 ms before, 5,000 of 5,000 polls held on 11 learners and 0 on voters, and 0 failed AppendEntries calls of any kind in 165 /stats samples, so zero overloaded (tests/m0/results/carried.md, LF-12). On the prototype it was 1 pass and 3 between in 4 runsDone

M0-WP1. Test bed, bindings, and placement facts

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
B0-12Member placement: 1,000 members created with each site class's hint, median of 20 binding round trips to each other voterAt least 95% of voter pairs at 30 ms or more and consistent with the hinted regionsUnder 80%
B0-13Bucket placement: 20 buckets per hint, location from the get-bucket API against a latency probe100% agreeThe API reports the hint, not the placement
B0-15Idle cost: 100,000 idle groups for 24 hours across a UTC midnight, with the daily Rtt recheck shortened to hourly, beside 1,000 control groups that serve one request an hourZero duration billed and zero Rtt checks for idle groups, while the control groups show bothAny duration billed, or any Rtt check, for idle groups, or a control group that shows neither
B0-16Account-owned deploy token with Workers Scripts Edit and Containers Edit runs a deploy with a container imageDeploysFails, which moves the deploy token to a user token held by the customer
B0-38Shard containers under the durable_object policy: 30 starts from objects created with each site's hint, then a new site-script version with a changed image map while three runEvery container in its site's container regions, and no running container restarts or changes imageA container outside its site's regions that the start check misses, or any restart on deploy

M0-WP2. Raft prototype on Durable Objects

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
B0-01Commit p99 on Tri, leader enam, 1 KB commands, 100 commands/s, member calls through cross-script DO bindingsAt most 600 msOver 1 s
B0-02Group commit on one group, one entry in flightAt least 300 commands/s at commit p99 at most 600 msUnder 100 commands/s
B0-03Write and read RTO, with one site script's member class failing every call (simulated region loss) under 100 writes/s, run once with every client in one colo and once with calls to that class blackholed (hung, not failing)New leader committing and serving fresh within 13 s at p99 in both runsOver 60 s in either run
B0-11Linearizability: 48 hours of Porcupine-checked histories while member calls to one site are dropped at random, members are reset, and leadership transfers, some abandoned mid-wayZero violations, zero lost acknowledged entriesAny one
B0-25Partial DO failure: inject 22% errors into member calls and into storage writes, which reset objects, and count whole pushesAt least 92% of pushes succeed within three attemptsUnder 84%
B0-26Reset storm: every member on all three sites reset within 60 s under 100 writes/s, with suspect reports from two colosEvery group commits within 12 s of the last resetAny group over 12 s

M0-WP3. Gray failure and learners

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
B0-06Gray site, with 30% errors injected on one site's member calls, then separately on its storage writesEvery group led there moves within 20 s and changes leader at most twice in an hourAny group slower, or more than two changes
B0-18Learners: 16 learners serving 16,000 reads/s on one groupLeader under 700 req/s, learner read p99 at most 1 sLeader returns overloaded

M0-WP4. Site days, verify refusals, and drain

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
B0-04Site deploy under 100 writes/s, with groups led from every site and clients on the leaders' continents: drain, deploy at 1%, 10%, 50%, 100%, undrainZero failed writes, commit p99 at most 1 s for clients on the leader's continentAny failed write
B0-05Follower regression: a release whose N+1 followers throw while applying one event kindThe apply gate stops it at 1% and graft-ops runs the pre-signed rollback, with no failed write and no human signatureIt reaches 10%, or any write fails
B0-07Gray site C during B's site day at 10%, with automatic drain onCallers on A and B report C, graft-ops undrains B under the 1% lead limit and drains C. Every group led on C that served a request or committed an entry in the 50 s before the drain moves within 20 s of it, and every other group led on C moves within 20 s of its next requestTwo sites drained at once, or any group on C slower
B0-08verify refusal by an older follower: release N+1 on site A writes a shape N cannot readEvery group A led moves to an undrained follower holding its last index; other writes keep committing; A stays non-leaderAny group without a leader for more than one lease
B0-09Rolling site refuses: N+1 followers on B refuse valid N entries at 10% of a site dayNo leader steps down, B is marked unable to follow, an alarm fires, and the site day halts and rolls backAny group without a leader for more than one lease
B0-10Two-site residual: A and B on N+1 commit a shape C on N cannot readEach group keeps its leader and commits on A and B, C is marked unable to follow, and the site day haltsAny group without a leader for more than one lease
B0-27Drain guard: two drain-key SiteMode { Drain } entries for different sites submitted straight to the Release group, then a test group whose voters are all drained by customer-key entriesThe second drain-key entry is refused and pages, and the all-drained group elects a leader within 12 sTwo sites drained by the drain key, or the all-drained group leaderless over 60 s

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
B0-17Account-separated option: member RPC over signed HTTPS through the site Worker, against a cross-script bindingp50 overhead at most 20 msOver 60 ms, which withdraws the option

M0-WP6. Git on Containers and R2

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
B0-14Daily working set of git cache shards per tier, in bytes, on a synthetic replayRecorded; cost.py shard counts updatedNone. The result sets the shard count
B0-19Container starts with the control plane out of reach: 100 new instances started with every account API token revoked100 of 100 startAny fails; the control-plane row stays contained either way and warm shards stay sized for a day
B0-20Push size: 2 GB push through the edge on Enterprise, teed to two buckets and a container; on Pro and Business, a push over the edge capEnterprise completes with Worker memory under 100 MB; Pro and Business print the edge's report-status naming Enterprise or splitting the pushFails, needs buffering, or shows a bare 413
B0-21Chunked push on Pro, Business, and EnterpriseRecorded either wayNone. The result sets each plan's stated cap
B0-22Container rehydration of a 5 GB network from R2At most 120 sOver 600 s
B0-23Clone cache: 33 CI clones/s of a monorepo whose tip moves 10 times a minuteHit rate at least 95%, time to first byte p99 at most 500 msUnder 80%

M0-WP7. SSH chain

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
B0-24aThe Pro and Business SSH option: clients through cloudflared access ssh as their ProxyCommand to a Tunnel public hostname of type ssh, then cloudflared in three containers, with writes through the outbound handler and one container killed every 10 minutes for a day. On Pro, 2,634 of 2,637 sessions (99.886%) over 2.2 h with 13 kills, then 28,230 of 28,347 (99.587%) over 24.0 h with 143 kills: between the lines. 85 of the 117 failures came within 2 min of a kill, and the other 32 were Tunnel-edge drops (stream closed before the banner, no banner within ConnectTimeout, 90 s timeouts), a background near 99.89%. Failures are dominated by sessions in flight on the killed replica, so a 10-minute kill period puts a floor of about one failure per kill under the rate. 9,418 pushes acknowledged, all 9,418 in the group log [measured, tests/m0/results/m0-wp7.md, B0-24a; tests/m0/results/raw/wp7/b0-24a-r4/summary.json]. Carried to M6-WP2's rerun on real codeAt least 99.9% of sessions succeedUnder 99%, which leaves Pro and Business HTTPS only
B0-24bEnterprise native SSH: Spectrum on port 22 to the Tunnel's virtual network, then the same containers, handler, and kills for a day. Not run: it needs an Enterprise zone. On Pro the Spectrum API refused a Tunnel origin (10012, 13002) measured, sameAt least 99.9% of sessions succeedUnder 99%, which drops native SSH on Enterprise

M0-WP8. Restore window

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
B0-28Restore window: delete a branch and force-push another on day 1 with GC and archiving running, then restore both to day 0 on day 30 (clock-skewed replay)Both refs return to their day-0 targets, every object they reach is readable, and archives are named by committed Archived records in two bucketsThe restore is refused, any object is missing, or a follower truncates past a range with no committed Archived

M0-WP9. Agent load on the base

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
B0-29Agent fleet replay (journey 9) on one Tri group, leader enam. 500 agents push distinct refs every 2 min (40 KB mean, 5% at 1 MiB), 27.6 commands/s with PrSync, test-merge records, check runs, task updates, and lease renewals, 45 feed polls/s on followers and learners, 10 merges/min, 33 ref reads/s, 200 API reads/s. Commit p99 on this row is set by storage stalls on the leader: of the seven fourth-to-seventh-pass runs, the five with 0 to 4 failed AppendEntries in 600 s passed at 378 to 531 ms, and the two with 10 and 12 missed at 798 and 809 ms [measured, tests/m0/results/m0-wp9.md, fourth to seventh passes]. The line stays one 600 s run at 600 ms, and M1-WP2 reruns it on real codeCommit p99 at most 600 ms, merge p99 at most 2 s, zero Stale on distinct refs, leader under 600 req/s with zero feed polls, API read p99 at most 2 s with at least 99% served ("Follower read")Commit p99 over 1 s, any queue merge waiting over 10 s, any feed poll on the leader, or API read p99 over 5 s
B0-30Hot-group storage. 30 days of the journey 9 replay on a group seeded with a 4 GB metadata tierMember SQLite under CHILD_GROUP_AT with zero SQLITE_FULL, and archived entries readable for RestoredToAny SQLITE_FULL, or member SQLite over CHILD_GROUP_AT without a child group taking the tier
B0-31Same replay with the leader on apac and every agent push at 1 MiB, INLINE_MAX_HOT on, then with every bucket's placement failing, under the design's STEP_DOWN with no variant. The placement-failing half gates Human and MergeQueue latency, not the leader's site: the apac leader may move once on the Human packs that the outage rule keeps inline. On the final Milestone 0 code both halves passed: buckets up, the apac leader held 600 s with commit p99 598 ms and every agent push staged; placement failing, Human and MergeQueue p99 1.11 s with 0 of 287 refused, 2,500 of 2,500 agent pushes refused at once in the ng shape, and one STEP_DOWN from apac at 8.9 s (6 and 3 failed calls) [measured, tests/m0/results/m0-wp9.md, B0-31, eighth pass]Commit p99 at most 1 s with buckets up. While placement fails, Human and MergeQueue commit p99 at most 3 s and 0 of them refused, and agent pushes over INLINE_MAX_HOT refused at once in the ng shapeOver 3 s, or any Human or MergeQueue push up to 1 MiB refused while placement fails
B0-32Burst of 500 agent pushes of 256 KiB each in 1 s with merges queued, admission on. The burst rides a hot group, so the row tests INLINE_MAX_HOT, which stages each 256 KiB pack before AGENT_INLINE_ENTRY is reached; a burst on a quiet group is covered by the budget's B0-33 evidence (fresh groups' first entries capped at 128 KiB, seventh pass). On the final Milestone 0 code 505 of 505 burst pushes committed within 5 s (client p99 1.40 s, stage p99 1.19 s), the burst's merges took 129 to 227 ms, each in the next entry (up to 2.30 s at a 256 KiB cap), queued agent bytes peaked at 119 KB, and there were 0 leader changes and 0 suspect [measured, tests/m0/results/m0-wp9.md, B0-32, eighth pass]Every push acknowledged within 5 s or answered in an HTTP 200 with ng <ref> agent admission full, retry after <n>s, queued agent bytes under AGENT_QUEUE_MAX, every merge in one of the next 2 entries, zero leader changes, and zero suspect reportsAny lost acknowledgment, any other refusal shape, an object reset from memory, a merge delayed over 2 s, any leader change, or any suspect report
B0-335,000 agents on one group, 252 agent-class commands/s offered with caused commands classed, plus one user token writing 100 commands a minute, admission on. With outcome folded into commit, AGENT_INLINE_ENTRY on every group, and hot from the log, three far-flung runs (ATL leader, weur and apac followers, fresh groups) held their leader with 0 suspect, Human and MergeQueue p99 746 to 943 ms, 253 agent commands/s admitted, and the leader at about 500 req/s [measured, tests/m0/results/m0-wp9.md, B0-33, seventh pass]. With commit_fail_min added, a fourth run passed again at Human and MergeQueue p99 674 ms, 250 agent commands/s, and 0 leader changes; no window reached the 5% rate measured, same, eighth pass. Before the fold the leader served 750 to 950 req/s and 2 of 3 runs lost it by election (fifth pass). The zero-leader-change claim is measured for this mix, with a margin of 57 to 326 ms under 1 s, least in the first 30 sHuman and merge-queue commit p99 at most 1 s, at least 140 agent commands/s admitted, the user token admitted as Agent, every refusal in the git ng or REST 429 retry-after shape, zero leader changes, and zero suspect reportsAny leader change or suspect report, any merge-queue command waiting over 2 s, or under 100 agent commands/s admitted, which reopens ref lanes with a parent Abort { txn } record (design/AGENT-LAYER.md)
B0-34Git CPU for one 5 GB network under the journey 9 replay, with the tip moving at 10 merges a minute along real history, then one warm site killed, then a second network that turns hot with the API unreachable, then a hostile owner making three of its own networks hot beside itk set in band from NetworkLoad and the hot pool, at most 8 per site, the hostile owner held to its per-owner pool budget, human push p99 at most 3 s beyond transfer, and clone time to first byte p99 at most 1 s throughoutk over 8, any Cloudflare API call, the monorepo's k reduced by another owner, or time to first byte p99 over 5 s
B0-35Lazy test merges, with the base moving 10 times a minute under 500 open PRs and 500 clients polling their PR every 5 sTest-merge vCPU within TEST_MERGE_BUDGET, Actions dispatch p99 at most 10 s, and zero computes from list fieldsOver budget, or dispatch p99 over 30 s

M0-WP10. Agent refusals and provenance on the base

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
B0-36Agent refusals, run once with a Graftyard-run token and once with a confined vendor App's token, used without the controller. Each tries approve, merge, ready-for-review, a bypass, dispatch, workflow re-run, a release, a label outside the allowlist, a graftyard/ status, a push outside its prefix, a push with no live task and a fork push after its task is cancelled (Graftyard-run token only; a confined App is bound by confine), a REST ref write into another task's Exclusive scope under AgentsExclusive, and a change under the protected list, over git, REST, and GraphQL where GHES 3.22 has a surface for the action. The Graftyard-run token runs once with its shipped permissions and once on an installation widened to every permission, and a refusal from either counts. It also pushes an outsider commit that changes a workflow under its ref, pushes after an approval and then merges, and posts graftyard/agent-gate. Then a flagged App without confine, Graftyard MCP, and a user token with X-Graftyard-Agent-Session push into the same Exclusive scope, and a user token pushes a commit whose only trailer is Co-authored-by100 of 100 refused for each, for both agent kinds. The other three are never refused by the lease, and the Co-authored-by push stays HumanAny accepted, any of the other three refused by a lease, or the Co-authored-by push classed Agent
B0-37Provenance and hidden refs. 10,000 pushes with malformed trailers and notes, forged Graftyard-Session trailers, and writer pushes to refs/graftyard/, then v0, v1, v2, and --mirror fetchesZero pushes fail on parsing, every forged trailer stays Claimed, every refs/graftyard/ update not by graftyard-workspace is refused, and session refs appear only to a v2 ref-prefix that starts with itAny push fails on parsing, a forged trailer links, or a session ref is advertised otherwise

M0-WP12. Leases, the activity feed, and the core tier (layer rows)

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
L0-01Gray enam (30% errors) during the A1 replayBase and fork groups move within 20 s, 99% of agent pushes succeed within three attemptsAny group slower, or under 97%
L0-12Leases and the feed. The A1 replay with a lease per agent and leader kills, then a 30-minute stall with every member running and member calls blackholed, then an idle 60 s lease followed 10 minutes later by an overlapping Exclusive acquire, then 100 conflicting pushes and 100 conflicting REST ref writes into another task's Exclusive scope under AgentsExclusive, run once per actor kind: Graftyard-run App, confined vendor App, flagged App without confine, Graftyard MCP, and a human user tokenNo two holders hold overlapping Exclusive leases for longer than DRIFT (500 ms), and measured member clock offsets stay under 500 ms. Every lease live before the stall is live after it, with expires moved by the recorded stall. The late Exclusive acquire succeeds. 500 long-pollers see each event within FEED_BATCH plus 2 s (3 s) at p99 outside leader kills. 100 of 100 refused for each of the two ActorKind::Agent kinds, 0 of 100 for the other threeAny overlap over 500 ms, any lease expired by the stall, the late acquire refused, feed p99 over 2 s, or a refusal count off for any kind
L0-13Lease limits. Exclusive acquires of **, *, and */x, one with no live task, and a second for one task; an App-token acquire with no session; 600 plan blocks from accounts without push access, then 600 from writers; a holder whose push access is removed; an App uninstalled while it holds leases and tasks; an Exclusive lease renewed for 5 h100 of 100 refused for each bad acquire. Zero plan leases from non-writers and at most 500 from writers. The removed holder's lease refuses nothing at the next decide. Uninstall releases every lease and cancels every task. The lease reads as Shared after 4 hAny accepted, any plan lease from a non-writer, or an Exclusive lease past 4 h
L0-16Core tier alone. Pushes, minting, and lease checks on the A1 replay with the metadata tier unloaded (a merge reads its pull row and writes Merged, both Metadata, so it waits for the tier)Every check passes or refuses with zero metadata loadsAny metadata load on those paths
LF-12A1 replay on one Tri group with no learners, plus 5,000 held feed polls with paths, then again after place_learners runs. On the final rules, with the own-stall exclusion and the all-lagging Retry-After of FEED_BATCH_ALL, 4 runs gave 1 pass and 3 between the lines, with no fail, no step-down, and every other line passing in every phase. The misses trace to Durable Object storage confirm stalls on the leader or the near follower and to the far follower's round trip, as for B0-02 and B0-29 [measured, tests/m0/results/m0-wp12.md, LF-12, tenth pass]. The gate is one run, as B0-29's isCommit p99, measured at the group's leader (admission to resolution), at most 600 ms once place_learners has run and at most 1 s before it, zero overloaded replies to AppendEntries, at most 1,000 polls held per member with the rest answered 429 with Retry-After, zero polls on the leader, and zero on followers after placementCommit p99 over 1 s once place_learners has run or over 2 s before it, any overloaded replication reply, any member above 1,000 held, a refused poll without 429 and Retry-After, any poll on the leader, or any poll on a follower after placement
LF-131,000 polls held on a follower, then its member calls failed while commits run and leadership moves to the other follower, so the new leader's first round leaves a gap, then its log truncated past the follower for an InstallSnapshot catch-up (one leader re-sends from the follower's match, so dropped rounds alone leave none, crates/raft/tests/lf13.rs; hosted, the follower's calls are severed until it stands behind the other follower, then leadership moves there, tests/am1/results/lf13-m3c.md)Every held poll returns within 1 s of the follower's stop, when it learns it is behind, with its cursor and no events, new polls are refused until it catches up, and no event is lost; the time from the fault's lift is reported too (A26, decisions/DECISIONS.md, "LF-13's 1 s runs from the follower's stop")Any poll held or accepted while it is behind, or any event skipped. Between the lift and its stop the follower cannot know it is behind: it admitted 0 to 2 of 50 probes there, and its stop answered them with nothing skipped [measured, graft-m3c]

M0-WP13. Token ceilings, tasks, the agent gate, and the trust view (layer rows)

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
L0-03Token ceilings with raw tokens, bypassing the router. A workspace token pushes to the base, to another task's prefix, and to .github/workflows/, a token minted with no live AgentTask pushes, a fork-mode token pushes to its fork after its task is cancelled, and the base token merges100 of 100 refused for eachAny accepted
L0-09Trust view. 200 injection payloads (HTML comments, zero-width, bidi, link titles, collapsed blocks) in bodies, comments, commit messages, and check-run text, plus a non-writer edit after assignment100% absent from visible and counted, body byte-identical, and the edited task pausedAny payload in visible, or the task runs on the edit
L0-14Task rules. With agents_enabled off, then on: Graftyard-run creates from an installation token, an AgentApp's user token, an MCP token, a demoted user, and a read-only user; a controller create from an issue assigned to another bot; 101 open tasks for one requester; PATCH of a cancelled task to in_progress; a token request naming another App's task or a repository outside the token's list; a body that sets prefix422 for every create while off. When on, 100 of 100 refused for each bad case, the prefix always derived from session, and every create and cancel at the controller within 5 s, at no other App's webhookAny accepted, or the delivery reaches any other App
L0-15Gate and read permissions. An MCP client with WriteApproveMerge approves an agent PR. Code Mode calls approve and merge through codemode.request() without that scope. A vendor's named requester approves its PR. Anonymous, metadata: read-only, and over-limit feed polls, and /log and prompt reads with metadata: read onlyAgentGate stays unsatisfied in all three cases, and Code Mode is refused 100 of 100. Feed polls get 401, 404 (no contents: read, as task and lease reads answer), or 429, the leader sees zero feed requests, and /log and prompts return 403Any gate satisfied, any Code Mode approve or merge, or any read served

M0-WP14. Sandbox containment and agent CI (layer rows)

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
L0-02Sandbox containment, 100 attempts each to reach an unlisted host, read a token from env, files, or /proc, push to another repository on the host with a sandbox-supplied token, create a gist, call GraphQL, write a status or check run, send a model request with mcp_servers or a server-side tool, and fetch an unknown Go module path0 succeedAny succeeds
L0-04Agent CI under ci: AutoReadOnly on the A1 replayAgent-push workflows start within 30 s with a read-only token and no secrets, and jobs that reference secrets waitAny secret reaches an agent-push job before AgentGate holds for its head

M0-WP15. MCP and unmodified agent clients (layer rows)

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
L0-07A2 oracles. Claude Code GHES manifest flow, Cursor's GHES app, Codex's app template, local github-mcp-server default toolsetsEach installs, gets webhooks, opens a PR, reads graftyard/overlap, writes a check, and revokes, with zero Graftyard-specific client code. A push 1 s after revoke is refused 100 of 100Any needs a patch, or a revoked token pushes
L0-08MCP. Three 2026-07-28 clients and one 2025-11-25 client, CIMD, PKCE, consent, 30 calls each100% succeed, tool overhead p50 at most 50 ms, 100 of 100 wrong-audience tokens refused, a refreshed token keeps its scope 100 of 100, a replayed refresh revokes the grant, a framed or skipped consent issues no code, and a refresh succeeds with the unscoped ghu_ held, which is deleted after itAny wrong-audience token accepted, any refresh wider than its grant, or any code without consent
L0-11Byte diff. 1,000 REST and GraphQL requests from unmodified clients, with and without the agent layer deployed, and the same requests against a GHES 3.22 oracle: with no GHES instance, the pinned ghes-3.22 OpenAPI and GraphQL schema for shape, and github.com for behavior on operations whose descriptions match, weaker than an appliance inferenceWith and without the layer: identical except request ids and times. Against the oracle: every difference listed, and none in a field a client readsAny other difference with and without the layer, or an unlisted oracle difference

M0-WP16. Prediction and provenance (layer rows)

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
L0-05Prediction on 1,000 replayed branch pairs of a real monorepo100% of textual conflicts on current tips found, at most 5% false conflicts, push to verdict p95 at most 30 s, overlap check on the head SHA before a PR existsAny missed conflict on current tips, or p95 over 120 s
L0-06Predictor under the A1 replay, then with its site leadingSecond-site shard CPU at most 50%, and zero predictions while that site leadsOver 60%, or any prediction while it leads
L0-10Provenance. 10,000 replayed agent commits, then a vault restore100% resolve to a session or PR before and afterAny lost

build/BUILD-PLAN.md

Milestone 1: groups and git

Milestone 1 status (2026-11-04)

Milestone 1 is closed: every gate row below is Done, the last being BM-1's 24 h half on lin24b (2026-11-04).

Every row ran on the test install graft-m1-site-a/b/c in the Milestone 0 account, or, where the table says so, in the crate's simulator or with a stub. Results are in tests/m1/results/, one file per work package, and each design fix they found went into the design first (decisions/DECISIONS.md, "Decisions from the M1 builds, 2026-11-01", "BM-1 and the M1-WP8 fixes, 2026-11-02", "The hosted closing runs, 2026-11-02", and "BM-1's 24-hour half and F43, 2026-11-02").

RowStateWhat closes it
BF-10Pass on Durable Objects: 1,000 of 1,000 chains with one group from two edges at once, and 100 of 100 during an owner move with the bootstrap claim (m1-wp7.md)Done
BF-11Pass on Durable Objects: 1,500 of 1,500 cold-cache requests found the group after a member retry, a ReplaceMember stand-in, and an owner move (m1-wp7.md). ReplaceMember without its Workflow, spare, or JointDone for M1; the full ReplaceMember is M6-WP1
BF-12Pass on Durable Objects: 10,000 of 10,000 kills kept their outcome on the register, and 1,000 of 1,000 on Push through the git edge (m1-wp2.md, m1-wp6.md)Done
BF-13Pass on Durable Objects: 0 refusals from uncommitted entries in 400 rounds, plus 120 with early AppendEntries; Porcupine Ok (m1-wp2.md)Done
BF-14Pass on the hosted edge: 2,400 commands across Push and identity (PutAccount) commands each missed its first receipt, half held 2.6 s and half dropped, and 0 were forwarded to a second member; also 0 in the simulator (m1-wp8.md, "Gate rows closed"; m1-wp3.md). The misses were injected by the test-faults build, not real network loss of a replyDone
BF-15Pass on Durable Objects: 1,000 of 1,000 queued commands refused 401 after a revocation, and after a suspension (m1-wp7.md)Done
BF-16Pass on the real groups (m1-wp6.md)Done
BF-17Pass on the real groups, 12 of 12 rows (m1-wp6.md)Done
BF-18Pass with a real ChainLog on a real repository group (run 2): git fsck clean on both shards, 9 of 9 clones matched, and the second shard served the newer pushes. Run 1 failed on a thin stored pack in the promoted range, now a design rule (Shape, "Packs"). Stubs: a test route triggers the archive, a key in one bucket is not re-put, last_promoted's index is an upper bound, consumed is empty, and RepoFold::check requires every archived range to be promoted (m1-wp5.md, "BF-18 on the hosted install")Done
BF-19Pass on the hosted cron with the real request table: 0 of 1,000 committed packs deleted, 1,000 of 1,000 missing markers repaired, and 1,000 of 1,000 failed placements deleted, none before 24 h. Stub: the 24 h ran on the sweeper's injected clock, not a day of waiting (m1-wp4.md, "BF-19 on the hosted cron")Done
BM-1, git halfPass with site 0's member class failing every call for the rest of each run: clients.sh 29 of 29 on git 2.39.5 and 2.51.0; t/ on 2.51.0 and 2.56.0 lost no baseline pass except t5549 #2, which sets a server-side receive.hiderefs the host does not apply inference (m1-wp8.md). It also passes a gray fault, site 0's calls failing 30%: clients.sh 29 of 29 on 2.39.5 and 2.51.0, every leader move an election and no STEP_DOWN on any member (m1-wp8.md, "BM-1 gray-fault rerun"). Not covered: t/ under the gray fault, other rates or sites, and a no-fault t/ baseline on 2.39.5 (compared with 2.51.0 instead)Done; the 2.39.5 baseline goes with BM-1's next rerun
BM-1, 24 h linearizabilityPass on lin24b (tests/m1/lin/raw/lin24b.report.json; m1-wp8.md, "The 24-hour half, 2026-11-04"; m1-wp2.md, "The 24-hour rerun's result, 2026-11-04"): repo:934783, 2026-11-03T00:55:07Z to 2026-11-04T00:55:07Z, 2,850,523 operations, 0 without a reply, p50 314 ms, p99 952 ms, under 4,176 resets (2,795 of a leader), 2,752 transfers (855 abandoned), and fault counters in 97 isolates. Porcupine Ok on all 6 keys widened 100 ms and 153 ms (the measured 305 ms ATL-SIN bound, halved); on the recorded clocks Illegal on k0 to k4 in 20 segments, 18 reads or pending writes of client rows lost in 15 of 12,535 cut chunks (Ok with them restored) and 2 cross-colo ties needing 1 ms and 4 ms of skew inside the measured bounds, none same-colo. 0 of 904,294 acknowledged writes lost, three identical logs of 513,232 entries, 0 of 805,201 refused CAS writes in the log, the member-side ack check 0 violations on 904,294 writes, and B0-11's same-clock early-ack test 0. That answers F44 as a client-clock artifact (decisions/DECISIONS.md, "F44 answered: the fast acks were a client-clock artifact"). Before it, lin24 did not pass: on repo:915635, 24 h to 2026-11-02T18:27Z: 2,739,691 operations, 0 without a reply, under 5,547 resets, 3,743 transfers (1,131 abandoned), and faults in 88 isolates. Porcupine Ok on all 6 keys widened 100 ms and 275 ms (the measured 550 ms LHR-SIN bound, halved); on the recorded clocks Illegal on k2, k4, and k5, all 5 segments reads of writes whose client rows were lost in 5 of 12,518 cut chunks, and Ok with them restored. 0 of 869,257 acknowledged writes lost, 0 of 774,418 refused CAS writes in the log, three identical logs of 479,965 entries, and B0-11's same-clock early-ack test 0. An early ack was not ruled out: entry 398032 needed 93 ms of ATL-LHR skew against a measured 49 ms, and 17 writes were acknowledged faster than any quorum could commit them on their own client's clock (m1-wp8.md, "The 24-hour half, 2026-11-02"; tests/m1/lin/raw/lin24.report.json). lin24b's 30-minute smoke had passed: 57,137 operations, Porcupine Ok on the recorded clocks and both widenings, and the ack check Ok on 18,225 of 18,225 writes (m1-wp8.md, "The 24-hour half, 2026-11-03"). Every history is checked per key with Porcupine's segmented mode, which now cuts each key on its own and keeps a no-reply write that a later operation observed; the old mode dropped every no-reply write, which can make a linearizable history look Illegal, and cut all keys of a file as one list. Rerun on the fixed checker, lin24's recorded-clock and 100 ms verdicts are unchanged; the 275 ms check needed over 7 GB for one key and was not rerun, and stands because its input has no no-reply writes and one key per file, the path both versions share (tests/m1/results/porcupine-seg.md). lin24b's finish builds the fixed checker. lin24b's pass rule was zero Porcupine violations, no acknowledged entry lost, and zero violations of the member-side ack check's six rules (tests/m1/results/m1-wp2.md, "Member-side ack evidence, 2026-11-03")Done

M1-WP2. Raft group core

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
BF-12Leader killed after quorum and before the reply, 10,000 times. The edge re-sends with the same RequestIdOriginal outcomes returned, zero double appliesAny double apply or changed outcome
BF-13Leader lost before commit, on entries with a Stale refusal caused by an earlier command in the same entryZero refusals returned from uncommitted entries (Porcupine)Any refusal returned from an uncommitted entry

M1-WP3. Leadership and health

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
BF-14Receipts that miss CALL_DEADLINEZero commands forwarded to a second memberAny command forwarded

M1-WP4. Pack store and R2 keys

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
BF-191,000 failed placements, plus 1,000 commits whose marker write is killedFailed ones deleted after 24 h, zero committed ones deleted, missing markers repairedAny committed pack deleted

M1-WP5. Git cache shards and the git container image

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
BF-18Archive a range of inline packs, kill all shards, rehydrate. A second shard behind its IngestCursorgit fsck clean, every ref clones, second shard serves the newer pushesAny missing object

M1-WP6. The edge for git over HTTPS and the clone cache

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
BF-16Edge on N+1 with a new message format that is not enabled, plus one forced unknown variantNothing sent early. Unknown variant gives 503 with Retry-After and no state changeAny other rendering or any state change
BF-17Atomic push of 10,000 refs with a 1 MiB pack. A push whose updates exceed COMMAND_MAXFirst: pack stored and push commits. Second: refused whole, no entryPartial push, or an entry over 1.5 MB

M1-WP7. Minimal identity and owner placement records

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
BF-101,000 chain creations with two edges at once, plus 100 during an owner record changeOne group per chain. Members without a committed config serve no read and grant no voteAny chain with two groups, or any read or vote served without a config
BF-11Edges with cold caches after a member retry, ReplaceMember, and an owner move100% of requests find the groupAny request that finds no group
BF-15Revoke a stored token, and separately suspend an account, each with 1,000 of its commands queued100% of commands decided afterwards get 401Any accepted

M1-WP8. Milestone 1 gate

Acceptance: the gate's pass and fail lines

RowGate (Pass)
BM-1The git conformance suite and git clone / fetch / push --atomic from unmodified git 2.4x pass with one site's member class failing mid-run. Linearizability rerun at 24 hours. The git conformance suite is SHARED's git-http and git-protocol-v2 oracle rows on unmodified git 2.39 and 2.51, plus git's own smart-HTTP transport tests from t/ that can be pointed at a remote host, run by tests/m1/wp6/conformance/run.sh (decisions/DECISIONS.md, "Decisions from the M1 builds, 2026-11-01")

build/BUILD-PLAN.md

Milestone 2: folds, issues, pull requests, and merge

Milestone 2 status (2026-11-07)

Milestone 2 is closed: every gate row below is Done, the last being BM-2 as reworded, on bm2-home (2026-11-06 to 2026-11-07).

Every hosted row ran on the test install graft-m1-site-a/b/c and graft-m1-edge, or on graft-m2b where the table says so (graft-m1 gained a fourth site script, graft-m1-site-d in wnam, on 2026-11-08 for BF-30, and graft-m2b gained graft-m2b-ops, Worker c53e4459 with the Secrets Store secret graft-m2b-mint-wrap-key, on 2026-11-09, when graft-m2b-edge became c4eb6915 with formats 32.8, 32.12, 32.14, and 18.3 added (AM1-WP6), then b9f4291e with the same formats after fanout's site day on 2026-11-10 (AM1-WP1); Milestone 2 ran before both), with real gh 2.102.1, git, and git-lfs 3.8.0 clients. LF-13 ran on a third test install, graft-m3c (tests/am1/results/install-m3c.md), built 2026-11-11 in account be2ae00c888cc5865939bb290fa07074: install 01M742HEA2SBCYP1VW26624KMW, sites graft-m3c-site-a (enam), -b (weur), and -c (apac), edge graft-m3c-edge with the agent formats and the tools edge graft-m3c-tools-edge, and no Secrets Store or graft-m3c-ops, because genesis hit maximum_stores_exceeded (the account's one store is graft-m2b's), so installation tokens do not mint there. Results are in tests/m2/results/, one file per work package, and each design fix they found went into the design first (decisions/DECISIONS.md, the groups from "BF-20 passes after the restore fix, 2026-11-04" to "BM-2 reworded by the operator, bm2-repack, and M3-WP4's pure half, 2026-11-06"). M2-WP6's final build rechecked the earlier rows at reduced counts, not as full reruns ("Per-row recheck on the final build", under M2-WP6).

RowStateWhat closes it
BF-20Pass at 300 commands/s on the 3.99 GB metadata tier of repo:196628: digest equal to a refold from both R2 copies and on all three members, apply p99 0 ms, no limit outcome, 423,000 of 423,000 commands (m2-wp1.md, "2026-11-04: restore fix and BF-20 rerun"). Rechecked on M2-WP6's final build only as a smoke: digests agree at two indexes after two site days, and 3 pushes; the 300/s campaign was not rerunDone
BF-22Pass in the pure half at the row's 10,000 check runs and statuses on two members, prune included (M2-WP2). Hosted at 500 statuses (bf22e, m2-wp6.md, "BF-22 reduced"): 500 of 500 read through gh pr checks and REST, core rows unchanged but for consumer_cursor, both tiers' digests equal on every member, after two fixes. The prune clause ran only in the pure test, because the install's entry clock cannot be movedDone for M2; a hosted prune needs a movable entry clock
BF-1Pass hosted, 3 of 3 (m2-wp3.md); rechecked 2 of 2 on M2-WP6's final build (bf1s) and on bm2-repack'sDone
BF-21Pass hosted, 100 runs with the leader killed in 10 (m2-wp3.md); rechecked 10 of 10 with 5 kills on M2-WP6's final build (bf21t), on bm2-latency's, and on bm2-repack'sDone
BF-23Pass hosted, 100 of 100 (m2-wp3.md); rechecked 20 of 20 on M2-WP6's builds, and 10 of 10 on bm2-latency's and on bm2-repack'sDone
BF-24Pass on graft-m2b, 75 of 75 within 10 s with the in-band first attempt, clean p99 6.44 s, conflict p99 5.81 s, with leader-change, reset, and purge runs (m2-wp5.md, "2026-11-05"); rechecked 20 of 20 on M2-WP6's builds (clean 0.83 to 2.10 s, conflicts 1.03 to 1.71 s) and once after the edge-wiring site day (6.3 s, m2-edge.md)Done
BF-25Pass with real git-lfs and gh release: every verify and asset 201 commits only after the member confirms two buckets in two regions, with each blob's marker in all three buckets (m2-edge.md); rechecked as a smoke on M2-WP6's build. The unreadable-bucket clause ran only in the pure test, 60 of 60 asset and LFS reads with each bucket in turn unreadable, plus one corrupt copy (crates/api/tests/blobs.rs), fewer than the row's 100, because the install has no hook to make a bucket unreadable (m2-wp6.md, "BF-25 and BF-26 smoke")Done for M2; a hosted unreadable-bucket run at 100 reads needs a bucket fault hook
BF-26Pass in the pure half with real git archive tar.gz and zip, and hosted: all 6 archive-cache/ objects deleted, then gh release download --archive=tar.gz gave the same bytes and rebuilt the object (m2-wp6.md, "BF-25 and BF-26 smoke")Done
BM-2Pass as reworded on bm2-home: home clause p99 1.36 s at n = 300 with an EWR client and edge, 0 over 2 s; off-home clause p99 1.83 s at n = 288 from LAX, 0 over 3 s; every sample on a new term. Journeys 2 and 3 passed 10 of 10 on bm2-repack's build and stale parent_tip 100 of 100 on bm2f, neither rerun on bm2-home's (M2-WP6, "BM-2 status")Done

M2-WP1. Folds in member SQLite

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
BF-20Snapshot of a 4 GB guess metadata tier under 300 commands/sApply p99 within 10% guess. Snapshot digest equals a refold at to. No request-limit errorsApply p99 more than 10% slower, a digest mismatch, or any request-limit error

M2-WP2. Issues, pull requests, reviews, and checks

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
BF-2210,000 non-required check runs and statuses on one repository, then entry time advanced past check_retention_daysgh pr checks and GET /commits/{sha}/statuses read them; core tier bytes unchanged; rows pruned on every member with equal fold_digestA non-required row in the core tier, or members disagreeing

M2-WP3. Merge and the merge queue

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
BF-1One entry holding a 5-PR merge-queue group plus a PrSync batchIt commits on all three sitesAny follower refuses it
BF-21A group of 5 PRs under HeadGreen, the 3rd failing its required check on its queue ref, 100 runs, with the base leader killed mid-landing in 10The base moves only by QueueMerge entries that land a whole group; the failing PR is ejected and the other 4 land after a rebuild; every base move is a fast-forwardAny partial group, any non-fast-forward base move, or a queued PR lost
BF-23A required check passes, then is re-requested before the mergeThe merge is refused until the new run concludes, 100 of 100A merge on the stale success

M2-WP4. The wire layer, the blob store, and the other T0 repository surfaces

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
BF-25git lfs push and gh release upload, then one site bucket made unreadableverify and the 201 come only after two buckets hold the blob; fetch and download succeed 100 of 100Any acknowledged blob unreadable
BF-26All ArchiveCache objects deleted, then gh release download --archive=tar.gzThe bytes match the earlier downloadAny difference or error

M2-WP5. Cursors, Queues, and the PrSync consumer

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
BF-24A head push, then git ls-remote origin refs/pull/N/merge; a conflicting pushThe TestMerged commit within 10 s with a matching mergeable; on a conflict the ref is deleted and mergeable: falseA stale ref

M2-WP6. Milestone 2 gate

Acceptance: the gate's pass and fail lines

RowGate (Pass)
BM-2gh pr and gh issue journeys 2 and 3 pass, the merge p99 on a freshly elected leader on the group's home site, with the client in that region, is at most 2 s, and on a freshly elected leader off the home site at most 3 s, and a queue entry with a stale parent_tip is refused 100 times out of 100
ClauseResult
Journey 2 (gh pr review, required check, merge, view, list)Pass, 10 of 10 (j2b): the merge is refused until the review and the check commit, then lands with parents (base, head), and gh pr view and gh pr list --state merged both show it
Journey 3 (gh label, gh issue create -l, gh issue list, REST issues)Pass, 10 of 10 (j3b)
Merge p99 on a freshly elected leader at most 2 s (old wording)After bm2-repack: p99 2.42 s, p50 1.52 s, n = 288 on 300 new terms (final); per new leader site p99 enam 2.01 s (n = 66), weur 1.96 s (n = 120), apac 2.71 s (n = 102); 8 of 288 over 2 s. Before it, after bm2-latency: p99 4.10 s, p50 2.11 s, p90 2.93 s, n = 288 on 288 new terms (final); per new leader site p99 enam 2.40 s, weur 4.15 s, apac 2.89 s; 178 of 288 over 2 s, where a pass at this n allows 2. The first run, before the cuts: p99 3.20 s, n = 91
Stale parent_tip refused 100 of 100Pass, 100 of 100 on the final build (bm2f); not rerun on bm2-repack, which does not touch the queue's decide
RowRecheckResult
BF-12 queue runs (bf1s)Pass 2 of 2: each landing's 5 Merged, Landed, and one 15-update Refs in one entry on all 3 members with one digest
BF-20Smoke: digests at 1720 and 1738 on repo:196628 after two site days, and 3 pushesSmoke passes: all three members agree; the first pushes after two idle days took 7.1 and 14.4 s (cold tier load); the 300/s campaign was not rerun
BF-2110 runs, 5 leader kills (bf21t, sd10)Pass 10 of 10, each kill ending whole in 93 to 120 s; the first try (bf21r, 9 of 10) stalled 9.5 min on a stopped container, fixed by 576badf and 4c3727e
BF-22500 statuses, 8 posters (bf22e, sd8)Pass after two fixes: 500 of 500 through gh pr checks and 5 REST pages, core rows unchanged but for consumer_cursor, both tiers' digests equal on all members; the prune clause stays with the pure test
BF-2310 + 10 (bf23r sd7, bf23s sd10)Pass 20 of 20 (merge 8, squash 6, rebase 6)
BF-2410 + 10, with conflicts and a transfer (sd7, sd10)Pass 20 of 20 within 30 s: clean 0.83 to 2.10 s, conflicts 1.03 to 1.71 s
BF-25git lfs and gh release smokeLFS and asset bytes match, both blobs in all 3 buckets; the unreadable-bucket clause has no hosted hook
BF-26All 6 archive-cache/ objects deleted, then gh release download --archive=tar.gzPass: same bytes, object rebuilt
BM-2 stale parent_tip100 (bm2f, sd10)Pass 100 of 100

build/BUILD-PLAN.md

Milestone 3: identity and authorization

Milestone 3 status (2026-11-09)

Milestone 3 is closed: every gate row below is Done, the last being BF-34's TestAccGithubRepositoryCollaborator on m3-repos (2026-11-09, tests/m3/results/m3-repos.md). BF-30 ran on graft-m1 (m3-finish, merged at b767933); the m3-rows rows ran on graft-m2b after four site days, the first onto main's build e2c67b5 and the last onto db1c6be (m3-rows.md, "What was deployed on graft-m2b"); m3-repos ran on graft-m2b after a site day from 3160687. The two test installs: graft-m1 has four site scripts since 2026-11-08, graft-m1-site-a to graft-m1-site-c and graft-m1-site-d in wnam (added for BF-30's move across placements), with graft-m1-edge, the tools edge graft-m1-wp7-edge, and graft-m1-ops; graft-m2b has three, graft-m2b-site-a to -c, with graft-m2b-edge and graft-m2b-ops. Three rows rest partly on runs outside the hosted install, and the table says which: BF-27's 1,000-queued count ran only in memory, BF-32's no-secret dump ran only in memory, and BM-3's learner half counts B0-18's graft-m2b run as a judgement.

RowStateWhat closes it
BF-2Pass on graft-m2b, 9 of 9 runs: 600 of 600 pushes held behind a revocation refused 401, 0 acknowledged at or after the revocation's index; the 3 controls acked 300 from reads below it (m3-rows.md, "BF-2")Done
BF-31Pass on graft-m2b, 10 of 10: a scoped github_pat_ refused outside its scope; revoking either of a ghu_/ghr_ pair refuses the other; DELETE /applications/{client_id}/token revokes; 0 uses after revocation (m3-rows.md, "BF-31")Done
BF-3Pass hosted on graft-m1: 200 of 200 pushes refused after a suspension, one key from two requests 100 of 100 with one owner (m3-close.md, "BF-3"); the SSH push is BF-52 (M6-WP2)Done
BF-30Pass on graft-m1, 20 of 20 transfers and 0 of 2,313 route samples lost: across enterprises, across placements (Tri [0,1,2] to Na3 [0,3,1] home 3, with the new graft-m1-site-d in wnam), both at once, home only, and back; deps named the new owner after Settings in 20 of 20; a finisher killed after Reserved finished by Audit alone in 76.2 s with 0 of 586 samples lost; an org's enterprise change gave 2 of 2 repositories their Settings; 9 of 9 planted bugs (m3-finish.md, "BF-30")Done
BF-33Pass on graft-m2b: with the new token's Tokens leader killed after UseCode committed and before the reply, 99 of 100 completed with exactly one token and 1 restarted cleanly; replay and codes older than 900 s refused 100 of 100 each, kept from an earlier run on the same install (m3-rows.md, "BF-33")Done
BF-34Pass on graft-m2b: the driver 10 of 10 with 170 of 170 replies valid against GHES 3.22 (m3-rows.md, "BF-34"), and Terraform's TestAccGithubRepositoryCollaborator as written 6 of 6 runs, 12 of 12 subtests (terraform-provider-github 1b9c5c1, Terraform 1.17.1), every one of the 12 repositories it created answering 404 afterwards; its third subtest skips without GH_TEST_COLLABORATOR, as written (m3-repos.md, "Terraform")Done
BF-35Pass on graft-m2b, 100 of 100 through gh ssh-key list and GET /user/keys with no Audit consumer (m3-rows.md, "BF-35")Done
BF-36Pass on graft-m2b, 10 of 10: a real OIDC code flow with PKCE against a test IdP, a Password sign-in WrongMethod after the switch, a non-admin's write 403 (m3-rows.md, "BF-36")Done
BF-27Pass in memory at the row's count, 1,000 of 1,000 refused (m3-wp4.md); hosted only through BM-3's 100 of 100 ghs_ refused after revoke, not with 1,000 queuedDone, on that evidence
BF-32Pass hosted, 12 of 12 twice with @octokit/app 16.1.4 (m3-hosted.md); the no-secret dump of every Identity chain ran in memoryDone
BF-28, BF-29Pass hosted on graft-m2b (M3-WP5 status)Done
BM-3Pass, journey 4 100 rounds on edge 8a824825, with the learner half counting B0-18's 6 of 6 as a judgement (M3-WP6, "BM-3 status")Done

M3-WP2. Token kinds and short-lived codes

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
BF-2Revoke a token while 100 of its pushes wait in admissionNone is acknowledged at an index after the revocation'sAny one is
BF-31A github_pat_ scoped to one repository with contents: read; a ghu_ and ghr_ refresh; DELETE /applications/{client_id}/tokenThe PAT is refused outside its scope; revoking either paired token refuses the other; the DELETE revokesAny use after revocation

M3-WP3. Authz, Identity, and sign-in

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
BF-3Suspend a user, then push with their PAT; separately, add one SSH key from two edges at onceEvery push refused; the key gets one ownerAny push accepted, or two owners
BF-30Transfer across enterprises and placements, plus an org changing enterpriseBoth routes reach the group throughout. deps names the new owner after SettingsAny lost route, or a stale owner after Settings
BF-33gh auth login device flow with the code shard's leader killed mid-flow, 100 runs; replay of a used code; a code older than 900 sEach login completes or restarts cleanly; replay and the expired code fail 100 of 100A token from a reused or expired code
BF-34TestAccGithubRepositoryCollaborator; an org invitation accepted and one declined; a Collaborator write for an org memberAccepting grants in the same entry; declining grants nothing; the member write is refusedAny overlap or lost grant
BF-35A new SSH key and PAT, listed at once with no Audit consumer deployedgh ssh-key list and GET /user/keys show them 100 of 100Any listing that misses them
BF-36A site admin switches InstanceAuth from Builtin to OIDCThe next sign-in uses OIDC; a non-admin write of it is refusedAny other result

M3-WP4. Stateless installation tokens in graft-ops

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
BF-27Revoke a jti with 1,000 commands under its installation token queued100% of commands decided afterwards get 401Any accepted
BF-32@octokit/app installation-token flow with a newly registered App; OAuth web flow with a registered OAuth app and a wrong secretThe JWT verifies against AppRecord.keys; the wrong secret gets 401; a dump of every Identity chain shows no client secretAny other result

M3-WP5. Learners and snapshot install

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
BF-28A learner on its voter's site, and two learners on one siteDistinct objectsA name collision
BF-29Follower offline past the archive boundary, then rejoinsCatches up by InstallSnapshot with matching fold digestsStuck or divergent

M3-WP6. Milestone 3 gate

Acceptance: the gate's pass and fail lines

RowGate (Pass)
BM-3Journey 4 passes over HTTPS, with revoked tokens, a revoked stateless installation token included, refused on the next write 100 times out of 100. Learners rerun the Milestone 0 learner gate

build/BUILD-PLAN.md

AM1: base agent changes, leases, and tasks (built with M2 and M3)

AM1-WP2. Agent actors and refusals in decide

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
LF-2Unflag an AgentApp (DELETE /agents/apps/{id}), then push under its old confine prefixThe next decide after the acknowledgment no longer classes it ActorKind::AgentA decision after the acknowledgment uses the old row

AM1-WP3. External tasks and the task read endpoints

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
LF-1Uninstall an App with 100 open tasks across 10 repositoriesEvery task reads cancelled at the next read with no sweep command, and its next command records itAny task reads live

AM1-WP4. Leases, PathSet, and the activity feed

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
LF-3Feed cursor from another group, wait=40, 33 paths410 with oldest_cursor, wait clamped to 25, 422 for the 33 pathsAny other status, or wait above 25
LF-5graftyard-plan blocks: a valid one, a repeated key, a bad glob, a second block, and a block over 4 KiBOne lease from the valid block. None from the others, and the body unchangedA lease from an invalid block, no lease from the valid one, or a changed body

AM1-WP5. The trust view

Acceptance: the gate's pass and fail lines

RowMeasurementPassFail
LF-4Trust view: one payload per HIDDEN_CHARS range in each graftyard_trust resource100% absent from visible and countedAny present

AM1-WP6. AM1 gate

Acceptance: the gate's pass and fail lines

RowGate (Pass)
LM-1Base journeys pass unchanged, the base's agent rows rerun on real code, the base's 30-day hot-group storage row passes on a group seeded with the 4 GB tier, the leases and feed, lease limits, gate and read permissions, and core-tier rows pass, and the byte diff passes. Fleets of A1's size on one repository are enabled only after metadata child groups (base Milestone 7)
DayCommitSites a / b / c / dEdgeOpsNotes
240708cbbe79725f / aed638b3 / 232474cc / 150bab9c19e72fe61e9eb629 (the task claim)New shard image sha256:5db19b4e…; --roll-shards stopped on a catch-up timeout and tests/am1/hosted/roll.mjs finished the roll
3a4dd646c2f4c5e4 / bde58f9a / 6fe455bf / 69beff5198de7bf4no roll
4f925d40ed0bff49 / 8686876c / 11ff63a5 / f47c2827b330c05cno roll
5c836d14b3d42554 / 7bed34dc / e57c7482 / 0b7bcdf4d3a8981fno roll; L0-16's two fixes
604e4f226553b6dc / 751b0701 / 98221437 / 002551dcce2e1ac3no roll; main through 5d18dda merged, with the read gate
7c7384ecb5bfd297 / a0ee9724 / da589789 / 9199c4e4382356edunchangedno roll; am1-fix's two fixes (tests/am1/results/am1-fix.md)
RowHosted result
Base journeysPass: j2 and j3 20 of 20 each; journey 4 PATs and ghs_ 10 of 10 refused after revoke
L0-11 (byte diff)Pass: day 2 0 of 2,048 replies differ; the day 5 and day 6 differences are only j3's new issues and the read gate's anonymous 404s
L0-12, refusalsPass for all six kinds (am1-runs): the confined vendor App 100 of 100 pushes and 100 of 100 REST ref writes refused naming the lease; the Graftyard-run App (graftyard-workspace) 100 of 100 of each refused, but by NoLiveTask ("push needs a live task") before the lease check, because the edge cannot yet create a Graftyard-run task (AM3-WP3); 0 of 100 for a flagged App without confine, an unflagged App, a human PAT, and Graftyard MCP
L0-12, stall**Pass after am1-fix** (day 7, lf1-5): a 30-minute stall with every member call failing, then a leader change; term 3's first entry carried extend { by: 1,761,355 ms }, all 20 leases moved by 1,761 or 1,762 s, exactly one entry carried extend, and clock spread was at most 65 ms. Before the fix (day 5) a new leader wrote no extend and leases moved 0 ms. The fix (96dc541): a follower marks the stall from the leader's late re-send of an uncommitted entry; the long sweep passed (335.75 s) and Porcupine was Ok on both histories. Uncovered: an old leader that never reaches the winner, which loses the stall as a lost marker does
L0-12, A1 replayFail (am1-runs, 500 tasks and 500 Exclusive leases, a leader kill every 60 s): 0 events reached the 500 pollers in 600 s, of 149,265 polls (74,432 with no status, 52,602 gone, 12,099 notServing, 6,277 lagging, 3,855 held); 57 of 2,491 agent pushes ok; the kills did not land (1 of 10 kill pushes ok, term unchanged); 5 of 1,451 lease renewals answered 200
L0-13Pass: 6 bad acquires 100 of 100 (am1-hosted); then (am1-runs) 600 + 600 plan blocks gave 0 plan leases from non-writers and 500 from writers (63 holders, at most 8 each); a holder removed from the org refused nothing at once (2 of 2 pushes refused before, 2 of 2 accepted after); the 5 h renew read shared at every read after 240 min
L0-15, feed halfPass with the row's 404 (contradiction 1, now in the row); the AgentGate, Code Mode, /log, and prompt halves need MCP and the controller
L0-16Pass: 0 metadata reads on 3 members for 8 checks; a merge waits for the tier by design (the row no longer lists merges)
LF-1, LF-2, LF-3, LF-5Pass
LF-12**Pass after am1-fix** (day 7, lf1-6): leader commit p99 474 ms after place_learners (6,481 ms before the fix) and 710 ms before it, with 5,000 of 5,000 polls held on 11 learners and 0 on voters. The cause was a member call: each poll on a stale learner sent its own owner-Authz read to that chain's leader, which delayed the repository leader's dep read; the fix (760f9c0) keeps one owner read in flight per member. Through the edge from the orb, client p99 is about 2.7 s even with no polls: that is the client path, not the commit, which the row measures at the leader
LF-13**Pass on graft-m3c** (lf13-m3c, site day 4, 9d59283; tests/am1/results/lf13-m3c.md), both modes, 3 of 3 runs each, 999 polls held. Gap: a real gap in 3 of 3; 999 of 999 held polls returned with their cursor and no events within 1 s of the lift (99, 306, 799 ms) and of F's stop (0 ms); 1 of 50 probes lagging per run, because a gap ends at F's next accepted round; 0 events missing. Install: InstallSnapshot in 3 of 3; 999 of 999 within 1 s of the lift (788, 83, 437 ms) and of the stop (0 ms); probes lagging 48, 50, and 49 of 50; 0 missing, and the pre-fault cursor answered gone. Between the lift and F's stop, F admitted 0 to 2 of 50 probes, which its stop then answered with nothing skipped. Fixes: 3597c6f (a stop counter, so a stop between two waits still answers a held poll; and the test-faults /sever route), 429eede (a lease acquired and released in one catch-up batch reports both), and 7972504, 606b403, 9d59283 (the ring restarts at the install's base at once, with the installed tier's live leases); 7 of 7 planted bugs caught. Cost $2.55. Earlier, on graft-m2b (am1-runs to lm1-rerun2): no gap formed, and held polls ran out their own 25 s
LF-4Pass, all eight resources (lf4-rest, graft-m3c; tests/am1/results/lf4-rest.md): issues, pull requests, issue comments, reviews, review comments, commits, check runs, and releases 16 of 16 each, every hidden range absent from visible, invisible_chars 2, raw text unchanged, the untrusted writer's objects trusted: false, no graftyard_trust without the media type. The six new reads are REST GETs on the edge (F61), validated against the pinned description locally (96 REST replies) and hosted (16), 0 failed; 16 of 16 planted bugs caught. Rerun on lf4-rest-2 (tests/am1/results/lf4-rest-2.md) with review comments and check runs written through the new REST POSTs instead of a test-only commit: 8 of 8 resources 16 of 16 again, every write 201. Rerun on lf4-rest-3 (tests/am1/results/lf4-rest-3.md) with the check-run create's id taken from the IdBlock cursor in decide (repository 32.18): 16 concurrent creates 16 × 201 with 16 distinct ids from one block, 8 of 8 resources 16 of 16. Earlier (am1-runs): issues and releases only
B0-29Fail (am1-runs, 500 agents): commit p50/p99 1,205/9,316 ms (n 321); 5 of 2,500 agent pushes ok (1,772 × 503, 722 client timeouts); 1 of 99 merges; leader moved 0 → 2 with 2,305 failed member calls; 0 feed polls reached the leader
B0-31Fail: 4 of 2,501 agent pushes ok (1,974 × 503, 522 timeouts); the apac leader did not hold (back to site 0 in setup, ending on site 1, term 30), so no commit sample on the final leader, and the placement-failing half was not reached; feed p99 42.3 s
B0-32Fail: 0 of 500 burst pushes within 5 s, 346 × HTTP 503 and 154 timeouts, neither in the ng shape; 0 leader changes; commit p50/p99 714/1,199 ms (n 84); 72 of 755 agent pushes ok
B0-33Fail (5,000 agents, 300 s): about 0.6 agent commands/s admitted against 140; 9 of 12,454 agent pushes ok (12,194 × 503); the fast user 6 of 508 × 200 and 502 × 503; humans 4 of 487; commit p99 1,086 ms (n 53); 0 leader changes; 3,195 failed member calls
B0-34Fail: clone time to first byte p50/p99 23.1/86.7 s (80 × 200, 255 × 503); human pushes 0 of 33 ok; agent pushes 0 of 2,500 ok; commit p50/p99 5,516/5,659 ms; 0 leader changes; k stayed 1, with no fanout
B0-35Fail, not a valid run of its line: setup opened 10 of 500 PRs; mergeable polls 187 MERGEABLE and 9,386 × 503; 63 of 2,500 agent pushes ok; commit p99 13,213 ms; dispatch not measurable (no Actions on graft-m2b)
B0-30Fail on its fail line (tests/am1/results/b0-30.md; graft-m1, branch b0-30, merged into main; pushes off from simulated day 1.34). Member SQLite crossed CHILD_GROUP_AT (8,589,934,592 B) at 23:09:41Z on 2026-11-11, simulated day 27.52: site 0 read 8,591,708,160 B, and sites 1 and 2 crossed within 30 s. No child group took the tier, because MetaSplit and RepoMeta are types only (G6). The rest held: SQLITE_FULL 0 on every member throughout, and the final verify read 81 of 81 sampled archived ranges from both buckets and resolved 10 of 10 RestoredTo lookups. Rates: while archiving kept up (simulated days 7.08 to 22.27), SQLite grew 0.0585 GB a simulated day, which projects to about 8.21 GB at day 30, under the line inference, linear; during the stall it grew 0.184 GB a day. Cause of the stall: from about 17:13Z fanout (k = 6, with no entry found that raised it; A31) forwarded the archive promotion's /validate to cold pool shards, each of which read the multi-GB log tail before validating, so archive runs exceeded RUN_FOR (120 s); site a's promotion validate median went from 0.17 s at 16Z to 7,551 s at 21Z, and site b archived its last range at 20:17:56Z (G7). Blocker 12's cursor skip came back on site c, and the group was left with no leader three more times (F67). Cost $64.90. The earlier substitution is withdrawn ("What LM-1 still needs")
B0-36Pass (am1-runs) on every surface the deployed edge routes, 100 per action, for the confined vendor App (m3-ci) and the Graftyard-run App (graftyard-agent) with its shipped permissions and on an installation widened to every permission; ready-for-review and REST issues/{n}/labels included. Graftyard MCP, the control, met no agent refusal. The Graftyard-run kind's git-surface refusals came from NoLiveTask; its lease refusal and task pushes moved to AM3-WP3 (operator, 2026-11-10). GraphQL addLabelsToLabelable, rerun on graft-m3c (lf4-rest): 100 of 100 refused by the allowlist rule for m3-ci and for graftyard-agent shipped and widened, 0 of 100 for Graftyard MCP, a label inside the allowlist accepted 5 of 5 for every kind; the same again on lf4-rest-2, with m3-ci flagged through PUT /api/graftyard/v1/agents/apps/{id} by the m3 admin, now a site admin of graft-m3c
B0-37Push half pass at 100 per kind (about 1,000 pushes, not 10,000): 0 parse failures, every forged trailer Claimed, 100 of 100 refs/graftyard/ writes refused. Fetch half partial: session refs stay hidden from v0, v2, and clone --mirror, but no session ref exists to list, because only graftyard-workspace under a live Graftyard-run task writes one (AM3-WP3); containers/git/server/sessions_test.go covers the listing
RowRerun on `graft-m2b`
B0-29Fail: 105 of 2,500 agent pushes ok (1,897 × 503 from the gate, 440 timeouts); commit p99 30,092 ms (n 9); merge p99 120 s; leader 0 → 2; 16,777 leader /reads and 20,199 /suspect in 600 s
B0-31Fail: 133 of 2,500 ok (1,777 × 503); commit p99 2,508 ms (n 12); the apac leader did not hold, and the placement-failing half was not reached
B0-32Fail: 0 of 500 burst pushes within 5 s (323 × 503, 132 × 500 "no member leads", 43 timeouts), none in the ng shape; leader 0 → 2
B0-33Fail (5,000 agents): about 2.5 agent commands/s admitted against 140; 62 to 331 of 12,546 agent pushes ok (a driver parser bug, 994266c, bounds the count); commit p99 3,415 ms; no leader change
B0-34Fail: k stayed 1 (a follower's /fanout error); clone time to first byte p99 172.4 s; 0 of 37 human pushes ok; commit p99 361 ms
B0-35Fail: the setup now opens 500 of 500 PRs (am1-runs' 10 of 500 was very likely the same parser bug); mergeable polls 10 MERGEABLE and 10,162 × 503; 0 of 2,500 agent pushes ok
L0-12, A1 replayFail: 0 events reached the 500 pollers in 600 s of 211,151 polls; 0 of 10 kill pushes landed
LF-13Fail: install mode reached its fault in 4 of 6 runs, refused new polls (45 to 47 of 50 lagging), and lost no event (the ring answers gone, dfa5410 deployed), but F stopped its held polls in only 2 of 4; gap mode reached a real gap in 1 of 6, with 0 lagging and one lease event fewer on F's ring than on the third voter, an open bug candidate. The row's "within 1 s" does not say of what: the lift or F's stop
Row`lm1-rerun` on `graft-m2b`
B0-29Fail: 274 of 2,500 agent pushes ok (1,425 × 503 from the gate, 224 × 500, 567 timeouts); commit p50/p99 1,063/1,505 ms (n 144); merges 2 of 3, p99 76 s; API reads 4,143 × 200, 9,576 × 503; leader held; exceededMemory 11,117 / 4,478 / 2,814 on sites a/b/c. With API reads off (diagnostic, 300 s): 600 of 1,249 pushes ok and clean for 2 minutes, then members ran out of memory
B0-31Fail: 48 of 2,500 ok (1,741 × 503); commit p99 2,359 ms (n 93); the placement-failing half was not reached
B0-32Fail: 0 of 500 burst pushes within 5 s (456 × 503, 44 timeouts), none in the ng shape; leader 0 → 2; 40,180 leader /reads and 51,362 /suspect
B0-33Fail (5,000 agents): 101 of 12,484 agent pushes ok (10,633 × 503); about 2.1 side commands/s admitted against 140; 0 of 29 human pushes; leader 0 → 1; members 331 / 262 / 250 MiB
B0-34Invalid: stopped at about 07:15Z on a false cost reading (account-wide rows that were B0-30's); 103 of 2,577 receive-packs answered 200; k stayed 1, though /fanout reports now answer ok (16 of 17 in B0-29)
B0-35Not run: it would have started at $11.42 against the $12 row stop
L0-12, A1 replayFail: 0 events timed at the pollers of 146,705 polls; 1 of 10 kill pushes landed; 137 of 2,490 agent pushes ok; members 283 / 280 / 265 MiB
LF-13Fail, with HELD=999. Install reached its fault in 2 of 3 runs: every held poll came back with its cursor and no events, 48 and 47 of 50 new polls answered lagging, and no event was lost (F answered gone), but only 630 and 479 of 999 returned within 1 s of the lift (the slowest at 1,003 and 1,004 ms), and F itself stopped only 193 and 0 of them; the rest returned before the lift at their own 25 s wait. Gap left no gap in 3 of 3 (O level with F each time) (A26)
Row`lm1-rerun2` on `graft-m2b`
Smoke, 250 agents, API reads off (ccd10b1 only)491 of 624 pushes ok; leader 0 → 2; members 280 / 113 / 105 MiB; 607 exceededMemory on site c
Smoke, 500 agents, API reads offccd10b1 only: 300 of 1,232 ok, 6,924 exceededMemory on site c. With eb8eb0b: 812 of 1,254 ok, still a leader change and 1,621 exceededMemory on site b
B0-29, twiceFail. With eb8eb0b: 118 of 2,501 agent pushes ok (1,690 × 503 from the gate), commit p99 37,954 ms, members 273 / 247 / 201 MiB. With a271fb3: 107 of 2,501 ok (1,770 × 503), commit p50/p99 1,757/3,967 ms, members 267 / 265 / 263 MiB though a whole-tier read was about 150 KB; 87,000 leader /reads and 108,000 /suspect
B0-35Fail: the setup opened 500 of 500 PRs, then from about 17:08:30Z every edge request answered 503. The group was left with no leader: at 17:21Z, after the load stopped, all three voters were followers at term 4 with no leader, after 150,399 /suspect reports and 78,842 /reads to one of them, and one test-only /campaign to each elected site 0 within 5 s. Member peak 279 MiB. Diagnosed by lm1-memory: the term-2 leader f0 was stuck in one alarm invocation for 641 s wall (622 s CPU, 41.9 M rows read), building dumps for about 730 reads whose callers had gone (fixed in f125a37), while f1 kept hitting exceededMemory; elections were not at fault and crates/raft is unchanged. Test-merge vCPU and list-field computes are still not instrumented (AM1-WP1, "Follow-up: the test-merge budget and B0-35's instruments")
B0-32Fail: 12 of 500 burst pushes ok, none in the ng shape, p50 27 s; leader 0 → 2; 63,043 /suspect on the leader; member peak 318 MiB
B0-31Fail: 126 of 2,501 ok; commit p99 9,754 ms; the term churned from 3 to 28 in the run, with one voter at a time not answering /status; 128,734 /suspect on the leader; member peak 265 MiB
B0-33, B0-34, L0-12 A1 replayNot run: the round's $13.40 no-new-row threshold was reached after B0-31
LF-13Fail on graft-m2b (gap left no gap in 3 of 3; install's held polls returned up to 1,228 and 1,741 ms after the lift), its last run there. LF-13 has moved to its own install, graft-m3c (thread T-01a1c666-2b83-7522-8afd-45a508a3cd74)
Row`lm1-memory` on `graft-m2b`, `d9ee594`
Smoke, 500 agents, 300 sEvery member under 80 MB (peaks 69 / 35 / 78 MB); 1,237 of 1,242 agent pushes ok (5 refused); commit p50/p99 793/2,956 ms at the leader; leader 0 → 2, on Confirm then CommitErrors
B0-29Fail: commit p99 3,867 ms against 600 ms (p50 1,000, n 2,077); leader 0 → 1; 1,089 of 2,500 agent pushes ok (666 stale, 406 timeouts, 205 missed); the apac member at 262 MB
B0-32Fail: a leader change, 71 suspect, commit p99 4,834 ms; 48 of the burst ok, 452 refused
B0-31Stopped early, before its row line
B0-33, B0-34, B0-35, L0-12 A1 replayNot run: the 500-agent bar did not hold at 600 s, and the rows would have passed the cost cap with nothing new to test
RowGate (pass / fail)Round 3 on `graft-m2b`, `54c45213`
Smoke, 500 agents, 300 sNonePeaks 75 / 55 / 112 MB, 0 exceededMemory; the leader stepped down on CommitErrors 15 s into load; commit p99 5,675 ms
B0-29Commit p99 ≤ 600 ms, merge p99 ≤ 2 s, zero Stale on distinct refs, leader < 600 req/s with zero feed polls / commit p99 > 1 s, a queue merge > 10 s, a feed poll on the leaderFail: commit p99 3,419 ms (p50 662, n 8,167); leader 0 → 1 → 2, the first step-down 7 minutes into load; 7,774 /suspect; 2,412 of 2,500 agent pushes ok; merge p99 9,303 ms; 306 req/s and 0 feed polls on the leader
B0-32Every push acked within 5 s or ng … admission full, merges in the next 2 entries, zero leader changes and suspectFail: zero leader changes but 3 suspect; burst 168 ok and 332 refused, not in the admission full shape, 1 within 5 s; merge p99 41,005 ms
B0-31Commit p99 ≤ 1 s with buckets up / > 3 sFail, between the lines: commit p99 2,541 ms; leader 0 → 1 in two moves (CommitErrors, then Confirm p99 1,377 ms)
B0-34k in band ≤ 8, human push p99 ≤ 3 s beyond transfer, clone TTFB p99 ≤ 1 s / TTFB p99 > 5 s, k > 8Fail: clone TTFB p99 58,554 ms (p50 1,643); k 1 → 2 in band; human push p99 240 s
B0-33, 5,000 agentsHuman and merge-queue p99 ≤ 1 s, ≥ 140 agent commands/s, zero leader changes and suspect / a leader change or suspect, < 100 commands/sFail: term 2 → 19 in 5 minutes, 27,703 /suspect; commit p99 15,980 ms; 127 of 12,523 agent pushes ok, 6,925 answered 503 by the read gate
L0-12, A1 replay with a leader kill every 60 sFeed p99 ≤ 2 s outside kills, no lease overlap over 500 ms, refusal counts rightFail: feed p99 64,031 ms (p50 2,647); f0 led through 10 kills (term 2 → 5); f0 peaked at 275 MB with 711 aborted /reads
B0-35, 500 open PRsTest-merge vCPU within TEST_MERGE_BUDGET, dispatch p99 ≤ 10 s, zero list-field computesNot judged: the budget was not instrumented and graft-m2b has no Actions; mergeable 3,876 MERGEABLE and 24,168 × 503; f1 logged 812 exceededMemory
RowRound 4 on `graft-m2b`, `48f8fae1`
Smoke, 500 agents, 300 s, 3 learnersLeader held term 2 with 0 /suspect; commit p99 3,966 ms; 1,218 of 1,241 agent pushes ok; read:api 15,913 × 200 and 36,422 × 503 with Retry-After: 1; leader peak 95 MB, no exceededMemory
B0-29Fail: commit p99 5,461 ms (p50 834, n 6,793) against 600 ms; 0 leader changes and 0 /suspect; 2,250 of 2,500 agent pushes ok; merges 13 of 28 answered, p99 18,971 ms; read:api 51,594 × 200 and 52,858 × 503; the org's Authz leader took 87 /read/s and stepped down
B0-32Fail: 0 leader changes and 0 /suspect, but the burst had 49 ok and 446 refused, not in the admission full shape, 0 within 5 s (p50 26.6 s)
B0-31Fail, between the lines: commit p99 2,311 ms (p50 658) against 1 s, under the 3 s fail line; 29 /suspect
L0-12, A1 replay with a leader kill every 60 sFail: feed p99 12,984 ms (p50 1,141) against 2 s; term 11 after 10 kills; voters peaked at 256 to 278 MB; lease overlap not reported by the driver
B0-34Fail: clone TTFB p99 158,542 ms (p50 846) against 1 s; k 1 → 2 in band; leader held term 2 with 0 /suspect; human push p99 121 s
B0-33, 5,000 agentsFail: term 2 → 8, 7,539 /suspect; commit p99 8,568 ms; 2,020 of 12,533 agent pushes ok
B0-35, 500 open PRsNot judged: the budget's instruments (merge-budget) were not in this build and graft-m2b has no Actions; 18,870 mergeable reads answered 503
RowRound 5 on `graft-m2b`, sites `7583378f`, edge `afb025ac`
B0-29Not judged on commit: the leader held (site 0, term 2, 0 changes, 0 feed polls), but its /stats split was lost when the isolate reset. Merges fail: 4 × 200 and 25 × 503. 2,305 of 2,500 agent pushes ok
B0-32Fail: burst 147 ok and 352 refused, not in the admission full shape (p50 23,063 ms); commit p99 932 ms; two CommitErrors holds, term 2 → 4
B0-31Between the lines: commit p99 1,163 ms against 1 s, under the 3 s fail line (round 4: 2,311 ms); the leader held
L0-12, A1 replay with a leader kill every 60 sFail: feed p99 14,291 ms against 2 s; leader 0 → 1, term 6; the last leadership's Authz dep reads p50 2,246 ms and 956 /suspect. Lease overlap is not measurable: the driver takes only Shared leases, which never conflict
B0-34Fail on human push p99, 121 s (client timeouts). Clone TTFB p99 2,253 ms is between the lines (round 4: 158,542 ms); k 1 → 2 in band; the leader held
B0-35Judged, and fails over budget: test merges used 0.274 vCPU against 0.25 (707 computed, 6,043 shed). List-field computes 0, which passes. Dispatch p99 not instrumented (no Actions). mergeable 1,949 MERGEABLE, 16,018 UNKNOWN, and 39,990 × 503
B0-33, 5,000 agentsNot run: the round's $10 stop came first

build/BUILD-PLAN.md

Gates not yet run

Later milestones' pass and fail lines, fixed before the code is written. Nothing below has a result yet.

Milestone 4: events, webhooks, and Actions

M4-WP1. Webhooks and the remaining consumers

RowMeasurementPassFail
BF-57One hook timing out at 10 s beside 49 healthy hooks of the same owner, 10 events/sHealthy hooks' first-attempt p99 at most 2 s; the slow hook gets its events in chain order; Webhooks cursor lag at most 5 sHealthy p99 over 10 s, an out-of-order delivery to one hook, or cursor lag that grows with the slow hook
BF-58Kill the site holding a hook owner's HookSequencer with deliveries pendingEvery event in the range reaches every hook at least once, with unchanged GUIDsAny event never delivered, or a redelivery under a new GUID
RowMeasurementPassFail
BF-5Star a repository and comment on a watched issue, then delete and rebuild the watcher's Inboxstargazers_count and /notifications show both within 10 s, and the rebuilt Inbox keeps read stateLag over 60 s, or read state lost
BF-67Delete an Inbox and rebuild it. Also 10,000 stars, then a refoldRead and done state kept, nothing from sources lost. /stargazers lists all 10,000Any lost read or done state, a lost event from a sources chain, or a missing star
BF-68EU-resident org's search documents. Kill one SearchShardDocuments only in the eu-jurisdiction shards. incomplete_results until the rebuild, then full resultsA document outside EU, a partial result without incomplete_results, or results wrong after the rebuild

M4-WP3. The Actions engine

RowMeasurementPassFail
BF-4Kill a runner mid-job, then leave its Actions group idleThe job fails within 10 minutes of its last renewjobIt is still running at 11 minutes
BF-37on: merge_group workflow, and go-playground/webhooks parsing merge_group, pull_request.enqueued, and pull_request.dequeuedA run on each built queue ref; every queue ref deleted within 5 min of landing or ejection guess boundA missing delivery, or a queue ref left over
BF-54Hold 2,500 runner polls on one broker scope (BROKER_HELD_MAX plus 500) with jobs queuedPolls past the cap get 503; those runners re-poll within 30 s with their sessions valid; job start p99 at most 35 s for themAny RunnerSessionInvalid, a runner that exits, or a job unstarted after 60 s
BF-55Kill the leader site of an Owner scope's registration shard during a 256-job matrixPolls reach a broker on the new leader site within 15 s; every job runs once; zero jobs assigned twiceA job assigned twice, or dispatch stalled over 60 s
BF-56Kill the repository's commit path (and, separately, RunOwner) between RunCreated and the RequiredCheck writes, then finish or cancel every jobEvery RequiredCheck on an open PR head has a conclusion within 60 s of its job's CompletedAny RequiredCheck with conclusion: None after its job completed
BF-59gh secret set for a repository, an environment, and an org, then a job reading all three; then rotate the SealKey and the wrapping keyThe job sees all three values before and after both rotations, and a fork-PR job sees noneA wrong value, a value shown to a fork PR, or a plaintext secret found in any entry or log

M4-WP4. Hosted runners on Sandboxes

RowMeasurementPassFail
BF-6050 ubuntu-latest jobs queued for one owner with the cap at 20, while two graft-runners sweeps overlapAt most 20 hosted runners at once; every job finishes; no Sandbox is left 10 minutes after the last jobMore than 20 at once, a job that never finishes, or a leaked Sandbox

M4-WP5. The web app inference placement

RowMeasurementPassFail
BF-38Web app ("Frontend and native app" in design/SHARED.md) under Playwright inference: sign in, open a repository, a pull request's diff, checks, and timeline, and a running job's live log (the actions-logs-results oracle), then force the checks region's query to failEvery page renders, the first route's data is inlined in the HTML shell with zero API requests before first paint, the log streams live, and with the checks query failing only the checks region shows an errorAny page fails, the first route waits on a second round trip, or the checks failure breaks another region
BF-61View a running job's live log in the web app, then kill its LogRelayLines appear within 2 s; after the kill, the viewer falls back to step log blobs and the final log is completeA line later than 2 s, no fallback after the kill, or missing lines in the final log

M4-WP6. Milestone 4 gate

RowGate (Pass)
BM-4An unmodified actions/runner runs a matrix, and a site is failed mid-run with no lost job state
Milestone 5: release control and site days

M5-WP1. The Release group, genesis, and keys

RowMeasurementPassFail
BF-6Replay every historical Release entry, pre-signed undos and McpPointer and ControllerPointer entries included, after Keys { vendor: None }100 of 100 refusedAny accepted
BF-40Submit a second intent while one is current, and an intent naming a release with no ReleaseUploadedBoth refusedEither committed
BF-41Two genesis entries sent to an empty Release group: one matching the anchor, one signed by another keyOnly the matching one commitsThe other commits
BF-42Sign an entry offline while graft-ops commits an IntentStep409 with the current digest; the re-signed entry commitsA stale prev commits
BF-45A forged ReleaseUploaded, ReplaceMember, or pointer signed by the executor key, for a site not drained 24 hOnly a correctly signed ReleaseUploaded commitsA pin, or a ReplaceMember of an undrained site

M5-WP2. graft-front, pin-only pointers, and gradual edge deploys

RowMeasurementPassFail
BF-8Blackhole every Release member for 10 minutes, then set EdgePointer to a version outside the deploymentRequests keep the last pointers; the mismatch alarms within POINTER_TTL and the EdgeRoll that dropped it was refusedAny request blocked on the group, or a silent mismatch
BF-43EdgePointer undo under 1% EdgeRoll with the API blocked, then the next EdgeRollEvery request pins to N within POINTER_TTL; the next EdgeRoll sets NoneAny request off N while pinned, or the pin survives
BF-48Blackhole the Release group, then send a customer-signed StaleModeScoped groups serve stale() within POINTER_TTL; the flag expires at untilNo stale reads, or a flag past STALE_MAX honored

M5-WP3. The graft-ops executor and record formats

RowMeasurementPassFail
BF-7After FormatsEnable, submit the pre-signed Rollback to N and Abort { intent, target: N }Both refused, and both pageEither applied

M5-WP4. Site days and automatic drain

RowMeasurementPassFail
BF-39Kill graft-ops at each step of a SiteDay and an EdgeRoll, 100 timesIt resumes after the last IntentStep; no step skipped; one IntentDoneA skipped or doubled step

M5-WP5. The change feed, the export endpoints, and the vault

RowMeasurementPassFail
BF-47release roll end to end, including SiteDay { Workflows } and graftyard vault upgradeOrder as in Release roll; vault upgraded with no install tokenAny other order, or an install token used
BF-49Vault pulls every export endpoint; then the export jti is revokedAll four endpoints served; refused within READ_CACHE after revocationAny served after READ_CACHE

M5-WP6. Revocation and rotation runbooks

RowMeasurementPassFail
BF-44A vendor-signed undo not in the parent's undo list, and a Resume after Keys { vendor: None } without customer re-signed undosBoth refusedEither accepted
BF-46Rotate of each SecretName with live trafficNo new script version; zero failed calls; old value unused after the switchA new version, or failures

M5-WP7. Milestone 5 gate

RowGate (Pass)
BM-5A deliberately unreadable format released to graft-site-a commits nothing and moves A's groups. With the vendor token revoked mid-roll, Halt, Keys { vendor: None }, then Abort { intent, target: N } returns every site and pinned script to N, and a vendor-signed intent submitted after Keys { vendor: None } is refused. A bad graft-ops at 1% swaps back by OpsPointer with the API blocked
AM2: graft-mcp (after M5)

AM2-WP1. graftyard apps init and the graft-mcp authorization server

RowMeasurementPassFail
LF-6token/scoped from Graftyard MCP without graftyard_grant, with another user's grant, and with graftyard_write above the grant. A stale generation on POST /mcp/grants/{id}/generation. Grant calls with wrong client credentials422 for each bad token request. 409 and the grant revoked for the stale generation. 401 for wrong credentials, 100 of 100Any accepted

AM2-WP4. AM2 gate

RowGate (Pass)
LM-2MCP row passes, and a bad graft-mcp at 1% swaps back with the API blocked
AM3: the controller and Graftyard-run agents (after M5)

AM3-WP1. The agents account, graft-agents-front, and the controller

RowMeasurementPassFail
LF-7Bad graft-controller at 1% with the deployments API blocked. Submit the pre-signed ControllerPointer undo, then blackhole GET release/entries for 10 minutesEvery graft-agents-front isolate pins N within POINTER_TTL of its next request. Webhooks keep flowing on the last viewAny webhook reaches N+1 after the pin, or blocks on the read
LF-8release/entries with an entry signed for another install id, a Keys entry not signed by the anchored customer key, and an entry out of index orderAll refused by the agents view, 100 of 100Any accepted

AM3-WP3. Graftyard-run tasks and workspaces

RowMeasurementPassFail
LF-9Token ceilings by kind: a graftyard-agent token pushes to the base and the fork. A graftyard-workspace token pushes to the base in fork mode. A graftyard_task request from a third App, or for a repository that is neither the base nor its fork100 of 100 refused for eachAny accepted
RowMeasurementPassFail
L0-12, Graftyard-run kind100 pushes and 100 REST ref writes by a graftyard-workspace token with a live Graftyard-run task, into another holder's Exclusive scope under AgentsExclusive100 of 100 of each refused naming the lease, not NoLiveTaskAny accepted, or any refused before the lease check
B0-36, Graftyard-run kindA graftyard-workspace token with a live Graftyard-run task: a REST ref write into another task's Exclusive scope under AgentsExclusive, a push with no live task, and a fork push after its task is cancelled, over git and REST, with shipped and widened permissions100 of 100 refused for each, the ref write naming the leaseAny accepted, or the ref write refused before the lease check
B0-37, fetch halfA session ref pushed by graftyard-workspace under a live Graftyard-run task, then v0, v1, v2, and --mirror fetches and a v2 ls-refsThe session ref appears only to a v2 ref-prefix that starts with itIt appears to any other listing, or not to the matching ref-prefix

AM3-WP4. Session logs and session pages

RowMeasurementPassFail
LF-10Fork-mode session URL in a trailer, read by a base reader with no access to the forkThe page and JSON serve the log from the fork404, or the URL names the fork

AM3-WP5. AM3 gate

RowGate (Pass)
LM-3A3 end to end in both modes, prompt-only tasks included, the task rules row, containment, ceiling, and agent CI rows rerun, a Sandbox killed mid-task resumes, a bad controller at 1% swaps back, the provenance row passes
Milestone 6: region loss, SSH, and the lifeboat

M6-WP1. Region loss and re-protection

RowMeasurementPassFail
BF-50Lose site A with every API token revoked; drain it 24 h; then recover itExecutor-signed ReplaceMember moves every group with a voter on A, using B's and C's registries; RestorePlacement refused until undrainedA group left out, or a restore while drained

M6-WP2. The SSH front

RowMeasurementPassFail
BF-51Rotate SSH host keysUpdateHostKeys clients learn the new set; /api/graftyard/v1/meta/ssh lists both slots for the overlap; same fingerprints on every replicaA mismatch, or a host-key error for an updated client
BF-52Suspend a user, then push over SSH with their key; separately, revoke an SSH key with 1,000 of its commands queuedEvery push from the suspended user refused; 100% of commands decided after the revocation get 401Any accepted

M6-WP3. The lifeboat

RowMeasurementPassFail
BF-53Lifeboat enroll, a daily LifeboatDecoder, and tail and pack pulls with the R2 feed stoppedToken refused by the main edge; no 7-day alarm; tail and packs fetched from followersAny leader read, or a main-edge accept

M6-WP4. Milestone 6 gate

RowGate (Pass)
BM-6After two release rolls and one FormatsEnable, in a journey 5 drill, one site's member and container classes fail for a day, and ReplaceMember runs with every account API token revoked. It passes only if no script version is created, the spare reads every enabled format, every acknowledged write is present, groups return home, and enrolled gh pr list works against the lifeboat with the edge blocked
AM4: insights, the predictor, and Code Mode (after M6)

AM4-WP2. The insights consumer and the agents page

RowMeasurementPassFail
LF-11graftyard apps init on an install with no agents account, then the MCP and A5 insights flowsMCP and overlap check runs work. No Graftyard MCP or graftyard-insights secret exists in any agents accountEither flow needs the agents account
LF-14A5 replay with every InsightJob delivered twice, plus a stale job after a newer pushOne InsightVerdict entry per changed result, none for a repeat, 409 for the stale head, one check run per changed check, and every feed verdict event at an InsightVerdict indexAny duplicate entry or check run, a verdict for a moved head, or a feed verdict event at another index
LF-151,000 agent pushes across a leader changeExactly one InsightJob per PathSet entry, all from the Insights cursor, none from applyAny second job for one entry, an entry with no job, or a job from apply
LF-16PRs 4, 7 and 9, where 4 overlaps 7, 7 overlaps 9, and 4 does not overlap 9. Close 4, then close 7overlap:4 on all three, then overlap:7 on 7 and 9, then no overlap: label on 9. Closed PRs keep their labelsAny other name, or a label on an open PR that is in no cluster

AM4-WP3. Code Mode behind OrgPolicy.code_mode

RowMeasurementPassFail
LF-18/codemode for an org repository with code_mode off, then on, then off; once for a user-owned repository; a PATCH of code_mode by a non-adminRefused while off and for the user repository, served while on, refused within 60 s of turning it off, and the non-admin PATCH refusedAny Code Mode call served while off, on a user repository, or 60 s after turning it off, or a non-admin PATCH accepted

AM4-WP4. AM4 gate

RowGate (Pass)
LM-4Both predictor rows pass, A5 and A6 pass on a seeded repository, and Code Mode is enabled only if it serves 2026-07-28
Milestone 7: breadth and modes

M7-WP3. Managed front and cell accounts

RowMeasurementPassFail
BF-6620 isolates hit one key at the limit, then the counter object is killedx-ratelimit-remaining and enforcement overshoot by at most 20 requests times the number of isolates. The kill refunds at most one windowAn overshoot above that bound, or more than one window refunded

M7-WP4. Child groups for oversized metadata

RowMeasurementPassFail
BF-9A monorepo member reaching CHILD_GROUP_AT, and octokit.paginate over /repositories/{id} links on a managed cellThe metadata tier moves to a child group with zero SQLITE_FULL; page 2 is served by the right cellAny SQLITE_FULL, or a page routed to the wrong cell
BF-62Monorepo member reaches CHILD_GROUP_AT under 300 commands/s with comments, PR opens, merges and releases with tagsZero SQLITE_FULL. No metadata write gets 503. Every both-tier entry has its MetaHalf within 60 s, with zero lost or doubled. A comment on #n is never delivered before #n opened, 100 of 100. Pre-split feed cursors continue with no 410Any SQLITE_FULL, any metadata 503, a lost or doubled half, an out-of-order delivery, or a 410
BF-63RepoReset to a t before the split and to a t after it, on a split repositoryBoth tiers equal a refold at tAny tier differs
LF-17Split the metadata tier of a repository that has live InsightVerdict rowsFeed cursors keep working, verdict rows and Verdict events stay on Repo(r), and the check runs and labels reach RepoMeta(r)Any lost verdict, event, check run or label

M7-WP5. Solo and workerd for evaluation

RowMeasurementPassFail
BF-64LeaveEvaluation on a Cloudflare Solo install under write load. Separately, a vendor-signed LeaveEvaluationSystem groups and owners end on Tri with moving_from cleared, zero lost acknowledged writes, Placement::solo refused afterwards. The vendor-signed one is refusedAny lost write, a group left on Solo, or the vendor-signed one accepted

M7-WP6. Import from GHES with the plan preflight

RowMeasurementPassFail
BF-65GHES archive import with hooks, a workflow, watchers, merged PRs and #N links, on a synthetic ghe-migrator archive built from the documented format while no GHES instance exists inferenceZero webhook deliveries, runs or notifications. Numbers, created_at and authors match. Merged PRs show merged. legacy_id resolves every idAny delivery, run or notification, or any mismatch

M7-WP8. Milestone 7 gate

RowGate (Pass)
BM-7A self-hosted install with no vendor key passes milestones 1 to 6 gates and enables a new format. A vault restore to a chosen cutoff and a lifeboat-export restore into new accounts each lose no entry up to their cutoff, and afterward a developer's PAT and SSH key, a user-owned repository, and an org name all work, while a PAT revoked after the cutoff fails. A Solo group moves to Tri under load with no failed write