bootjp · bootjp · Jul 13, 2026 · Jul 13, 2026 · Jul 13, 2026 · Jul 13, 2026
diff --git a/docs/design/2026_02_18_partial_hotspot_shard_split.md b/docs/design/2026_02_18_partial_hotspot_shard_split.md
@@ -293,12 +293,13 @@ Add RPCs:
 2. Job phases: BACKFILL/FENCE/DELTA/CUTOVER
 3. Manual split with target-group relocation
 
-Status: partial. SplitJob catalog/codec, MVCC export/import primitives, staged
-visibility, write-fence checks, target readiness, and target-promotion catalog
-components exist in the M2 stack, but no production runner advances cross-group
-jobs to `DONE` yet. M2 remains open in
-[`2026_06_11_partial_hotspot_split_milestone2_migration.md`](2026_06_11_partial_hotspot_split_milestone2_migration.md)
-until the cross-group acceptance criteria and Jepsen workload pass.
+Status: implemented. The production runner advances durable cross-group jobs
+through BACKFILL, FENCE, DELTA_COPY, CUTOVER, CLEANUP, and `DONE`; current-voter
+readiness and cleanup barriers protect leadership changes; and the deterministic
+Jepsen workload covers the split alongside leader-kill and partition packages.
+The completed M2 contract is recorded in
+[`2026_06_11_implemented_hotspot_split_milestone2_migration.md`](2026_06_11_implemented_hotspot_split_milestone2_migration.md)
+and its final runner/lifecycle slice landed in PR #1096.
 
 ### Milestone 3: Automation
 

diff --git a/...ial_hotspot_split_milestone2_migration.md → ...ted_hotspot_split_milestone2_migration.md b/...ial_hotspot_split_milestone2_migration.md → ...ted_hotspot_split_milestone2_migration.md
@@ -1,13 +1,13 @@
 # Hotspot Shard Split — Milestone 2: Migration Plane
 
-Status: Partial
+Status: Implemented
 Author: bootjp
 Date: 2026-06-11
 
 Parent: [2026_02_18_partial_hotspot_shard_split.md](2026_02_18_partial_hotspot_shard_split.md).
 M1 (control plane) is as-built in [2026_02_18_implemented_hotspot_split_milestone1_pr.md](2026_02_18_implemented_hotspot_split_milestone1_pr.md).
 
-Current implementation status: partial. The SplitJob catalog/codec, versioned route descriptor storage, range export/import primitives, staged-read visibility, write-fence checks, target readiness, and target-promotion catalog pieces have landed or are covered by the active M2 stack. The feature is not implemented end to end until a production runner advances cross-group jobs to `DONE`, `ListRoutes` exposes the moved child on its target group after cutover, and the Jepsen cross-group split workload passes the acceptance criteria in §13.
+Current implementation status: implemented. The M2 stack provides the durable SplitJob catalog, versioned route descriptors, resumable range export/import, staged/live visibility, source write and read fences, current-voter readiness and cleanup barriers, constant-time route cutover, incremental target promotion, bounded source/target cleanup, terminal `DONE` history, and the deterministic cross-group Jepsen workload. PR #1096 completed the production runner and lifecycle wiring. Automatic hotspot detection and split scheduling remain M3 scope; the broader route-shuffle and fault campaign remains M4 scope.
 
 ## 1. Background
 
@@ -1243,7 +1243,7 @@ Phased into reviewable PRs, each lands behind its own doc-or-test gate:
 | M2-PR5 | Coordinator + FSM FENCE rejection (`ErrRouteWriteFenced`) with point and `DEL_PREFIX` range-footprint checks + route-faithful txn-lock drain (§3.2a.0a) + same-group `SplitRange` overlap rejection while a SplitJob is live | fsm + coordinator unit + `migrator_lock_drain_test` + `catalog_test` overlap red controls |
 | M2-PR6 | Cross-group end-to-end: ExportRangeVersions / ImportVersions server-side handlers + migrator BACKFILL/DELTA_COPY + raw-candidate staged/live merge read path in both scan directions and `LatestCommitTS` + §7.2.2e source-side cutover read-fence arm with every-current-source-voter ACK, membership-epoch re-ACK, catalog-version waiter, route-key-normalized scan ownership checks, and server-stamped RawKV read versions | integration incl. delete/TTL DELTA_COPY, Redis list/hash/set/zset/stream migration, HLC restart/fence-floor cases + `kv/fsm_cutover_read_fence_test.go` |
 | M2-PR7 | Target-local `PromotionState` + background promoter + ordered default-group promotion-complete CAS retaining `min_write_ts_exclusive` + source/target cleanup before DONE history move; readiness guard accepts matching cleared descriptors while retained; `AbandonSplitJob` with durable `ABANDONING` cleanup witness + CLEANUP GC + Jepsen split workload | `kv/fsm_promote_staged_test.go` raw hidden-version/LatestCommitTS merge + jepsen suite |
-| M2-PR8 | Rename `*_proposed_*` → `*_partial_*` after PR1 ships; update parent partial doc M2 status; rename to `*_implemented_*` after PR7 |  |
+| M2-PR8 | Rename `*_proposed_*` → `*_partial_*` after PR1 ships; update parent partial doc M2 status; rename to `*_implemented_*` after PR7 | Completed after the runner, voter barriers, cleanup lifecycle, route publication, and Jepsen workload landed in PR #1096. |
 
 Each PR follows the five-lens self-review and is gated by its tests + `make lint`.
 
@@ -1283,9 +1283,12 @@ This is independent of the existing rolling-upgrade protocol for unrelated subsy
 
 ## 14. Lifecycle
 
-This document is `*_partial_*` because M2-PR1 and later component slices have landed, but the cross-group migration plane is not complete end to end.
+This document is `*_implemented_*` because the M2 cross-group migration plane is complete as a central subsystem. The production runner now resumes durable jobs across leadership changes, drives every phase through `CLEANUP`, waits for current source and target voter proofs, publishes the target route, removes bounded migration state, and moves the job to `DONE` history. The deterministic Jepsen split workload exercises the cross-group operation alongside leader-kill and partition fault packages.
 
-- Track per-PR landing under §11.
-- Rename to `*_implemented_*` only after the acceptance criteria in §13 pass: cross-group `StartSplitMigration` reaches `phase=DONE`, `ListRoutes` shows the target-group child after cutover, leader-kill recovery is automatic, and the Jepsen split workload is green.
+The remaining work is intentionally outside M2's central scope:
+
+- M3 owns automatic hotspot detection, target selection, and split scheduling.
+- M4 owns the broader route-shuffle nemesis matrix and production-scale fault campaigns.
+- Reverse migration after CUTOVER and concurrent migration jobs remain future extensions.
 
 `git mv` is used so history follows.
diff --git a/docs/design/2026_06_12_proposed_scaling_roadmap.md b/docs/design/2026_06_12_proposed_scaling_roadmap.md
@@ -343,9 +343,9 @@ control-plane (`*_proposed_*` doc TBD).**
 
 - M1 standalone but doesn't enable cross-region writes — it just
   makes Raft survive cross-WAN partition.
-- M2 depends on the M2 hotspot-split migration contract
-  (`2026_06_11_partial_hotspot_split_milestone2_migration.md`)
-  being implemented so the monotone-merge primitive exists.
+- M2's hotspot-split migration dependency is implemented in
+  [2026_06_11_implemented_hotspot_split_milestone2_migration.md](2026_06_11_implemented_hotspot_split_milestone2_migration.md), including
+  the monotone-merge primitive.
 - M3 depends on M1's region-aware membership and M2's per-region
   ceiling.
 - M4 depends on M2 and M3.
@@ -538,7 +538,7 @@ the ceiling shape:
 Composability invariant: **every monotone-merge happens via the
 same `SetPhysicalCeiling` + `Observe` primitive**. The M2
 hotspot-split contract (§6.2.1 of
-`2026_06_11_partial_hotspot_split_milestone2_migration.md`) is the
+[2026_06_11_implemented_hotspot_split_milestone2_migration.md](2026_06_11_implemented_hotspot_split_milestone2_migration.md)) is the
 reference implementation; per-region and per-group merges reuse it.
 
 ### 7.2 Capability bits

diff --git a/docs/design/2026_06_23_proposed_scaling_roadmap.md b/docs/design/2026_06_23_proposed_scaling_roadmap.md
@@ -98,10 +98,10 @@ memory each group's private cache/memtable pins.
 
 - **Range split — distribute a range across groups.** Same-group split
   shipped in M1 (`distribution/`). Cross-group migration (the part that
-  actually relocates data and reduces per-node volume) is tracked by **PR #945**
-  and the active M2 stack
-  (`docs/design/2026_06_11_partial_hotspot_split_milestone2_migration.md`,
-  branch `docs/hotspot-split-m2-proposal`): a resumable `SplitJob` with
+  actually relocates data and reduces per-node volume) is implemented by the
+  M2 stack recorded in
+  [2026_06_11_implemented_hotspot_split_milestone2_migration.md](2026_06_11_implemented_hotspot_split_milestone2_migration.md):
+  a resumable `SplitJob` with
   `PLANNED → BACKFILL → FENCE → DELTA_COPY → CUTOVER → CLEANUP → DONE` phases
   driven by a migrator on the default-group leader. M2 is the required
   ownership-migration mechanism, but it reduces per-node bytes only when the

diff --git a/main_encryption_registration.go b/main_encryption_registration.go
@@ -126,19 +126,19 @@ func retryUntilRegistered(ctx context.Context, what string, fn func() error) err
 func setupDistributionAndRegistration(
 	runCtx context.Context,
 	eg *errgroup.Group,
-	runtimes []*raftGroupRuntime,
+	distCatalog *distribution.CatalogStore,
 	engine *distribution.Engine,
 	coordinate *kv.ShardedCoordinator,
 	defaultGroup *kv.ShardGroup,
 	w encryptionWriteWiring,
 	raftID string,
 	sidecarPath string,
-) (*distribution.CatalogStore, error) {
+) error {
 	if err := validateRaftRegistrationStartupEpoch(defaultGroup, w, raftID, sidecarPath); err != nil {
-		return nil, err
+		return err
 	}
 	if err := installProcessStartRegistrationGate(runCtx, eg, coordinate, defaultGroup, w, raftID); err != nil {
-		return nil, err
+		return err
 	}
 	installRaftRegistrationVerifier(defaultGroup, w, raftID)
 	installRuntimeRaftRegistrationWatcher(runCtx, eg, coordinate, defaultGroup, w, raftID, sidecarPath)
@@ -153,11 +153,38 @@ func setupDistributionAndRegistration(
 	installRuntimeRegistrationWatcher(runCtx, eg, coordinate, defaultGroup, w, raftID)
 	// Bootstrap + registration both run under runCtx so a shutdown
 	// cancels the bounded retry rather than hanging.
-	distCatalog, err := setupDistributionCatalog(runCtx, runtimes, engine)
-	if err != nil {
-		return nil, err
+	return ensureDistributionCatalogSnapshot(runCtx, distCatalog, engine)
+}
+
+func ensureDistributionCatalogSnapshot(
+	ctx context.Context,
+	distCatalog *distribution.CatalogStore,
+	engine *distribution.Engine,
+) error {
+	if distCatalog == nil {
+		return errors.New("distribution catalog store is not available")
+	}
+	// EnsureCatalogSnapshot may Save through the direct (non-raft) write
+	// path. When the §7.1 storage envelope is active and this load's
+	// writer registration has not yet committed, that Save fails closed
+	// with store.ErrWriterNotRegistered (Stage 7a-2). retryUntilRegistered
+	// retries the bootstrap until the registration goroutine — armed
+	// before this call in setupDistributionAndRegistration — commits and
+	// the gate clears. The common cases (populated catalog → no-op Save,
+	// or pre-cutover → cleartext Save) never hit the gate and return on
+	// the first attempt.
+	//
+	// Idempotency requirement: the retry re-invokes EnsureCatalogSnapshot
+	// from scratch on each ErrWriterNotRegistered, so it MUST be
+	// re-entrant — on the populated-catalog path it is a version-unchanged
+	// no-op Save (no mutation, no nonce), so re-running it is safe.
+	if err := retryUntilRegistered(ctx, "distribution catalog bootstrap", func() error {
+		_, e := distribution.EnsureCatalogSnapshot(ctx, distCatalog, engine)
+		return errors.Wrap(e, "ensure catalog snapshot")
+	}); err != nil {
+		return errors.Wrapf(err, "initialize distribution catalog")
 	}
-	return distCatalog, nil
+	return nil
 }
 
 func installRaftRegistrationVerifier(defaultGroup *kv.ShardGroup, w encryptionWriteWiring, raftID string) {