Add fenced Raft member replacement operation#1122
Conversation
|
Warning Review limit reached
Next review available in: 39 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (8)
📝 WalkthroughWalkthroughRaftの設定インデックスと保留中の構成変更をステータスAPI・CLIへ追加し、フェンス付き同一IDメンバー置換を再開可能な状態機械として実装しました。関連する安全検証、障害復旧テスト、運用ドキュメントも追加されています。 ChangesRaftステータスとメンバー置換
Estimated code review effort: 5 (Critical) | ~120 minutes Sequence Diagram(s)sequenceDiagram
participant Operator
participant ReplacementScript as raft-member-replace.sh
participant RaftAdmin as raftadmin
participant RollingUpdate as rolling-update.sh
Operator->>ReplacementScript: --execute with target and fence controls
ReplacementScript->>RaftAdmin: read status and configuration
ReplacementScript->>RaftAdmin: remove voter and add learner
ReplacementScript->>RollingUpdate: deploy learner
ReplacementScript->>RaftAdmin: wait for catch-up and promote learner
ReplacementScript->>RollingUpdate: redeploy promoted voter
ReplacementScript-->>Operator: report stability and application verification
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Implemented the fenced same-ID replacement state machine with quorum, identity, fencing, configuration-index, catch-up, restart, and application verification gates. Verification on 9bfc113:
@codex review |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new script, scripts/raft-member-replace.sh, along with design documentation and tests to automate safe, fenced, and resumable same-ID voter replacement in a Raft group. To support this workflow, the Raft status response has been extended with configuration_index and pending_conf_change fields across the proto, engine, and admin CLI. Feedback on the changes identifies a high-severity issue in the remote SSH execution block of the replacement script, where a standalone sudo -n true check under set -e can cause premature script termination if passwordless sudo is not configured. A suggestion is provided to safely evaluate the sudo check within the conditional statement.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
9bfc113 to
a23368c
Compare
|
Updated the signed head to a23368c after an additional shell audit.
Previous reviews are stale against the amended head. @codex review |
|
@coderabbitai review |
✅ Action performedReview finished.
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 55d863c1e2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Addressed all latest-head P1/P2 findings in signed commit bc84f3e.\n\nChanges:\n- force execute rollouts even when deployment inventory contains DRY_RUN=true\n- persist and compare the complete membership signature plus configuration index across Remove/Add/Promote\n- ignore replacement controls sourced from deploy.env and require fresh invocation-level authorization\n- reject completed state reuse\n- normalize DATA_DIR and reject filesystem root\n\nVerification:\n- ShellCheck: no findings\n- replacement lifecycle, response-loss resume, config-drift rejection, invocation authorization, quorum, and lock tests: pass\n- affected root tests under race: pass\n- RaftAdmin and etcd engine packages under race: pass\n- golangci-lint commit hook: 0 issues\n- git diff --check: pass\n- the concurrent full local suite hit unrelated adapter/store election-resource failures while other design worktrees were compiling; current-head CI is the authoritative full-suite rerun\n\n@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bc84f3e587
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Addressed all current-head P2 findings in signed commit df27d70. Changes:
Verification:
@codex review |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/raft-member-replace.sh`:
- Line 345: bash 4.3 以前の set -u 環境で空配列展開が失敗しないよう、scripts/raft-member-replace.sh
の345行では NODE_IDS の要素数が0より大きい場合 בלבד contains_value を実行し、768-771行では L788/L793 の
sudo_cmd 配列展開を未設定・空配列対応の形式に置き換える。
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: bd8a6fb2-0233-4c51-b57c-9480bdf574b3
⛔ Files ignored due to path filters (1)
proto/service.pb.gois excluded by!**/*.pb.go
📒 Files selected for processing (13)
cmd/raftadmin/main.gocmd/raftadmin/main_test.godocs/design/2026_07_18_implemented_fenced_raft_member_replacement.mddocs/raft_learner_operations.mdinternal/raftadmin/server.gointernal/raftadmin/server_test.gointernal/raftengine/engine.gointernal/raftengine/etcd/engine.gointernal/raftengine/etcd/engine_test.goproto/service.protoraft_member_replace_script_test.goscripts/raft-member-replace.shscripts/rolling-update.env.example
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: df27d7089e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Addressed every latest-head P1/P2 and compatibility finding in signed commit b1e6e5a. Changes:
Verification:
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b1e6e5a40e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Addressed all six latest-head P2 findings in signed commit 4d54f11. Changes:
Verification:
@codex review |
|
Codex Review: Didn't find any major issues. Breezy! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Summary
This is stacked on #1118 because the replacement deploy uses its fresh learner join mode.
Safety
previous_indexVerification
go test -count=1 . -run TestRaftMemberReplaceScriptgo test -race -count=1 . ./cmd/raftadmin ./internal/raftadmin -run "TestRaftMemberReplaceScript|TestExecuteCommandLeaderAndStateOutput|TestServerMapsEngineAdminMethods"go test -count=1 ./internal/raftengine/etcdgolangci-lint run ./... --timeout=5mmake -C proto gen(includes breaking-change check)go test ./...compiled all packages but hit an etcd test failure under concurrent package load; the etcd package passed when rerun aloneAuthor: bootjp
Summary by CodeRabbit
新機能
raftadmin statusに構成インデックスと構成変更保留状態を表示する項目を追加。ドキュメント
テスト