Skip to content

coordinator: preserve stop operator on repeated warnings - #6135

Merged
ti-chi-bot[bot] merged 1 commit into
pingcap:masterfrom
wk989898:stop-op
Sep 2, 2026
Merged

coordinator: preserve stop operator on repeated warnings#6135
ti-chi-bot[bot] merged 1 commit into
pingcap:masterfrom
wk989898:stop-op

Conversation

@wk989898

@wk989898 wk989898 commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

What problem does this PR solve?

Issue Number: close #6134

What is changed and how it works?

Introduce a stop operator kind to distinguish current-placement stops from stale-owner cleanup.

Repeated non-removing stop requests now reuse the existing current-placement operator, preserving its target node and maintainer epoch. Stale-owner cleanup operators are not reused when the current owner needs to be stopped. This prevents maintainer recovery from getting stuck after repeated warning events.

Check List

Tests

  • Unit test
  • Integration test

Questions

Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?

Release note

Fix a changefeed stuck issue after downstream failures.

Summary by CodeRabbit

  • Bug Fixes
    • Improved changefeed stop handling when ownership changes.
    • Prevented duplicate stop requests from replacing an existing operation unnecessarily.
    • Ensured cleanup requests for stale owners do not interfere with stops targeting the current owner.
    • Preserved the original target and maintainer epoch when repeated stop requests are received.

Signed-off-by: wk989898 <nhsmwk@gmail.com>
@ti-chi-bot ti-chi-bot Bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/needs-triage-completed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. needs-cherry-pick-release-nextgen-202603 Should cherry pick this PR to release-nextgen-202603 branch. and removed do-not-merge/needs-triage-completed labels Aug 31, 2026
@coderabbitai

coderabbitai Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 432d21db-e273-4bdc-be42-63196e916994

📥 Commits

Reviewing files that changed from the base of the PR and between 51db518 and e9f5194.

📒 Files selected for processing (4)
  • coordinator/operator/operator_controller.go
  • coordinator/operator/operator_controller_test.go
  • coordinator/operator/operator_stop.go
  • coordinator/operator/operator_stop_test.go

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The change adds stop-operator classifications for current placements and stale owners. The controller now reuses compatible non-removing stop operators while creating new operators when stale-owner cleanup targets differ. Tests validate node targeting, epoch preservation, and cleanup replacement.

Changes

Stop operator reuse

Layer / File(s) Summary
Stop operator classification contract
coordinator/operator/operator_stop.go, coordinator/operator/operator_stop_test.go
The operator stores a stopChangefeedKind. Constructor calls pass the current-placement classification.
Controller stop request reuse
coordinator/operator/operator_controller.go
Remote stops use the stale-owner classification. Current-placement requests reuse existing compatible operators. Stale-owner requests reuse only matching node and maintainer epoch operators.
Stop request reuse validation
coordinator/operator/operator_controller_test.go
Tests verify owner targeting, original epoch preservation, current-placement reuse, and replacement of stale-owner cleanup operators.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to e9f51

The change is a localized coordinator behavior update, and no actionable merge-blocking risk remains after normal checks and review.

Suggested reviewers: hongyunyan

Poem

A rabbit sorts stop notes in a neat little row
Current homes stay, stale owners must go
Epochs remain with the operators they knew
Fresh owners receive fresh work to do
The changefeed hops onward, steady and bright

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 4 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes address issue #6134 by distinguishing current-placement stops from stale-owner cleanup, reusing valid repeated stop operators, and scheduling a fresh stop for the current owner. These beha…
Out of Scope Changes check ✅ Passed All production and test changes support the stated objective in issue #6134. No unrelated code or documentation changes are present.
Title check ✅ Passed The title clearly and concisely describes the main change: preserving the stop operator when repeated warnings occur.
Description check ✅ Passed The description includes the required issue reference, explains the stop operator behavior, identifies unit and integration tests, addresses the release note, and matches the pull request objectives. …
Full details: Linked Issues check

Explanation

The changes address issue #6134 by distinguishing current-placement stops from stale-owner cleanup, reusing valid repeated stop operators, and scheduling a fresh stop for the current owner. These behaviors support maintainer recovery and checkpoint progress after downstream recovery.

Full details: Description check

Explanation

The description includes the required issue reference, explains the stop operator behavior, identifies unit and integration tests, addresses the release note, and matches the pull request objectives. The two question fields are blank, but this is non-critical because the description is otherwise complete.

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@wk989898 wk989898 added the needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. label Aug 31, 2026
@ti-chi-bot ti-chi-bot Bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Sep 2, 2026
@ti-chi-bot

ti-chi-bot Bot commented Sep 2, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: asddongmen, lidezhu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Sep 2, 2026
@ti-chi-bot

ti-chi-bot Bot commented Sep 2, 2026

Copy link
Copy Markdown

[LGTM Timeline notifier]

Timeline:

  • 2026-09-02 06:39:30.825983833 +0000 UTC m=+1261405.997077940: ☑️ agreed by asddongmen.
  • 2026-09-02 08:58:14.266074257 +0000 UTC m=+1269729.437168374: ☑️ agreed by lidezhu.

@ti-chi-bot
ti-chi-bot Bot merged commit ea94ac1 into pingcap:master Sep 2, 2026
40 checks passed
@ti-chi-bot

Copy link
Copy Markdown
Member

In response to a cherrypick label: new pull request created to branch release-8.5: #6150.

@ti-chi-bot

Copy link
Copy Markdown
Member

In response to a cherrypick label: new pull request created to branch release-nextgen-202603: #6151.
But this PR has conflicts, please resolve them!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved lgtm needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. needs-cherry-pick-release-nextgen-202603 Should cherry pick this PR to release-nextgen-202603 branch. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

new architecture: changefeed remains unbootstrapped after downstream cluster restart due to stale maintainer epoch

4 participants