Skip to content

fix(view): activate durable metadata refresh - #27431

Closed
ck89119 wants to merge 31 commits into
matrixorigin:mainfrom
ck89119:issue-26227-activation-main
Closed

fix(view): activate durable metadata refresh#27431
ck89119 wants to merge 31 commits into
matrixorigin:mainfrom
ck89119:issue-26227-activation-main

Conversation

@ck89119

@ck89119 ck89119 commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

What type of PR is this?

  • API-change
  • BUG
  • Improvement
  • Documentation
  • Feature
  • Test and CI
  • Code Refactoring

Which issue(s) this PR fixes:

issue #26227

What this PR does / why we need it:

This is the final activation layer for the View metadata lifecycle built by #27267 and #27370 on top of the durable admission contract from #27430.

It:

  • advertises refresh support only after a CN observes the exact final catalog version and offset in READY state;
  • lets HAKeeper activate refresh only after every HAKeeper replica and known CN supports the lifecycle;
  • fences public metadata readers before a new membership epoch can require catalog revalidation;
  • runs one bounded, stopper-owned recovery worker per CN and acknowledges an epoch only after the durable catalog pass reaches a terminal state;
  • keeps refresh sealed when an epoch advance is canceled, rejects metadata-sensitive work from an older transaction epoch, and fences lifecycle-unaware CN or HAKeeper rollback;
  • enables fail-closed behavior for DESC, SHOW COLUMNS, information_schema.columns, and current-catalog CTAS while ordinary View queries continue to rebind the stored SQL;
  • preserves snapshot, subscription, restore, identity, and typed catalog-readiness behavior;
  • seeds revalidation through an idempotent version-offset upgrade.

The activation authority is the replicated HAKeeper admission epoch. No local MOCluster snapshot can independently authorize refresh.

Validation:

Design artifact

  • Approved/versioned design: View Metadata Lifecycle Admission, Activation, and Recovery
  • Revision: 2026-08-27 / v1, status Approved.
  • The document records the problem evidence, invariant and negation, activation state machine, durable first owner, consumer matrix, rollback/restart/restore behavior, and validation matrix. Any semantic change requires re-approval.

Review closure evidence

  • Metadata-sensitive statement classification is now AST-based at the frontend admission boundary. It walks table references for information_schema.columns instead of matching raw SQL text, so comments/formatting cannot bypass the fence; CTE aliases and string literals do not create false positives.
  • Exact PR-head binary: 483138ea57e7e5e5189d5a7d12cf6883f3ff31a8; exact old/base binary: 424fe3d821e45fe883ab93af5b404d77686d05a0.
  • Real 1 LogService/HAKeeper + 1 TN + 2 independently replaced CN sequence passed: all-old stale metadata (VARCHAR(12)), one-new/one-old fail-closed (0 visible rows), all-new activation (VARCHAR(48)), same-UUID CN restart, old-binary rollback denied before ingress, new-binary recovery, and immediate whole-cluster restart with durable revalidation.
  • Snapshot oracle passed on the same exact-head cluster: ALTER revalidated VARCHAR(12) to VARCHAR(48); RESTORE TABLE ... {SNAPSHOT=...} revalidated back to VARCHAR(12).
  • The first real restart run exposed a 30s CN startup deadline vs. 30s old-generation expiry race. The final head keeps the old-owner safety fence and gives admission a second discovery window; CN/HAKeeper normal and race owning-package tests pass.
  • Upstream matrixorigin/CI/.github/workflows/e2e-upgrade.yaml@main currently hard-disables Target/Release jobs with if: false; those SKIPPED jobs were not counted as validation evidence.

@ck89119 ck89119 closed this Aug 27, 2026
@ck89119
ck89119 deleted the issue-26227-activation-main branch August 27, 2026 06:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/bug Something isn't working size/XL Denotes a PR that changes [1000, 1999] lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants