ci: make critical artifact uploads resilient

[XuPeng-SH]

Summary

• add one reusable composite action that retries artifact publication up to three times with 5s and 15s backoff
• use it for the shared build, UT coverage, both required BVT coverage producers, the final coverage result, and both BVT execution logs
• reuse completed producer files instead of rerunning build or test work
• fail closed after the third failure for required artifacts while keeping diagnostic publication best-effort
• preserve cancellation and failure-path upload behavior
• make every failure-diagnostic upload in the current main entrypoint descriptive, best-effort, and isolated by run attempt
• add permanent workflow and retry-state-machine validation so later edits cannot silently break these contracts
• baseline the exact 9 deliberately disabled jobs so actionlint exceptions cannot hide a new constant-false condition
• migrate Darwin/x86 jobs from retired GitHub-hosted macos-12 and macos-13 labels to macos-15-intel
• remove the unnecessary mutable-tag changed-files action from the self-hosted validation job and pin every remaining third-party action in the new protection chain to a verified commit

Correctness and generation handling

Retry attempts use the same job-owned artifact name. Attempts two and three overwrite any partial artifact left by an ambiguous finalize response, so the downstream merge sees at most one profile from that producer.

Required producer artifacts include the current run attempt or shared assignment generation. BVT execution logs and all failure-diagnostic artifacts defined by the current main entrypoint workflows use immutable attempt-specific names. A rerun cannot collide with, overwrite, or misrepresent evidence from an earlier attempt.

The fixed-name final-result-files artifact retains its established consumer contract and explicitly overwrites on its first attempt.

Every internal composite step has an explicit not-cancelled status condition. This avoids the implicit success-only gate inside a composite action and preserves callers that intentionally upload artifacts after an earlier job step has failed.

BVT execution logs are diagnostic. They are published after container shutdown and required coverage publication, retry publication failures, and use continue-on-error so their final failure cannot turn a successful coverage producer into a failed prerequisite.

Missing UT profiles and other required single-file inputs remain errors. BVT coverage uploads retain their existing warning behavior for legacy branches where coverage is intentionally unsupported.

Observability and user experience

• each failed attempt emits a visible warning with the next backoff
• recovery on attempt two or three emits a notice
• exhausting all three attempts emits one clear wrapper-level error while preserving the root upload errors in preceding steps
• observability and backoff helper steps are best-effort and cannot override a successful upload result
• artifact-id, artifact-url, and artifact-digest are forwarded from the final successful generation
• all 12 failure-diagnostic uploads defined in the current main entrypoint workflows have descriptive step names, continue-on-error, and run_attempt in their artifact names
• coverage merge Step Summary distinguishes prepared output from published output, reports the actual terminal upload result, and links the successful final artifact directly
• coverage merge Step Summary points to the exact best-effort diagnostic artifact for the current attempt
• AJV strict-mode validation is warning-free, so real schema drift is not buried in validator noise
• disablement drift reports the exact workflow/job findings and fails even for non-literal constant-false expressions

Performance

The successful producer path still invokes actions/upload-artifact only once and executes no backoff shell step. The final coverage summary adds one small reporting shell step after publication. Retries operate on existing producer output.

The shared build is already a tar.gz file, so it uses compression level 0 to avoid redundant zlib work; text coverage profiles retain the default compression level 6.

Diagnostic retries run only after service cleanup and required coverage publication. Failure diagnostics remain direct single-attempt best-effort uploads so an already-failed job does not incur additional retry delay.

The protected Check Yaml job downloads one checksum-pinned ARM64 actionlint binary only when workflow/action validation is triggered and has a 10-minute job bound. The raw disablement-baseline pass reuses that binary; the full second scan completed locally in under 0.1 seconds. Removing changed-files removes a 7-second action step; unconditional validation costs the same measured 7 seconds on action-only changes and saves about 7 seconds when workflows change.

Validation

• all changed YAML parsed successfully
• every action plus all .yaml and .yml workflow files passed the current SchemaStore schemas (19 files)
• full-repository actionlint 1.7.12 passed both via the Go binary and the official multi-architecture container
• the actionlint runner configuration lists only real self-hosted labels and narrows existing disabled/legacy exceptions by workflow
• a second unfiltered actionlint pass baselines all 9 deliberate constant-false conditions by workflow, job, column, and source hash; an injected non-literal false expression was rejected
• the ARM64 actionlint release archive matched its pinned official SHA-256; installer shell syntax passed
• a local Draft-07 state-machine schema now enforces required-input semantics, stable defaults, the pinned upload implementation, three attempts, exact helper scripts, failure transitions, cancellation gates, retry overwrites, bounded backoff, fail-closed terminal, and successful-generation outputs
• the same schema compiled and validated cleanly with AJV 8.18 strict mode after closing every required/property and object-type declaration
• 14 injected regressions were rejected, covering status gating, overwrite, terminal failure, output generation, backoff removal/extension, fake exit, mutable action refs, all optional defaults, and defaults on required inputs
• temporary local callers passed actionlint for every input, all forwarded outputs, failure-path invocation, and successful-generation output selection
• focused structural assertions covered 7 retry callers, all 12 failure diagnostics in the current-main workflow set, required-before-diagnostic ordering, final publication reporting, and the fixed final-result-files consumer contract
• organization-wide code search found no automated consumers of the renamed diagnostic artifact names
• git diff --check passed
• GitHub runner-images documents macos-15-intel as the supported x86_64 successor to the fully retired macos-13; actionlint recognizes both migrated jobs
• checkout, JSON-schema validation, and all three internal upload attempts resolve to verified full commit SHAs; checkout credentials are not persisted
• the protected check name remains Validate GitHub Actions workflows
• Q1-Q3 audit passed: one artifact owner per name and generation, bounded and cancellable waits, three-attempt upper bound

Decision log

• required and high-value artifacts retry; failure diagnostics intentionally stay single-attempt best-effort to avoid extending an already-failed job by another 20 seconds
• actions/upload-artifact exposes only a step outcome, not a reliable error classification; the wrapper therefore retries any publication failure, preserves the root error, and bounds added backoff to 20 seconds instead of claiming every error is transient
• actionlint's workflow/expression checks are enforced; its external shellcheck and pyflakes integrations remain disabled because the repository has a separate existing backlog that is outside this artifact-resilience change
• final-result-files intentionally keeps its fixed external name; overwrite is explicit and publication status is reported after the terminal upload state
• MatrixOne's legacy coverage workflow is pinned to CI@release/3.0-dev and is outside this main-branch PR's delivery boundary; it must be changed on that release branch rather than by editing the unused main copy
• macos-15-intel preserves the Darwin/x86 release architecture while replacing hosted images that GitHub no longer schedules
• the validation job always checks all 18 small workflow files instead of executing a mutable change-detection dependency; repository-owned reusable actions intentionally remain on @main so fixes propagate to MatrixOne callers

Mitigates matrixorigin/matrixone#27526 and hardens the broader artifact publication and diagnostic paths.

[XuPeng-SH]

Follow-up deep review completed on b46ddbb.

Findings fixed:

1. The green state-machine validation still emitted AJV strict-mode warnings. The schema now closes every object type and required/property declaration and compiles warning-free under AJV 8.18 strict mode.
2. The global constant-false actionlint exception could hide future accidental job or step disablements. Exceptions are now path-scoped and an unfiltered second pass baselines exactly 9 intentional findings by workflow/job/column/source hash.
3. macos-12 and macos-13 were incorrectly modeled as self-hosted labels even though these jobs use retired GitHub-hosted x86 images. Both Darwin/x86 jobs now use macos-15-intel.

Fresh evidence: full actionlint passed; 18 workflows plus the composite action passed current SchemaStore schemas; AJV strict validation passed; 5/5 state-machine counterexamples and a non-literal constant-false counterexample were rejected; artifact closure remains 7 retry callers and 12 isolated failure diagnostics; git diff --check passed.

No artifact-path correctness or performance blockers remain in the complete diff.

[XuPeng-SH]

Follow-up deep review — 82dd114

本轮从供应链、状态机契约、失败路径和性能四个方向复核，并已直接修复发现的问题。

已修复

• 移除自托管校验任务里不必要的 tj-actions/changed-files@v47，消除 mutable tag/历史供应链事件带来的攻击面。
• 将新增保护链所需的第三方 action 固定到经签名验证的精确 commit：
  • actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803（v6），并设置 persist-credentials: false
  • dsanders11/json-schema-validate-action@eddf079f55830cc9a916a3c512ba9086240d2fea（v2.0.0）
  • actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a（v7.0.1）
• 收紧状态机 schema：退避、恢复和终态脚本改为精确匹配，避免附加无限等待或伪造 exit 1 绕过校验。
• 锁定输入契约：必填 name/path 不允许默认值；4 个可选输入的默认值和 required: false 均纳入 schema。
• 工作流 schema 改为稳定校验全部 18 个 workflow，不再依赖变更文件检测。

性能核对

历史 job 的实测阶段耗时：changed-files 约 7s，workflow schema 约 7s。移除 changed-files 后：

• workflow 变更 PR：约节省 7s；
• 仅 action 变更 PR：从“7s 检测后跳过”变成“7s 全量 schema 校验”，总体近似持平；
• 18 个 workflow 的校验集合有界，不引入随仓库内容无界增长的等待。

验证结果

• SchemaStore：18 个 workflow + composite action 全通过
• actionlint 1.7.12：通过
• AJV 8.18 strict：schema 编译及当前 action 校验通过、零 warning
• 14/14 反例均被拒绝，包括隐式成功、错误 overwrite、best-effort 终态、陈旧输出、缺少退避、mutable upload ref、额外无界等待、伪造终态退出和输入默认值漂移
• artifact 闭环：7 个 retry caller、2 个 best-effort execution log、12 个隔离失败诊断
• git diff --check：通过

结论：本轮发现的问题已在 82dd114 系统性修复；当前代码层面没有剩余 blocker，可以进入外部 reviewer approval。