feat(storage): standardize storage SPI capabilities + TCK (issue #5303) - #5323
Merged
Merged
Conversation
…he#5303) Add a typed capability-marker pattern on top of MeshStoragePlugin so callers can discover backend support for each optional feature (end-offset query, pull-cursor rewind, deferred POP ACK, lite topic) via `instanceof`, and so the per-backend TCK can assert that each declared capability is in fact implemented. What ships in this PR (single merged PR per the apache#5296 plan): 1. StorageCapabilities — outer interface in eventmesh-storage-api/.../storage/ that groups 7 capability sub-interfaces (3 universal + 4 backend-specific). Each sub-interface `extends StorageCapabilities` so a single `Class.isAssignableFrom` check covers all of them. Universal (every backend): - TopicManagement (createTopic) - PartitionAssignment (assignPartitions) - ExplicitOffsetCommit (commitOffset) Backend-specific: - EndOffsetQuery (Kafka only) - AlignPullOffset (Kafka + RocketMQ 4.x) - DeferredPopAck (RocketMQ 5.x) - LiteTopic (RocketMQ 5.x; mirrors the historical LiteTopicCapable) 2. MeshStoragePluginTCK — abstract JUnit 5 test base in eventmesh-storage-api/.../storage/tck/. Subclasses provide a fresh plugin and the set of capabilities they declare. The TCK runs the universal lifecycle tests unconditionally and gates the capability-specific tests on the declared set. A capability-declaration assertion catches the case where a capability is removed from `implements` but the author forgot to remove it from the test. 3. InMemoryStoragePlugin + MeshStoragePluginTCKSelfTest — in-memory fixture that implements ALL 7 capabilities. The TCK self-test runs the TCK against itself; if any TCK test fails against this plugin, the TCK is broken (too strict, wrong signature, or assumes a behavior the real backends can't provide). 4. 3 plugin capability declarations: - KafkaMeshStoragePlugin declares 5 (universal 3 + EndOffsetQuery + AlignPullOffset) - RocketMQRemotingStoragePlugin declares 4 (universal 3 + AlignPullOffset) - RocketMQ5RemotingStoragePlugin declares 5 (universal 3 + DeferredPopAck + LiteTopic) 5. 3 plugin TCK wirings + build.gradle updates (junit-jupiter + useJUnitPlatform on the 3 storage-plugin modules that didn't have it). Capability matrix (the single source of truth for "what does each backend support"): Capability Kafka RocketMQ 4.x RocketMQ 5.x -------------------------- ----- ------------ ------------ TopicManagement yes yes yes PartitionAssignment yes yes yes ExplicitOffsetCommit yes yes yes EndOffsetQuery yes no no AlignPullOffset yes yes no DeferredPopAck no no yes LiteTopic no no yes Test results: 40 tests, 0 failures, 0 errors across the 4 modules (eventmesh-storage-api self-test + 3 plugin TCK wirings). Closes apache#5303.
…agePluginTCK
The CI build failed on `checkstyleMain` with 2 warnings:
MeshStoragePluginTCK.java:35:1: Wrong order for 'java.util.Properties' import. [ImportOrder]
MeshStoragePluginTCK.java:38:1: Wrong order for 'org.junit.jupiter.api.Assertions.assertDoesNotThrow' import. [ImportOrder]
The checkstyle ImportOrder rule in `style/checkStyle.xml` defines groups as:
org.apache.eventmesh, org.apache, java, javax, org, io, net, junit, com, lombok
with `separatedStaticGroups=true` and `option=top` (static imports on top, in
the same group order).
My original import order was:
1. org.apache.eventmesh.* and org.apache.eventmesh.common.wire.*
2. org.junit.jupiter.api.* ← should be AFTER java.*
3. java.util.* ← should be BEFORE org.*
4. static org.junit.jupiter.api.Assertions.*
Reordered to:
1. static org.junit.jupiter.api.Assertions.* (top, 'org' group)
2. org.apache.eventmesh.* and org.apache.* ('org.apache' group)
3. java.util.* ('java' group, before org)
4. org.junit.jupiter.api.* ('org' group, after java)
Verified locally:
./gradlew :eventmesh-storage-plugin:eventmesh-storage-api:checkstyleMain PASS
./gradlew :eventmesh-storage-plugin:eventmesh-storage-api:check \
:eventmesh-storage-plugin:eventmesh-storage-kafka:check \
:eventmesh-storage-plugin:eventmesh-storage-rocketmq:check \
:eventmesh-storage-plugin:eventmesh-storage-rocketmq5:check PASS
Contributor
Author
|
Force-pushed to fix checkstyle ImportOrder violations in MeshStoragePluginTCK.java (2 warnings on ). No code changes — only import group ordering. Verified locally: all 4 storage modules pass |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Single PR that closes #5303 — standardizes storage SPI capabilities via a typed marker pattern, and ships a JUnit 5 TCK every backend must pass.
Why
The existing
MeshStoragePlugininterface is rich but backend-specific behaviors (end-offset query, pull-cursor rewind, deferred POP ACK, lite topic) are scattered betweenMeshStoragePlugindefault methods, the standaloneLiteTopicCapableinterface, and ad-hocinstanceofchecks on concrete classes (e.g.createTopic). Callers have to know per-backend which features are supported, and there's no shared test base that catches an accidental removal of a capability override.What changes
New (in
eventmesh-storage-api)StorageCapabilities— outer interface grouping 7 capability sub-interfacestck.MeshStoragePluginTCK— abstract JUnit 5 test baseInMemoryStoragePlugin(test scope) — fixture implementing all 7 capabilitiestck.MeshStoragePluginTCKSelfTest(test scope) — runs the TCK against the in-memory fixtureModified
KafkaMeshStoragePlugindeclares 5 capabilities (universal 3 +EndOffsetQuery+AlignPullOffset)RocketMQRemotingStoragePlugindeclares 4 (universal 3 +AlignPullOffset)RocketMQ5RemotingStoragePlugindeclares 5 (universal 3 +DeferredPopAck+LiteTopic)*MeshStoragePluginTCKTestclasses wire the TCK for each backendbuild.gradlefiles addjunit-jupiter+useJUnitPlatformCapability matrix
TopicManagementPartitionAssignmentExplicitOffsetCommitEndOffsetQueryAlignPullOffsetDeferredPopAckLiteTopicThe 3 universal capabilities are part of the
MeshStoragePlugincontract already; declaring them viaStorageCapabilitiesis a self-audit so dropping a method body by accident fails the TCK, not a smoke test. The 4 backend-specific capabilities are new — they correspond to existing behavior that was previously undiscoverable.Test results
40 tests across 4 modules, 0 failures:
Out of scope (separate issues)
eventmesh-storage-*implements the 3 universal capabilities — needs theeventmesh-architecture-guardmodule that [Architecture Review][P2] Gradle dependency guardrails (architectureCheck) #5305 added to develop. This PR stays on master (ba267195c); when this PR merges, a follow-up can carry the ArchUnit rule over.LiteTopicCapabledeprecation — the historical interface is still around and still used by callers; consolidating it withStorageCapabilities.LiteTopiccan be a follow-up.How a backend wires in
Closes #5303.