Skip to content

feat(storage): standardize storage SPI capabilities + TCK (issue #5303) - #5323

Merged
qqeasonchen merged 2 commits into
apache:developfrom
qqeasonchen:feat/5303-storage-spi
Sep 2, 2026
Merged

feat(storage): standardize storage SPI capabilities + TCK (issue #5303)#5323
qqeasonchen merged 2 commits into
apache:developfrom
qqeasonchen:feat/5303-storage-spi

Conversation

@qqeasonchen

Copy link
Copy Markdown
Contributor

What

Single PR that closes #5303 — standardizes storage SPI capabilities via a typed marker pattern, and ships a JUnit 5 TCK every backend must pass.

Why

The existing MeshStoragePlugin interface is rich but backend-specific behaviors (end-offset query, pull-cursor rewind, deferred POP ACK, lite topic) are scattered between MeshStoragePlugin default methods, the standalone LiteTopicCapable interface, and ad-hoc instanceof checks on concrete classes (e.g. createTopic). Callers have to know per-backend which features are supported, and there's no shared test base that catches an accidental removal of a capability override.

What changes

New (in eventmesh-storage-api)

  • StorageCapabilities — outer interface grouping 7 capability sub-interfaces
  • tck.MeshStoragePluginTCK — abstract JUnit 5 test base
  • InMemoryStoragePlugin (test scope) — fixture implementing all 7 capabilities
  • tck.MeshStoragePluginTCKSelfTest (test scope) — runs the TCK against the in-memory fixture

Modified

  • KafkaMeshStoragePlugin declares 5 capabilities (universal 3 + EndOffsetQuery + AlignPullOffset)
  • RocketMQRemotingStoragePlugin declares 4 (universal 3 + AlignPullOffset)
  • RocketMQ5RemotingStoragePlugin declares 5 (universal 3 + DeferredPopAck + LiteTopic)
  • 3 new *MeshStoragePluginTCKTest classes wire the TCK for each backend
  • 3 build.gradle files add junit-jupiter + useJUnitPlatform

Capability matrix

Capability Kafka RocketMQ 4.x RocketMQ 5.x
TopicManagement
PartitionAssignment
ExplicitOffsetCommit
EndOffsetQuery
AlignPullOffset
DeferredPopAck
LiteTopic

The 3 universal capabilities are part of the MeshStoragePlugin contract already; declaring them via StorageCapabilities is a self-audit so dropping a method body by accident fails the TCK, not a smoke test. The 4 backend-specific capabilities are new — they correspond to existing behavior that was previously undiscoverable.

Test results

40 tests across 4 modules, 0 failures:

eventmesh-storage-api:        10/10  (TCK self-test against InMemoryStoragePlugin)
eventmesh-storage-kafka:      10/10  (TCK wiring for KafkaMeshStoragePlugin)
eventmesh-storage-rocketmq:   10/10  (TCK wiring for RocketMQRemotingStoragePlugin)
eventmesh-storage-rocketmq5:  10/10  (TCK wiring for RocketMQ5RemotingStoragePlugin)

Out of scope (separate issues)

  • ArchUnit rule asserting every plugin in eventmesh-storage-* implements the 3 universal capabilities — needs the eventmesh-architecture-guard module that [Architecture Review][P2] Gradle dependency guardrails (architectureCheck) #5305 added to develop. This PR stays on master (ba267195c); when this PR merges, a follow-up can carry the ArchUnit rule over.
  • LiteTopicCapable deprecation — the historical interface is still around and still used by callers; consolidating it with StorageCapabilities.LiteTopic can be a follow-up.

How a backend wires in

class KafkaMeshStoragePluginTCKTest extends MeshStoragePluginTCK<KafkaMeshStoragePlugin> {
    @Override protected KafkaMeshStoragePlugin newPlugin() { return new KafkaMeshStoragePlugin(); }
    @Override protected Set<Class<?>> expectedCapabilities() {
        return Set.of(TopicManagement.class, PartitionAssignment.class,
                      ExplicitOffsetCommit.class, EndOffsetQuery.class,
                      AlignPullOffset.class);
    }
}

Closes #5303.

…he#5303)

Add a typed capability-marker pattern on top of MeshStoragePlugin so callers can
discover backend support for each optional feature (end-offset query, pull-cursor
rewind, deferred POP ACK, lite topic) via `instanceof`, and so the per-backend
TCK can assert that each declared capability is in fact implemented.

What ships in this PR (single merged PR per the apache#5296 plan):

  1. StorageCapabilities — outer interface in eventmesh-storage-api/.../storage/
     that groups 7 capability sub-interfaces (3 universal + 4 backend-specific).
     Each sub-interface `extends StorageCapabilities` so a single
     `Class.isAssignableFrom` check covers all of them.

     Universal (every backend):
       - TopicManagement        (createTopic)
       - PartitionAssignment    (assignPartitions)
       - ExplicitOffsetCommit   (commitOffset)

     Backend-specific:
       - EndOffsetQuery         (Kafka only)
       - AlignPullOffset        (Kafka + RocketMQ 4.x)
       - DeferredPopAck         (RocketMQ 5.x)
       - LiteTopic              (RocketMQ 5.x; mirrors the historical LiteTopicCapable)

  2. MeshStoragePluginTCK — abstract JUnit 5 test base in
     eventmesh-storage-api/.../storage/tck/. Subclasses provide a fresh plugin
     and the set of capabilities they declare. The TCK runs the universal
     lifecycle tests unconditionally and gates the capability-specific tests
     on the declared set. A capability-declaration assertion catches the case
     where a capability is removed from `implements` but the author forgot
     to remove it from the test.

  3. InMemoryStoragePlugin + MeshStoragePluginTCKSelfTest — in-memory
     fixture that implements ALL 7 capabilities. The TCK self-test runs the
     TCK against itself; if any TCK test fails against this plugin, the TCK
     is broken (too strict, wrong signature, or assumes a behavior the
     real backends can't provide).

  4. 3 plugin capability declarations:
       - KafkaMeshStoragePlugin         declares 5 (universal 3 + EndOffsetQuery + AlignPullOffset)
       - RocketMQRemotingStoragePlugin  declares 4 (universal 3 + AlignPullOffset)
       - RocketMQ5RemotingStoragePlugin declares 5 (universal 3 + DeferredPopAck + LiteTopic)

  5. 3 plugin TCK wirings + build.gradle updates (junit-jupiter + useJUnitPlatform
     on the 3 storage-plugin modules that didn't have it).

Capability matrix (the single source of truth for "what does each backend support"):

  Capability                  Kafka  RocketMQ 4.x  RocketMQ 5.x
  --------------------------  -----  ------------  ------------
  TopicManagement             yes    yes            yes
  PartitionAssignment         yes    yes            yes
  ExplicitOffsetCommit        yes    yes            yes
  EndOffsetQuery              yes    no             no
  AlignPullOffset             yes    yes            no
  DeferredPopAck              no     no             yes
  LiteTopic                   no     no             yes

Test results: 40 tests, 0 failures, 0 errors across the 4 modules
(eventmesh-storage-api self-test + 3 plugin TCK wirings).

Closes apache#5303.
…agePluginTCK

The CI build failed on `checkstyleMain` with 2 warnings:

  MeshStoragePluginTCK.java:35:1: Wrong order for 'java.util.Properties' import. [ImportOrder]
  MeshStoragePluginTCK.java:38:1: Wrong order for 'org.junit.jupiter.api.Assertions.assertDoesNotThrow' import. [ImportOrder]

The checkstyle ImportOrder rule in `style/checkStyle.xml` defines groups as:

  org.apache.eventmesh, org.apache, java, javax, org, io, net, junit, com, lombok

with `separatedStaticGroups=true` and `option=top` (static imports on top, in
the same group order).

My original import order was:
  1. org.apache.eventmesh.* and org.apache.eventmesh.common.wire.*
  2. org.junit.jupiter.api.*   ← should be AFTER java.*
  3. java.util.*               ← should be BEFORE org.*
  4. static org.junit.jupiter.api.Assertions.*

Reordered to:
  1. static org.junit.jupiter.api.Assertions.*  (top, 'org' group)
  2. org.apache.eventmesh.* and org.apache.*      ('org.apache' group)
  3. java.util.*                                  ('java' group, before org)
  4. org.junit.jupiter.api.*                      ('org' group, after java)

Verified locally:
  ./gradlew :eventmesh-storage-plugin:eventmesh-storage-api:checkstyleMain   PASS
  ./gradlew :eventmesh-storage-plugin:eventmesh-storage-api:check \
            :eventmesh-storage-plugin:eventmesh-storage-kafka:check \
            :eventmesh-storage-plugin:eventmesh-storage-rocketmq:check \
            :eventmesh-storage-plugin:eventmesh-storage-rocketmq5:check   PASS
@qqeasonchen

Copy link
Copy Markdown
Contributor Author

Force-pushed to fix checkstyle ImportOrder violations in MeshStoragePluginTCK.java (2 warnings on ). No code changes — only import group ordering. Verified locally: all 4 storage modules pass ./gradlew :check (compile + test + checkstyle + spotbugs). Awaiting CI re-run.

@qqeasonchen
qqeasonchen merged commit 054127e into apache:develop Sep 2, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Architecture Review][P1] Standardize Storage SPI capabilities + TCK

1 participant