Skip to content

Respect split stream events when building decision models - #925

Closed
DavidBadura wants to merge 1 commit into
append-store-query-fromfrom
decision-model-split-stream
Closed

DavidBadura wants to merge 1 commit into
append-store-query-fromfrom
decision-model-split-stream

Conversation

@DavidBadura

Copy link
Copy Markdown
Member

Builds on #924, the base will switch to 4.0.x once that is merged.

An event marked with #[SplitStream] holds the whole state that is relevant from that point on. Aggregates use that to skip older events, but in DCB it had no effect at all: StoreEventAppender doesn't run message decorators, append() doesn't archive and query() ignores the archived flag. Archiving wouldn't be the right tool here anyway. It works per stream, while an event in DCB often belongs to several boundaries (e.g. account:1 and customer:7), so archiving it for one projection would hide it from the others.

So the cut happens per projection while building the decision model, the store doesn't need to know anything about split events:

  1. BasicProjection::splitEvents() returns the event classes of its apply methods that have #[SplitStream].
  2. The builder looks up the last of these events per projection, with the projection's own tags and stream, in a single onlyLastEvent query.
  3. It reads the normal query from the smallest of these positions via query($query, $from) and every projection only gets messages from its own split event onwards.

The append condition doesn't change. Events before the split are older than after anyway, and anything appended concurrently is newer. If one projection in the decision model has no split events, the read starts at 0, so mixed decision models are correct but don't save anything yet. A projection that handles a split event but needs the whole history, e.g. to count them, can override splitEvents() and return [].

The same applies to StoreProjectionBuilder.

An event marked with #[SplitStream] holds the whole state that is relevant
from that point on. Aggregates already use it to skip older events, but
decision models always read the complete history of their projections.

Before reading, the builders now look up the last split event of every
projection that handles one, using the projection's own tags, and read from
there. Projections only get events from their own split event onwards, so a
projection without split events in the same decision model still gets its
whole history. Nothing is archived, the append condition stays the same.
A projection can opt out by overriding splitEvents().
@DavidBadura
DavidBadura marked this pull request as draft October 3, 2026 19:33
@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown

Hello 👋

here is the most recent benchmark result:

SplitStreamBench
================

+-------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
|                         | time (kde mode)                                     | memory                                     |
+-------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| subject                 | Tag: <current>     | Tag: base          | time-diff | Tag: <current>  | Tag: base  | memory-diff |
+-------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| benchLoad10000Events () | 2.804ms (±0.00%)   | 3.074ms (±0.00%)   | -8.79%    | 35.774mb        | 36.295mb   | -1.43%      |
| benchSave10000Events () | 472.040ms (±0.00%) | 469.324ms (±0.00%) | +0.58%    | 35.909mb        | 36.430mb   | -1.43%      |
+-------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+

SimpleSetupTaggableStoreBench
=============================

+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
|                                        | time (kde mode)                                     | memory                                     |
+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| subject                                | Tag: <current>     | Tag: base          | time-diff | Tag: <current>  | Tag: base  | memory-diff |
+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| benchLoad1Event ()                     | 1.131ms (±0.00%)   | 1.109ms (±0.00%)   | +1.98%    | 36.312mb        | 36.312mb   | 0.00%       |
| benchLoad10000Events ()                | 73.639ms (±0.00%)  | 73.973ms (±0.00%)  | -0.45%    | 36.312mb        | 36.312mb   | 0.00%       |
| benchSave1Event ()                     | 1.635ms (±0.00%)   | 1.749ms (±0.00%)   | -6.52%    | 36.312mb        | 36.312mb   | 0.00%       |
| benchSave10000Events ()                | 324.784ms (±0.00%) | 313.958ms (±0.00%) | +3.45%    | 36.312mb        | 36.312mb   | 0.00%       |
| benchSave10000Aggregates ()            | 8.496s (±0.00%)    | 8.497s (±0.00%)    | -0.01%    | 36.312mb        | 36.312mb   | 0.00%       |
| benchSave10000AggregatesTransaction () | 5.970s (±0.00%)    | 5.975s (±0.00%)    | -0.09%    | 36.312mb        | 36.312mb   | 0.00%       |
| benchAppend1Event ()                   | 1.570ms (±0.00%)   | 1.577ms (±0.00%)   | -0.40%    | 36.312mb        | 36.312mb   | 0.00%       |
| benchAppend100Events ()                | 8.002ms (±0.00%)   | 7.979ms (±0.00%)   | +0.28%    | 36.312mb        | 36.312mb   | 0.00%       |
+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+

SubscriptionEngineBench
=======================

+---------------------------+-----------------+-----------------+-----------+-----------------+------------+-------------+
|                           | time (kde mode)                               | memory                                     |
+---------------------------+-----------------+-----------------+-----------+-----------------+------------+-------------+
| subject                   | Tag: <current>  | Tag: base       | time-diff | Tag: <current>  | Tag: base  | memory-diff |
+---------------------------+-----------------+-----------------+-----------+-----------------+------------+-------------+
| benchHandle10000Events () | 3.293s (±0.00%) | 3.252s (±0.00%) | +1.26%    | 48.301mb        | 48.301mb   | 0.00%       |
+---------------------------+-----------------+-----------------+-----------+-----------------+------------+-------------+

NoopSubscriptionEngineBench
===========================

+---------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
|                           | time (kde mode)                                     | memory                                     |
+---------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| subject                   | Tag: <current>     | Tag: base          | time-diff | Tag: <current>  | Tag: base  | memory-diff |
+---------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| benchHandle10000Events () | 113.038ms (±0.00%) | 102.662ms (±0.00%) | +10.11%   | 48.301mb        | 48.301mb   | 0.00%       |
+---------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+

SimpleSetupStreamStoreBench
===========================

+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
|                                        | time (kde mode)                                     | memory                                     |
+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| subject                                | Tag: <current>     | Tag: base          | time-diff | Tag: <current>  | Tag: base  | memory-diff |
+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| benchLoad1Event ()                     | 1.328ms (±0.00%)   | 1.350ms (±0.00%)   | -1.66%    | 35.146mb        | 35.146mb   | 0.00%       |
| benchLoad10000Events ()                | 72.064ms (±0.00%)  | 71.558ms (±0.00%)  | +0.71%    | 35.146mb        | 35.146mb   | 0.00%       |
| benchSave1Event ()                     | 1.521ms (±0.00%)   | 1.508ms (±0.00%)   | +0.86%    | 35.146mb        | 35.146mb   | 0.00%       |
| benchSave10000Events ()                | 291.034ms (±0.00%) | 286.495ms (±0.00%) | +1.58%    | 35.146mb        | 35.146mb   | 0.00%       |
| benchSave10000Aggregates ()            | 8.421s (±0.00%)    | 8.446s (±0.00%)    | -0.29%    | 35.147mb        | 35.147mb   | 0.00%       |
| benchSave10000AggregatesTransaction () | 5.921s (±0.00%)    | 5.926s (±0.00%)    | -0.09%    | 35.147mb        | 35.147mb   | 0.00%       |
+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+

SubscriptionEngineBatchBench
============================

+---------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
|                           | time (kde mode)                                     | memory                                     |
+---------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| subject                   | Tag: <current>     | Tag: base          | time-diff | Tag: <current>  | Tag: base  | memory-diff |
+---------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| benchHandle10000Events () | 107.638ms (±0.00%) | 106.321ms (±0.00%) | +1.24%    | 35.726mb        | 35.726mb   | 0.00%       |
+---------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+

PersonalDataBench
=================

+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
|                                        | time (kde mode)                                     | memory                                     |
+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| subject                                | Tag: <current>     | Tag: base          | time-diff | Tag: <current>  | Tag: base  | memory-diff |
+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| benchLoad1Event ()                     | 927.300μs (±0.00%) | 906.900μs (±0.00%) | +2.25%    | 37.008mb        | 37.008mb   | 0.00%       |
| benchLoad10000Events ()                | 172.518ms (±0.00%) | 175.419ms (±0.00%) | -1.65%    | 37.008mb        | 37.008mb   | 0.00%       |
| benchSave1Event ()                     | 2.280ms (±0.00%)   | 2.193ms (±0.00%)   | +3.99%    | 37.008mb        | 37.008mb   | 0.00%       |
| benchSave10000Events ()                | 424.981ms (±0.00%) | 415.593ms (±0.00%) | +2.26%    | 37.063mb        | 37.063mb   | 0.00%       |
| benchSave10000Aggregates ()            | 13.417s (±0.00%)   | 13.690s (±0.00%)   | -2.00%    | 37.008mb        | 37.008mb   | -0.00%      |
| benchSave10000AggregatesTransaction () | 10.989s (±0.00%)   | 10.919s (±0.00%)   | +0.64%    | 37.008mb        | 37.008mb   | 0.00%       |
+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+

SnapshotsBench
==============

+----------------------------------------+-------------------+-------------------+-----------+-----------------+------------+-------------+
|                                        | time (kde mode)                                   | memory                                     |
+----------------------------------------+-------------------+-------------------+-----------+-----------------+------------+-------------+
| subject                                | Tag: <current>    | Tag: base         | time-diff | Tag: <current>  | Tag: base  | memory-diff |
+----------------------------------------+-------------------+-------------------+-----------+-----------------+------------+-------------+
| benchLoad10000EventsMissingSnapshot () | 72.199ms (±0.00%) | 72.622ms (±0.00%) | -0.58%    | 35.151mb        | 35.151mb   | 0.00%       |
| benchLoad10000Events ()                | 1.130ms (±0.00%)  | 1.105ms (±0.00%)  | +2.27%    | 35.151mb        | 35.151mb   | 0.00%       |
+----------------------------------------+-------------------+-------------------+-----------+-----------------+------------+-------------+

CommandToQueryBench
===================

+----------------+------------------+------------------+-----------+-----------------+------------+-------------+
|                | time (kde mode)                                 | memory                                     |
+----------------+------------------+------------------+-----------+-----------------+------------+-------------+
| subject        | Tag: <current>   | Tag: base        | time-diff | Tag: <current>  | Tag: base  | memory-diff |
+----------------+------------------+------------------+-----------+-----------------+------------+-------------+
| benchCreate () | 3.800ms (±0.00%) | 3.843ms (±0.00%) | -1.12%    | 5.057mb         | 5.057mb    | 0.00%       |
| benchUpdate () | 5.433ms (±0.00%) | 5.279ms (±0.00%) | +2.92%    | 5.067mb         | 5.067mb    | 0.00%       |
| benchBoth ()   | 8.865ms (±0.00%) | 8.735ms (±0.00%) | +1.49%    | 5.389mb         | 5.389mb    | 0.00%       |
+----------------+------------------+------------------+-----------+-----------------+------------+-------------+

This comment gets update everytime a new commit comes in!

@DavidBadura
DavidBadura deleted the branch append-store-query-from October 3, 2026 20:21
@DavidBadura DavidBadura closed this Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant