Conversation
Create the event log table to store each event's sequence number, kind, payload, transaction data, and tip hash. Add `applyEventTx` to run an event's database changes and append its log entry in one transaction. Both are committed together, and a mismatch with a supplied sequence number or expected tip hash rolls back the changes. This helper is for use by event appliers. Add `EventLogFrontier` and `EventLogEntriesAfter` so the mesh package can query the event log table.
Define which tables are included in snapshots, which must be empty before loading a snapshot, and which are preserved as local configuration. Check at startup that every table in `public` and the stored market schemas has a classification to avoid future developers adding tables without considering how snapshotting will be affected. Add `HasNoEventSourcedState` to check whether all event-sourced data is absent, as required before loading a snapshot. Add `WipeEventSourcedState` to clear all event-sourced data before restoring from a peer.
Add `WriteSnapshot`, used by the master to create a snapshot that is sent to the slave, and also add `LoadSnapshot`, used by the slave to populate its DB with the master's data. The snapshot includes: - The event-log sequence number and tip hash corresponding to the snapshot. - All accounts, bonds, prepaid bonds, and reputation points. For configured markets, it also includes: - Market lifecycle state. - Active orders, cancels, and matches. - The past 30 days of archived orders, archived cancels, and matches, plus older archived orders referenced by included matches or archived cancels. - The latest 1,000 candles from each candle table. - Recent epoch reports needed to rebuild unfinished candles.
Add the v9 upgrade and update table schemas. The upgrade makes the following changes: - Create the `event_log` table. - Create the `market_lifecycle` table to store market state, scheduled suspensions and resumptions, epoch progress, and trading parameters. - Remove the legacy `accounts.fee_asset` column. - Remove commitment and preimage uniqueness constraints from `orders_archived` and `cancels_archived`. Nodes may retain different archived histories, so these constraints could make archiving the same order succeed on one node and fail on another. - Add non-unique commitment indexes to `orders_archived` and `cancels_archived` for lookups. - Add indexes on `makerOrder` and `takerOrder` for active matches, to support finding an order's active matches during future swap processing. - Add a genesis event when a database has existing event-sourced data and an empty event log. It includes a random nonce, so each legacy database initialized this way has a distinct history. Databases with no event-sourced data do not receive a genesis event.
martonp
force-pushed
the
mesh-db-foundation
branch
3 times, most recently
from
September 22, 2026 06:40
cee820a to
e391aa3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR builds on #3627 and adds PostgreSQL support for mesh replication, including the event log, snapshot export and restore, and the v9 schema upgrade.
server/db/pg: Add event log with atomic state updates
Create the event log table to store each event's sequence number, kind,
payload, transaction data, and tip hash.
Add
applyEventTxto run an event's database changes and append its logentry in one transaction. Both are committed together, and a mismatch
with a supplied sequence number or expected tip hash rolls back the
changes. This helper is for use by event appliers.
Add
EventLogFrontierandEventLogEntriesAfterso the mesh package canquery the event log table.
server/db/pg: Classify tables and support clearing event-sourced state
Define which tables are included in snapshots, which must be empty
before loading a snapshot, and which are preserved as local configuration.
Check at startup that every table in
publicand the stored market schemashas a classification to avoid future developers adding tables without
considering how snapshotting will be affected.
Add
HasNoEventSourcedStateto check whether all event-sourced data isabsent, as required before loading a snapshot. Add
WipeEventSourcedStateto clear all event-sourced data before restoring from a peer.
server/db/pg: Add snapshot export and restore
Add
WriteSnapshot, used by the master to create a snapshot that issent to the slave, and also add
LoadSnapshot, used by the slave topopulate its DB with the master's data.
The snapshot includes:
For configured markets, it also includes:
older archived orders referenced by included matches or archived cancels.
server/db/pg: Add v9 schema upgrade for mesh replication
Add the v9 upgrade and update table schemas. The upgrade makes the
following changes:
event_logtable.market_lifecycletable to store market state, scheduledsuspensions and resumptions, epoch progress, and trading parameters.
accounts.fee_assetcolumn.orders_archivedandcancels_archived. Nodes may retain differentarchived histories, so these constraints could make archiving the same
order succeed on one node and fail on another.
orders_archivedandcancels_archivedfor lookups.makerOrderandtakerOrderfor active matches, tosupport finding an order's active matches during future swap processing.
an empty event log. It includes a random nonce, so each legacy database
initialized this way has a distinct history. Databases with no
event-sourced data do not receive a genesis event.