test(proxy): harden transactional schema lifecycle coverage

[tobyhede]

Summary

• add property-based coverage that keeps schema and encryption overlays aligned across successful and failed DDL execution and rollback
• verify failed-transaction readiness does not publish schema state
• cover concurrent commit visibility, blocked-DDL cancellation, and client disconnection through database-backed tests
• share PostgreSQL cancellation routing across connections so cancellation requests reach the original upstream connection
• verify native temporary tables remain usable without blocking subsequent encrypted schema mapping

Validation

• focused schema lifecycle and backend adapter unit tests
• integration test suite compilation
• formatting and diff checks
• CI across PostgreSQL 14, 15, 16, and 17
• encrypted burn-in and performance regression checks

Acknowledgment

By submitting this pull request, I confirm that CipherStash can use, modify, copy, and redistribute this contribution, under the terms of CipherStash's choice.

[freshtonic]

Review — PR #461, test(proxy): harden transactional schema lifecycle coverage

Scope: origin/main...c74d9900 (3 commits, 6 files, +402/−5). The test additions are well-constructed and the cancellation fix is directionally correct — cancellation through Proxy was genuinely broken before, because each connection built its own registry and a CancelRequest arrives on a fresh connection that could never resolve the key. One finding blocks the production change.

1. Blocking — shared cancellation-registry entries are never detached

packages/cipherstash-proxy/src/postgresql/driver.rs:24-25 and :88

The static registry makes every route entry process-global, but nothing removes entries. In pg-proto 0.11.1, detach_cancellation is called only inside accept error paths and teardown(), and there is no Drop-based cleanup of the registry. The driver never calls teardown() or detach_cancellation(): the session loop breaks on Terminate (driver.rs:153-154, :207) or returns an error, and the session is simply dropped.

Every session registers a route during accept (CancellationPolicy::Forward plus BackendKeyData), so with this change each client connection leaks one CancelKey → CancellationRoute entry for the life of the process. Before this change the map died with the per-connection registry, which is also why cancellation never worked.

Consequences:

• Unbounded growth proportional to total connections served. A long-running Proxy in front of a connection-churning workload accumulates entries indefinitely.
• A stale entry keeps forwarding cancellations for a dead backend. Harmless to PostgreSQL, but each stale request opens an upstream connection.
• register returns DuplicateKey on collision, and accept then refuses the new connection (intermediary_component.rs:2180-2199). PostgreSQL recycles pids, so a stale entry that matches a new backend's (pid, secret) pair blocks that backend's clients permanently — the stale entry never leaves until Proxy restarts. The probability per instance is small, but the failure is persistent when it lands.

Fix: detach on every session exit. session.detach_cancellation() is public; run the session loop to a result, detach, then propagate — covering the Terminate break, the capacity-retry break, the timeout returns, and the error returns. Prefer detach_cancellation() over teardown(), which asserts backend_hold is empty and would panic on an abnormal exit.

2. CHANGELOG entry for the cancellation fix

Commit 2cef94b5 is a user-facing behavior fix: query cancellation through Proxy previously never reached the upstream connection. Add a ### Fixed entry under [Unreleased].

Observations (no change requested)

• disconnecting_during_blocked_ddl_does_not_cancel_the_upstream_schema_change tacitly documents a real gap: the orphaned ALTER commits upstream with no publication until the periodic reload, and the test deliberately touches only the native column afterwards. That window predates this PR, but it deserves a tracking issue if none exists.

Checked and cleared

• InMemoryCancellationRegistry clones share Arc<Mutex<HashMap>> state, so the static + .clone() pattern does share one registry. Verified against the pg-proto 0.11.1 source.
• The property test's retention of overlay tables across ready_for_query(FailedTransaction) matches the middleware: a failed transaction still holds its uncommitted DDL until ROLLBACK, and only ROLLBACK clears the overlay.
• The barrier-raced concurrent-commit test is deterministic: each COMMIT returns only after its own publication, and the later catalog read begins after both database commits, so the observer's adopted snapshot contains both tables.
• cancelling_blocked_ddl_does_not_activate_a_schema_change is the regression that requires the shared registry, and its post-cancel assertions (execution_failed discards the intent; the connection keeps mapping) match the middleware contract.

Requested changes addressed

[freshtonic]

The follow-up commit resolves my previous blocking feedback: the shared cancellation route is now detached after every session-loop result, including normal termination and error paths, and the cancellation fix has an Unreleased changelog entry.

I verified the shared registry and cleanup behavior, the transaction-aware schema lifecycle coverage, formatting, and Clippy. Focused tests pass, and the PostgreSQL 14–17, encrypted burn-in, and performance checks are green.

Non-blocking documentation note: because packages/cipherstash-proxy/CONTEXT.md is the source of truth for connection architecture, consider recording that cancellation routing is process-global and that each route is detached when its client connection exits.