Repository navigation
PBS-54 feature: Support for encrypted storages in COM_BINLOG_DUMP - #196
Merged
Merged
Conversation
https://perconadev.atlassian.net/browse/PBS-54 Problem: storage_core::fetch_event_block() served the bytes returned by the storage backend verbatim, ignoring the per-binlog encryption metadata recorded by the write side. Downstream replicas connected via COM_BINLOG_DUMP against an encryption-enabled storage therefore received the on-disk ciphertext instead of a plaintext binlog stream. Solution: Decrypt the fetched block on the read path with the same two-level key hierarchy the write path already uses: the per-file data key is unwrapped with the KEK from the keyring, then the data cipher is applied via cipher_context::create_with_offset() so the CTR counter aligns with the block's actual offset in the file. The decrypt happens in place: cipher_context::update() is invoked with a single span aliased as both input and output. OpenSSL's EVP_EncryptInit(3) guarantees in-place operation when every prior update has processed a multiple of the cipher block size, a precondition the wrapper already enforces on every call (and which is trivially satisfied by the streaming CTR / GCM modes whose block size is 1). Doing so eliminates the per-fetch second buffer allocation and move that a copy-and-swap approach would need. The KEK-lookup and file-key-unwrap logic that used to live inline in storage_core::write_data_to_stream() is extracted into a small storage_core::decrypt_file_key() helper so both paths share exactly one copy of a security-critical routine that must not drift between the encrypt and decrypt sides. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…torage https://perconadev.atlassian.net/browse/PBS-54 * Added new 'binlog_streaming.rs_roundtrip_encryption' MTR test case which checks that data collected by PBS into encrypted storage and then extracted from PBS via 'mysqlbinlog --read-from-remote-server' is identical to what was originally added to MySQL Server. Exercises the decrypt path added to 'binsrv::storage_core::fetch_event_block()' by PBS-54 end-to-end: without the fix the dumped stream is the raw on-disk ciphertext and the restore step fails. * Extracted the shared roundtrip flow out of the existing 'binlog_streaming.rs_roundtrip' test into a new 'rs_roundtrip_body.inc' include so that the plain and encrypted variants differ only in their setup / teardown. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
kamil-holubicki
force-pushed
the
PBS-54
branch
from
October 1, 2026 08:46
099ab87 to
a9af942
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://perconadev.atlassian.net/browse/PBS-54
Problem:
storage_core::fetch_event_block() served the bytes returned by the storage backend verbatim, ignoring the per-binlog encryption metadata recorded by the write side. Downstream replicas connected via COM_BINLOG_DUMP against an encryption-enabled storage therefore received the on-disk ciphertext instead of a plaintext binlog stream.
Solution:
Decrypt the fetched block on the read path with the same two-level key hierarchy the write path already uses: the per-file data key is unwrapped with the KEK from the keyring, then the data cipher is applied via cipher_context::create_with_offset() so the CTR counter aligns with the block's actual offset in the file.
The decrypt happens in place: cipher_context::update() is invoked with a single span aliased as both input and output. OpenSSL's EVP_EncryptInit(3) guarantees in-place operation when every prior update has processed a multiple of the cipher block size, a precondition the wrapper already enforces on every call (and which is trivially satisfied by the streaming CTR / GCM modes whose block size is 1). Doing so eliminates the per-fetch second buffer allocation and move that a copy-and-swap approach would need.
The KEK-lookup and file-key-unwrap logic that used to live inline in storage_core::write_data_to_stream() is extracted into a small storage_core::decrypt_file_key() helper so both paths share exactly one copy of a security-critical routine that must not drift between the encrypt and decrypt sides.