fix(pyo3-bytes): don't panic on b"" in Bytes or a huge Bytes * n - #788
Merged
Merged
Conversation
Temporary, so the Python tests exercise the pyo3-bytes fix for #786 before pyo3-bytes 0.7.2 is published. Switch back to the crates.io release once it is out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`__contains__` passed the needle length to `slice::windows`, which panics for a size of 0. Like `bytes`, the empty needle is now in every value. `__mul__` passed `len * n` straight to `BytesMut::with_capacity`, which panics past `isize::MAX` and aborts the process when the allocation fails. It now raises `OverflowError` for a result that is too long and `MemoryError` when the allocation fails, matching `bytes`. An empty `Bytes` times a huge `n` also looped `n` times; it now returns at once. Bump pyo3-bytes to 0.7.2 so the fix can be published. Closes #786 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stub declared `__mul__(self, other: Buffer) -> int`, but the method takes an `int` count and returns `Bytes`. Pyright rejected the new repetition tests because of it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
This PR was written by Claude (Claude Code), not by @kylebarron.
Makes
inand*onobstore.Bytesbehave likebytesinstead of raisingPanicExceptionor aborting the interpreter.Closes #786
Changes
__contains__: an empty needle now returnsTrue. Before, it went toslice::windows(0), which panics even for an empty haystack.__mul__: now raisesOverflowError("repeated bytes are too long")whenlen * noverflows or exceedsisize::MAX, andMemoryErrorwhen the allocation fails (viaVec::try_reserve_exact). Before, it passedlen * nstraight toBytesMut::with_capacity, which panics pastisize::MAXand aborts the process on allocation failure.__mul__on an emptyBytes: now returns immediately. Found while testing, not in the issue:Bytes(b"") * (1 << 62)loopedntimes appending an empty slice, so it hung, wherebytesreturnsb""right away.Bytes.__mul__type hint, which claimed(other: Buffer) -> int. It now takes anintand returnsBytes. Pyright rejected the new tests without this.bytesBytesbeforeBytesafterb"" in B(b"abc")TruePanicExceptionTrueB(b"abc") * (1 << 62)OverflowErrorPanicException: capacity overflowOverflowErrorB(b"a") * (1 << 62)MemoryErrorMemoryErrorB(b"") * (1 << 62)b""BytesTemporary path dependency
obstore depends on pyo3-bytes from crates.io, so the Python tests would otherwise keep exercising the published 0.7.1. The first commit points obstore at
../pyo3-bytesso CI tests this fix. Once this merges andpyo3-bytes-v0.7.2is tagged and published, obstore can go back to the crates.io release.Tests
New tests in
tests/test_bytes.py:incompared againstbytesfor empty and non-empty needles and haystacks, ordinary repetition compared againstbytes, and one test each for theOverflowError,MemoryError, and empty-Bytescases. Against the unfixed code, the empty-needle cases fail withPanicException, the overflow test fails withPanicException, theMemoryErrortest aborts pytest, and the empty-Bytestest hangs.Locally (macOS arm64): the full suite passes on 3.14 and on 3.14t with
--parallel-threads=2(445 passed, 2 xfailed).cargo fmt,cargo clippy -D warnings(root and pyo3-bytes),cargo test --all, pyright 1.1.406, and pre-commit all pass.🤖 Written by Claude Code