Skip to content

fix(pyo3-bytes): don't panic on b"" in Bytes or a huge Bytes * n - #788

Merged
kylebarron merged 4 commits into
mainfrom
kyle/fix-bytes-panics
Sep 29, 2026
Merged

kylebarron merged 4 commits into
mainfrom
kyle/fix-bytes-panics

Conversation

@kylebarron

@kylebarron kylebarron commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Note

This PR was written by Claude (Claude Code), not by @kylebarron.

Makes in and * on obstore.Bytes behave like bytes instead of raising PanicException or aborting the interpreter.

Closes #786

Changes

  • __contains__: an empty needle now returns True. Before, it went to slice::windows(0), which panics even for an empty haystack.
  • __mul__: now raises OverflowError("repeated bytes are too long") when len * n overflows or exceeds isize::MAX, and MemoryError when the allocation fails (via Vec::try_reserve_exact). Before, it passed len * n straight to BytesMut::with_capacity, which panics past isize::MAX and aborts the process on allocation failure.
  • __mul__ on an empty Bytes: now returns immediately. Found while testing, not in the issue: Bytes(b"") * (1 << 62) looped n times appending an empty slice, so it hung, where bytes returns b"" right away.
  • Fixes the Bytes.__mul__ type hint, which claimed (other: Buffer) -> int. It now takes an int and returns Bytes. Pyright rejected the new tests without this.
  • Bumps pyo3-bytes to 0.7.2, with a changelog entry.
expression bytes Bytes before Bytes after
b"" in B(b"abc") True PanicException True
B(b"abc") * (1 << 62) OverflowError PanicException: capacity overflow OverflowError
B(b"a") * (1 << 62) MemoryError process abort (exit 134) MemoryError
B(b"") * (1 << 62) b"" hangs empty Bytes

Temporary path dependency

obstore depends on pyo3-bytes from crates.io, so the Python tests would otherwise keep exercising the published 0.7.1. The first commit points obstore at ../pyo3-bytes so CI tests this fix. Once this merges and pyo3-bytes-v0.7.2 is tagged and published, obstore can go back to the crates.io release.

Tests

New tests in tests/test_bytes.py: in compared against bytes for empty and non-empty needles and haystacks, ordinary repetition compared against bytes, and one test each for the OverflowError, MemoryError, and empty-Bytes cases. Against the unfixed code, the empty-needle cases fail with PanicException, the overflow test fails with PanicException, the MemoryError test aborts pytest, and the empty-Bytes test hangs.

Locally (macOS arm64): the full suite passes on 3.14 and on 3.14t with --parallel-threads=2 (445 passed, 2 xfailed). cargo fmt, cargo clippy -D warnings (root and pyo3-bytes), cargo test --all, pyright 1.1.406, and pre-commit all pass.

🤖 Written by Claude Code

kylebarron and others added 2 commits September 29, 2026 18:42
Temporary, so the Python tests exercise the pyo3-bytes fix for #786
before pyo3-bytes 0.7.2 is published. Switch back to the crates.io
release once it is out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`__contains__` passed the needle length to `slice::windows`, which
panics for a size of 0. Like `bytes`, the empty needle is now in every
value.

`__mul__` passed `len * n` straight to `BytesMut::with_capacity`, which
panics past `isize::MAX` and aborts the process when the allocation
fails. It now raises `OverflowError` for a result that is too long and
`MemoryError` when the allocation fails, matching `bytes`. An empty
`Bytes` times a huge `n` also looped `n` times; it now returns at once.

Bump pyo3-bytes to 0.7.2 so the fix can be published.

Closes #786

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ds-release-bot ds-release-bot Bot added the fix label Sep 29, 2026
kylebarron and others added 2 commits September 29, 2026 18:57
The stub declared `__mul__(self, other: Buffer) -> int`, but the method
takes an `int` count and returns `Bytes`. Pyright rejected the new
repetition tests because of it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kylebarron
kylebarron enabled auto-merge (squash) September 29, 2026 23:26
@kylebarron
kylebarron merged commit 42ec7a4 into main Sep 29, 2026
9 checks passed
@kylebarron
kylebarron deleted the kyle/fix-bytes-panics branch September 29, 2026 23:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pyo3-bytes: b"" in Bytes panics and Bytes * n can abort Python

1 participant