Skip to content

Loosen regex pin to allow calendar versions past 2024 - #157

Open
eljero66 wants to merge 1 commit into
vmenger:mainfrom
eljero66:fix/regex-calendar-version-pin
Open

eljero66 wants to merge 1 commit into
vmenger:mainfrom
eljero66:fix/regex-calendar-version-pin

Conversation

@eljero66

@eljero66 eljero66 commented Sep 8, 2026

Copy link
Copy Markdown

Problem

regex uses date-based (calendar) versioning, not semver. The current constraint

regex = "^2024.11.6"

expands (via Poetry's caret rule) to >=2024.11.6,<2025.0.0, which locks out every 2025+ release of regex — treating the calendar year 2024 as if it were a semver major. Those newer releases are not breaking changes; they're routine bugfix/feature releases under the library's date-tag scheme.

This blocks any environment that needs a newer regex alongside deduce — for example co-installing transformers>=5.5.0, which requires regex>=2025. Today the two are mutually exclusive purely because of this caret.

Why it's safe

deduce uses the regex library in exactly one place:

# deduce/tokenizer.py
_TOKENIZER_PATTERN = regex.compile(r"\w+|[\n\r\t]| {4,}|[^ ]", flags=regex.I | regex.M)

That's a single regex.compile using \w+ and the I/M flags — bedrock API that has been stable for years. (The BSN and phone annotators use the stdlib re, not this library.)

Change

Turn the cap into a floor:

-regex = "^2024.11.6"
+regex = ">=2024.11.6"

poetry.lock regenerated to match (still resolves regex to 2024.11.6 by default — the change only permits newer versions, it doesn't force them).

Verification

With regex 2026.9.3 installed (≈2 years past the old cap):

  • Full test suite: 122 passed, 0 failures.
  • End-to-end smoke test unaffected:
    in : Jan de Vries woont in Amsterdam en belt vaak naar 0612345678.
    out: [PERSOON-1] woont in [LOCATIE-1] en belt vaak naar [ID-1].
    

Happy to adjust the constraint form (e.g. an explicit upper bound at a future known-good version) if you prefer a different policy for this dep.

`regex` uses date-based (calendar) versioning, so the caret constraint
`^2024.11.6` resolves to `>=2024.11.6,<2025.0.0` and locks out every
2025+ release — even though those are not semver-major breaks. deduce
uses `regex` in exactly one place (tokenizer.py: a single
`regex.compile` with `\w+`, `regex.I`, `regex.M`), a bedrock API that is
stable across these releases; the BSN/phone annotators use the stdlib
`re`, not this library.

Change the constraint to a floor (`>=2024.11.6`). Full test suite (122
tests) passes with regex 2026.9.3 installed, and end-to-end
de-identification is unaffected.

This also unblocks downstream environments that need a newer `regex`
(e.g. co-installed with transformers>=5.5.0, which requires
regex>=2025).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants