Skip to content

ci: add weekly audit for uv override-dependencies#1887

Open
thomasdhc wants to merge 6 commits intomainfrom
donghyukc/override_audit
Open

ci: add weekly audit for uv override-dependencies#1887
thomasdhc wants to merge 6 commits intomainfrom
donghyukc/override_audit

Conversation

@thomasdhc
Copy link
Copy Markdown
Contributor

@thomasdhc thomasdhc commented Apr 28, 2026

Description

  • Adds a weekly CI job that audits each entry in [tool.uv] override-dependencies for staleness, by removing it from pyproject.toml and re-running uv lock.
  • Catches overrides that became no-ops after upstream releases -- currently there's no signal until something breaks.

Usage

uv run --with packaging --no-project python .github/scripts/audit_overrides.py

Output

## Stale (safe to remove)

- `apex; sys_platform == 'never'` -- apex is not in the lock without the override
- `distance; sys_platform == 'never'` -- distance is not in the lock without the override
- `kaldiio;  sys_platform == 'never'` -- kaldiio is not in the lock without the override
- `levenshtein;  sys_platform == 'never'` -- levenshtein is not in the lock without the override
- `protobuf>=5.29.5,<7.0` -- resolves to protobuf==5.29.6, already satisfies <7.0,>=5.29.5; inverse range is unsatisfiable -- truly redundant
- `nixl-cu12>=0.10.0; (platform_machine == 'x86_64' and platform_system != 'Darwin')` -- resolves to nixl-cu12==1.0.1, already satisfies >=0.10.0; inverse range is unsatisfiable -- truly redundant

## Defensive (redundant today, protects against upstream regression)

- `setuptools>=80.10.1` -- natural resolution satisfies the override, but `setuptools<80.10.1` is also resolvable -- override is protective
- `torchvision>=0.23.0` -- natural resolution satisfies the override, but `torchvision<0.23.0` is also resolvable -- override is protective
- `torchcodec>=0.9.0` -- natural resolution satisfies the override, but `torchcodec<0.9.0` is also resolvable -- override is protective

## Shaping (review -- override actively constrains resolution)

- `numpy>=2.0.0,<=2.2.0` -- resolves to numpy==2.2.6 without override (override forces <=2.2.0,>=2.0.0)
- `xgrammar>=0.1.32` -- resolves to xgrammar==0.1.29 without override (override forces >=0.1.32)

## Load-bearing (keep)

- `fsspec>=2025.3.0` -- removing the override breaks resolution
- `huggingface-hub>=0.34,<1.0` -- removing the override breaks resolution
- `torch>=2.8.0` -- removing the override breaks resolution
- `torchaudio>=2.8.0` -- removing the override breaks resolution

Checklist

  • I am familiar with the Contributing Guide.
  • New or Existing tests cover these changes.
  • The documentation is up to date with these changes.

Signed-off-by: Dong Hyuk Chang <9426164+thomasdhc@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@thomasdhc thomasdhc requested a review from a team as a code owner April 28, 2026 17:39
Signed-off-by: Dong Hyuk Chang <9426164+thomasdhc@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comment on lines +75 to +86
def remove_override_line(pyproject: Path, spec: str) -> None:
"""Strip the single line containing the given override spec."""
text = pyproject.read_text()
pattern = re.compile(
r'^[ \t]*"' + re.escape(spec) + r'"[ \t]*,?[ \t]*(#.*)?\n',
re.MULTILINE,
)
new_text, count = pattern.subn("", text, count=1)
if count != 1:
msg = f"Could not locate override line for {spec!r} in {pyproject}"
raise RuntimeError(msg)
pyproject.write_text(new_text)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 remove_override_line silently misclassifies single-quoted specs

TOML allows both "double-quoted" and 'single-quoted' string literals. tomllib returns the same Python string either way, but the regex only anchors on ". Any spec written with single quotes in pyproject.toml (e.g. 'packaging>=21.0') will fail to match, raise a RuntimeError, get caught by the broad except Exception in main, and appear in the report as "error: audit failed: Could not locate override line …" rather than being correctly audited. Fix by matching either quote style:

pattern = re.compile(
    r'^[ \t]*["\']' + re.escape(spec) + r'["\'][ \t]*,?[ \t]*(#.*)?\n',
    re.MULTILINE,
)

Comment on lines +75 to +86
def remove_override_line(pyproject: Path, spec: str) -> None:
"""Strip the single line containing the given override spec."""
text = pyproject.read_text()
pattern = re.compile(
r'^[ \t]*"' + re.escape(spec) + r'"[ \t]*,?[ \t]*(#.*)?\n',
re.MULTILINE,
)
new_text, count = pattern.subn("", text, count=1)
if count != 1:
msg = f"Could not locate override line for {spec!r} in {pyproject}"
raise RuntimeError(msg)
pyproject.write_text(new_text)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Inline or compact TOML arrays will break the regex

If override-dependencies is written as a single-line inline array — override-dependencies = ["pkg>=1.0", "other==2.0"] — the ^ anchor never matches because the spec is not at the start of a line. The RuntimeError is again caught as an "error" result, meaning those overrides are silently mis-reported rather than audited. The function's docstring says "Strip the single line containing the given override spec", so this limitation should at least be documented, or validated up-front by asserting the array is written in multi-line form before entering the per-spec loop.

Signed-off-by: Dong Hyuk Chang <9426164+thomasdhc@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
thomasdhc and others added 2 commits April 28, 2026 14:16
Signed-off-by: Dong Hyuk Chang <9426164+thomasdhc@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented Apr 28, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@thomasdhc
Copy link
Copy Markdown
Contributor Author

/ok to test c9da3bd

Comment on lines +191 to +192
if ver is None:
return "stale", f"{req.name} is not in the lock without the override"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Misleading "stale" classification when ver is None for a specifier override

_categorize falls through to return "stale", f"{req.name} is not in the lock without the override" for any version-constraining spec when the package is absent from the freshly generated lock. For a normal version override, the package should still be pulled in (overrides constrain versions, they don't add deps). The only realistic paths to ver is None here are:

  1. Package-name canonicalization failure (e.g., a dotted name — see the separate thread on locked_version), which causes a false "stale" classification rather than a lookup error.
  2. The package is only an optional extra and no extras were locked.

In case 1 the silent misclassification makes the audit trustworthy only for packages with no dots or extra-only specs. Consider distinguishing these cases — e.g., log a warning or return an intermediate category when ver is None but lock succeeded for a specifier override.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant