Skip to content

feat: baseline / false-positive suppression (closes #88)#106

Open
assinchu wants to merge 1 commit into
NVIDIA:mainfrom
assinchu:feature/baseline-suppression
Open

feat: baseline / false-positive suppression (closes #88)#106
assinchu wants to merge 1 commit into
NVIDIA:mainfrom
assinchu:feature/baseline-suppression

Conversation

@assinchu

Copy link
Copy Markdown

Summary

Implements the baseline / false-positive suppression mechanism requested in #88.
Lets users suppress known/accepted findings so the risk score reflects only
un-triaged issues and re-scans surface only new findings — addressing the
"~95% false positives from framework patterns, every scan re-reports the same
issues" problem raised in that issue.

Without a baseline, behavior is unchanged.

What it adds

Two suppression mechanisms in one YAML/JSON baseline:

  • rules — glob-based, drift-tolerant: match on rule id, file path, and/or
    message (covers both global pattern suppression and skill/file-scoped
    suppression). Each rule carries a required reason.
  • fingerprints — exact per-finding hashes, machine-generated for incremental
    CI use.

CLI:

  • scan --baseline/-b PATH and --show-suppressed
  • new skillspector baseline <path> command to generate a baseline from a scan

Design

Suppression is applied in the report node (the single scoring/formatting
point, reused by CLI and any future REST API). Suppressed findings never count
toward the risk score and are excluded from SARIF results; they appear in
terminal/markdown reports only with --show-suppressed, and are always listed
in JSON output under suppressed/suppressed_count.

New module: skillspector/suppression.py (Baseline, SuppressionRule,
load_baseline, partition_findings, finding_fingerprint, build_baseline_dict).

Tests & docs

  • New tests/unit/test_suppression.py + suppression cases in test_report.py
    and test_cli.py. Full suite passes; ruff check/format clean.
  • New docs/SUPPRESSION.md, .skillspector-baseline.example.yaml, and
    README/DEVELOPMENT.md updates.

Example

# Accept current findings, then re-scan against the baseline
skillspector baseline ./my-skill/ -o .skillspector-baseline.yaml
skillspector scan ./my-skill/ --baseline .skillspector-baseline.yaml --show-suppressed

Closes #88

Adds a baseline mechanism so known/accepted findings can be suppressed,
letting the risk score reflect only un-triaged issues and re-scans surface
only new findings. Suppressed findings never count toward the score and are
excluded from SARIF; they appear in reports only with --show-suppressed and
are always listed in JSON output.

Two complementary suppression mechanisms in one YAML/JSON baseline:
- rules: glob-based (rule id / file path / message), drift-tolerant
- fingerprints: exact per-finding hashes, machine-generated

CLI:
- scan --baseline/-b PATH, --show-suppressed
- new `skillspector baseline <path>` command to generate a baseline

Suppression is applied in the report node (the single scoring/formatting
point) so the CLI and any future REST API behave identically. Includes a
new skillspector.suppression module, unit/report/CLI tests, an example
baseline, and docs (docs/SUPPRESSION.md).

Closes NVIDIA#88

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Aravinda Sharma <7734009+assinchu@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: False-positive suppression / baseline mechanism

1 participant