feat: baseline / false-positive suppression (closes #88)#106
Open
assinchu wants to merge 1 commit into
Open
Conversation
Adds a baseline mechanism so known/accepted findings can be suppressed, letting the risk score reflect only un-triaged issues and re-scans surface only new findings. Suppressed findings never count toward the score and are excluded from SARIF; they appear in reports only with --show-suppressed and are always listed in JSON output. Two complementary suppression mechanisms in one YAML/JSON baseline: - rules: glob-based (rule id / file path / message), drift-tolerant - fingerprints: exact per-finding hashes, machine-generated CLI: - scan --baseline/-b PATH, --show-suppressed - new `skillspector baseline <path>` command to generate a baseline Suppression is applied in the report node (the single scoring/formatting point) so the CLI and any future REST API behave identically. Includes a new skillspector.suppression module, unit/report/CLI tests, an example baseline, and docs (docs/SUPPRESSION.md). Closes NVIDIA#88 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Aravinda Sharma <7734009+assinchu@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the baseline / false-positive suppression mechanism requested in #88.
Lets users suppress known/accepted findings so the risk score reflects only
un-triaged issues and re-scans surface only new findings — addressing the
"~95% false positives from framework patterns, every scan re-reports the same
issues" problem raised in that issue.
Without a baseline, behavior is unchanged.
What it adds
Two suppression mechanisms in one YAML/JSON baseline:
rules— glob-based, drift-tolerant: match on rule id, file path, and/ormessage (covers both global pattern suppression and skill/file-scoped
suppression). Each rule carries a required
reason.fingerprints— exact per-finding hashes, machine-generated for incrementalCI use.
CLI:
scan --baseline/-b PATHand--show-suppressedskillspector baseline <path>command to generate a baseline from a scanDesign
Suppression is applied in the report node (the single scoring/formatting
point, reused by CLI and any future REST API). Suppressed findings never count
toward the risk score and are excluded from SARIF results; they appear in
terminal/markdown reports only with
--show-suppressed, and are always listedin JSON output under
suppressed/suppressed_count.New module:
skillspector/suppression.py(Baseline,SuppressionRule,load_baseline,partition_findings,finding_fingerprint,build_baseline_dict).Tests & docs
tests/unit/test_suppression.py+ suppression cases intest_report.pyand
test_cli.py. Full suite passes;ruff check/formatclean.docs/SUPPRESSION.md,.skillspector-baseline.example.yaml, andREADME/DEVELOPMENT.md updates.
Example
# Accept current findings, then re-scan against the baseline skillspector baseline ./my-skill/ -o .skillspector-baseline.yaml skillspector scan ./my-skill/ --baseline .skillspector-baseline.yaml --show-suppressedCloses #88