Skip to content

Commit ed539ba

Browse files
dineshmistryomer-roth
authored andcommitted
PM-4777: Add binary scanning for Java artifacts (JAR, WAR, EAR, Spring Boot)
Adds `cycode scan -t sca binary <path>` and `cycode report sbom binary <path>`. The CLI opens a Java archive locally, identifies the open-source components inside it from embedded Maven metadata (pom.properties, then optionally a Maven Central SHA-1 lookup, then MANIFEST.MF), synthesises a CycloneDX 1.4 document, and feeds it to the existing SCA scan path. The archive itself is never uploaded. What ships: - Hardened zip reader (zip-slip, symlinks, absolute paths, compression and entry-count bombs, duplicate entry names read by central-directory record). - Identification ladder with an explicit `unidentified` result; no coordinate is ever guessed from a filename or an unshaped manifest value. - Dependency graph from containment plus real edges from embedded pom.xml (parsed with a DOCTYPE/ENTITY guard, no defusedxml needed). - `--max-depth`, `--offline`, `--maven-central` (opt-in, sends the hash only), `--project-name`, `--keep-bom`, `--include-binaries`. - Coverage line and JSON fields so CI can gate on identification, not guess. - `DigestResolver` seam so the Cycode backend digest index can replace the Maven Central implementation without caller changes. No new runtime dependency. All HTTP in tests is mocked with `responses`.
1 parent bf2d31f commit ed539ba

43 files changed

Lines changed: 5794 additions & 8 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 151 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -45,10 +45,16 @@ This guide walks you through both installation and usage.
4545
1. [Branch Option](#branch-option)
4646
3. [Path Scan](#path-scan)
4747
1. [Terraform Plan Scan](#terraform-plan-scan)
48-
4. [Commit History Scan](#commit-history-scan)
48+
4. [Binary Scan](#binary-scan)
49+
1. [Supported Artifacts](#supported-artifacts)
50+
2. [Binary Scan Options](#binary-scan-options)
51+
3. [How Components Are Identified](#how-components-are-identified)
52+
4. [Unidentified Components](#unidentified-components)
53+
5. [Limitations](#binary-scan-limitations)
54+
5. [Commit History Scan](#commit-history-scan)
4955
1. [Commit Range Option (Diff Scanning)](#commit-range-option-diff-scanning)
50-
5. [Pre-Commit Scan](#pre-commit-scan)
51-
6. [Pre-Push Scan](#pre-push-scan)
56+
6. [Pre-Commit Scan](#pre-commit-scan)
57+
7. [Pre-Push Scan](#pre-push-scan)
5258
2. [Scan Results](#scan-results)
5359
1. [Show/Hide Secrets](#showhide-secrets)
5460
2. [Soft Fail](#soft-fail)
@@ -67,6 +73,9 @@ This guide walks you through both installation and usage.
6773
6. [Ignoring via a config file](#ignoring-via-a-config-file)
6874
9. [Report command](#report-command)
6975
1. [Generating SBOM Report](#generating-sbom-report)
76+
1. [Repository](#repository)
77+
2. [Local Project](#local-project)
78+
3. [Built Artifact](#built-artifact)
7079
10. [Import command](#import-command)
7180
11. [Scan logs](#scan-logs)
7281
12. [Syntax Help](#syntax-help)
@@ -997,6 +1006,127 @@ If you just have a configuration file, you can generate a plan by doing the foll
9971006
9981007
`cycode scan -t iac path ~/PATH/TO/YOUR/{tfplan}.json`
9991008
1009+
### Binary Scan
1010+
1011+
A binary scan examines a built Java artifact instead of a source tree.
1012+
Point the CLI at a JAR, WAR, EAR or Spring Boot fat JAR and it identifies the open-source components packaged inside it, then scans them exactly as an SCA scan of the source would.
1013+
1014+
This closes the gap between what was scanned and what was shipped.
1015+
It is the only option when you have no source at all: a vendor-supplied JAR, a legacy EAR whose build job no longer exists, or a release gate that should check the actual deployable rather than the commit that supposedly produced it.
1016+
1017+
To scan an artifact, execute the following:
1018+
1019+
`cycode scan -t sca binary {{path}}`
1020+
1021+
For example:
1022+
1023+
```shell
1024+
# a single deployable
1025+
cycode scan -t sca binary ./dist/payments.war
1026+
1027+
# every Java archive under a directory
1028+
cycode scan -t sca binary ./dist
1029+
1030+
# recurse further into deeply nested archives
1031+
cycode scan -t sca --max-depth 5 binary ./dist/payments.ear
1032+
```
1033+
1034+
Everything an SCA scan normally supports still applies: `--severity-threshold`, `--soft-fail`, `cycode ignore` rules, `--export`, `--cycode-report` and the usual exit codes.
1035+
1036+
> [!IMPORTANT]
1037+
> The artifact never leaves your machine.
1038+
> The CLI opens it locally and uploads only the resulting component inventory, which is typically a few tens of kilobytes regardless of how large the artifact is.
1039+
1040+
You can also extract archives encountered during an ordinary path scan, using `--include-binaries`:
1041+
1042+
`cycode scan -t sca --include-binaries path ./target`
1043+
1044+
This is off by default.
1045+
Extracting a large tree of artifacts takes real time, and a path scan that silently got slower would be a worse surprise than an opt-in flag.
1046+
1047+
#### Supported Artifacts
1048+
1049+
| Artifact | Recognized layouts |
1050+
|---|---|
1051+
| JAR | `lib/*.jar`, embedded Maven metadata |
1052+
| WAR | `WEB-INF/lib/*.jar` |
1053+
| EAR | nested `*.war` and `*.jar` modules, `APP-INF/lib/*.jar` |
1054+
| Spring Boot fat JAR | `BOOT-INF/lib/*.jar` |
1055+
1056+
Nested archives are opened recursively, so a JAR inside a WAR inside an EAR is scanned.
1057+
1058+
#### Binary Scan Options
1059+
1060+
> [!NOTE]
1061+
> These options belong to the `scan` command, so they must appear **before** the `binary` subcommand:
1062+
> `cycode scan -t sca --max-depth 5 binary app.ear`
1063+
1064+
| Option | Default | Description |
1065+
|---|---|---|
1066+
| `--max-depth` | `3` | Nested-archive recursion limit. An EAR containing WARs containing JARs is depth 3. |
1067+
| `--offline` | off | Identify components from embedded metadata only. Acknowledges and silences the partial-results warning. |
1068+
| `--maven-central` | off | Look up archives that embedded metadata cannot identify on Maven Central by SHA-1. Sends only the digest, never the archive. Cannot be combined with `--offline`. |
1069+
| `--project-name` | inferred | Override the platform identity when the artifact is detached from its source repository. |
1070+
| `--keep-bom` | off | Write the generated component inventory beside each artifact, for inspection or audit. |
1071+
| `--include-binaries` | off | On `scan path` only. Extract any Java archives encountered during the walk. |
1072+
1073+
Platform identity is inferred from the Git remote when you run inside a repository, and falls back to the artifact filename otherwise.
1074+
`--monitor` is refused on a bare filename identity, because monitoring keyed on `app.jar` would merge unrelated projects into one and quietly corrupt the trend data.
1075+
Use `--project-name` or run from inside the repository the artifact was built from.
1076+
1077+
#### How Components Are Identified
1078+
1079+
Components are identified from metadata the build itself wrote into the archive:
1080+
1081+
| Source | Confidence | Notes |
1082+
|---|---|---|
1083+
| `META-INF/maven/.../pom.properties` | exact | Written by Maven. Authoritative group, artifact and version. |
1084+
| Maven Central digest lookup (`--maven-central`) | exact | The SHA-1 of the archive, matched against what Maven Central published. Opt-in, because it is the one step that sends anything about the artifact off the machine. |
1085+
| `META-INF/MANIFEST.MF` attributes | low | `Implementation-Title`, OSGi `Bundle-SymbolicName` and similar, with the group taken from `Implementation-Vendor-Id`. Used only when all three parts are shaped like Maven coordinates; a product name or a build banner is not one. |
1086+
1087+
Every finding reports which source identified its component and where inside the artifact that component sits, so a hit on a large EAR points at a specific nested JAR rather than the whole file.
1088+
1089+
> [!NOTE]
1090+
> Findings from a low-confidence match are printed and exported, but do **not** affect the exit code.
1091+
> A wrong coordinate produces a wrong vulnerability list, and a fabricated CVE breaking a release costs more trust than the extra coverage is worth.
1092+
1093+
#### Unidentified Components
1094+
1095+
Archives that carry no usable metadata are listed in their own section by path, digest and size, and appear under an `unidentified` key in `--output json` so CI can assert on coverage:
1096+
1097+
```
1098+
╭─ 🔎 Unidentified (1) ────────────────────────────────────────────╮
1099+
│ Path SHA-1 Size │
1100+
│ payments.war > WEB-INF/lib/internal-shim.jar 5fe08079… 142 B │
1101+
╰──────────────────────────────────────────────────────────────────╯
1102+
1103+
3 identified (1 low confidence) | 1 unidentified | 17 vulnerabilities
1104+
```
1105+
1106+
The coverage line counts manifest-only matches as identified but calls them out, and `--output json` reports the same number as `binary.low_confidence_components`.
1107+
1108+
We do not guess a component's identity from its filename.
1109+
`internal-shim.jar` is not evidence of anything, and an admitted gap is more useful than an invented coordinate.
1110+
Unidentified components do not set the exit code on their own.
1111+
1112+
#### Binary Scan Limitations
1113+
1114+
Read this before relying on a binary scan as your only check.
1115+
1116+
- **Relocated and shaded classes are not detected.**
1117+
When a build rewrites `com.google.common` into `com.acme.shaded.common` and merges it into the parent JAR, there is no separate JAR to identify and no metadata left to read.
1118+
Those components will not appear in the results at all.
1119+
Detecting them requires class-level fingerprinting, which this feature does not do.
1120+
- **Source scanning gives a truer picture.**
1121+
A source scan resolves the real dependency graph from your lockfiles.
1122+
A binary scan sees what is physically packaged, and infers relationships from archive nesting plus any embedded `pom.xml` files it finds.
1123+
Where you have source, scan the source; use binary scanning for the artifacts you cannot scan any other way.
1124+
- **Coverage is reported, not assumed.**
1125+
The coverage line is always printed and always true.
1126+
If it says components could not be identified, the scan is genuinely incomplete for those components rather than clean.
1127+
- **Java only, for now.**
1128+
.NET, npm and container artifacts are not supported yet.
1129+
10001130
### Commit History Scan
10011131

10021132
> [!NOTE]
@@ -1547,6 +1677,7 @@ The following commands are available for use with this command:
15471677
|------------------|-----------------------------------------------------------------|
15481678
| `path` | Generate SBOM report for provided path in the command |
15491679
| `repository-url` | Generate SBOM report for provided repository URI in the command |
1680+
| `binary` | Generate SBOM report for a built Java artifact (JAR, WAR, EAR) |
15501681
15511682
### Repository
15521683
@@ -1572,6 +1703,23 @@ The `path` subcommand supports the following additional options:
15721703
| `--gradle-all-sub-projects` | Run the Gradle restore command for all sub-projects (use from the root of a multi-project Gradle build). |
15731704
| `--maven-settings-file` | For Maven only, allows using a custom [settings.xml](https://maven.apache.org/settings.html) file when building the dependency tree. |
15741705
1706+
### Built Artifact
1707+
1708+
To create an SBOM report for a built Java artifact, without scanning it for vulnerabilities:\
1709+
`cycode report sbom --format <sbom format> --output-file </path/to/file> binary </path/to/artifact>`
1710+
1711+
For example:\
1712+
`cycode report sbom --format spdx-2.3 --output-file payments-sbom.json binary ./dist/payments.war`
1713+
1714+
This answers the compliance case directly: an SBOM of what you actually shipped, rather than of what was committed.
1715+
It uses the same extraction as [Binary Scan](#binary-scan), so the [limitations](#binary-scan-limitations) documented there apply here too — in particular, shaded and relocated components will be missing from the SBOM.
1716+
1717+
The `binary` subcommand supports the following additional option:
1718+
1719+
| Option | Description |
1720+
|---------------|----------------------------------------------------------------------------------------------|
1721+
| `--max-depth` | Nested-archive recursion limit. Defaults to 3. An EAR containing WARs containing JARs is depth 3. |
1722+
15751723
# Import Command
15761724
15771725
## Importing SBOM

cycode/cli/apps/report/sbom/__init__.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
import typer
22

3+
from cycode.cli.apps.report.sbom.binary.binary_command import binary_command
34
from cycode.cli.apps.report.sbom.path.path_command import path_command
45
from cycode.cli.apps.report.sbom.repository_url.repository_url_command import repository_url_command
56
from cycode.cli.apps.report.sbom.sbom_command import sbom_command
@@ -10,6 +11,7 @@
1011
app.command(name='repository-url', short_help='Generate SBOM report for provided repository URI in the command.')(
1112
repository_url_command
1213
)
14+
app.command(name='binary', short_help='Generate SBOM report for a built Java artifact (JAR, WAR, EAR).')(binary_command)
1315

1416
# backward compatibility
1517
app.command(hidden=True, name='repository_url')(repository_url_command)

cycode/cli/apps/report/sbom/binary/__init__.py

Whitespace-only changes.
Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
import time
2+
from pathlib import Path
3+
from typing import Annotated
4+
5+
import typer
6+
7+
from cycode.cli import consts
8+
from cycode.cli.apps.report.sbom.common import create_sbom_report, send_report_feedback
9+
from cycode.cli.exceptions.handle_report_sbom_errors import handle_report_exception
10+
from cycode.cli.files_collector.binary.collector import collect_binary_documents
11+
from cycode.cli.files_collector.zip_documents import zip_documents
12+
from cycode.cli.utils.get_api_client import get_report_cycode_client
13+
from cycode.cli.utils.progress_bar import SbomReportProgressBarSection
14+
15+
_REPORT_COMMAND_TYPE = 'binary'
16+
17+
18+
def binary_command(
19+
ctx: typer.Context,
20+
path: Annotated[
21+
Path,
22+
typer.Argument(
23+
exists=True,
24+
resolve_path=True,
25+
help='Path to the built artifact to generate an SBOM for.',
26+
show_default=False,
27+
),
28+
],
29+
max_depth: Annotated[
30+
int,
31+
typer.Option('--max-depth', help='Nested-archive recursion limit.', min=1),
32+
] = consts.BINARY_MAX_DEPTH,
33+
maven_central: Annotated[
34+
bool,
35+
typer.Option(
36+
'--maven-central',
37+
help='Look archives that embedded metadata cannot identify up on Maven Central by SHA-1. '
38+
'Sends the digest of each such archive, never the archive itself, to search.maven.org.',
39+
),
40+
] = False,
41+
) -> None:
42+
""":package: [bold cyan]Generate an SBOM for a built Java artifact.[/]
43+
44+
Reads a JAR, WAR, EAR or Spring Boot fat JAR and produces an SBOM of the open-source components inside it,
45+
without scanning them for vulnerabilities. Answers the compliance case directly: an SBOM of what shipped,
46+
rather than of what was committed.
47+
48+
Example usage:
49+
* `cycode report sbom --format cyclonedx-1.4-json binary app.war`
50+
* `cycode report sbom --format spdx-2.3-json binary app.ear`
51+
52+
Format conversion happens server-side, so every format the path command supports is supported here too.
53+
54+
"""
55+
ctx.obj['binary_max_depth'] = max_depth
56+
ctx.obj['maven_central'] = maven_central
57+
58+
client = get_report_cycode_client(ctx)
59+
report_parameters = ctx.obj['report_parameters']
60+
output_format = report_parameters.output_format
61+
output_file = ctx.obj['output_file']
62+
63+
progress_bar = ctx.obj['progress_bar']
64+
progress_bar.start()
65+
66+
start_scan_time = time.time()
67+
report_execution_id = -1
68+
69+
try:
70+
# the only difference from the path command: our collector in place of the manifest walk. Everything from
71+
# zip_documents onward is reused verbatim, and the server generates the document.
72+
collection = collect_binary_documents(
73+
ctx,
74+
(str(path),),
75+
stop_on_error=ctx.obj.get('stop_on_error', False),
76+
progress_bar_section=SbomReportProgressBarSection.PREPARE_LOCAL_FILES,
77+
)
78+
ctx.obj['binary_result'] = collection
79+
80+
if not collection.documents:
81+
raise typer.BadParameter(
82+
f'No supported binary artifacts were found at {str(path)!r}. '
83+
'Supported artifacts are .jar, .war and .ear files.',
84+
param_hint='PATH',
85+
)
86+
87+
zipped_documents = zip_documents(consts.SCA_SCAN_TYPE, collection.documents)
88+
report_execution = client.request_sbom_report_execution(report_parameters, zip_file=zipped_documents)
89+
report_execution_id = report_execution.id
90+
91+
create_sbom_report(progress_bar, client, report_execution_id, output_file, output_format)
92+
93+
send_report_feedback(
94+
client=client,
95+
start_scan_time=start_scan_time,
96+
report_type='SBOM',
97+
report_command_type=_REPORT_COMMAND_TYPE,
98+
request_report_parameters=report_parameters.to_dict(without_entity_type=False),
99+
report_execution_id=report_execution_id,
100+
request_zip_file_size=zipped_documents.size,
101+
)
102+
except Exception as e:
103+
progress_bar.stop()
104+
105+
send_report_feedback(
106+
client=client,
107+
start_scan_time=start_scan_time,
108+
report_type='SBOM',
109+
report_command_type=_REPORT_COMMAND_TYPE,
110+
request_report_parameters=report_parameters.to_dict(without_entity_type=False),
111+
report_execution_id=report_execution_id,
112+
error_message=str(e),
113+
)
114+
115+
handle_report_exception(ctx, e)

cycode/cli/apps/scan/__init__.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
import typer
22

3+
from cycode.cli.apps.scan.binary.binary_command import binary_command
34
from cycode.cli.apps.scan.commit_history.commit_history_command import commit_history_command
45
from cycode.cli.apps.scan.path.path_command import path_command
56
from cycode.cli.apps.scan.pre_commit.pre_commit_command import pre_commit_command
@@ -23,6 +24,7 @@
2324

2425
app.command(name='path', short_help='Scan the files in the paths provided in the command.')(path_command)
2526
app.command(name='repository', short_help='Scan the Git repository included files.')(repository_command)
27+
app.command(name='binary', short_help='Scan built Java artifacts (JAR, WAR, EAR, Spring Boot).')(binary_command)
2628
app.command(name='commit-history', short_help='Scan commit history or perform diff scanning between specific commits.')(
2729
commit_history_command
2830
)

cycode/cli/apps/scan/binary/__init__.py

Whitespace-only changes.
Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
from pathlib import Path
2+
from typing import Annotated
3+
4+
import typer
5+
6+
from cycode.cli.apps.scan.binary.identity import (
7+
assert_monitor_has_an_explicit_identity,
8+
resolve_platform_identity,
9+
)
10+
from cycode.cli.apps.scan.code_scanner import scan_binary_artifacts
11+
from cycode.cli.logger import logger
12+
13+
14+
def binary_command(
15+
ctx: typer.Context,
16+
paths: Annotated[
17+
list[Path],
18+
typer.Argument(
19+
exists=True,
20+
resolve_path=True,
21+
help='Paths to the built artifacts to scan',
22+
show_default=False,
23+
),
24+
],
25+
) -> None:
26+
""":package: [bold cyan]Scan built Java artifacts for open-source vulnerabilities.[/]
27+
28+
Opens a JAR, WAR, EAR or Spring Boot fat JAR, identifies the open-source components inside it, and scans them
29+
exactly as a source scan would. The artifact never leaves your machine: only the component inventory is
30+
uploaded.
31+
32+
Example usage:
33+
* `cycode scan -t sca binary app.war`: Scan a single deployable.
34+
* `cycode scan -t sca binary dist/`: Scan every Java archive under a directory.
35+
* `cycode scan -t sca --max-depth 5 binary app.ear`: Recurse further into nested archives.
36+
37+
Components are identified from embedded Maven metadata. Anything that cannot be identified is reported in its
38+
own section rather than guessed at. Relocated and shaded classes are not detected: where source is available,
39+
a source scan gives a truer dependency graph.
40+
41+
"""
42+
tuple_paths = tuple(str(path) for path in paths)
43+
44+
identity = resolve_platform_identity(ctx, tuple_paths)
45+
if ctx.obj.get('monitor'):
46+
assert_monitor_has_an_explicit_identity(identity)
47+
48+
ctx.obj['binary_identity'] = identity
49+
50+
progress_bar = ctx.obj['progress_bar']
51+
progress_bar.start()
52+
53+
logger.debug(
54+
'Starting binary scan process, %s',
55+
{'paths': paths, 'identity': identity.value, 'identity_source': identity.source},
56+
)
57+
58+
scan_binary_artifacts(ctx, tuple_paths)

0 commit comments

Comments
 (0)