Skip to content

Commit 9c77a54

Browse files
PhysShellPhysShell
authored
docs: P-024 security audit profile — external tools + SARIF, reject own scanner engine (#169)
Records the design-discussion verdict on the Own.SecurityChecks idea: do not build a custom security scanner engine — it duplicates Nuclei/ testssl.sh/ZAP/Trivy and violates the audit charter in Plan.md (orchestrator-not-analyzer, no regex heuristics, take ready tools). Instead: a security profile for the existing audit fleet — tool-run manifests + thin raw→SARIF adapters (v0.1), a typed .NET config analyzer as the only own-code niche (v0.2), cross-tool correlation reusing the oracle scorer (v0.3). Non-goals section pins the rejected engine. Renumbered P-023 -> P-024 after #171 took P-023 (architecture guard). Review fixes: MD040 fence label (CodeRabbit), --include-transitive on the NuGet vulnerability scan (Codex).
1 parent 33a7302 commit 9c77a54

2 files changed

Lines changed: 197 additions & 0 deletions

File tree

‎docs/proposals/P-024-security-audit-profile.md‎

Lines changed: 196 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,196 @@
1+
# P-024 — Security audit profile (external tools + SARIF adapters)
2+
3+
- **Status:** draft — direction accepted in design discussion; implementation not
4+
started. Supersedes and **rejects** the earlier "Own.SecurityChecks" scanner-engine
5+
idea (recorded below so it is not re-proposed).
6+
- **Depends on:** the audit orchestrator design ([`Plan.md`](../../Plan.md)) — SARIF
7+
normalization, cross-tool confidence scoring, coverage map. Relates to
8+
[P-015](P-015-configuration-surface.md) (check selection / severity) for how the
9+
profile's categories surface in configuration.
10+
- **Where it lives:** the design is recorded here; per
11+
[`audit/README.md`](../../audit/README.md) active audit development currently
12+
happens in the `OwnAudit` repo, and this profile follows the audit code — it is an
13+
audit extension, not core. Same contract as everything else in the fleet:
14+
consumed through CLI + SARIF only, zero coupling to `ownlang/`.
15+
16+
## Decision (read this first)
17+
18+
**Do not build a security scanner.** A proposal was drafted for
19+
"Own.SecurityChecks": a C# engine interpreting a custom YAML detection DSL
20+
(request/expect matchers, HTTP/SSH/DB modules, its own severity model, an eventual
21+
NASL/OpenVAS/Nessus export). It is rejected, permanently, for reasons that are
22+
already codified as repository principles in `Plan.md`:
23+
24+
- **"Оркестратор, не анализатор"** — the audit layer runs mature external tools and
25+
aggregates evidence; it does not grow its own detectors.
26+
- **"Ни одной собственной эвристики «на регулярках»"** — the engine's example checks
27+
were regex-over-config and regex-over-banner, exactly the FP factory the charter
28+
forbids. (A banner regex like `OpenSSL 1\.0\.` misses vulnerable `1.1.0` while the
29+
remediation demands `>=1.1.1` — the failure mode is intrinsic, not a draft bug.)
30+
- **"Берём готовое"** — the niche is occupied. Nuclei *is* the "YAML checks with
31+
id/severity/matchers + CI + community corpus" product, with native SARIF export;
32+
testssl.sh owns TLS; ZAP baseline owns the safe web pass; Trivy and
33+
`dotnet list package --vulnerable` own dependencies. Rebuilding any of these is
34+
months of engine work plus an unbounded corpus-maintenance tail (Greenbone staffs
35+
a team for ~100k NVTs).
36+
37+
What survives from the old idea is **not** the engine but the *artifact model*: a
38+
security check is a first-class, reviewable, versioned thing with an id, a
39+
severity, and a defined FP policy. We keep that — as a **tool-run manifest**, not a
40+
detection DSL.
41+
42+
> **Own.NET Audit Security Profile** — a set of profiles and thin adapters that run
43+
> ready-made security tools inside the existing SARIF pipeline of the audit
44+
> orchestrator. Not a new product; a new audit profile.
45+
46+
## Motivation
47+
48+
The audit orchestrator answers "where does the legacy target hurt" for code
49+
health. The same fleet-of-tools / SARIF / cross-tool-agreement machinery applies
50+
unchanged to a second question — "what is insecure in the deployed surface":
51+
missing HSTS, legacy TLS, vulnerable packages, debug/detailed-errors leaking into
52+
production, dev signing credentials. Today none of this is covered, and the honest
53+
coverage map should say so (`NO-TOOL: skipped`) rather than pretend. The cheap,
54+
charter-compliant fix is to add security tools to the fleet, not to write one.
55+
56+
## Scope
57+
58+
### v0.1 — the fleet (external tools only, no own detectors)
59+
60+
| Concern | Tool | Output |
61+
|---|---|---|
62+
| Web baseline (headers, exposures, known CVE templates) | Nuclei (`-sarif-export`) | SARIF native |
63+
| Safe passive web scan (spider + passive rules, no attacks) | OWASP ZAP baseline | raw → adapter |
64+
| TLS/SSL protocols, ciphers, crypto flaws | testssl.sh (JSON output) | raw → adapter |
65+
| Dependencies, container images, IaC misconfig, secrets | Trivy (`--format sarif`) | SARIF native |
66+
| .NET package vulnerabilities (incl. transitive) | `dotnet list package --vulnerable --include-transitive` | raw → adapter |
67+
68+
Deliverables:
69+
70+
- `audit/security/profiles/*.yaml` — run manifests (see Sketch), e.g.
71+
`baseline-web`, `tls`, `supply-chain`, `dotnet-config` (the last lands in v0.2).
72+
- `audit/security/adapters/` — thin `raw → SARIF` converters for tools without
73+
native SARIF, same shape as the existing static-layer adapters.
74+
- Findings enter the **existing** aggregation: normalize → score → report. No
75+
separate security report pipeline.
76+
- Coverage map rows per category: `CHECKED` (tool ran), `SKIPPED` (no reliable
77+
tool), `NEEDS-RUNTIME` (target must be running/reachable), `NEEDS-AUTH`
78+
(credentialed scan not configured). Honest skip extends to security verbatim.
79+
80+
### v0.2 — `OwnAudit.DotNetConfig` (the one place own code is justified)
81+
82+
The single niche no mature tool covers well: **typed** .NET configuration audit.
83+
Inputs: `web.config`, `app.config`, `appsettings*.json`, `launchSettings.json`,
84+
`*.csproj`, `packages.config`, Kestrel/IIS hosting config where available.
85+
86+
Candidate checks (each ships with confidence + limitations, per finding):
87+
88+
- ASP.NET `compilation debug="true"` / detailed errors (`customErrors`) exposed
89+
- cookie policy: missing `Secure` / `HttpOnly` / unsafe `SameSite` defaults
90+
- DataProtection keys ephemeral in production; weak or hardcoded `machineKey`
91+
- `AllowedHosts` wildcard in production profiles
92+
- IdentityServer dev signing credential (`AddDeveloperSigningCredential`) outside dev
93+
- forwarded-headers misconfiguration behind a proxy
94+
- HTTPS redirection / HSTS absent in production profile
95+
96+
Discipline (non-negotiable, mirrors the core's culture):
97+
98+
1. **No regex-first rules.** XML via an XML parser, JSON via a JSON parser,
99+
`csproj` via MSBuild APIs or a resilient XML model.
100+
2. Where production context cannot be proven (which `appsettings.*.json` wins,
101+
what the environment is), the finding is `needs-review`, never `high`.
102+
3. Output is SARIF only; findings carry confidence and a limitations note.
103+
4. No reliable signal ⇒ `NO-TOOL / skipped`, not a guess.
104+
5. Test fixtures: paired good/bad configs (`web.config`, `appsettings.json`,
105+
IdentityServer setup) gating every rule in CI.
106+
107+
### v0.3 — cross-tool correlation
108+
109+
Direct reuse of the oracle pattern: ZAP **and** Nuclei both flag a header issue ⇒
110+
confidence up; Trivy **and** `dotnet list package` both flag a package ⇒
111+
confidence up; a single noisy tool ⇒ candidate / needs-review. No new machinery —
112+
this is the existing cross-tool-agreement scorer fed with security findings.
113+
114+
## Non-goals
115+
116+
The most important section. None of the following will be built, and future
117+
proposals re-introducing them should cite this section and explain what changed:
118+
119+
- an own YAML **detection** DSL (request/expect/matchers) or its interpreter
120+
- an own HTTP/TLS/SSH/DB scanner or probe modules
121+
- an own CVE-check corpus (that is Nuclei templates / Trivy DBs / GVM feeds)
122+
- export or conversion of checks to NASL / OpenVAS / Nessus plugin formats
123+
- regex heuristics as a primary detection mechanism anywhere in the profile
124+
- active/attacking scans; the web pass stays at ZAP *baseline* (passive) and
125+
Nuclei templates vetted as non-intrusive
126+
- scanning targets outside an explicit, configured allowlist (authorization is a
127+
precondition, and profiles must name their targets; nothing scans "the network")
128+
129+
## Sketch
130+
131+
A profile entry is a **tool-run manifest** — it says *what to run and how to file
132+
the results*, never how to detect:
133+
134+
```yaml
135+
id: OWNSEC-WEB-001
136+
title: Web security baseline
137+
tool: nuclei
138+
target: web
139+
sarifCategory: security/nuclei/web-baseline
140+
command:
141+
executable: nuclei
142+
args:
143+
- "-l"
144+
- "targets/web.txt"
145+
- "-t"
146+
- "audit/security/nuclei-templates/"
147+
- "-sarif-export"
148+
- "artifacts/security/nuclei-web.sarif"
149+
confidence:
150+
source: tool-native
151+
fpPolicy: external-tool
152+
```
153+
154+
The contrast that keeps the design honest:
155+
156+
| Rejected (engine) | Adopted (profile) |
157+
|---|---|
158+
| `match regex: OpenSSL 1\.0\.` | run testssl.sh, adapt findings |
159+
| `expect header: Strict-Transport-Security` | run Nuclei/ZAP baseline, accept tool finding |
160+
| own YAML request/matcher DSL | Nuclei templates (write *templates*, not an engine) |
161+
| own HTTP/SSH/DB runner | external tools + adapters |
162+
| own severity model | tool severity + cross-tool confidence |
163+
| "regex said something" | `NO-TOOL: skipped` |
164+
165+
Layout (inside the audit subtree, liftable with it):
166+
167+
```text
168+
audit/security/
169+
profiles/ # run manifests: baseline-web, tls, supply-chain, dotnet-config
170+
nuclei-templates/ # own *templates* for the external engine, if any
171+
adapters/ # zap_to_sarif, testssl_to_sarif, dotnet_vuln_to_sarif
172+
tools/ # run_security_profile entrypoint
173+
docs/
174+
```
175+
176+
Tool versions are pinned and printed in the report header, findings are
177+
reproducible per commit — same as the rest of the fleet.
178+
179+
## Open questions
180+
181+
- **Where does v0.2 code live?** `OwnAudit.DotNetConfig` as a .NET project vs. a
182+
Python typed-parser module in the aggregation layer. The .NET route gets MSBuild
183+
APIs for free; the Python route keeps the audit subtree dependency-free. Decide
184+
at v0.2 start, not before.
185+
- **Nuclei template curation.** Which subset of community templates is
186+
non-intrusive enough for the default profile, and who reviews additions.
187+
- **Target declaration.** Format for the scan allowlist (`targets/*.txt` vs. a
188+
section in the profile) and how CI proves the target is a lab/staging host, not
189+
production, before running network tools.
190+
- **Licensing hygiene.** All fleet tools are consumed as external executables
191+
(Nuclei MIT, Trivy Apache-2.0, testssl.sh GPLv2, ZAP Apache-2.0) — GPL tools are
192+
invoked, never linked, matching how GPL analyzers are already handled.
193+
- **Consolidation timing.** Whether v0.1 lands in `OwnAudit` first (current live
194+
audit repo) and rides the deferred consolidation back into `audit/`, or waits
195+
for consolidation. Default: follow wherever the static-layer fleet lives at
196+
implementation time.

‎docs/proposals/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ proposal is marked `done` with a pointer.
4343
| [P-021](P-021-async-audit-pack.md) | Async audit pack (`Own.Async`) | draft |
4444
| [P-022](P-022-rust-core-migration.md) | Rust core migration: crate DAG, patterns, prior art, differential oracle (Python = golden) | draft / exploratory |
4545
| [P-023](P-023-architecture-guard.md) | Architecture guard (`Own.Arch`): rules.yaml intent model + dependency-graph gate + baseline ratchet | draft |
46+
| [P-024](P-024-security-audit-profile.md) | Security audit profile (external tools + SARIF adapters; rejects own scanner engine) | draft |
4647

4748
> For priorities, milestones, the framing, and the design philosophy across all
4849
> of these, see the strategy hub: [`docs/ROADMAP.md`](../ROADMAP.md). P-004 … P-016

0 commit comments

Comments
 (0)