Skip to content

sandboy: S0 GitHub-hosted-runner feasibility gate — ACCEPTED - #277

Merged
PhysShell merged 2 commits into
mainfrom
claude/qodec-benchmark-v2-representative-logs-n2-dqn3a1
Jul 14, 2026
Merged

PhysShell merged 2 commits into
mainfrom
claude/qodec-benchmark-v2-representative-logs-n2-dqn3a1

Conversation

@PhysShell

@PhysShell PhysShell commented Jul 14, 2026

Copy link
Copy Markdown
Owner

Что и зачем

Sandboy S0: ACCEPTED. Capability-probe gate (часть sourcing addendum к QODEC Scope N2), проверяющий, реально ли текущий коммит Sandboy (Landlock + seccomp, без VM) конфайнит враждебное дерево процессов на обычном GitHub-hosted ubuntu-24.04 runner'е — до того, как N2 начнёт майнить реальные публичные репозитории через него. Ничего стороннего не исполняется: только синтетические fixtures под sandboy/tests/feasibility/.

Попутно закрыт реальный пробел, найденный при проектировании canary-теста: sandboy run раньше пробрасывал весь launcher-окружение в обёрнутую команду без изменений. Добавлен env_clear() + env_allow-allowlist (по умолчанию — deny-all).

Итоговый результат: PASS (25/25 протестированных контролей)

Реальный прогон на ubuntu-24.04 (GitHub-hosted): workflow run 29350379408, commit e925058.

  • Landlock ABI v7, fully_enforced — filesystem read/write/symlink-escape/unix-socket/HOME/.ssh/unrelated-workspace вне allowlist корректно денайятся.
  • seccomp денайит ptrace/mount/setns на каждой глубине процесса (self/child/grandchild/compiler-tree-shaped). Кейс с ptrace — чистое доказательство: без sandboy успевает выполниться, под sandboy — денайится, изолируя вклад именно seccomp.
  • Credential canaries (фейковые GITHUB_TOKEN/AZURE_DEVOPS_PAT/NUGET_AUTH_TOKEN/AWS_SECRET_ACCESS_KEY/SSH_AUTH_SOCK) не достигают ни сам процесс, ни потомков — подтверждает работу env_clear()+allowlist.
  • Network: TCP корректно scoped по порту; UDP подтверждён как неэнфорсируемый (документированный gap, не регрессия); cloud-metadata address-vs-port — задокументированное ограничение (Layer 3, не построен), не тестировалось вживую.

Принятый коммит для N2: e925058ddea405b5821fc0aed4882c76650dcbe9 — уже использован как pinned Sandboy commit в N2-A miner-canary (007#50), где реальный dotnet build под этим Sandboy успешно и воспроизводимо прошёл.

Тип изменения

  • feat — новая возможность (probe subcommand, feasibility gate)
  • fix — исправление бага (env leak в sandboy run)
  • docs — документация
  • refactor / chore / test / ci — без изменения поведения

Как проверено

  • cargo test --release (sandboy) — 4 unit-теста (env_allow default/parse, seccomp_deny default/hard-error)
  • cargo fmt --check, cargo clippy --all-targets --release — чисто (одно pre-existing warning не в новом коде)
  • Реальный прогон на ubuntu-24.04 GitHub-hosted runner — 25/25 контролей PASS (см. выше)
  • Локальный прогон в dev-контейнере без Landlock корректно fail-closed (ожидаемо, не баг)

Связанные issue

Нет — часть Scope N2 sourcing addendum (Sandboy S0 feasibility gate), без отдельного issue.

Чеклист

  • изменение покрыто тестом/селфтестом
  • README/docs обновлены (sandboy/README.md, sandboy/policy.example.toml)
  • коммиты в conventional-commit стиле
  • CI зелёный (39/39 check runs)
  • zero unresolved review threads

Generated by Claude Code

…be, hostile fixtures)

Scope: Sandboy S0 capability probe for the QODEC Scope N2 sourcing addendum.
Determines whether the existing Sandboy commit (Landlock + seccomp, no VM —
Layer 2 of the isolation ADR) actually confines a hostile process tree on a
standard ubuntu-24.04 GitHub-hosted runner, which supplies the disposable-VM
boundary this gate assumes rather than builds. No third-party repository is
executed, no RepoLogMiner, no microVM layer.

- sandboy: env_clear() + env_allow allowlist before exec — closes a real gap
  found while designing the gate's credential-canary test (sandboy previously
  passed its full launcher environment straight through to the wrapped
  command). Default (no env_allow) is deny-all.
- sandboy: new `probe` subcommand — reuses the same landlock_status/
  apply_seccomp code `run` uses against a synthetic default policy, reports
  host capability (Landlock ABI/status via both the crate and a raw
  landlock_create_ruleset(..., VERSION) syscall, seccomp, no_new_privs,
  close_range, kernel/uid/container signals) as JSON. Always exits 0; a
  caller decides pass/fail. Distinct from the still-unimplemented `run
  --report` (per-step outcome) noted in the README.
- tests/feasibility/: argv-only (no shell strings) synthetic hostile
  fixtures — filesystem escape (incl. symlink, unix-socket-bind), seccomp
  syscall denial with fork/child/grandchild/compiler-tree-shaped process
  trees and an unsandboxed baseline to isolate seccomp's actual
  contribution, credential-canary leak scanning, and TCP port-scope network
  tests — plus run_gate.py, which also runs the existing tests/demo.sh smoke
  and writes sandboy-host-capabilities.json, sandboy-feasibility-report.json,
  sandbox-enforcement-matrix.json, and sandboy-feasibility-summary.md.
  Documents (without live-testing against a real address) the two
  structural network gaps: no UDP coverage, and TCP scoped by port only,
  never destination address — so a metadata-endpoint port collision is not
  something Landlock can distinguish.
- .github/workflows/sandboy-feasibility-gate.yml: read-only, pinned actions,
  no secrets, ubuntu-24.04, injects fake canary env vars, always uploads the
  four reports as CI artifacts even when the gate fails.
- policy.rs unit tests for env_allow parsing/default and seccomp_deny
  resolution/hard-error-on-unknown-name.

Locally (this dev container) the gate fails closed end-to-end: Landlock is
not implemented in this kernel, so every sandboxed invocation is refused
before it ever execs — which is exactly the scenario this gate exists to
tell apart from a real ubuntu-24.04 runner. Real verification happens in CI.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146FG6sjNiVaQEbCUZkzSy6
@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@PhysShell, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 28 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 9b85189d-de91-4f96-8972-cdff28c6417b

📥 Commits

Reviewing files that changed from the base of the PR and between 53bf05a and e925058.

⛔ Files ignored due to path filters (1)
  • sandboy/Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (11)
  • .github/workflows/sandboy-feasibility-gate.yml
  • sandboy/.gitignore
  • sandboy/README.md
  • sandboy/policy.example.toml
  • sandboy/src/main.rs
  • sandboy/src/policy.rs
  • sandboy/tests/feasibility/fixtures/env_probe.py
  • sandboy/tests/feasibility/fixtures/fs_probe.py
  • sandboy/tests/feasibility/fixtures/net_probe.py
  • sandboy/tests/feasibility/fixtures/syscall_probe.py
  • sandboy/tests/feasibility/run_gate.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/qodec-benchmark-v2-representative-logs-n2-dqn3a1

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…assification

Two real bugs found by actually running the gate on a GitHub-hosted
ubuntu-24.04 runner (Landlock ABI v7, fully_enforced there):

- python_fs_ro_dirs() walked dirname() up from the interpreter path with no
  floor, so a shallow system `/usr/bin/python3` reached "/" itself within a
  few hops and silently allowlisted the ENTIRE filesystem for reads. Every
  fs_read_* denial probe passed regardless of Landlock, because the policy
  itself was wrong, not because confinement failed — writes were correctly
  denied (fs_ro only grants read+exec) but reads were not exercised at all.
  Fixed: never add "/" (or an empty path) to the walk-up, and stop the walk
  before reaching it.
- The credential-canary controls compared env_probe.py's own CLEAN/LEAK
  judgement against the generic ALLOWED/DENIED classifier, so every clean
  run was misreported as "fail" in the per-control table (the overall
  credential_canaries_absent acceptance criterion was unaffected — it reads
  the independent leak scan directly, not pass_fail). Added a
  category=="credential_canary" branch that classifies on CLEAN/LEAK.

Also ruff/mypy cleanup (line length, unused imports, iterable unpacking) —
the repo's existing CI lint job runs ruff across the whole tree, including
this new script.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146FG6sjNiVaQEbCUZkzSy6

Copy link
Copy Markdown
Owner Author

Sandboy S0 result: PASS

Real run on ubuntu-24.04 (GitHub-hosted): workflow run 29350379408, commit e925058. All 25 tested controls pass; 2 informational findings; 1 gap documented (not live-tested).

Host: kernel_release=6.17.0-1018-azure, arch=x86_64, Landlock ABI v7, fully_enforced.

Category Result
Filesystem (read/write/symlink-escape/unix-socket/HOME/.ssh/unrelated-workspace) 9/9 pass
seccomp × process depth (ptrace/mount/setns × self/child/grandchild/compiler-tree) 12/12 pass
Credential canaries (fake GITHUB_TOKEN/AZURE_DEVOPS_PAT/NUGET_AUTH_TOKEN/AWS_SECRET_ACCESS_KEY/SSH_AUTH_SOCK) absent everywhere
Network (TCP port-scope allow/deny/wrong-port) 3/3 pass
Network (UDP — no Landlock coverage) confirmed as documented gap, not a regression
Network (cloud-metadata address-vs-port) documented limitation, not live-tested (Layer 3 unbuilt)

Note on the first run (commit a87f5a4): it reported FAIL because of two bugs in the test harness itself (not Sandboy) — an fs-allowlist path-walk that reached / on this runner's shallow python3 path (making every read probe pass regardless of confinement), and a canary-check status-code mismatch. Both fixed in e925058; see that commit message for detail.

Decision per the N2 Sandboy-dogfooding addendum: gate passes → accepted Sandboy commit for N2 is e925058ddea405b5821fc0aed4882c76650dcbe9, no VM layer implemented (none required — the GitHub-hosted runner supplies the disposable-VM boundary).

Artifacts: sandboy-s0-feasibility-reports (host-capabilities, feasibility-report, enforcement-matrix, summary).

Still DO NOT MERGE — this PR remains CI/review-only for the S0 probe.


Generated by Claude Code

@PhysShell PhysShell changed the title sandboy: S0 GitHub-hosted-runner feasibility gate (DO NOT MERGE — CI probe only) sandboy: S0 GitHub-hosted-runner feasibility gate — ACCEPTED Jul 14, 2026
@PhysShell
PhysShell marked this pull request as ready for review July 14, 2026 19:00
@PhysShell
PhysShell merged commit 2baa528 into main Jul 14, 2026
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants