Skip to content

feat(insights): audit-record checks — the control-plane log against the state it narrates - #309

Open
ophiocus wants to merge 2 commits into
theam:mainfrom
ophiocus:feat/audit-integrity-observe
Open

ophiocus wants to merge 2 commits into
theam:mainfrom
ophiocus:feat/audit-integrity-observe

Conversation

@ophiocus

@ophiocus ophiocus commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

TL;DR — Read-only checks that the audit log agrees with the state it describes, in both directions: every API key has its key.issued, every revocation its key.revoked, every archived project its project.archived, every budget its budget.updated — and every such event names state that exists and is in that condition. One module that returns findings; it surfaces nothing, gates nothing, needs no migration. 16 tests. Opens with a question rather than a claim.

The question first. audit_events came through #289 as a plain append log — actor, action, target, payload, createdAt, indexed by time. It has no ordering of its own, so an event removed from the middle leaves no trace the log itself can show; a gap and a quiet hour look the same. That may well be the intended shape now that trust is architectural and the log is a record rather than an instrument — in which case this PR is a small read that costs nothing, and the question is answered. If the log is meant to stand as the account of who did what to the control plane, then what follows is the least that makes a gap visible without giving the log any machinery of its own.

Sibling of #308 (the delivery-record checks), same shape, same rule: every check enumerates from a ledger other than the one it verifies. Here the second ledger is the state the log narrates.

What the log already gives us

Three explicit audits carry the entity as target — key.issued and key.revoked with {type: "key", id} (me-members-roles.ts), auth.logout with the principal. Fifteen more arrive through the generic onResponse hook on routes that declare auditAction; those carry the project as target when the route has one, otherwise the route path. So the log names an entity for keys, and a project for project.archived and budget.updated. Those are the pairs this reads.

state (enumerate from) expects in the log and back
api_keys row key.issued for that id key.issued names a key that exists
api_keys.revoked_at set key.revoked for that id key.revoked names a key that is revoked
projects.status = archived project.archived for that project project.archived names a project that is archived
project_budgets row budget.updated for that project budget.updated names a project with a budget

What it does not claim

  • member.*, role.*, repo.*, project.created, org.updated audit with the route as target, not the entity, so they cannot be paired by id. They are outside this read, stated rather than approximated.
  • auth.logout leaves no durable state (it clears a cookie); nothing to pair.
  • State written outside audited routes — the insecure dev login's default org and membership, the seed — has no event by construction. On a dev instance that shows up as findings; it is a fact about the writer, and the module says so.
  • Ordering: the log has createdAt only. This PR does not add a sequence or a chain; it makes gaps visible from the state side, which is the only side that survives a deletion.

Proof — every shape is "break the pair, watch the report"

services/api/test/audit-record.integration.test.ts (10 tests, scoped fixture orgs):

mutate the report says
key issued with its event; key revoked with both events; archived project with its event; budget with its event 0 findings
key exists, log never saw it issued keys: state-without-event (key.issued)
key revoked, log never saw the revocation keys: state-without-event (key.revoked)
key.issued in the log, key row deleted keys: event-without-state — the deletion is visible from the log that still expects the row
key.revoked in the log, key still active keys: event-without-state
project archived, log never saw it projects: state-without-event
project.archived in the log, project is active projects: event-without-state — "the project is active"
budget set, log never saw it budgets: state-without-event
budget.updated in the log, no budget budgets: event-without-state
walk with pageSize: 2 same findings, same counts
the other org never inspected

Review round — @adrian-lorenzo's findings

services/api/test/audit-record-review.integration.test.ts (6 tests). Fixtures are modelled on the writers: request.audit writes one key.issued per issue with the key id as target; the onResponse hook writes auditAction only on 2xx; bundled roles come from the seed, never from a migration.

finding now proof
with pageSize: 2 the matching-event query still loaded 1,002 events matchingActions selects distinct (action, target id) pairs server-side and caps the result at actions × targets — the most distinct pairs there can be, so the cap never truncates; exported so the bound is observable 1,002 duplicate key.issued rows → 1 pair; n targets × k actions never exceed n × k rows; the pre-fix query shape run raw returns 1,002 (kept in the suite as the red side); the report is unchanged at pageSize: 2
the suite fails on a fresh database: role_bundled_viewer missing the fixture calls seed() before inserting keys — the convention budget-routes.integration.test.ts already uses no assertion changed

Found while enumerating the writers: DELETE /v1/projects/:projectId updates by id without checking a row was hit and answers 200 either way, so the hook audits project.archived for an id that may not exist. The checker reports that as event-without-state ("no such project exists") — a signal about the route, kept and tested both ways (existing project: clean; unknown id: reported). The sibling PATCH route throws notFound in the same situation; DELETE should too. Follow-up, not this PR.

Both suites: 16/16, three consecutive runs; tsc and biome clean on the rebased branch.

On the open question — who calls this. Proposal, not a claim: an insights read behind analytics:read that returns the report for the caller's org, with no scheduling and no surface in this PR. If a CLI doctor sub-check fits the operator story better, that works just as well and I would rather follow the shape you prefer. Either caller would also justify an expression index on target->>'id'; today each page is a scan of the org's log, and DISTINCT bounds what is returned, not what is read.

Bounded by construction

Read-only; keyset-paginated over api_keys and over the paired events; projects and budgets are org-scoped and read whole; every lookup is an IN over the page and returns distinct pairs capped at the page's size. Counts are reported over everything read — checkedKeys, checkedProjects, checkedBudgets, checkedEvents — never over survivors.

Non-goals

No sequence column, no hash, no chain, no trigger, no gating, no surface, no migration. The log stays exactly what it is; this only reads it against the state beside it.

Files

  • services/api/src/insights/audit-record.tsverifyAuditRecord, matchingActions (new)
  • services/api/test/audit-record.integration.test.ts
  • services/api/test/audit-record-review.integration.test.ts

Verify locally

docker compose -f docker-compose.dev.yml up -d postgres
pnpm --filter @facility/api exec vitest run test/audit-record.integration.test.ts test/audit-record-review.integration.test.ts
pnpm --filter @facility/api test
pnpm typecheck && pnpm lint

🤖 Generated with Claude Code

@adrian-lorenzo adrian-lorenzo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for exploring this approach.

Before merging, please bound the matching-event query and make the test fixture seed its own role. With pageSize: 2, my probe still loaded 1,002 events in one query; the focused suite also fails on a fresh database because role_bundled_viewer is missing.

I’d also like us to agree on how operators will use this—the checker currently has no caller.

ophiocus added a commit to ophiocus/facility that referenced this pull request Sep 14, 2026
…lf-seeding fixture

Addresses the two code findings on theam#309.

- Bounded reads: `matchingActions` selects distinct (action, target id)
  pairs server-side and caps the result at actions × targets, the most
  distinct pairs there can be, so a page of n keys never loads more than
  2n rows however often the log repeats a pair. `actionsFor` builds its
  map from it. Exported so the bound is observable in a test.
- Self-seeding fixture: the suite calls `seed()` before inserting keys.
  The bundled roles the keys reference are written by the seed, never by
  a migration, so a fresh database has none. Same convention as
  budget-routes.integration.test.ts.

Tests: no assertion changed in the existing suite. New suite
audit-record-review.integration.test.ts (6 tests): 1,002 duplicate
key.issued rows return one pair; n targets × k actions never exceed
n × k rows; the pre-fix query shape run raw returns 1,002 rows; the report
is unchanged at pageSize 2; and the archive route modelled both ways —
DELETE on an existing project is clean, DELETE on an unknown id answers
200 and audits an archive it did not perform, which the report names.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@ophiocus
ophiocus force-pushed the feat/audit-integrity-observe branch from d35df83 to b332d3d Compare September 14, 2026 14:40
@ophiocus

Copy link
Copy Markdown
Contributor Author

Thanks @adrian-lorenzo. Both addressed in b332d3d: the matching-event read is distinct and capped at actions × targets, and the fixture seeds its own role. On the caller, the body now carries a proposal (an insights read behind analytics:read, or a doctor sub-check if that fits operators better); I would rather follow the shape you prefer than assume one. self-host-build failed on the same moby source fetch as #308; a rerun should clear it.

ophiocus and others added 2 commits September 15, 2026 09:01
…he state it narrates

Read-only checks over the audit log, returning findings; nothing gates and
nothing is surfaced.

`audit_events` is an append log ordered by createdAt only, so on its own it
cannot tell a removed event from a quiet hour. Where an event names its
entity, the log and the state can be read against each other both ways:

- keys: every API key has a key.issued event; every revoked key has a
  key.revoked event; every such event names a key that exists and is in
  that state.
- projects: an archived project has a project.archived event, and that
  event names a project that is archived.
- budgets: a project with a budget has a budget.updated event, and that
  event names a project that has one.

Not evaluable and not claimed: actions whose target is the route rather
than the entity (member.*, role.*, repo.*, project.created, org.updated),
actions that leave no durable state (auth.logout), and state written by
bootstrap paths outside audited routes. The module says so.

Keyset-paginated over keys and over paired events; projects and budgets
are org-scoped and read whole; every lookup is an IN over the page.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…lf-seeding fixture

Addresses the two code findings on theam#309.

- Bounded reads: `matchingActions` selects distinct (action, target id)
  pairs server-side and caps the result at actions × targets, the most
  distinct pairs there can be, so a page of n keys never loads more than
  2n rows however often the log repeats a pair. `actionsFor` builds its
  map from it. Exported so the bound is observable in a test.
- Self-seeding fixture: the suite calls `seed()` before inserting keys.
  The bundled roles the keys reference are written by the seed, never by
  a migration, so a fresh database has none. Same convention as
  budget-routes.integration.test.ts.

Tests: no assertion changed in the existing suite. New suite
audit-record-review.integration.test.ts (6 tests): 1,002 duplicate
key.issued rows return one pair; n targets × k actions never exceed
n × k rows; the pre-fix query shape run raw returns 1,002 rows; the report
is unchanged at pageSize 2; and the archive route modelled both ways —
DELETE on an existing project is clean, DELETE on an unknown id answers
200 and audits an archive it did not perform, which the report names.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@ophiocus
ophiocus force-pushed the feat/audit-integrity-observe branch from b332d3d to 45315f0 Compare September 15, 2026 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants