Skip to content

feat: ongoing monitoring engine (continuous re-screening + alerts) - #158

Merged
munisp merged 4 commits into
mainfrom
feat/ongoing-monitoring-v2
Sep 13, 2026
Merged

munisp merged 4 commits into
mainfrom
feat/ongoing-monitoring-v2

Conversation

@munisp

@munisp munisp commented Sep 13, 2026

Copy link
Copy Markdown
Owner

What

Ongoing-monitoring (continuous re-screening) engine for investigation subjects:

  • drizzle/0025_monitoring.sql — three raw-SQL tables (consumed via pg only, same precedent as the informal_verification tables in 0010; intentionally not in drizzle/schema.ts):
    • monitoring_enrollments (tenant-scoped subject subscription: identifiers jsonb, list_set text[] ⊆ {sanctions,pep,watchlist}, frequency daily|weekly|monthly, status active|paused|cancelled, last_run_at/next_run_at, baseline_snapshot jsonb, partial unique index = one live enrollment per subject+investigation+tenant, due-index on next_run_at WHERE status='active')
    • monitoring_runs (immutable per-enrollment run history: result no_change|change_detected|error, snapshot, error)
    • monitoring_alerts (tenant-scoped: alert_type new_hit|status_change|removed_hit, severity, delta jsonb, acknowledged_at/by)
    • journal entry appended to drizzle/meta/_journal.json (idx 25; main already ends at 0024_share_links_and_plan_signups, hence 0025).
  • server/monitoring.tsmonitoringRouter + pure core, modelled on informalVerification.ts/shareableReports.ts (zod inputs, ctx.user + ctx.tenantId tenant guard on every procedure, getPgPool transactions, HMAC writeAuditLog, publishEvent):
    • enroll (writeProcedure): validates the investigation belongs to the caller's tenant (FOR SHARE), runs the subject's current screening via the existing pipelines (gateway GET /v1/sanctions/:name, GET /v1/pep/:name — the same entry points as kycScheduledRerunExecutor; watchlist via the ngScreening verify pipeline check_type=efcc_watchlist), stores a baseline snapshot of stable hit IDs (sha256 of list|sourceList|matchedName) + status bands, sets next_run_at per frequency. Transactional + audit + MONITORING_ENROLLED event. Fail-closed: if any requested provider is unavailable, enroll returns SERVICE_UNAVAILABLE and stores nothing.
    • pause / resume / cancel — state machine (active↔paused, cancel terminal), each audited; resume recomputes next_run_at.
    • list, getEnrollment (with last N runs), getAlerts (unacknowledged first) — all tenant-scoped.
    • acknowledgeAlert — admin/supervisor only, tenant-scoped, single-ack (conflict on re-ack), audited.
    • Exported pure functions runSubjectScreening, diffSnapshots, nextRunAt, scoreBand, alertSeverity — no list matching is reimplemented; provider responses are only normalized into comparable snapshots.
  • server/monitoringScheduler.ts — every 60s claims due enrollments WHERE status='active' AND next_run_at <= now() ... FOR UPDATE SKIP LOCKED (leased-claim precedent from paymentIntentOutbox.ts, safe for multiple BFF replicas). Per enrollment in ONE transaction: re-run screening → diff vs baseline → insert monitoring_runs → on delta insert monitoring_alerts + writeAuditLog → advance baseline_snapshot + next_run_at. After COMMIT, publishEvent('MONITORING_ALERT', …) per alert (severity critical for a new sanctions hit, high for new PEP/watchlist, medium for status change, low for removed hit). Per-enrollment failure rolls back and records a result='error' run in a fresh transaction (baseline untouched, schedule advanced — fail-closed, no hot loop); the loop never crashes.
  • server/_core/index.ts — starts the scheduler after startPaymentIntentOutboxDispatcher(), guarded by MONITORING_SCHEDULER_ENABLED (default on; set to "false" to disable).
  • server/monitoring.test.ts — 24 vitest cases (below).

Why

Sanctions/PEP/watchlist posture changes after a report is issued. This gives tenants a durable, auditable subscription that re-screens enrolled subjects on a daily/weekly/monthly cadence against the same provider pipelines used at onboarding, detects deltas against a stored baseline, and fans out severity-ranked alerts — without reimplementing any matching logic.

⚠️ Required manual step — server/routers.ts registration (NOT in this branch)

server/routers.ts is 374KB, too large for the MCP file-push API, so the 2-line registration is not in this branch. Apply exactly:

1. Import — immediately after line 142 (import { piiKeyCustodyRouter } from "./piiKeyCustody";):

import { monitoringRouter } from "./monitoring";

2. Register — at the END of the appRouter object, immediately after kycDocumentEvidence: kycDocumentEvidenceRouter, (~line 7692, before the closing });):

  monitoring: monitoringRouter,

Both anchors were verified unique in server/routers.ts on current main (exactly one occurrence each), and the patch was applied locally for all verification below. The stale 265KB duplicate at the repo root was not touched.

How tested (real output, run against the exact content of this branch + the routers.ts patch above)

$ npx vitest run server/monitoring.test.ts
 ✓ server/monitoring.test.ts (24 tests) 59ms

 Test Files  1 passed (1)
      Tests  24 passed (24)
   Duration  805ms

$ npx tsc --noEmit
TSC_EXIT=0

$ npx vitest run server/share-subscribe.test.ts server/investigations.screening.test.ts   # regression
 Test Files  2 passed (2)
      Tests  31 passed (31)

Coverage: diff detection (new hit / removed hit / status change / no-change), frequency→next_run_at math (daily +24h, weekly +7d, monthly +1 calendar month), enroll baseline + tenant isolation (cross-tenant investigation → FORBIDDEN; cross-tenant reads → NOT_FOUND/empty), fail-closed enroll on provider outage (SERVICE_UNAVAILABLE, nothing stored), pause→resume→cancel state machine, scheduler critical alert + MONITORING_ALERT fan-out on new sanctions hit, no-change run, removed-hit low-severity alert, provider-outage → result='error' run without crashing, paused/not-due enrollments skipped, acknowledgeAlert authZ (analyst → FORBIDDEN; supervisor OK; re-ack → CONFLICT; cross-tenant → CONFLICT), unacknowledged-first alert ordering.

The pg pool is replaced by a stateful in-memory handler executing the real SQL strings (same precedent as share-subscribe.test.ts); external HTTP boundaries (gateway, verify, event processor) are intercepted with a stubbed fetch. All business logic under test is production code. No mocks/stubs in the shipped paths.

Risks / notes

  • Gateway PEP endpoint currently returns 503 when no live PEP provider is configured (existing fail-closed behavior in services/gateway). Enrollments including pep in list_set will fail-closed (enroll → SERVICE_UNAVAILABLE; scheduled runs → result='error') until a provider is configured. This is intentional — a monitored subject is never treated as "clear" by an unavailable provider.
  • Migration was validated by inspection against the 0010/0014 raw-SQL precedents (gen_random_uuid(), partial indexes, BEGIN/COMMIT); no live PostgreSQL was available in the build sandbox. Recommend running it against staging before merge.
  • MONITORING_ALERT events are published after COMMIT; a crashed BFF between COMMIT and publish loses the event, but the monitoring_alerts rows are durable — a consumer reconciliation job can replay from the table if desired.
  • server/routers.ts registration must be applied manually (above) before the tRPC routes are live; the scheduler and migration are functional independently.

Supersedes the stale attempt on feat/ongoing-monitoring (which targeted the now-conflicting 0024 migration number).

@munisp
munisp merged commit 43e9f2e into main Sep 13, 2026
8 of 10 checks passed
munisp added a commit that referenced this pull request Sep 14, 2026
…gration) (#160)

- WP1 (#154): entitySearchRouter import + appRouter registration
- WP2 (#158): monitoringRouter import + appRouter registration
- WP3 (#155): subjectPortalRouter + computeDataCompleteness imports; subjectPortal registration; removed routers.ts-local getFallbackSuggestion (now shared in server/dataCompleteness.ts); getDataCompleteness delegates to computeDataCompleteness; consentPurposeEnum gains consumer_self_check; subjectAccessTokens/subjectDisputes pgTable declarations (matches drizzle/0023_subject_portal.sql)
- WP4 (#157): shareableReportsRouter + selfServiceBillingRouter imports + registrations; reportShareLinks/planSignups pgTable declarations (matches drizzle/0024_share_links_and_plan_signups.sql)
- WP5 (#156): lookup.phone procedure (gatewayFetch /v1/phone/:number, validated input)

Co-authored-by: bis-integration <integration@bis.local>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant