feat: ongoing monitoring engine (continuous re-screening + alerts) - #158
Merged
Merged
Conversation
munisp
added a commit
that referenced
this pull request
Sep 14, 2026
…gration) (#160) - WP1 (#154): entitySearchRouter import + appRouter registration - WP2 (#158): monitoringRouter import + appRouter registration - WP3 (#155): subjectPortalRouter + computeDataCompleteness imports; subjectPortal registration; removed routers.ts-local getFallbackSuggestion (now shared in server/dataCompleteness.ts); getDataCompleteness delegates to computeDataCompleteness; consentPurposeEnum gains consumer_self_check; subjectAccessTokens/subjectDisputes pgTable declarations (matches drizzle/0023_subject_portal.sql) - WP4 (#157): shareableReportsRouter + selfServiceBillingRouter imports + registrations; reportShareLinks/planSignups pgTable declarations (matches drizzle/0024_share_links_and_plan_signups.sql) - WP5 (#156): lookup.phone procedure (gatewayFetch /v1/phone/:number, validated input) Co-authored-by: bis-integration <integration@bis.local>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Ongoing-monitoring (continuous re-screening) engine for investigation subjects:
drizzle/0025_monitoring.sql— three raw-SQL tables (consumed viapgonly, same precedent as the informal_verification tables in 0010; intentionally not indrizzle/schema.ts):monitoring_enrollments(tenant-scoped subject subscription: identifiers jsonb,list_set text[]⊆ {sanctions,pep,watchlist},frequencydaily|weekly|monthly,statusactive|paused|cancelled,last_run_at/next_run_at,baseline_snapshotjsonb, partial unique index = one live enrollment per subject+investigation+tenant, due-index onnext_run_at WHERE status='active')monitoring_runs(immutable per-enrollment run history:resultno_change|change_detected|error, snapshot, error)monitoring_alerts(tenant-scoped:alert_typenew_hit|status_change|removed_hit, severity, delta jsonb, acknowledged_at/by)drizzle/meta/_journal.json(idx 25; main already ends at 0024_share_links_and_plan_signups, hence 0025).server/monitoring.ts—monitoringRouter+ pure core, modelled oninformalVerification.ts/shareableReports.ts(zod inputs,ctx.user+ctx.tenantIdtenant guard on every procedure,getPgPooltransactions, HMACwriteAuditLog,publishEvent):enroll(writeProcedure): validates the investigation belongs to the caller's tenant (FOR SHARE), runs the subject's current screening via the existing pipelines (gatewayGET /v1/sanctions/:name,GET /v1/pep/:name— the same entry points askycScheduledRerunExecutor; watchlist via the ngScreening verify pipelinecheck_type=efcc_watchlist), stores a baseline snapshot of stable hit IDs (sha256 of list|sourceList|matchedName) + status bands, setsnext_run_atper frequency. Transactional + audit +MONITORING_ENROLLEDevent. Fail-closed: if any requested provider is unavailable, enroll returnsSERVICE_UNAVAILABLEand stores nothing.pause/resume/cancel— state machine (active↔paused, cancel terminal), each audited; resume recomputesnext_run_at.list,getEnrollment(with last N runs),getAlerts(unacknowledged first) — all tenant-scoped.acknowledgeAlert— admin/supervisor only, tenant-scoped, single-ack (conflict on re-ack), audited.runSubjectScreening,diffSnapshots,nextRunAt,scoreBand,alertSeverity— no list matching is reimplemented; provider responses are only normalized into comparable snapshots.server/monitoringScheduler.ts— every 60s claims due enrollmentsWHERE status='active' AND next_run_at <= now() ... FOR UPDATE SKIP LOCKED(leased-claim precedent frompaymentIntentOutbox.ts, safe for multiple BFF replicas). Per enrollment in ONE transaction: re-run screening → diff vs baseline → insertmonitoring_runs→ on delta insertmonitoring_alerts+writeAuditLog→ advancebaseline_snapshot+next_run_at. After COMMIT,publishEvent('MONITORING_ALERT', …)per alert (severity critical for a new sanctions hit, high for new PEP/watchlist, medium for status change, low for removed hit). Per-enrollment failure rolls back and records aresult='error'run in a fresh transaction (baseline untouched, schedule advanced — fail-closed, no hot loop); the loop never crashes.server/_core/index.ts— starts the scheduler afterstartPaymentIntentOutboxDispatcher(), guarded byMONITORING_SCHEDULER_ENABLED(default on; set to"false"to disable).server/monitoring.test.ts— 24 vitest cases (below).Why
Sanctions/PEP/watchlist posture changes after a report is issued. This gives tenants a durable, auditable subscription that re-screens enrolled subjects on a daily/weekly/monthly cadence against the same provider pipelines used at onboarding, detects deltas against a stored baseline, and fans out severity-ranked alerts — without reimplementing any matching logic.
server/routers.tsregistration (NOT in this branch)server/routers.tsis 374KB, too large for the MCP file-push API, so the 2-line registration is not in this branch. Apply exactly:1. Import — immediately after line 142 (
import { piiKeyCustodyRouter } from "./piiKeyCustody";):2. Register — at the END of the
appRouterobject, immediately afterkycDocumentEvidence: kycDocumentEvidenceRouter,(~line 7692, before the closing});):Both anchors were verified unique in
server/routers.tson currentmain(exactly one occurrence each), and the patch was applied locally for all verification below. The stale 265KB duplicate at the repo root was not touched.How tested (real output, run against the exact content of this branch + the routers.ts patch above)
Coverage: diff detection (new hit / removed hit / status change / no-change), frequency→next_run_at math (daily +24h, weekly +7d, monthly +1 calendar month), enroll baseline + tenant isolation (cross-tenant investigation → FORBIDDEN; cross-tenant reads → NOT_FOUND/empty), fail-closed enroll on provider outage (SERVICE_UNAVAILABLE, nothing stored), pause→resume→cancel state machine, scheduler critical alert +
MONITORING_ALERTfan-out on new sanctions hit, no-change run, removed-hit low-severity alert, provider-outage →result='error'run without crashing, paused/not-due enrollments skipped,acknowledgeAlertauthZ (analyst → FORBIDDEN; supervisor OK; re-ack → CONFLICT; cross-tenant → CONFLICT), unacknowledged-first alert ordering.The pg pool is replaced by a stateful in-memory handler executing the real SQL strings (same precedent as
share-subscribe.test.ts); external HTTP boundaries (gateway, verify, event processor) are intercepted with a stubbed fetch. All business logic under test is production code. No mocks/stubs in the shipped paths.Risks / notes
services/gateway). Enrollments includingpepinlist_setwill fail-closed (enroll → SERVICE_UNAVAILABLE; scheduled runs →result='error') until a provider is configured. This is intentional — a monitored subject is never treated as "clear" by an unavailable provider.gen_random_uuid(), partial indexes, BEGIN/COMMIT); no live PostgreSQL was available in the build sandbox. Recommend running it against staging before merge.MONITORING_ALERTevents are published after COMMIT; a crashed BFF between COMMIT and publish loses the event, but themonitoring_alertsrows are durable — a consumer reconciliation job can replay from the table if desired.server/routers.tsregistration must be applied manually (above) before the tRPC routes are live; the scheduler and migration are functional independently.Supersedes the stale attempt on
feat/ongoing-monitoring(which targeted the now-conflicting0024migration number).