fix: unblock master (3 regressions) + fleet-pat-health watched 3 of 9 PAT repos + census roster reconcile

[github-actions]

📋 Description

Scope grew after this PR was opened. The urgent part is the first section, not the last. master went red at 05:07Z today and stayed red; three separate regressions landed inside one nine-minute merge window. Each is reproduced locally against PostgreSQL 16 and confirmed green in this PR's CI.

1. Three master regressions (the reason this is urgent)

#	Defect	Introduced by	Confirmed green by
1	#809's BOLA fixture declared otherOrgId, assigned it a UUID and used it as a foreign key — but never inserted the organization row	d44ee551 (#809) 05:06Z	Backend tests (Node 24.x)
2	#680 changed migration 022 to adopt a divergent platform org, and left rootOrgAbsentGuards.test.ts asserting the contract it had just replaced	73c30aae (#680) 05:15Z	same job, coverage step
3	applications-service migration 011 threw unconditionally when the root org was absent — including when there was nothing to backfill	d44ee551 (#809) 05:06Z	Applications service (unit + integration)

1 — backend/tests/apps.test.ts

otherOrgId was a live FK reference to a row that was never created, so every insert against it violated apps_organization_id_foreign. Fixed by actually inserting the org, and cleaning it up in afterAll alongside scopedOrgId.

2 — backend/tests/rootOrgAbsentGuards.test.ts

Before #680, migration 022 hardcoded ROOT_ORG_ID, which crashlooped fuzefront-backend and fuzefront-security on 2026-08-16 against a prod DB whose platform-root org has a different id (the 2026-07-29 rebuild). #680 correctly changed 022 to resolve the root org the way ensureRootPortal() does — prefer ROOT_ORG_ID, else the oldest type='platform' row — but the guard test still demanded a skip for exactly that fixture.

The assertion is now the post-#680 contract: the backfill runs, bound to the adopted org's id and never to the constant.

expect(inserts[0].bindings).toContain('legacy-platform-id')
expect(inserts[0].bindings).not.toContain(ROOT_ORG_ID)

Flipping an assertion removes coverage unless something replaces it, so the genuine "no platform org exists at all" case is added as its own test — with users present deliberately, since the pre-existing empty-fixture test has none and therefore cannot distinguish "skipped the backfill" from "had nobody to backfill".

3 — backend/applications/src/migrations/011_apps_organization_id_not_null.ts

The root org is seeded by a different deployable's migration tree (backend/src's 015). This service's 011 threw whenever that row was absent — and a throwing migration is a boot crashloop, not a warning. tests/migrations.idempotency.integration.test.ts runs precisely this tree against a bare schema and expects a clean no-op, so it failed too.

The throw is narrowed to the case that warrants it: root org absent and org-less rows actually present to backfill. Otherwise it logs and returns.

Two things worth stating explicitly, because both are easy to get wrong:

• "It self-heals on the next boot" would be false. Knex records a migration as applied and never re-runs it. What actually closes the gap is the sibling migration against the same shared table — backend/src's 026_apps_organization_id_not_null, in the tree that also owns 015 and therefore always has the root org by the time it runs.
• The skip branch deliberately sets neither DEFAULT nor NOT NULL. A DEFAULT pointing at a non-existent org would reintroduce the exact FK hazard the guard exists to prevent.

The invariant the query-side half depends on still holds: service.ts's list() may drop its organization_id IS NULL arm only once no row can be null, and the skip fires precisely when there are zero org-less rows — so nothing that currently appears can silently disappear.

2. fleet-pat-health.yml — the original change

This workflow is the fleet's only detector for a GH_RELEASE_PAT expiry. Its own header says "Add new fleet repos here when they receive GH_RELEASE_PAT." Nine repos hold the secret; FLEET_REPOS named three. The other six had no expiry detection at all — the single thing this workflow exists to provide, absent for two thirds of its subjects.

Repo	Holds GH_RELEASE_PAT	Watched before	Watched now
FuzeX	✅	✅	✅
FuzeFinance	✅	✅	✅
FuzeExecutive	✅	✅	✅
FuzePlan	✅	❌	✅
FuzeCall	✅ (today)	❌	✅
FuzeDeploy	✅ (today)	❌	✅
FuzeMerchandize	✅ (today)	❌	✅ =build-and-push.yml
FuzeFront	✅	—	deliberately excluded
FuzeSDLC	✅	—	deliberately excluded

FuzeCall, FuzeDeploy and FuzeMerchandize were provisioned the credential today by FuzeSDLC's provision-secrets.yml (run 32948657038 — 3× SET GH_RELEASE_PAT), so they are added in the same change rather than left to be noticed later. FuzePlan already held it and was simply never listed.

The list is not hand-assembled. It is exactly the repos whose own workflows reference secrets.GH_RELEASE_PAT to bump a prod tag, from the fleet-wide survey in provision_secrets.py.

Two exclusions, on purpose. FuzeFront and FuzeSDLC hold the secret but neither uses it to cut a release — FuzeFront for this health check itself, FuzeSDLC for secret provisioning. Listing them would query a release workflow that does not exist and report a failure that is not one.

The per-repo workflow override isn't cosmetic. That same hazard is live for a repo that is in scope: FuzeMerchandize references the PAT from build-and-push.yml, not release.yml. Querying a workflow file a repo does not have returns the same Could not query error as a genuinely expired credential — indistinguishable in the alert issue. A false alarm here is worse than a missing one: it trains people to ignore the alert this workflow exists to raise. Entries now accept an optional =workflow.yml suffix, defaulting to WORKFLOW_FILE (release.yml) when absent.

izzywdev/FuzeCall                            -> release.yml
izzywdev/FuzeMerchandize=build-and-push.yml  -> build-and-push.yml

3. scripts/expected-portal-apps.json — the census roster disagreed with the live registry

The census kept reporting products as "missing" that are in fact registered under different slugs. Reconciled per the file's own _meta.notASlugMigrationWorklist rule — "If a live registry slug genuinely differs from an entry below, FIX THIS FILE to match the registry — never the other way around":

Roster said	Registry says	Action
keys	fuzekeys	roster corrected
fuzecontact	contact	roster corrected
fuzehub-ventures	fuzehub	roster corrected
(absent)	clock	added
fuzequality	fuzequality, activated	kept — see below

No slug is edited anywhere. PUT /apps/{slug} has no rename operation, so changing one registers a second app and strands the first.

Correction, made on this branch rather than left in history

An earlier commit here removed fuzequality from the roster on the premise that it was not registered. That premise was wrong, and commit 77096fa1 restores it. Live census run 32960147607 (2026-08-26 10:49:51Z) against app.fuzefront.com:

fuzequality  FuzeQuality  activated  FAIL  https://app.fuzefront.com/apps/fuzequality/assets/remoteEntry.js
  — remoteEntry returned 200 but is HTML (content-type 'text/html') — SPA fallback answering a 404 with 200

It is activated, and it is one of the four entries in BUILTIN_MANIFESTS (fuzesocial, fuzeagent, clock, fuzequality) — the only four whose slug comes from FuzeFront's own seed.

The consequence of dropping it was not cosmetic: this file is the only thing that makes a MISSING app detectable, so an app absent from both the registry and the roster is invisible to the census entirely. Removing a real product converts it from a reported failure into a blind spot — the exact failure mode the file exists to prevent.

To be precise about what restoring it claims: fuzequality is not healthy — it is the 200-serving-HTML row. The entry asserts the product is expected, so its brokenness keeps being counted rather than disappearing.

4. Two .gitignore entries

backend/database.sqlite (written by npm run db:init) and an unanchored coverage/ (jest writes backend/coverage/, which the existing anchored pattern missed). Both were showing up as untracked noise after any local test run.

🧪 Testing

• All three regressions reproduced locally against PostgreSQL 16 before fixing, and re-run green after.
• CI on head 77096fa1: 61 checks, 0 failures — Backend tests (Node 24.x), Applications service (unit + integration), Integration Tests, Playwright sign-in flow, OIDC plumbing and the full gate set all green.
• Roster change verified with the checker's own fixture suite: node --test scripts/check-portal-federation-health.selftest.mjs → 5/5 pass.
• fleet-pat-health YAML parses; the entry parser is exercised directly (plain entries → release.yml, suffixed entry → build-and-push.yml). workflow_dispatch is enabled so the widened list can be run on demand after merge. Baseline: run 32948915104 passed on the old 3-repo list today, including Require the release credential, so the PAT itself is valid.

🚨 Breaking Changes

None. No production code path changes behaviour: two test fixtures, one migration guard narrowed strictly toward fewer throws, one workflow's watch list, one JSON roster, two ignore rules.

Note on the auto-merge label

This PR was opened by repo automation on push, and that automation applied the auto-merge label — I did not. master is deployOnPush: true, so merging is a production deploy; #794 deliberately withheld the label for that reason, while #810 shipped with it earlier today. The owner has since instructed that this PR be merged.

[izzywdev]

Backend tests (Node 24.x) is red on this PR, and it is not this PR's

What is failing: Backend Tests (.github/workflows/backend-tests.yml), job Backend tests (Node 24.x), on head 03e803dc.

Why it is not this PR's. The diff is two files:

• .github/workflows/fleet-pat-health.yml — a scheduled/dispatch-only workflow that probes other repos' release runs
• scripts/expected-portal-apps.json — a data roster read only by check-portal-federation-health.mjs

Neither is imported, executed, or read by the backend test suite. There is no path from this diff to that job.

Confirmed red on the base branch. The last three Backend Tests runs on master all failed:

head	created	conclusion
73c30aae	2026-08-26T05:15:34Z	failure
455f6adc	2026-08-26T05:07:04Z	failure
d44ee551	2026-08-26T05:06:58Z	failure
02c2b449	2026-08-26T05:06:53Z	success
c1c4a1d7	2026-08-26T05:06:13Z	success

So master broke around 05:07Z today, roughly an hour before this branch existed.

No fix ported, because I could not identify one to port. I have deliberately not pushed a speculative change. Two honest limits on the diagnosis:

1. I have not isolated the causing merge. The transition sits inside a ~50-second window with several heads completing near-simultaneously, so the success/failure ordering above is concurrent runs on different commits, not sequential states of master. Attributing it to any one PR from this data would be a guess.
2. I could not retrieve the failing assertion. The job-logs endpoint redirects to blob storage, which this environment's network policy denies (CONNECT tunnel failed, 403), and tail-based retrieval returns only post-job output — service-container teardown and Postgres logs. The test-results-24.x artifact is on the same blocked host, and the check run carries no output summary.

What the retrievable portion does show is Postgres constraint violations that read as expected negative-path test output (FK violations on apps.organization_id, uniqueness on uq_personal_org_per_owner, portals_slug_unique, portal_domains_domain_unique) rather than an infrastructure fault — the database came up cleanly and served the whole run.

Not re-run. A re-run is only warranted to confirm a failure is not this PR's, and the base-branch evidence already establishes that; spending the one re-run here would tell us nothing new.

This is a pre-existing master regression that needs its own fix, and it blocks every PR in this repo — including this one — until it lands.

---

Generated by Claude Code

[izzywdev]

Correction + isolation: the master regression is #809, and I can now name it

My previous comment said the causing merge could not be isolated from the run data. That was wrong — it can be, and I'm correcting it here rather than leaving it standing.

I read the near-simultaneous run completions as concurrent runs on unrelated heads. They are not: they are three sequential merges, each triggering its own master run. Mapping commits to runs settles it:

Commit	PR	Migration added	Backend Tests
02c2b449	#808	010_clock_remoteentry_assets_segment_fix	✅ success
d44ee551	#809	011_apps_organization_id_not_null	❌ failure
455f6adc	#810	011_suspend_phantom_fuzequality_builtin	❌ failure
73c30aae	—	—	❌ failure

Green at #808's merge commit, red at #809's, red on every run since. The commit timestamps read 08:06/08:07 in +03:00, which is 05:06/05:07 UTC — matching the run times exactly. So my earlier attribution guess toward #810 was also wrong; #810 merged into an already-red master.

Also failing on this head, same root cause

Applications service (unit + integration), Integration Tests, and Notify Team are all red on 03e803dc for the same reason. Not commenting per check — one cause, one note.

Applications service has no usable base-branch comparison: it lives in ci.yml, whose last master run was 2026-08-20. The bisect above stands on Backend Tests, which does run on every master push.

A second, latent defect this surfaced

#809 and #810 each added a migration numbered 011, an hour apart, in the same tree:

backend/applications/src/migrations/011_apps_organization_id_not_null.ts        (#809)
backend/applications/src/migrations/011_suspend_phantom_fuzequality_builtin.ts  (#810)

Ordering between them is now settled by alphabetical tiebreak rather than intent. Worth fixing regardless of the test failure — neither PR could see the other.

What is still unproven, and why I am not pushing a fix

The bisect identifies which commit, not which assertion. #809 is not a careless change — it updated backend/tests/apps.test.ts (+160), app-registry.unit.test.ts and portal-catalog.integration.test.ts alongside the migration, and the migration itself sets DEFAULT ROOT_ORG_ID before SET NOT NULL, so org-less inserts are backfilled rather than rejected. The FK violations visible in the surviving log fragment supply an organization_id that is simply absent from organizations — consistent with negative-path tests, not obviously the failure.

I cannot read the failing assertion: the job-logs endpoint redirects to blob storage, which this environment's network policy denies (CONNECT tunnel failed, 403); the log is 1718 lines and tail-based retrieval returns only post-job teardown; the test-results-24.x artifact sits on the same blocked host; and the check run carries no output summary. There is no Postgres server available locally to reproduce against.

Guessing whether the fix belongs in #809's migration or in test fixtures — on someone else's change — is exactly the speculative push that costs a cycle and reviewer trust. So: isolation reported, no fix pushed.

This blocks every PR in the repo, not just this one. It also explains why #810's own migration never reached production: fuzequality is still activated there, four hours after it merged.

---

Generated by Claude Code