fix(apps): a 404 is not healthy — probe the module, not "did anything answer"

[github-actions]

📋 Description

checkAppHealth in backend/src/routes/apps.ts was:

const healthUrl = `${app.url}`
...
return response.status < 500   // "Consider 2xx, 3xx, 4xx as healthy"

So an app whose URL returned 404 was reported healthy. A production probe of all 16 registered remotes found 0 of them serving a module, and the portal calling every one green:

result	count	meaning
HTTP 404	12	nothing mounted at /apps/<slug>/ — no ingress route resolves
HTTP 503	3	route exists, no healthy backend
HTTP 200 + HTML	1	the host shell's own SPA fallback answering — the "200 that isn't"

A check that cannot separate "nothing is served here" from "this works" is not a health check. This one reported the entire federated surface green while none of it loaded.

🔧 Implementation Details

Backend only. No frontend, SDK, or deploy changes.

Health now means something different per integration type:

• module-federation — fetch the registered remote entry and require HTTP 200 and a JavaScript body. The content-type/HTML test is not belt-and-braces: it is the only thing that catches the SPA-fallback case, where the status line says 200 and the bytes are index.html. Same rule as scripts/check-federated-assets.mjs, deliberately, so the CI probe and the runtime check cannot disagree about what "serving" means.
• everything else (iframe, spa, web-component) — fetch the app URL and require < 400. A 404 on an app's own root means nothing is there.

remote_url resolution

remote_url may be same-origin-relative (/apps/<slug>/remoteEntry.js, the stored form since migration 011) or absolute (legacy cross-origin registrations). Relative resolves against the request origin, matching frontend/src/utils/loadFederatedApp.ts in the browser. Absolute is probed verbatim, so a cross-origin registration is judged on the host it actually names — probing those same-origin is what made the first production probe score fuzekeys as a 404 for a path it never claimed. That was a fault in the instrument, and there is a test pinning it.

Unhealthy is no longer an unexplained false

The response carries reason and httpStatus. 404 (no route), 503 (route up, backend down) and 200-with-HTML have different fixes and different owners; the old boolean threw that away. Both fields are additive, and no OpenAPI contract covers this route.

Not downloading the bundle

A JavaScript content-type returns without reading the body. A remoteEntry bundle is hundreds of KB and this runs once per app on every listing request — downloading it to confirm what the header already said is the expensive way to learn nothing. The body is sniffed only when the content-type is inconclusive.

Extracted for testability

Moved to backend/src/routes/appHealth.ts, with no express and no database import. routes/apps.ts needs a live Postgres to import, which is why this had no unit test before.

🧪 Testing

backend/tests/appHealth.test.ts, weighted deliberately toward the cases that must report unhealthy — 404, 503, 200+HTML by content-type, 200+HTML by body sniff, missing remote_url, network error, and 404 on a non-federated root. A suite that only asserted "a working app is healthy" would have passed against the old code too. It also asserts the bundle is not downloaded when the content-type settles it, and that an absolute remote_url is probed verbatim.

What I verified, and what I did not:

• tsc --noEmit introduces exactly one name beyond the pre-existing baseline (URL). Every TS2304 in this file is a missing-@types/node artefact of my worktree — confirmed by type-checking the unmodified master version in the same worktree and getting the same four.
• jest was NOT run — node_modules is not installed in this environment. CI on this PR is the proof for the test suite.

🚨 Breaking Changes

None for any caller that exists today, but one thing to know:

Nothing in frontend/ or sdk/ passes healthyOnly (verified by grep), so the portal lists apps regardless of health and this fix cannot shrink what users see. That is called out in a comment at the call site, because with production at 0-of-16 serving, a future caller turning it on would filter the list to nothing — which would be the check telling the truth, but should be a deliberate choice rather than a surprise.

📝 Additional Notes

A separate defect this surfaced — reported, not fixed here

There are two implementations of the same visibility-scoping rule, and they disagree:

• backend/src/routes/apps.ts scopeAppsQuery — visibility IN ('public','marketplace') OR organization_id IN (memberOrgIds). An app with organization_id IS NULL matches neither branch and is invisible to everyone.
• backend/applications/src/app-registry/service.ts:235 — .whereIn('visibility', [...]).orWhereNull('organization_id'), which does show org-less apps.

All 17 products with a registration/manifest.json register as organization or private visibility; not one is public/marketplace. So which of these two runs decides how many apps a user sees. Both are mounted — backend/src/index.ts:315 mounts routes/apps, while frontend/src/services/api.ts comments that the applications-service owns /api/apps — and I could not determine from here which one production's ingress actually routes to. Reporting the divergence rather than guessing which is authoritative. It is a plausible cause of the portal listing fewer apps than are registered, and it wants its own fix.

Deployment Notes

No migration, no dependency change. One optional new env var, FEDERATION_PROBE_ORIGIN, overriding the origin a relative remote_url resolves against — needed if the API pod cannot reach its own public hostname from inside the cluster. When it cannot, the probe reports unhealthy with the network error named, rather than reporting healthy the way the old one did.

[izzywdev]

All CI gates pass (gate-authz, gate-ds-conformance, gate-identifier, gate-frames-first, gate-test, gate-lint, gate-build, gate-sast, gate-toolchain, gate-version, gate-localup, etc.). Approving per governance policy.

[izzywdev]

Approving on verified-green CI (head d3dcff3a).

Review

• Backend-only change. Replaces a health probe that returned response.status < 500 — so a 404 counted as healthy — with appHealth.ts, which per integration type requires a module-federation remote entry to return 200 AND a JavaScript body (catching the SPA-fallback "200 that isn't"), and everything else to return < 400. Correct and well-motivated by the production 0-of-16 probe.
• Security hardening is sound: probeOrigin never derives the probe target from caller-controlled Host/x-forwarded-* headers, and fails closed (reports unhealthy naming the missing FEDERATION_PROBE_ORIGIN) rather than guessing — a probe whose target the caller can pick is not a health check.
• Absolute remote_urls are probed verbatim; the JS content-type short-circuit avoids downloading multi-hundred-KB bundles. Additive reason/httpStatus fields, no contract covers this route.
• Test suite is weighted toward the must-be-unhealthy cases (404/503/200-HTML by content-type and by body sniff, missing remote_url, network error) — i.e. non-vacuous.

Merge-conflict resolution I applied: master had since landed a competing inline health check (from a rival PR) that only probed absolute remote_urls and skipped the same-origin-relative form migration 011 made the default. I resolved in favor of this PR's superior extracted module and preserved master's unrelated ROOT_ORG_ID import; both call sites use the new checkAppHealth(app, origin) signature.

CI: all 18 checks green on d3dcff3a, including Backend Tests, Security Scanning, OIDC/Google/sign-in E2E, Harden Gate, and every gate-*. The Backend-Test failures seen on an earlier head were pre-existing master regressions since fixed by #811 and merged in here — not this PR's.

The separate visibility-scoping divergence this PR's description flags is correctly left for its own change.

---

Generated by Claude Code