Skip to content

fix: trust the OS certificate store so login works behind TLS proxies#205

Merged
quickbeard merged 2 commits into
mainfrom
fix/trust-system-ca-store
Jul 17, 2026
Merged

fix: trust the OS certificate store so login works behind TLS proxies#205
quickbeard merged 2 commits into
mainfrom
fix/trust-system-ca-store

Conversation

@quickbeard

@quickbeard quickbeard commented Jul 17, 2026

Copy link
Copy Markdown
Owner

The report

Users hit this at the Login step of codevhub install:

Login failed: fetch failed (self-signed certificate in certificate chain)

Root cause

Node verifies TLS against its own bundled Mozilla CA snapshot and never consults the OS trust store (tls.rootCertificates: "fixed at release time… identical on all supported platforms"). A TLS-intercepting proxy (Zscaler/Netskope/Fortinet) or HTTPS-scanning AV re-signs every chain with a corporate root that MDM/GPO installed into the OS store — so the user's browser works and we fail. That per-machine variation is exactly why only some users are affected.

The error code pins it. Fetching two local servers through the same path as loggedFetch:

Server presents Result
Leaf + self-signed root (proxy shape) fetch failed (self-signed certificate in certificate chain)SELF_SIGNED_CERT_IN_CHAIN
Lone self-signed leaf fetch failed (self-signed certificate; …try --use-system-ca)DEPTH_ZERO_SELF_SIGNED_CERT

The first matches the report exactly: an untrusted self-signed root in the chain — interception, not a misconfigured server.

The fix

src/lib/tls.ts merges the OS store into Node's default CA set via tls.getCACertificates("system"), which reads that store even without the --use-system-ca flag. Affected users need no configuration.

The merge happens only after a certificate failure, never up front. loggedFetch runs the request; if it fails with a cert error, fetchTrustingSystemCa merges and retries once. This matters — the first revision of this PR merged eagerly and regressed Windows CI badly (see below).

Other load-bearing details:

  • At most one retry per process. applySystemCaCertsOnce returns null once it has run, so a chain that stays untrusted surfaces its error instead of looping. Replay is safe: a TLS handshake fails before any body is sent, and every call site passes a replayable body (string/URLSearchParams/FormData/Buffer), never a stream.
  • Merge default + system, never bundled + system. "default" already folds in NODE_EXTRA_CA_CERTS (verified: 146 = bundled 145 + the extra one), so users who fixed this the documented way keep working. This was the one way the fix could itself cause a regression.
  • Every failure mode is a no-op, including an empty system store (setDefaultCACertificates then never runs, keeping Node byte-identical). Trust config isn't ours to have opinions about, and a bad merge would break users whose certs already work.
  • APIs need Node ≥22.19/24.5 (our floor is 22.5), hence tlsApi.supported(). Accessed off the default import, never destructured — a named ESM import of a missing builtin export is a link-time error on older Node.

Why merge-on-failure, not merge-up-front

The first revision merged before the first request. tls.getCACertificates("system") is synchronous and costs ~20ms on macOS but ~300ms+ on Windows, where it blocks the event loop. Windows per-file test durations against the same runner on #204:

file baseline eager merge now
backend.test.ts 172ms 511ms (+197%) 156ms
TaskList.test.tsx 7,644ms 19,541ms (+156%) 3,278ms
ConfigApp.test.tsx 10,091ms 17,240ms (+71%) 8,707ms
InstallApp.test.tsx 57,903ms 87,657ms (+51%) 43,485ms

Everything slowed, including TaskList, which never fetches — the stalls starved Ink's render timers, and the three tests with the tightest budgets failed. That was never just a test problem: every networked command on Windows would have eaten the same stall.

Inverting it makes a cert error the trigger, which is a precise signal that the user is one of the affected minority. The happy path now costs exactly zero, and only affected users pay. A test pins that a successful request never touches the OS store.

The hint

describeNetworkError unwraps Node's bare fetch failed (the real reason hides on err.cause) and appends a remedy for cert codes. Now used at all three login surfaces (components/Login.tsx — the one in the report, components/AdminLogin.tsx, LoginApp.tsx), replacing hand-rolled unwrapping.

It exists because Node's own --use-system-ca hint pointedly excludes SELF_SIGNED_CERT_IN_CHAINcrypto_common.cc gates it on DEPTH_ZERO_SELF_SIGNED_CERT / UNABLE_TO_VERIFY_LEAF_SIGNATURE / UNABLE_TO_GET_ISSUER_CERT only (visible in the table above: only the self-signed-leaf case got a hint). The corporate-proxy case, the one the hint most exists for, prints nothing. When a cert error survives the merge the root isn't in the OS store either, so our hint names NODE_EXTRA_CA_CERTS rather than --use-system-ca, which would be a dead end.

Verification

Drove the real loggedFetch against a proxy-shaped chain (leaf + untrusted self-signed root):

  • Root in the OS store — the user's actual situation, since their browser works: login succeeds (HTTP 200), where it previously failed with the exact reported error.
  • Root absent: the same error, now carrying an actionable remedy.
  • The OS store is read only on the failure path, exactly once.

Unit tests cover the merge, the one-retry bound, the old-Node path, and that every failure mode leaves TLS untouched. pnpm fix / typecheck / test (944 passing) / build && node dist/index.js --version all green, Windows CI included.

Note for reviewers

Blast radius is our process only. npm i -g during install and the agents themselves are separate processes behind the same proxy, and will still fail on their own TLS — they need NODE_EXTRA_CA_CERTS or npm's cafile. Worth confirming with an affected user whether they now get past login and stall at agent launch; if so, that's a follow-up.

🤖 Generated with Claude Code

quickbeard and others added 2 commits July 17, 2026 14:58
Users behind a corporate proxy hit this at the Login step:

    Login failed: fetch failed (self-signed certificate in certificate chain)

Node verifies TLS against its own bundled Mozilla CA snapshot and never
consults the OS trust store (tls.rootCertificates: "fixed at release
time... identical on all supported platforms"). A TLS-intercepting proxy
(Zscaler/Netskope/Fortinet) or HTTPS-scanning AV re-signs every chain
with a corporate root that MDM/GPO installed into the *OS* store — so
the user's browser works and we fail. That per-machine variation is why
only some users are affected.

The error code pins it: SELF_SIGNED_CERT_IN_CHAIN is an untrusted
self-signed ROOT in the presented chain (interception), not a
self-signed leaf, which reports DEPTH_ZERO_SELF_SIGNED_CERT instead.

Merge the OS store into Node's default CA set via
tls.getCACertificates("system"), which reads that store even without the
--use-system-ca flag — so affected users need no configuration.

Load-bearing details:

  - Merge "default" + "system", never "bundled" + "system". "default"
    already folds in NODE_EXTRA_CA_CERTS; narrowing it would silently
    drop the certs of users who fixed this the documented way.
  - Call it from loggedFetch, not at startup: it costs ~25ms (a native
    OS-store read) and commands that never open a socket shouldn't pay.
    Every network path goes through loggedFetch (16 call sites, zero raw
    fetch), so first-request is the cheapest and completest hook.
  - Every failure mode is a no-op, including an empty system store.
    Trust config isn't ours to have opinions about, and a bad merge would
    break users whose certs already work.
  - The APIs need Node >=22.19/24.5 (our floor is 22.5), hence
    tlsApi.supported(). Accessed off the default import, never
    destructured — a named ESM import of a missing builtin export is a
    link-time error on older Node.

Also add describeNetworkError: unwrap Node's bare `fetch failed` (the
real reason hides on err.cause) and append a remedy for cert codes. Node
only volunteers its own --use-system-ca hint for DEPTH_ZERO /
UNABLE_TO_VERIFY_LEAF / UNABLE_TO_GET_ISSUER, which pointedly excludes
SELF_SIGNED_CERT_IN_CHAIN — the corporate-proxy case it most exists for
prints nothing. When a cert error survives the merge the root isn't in
the OS store either, so the hint names NODE_EXTRA_CA_CERTS rather than
--use-system-ca, which would be a dead end.

Verified by driving the real loggedFetch against a proxy-shaped chain
(leaf + untrusted self-signed root): with the root in the OS store login
now succeeds (HTTP 200) where it previously failed with the exact
reported error; without it, the error carries an actionable remedy.

Note the blast radius: this covers our own process only. `npm i -g`
during install and the agents themselves are separate processes behind
the same proxy and need NODE_EXTRA_CA_CERTS / npm's cafile of their own.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The eager merge regressed Windows CI. tls.getCACertificates("system") is
synchronous and costs ~20ms on macOS but ~300ms+ on Windows, where it
blocks the event loop — and loggedFetch ran it before the first request
of every command, including in tests.

Windows per-file durations vs the same runner on #204:

    backend.test.ts       172ms ->    511ms  (+197%)
    TaskList.test.tsx    7,644ms -> 19,541ms (+156%)
    ConfigApp.test.tsx  10,091ms -> 17,240ms  (+71%)
    InstallApp.test.tsx 57,903ms -> 87,657ms  (+51%)

Everything slowed, including TaskList, which never fetches — the stalls
starved Ink's render timers, and the three tests with the tightest budgets
failed. Not a test-only problem: every networked command on Windows would
have eaten the same stall.

Invert it. Run the request first; only when it fails with a cert error do
we merge and retry once (fetchTrustingSystemCa). A cert error is a precise
signal that the user is one of the affected minority, so the happy path
now costs exactly zero and only affected users pay. applySystemCaCertsOnce
returns null once it has run, bounding this at one retry per process — a
chain that stays untrusted surfaces its error instead of looping. Replay
is safe: a TLS handshake fails before any body is sent, and every call site
passes a replayable body (string/URLSearchParams/FormData/Buffer).

certHint is now pure for the same reason: reading the store to word a
sentence would put the stall back on the error path, and the retry already
read it.

Tests pin both halves — a successful request never touches the OS store,
and a cert failure merges, retries once, then gives up. Re-verified
end-to-end: with the root in the OS store login still succeeds (HTTP 200);
without it the error still carries the remedy; the store is read only on
the failure path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@quickbeard
quickbeard merged commit b0b23b0 into main Jul 17, 2026
4 checks passed
@quickbeard
quickbeard deleted the fix/trust-system-ca-store branch July 17, 2026 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant