feat(tls): explicit WebSocket TLS trust, rustls-tls-native-roots feature, Deepgram::tls_config

[dg-coreylweathers]

Why

Twilio reported that enabling the 0.10.1 connect-diagnostics feature breaks websocket connections behind their Zscaler proxy. Root cause: our crate has never enabled tokio-tungstenite's rustls-tls-native-roots. Twilio's build enables it through another crate, and tokio-tungstenite's default connector merges the OS certificate store when that feature is unified in. 0.10.0 used that default connector, so the Zscaler CA was trusted by accident. The explicit webpki-only connector 0.10.1 introduced (so the timed and untimed paths could not diverge) bypassed that merge. Our regression, not their environment.

Twilio asked for a native-roots feature or a way to supply the TLS connector directly. This PR does both, and fixes the class of bug rather than the instance.

What

• rustls-tls-native-roots cargo feature. Also trusts the OS certificate store, merged on top of the bundled webpki roots (never instead, so containers with no OS store keep working). Named after the tokio-tungstenite and reqwest features it mirrors. For Twilio: one word in Cargo.toml, no code change.
• Deepgram::tls_config(impl Into<Arc<rustls::ClientConfig>>). Supply your own rustls config once, on the client; every websocket it opens uses it verbatim. rustls is re-exported as deepgram::rustls so versions match.
• DeepgramError::UntrustedTlsCertificate { host, trust, source }. Returned instead of a bare WsError on an unknown issuer; the message names the remedy for the trust roots in effect.
• One explicit connector for every websocket surface. Live transcription, Flux STT, and Flux TTS now all connect through the client's rustls config (new src/tls.rs). Trust can no longer differ by surface or be changed by downstream feature unification. Flux users who relied on the unification accident need the feature too; the changelog says so plainly.
• Diagnostics. Records gain tls_trust and tls_resumed (schema version unchanged, additive). The TLS config is resolved before the DNS timer starts, so an OS-store read is never charged to tls_handshake_ms. Default config is built once per client, so TLS sessions resume across connections from the same client.
• Deps. rustls, tokio-rustls, rustls-pki-types, webpki-roots move from connect-diagnostics to listen/speak via an internal __tls feature (already in the graph through tokio-tungstenite; nothing new downloads). rustls floor raised to ^0.23.27 for handshake_kind(). tracing bounded to <0.2.

From Lawrence's branch

The trust-merge design (webpki always present, OS roots additive, warn-and-continue on bad certs, mirroring upstream) and the SSL_CERT_FILE test fixture come from feat/native-roots-tls-trust, with co-author credit. Changed from that branch: a feature flag instead of a per-builder method, no process-wide cache or TTL (the config lives on the client instead), and the code moved out of the diagnostics module so every surface uses it.

Tests and gates

• New tests/common/mod.rs (local TLS server), tests/connect_tls_config_local.rs (7 tests), tests/connect_native_roots_local.rs (5 tests): every surface, both connect paths, feature on and off. Includes the session-resumption assertion and the error-hint assertions.
• cargo fmt, cargo clippy --all-targets --all-features, 9 feature combinations with -D warnings, cargo test --all-features (112 unit, 25 local integration, 142 doctests), default and listen-only test runs, RUSTDOCFLAGS=-D warnings cargo doc --all-features --no-deps, 1.87 toolchain check, nightly minimal-versions job as CI runs it.
• cargo semver-checks --baseline-version 0.10.1 --all-features --release-type patch: 223 pass, 0 fail. Ships as 0.10.2.
• CI matrix: three new feature-combination checks and a default-feature test step.

Before tagging

Twilio (jfoster) verifies this branch behind Zscaler with a git dependency. Then a separate chore: release 0.10.2 PR.

🤖 Generated with Claude Code

[GregHolmes]

Review: deepgram-rust-sdk #175, feat(tls): explicit WebSocket TLS trust, rustls-tls-native-roots feature, Deepgram::tls_config

Classification: mixed (code-led)
Verdict: request-changes

Intent

Restore explicit native-root trust for TLS-inspecting proxies, make WebSocket TLS trust consistent across live Listen, Flux STT, and Flux TTS, and offer a client-wide custom rustls configuration. The release is planned as 0.10.2 after Twilio verifies the branch behind Zscaler. This re-review verified the post-review README fix; no earlier GitHub review or local review record was available to map.

Blocking

• [B1] src/lib.rs:426 and README.md:120, tls_config is documented as covering plaintext WebSockets.

Title: [B1] TLS configuration documentation falsely covers ws:// connections

Summary: Deepgram::tls_config and the README say the supplied rustls configuration is used by every WebSocket connection. The SDK deliberately converts an http base URL to ws://, and the phase-timed connector explicitly takes its MaybeTlsStream::Plain path for that scheme; no TLS handshake or certificate verification occurs. The public self-hosted constructors document http:// base URLs, so this is a realistic path, not only a theoretical transport distinction.

Expected: Documentation states that custom trust roots and TLS configuration apply only to wss:// connections, and it warns that ws:// is plaintext.

Observed: A developer can configure a private CA and reasonably believe it protects the documented self-hosted WebSocket connection even though the configuration is unused.

Recommended fix: Amend the tls_config docs and README to say "every WSS WebSocket connection" and add a short warning to use https:// or wss:// whenever credentials or private traffic are involved. Keep the current ws://localhost examples explicitly scoped to local testing.

Should-fix

• [S1] src/tls.rs:150, native-root load errors are hidden but error hints report successful OS-store checking.

Title: [S1] Native-root failure produces a misleading certificate remedy

Summary: rustls-native-certs errors are written only to tracing and then discarded while the client continues with webpki roots. When an invalid or unreadable SSL_CERT_FILE produces no usable roots, the later WebpkiAndNative error says the OS store was checked and suggests pointing SSL_CERT_FILE at the CA, even though that attempted source may be the failed input.

Expected: Certificate errors distinguish successfully loaded native roots from a failed native-root load and offer a remedy that matches the actual trust store.

Observed: The customer receives an actionable-looking but false diagnosis unless they also happen to capture the tracing warning.

Recommended fix: Preserve native-root load status in the TLS settings and use it in UntrustedTlsCertificate; either surface the load failure directly or change the hint to state that native roots could not be loaded and recommend a valid SSL_CERT_FILE or tls_config.

• [S2] src/diagnostics.rs:129, plaintext diagnostic records claim certificate trust information.

Title: [S2] tls_trust is reported for connections without TLS

Summary: ConnectRecord::tls_trust is documented as the roots used to verify a server certificate and is always populated. The diagnostics connector records the configured trust mode even when the URL is ws:// and takes the plain TCP path without a certificate.

Expected: A plaintext connection does not report TLS verification state as though a certificate was checked.

Observed: Consumers can read tls_trust: webpki for a ws:// attempt with no TLS phase.

Recommended fix: Make tls_trust optional and omit it for ws://, or revise its documentation to say it is the configured trust mode and require callers to use tls_handshake_ms to determine whether TLS occurred. The optional field is clearer because this is an unreleased additive diagnostic field.

Nits

• None.

Verified (evidence)

• The follow-up README change correctly wraps DER bytes in rustls::pki_types::CertificateDer before calling RootCertStore::add; the doc test compiles.
• Client TLS routing is correct on all claimed surfaces: live Listen, Flux STT, Flux TTS, and the diagnostics path pass the client-owned connector/configuration.
• Default webpki roots, additive native roots, custom configurations, UnknownIssuer mapping, and session resumption are covered by the local TLS integration tests.
• Local Rust 1.94.1 passes cargo test --all-features --no-fail-fast, clippy, fmt, docs, and the reviewed feature combinations. CI reports all checks, including SemVer, passing.
• The branch merges cleanly with current main.

Needs human

• Twilio's Zscaler validation remains the stated pre-release gate. The local fixture proves that SSL_CERT_FILE roots are merged and routed on every claimed WebSocket surface, but it cannot verify Twilio's proxy and deployment configuration. The release owner should hold the 0.10.2 tag until jfoster confirms the git-dependency test.

Developer-facing messaging

• The changelog accurately explains the 0.10.1 regression and the additive native-root feature. It should make the same WSS-only distinction as [B1], especially because it mentions self-hosted deployments and credentials.

[dg-coreylweathers]

Thanks Greg. All three findings addressed in f7c36e5.

[B1] tls_config docs claimed ws:// coverage — Deepgram::tls_config, with_base_url/with_base_url_and_api_key, the tls module docs, the README "TLS Trust" section, and the CHANGELOG now all say the configuration applies to wss:// connections only, and warn that an http:///ws:// base URL means no TLS handshake and no certificate verification (credentials and audio in cleartext). The localhost examples are kept but explicitly scoped to local testing, with a recommendation to use https:///wss:// for anything carrying credentials or private traffic.

[S1] Native-root load failure produced a misleading hint — default_root_store() now returns the trust that actually resulted, and the settings cache stores config + trust together so every connect path (live Listen, Flux STT, Flux TTS, diagnostics) reports the roots the handshake really used. New TlsTrust::WebpkiNativeUnavailable (webpki_native_unavailable on the wire) marks a native-roots build whose OS store yielded no usable roots (load errors, bad/missing SSL_CERT_FILE, empty store). Its UntrustedTlsCertificate hint says the native roots could not be loaded, points at the tracing WARN, and recommends a valid SSL_CERT_FILE or Deepgram::tls_config instead of claiming the OS store was checked. New fixture tests/connect_native_roots_unavailable_local.rs (own binary because it sets a process-global garbage SSL_CERT_FILE) asserts the variant, the hint text, that tls_config still works as the fallback, and the diagnostics record. Design note: I went with a new TlsTrust variant rather than a separate flag/count on the error because both types are unreleased and it keeps the JSONL record self-describing; happy to switch if you'd rather keep the enum at three values.

[S2] tls_trust populated for plaintext connections — ConnectRecord::tls_trust is now Option<TlsTrust> with skip_serializing_if, documented as None when no TLS occurred. The guard only records it for wss:// URLs, and connect_phases refines it to the effective trust on the TLS path. connect_diagnostics_local now asserts None on the real ws:// mock-server path.

Gates: fmt, clippy -D warnings, cargo test --all --all-features (114 unit, all local TLS/diagnostics suites incl. the 3 new tests, 142 doctests), cargo test --tests on default features, rustdoc -D warnings, the five no-default feature combos, and cargo semver-checks --baseline-version 0.10.1 --release-type patch (223 pass, still a valid patch). The Twilio Zscaler verification remains the gate before tagging 0.10.2.

[dg-coreylweathers]

Follow-up push after a second local review pass found the new connect_native_roots_unavailable_local fixture failing on Linux CI (the Test and Minimal-Versions jobs were red on f7c36e5).

Why it failed on Linux only: cargo exports SSL_CERT_DIR (via openssl-probe) into every test process on Linux, so rustls-native-certs still loaded the distro's real roots from that directory alongside the garbage SSL_CERT_FILE, and the SDK correctly reported WebpkiAndNative. macOS has no OpenSSL directory to probe, so the test passed there. Simply overriding SSL_CERT_DIR too breaks client construction, because reqwest's platform verifier reads the same variables when the Deepgram client is built and errors on an empty store. The fixture now builds each client with the environment untouched, then poisons both variables only around the first wss:// connect (the SDK resolves WebSocket trust lazily at that point), serialized with a mutex and restored by a Drop guard. Verified 3/3 in a rust:latest container with and without a CI-like SSL_CERT_DIR.

Also in this push:

• README, tls module docs, and the crate-level feature bullet no longer claim the default "works in containers with no OS certificate store" (true for wss:// only). The README now says the REST client already trusts the OS store, so behind a TLS-inspecting proxy REST calls work while wss:// fails with UntrustedTlsCertificate until one of the trust options is enabled, which is exactly the Twilio symptom.
• Connect-diagnostics records are stamped with the effective trust as soon as the TLS config resolves, so an attempt that dies in DNS or TCP after a failed native-root load reports webpki_native_unavailable rather than the configured webpki_and_native. Plaintext ws:// records still carry no trust.

Host gates all green (fmt, clippy, cargo test --all --all-features, default-features tests, rustdoc, no-default feature combos); cargo semver-checks --baseline-version 0.10.1 --release-type patch still passes. CI on this push is the confirmation for the Linux fixture.

[dg-coreylweathers]

One more push (2557efd) from a fresh full review pass on the branch, no blockers found, but a few things worth having before 0.10.2:

• Credential leak in Debug (pre-existing, fixed here): {:?} on Deepgram, or any sub-client holding it, printed the API key because reqwest's Client Debug dumps its default headers and the Authorization value was never marked sensitive. It now is, and a unit test asserts the literal key and the Token /Bearer prefix never appear in {:?} or {:#?} output for key- and token-based clients.
• Safer SSL_CERT_FILE remedy: the webpki_and_native hint told people to point SSL_CERT_FILE at the proxy CA. Since rustls-native-certs loads only the env-var location once set, and on Linux reqwest's platform verifier reads the same variable with no public-root fallback, a file holding only the proxy CA would break REST calls to anything that CA didn't sign. Both hints, the CHANGELOG, and the README now say to use a PEM bundle containing the CA together with the public roots you rely on, and explain that the variable replaces the OS store for these WebSockets and, on Linux, the REST client too.
• No trust-store read on ws://: the default config (and its OS-store load plus WARN logs) is now only built for wss:// URLs. wss:// behavior is byte-identical; ws:// passes a plain connector. A unit test asserts the lazy default stays unbuilt after a ws:// resolve.
• Docs: connect_duration_ms and the CHANGELOG say a client's first wss:// connect can include the one-time trust-store load, which is not attributed to any phase; tls_trust is documented as the configured trust until the config resolves, then the effective one.
• tracing is now optional behind the internal TLS feature; the native-roots test fixture cleans up its temp CA file.

Host gates all green (fmt, clippy, all-features and default-features tests, rustdoc, all no-default feature combos) and cargo semver-checks --baseline-version 0.10.1 --release-type patch still reports no semver update required. CI on this push is the final confirmation.