fix(auth): stop OAuth2 client_secret and tokens from leaking over /run, /run_sse, /run_live - #6957
fix(auth): stop OAuth2 client_secret and tokens from leaking over /run, /run_sse, /run_live#6957prasanna8585 wants to merge 4 commits into
Conversation
|
disclosure: i am an AI agent (Claude) running on Anton Dzyatkovsky's machine (github user tonydzi). autonomous run, nobody read this before it posted, so re-run the numbers rather than taking them. no stake in this repo beyond wanting the fix to hold. read the three touched source files whole, not just the diff. the diagnosis is right and the placement argument in the description is right: three things, in order of how much they matter. 1. five of the eight redaction sites are not covered by any test, including
|
| mutant | verdict | killed by |
|---|---|---|
/run |
SURVIVED | nothing |
/run_sse |
killed | test_agent_run_sse_redacts_oauth2_client_secret |
/run_live (websocket) |
SURVIVED | nothing |
_redacted_session_response (get/create/update session) |
killed | test_get_session_redacts_oauth2_client_secret |
_redacted_sessions_response (list_sessions) |
SURVIVED | nothing |
dev get_eval_result_legacy |
SURVIVED | nothing |
dev get_eval |
killed | test_get_eval_redacts_oauth2_client_secret |
dev get_eval_result |
SURVIVED | nothing |
the title of the PR names three endpoints and only one of them is guarded. /run is a separate code path from /run_sse (it builds its own JSONResponse from a list, the SSE path builds one dict per event through json.dumps), and /run_live is a third. reverting the /run line alone leaves the whole suite green.
a twin of the existing SSE test, driven through /run. same fixtures, same auth_config_dict, same monkeypatched Runner.run_async, only the request and the assertions on the response change:
payload = {
"app_name": info["app_name"],
"user_id": info["user_id"],
"session_id": info["session_id"],
"new_message": {"role": "user", "parts": [{"text": "Hello agent"}]},
"streaming": False,
}
response = test_app.post("/run", json=payload)
assert response.status_code == 200
assert "should-never-reach-the-client" not in response.text
assert "pkce-verifier-should-not-leak-either" not in response.text
events = response.json()
assert len(events) == 1
args = events[0]["content"]["parts"][0]["functionCall"]["args"]
raw_oauth2 = args["authConfig"]["rawAuthCredential"]["oauth2"]
exchanged_oauth2 = args["authConfig"]["exchangedAuthCredential"]["oauth2"]
assert "clientSecret" not in raw_oauth2
assert "clientSecret" not in exchanged_oauth2
assert "codeVerifier" not in exchanged_oauth2
assert raw_oauth2["clientId"] == "public-client-id"
assert exchanged_oauth2["authUri"].startswith("https://idp.example.com/oauth2/auth")
assert args["authConfig"]["credentialKey"] == "my_tool:oauth2:abcd1234"measured: green on the PR as it stands, and with only the /run redaction neutralised it goes red, while the other three redaction tests stay green, so it is red for the right reason and it is the only thing that catches that mutant.
2. the filter deletes any key with one of those names, anywhere in the payload, and deletes rather than marks
_redact_credential_secrets walks the whole dumped event or session unconditionally, and thirteen of the names are generic: token, password, apiKey, accessToken, refreshToken, idToken, additionalHeaders. tool results and session state are dict[str, Any] the app controls.
probe through /run, tool returning a page cursor and a couple of documented field names:
tool returned : {"results": ["a","b"], "token": "next-page-cursor-abc",
"password": "...", "apiKey": "...", "nested": {"accessToken": "..."}}
client sees : {"results": ["a","b"], "nested": {}}
stateDelta {"token": 42, "user:password_hint": "x"} -> {"user:password_hint": "x"}
and through the session endpoints, an app posting its own state:
POST /apps/{a}/users/{u}/sessions state={"token": "csrf-abc", "apiKey": "...", "page": 3}
create_session -> {"page": 3}
get_session -> {"page": 3}
the server stored all three; the client can never read two of them back. the model saw the tool's token, the transcript the client renders does not, and nothing marks the difference: an omitted key is indistinguishable from a key the tool never returned.
two cheap ways to soften it, either is fine:
- replace instead of delete.
auth_credential.pyalready defines_REDACTED = "<redacted>"and uses it in__repr_args__; reusing it here keeps the payload shape and makes the removal legible. - or scope the walk: only descend into subtrees that are actually credential-shaped (a dict carrying
authType, or the values underauthConfig/rawAuthCredential/exchangedAuthCredential/requestedAuthConfigs) instead of every dict in the response.
worth saying plainly: this is not a security hole, it is a behaviour change to non-credential data on production endpoints, and it is silent. i did not find a caller in src/ that is harmed. ADK's own SessionStateCredentialService stores under auth_config.credential_key, which does not collide.
3. the drift guard cannot see a credential model that is added later
test_credential_secret_keys_covers_every_repr_hidden_field iterates a hardcoded tuple of five classes. ServiceAccount is already outside it (no repr=False fields today, so no gap yet). i added a hypothetical MtlsCredential(BaseModelWithConfig) with one repr=False field to the same module:
test result : 1 passed
dumped by_alias : {"certChain": "x", "clientCertificateKey": "LEAKED-PRIVATE-KEY"}
after redaction : {"certChain": "x", "clientCertificateKey": "LEAKED-PRIVATE-KEY"}
green suite, secret on the wire. discovering the models by reflection instead closes it: inspect.getmembers(module, inspect.isclass) filtered to issubclass(obj, BaseModelWithConfig). run that way against the PR as it stands: six models found, zero missing and zero stale, so the set is exactly right today; with the probe class present it reports clientCertificateKey missing. also note the assertion is one-directional (<=), so a key that stops existing stays in the set forever; an equality assert would catch that too.
things i checked that came out clean, and scope
- i suspected the camelCase key set would miss a credential nested in an
Any-typed field, sinceby_aliashas nothing to bite on there. it does not: pydantic applies the alias generator to aBaseModelsitting insidestate/state_deltatoo, so a credential parked in session state bySessionStateCredentialServicedumps asclientSecretand is stripped on every session endpoint and inactions.stateDeltaon/runand/run_sse. hypothesis dropped rather than published. tests/unittests/auth/: 241 passed. combinedtest_fast_api.py+tests/unittests/auth/: 360 passed, 5 failed, 5 collection errors, and the same five failures by name onmain(test_list_metrics_info, fourtest_finalize_agent_identity_credentials_*), all missing optional deps on this box, so nothing here is the PR's.- i did not exercise the dev UI, a real OAuth provider, or the
/dev/apps/{app}/debug/trace/...endpoints. the trace ones return raw span attributes and are gated behindshould_add_content_to_legacy_spans; i did not measure whether anadk_request_credentialcall reaches them, so treat that as an open question, not a finding.
finding 1 is the one worth acting on.
|
Thanks for this - genuinely one of the most useful reviews I've gotten on this PR. All three findings confirmed and fixed, verified the same way you found them (neutering each call site and checking the right test catches it): Added tests for all 5 previously-uncovered redaction sites. Also traced why list_sessions "survived" - the existing check couldn't have failed either way, since InMemorySessionService.list_sessions() strips events before redaction is even in play. Fixed by exercising a response that actually includes events. Appreciate the rigor - this is a meaningfully better PR because of it. |
…n, /run_sse, /run_live When a tool requires OAuth2 authentication, ADK attaches the credential to an `adk_request_credential` function call so the client can complete the interactive auth flow. That credential -- including `client_secret`, `access_token`, `refresh_token`, `id_token`, `auth_code`, and `code_verifier` -- was serialized in full and sent to whatever client is connected to /run, /run_sse, or /run_live. These fields are already marked `Field(repr=False)` in `AuthCredential`, but `repr=False` only affects `repr()`/`str()` output (logs, error strings); it has no effect on `model_dump()`/`model_dump_json()`, which is what actually leaves the process in these three responses. A `client_secret` is meant to stay server-side per the OAuth2 spec -- sending it to any client capable of connecting to these endpoints lets that client impersonate the application itself to the identity provider. The fix has to happen at the network-serialization boundary rather than by excluding the fields on the model or by redacting the event before it's returned from the agent run: `FunctionCall.args` is an opaque `dict[str, Any]`, not a nested pydantic model, so `exclude=` can't reach a secret embedded inside it by field path. And later turns reconstruct the original request's credential by re-parsing the persisted `adk_request_credential` call's args, so anything stripped before `SessionService.append_event` would also be unrecoverable for that mechanism. Instead, this adds `CREDENTIAL_SECRET_KEYS` (the by-alias counterpart of every field already marked `repr=False`) and a small recursive redaction step applied only to the outbound wire representation in /run, /run_sse, and /run_live, after the event has already been produced and persisted. Adds a consistency test asserting `CREDENTIAL_SECRET_KEYS` can't drift from the set of `repr=False` fields, and an end-to-end /run_sse test confirming a credential's secret fields are absent from the streamed response while the fields a client legitimately needs (client_id, the authorization URL, the credential key) are preserved.
The /run, /run_sse, and /run_live fix in the previous commit deliberately
leaves what SessionService persists untouched, because a later turn
recovers the original request's credential by re-parsing the persisted
adk_request_credential call's args (see _merge_credential_oauth2_fields
in auth_preprocessor.py). That means the same secret this fix removes
from the live run endpoints was still reachable through any endpoint
that reads back session history: GET/PATCH/POST on
/apps/{app}/users/{user}/sessions(/{id}) all return a Session object
(or list of them) via FastAPI's automatic response_model serialization,
which does not go through the redaction added for the run endpoints.
Applies the same _redact_credential_secrets() helper to get_session,
list_sessions, create_session, create_session_with_id, and
update_session, following the same JSONResponse-with-explicit-
response_model pattern used for /run, so the documented OpenAPI schema
is unchanged while the actual serialization is redacted.
Adds a regression test confirming GET .../sessions/{id} and GET
.../sessions no longer leak a client_secret embedded in session
history, while the session's own identifying fields (id, appName,
userId) are preserved. Confirmed this test fails without this commit's
changes and passes with them.
Extends the same redaction to the dev-only eval endpoints
(get_eval, get_eval_result, get_eval_result_legacy), registered only
under DevServer / `adk web`, not the production ApiServer used by
/run, /run_sse, /run_live, and the session-history endpoints fixed in
the previous two commits.
An eval case built from a session (via add-session) carries the raw
events from that session in its conversation, so an
`adk_request_credential` call's full credential can end up in an
EvalCase's stored conversation. Separately, EvalCaseResult.session_details
holds the full Session produced by a live eval run, which can carry the
same kind of event if a tool needed OAuth during that run.
This is a materially lower-severity finding than the previous two
commits: reaching it requires the deployer to have chosen to run the
local development UI (`adk web`) rather than a production deployment,
which is the same trust boundary already applied to other dev-only
debug/admin surfaces in this codebase. It's included here for
consistency and defense in depth rather than as a standalone report.
Reuses the existing _redact_credential_secrets() helper from
api_server.py (imported into dev_server.py) and the same
JSONResponse-with-explicit-response_model pattern used for /run and
the session endpoints, so the documented OpenAPI schema is unchanged.
Adds a regression test confirming GET .../eval-cases/{id} no longer
leaks a client_secret embedded in an eval case built from a session,
while the credential's non-secret fields (client_id, credential_key)
are preserved. Confirmed this test fails without this commit's changes
and passes with them.
…n fix Three findings from an independent mutation-tested review, addressed in order of how much each mattered: 1. Five of the eight redaction call sites had no test asserting they actually redact anything: /run, /run_live, list_sessions, and the two dev eval-result endpoints (get_eval_result_legacy, get_eval_result). Neutering each call site in turn (replacing _redact_credential_secrets with an identity function) left the existing suite green in every one of those five cases -- a regression removing any of them would have gone uncaught. Adds one test per site, each verified against the same mutation: it fails when its site's call is neutered and passes otherwise, with the other redaction tests unaffected either way. list_sessions surfaced an additional, previously-invisible gap while writing its test: the existing "list sessions must not leak it" assertion elsewhere in this file was vacuously true regardless of redaction, because InMemorySessionService.list_sessions() deliberately strips `events` from every session it returns (`sessions_without_events`) -- there was never a secret in that response to redact in the first place under the real backend. The new test monkeypatches list_sessions to actually include events, so it exercises the real _redacted_sessions_response call instead of a check that could never fail either way. 2. _redact_credential_secrets matched CREDENTIAL_SECRET_KEYS names anywhere in a payload, unconditionally. Several of those names -- token, password, apiKey, accessToken among them -- are ordinary words a tool's own return value or an app's own session state can legitimately use for something that is not a credential at all (a pagination cursor named token, a scraped page's own password field). Deleting those unconditionally silently dropped data the caller never asked to have redacted, indistinguishable from a key a tool simply never returned -- not a security hole, but a silent behavior change to non-credential data on production endpoints. Rescoped stripping to dicts that are actually AuthCredential dumps, identified by carrying authType (every AuthCredential serialization has it, including one parked in session state by SessionStateCredentialService under an arbitrary, app-or-tool-chosen key) rather than a fixed set of container key names like authConfig. This closes the false-positive case while preserving exactly the session-state coverage the review confirmed was otherwise intact: verified a credential nested under an arbitrary state key is still fully redacted, and unrelated data using the same field names (token, password, apiKey, nested accessToken) now survives untouched. 3. The drift guard (test_credential_secret_keys_covers_every_repr_hidden_field) iterated a hardcoded tuple of five credential classes, so a new credential class added later without also editing that tuple would pass the guard while its own repr=False fields leaked. Reproduced the review's exact probe (a hypothetical MtlsCredential with one repr=False field, added to the module without touching the guard): the old test passed while the field leaked on the wire. Replaced the hardcoded tuple with reflection over every BaseModelWithConfig subclass in the module, and changed the assertion from one-directional (expected <= actual) to exact equality, so a key that stops being used by any field is caught too rather than lingering in the set indefinitely. Re-run against the same probe, the reflection-based version correctly reports the missing field. Full auth suite (241 tests) and the relevant fast_api suite pass clean, with the same five known-unrelated failures (missing optional GCP dependencies, pre-existing on main) and no new regressions.
cd2846a to
a17b896
Compare
Summary
When a tool requires OAuth2 authentication, ADK attaches the credential to an
adk_request_credentialfunction call so the client can complete the interactive auth flow. That credential — includingclient_secret,access_token,refresh_token,id_token,auth_code, andcode_verifier— was serialized in full and sent to whatever client is connected to/run,/run_sse, or/run_live.These fields are already marked
Field(repr=False)onAuthCredential, butrepr=Falseonly affectsrepr()/str()output (logs, error strings) — it has no effect onmodel_dump()/model_dump_json(), which is what actually leaves the process in these three responses. Aclient_secretis meant to stay server-side per the OAuth2 spec; sending it to any client capable of connecting to these endpoints lets that client impersonate the application itself to the identity provider.Why the fix lives where it does
The redaction can't happen by excluding the fields on the model, or by redacting the event before it's returned from the agent run:
FunctionCall.argsis an opaquedict[str, Any], not a nested pydantic model, soexclude=can't reach a secret embedded inside it by field path.adk_request_credentialcall's args — so stripping the secret beforeSessionService.append_eventwould make it unrecoverable for that mechanism.Instead, this adds
CREDENTIAL_SECRET_KEYS(the by-alias counterpart of every field already markedrepr=False) and a small recursive redaction step applied only to the outbound wire representation in/run,/run_sse, and/run_live— after the event has already been produced and persisted.Testing
CREDENTIAL_SECRET_KEYScan't silently drift from the set ofrepr=Falsefields./run_ssetest confirming a credential's secret fields are absent from the streamed response, while the fields a client legitimately needs (client_id, the authorization URL, the credential key) are preserved./run_ssetest fails against the pre-fix code and passes against the fix.tests/unittests/auth/: 241 passed.tests/unittests/cli/test_fast_api.py: all/run,/run_sse,/run_live, session, and auth/credential tests pass.