Release/1.6.1#663
Merged
Merged
Conversation
…phaned
When a response is interrupted (user Stop, refresh, or dropped socket) the
user turn was already persisted but the assistant reply was not, leaving a
dangling user turn — no context on reload and malformed history the model
must paper over.
Backend:
- set/clear_interrupted_turn markers (metadata.py) with race-safe reason
precedence: user_stopped (authoritative client signal) always wins over
the connection_lost cancellation fallback. clear is an atomic pop
(ReturnValues=UPDATED_OLD) returning the settled reason.
- stream_coordinator: (CancelledError, GeneratorExit) backstop persists the
IN-FLIGHT partial assistant text (accumulator scoped to the current
message so already-persisted mid-turn messages aren't duplicated) via
asyncio.shield; empty-partial placeholder gated on a user-tail to repair
role alternation only where needed.
- app-api POST /sessions/{id}/interrupt: authoritative user_stopped carrier
(cookie auth; fetch keepalive + X-CSRF-Token, not sendBeacon). Lives on
app-api because the AgentCore Runtime data plane only proxies
/invocations + /ping.
- inference-api: pop the marker at turn start and prepend a reason-specific
ephemeral interruption note at the pending_ctx_block seam (invisible to
the user via the displayText split; ages out via compaction).
Remaining: SPA stop signal in cancelChatRequest + reload chip/Continue.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Completes the interrupted-turn feature on the SPA side:
- cancelChatRequest fires a best-effort keepalive fetch to
POST /sessions/{id}/interrupt with {reason:'user_stopped'} + X-CSRF-Token
(not sendBeacon — it can't set the CSRF header) before aborting, and
reflects the interruption locally so the chip shows without a reload.
- lastTurnInterrupted + reason are per-session ChatState signals (mirroring
lastTurnContinuable): hydrated from session metadata on reload (set-true
only), cleared beside every continuable clear (new send / continuation /
new streamed turn).
- message-actions renders a chip on the last message: connection_lost →
"Response interrupted" + Continue (reuses continueTruncatedTurn against
the persisted partial); user_stopped → "You stopped this response", no
Continue. Gated on last-message + not-loading; max_tokens Continue wins.
Specs cover both chip variants, Continue emit + precedence, the keepalive
signal shape, local reflect, failure isolation, and continuation clears.
Full app build + affected unit specs green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ted-turn-context feat: interrupted-turn context (partial + marker + reload UX)
Sweeper-not-scheduler architecture with layered runaway guards; all five open questions resolved against AgentCore Identity docs, Drive API docs, and code inspection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…scades PR-1 of the KB sync feature (docs/specs/assistant-kb-sync.md): - SyncPolicy model + repository in apis.shared.sync_policies (adjacency list SYNCPOL# items on the assistants table) - Sparse DueSyncIndex (GSI4) — keys present only while state=active, so paused policies are physically invisible to the dispatcher sweep - Conditional re-arm for idempotent dispatch; breaker counters (consecutive failures / source-gone) maintained by record_sync_result - Assistant and document delete paths eagerly cascade sync policies so no schedule outlives its source - Document gains contentHash/lastSyncedAt/syncPolicyId; Assistant gains lastUsedAt (inactivity-pause input) Inert until the PR-2 dispatcher exists: nothing reads DueSyncIndex yet. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…data feat(kb-sync): PR-1 — SyncPolicy data layer, DueSyncIndex GSI, delete cascades
…c image pipeline PR-2 of the KB sync feature (docs/specs/assistant-kb-sync.md): Backend: - kb-sync dispatcher (apis/app_api/kb_sync/dispatcher.py) — sweeps the sparse DueSyncIndex each tick and applies the runaway guards in order: kill switch, liveness (orphan policies hard-deleted), circuit breaker (failure/not-found streaks -> paused_error), 30-day inactivity pause, in-flight skip via syncRunStartedAt, re-arm-with-backoff BEFORE async-invoking the worker - worker stub (records "skipped", clears the run stamp); Drive sync lands in PR-3, web re-crawl in PR-4 - rearm_policy gains mark_run_started so claiming the in-flight slot is atomic with winning the re-arm - dispatcher reads assistant/source records by raw adjacency-list key (keeps the image surface to apis.shared.sync_policies + kb_sync); tests create them through the real services so schema drift breaks Infra: - KbSyncConstruct: two DockerImageFunctions sharing one image with ImageConfig.Command overrides; rate(15 min) EventBridge rule — DISABLED unless CDK_KB_SYNC_ENABLED=true (dark by default), env kill switch mirrors the flag; namespace-conditioned PutMetricData; error alarms; function-name SSM params for the code-deploy step - platform-as-bootstrap: byte-stable bootstrap-assets/kb-sync stub Pipeline: - backend/Dockerfile.kb-sync (lightweight: boto3+pydantic, no ML deps) - build-one.sh kb-sync case, deploy-image-lambda-one.sh kb-sync-dispatcher/worker cases sharing one image tag - deploy-image-lambda-one.sh: first-deploy grace skip when the function-name SSM param doesn't exist yet (backend.yml runs before the platform deploy on the introducing PR) - backend.yml build+deploy jobs; nightly scan-images + supply-chain test lists include the new Dockerfile Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…dispatcher feat(kb-sync): PR-2 — dispatcher/worker Lambdas, EventBridge sweep, image pipeline
…staged re-ingest
PR-3 of the KB sync feature (docs/specs/assistant-kb-sync.md §6.1):
Worker (replaces PR-2 stub for drive_file policies):
- resolves the policy creator's stored Google token from the AgentCore
Identity vault (GetWorkloadAccessTokenForUserId -> GetResourceOauth2Token,
no live user session; same customParameters as consent — vault-key rule)
- two-gate change detection: Drive files.get version vs stored sourceEtag
(continuous with import provenance), then sha256 vs contentHash —
identical bytes never reach Docling/Titan
- changed bytes staged to the document's EXISTING S3 key -> the untouched
ingestion pipeline re-chunks/re-embeds via its S3 event
- pause semantics per spec §7: requires_consent / Google 401 ->
paused_reauth (not a failure streak); Drive 404 (deleted-or-unshared,
indistinguishable) -> not_found strike, dispatcher pauses at 2;
trashed=true -> grace skip; provider/adapter/provenance gone ->
paused_error; every path records the run so the stamp always clears
Shrinkage cleanup:
- worker stashes previousChunkCount before staging; ingestion completion
atomically pops it (REMOVE + UPDATED_OLD — duplicate S3 events can't
double-delete) and deletes stale tail vectors {doc}#{new..prev-1} via
new delete_vector_tail; uses post-split len(chunks), the true vector
count this run wrote
Adapter/image/infra:
- GoogleDriveAdapter.get_file_metadata (cheap files.get, trashed +
version/md5Checksum/modifiedTime fields)
- kb-sync image gains apis.shared.oauth + file_sources (FastAPI-free,
verified by import-surface simulation); httpx + bedrock-agentcore pins
- worker IAM: vault token read (two Get* actions only — no consent
completion, no provider CRUD), oauth-providers table read, documents
bucket put; env: workload identity name + callback URL (same values
app-api uses)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…drive-worker feat(kb-sync): PR-3 — Drive-file sync path (vault token, change detection, staged re-ingest)
…ting, TTL fix PR-4 of the KB sync feature (docs/specs/assistant-kb-sync.md §6.2): Crawler refresh mode (opt-in via RefreshState; normal crawls unchanged): - upsert-by-URL: known pages reuse their document records — a re-crawl never duplicates documents - conditional GET with the stored ETag (304 = no body, no re-extract) plus a content-hash gate; identical bytes never re-embed - 'changed' emitted BEFORE the S3 overwrite so the worker stashes the previous chunk count ahead of the ingestion event (shrinkage cleanup) - transient fetch errors never flip an existing indexed doc to failed; its last-good content keeps serving - link-enqueue block deduplicated into a nested helper CrawlJob TTL fix (the silent-kill trap): terminal crawl jobs carry a 30-day TTL — a sync-covered job auto-expiring would trip the dispatcher's liveness check and delete the policy. finalize_crawl gains set_ttl; sync re-crawls finalize with the ttl REMOVED, and reset_crawl_for_refresh rearms the job (running, zeroed counters, no ttl) before each run. Worker web path: - re-runs the policy's crawl with its stored (already-capped) settings, 13-min crawler budget inside the 15-min Lambda so finalize + miss accounting always complete - miss accounting: pages absent from 2 CONSECUTIVE re-crawls are soft-deleted with cleanup run inline; fetch failures count as seen (outage != gone), robots-disallowed pages do not; misses reset on reappearance; missing root doc is recreated route-style Image: web_sources + documents + embeddings trees; beautifulsoup4, trafilatura, lxml pins. Worker Lambda timeout 10 -> 15 min. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…web-recrawl feat(kb-sync): PR-4 — web re-crawl (refresh/upsert mode, miss accounting, CrawlJob TTL fix)
… reauth/inactivity resume
- New /assistants/{id}/sync-policies router (edit-gated like documents):
create/list/patch/delete + POST run-now (202, atomic 10-min cooldown on
lastManualRunAt). Resume of paused_reauth is 409 — only a fresh OAuth
consent resumes it.
- Reauth markers: worker pause writes USER#/SYNCREAUTH# marker so
complete_consent() resumes exactly the right policies with one query;
markers are advisory (resume re-verifies state) and self-clean.
- Inactivity resume: throttled lastUsedAt bump on chat use (conditional
write, one winner/day) wakes paused_inactive policies due-immediately.
- restore_crawl_ttl: deleting a crawl's policy puts the job back on
normal 30-day expiry (running jobs untouched).
- drive_file policies keep a syncPolicyId back-pointer on the document,
cleared on policy delete.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
feat(kb-sync): PR-5 — sync-policy API + resume hooks
PR-6 (final) of the KB-sync series. Per-source "Keep in sync" controls on the assistant knowledge editor for imported Drive documents and web crawls: interval select (Manual only/Daily/Weekly/Monthly), status line (state, reason, last/next sync), pause/resume, Sync now (cooldown-aware), and a Reconnect affordance for paused_reauth policies that routes through the existing OAuth consent popup (a fresh consent auto-resumes server-side). Controls are owner/editor-only; device uploads show no control. Backend (additive, same-PR per cross-package contract): - DocumentResponse now exposes sourceConnectorId/sourceAdapterKey/ sourceFileId/syncPolicyId/lastSyncedAt so the SPA can tell imported docs from device uploads and route reconnect - GET /web-sources/crawls without ?active=true returns full history (list_all_crawls) — completed crawls are the syncable web sources Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
feat(kb-sync): SPA sync-policy controls on assistant knowledge page (PR-6, final)
…ge badges survive A Stop aborts the fetch before the stream's terminal `metadata` SSE (usage / cost / context) can arrive, so both the per-message metadata badges and the session cost badge stayed blank for that turn — and blank even after reload, because the cancellation path persisted only the partial text + interrupted marker, never any metadata. The abort is load-bearing (killing the socket is what stops generation and billing), so the backend can't push the metadata to the dead socket. Instead: - Backend: `_persist_interruption` now also calls `_store_message_metadata` (the same call the `done` path uses), which writes the per-message row AND bumps the denormalized session aggregates that hydrate the cost badge on reload. Keyed to the interrupted message's odd-position index. A cut generation often never delivered Bedrock's terminal usage event, so new `_projected_input_usage` falls back to the context-attribution projection for input-side tokens/cost (output unknown → priced at zero). - Frontend: `refreshAggregatesAfterStop` (delayed + one retry, mirroring refreshTitleFromServer, since the write races the abort) re-pulls session metadata and re-seeds the cost badge live. Per-message badges hydrate from the same persisted row on the next reload. Known limitation: the per-message contextBreakdown partition badge is not a persisted field (live-only even for normal turns), so it stays blank on a stopped/reloaded turn — a separate change affecting all turns. Tests: backend test_interrupted_turn_persistence.py (+2) and full streaming dir (175) green; frontend chat-http.service.spec.ts (+2) green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ted-turn-metadata feat(interrupted-turn): persist stopped-turn metadata so cost + message badges survive
The KB sync worker resolves the policy creator's Google token from the AgentCore Identity vault via GetResourceOauth2Token, but that call reads the refresh token *through* the Secrets Manager secret the vault auto-creates per provider (bedrock-agentcore-identity!default/oauth2/<id>). The worker role granted only the two bedrock-agentcore:* actions, so the call failed with AccessDenied on secretsmanager:GetSecretValue and every Drive sync returned "failed". Add the read-only AgentCoreIdentityOAuthSecrets grant (GetSecretValue + DescribeSecret on ...!default/oauth2/*) — the same grant app-api and inference-api already carry, minus the write lifecycle a background fetcher never needs. Covered by a new assertion in kb-sync.test.ts. Verified live in dev-ai: with the grant, the worker gets past the vault read (the remaining paused_reauth is a genuine expired-refresh-token, not this IAM gap). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…er-vault-secret-read fix(kb-sync): grant worker read on the vault's backing OAuth secrets
…r baseline `custom_parameters_for` previously injected vendor-specific OAuth params (Google `access_type=offline`, plus `prompt=consent` on the force-auth path) on top of the connector's admin-configured extras, keyed off `provider_type`. That hid a documented requirement inside the code and made the `customParameters` map depend on the call site (grant vs retrieval), which AgentCore factors into its vault key. Simplify to a single rule: every call site forwards the connector's admin-configured `customParameters` verbatim via `custom_parameters_for(admin_extras)`. Admins set `access_type=offline` and `prompt=consent` in the connector's "Custom OAuth Parameters" field (the seeded Google connector already carries both), so the same map is sent on consent and on every retrieval — no baseline merge, no `provider_type`/`force_authentication` branching, no per-call-site drift. Removes `_vendor_baseline_params`, the `provider_type_lookup` plumbing on the consent hook, and the `force_authentication=True` customParameters argument at all read sites (the SDK-level `force_authentication` flag is unchanged). Behaviour for the existing Google connector is identical (same resulting map); the win is that the vault key is now provably consistent and there are no hidden customizations in the token path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two design specs drafted alongside recent exploration work: - tool-search-token-bloat-strategy: cross-source tool-search plan for MCP token bloat (tiered discovery, AWS Agent Registry tier). - user-markdown-memory: per-user markdown "second brain" memory, re-scoped skills reference-file mechanism, prompt-cache injection. Docs only; no code or behaviour change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…dmin-configured-custom-params refactor(oauth): forward admin customParameters, drop hardcoded vendor baseline
…-and-user-memory-specs docs: add tool-search token-bloat + per-user markdown memory specs
A reauth pause was opaque — "credentials need re-consent" gave no hint why. The vault token is keyed by (workload identity, userId), so the overwhelmingly common cause is that the token was vaulted under a DIFFERENT workload identity than the worker queries: e.g. a consent done through local dev (AGENTCORE_RUNTIME_WORKLOAD_NAME=local_dev_inference) can't be read by the deployed worker (platform-workload), and vice versa. That exact mismatch cost hours to diagnose. Surface the workload name, userId, and provider the lookup used in the pause warning so the mismatch is obvious in one log line. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…reauth-diagnostics feat(kb-sync): log workload identity + userId on a reauth pause
KB sync ships dark: the EventBridge rule is created disabled and the
KB_SYNC_ENABLED kill switch is false unless CDK synths with
CDK_KB_SYNC_ENABLED=true (config.ts). platform.yml never forwarded that
var, so a CDK deploy always synthed the feature off — the only way to
enable it was an out-of-band CLI flip that reverts on the next deploy.
Forward it as an environment-scoped `${{ vars.CDK_KB_SYNC_ENABLED }}`
like every other CDK_ var. The development environment variable is set
to "true" (enables the rule + kill switch in dev-ai durably); production
leaves it unset, so it stays dark until validated there.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…able-dev chore(kb-sync): plumb CDK_KB_SYNC_ENABLED into the platform deploy
KB sync shipped opt-in (default off), a holdover from ship-dark
incremental development. The feature is complete and guarded, so a
deployer/cloner of the public stack should get it working by default —
a hidden default-off flag is bad DX. Invert to opt-out: enabled unless
CDK_KB_SYNC_ENABLED=false (or a `kbSync.enabled: false` cdk.json context).
The workflow forwards `${{ vars.CDK_KB_SYNC_ENABLED }}`, which is an empty
string when unset — a plain `?? false` fallback wouldn't flip the default,
so config.ts treats empty/unset as "use the default (on)" and only the
literal "false" as the kill switch. Adds config-resolution tests covering
unset / empty / "true" / "false" / context-disable.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…fault-on chore(kb-sync): default the feature ON with a kill switch
…ces-editor-permission fix(web-sources): let editors start and view crawls
The model admin page and the role admin page wrote to two different,
unlinked fields. Enabling a model for a role on the model page wrote
`allowedAppRoles` onto the model record — a field no access check ever
read — so the grant silently did nothing: the role page still showed the
model unchecked, and users never saw it in the chat picker. Only editing
the role's `grantedModels` had any effect.
Make the role record the single source of truth, matching the pattern
tools and skills already use:
- The model form's role picker now writes THROUGH to each selected role's
`grantedModels` (new ModelRoleService.set_roles_for_model), mirroring
set_roles_for_tool. Create/update/delete routes wire it up, migrating
grants on a modelId rename and revoking them on delete.
- `allowedAppRoles` is no longer persisted on the model; it is derived
from the role records on read (hydrate_model_roles), so the model page
and role page can no longer disagree. Adds `inheritedAppRoles` for
wildcard/inherited grants, surfaced read-only in the form.
- can_access_model and filter_accessible_models both delegate to one
`_grants_access` predicate. They previously diverged (one gated on
allowed_app_roles, one didn't), so a model could be listed by the
catalog yet denied on use.
- Removes the dead POST /sync-roles endpoint (never called; only existed
to paper over the drift); replaces it with GET /managed-models/{id}/roles.
- Drops Validators.required on the picker, since a model reachable only
via a wildcard grant legitimately has zero direct grants.
Adds regression coverage for the write-through, the derived read, and the
two access checks agreeing. Full backend + frontend suites green (the 8
pre-existing get_metadata_storage failures are unrelated).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ermission-source-of-truth fix(rbac): make AppRole the single source of truth for model access
Two pre-existing failures on develop, both unrelated to the code under test: - test_cache_savings.py patched apis.app_api.storage.get_metadata_storage, but that accessor moved to apis.shared.storage (the app_api.storage module is now an empty stub). Repoint all 5 patch targets. Production code in sessions/services/metadata.py already imports from the new location. - test_compaction_integration.py gated its real-AWS integration tests on AGENTCORE_MEMORY_ID. That variable leaks into the process mid-suite when other tests reload apis.app_api.main (load_dotenv(override=True) injects a local backend/src/.env), so the tests ran order-dependently against invalid credentials instead of skipping. Gate on an explicit RUN_AGENTCORE_INTEGRATION_TESTS=1 opt-in instead. Full suite: 4771 passed, 6 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…rage-test-patches fix(tests): repoint moved storage patch target, harden AWS integration gate
@microsoft/fetch-event-source defaults to openWhenHidden:false, which aborts the SSE connection on visibilitychange-to-hidden and reopens it — issuing a fresh POST /invocations for the SAME turn — when the tab becomes visible again. That reopen happens inside the library, reusing the request and bypassing the SPA's per-session double-submit and streamId supersession guards. Because a client abort does not propagate through the AgentCore Runtime data plane, the original backend agent keeps running while the reopened one runs the same turn concurrently. Both persist tool-use/tool-result events to the same AgentCore Memory session, corrupting history with duplicate / interleaved toolResult turns and bricking the conversation with a Bedrock "toolResult blocks exceed toolUse blocks" ValidationException. Set openWhenHidden:true on both fetchEventSource call sites so a single stream stays alive across tab switches (also correct for long agentic turns). The server-side restore-time repair is the safety net for already -corrupted histories. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bedrock Converse rejects any history where a user turn's toolResult blocks
do not exactly match the preceding assistant turn's toolUse blocks
("The number of toolResult blocks at messages.N exceeds the number of
toolUse blocks of previous turn"). A single such violation anywhere in a
session's persisted history makes every subsequent turn fail, permanently
bricking the conversation.
Such corruption can be written by concurrent/interrupted turns with parallel
tool calls (e.g. a duplicate invocation spawned by a tab switch): duplicate
toolResult turns, toolResults reordered away from their toolUse turn
(assistant/assistant/user/user), or toolResults orphaned after a synthetic
error turn. The SDK's own _fix_broken_tool_use only rebuilds the single
message after each toolUse turn, so it does not repair these shapes.
Add TurnBasedSessionManager._repair_tool_pairing, an unconditional
restore-time normalizer (sibling to _strip_document_bytes) that rebuilds a
Bedrock-valid history on the final agent.messages: every toolUse turn is
immediately followed by exactly one matching result turn (missing ones
synthesized as errors), duplicate/orphaned result turns are dropped, and
consecutive same-role turns are merged. No-op (identity) on healthy history.
Kill switch: AGENTCORE_MEMORY_HISTORY_REPAIR_ENABLED=false.
Validated against a real bricked production history (24 violations -> 0,
idempotent). Self-heals affected sessions on their next turn.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two pre-existing compaction tests fed the session manager a user toolResult turn with no preceding assistant toolUse (make_tool_result_message alone) — an invalid Converse history that Bedrock would also reject. The new restore-time _repair_tool_pairing correctly drops/merges those orphaned turns, changing the message counts the tests asserted. Give each fixture a matching toolUse turn before the toolResult so the repair no-ops and the tests exercise compaction/truncation and checkpoint slicing in isolation. Counts updated accordingly (4->5 kept; slice 2->3). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…tion breaks When a turn errors inside the agent stream, the handler persists a synthetic "⚠️ Something went wrong" assistant turn. If the last persisted message was already an assistant turn (a dangling assistant toolUse, or a prior synthetic error turn), this appends a second consecutive assistant message, breaking Bedrock's strict user/assistant alternation. The next turn then fails and persists yet another assistant error turn — an amplifier that turns one bad turn into a permanently bricked session. Add a centralized role-alternation guard in persist_synthetic_messages via a new last_persisted_role param: any synthetic turn that would land adjacent to a same-role turn is dropped (the error stays a live-only UI affordance, the same choice the max_tokens path already makes). Callers in stream_coordinator pass the history tail role via a new _last_persisted_role(agent) helper. Complements PR #653's restore-time _repair_tool_pairing, which masks this on the model-request path; this fixes the write side so storage and the message display stay clean too. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…-restore-sanitizer fix: prevent + repair tool-use/tool-result history corruption from tab-switch duplicate invocations
…ror-alternation-guard fix(sessions): guard synthetic error persistence against role-alternation breaks
…e-flight lease
A client-side abort (Stop, tab switch, dropped socket, retry) does not
propagate through the AgentCore Runtime data plane, and the Runtime can route
a duplicate POST /invocations to a different container. Two agent loops then
run concurrently against one AgentCore Memory session and corrupt tool-pairing
history, which Bedrock Converse rejects on every subsequent turn
("toolResult blocks exceed toolUse blocks"). This bricked prod session
f761f59b. Follow-up to PR #653, which closed the frontend tab-switch vector.
Add a distributed single-flight guard at the inference-api /invocations
turn-start chokepoint:
- session_lease.py: acquire/renew/release on a dedicated sessions-metadata item
(PK=USER#{uid}, SK=LEASE#{sid}) via an atomic conditional write. leaseExpiresAt
is the app-level check; ttl is a coarse auto-reap backstop. Owner-scoped renew
and release. Fail-open on any non-conflict DynamoDB error.
- routes.py: acquire at turn-start; reject a duplicate with 409. Resume /
max-tokens continuation re-enter an already-ended loop, so they acquire with
force=True (never blocked, still install a lease). Heartbeat renews the lease
while the turn streams; release in the generator finally + both except
handlers. Preview / no-DynamoDB paths skip the guard.
- SPA: handle the 409 as a soft "Already responding" notice (AlreadyStreamingError)
instead of a hard "Chat Request Failed" toast; unwrap the BFF's double-encoded
detail; loading clears so the user can retry once the prior turn finishes.
Design note + distributed-cancel follow-on: docs/specs/session-single-flight-guard.md
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…le-flight-guard fix(chat): reject duplicate concurrent turns with a per-session single-flight lease
…he server turn Follow-on to the single-flight lease (#655). A client abort doesn't propagate through the AgentCore Runtime data plane, so Stop was cosmetic server-side: the container ran to completion, held the lease, and burned model/tool spend. That left "Stop → resend" returning 409 until the prior turn finished naturally. Reuse the lease as the cross-container signalling channel: - Signal: the app-api user_stopped endpoint calls request_session_cancel, which stamps cancelRequestedFor=<leaseOwner> on the lease item (owner-scoped, so a stale Stop can't kill a later turn). Best-effort — never fails the Stop. - Observe: the inference-api lease heartbeat (tightened 30s→10s) renews with ReturnValues=ALL_NEW and, on cancelRequestedFor==owner, flips session_manager.cancelled. - Effect A (tools): the always-on StopHook cancels the next tool call. - Effect B (model stream): a cooperative check at the top of the StreamCoordinator loop raises _CooperativeStopSignal; a dedicated arm persists the partial via _persist_interruption (marked user_stopped), emits terminal SSE frames, and ends cleanly (no re-raise) so a still-connected client closes and the lease releases. This is what ends a pure-chat turn, which has no tool boundary for StopHook. - acquire clears any stale cancel marker (REMOVE) on takeover. Net: Stop ends the server turn; the 409-on-resend window shrinks from a full turn to ~one heartbeat (10s), and wasted spend after Stop is halted. Rides the existing interrupted-turn teardown/persist path (hardened by #653's _repair_tool_pairing), so stopping mid-stream never orphans or corrupts history. Residual (documented): in-flight tool calls finish before cancel is seen; already- generated Bedrock tokens are billed. Design note: docs/specs/session-single-flight-guard.md Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…-turn-cancellation feat(chat): distributed turn cancellation — make Stop actually stop the server turn
The shared-conversations DynamoDB table was threaded into the app-api
container as an env var (SHARED_CONVERSATIONS_TABLE_NAME) but never
granted on the task role. Every conversation-share operation therefore
failed against DynamoDB:
- POST /conversations/{id}/share -> PutItem AccessDeniedException
- GET /conversations/{id}/shares -> Query AccessDeniedException on
the SessionShareIndex GSI
Both surfaced to users as a generic 500 "Failed to create share".
Add `SharedConversationsAccess` to the app-api coreTables grant list so
the role gets the standard DynamoDB action set on the table and its GSIs
(index/*), matching every other table the app-api touches.
Also add a regression test that synthesizes PlatformStack and asserts
the app-api role has a SharedConversationsAccess statement granting
PutItem/Query/GetItem with a GSI resource and no wildcard. Verified the
test fails without the grant.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ed-conversations-iam-grant fix(infra): grant app-api task role access to shared-conversations table
Sharing a large conversation failed: ShareService.create_share inlined the full message list into a single DynamoDB item, exceeding the 400 KB item limit and surfacing to users as a bare 500 (observed in prod-ai as a PutItem ValidationException). This is separate from the IAM-grant bug in PR #657. Offload the snapshot body (messages + metadata) to a new private shared-conversations S3 bucket, keeping only control fields plus a body_ref pointer in DynamoDB — mirroring the Memory Spaces / Artifacts / Skills S3-offload pattern. Reads fall back to inline for legacy shares, so existing shares keep working with no migration and the SPA contract is unchanged. - New ShareSnapshotStore (content-addressed S3 put/get/delete, SSE-S3, dedupe) - create_share writes body to S3 + body_ref item; revoke/session-cleanup best-effort delete the object - _load_snapshot_body reads from S3 or falls back to legacy inline items - ShareStorageUnavailableError -> friendly 503 instead of a bare 500 - CDK: shared-conversations bucket + SSM param, compute-ref, app-api env (SHARED_CONVERSATIONS_BUCKET_NAME), and SharedConversationsBucketReadWrite IAM grant (app-api only) - Tests: store round-trip/dedupe, >400 KB regression, S3 + legacy reads, export-from-S3, revoke cleanup, storage-unavailable Spec: docs/specs/share-large-conversations-s3-offload.md Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…rge-conversations-s3-offload feat(shares): offload large-conversation snapshots to S3
…evelop-1.6.0 # Conflicts: # backend/src/agents/main_agent/session/tests/test_history_repair.py # backend/src/apis/inference_api/chat/routes.py
…nto-develop-1.6.0 Backmerge: main → develop (1.6.0)
Agent (assistant) model bindings persist only `model_id` — never `provider` — so previewing/invoking an agent bound to a Mantle model (e.g. `openai.gpt-5.4`) resolved to provider=None. That misroutes the model to Bedrock ConverseStream, which rejects it with "The provided model identifier is invalid", even though the same model works from the normal chat path (which always sends `provider` alongside `model_id`). Two complementary fixes: - Backend (server-authoritative): `_resolve_model_settings` now also returns the model's registered `provider` from the managed-model registry, and the invocation path backfills `effective_provider` from it when the request/binding didn't carry one. This fixes all existing agents with a provider-less stored binding — no data backfill needed — and mirrors how `mantle_api_mode`/`mantle_region` are already recovered. The app-tool-call / app-context-update rebuild paths get the same fallback so a rebuilt agent keys on the same provider as its main turn. - Frontend: the Agent Designer save payload now persists the selected model's `provider` (from the catalog `meta.provider`) alongside `modelId`, so newly created/edited bindings are self-describing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resume turns (interrupt_responses set — OAuth-gated MCP consent or tool-approval) crashed with `NameError: cannot access free variable 'effective_enabled_tools'`. The variable is referenced unconditionally by the `stream_with_quota_warning` streaming closure (attachment guidance + tabular inventory) but was only assigned in the non-resume branch. On resume the closure raised before its first yield, the inference-api container returned 500, and the AgentCore Runtime data plane translated that into a 424 Failed Dependency to app-api and the SPA. This broke every interrupt-resume turn since the agent-designer tool-binding refactor (0b9b039) — most visibly "connect to Gmail for employees", which completes via an OAuth-consent resume. Bind effective_enabled_tools from the paused-turn snapshot on the resume branch (the same source the resume get_agent call uses). Adds a resume-path regression test to tests/routes/test_inference.py that drives /invocations with interrupt_responses and asserts a 200 stream; without the fix it fails with the NameError. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…er-resolution fix(agents): resolve model provider for agent-bound invocations
…tive-enabled-tools-nameerror fix(inference): bind effective_enabled_tools on resume path (prod 424 on OAuth-gated MCP connect)
Patch release fixing two agent-invocation regressions. - fix(agents): resolve model provider for agent-bound invocations so agents bound to Mantle models (e.g. openai.gpt-5.4) no longer misroute to Bedrock and fail with an invalid-model-identifier error; backfill effective_provider from the managed-model registry and persist provider in the Agent Designer (#661) - fix(inference): bind effective_enabled_tools on the resume path so interrupt-resume turns (OAuth consent / tool approval, e.g. connect-to-Gmail) no longer 500/424 with a NameError (#662) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Patch release fixing two agent-invocation regressions since v1.6.0.
🐛 Fixed
model_id, so an agent bound to a Mantle model (e.g.openai.gpt-5.4) resolved toprovider=Noneand misrouted to Bedrock ConverseStream (rejected as "invalid model identifier"), even though the same model works from normal chat._resolve_model_settingsnow also returns the registeredproviderand the invocation path backfillseffective_provider— fixing existing provider-less bindings with no data backfill. Agent Designer now persistsprovideron save so new bindings are self-describing.NameError: effective_enabled_tools; the streaming closure referenced it unconditionally but it was only assigned on the non-resume branch → container 500 → Runtime 424. Now bound from the paused-turn snapshot on the resume branch. Fixes "connect to Gmail" and every other interrupt-resume flow. Regression test added.Version
VERSION1.6.0 → 1.6.1; manifests + lockfiles synced viascripts/common/sync-version.sh(--checkPASS).Deployment
No CDK / infra change, no migration. Ships through
backend.yml(app-api + inference-api) and the frontend deploy.🤖 Generated with Claude Code