diff --git a/CHANGELOG.md b/CHANGELOG.md index 34caa11ea..6831d845d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,40 @@ All notable changes to this project are documented in this file. Format follows For narrative release notes written for operators and product owners, see [RELEASE_NOTES.md](RELEASE_NOTES.md). +## [1.5.0] - 2026-07-13 + +Model-catalog and MCP-admin expansion, plus UI polish. Admins can now discover the tools behind OAuth-gated MCP servers (e.g. the GitHub remote MCP server) using their own vaulted 3LO token, two new curated model cards land (Claude Sonnet 5, GPT-5.4), and the max-output-tokens field goes optional so reasoning/Responses-API models without a fixed cap can be added. Alongside: sticky admin/settings sidebars, a redesigned 404 page, chat-scroll and sticky-nav fixes, a vitest flake fix, and a Docker curl security patch. No CDK deploy, no data migration, no breaking changes β€” ships via the backend and frontend pipelines. + +### πŸš€ Added + +- Admin tool "Discover" now works against OAuth-gated MCP servers: pass the OAuth provider id and discovery connects with the admin's own vaulted 3LO token (fetched via AgentCore Identity `get_token_for_user`, injected as a bearer), validating the admin's connection and listing the tools their token can see. Previously OAuth-gated servers were refused outright (400) or connected unauthenticated and 401'd. `requires_consent` β†’ 409; unknown provider / forward-auth conflict / oauth2-without-provider β†’ 400 (#639) +- Two curated model catalog cards: **Claude Sonnet 5** (Bedrock, `global.anthropic.claude-sonnet-5`, 1M context, effort-based reasoning, caching on) and **GPT-5.4** (Mantle, `openai.gpt-5.4`, Responses API). Bedrock Claude cards now order most-capable-first and the "Bedrock Mantle" tab sits next to "Bedrock" (#641) + +### ✨ Improved + +- Max output tokens is now optional on the admin model form β€” reasoning / Responses-API models (GPT-5.x, adaptive-thinking Claude) that share output with reasoning budget have no fixed cap to enter. `max_output_tokens` is `Optional[int]` end to end; it was only ever a ceiling for the admin-configured `max_tokens` param and is never sent to the provider, so leaving it unset is safe (#643, #644) +- Admin and user-settings sidebar navs are now sticky on desktop, staying in view as the content column scrolls (#632, #638) +- The 404 page was redesigned to match the auth/first-boot screens (frosted glass, animated blobs, graph-paper grid) (#633) +- Chat scroll space is now sized to the pending response and adapts to the shell scroll container, so the last turn spaces correctly (#637) + +### πŸ› Fixed + +- Mantle `google.gemma-4-*` models 401'd with `access_denied` because Gemma 4 is served only on Mantle's `/openai/v1` path but the Strands SDK routed it to `/v1`. The SDK's OpenAI-path prefix table is now extended with `google.gemma-4-` at build time (lazy, idempotent, guarded); Gemma 3 stays on `/v1` (#641) +- The admin/settings sticky sidebars needed a real shell scroll container to anchor against β€” the app shell now scrolls on the correct element so `position: sticky` engages (#634) +- Vitest runs now guarantee the Angular JIT compiler is present, stopping the intermittent `PlatformLocation` provider flake in the SPA unit suite (#636) + +### πŸ”’ Security + +- Dockerfiles pin curl to `+deb13u*` instead of an exact point release, so the security patch floats with Debian's mirror (which purges superseded point versions on each CVE update) and builds don't break; the digest-pinned base image still provides reproducibility (#645) + +### πŸ”§ CI/CD + +- `scripts/common/sync-version.sh` is now portable across GNU and BSD userlands (macOS `sed`/`grep`), so the version bump runs outside the dev container (#631) + +### πŸ“š Docs + +- Added the quota cooldown-windows + platform-ceiling spec and committee one-pager (`docs/specs/`) (#635); added a design note proposing `mantleEndpointPath` as a live admin setting (#641) + ## [1.4.0] - 2026-07-10 Opt-in MCP user-identity forwarding, plus an admin OAuth-provider repair. A new Cognito Pre-Token-Generation Lambda can copy configured user-pool attributes into namespaced claims on the access token β€” the only token forwarded end-to-end to MCP servers β€” so personalized MCP tools can identify the caller. It ships disabled by default (a fork that configures nothing gets zero resources). Also fixes a 502 when adding the first admin OAuth provider. A platform (CDK) deploy is required for the OAuth-provider IAM fix. diff --git a/README.md b/README.md index 880541351..443604ebe 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ **An open-source, production-ready Generative AI platform for institutions** *Built by Boise State University, designed for everyone.* -[![Release](https://img.shields.io/badge/Release-v1.4.0-6366f1?style=flat&logo=github&logoColor=white)](RELEASE_NOTES.md) +[![Release](https://img.shields.io/badge/Release-v1.5.0-6366f1?style=flat&logo=github&logoColor=white)](RELEASE_NOTES.md) [![Nightly](https://github.com/Boise-State-Development/agentcore-public-stack/actions/workflows/nightly.yml/badge.svg)](https://github.com/Boise-State-Development/agentcore-public-stack/actions/workflows/nightly.yml) ![Python](https://img.shields.io/badge/Python-3.13+-3776AB?style=flat&logo=python&logoColor=white) @@ -296,7 +296,7 @@ agentcore-public-stack/ See [RELEASE_NOTES.md](RELEASE_NOTES.md) for the full changelog, including new features, bug fixes, platform upgrades, and deployment notes for each release. -**Current release:** v1.4.0 +**Current release:** v1.5.0 --- diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index 2e07bd2cb..859058fbf 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -1,3 +1,72 @@ +# Release Notes β€” v1.5.0 + +**Release Date:** July 13, 2026 +**Previous Release:** v1.4.0 (July 10, 2026) + +--- + +> βœ… **No platform (CDK) deploy required.** This release is application code and frontend only β€” it ships through the `backend.yml` (app-api / inference-api image rebuild) and `frontend-deploy.yml` pipelines. No new AWS resources, no IAM changes, no data migration, no breaking changes. + +--- + +## Highlights + +v1.5.0 expands the **model catalog** and the **MCP admin** surface, then rounds out the UI. Admins can now run tool **Discovery against OAuth-gated MCP servers** β€” such as the GitHub remote MCP server β€” using their own vaulted 3LO token, rather than being refused outright. Two curated model cards land β€” **Claude Sonnet 5** and **GPT-5.4** β€” and the **max-output-tokens** field becomes optional so reasoning / Responses-API models that have no fixed output cap can be added at all. The remainder is polish and hardening: sticky admin/settings sidebars, a redesigned 404 page, chat-scroll and sticky-nav fixes, a vitest flake fix, a Mantle Gemma-4 routing fix, and a Docker curl security patch. + +## Discover OAuth-gated MCP servers with the admin's token + +Admins configuring a tool couldn't see what an OAuth-gated MCP server actually offered: the "Discover" flow refused `auth_type=oauth2` outright (400) or connected unauthenticated and got a 401 (wrapped to a 400), so servers like the GitHub remote MCP server (`api.githubcopilot.com/mcp/`) were undiscoverable. Discovery now connects with the admin's *own* vaulted token for the provider and lists the tools that token can see. + +### Backend + +- **Provider-aware discovery (#639)** β€” `MCPDiscoverRequest` gains `requires_oauth_provider` (alias `requiresOauthProvider`). The handler loads the provider, fetches the admin's vaulted 3LO token via AgentCore Identity (`get_token_for_user`) and injects it as `oauth_token` into `create_external_mcp_client` β€” mirroring how the agent loop attaches the end-user's provider token at runtime, and reusing the exact path `connector_status` already uses. It fetches the **admin's own** token only; it cannot mint an arbitrary end-user's token. Providers such as GitHub scope-filter the tool list to the token's grants, so the result reflects what the admin's connection can actually reach. `requires_consent` β†’ 409; unknown provider / conflict with `forward_auth` / oauth2-without-provider β†’ 400. + +### Frontend + +- The discover payload now sends `requiresOauthProvider` (the form control already existed) and the `OAuth2CallbackUrl` header (bare `/oauth-complete`, no query string) so the backend can resolve the admin's token. + +### Test Coverage + +5 backend tests for the OAuth-provider discovery path; 2 SPA specs for the discover payload. + +## Model catalog β€” Sonnet 5, GPT-5.4, and optional output caps + +Two new curated cards and one form change together let admins add the current generation of frontier models with a single click. + +### Frontend + +- **New curated cards (#641)** β€” **Claude Sonnet 5** (`global.anthropic.claude-sonnet-5`, 1M context, effort-based reasoning, caching on) and **GPT-5.4** (Mantle, `openai.gpt-5.4`, Responses API surface β€” its `openai.gpt-5.*` id matches the SDK's `/openai/v1` routing prefixes so one-click create routes correctly). The Bedrock Claude list now orders most-capable-first (Opus 4.7, Sonnet 5, Sonnet 4.6, Haiku 4.5), GPT-5.4 sits ahead of Qwen in the Mantle list, and the "Bedrock Mantle" tab moves next to "Bedrock" in the catalog selector. +- **Optional max output tokens (#643, #644)** β€” newer reasoning / Responses-API models don't publish a discrete max-output-tokens value (output shares the context budget with reasoning tokens), so the admin form field is now optional. `max_output_tokens` becomes `Optional[int]` across `ManagedModelCreate` / `ManagedModel` and the SPA interfaces (`number | null`); the DynamoDB write omits it when absent, the form drops `Validators.required`, and the catalog card null-guards to show "β€” out". It was only ever a ceiling for the admin-configured `max_tokens` param and is never sent to the provider, so an unset value is safe at inference time. + +## πŸ› Bug fixes + +- **Mantle Gemma 4 models returned `access_denied` (#641).** Gemma 4 is served only on Mantle's `/openai/v1` path (per its AWS model card), but the Strands SDK's `_OPENAI_PATH_MODEL_PREFIXES` shipped only `openai.gpt-5.`, so `google.gemma-4-*` fell through to `/v1` and inference 401'd. The SDK's prefix table is now extended with `google.gemma-4-` at build time (`_ensure_gemma4_openai_v1_routing`: lazy, idempotent, guarded) until it lands upstream β€” scoped to the 4.x family so Gemma 3 stays on `/v1`. Covered by guard tests. +- **Sticky sidebars didn't engage (#634).** The redesigned admin/settings sticky sidebars need a real scroll container to anchor against; the app shell now scrolls on the correct element so `position: sticky` takes effect and the chat scroll space sizes to the pending response (#637). +- **Intermittent SPA unit-test flake (#636).** Vitest runs now guarantee the Angular JIT compiler is present, eliminating the sporadic `PlatformLocation` provider error in the unit suite. + +## πŸ”’ Security + +- **Docker curl patch floats with the mirror (#645).** Debian removes the superseded point version of curl from the trixie mirror on each security update, so an exact `+deb13uN` pin broke every build once the next CVE landed. Both Dockerfiles now pin `+deb13u*` β€” tracking the live security patch while keeping the minor version fixed. The digest-pinned base image is what actually provides reproducibility. + +## ✨ UI polish + +- **Sticky navigation (#632, #638)** β€” the admin and user-settings sidebar navs stay in view on desktop as the content column scrolls. +- **Redesigned 404 page (#633)** β€” the not-found page now matches the auth / first-boot screens (frosted glass, animated blobs, graph-paper grid in Boise blue). + +## πŸ”§ CI/CD + +- **Portable version sync (#631)** β€” `scripts/common/sync-version.sh` now runs across GNU and BSD userlands (macOS `sed`/`grep`), so the release bump works outside the dev container. + +## πŸ“š Docs + +- Added the quota cooldown-windows + platform-ceiling spec and committee one-pager under `docs/specs/` (#635), and a design note proposing `mantleEndpointPath` as a live admin setting as the durable alternative to patching the SDK's hardcoded routing table (#641). + +## πŸš€ Deployment notes + +- **No special steps.** No CDK deploy, no IAM changes, no data migration, no breaking changes. The backend changes (Mantle Gemma-4 routing, optional `max_output_tokens`, Docker curl pin) ship on the next `backend.yml` image rebuild; the SPA changes ship via `frontend-deploy.yml`. + +--- + # Release Notes β€” v1.4.0 **Release Date:** July 10, 2026 diff --git a/VERSION b/VERSION index e21e727f9..3e1ad720b 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.4.0 \ No newline at end of file +1.5.0 \ No newline at end of file diff --git a/backend/Dockerfile.app-api b/backend/Dockerfile.app-api index 2f31de097..11e9d7f7c 100644 --- a/backend/Dockerfile.app-api +++ b/backend/Dockerfile.app-api @@ -39,7 +39,7 @@ WORKDIR /app # Install runtime dependencies (curl required for HEALTHCHECK) RUN apt-get update && apt-get install -y \ - curl=8.14.1-2+deb13u3 \ + curl=8.14.1-2+deb13u* \ && rm -rf /var/lib/apt/lists/* # Copy the virtual environment from builder diff --git a/backend/Dockerfile.inference-api b/backend/Dockerfile.inference-api index 0c7d17e10..d03b2d5f7 100644 --- a/backend/Dockerfile.inference-api +++ b/backend/Dockerfile.inference-api @@ -39,7 +39,7 @@ WORKDIR /app # Install runtime dependencies (curl required for HEALTHCHECK) RUN apt-get update && apt-get install -y \ - curl=8.14.1-2+deb13u3 \ + curl=8.14.1-2+deb13u* \ && rm -rf /var/lib/apt/lists/* # Copy the virtual environment from builder diff --git a/backend/pyproject.toml b/backend/pyproject.toml index 15b3b849c..e6b648b19 100644 --- a/backend/pyproject.toml +++ b/backend/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "agentcore-stack" -version = "1.4.0" +version = "1.5.0" requires-python = ">=3.10" description = "Multi-agent conversational AI system with AWS Bedrock AgentCore" readme = "README.md" diff --git a/backend/src/apis/app_api/admin/tools/routes.py b/backend/src/apis/app_api/admin/tools/routes.py index f45191534..496350d69 100644 --- a/backend/src/apis/app_api/admin/tools/routes.py +++ b/backend/src/apis/app_api/admin/tools/routes.py @@ -9,6 +9,16 @@ from fastapi import APIRouter, Depends, HTTPException, Query from apis.shared.auth import User, require_admin +from apis.shared.oauth.agentcore_identity import ( + CallbackUrlUnavailableError, + WorkloadTokenUnavailableError, + custom_parameters_for, + get_agentcore_identity_client, +) +from apis.shared.oauth.provider_repository import ( + OAuthProviderRepository, + get_provider_repository, +) from apis.app_api.tools.service import get_tool_catalog_service from apis.app_api.tools.discovery import discover_tools_for_saved_tool from apis.shared.tools.gateway_target_service import ( @@ -358,6 +368,7 @@ def _discovery_failure_detail(status: int) -> str: async def admin_discover_mcp_tools( request: MCPDiscoverRequest, admin: User = Depends(require_admin), + provider_repo: OAuthProviderRepository = Depends(get_provider_repository), ): """Connect to an MCP server with the given config and return its tool list. @@ -385,11 +396,55 @@ async def admin_discover_mcp_tools( create_external_mcp_client, ) - if request.auth_type in (MCPAuthType.OAUTH2.value, MCPAuthType.OAUTH2): + # An OAuth (3LO) gated server is discovered using the admin's *own* vaulted + # token for the named provider β€” mirroring how the agent loop attaches the + # end-user's provider token at runtime. This validates the admin's own + # connection and returns the tools their token can see (providers such as + # GitHub scope-filter the tool list to the token's grants). It fetches the + # admin's token only; it can't mint an arbitrary end-user's token, so the + # discovered list reflects the admin's connection, not every user's. + oauth_token: Optional[str] = None + if request.requires_oauth_provider: + if request.forward_auth_token: + raise HTTPException( + status_code=400, + detail="Choose either an OAuth provider or forwarded-auth " + "discovery, not both.", + ) + provider = await provider_repo.get_provider(request.requires_oauth_provider) + if provider is None: + raise HTTPException( + status_code=400, + detail=f"Unknown OAuth provider " + f"'{request.requires_oauth_provider}'.", + ) + identity = get_agentcore_identity_client() + try: + result = await identity.get_token_for_user( + provider_name=provider.provider_id, + scopes=provider.scopes, + user_id=admin.user_id, + # Match the customParameters used at consent time β€” AgentCore + # factors them into the vault key, so omitting them would + # falsely report consent-required for a vaulted token. + custom_parameters=custom_parameters_for(provider.custom_parameters), + ) + except (WorkloadTokenUnavailableError, CallbackUrlUnavailableError) as err: + logger.warning("OAuth discovery token unavailable: %s", err) + raise HTTPException(status_code=503, detail=str(err)) + if result.requires_consent or not result.access_token: + raise HTTPException( + status_code=409, + detail=f"You haven't connected '{provider.display_name}' yet. " + "Connect it in your connector settings, then retry discovery.", + ) + oauth_token = result.access_token + + elif request.auth_type in (MCPAuthType.OAUTH2.value, MCPAuthType.OAUTH2): raise HTTPException( status_code=400, - detail="OAuth-gated MCP servers can't be discovered server-side; " - "list the tool names manually.", + detail="Select the OAuth provider for this server to discover its " + "tools, or list the tool names manually.", ) # Forward-auth servers (same-team MCP that validates a forwarded JWT, behind @@ -400,8 +455,7 @@ async def admin_discover_mcp_tools( # trust-boundary note above); forwarding their own session token to that URL # is a strictly lower bar than the end-user token forwarding they're # configuring for the tool. - oauth_token: Optional[str] = None - if request.forward_auth_token: + elif request.forward_auth_token: if not admin.raw_token: raise HTTPException( status_code=400, diff --git a/backend/src/apis/shared/models/managed_models.py b/backend/src/apis/shared/models/managed_models.py index 1a5aed664..a50f56973 100644 --- a/backend/src/apis/shared/models/managed_models.py +++ b/backend/src/apis/shared/models/managed_models.py @@ -265,7 +265,6 @@ async def _create_managed_model_cloud(model_data: ManagedModelCreate, table_name 'inputModalities': model_data.input_modalities, 'outputModalities': model_data.output_modalities, 'maxInputTokens': model_data.max_input_tokens, - 'maxOutputTokens': model_data.max_output_tokens, 'allowedAppRoles': model_data.allowed_app_roles, 'availableToRoles': model_data.available_to_roles, 'enabled': model_data.enabled, @@ -278,6 +277,8 @@ async def _create_managed_model_cloud(model_data: ManagedModelCreate, table_name } # Add optional fields + if model_data.max_output_tokens is not None: + item['maxOutputTokens'] = model_data.max_output_tokens if model_data.cache_write_price_per_million_tokens is not None: item['cacheWritePricePerMillionTokens'] = model_data.cache_write_price_per_million_tokens if model_data.cache_read_price_per_million_tokens is not None: diff --git a/backend/src/apis/shared/models/mantle.py b/backend/src/apis/shared/models/mantle.py index 445750958..7114d060d 100644 --- a/backend/src/apis/shared/models/mantle.py +++ b/backend/src/apis/shared/models/mantle.py @@ -64,6 +64,29 @@ def param_map_for(api_mode: MantleApiMode) -> Dict[str, str]: ) +def _ensure_gemma4_openai_v1_routing() -> None: + """Teach the Strands SDK to route ``google.gemma-4-*`` to ``/openai/v1``. + + Per its AWS model card, Gemma 4 is served *only* on the Mantle + ``/openai/v1`` base path β€” "different from the ``v1`` path used by other + models." But the SDK's ``_OPENAI_PATH_MODEL_PREFIXES`` only lists + ``openai.gpt-5.``, so ``google.gemma-4-*`` falls through to ``/v1`` and the + endpoint 401s ("... is not enabled for this account"). Append the family + prefix at build time until it lands upstream (strands-agents/sdk-python). + + ``google.gemma-4-`` specifically β€” NOT ``google.gemma-``: Gemma 3 is served + on ``/v1`` and must not be rerouted. Idempotent; safe to call on every build. + """ + from strands.models import _openai_bedrock as _sdk + + prefix = "google.gemma-4-" + if prefix not in _sdk._OPENAI_PATH_MODEL_PREFIXES: + _sdk._OPENAI_PATH_MODEL_PREFIXES = ( + *_sdk._OPENAI_PATH_MODEL_PREFIXES, + prefix, + ) + + def build_mantle_model( model_id: str, api_mode: MantleApiMode, @@ -88,6 +111,9 @@ def build_mantle_model( from strands.models import OpenAIResponsesModel from strands.models.openai import OpenAIModel + # Bridge until google.gemma-4- lands in the SDK's base-path prefix table. + _ensure_gemma4_openai_v1_routing() + bedrock_mantle_config: Dict[str, Any] = {} if region: bedrock_mantle_config["region"] = region diff --git a/backend/src/apis/shared/models/models.py b/backend/src/apis/shared/models/models.py index d1b5f5bd3..1234aee6b 100644 --- a/backend/src/apis/shared/models/models.py +++ b/backend/src/apis/shared/models/models.py @@ -156,7 +156,11 @@ class ManagedModelCreate(BaseModel): input_modalities: List[str] = Field(..., alias="inputModalities", min_length=1) output_modalities: List[str] = Field(..., alias="outputModalities", min_length=1) max_input_tokens: int = Field(..., alias="maxInputTokens", ge=1) - max_output_tokens: int = Field(..., alias="maxOutputTokens", ge=1) + # Optional: newer reasoning/Responses-API models don't publish a discrete + # output cap (output shares the context budget with reasoning tokens). This + # value is only a ceiling for the admin-configured max_tokens inference + # param β€” it is never sent to the provider β€” so leaving it unset is safe. + max_output_tokens: Optional[int] = Field(None, alias="maxOutputTokens", ge=1) # Access control: AppRoles (preferred) or legacy JWT roles allowed_app_roles: List[str] = Field( default_factory=list, @@ -322,7 +326,7 @@ class ManagedModel(BaseModel): input_modalities: List[str] = Field(..., alias="inputModalities") output_modalities: List[str] = Field(..., alias="outputModalities") max_input_tokens: int = Field(..., alias="maxInputTokens") - max_output_tokens: int = Field(..., alias="maxOutputTokens") + max_output_tokens: Optional[int] = Field(None, alias="maxOutputTokens") # Access control: AppRoles (preferred) or legacy JWT roles allowed_app_roles: List[str] = Field( default_factory=list, diff --git a/backend/src/apis/shared/tools/models.py b/backend/src/apis/shared/tools/models.py index a71443cbe..71c2f9ea5 100644 --- a/backend/src/apis/shared/tools/models.py +++ b/backend/src/apis/shared/tools/models.py @@ -1354,9 +1354,17 @@ class MCPDiscoverRequest(BaseModel): """Request body for POST /api/admin/tools/discover. Same fields as MCPServerConfigRequest minus the `tools` list β€” the - point of discovery is to populate that list. Provider-gated OAuth (3LO) - servers can't be discovered admin-side (no end-user provider token - available); the route returns a 400 in that case. + point of discovery is to populate that list. + + `requires_oauth_provider` mirrors the catalog flag of the same name: when + set, the route discovers an OAuth (3LO) gated server using the *admin's + own* vaulted token for that provider (fetched via AgentCore Identity), + injected as a bearer. This validates the admin's connection and lists the + tools their token can see β€” providers like GitHub scope-filter the tool + list to the token's grants. It fetches the admin's token only; it cannot + mint an arbitrary end-user's token, so the discovered list reflects the + admin's own connection. If the admin hasn't connected the provider the + route returns 409. Mutually exclusive with `forward_auth_token`. `forward_auth_token` mirrors the catalog flag of the same name: when set, the route signs the discovery request with the *admin's own* OIDC token @@ -1373,6 +1381,9 @@ class MCPDiscoverRequest(BaseModel): api_key_header: Optional[str] = Field(None, alias="apiKeyHeader") secret_arn: Optional[str] = Field(None, alias="secretArn") forward_auth_token: bool = Field(default=False, alias="forwardAuthToken") + requires_oauth_provider: Optional[str] = Field( + None, alias="requiresOauthProvider" + ) model_config = {"populate_by_name": True, "use_enum_values": True} diff --git a/backend/tests/apis/app_api/admin/tools/test_discover_oauth_provider.py b/backend/tests/apis/app_api/admin/tools/test_discover_oauth_provider.py new file mode 100644 index 000000000..d490c0c33 --- /dev/null +++ b/backend/tests/apis/app_api/admin/tools/test_discover_oauth_provider.py @@ -0,0 +1,205 @@ +"""Tests for OAuth (3LO) MCP discovery on `/admin/tools/discover`. + +An OAuth-gated MCP server (e.g. the GitHub remote MCP server) is discovered +using the *admin's own* vaulted token for the named provider β€” fetched via +AgentCore Identity and injected as a bearer, mirroring how the agent loop +attaches the end-user's provider token at runtime. See +`apis/app_api/admin/tools/routes.py`. +""" + +import pytest +from fastapi import HTTPException + +from apis.app_api.admin.tools import routes +from apis.shared.auth.models import User +from apis.shared.tools.models import MCPAuthType, MCPDiscoverRequest + +_CREATE_TARGET = ( + "agents.main_agent.integrations.external_mcp_client.create_external_mcp_client" +) +_URL = "https://api.githubcopilot.com/mcp/" + + +def _admin(raw_token="admin-tok"): + return User( + email="admin@example.edu", + user_id="u1", + name="Admin", + roles=["system_admin"], + raw_token=raw_token, + ) + + +class _FakeSpec: + def __init__(self, name, description=None): + self.name = name + self.description = description + + +class _FakeTool: + def __init__(self, name, description=None): + self.mcp_tool = _FakeSpec(name, description) + self.tool_name = name + + +class _FakeClient: + def __init__(self, tools): + self._tools = tools + + def __enter__(self): + return self + + def __exit__(self, *exc): + return False + + def list_tools_sync(self): + return list(self._tools) + + +class _FakeProvider: + def __init__(self, provider_id="github", display_name="GitHub"): + self.provider_id = provider_id + self.display_name = display_name + self.scopes = ["repo", "read:org"] + self.custom_parameters = None + + +class _FakeProviderRepo: + def __init__(self, provider): + self._provider = provider + + async def get_provider(self, provider_id): + if self._provider and self._provider.provider_id == provider_id: + return self._provider + return None + + +class _FakeTokenResult: + def __init__(self, access_token=None, requires_consent=False): + self.access_token = access_token + self.requires_consent = requires_consent + + +class _FakeIdentity: + def __init__(self, result): + self._result = result + self.calls = [] + + async def get_token_for_user(self, **kwargs): + self.calls.append(kwargs) + return self._result + + +@pytest.mark.asyncio +async def test_oauth_provider_discovery_uses_admin_vaulted_token(monkeypatch): + """A connected provider yields a vaulted bearer injected into the client.""" + captured = {} + + def fake_create(config, oauth_token=None, **kwargs): + captured["oauth_token"] = oauth_token + return _FakeClient([_FakeTool("search_repositories", "Find repos")]) + + monkeypatch.setattr(_CREATE_TARGET, fake_create) + + identity = _FakeIdentity(_FakeTokenResult(access_token="gho_vaulted")) + monkeypatch.setattr(routes, "get_agentcore_identity_client", lambda: identity) + + req = MCPDiscoverRequest( + serverUrl=_URL, + authType=MCPAuthType.OAUTH2, + requiresOauthProvider="github", + ) + resp = await routes.admin_discover_mcp_tools( + req, admin=_admin(), provider_repo=_FakeProviderRepo(_FakeProvider()) + ) + + assert captured["oauth_token"] == "gho_vaulted" + assert [t.name for t in resp.tools] == ["search_repositories"] + # The token was fetched for the admin's own actor id. + assert identity.calls[0]["user_id"] == "u1" + assert identity.calls[0]["provider_name"] == "github" + + +@pytest.mark.asyncio +async def test_oauth_provider_discovery_requires_consent_returns_409(monkeypatch): + """An unconnected admin gets an actionable 409, and no client is built.""" + + def fake_create(*args, **kwargs): # pragma: no cover - must not run + raise AssertionError("client must not be built without a vaulted token") + + monkeypatch.setattr(_CREATE_TARGET, fake_create) + + identity = _FakeIdentity(_FakeTokenResult(requires_consent=True)) + monkeypatch.setattr(routes, "get_agentcore_identity_client", lambda: identity) + + req = MCPDiscoverRequest( + serverUrl=_URL, + authType=MCPAuthType.OAUTH2, + requiresOauthProvider="github", + ) + with pytest.raises(HTTPException) as exc_info: + await routes.admin_discover_mcp_tools( + req, admin=_admin(), provider_repo=_FakeProviderRepo(_FakeProvider()) + ) + + assert exc_info.value.status_code == 409 + assert "connect" in exc_info.value.detail.lower() + + +@pytest.mark.asyncio +async def test_unknown_oauth_provider_returns_400(monkeypatch): + def fake_create(*args, **kwargs): # pragma: no cover - must not run + raise AssertionError("client must not be built for an unknown provider") + + monkeypatch.setattr(_CREATE_TARGET, fake_create) + + req = MCPDiscoverRequest( + serverUrl=_URL, + authType=MCPAuthType.OAUTH2, + requiresOauthProvider="ghost", + ) + with pytest.raises(HTTPException) as exc_info: + await routes.admin_discover_mcp_tools( + req, admin=_admin(), provider_repo=_FakeProviderRepo(_FakeProvider()) + ) + + assert exc_info.value.status_code == 400 + + +@pytest.mark.asyncio +async def test_oauth_provider_and_forward_auth_conflict_returns_400(monkeypatch): + def fake_create(*args, **kwargs): # pragma: no cover - must not run + raise AssertionError("client must not be built for a conflicting request") + + monkeypatch.setattr(_CREATE_TARGET, fake_create) + + req = MCPDiscoverRequest( + serverUrl=_URL, + authType=MCPAuthType.OAUTH2, + requiresOauthProvider="github", + forwardAuthToken=True, + ) + with pytest.raises(HTTPException) as exc_info: + await routes.admin_discover_mcp_tools( + req, admin=_admin(), provider_repo=_FakeProviderRepo(_FakeProvider()) + ) + + assert exc_info.value.status_code == 400 + + +@pytest.mark.asyncio +async def test_oauth2_authtype_without_provider_returns_400(monkeypatch): + """auth_type=oauth2 with no provider named can't be discovered β€” 400.""" + + def fake_create(*args, **kwargs): # pragma: no cover - must not run + raise AssertionError("client must not be built without a provider") + + monkeypatch.setattr(_CREATE_TARGET, fake_create) + + req = MCPDiscoverRequest(serverUrl=_URL, authType=MCPAuthType.OAUTH2) + with pytest.raises(HTTPException) as exc_info: + await routes.admin_discover_mcp_tools( + req, admin=_admin(), provider_repo=_FakeProviderRepo(None) + ) + + assert exc_info.value.status_code == 400 diff --git a/backend/tests/shared/test_mantle.py b/backend/tests/shared/test_mantle.py index 90c4749fe..342d856fb 100644 --- a/backend/tests/shared/test_mantle.py +++ b/backend/tests/shared/test_mantle.py @@ -69,6 +69,63 @@ def test_params_forwarded(self, mock_openai_cls): assert mock_openai_cls.call_args.kwargs["params"] == {"temperature": 0.5, "max_tokens": 128} +class TestGemma4Routing: + """The build path teaches the SDK to serve google.gemma-4-* from /openai/v1. + + Gemma 4's model card pins it to the Mantle /openai/v1 base path, but the + SDK only lists openai.gpt-5. β€” so the builder appends the family prefix. + """ + + def _prefixes(self): + from strands.models import _openai_bedrock as sdk + + return sdk._OPENAI_PATH_MODEL_PREFIXES + + @patch("strands.models.openai.OpenAIModel") + def test_build_registers_gemma4_prefix(self, _mock_openai_cls): + build_mantle_model( + model_id="google.gemma-4-31b", + api_mode=MantleApiMode.CHAT_COMPLETIONS, + region="us-east-1", + ) + assert "google.gemma-4-" in self._prefixes() + + @patch("strands.models.openai.OpenAIModel") + def test_gemma4_variants_route_to_openai_v1(self, _mock_openai_cls): + from strands.models._openai_bedrock import _resolve_mantle_base_path + + build_mantle_model( + model_id="google.gemma-4-31b", + api_mode=MantleApiMode.CHAT_COMPLETIONS, + ) + for model_id in ( + "google.gemma-4-31b", + "google.gemma-4-26b-a4b", + "google.gemma-4-e2b", + ): + assert _resolve_mantle_base_path(model_id) == "/openai/v1" + + @patch("strands.models.openai.OpenAIModel") + def test_gemma3_stays_on_v1(self, _mock_openai_cls): + # Gemma 3 is served on /v1 β€” the narrower prefix must not reroute it. + from strands.models._openai_bedrock import _resolve_mantle_base_path + + build_mantle_model( + model_id="google.gemma-4-31b", + api_mode=MantleApiMode.CHAT_COMPLETIONS, + ) + assert _resolve_mantle_base_path("google.gemma-3-27b-it") == "/v1" + + @patch("strands.models.openai.OpenAIModel") + def test_registration_is_idempotent(self, _mock_openai_cls): + for _ in range(3): + build_mantle_model( + model_id="google.gemma-4-31b", + api_mode=MantleApiMode.CHAT_COMPLETIONS, + ) + assert self._prefixes().count("google.gemma-4-") == 1 + + class TestParamMapFor: def test_chat_mode_map(self): assert param_map_for(MantleApiMode.CHAT_COMPLETIONS) is MANTLE_CHAT_PARAM_MAP diff --git a/backend/uv.lock b/backend/uv.lock index 25fd970e7..f9ce7188f 100644 --- a/backend/uv.lock +++ b/backend/uv.lock @@ -12,7 +12,7 @@ resolution-markers = [ [[package]] name = "agentcore-stack" -version = "1.4.0" +version = "1.5.0" source = { editable = "." } dependencies = [ { name = "aiofiles" }, diff --git a/docs/specs/mantle-endpoint-path-admin-setting.md b/docs/specs/mantle-endpoint-path-admin-setting.md new file mode 100644 index 000000000..380ecb67c --- /dev/null +++ b/docs/specs/mantle-endpoint-path-admin-setting.md @@ -0,0 +1,111 @@ +# Design note: revive `mantleEndpointPath` as a live admin setting + +**Status:** Proposal (not started) +**Author:** (drafted with Claude) +**Date:** 2026-07-13 +**Related:** `apis/shared/models/mantle.py`, `apis/shared/bedrock/bearer_token.py`, +`frontend/.../admin/manage-models/models/curated-models.ts` + +## Problem + +Bedrock Mantle serves different models at different OpenAI-compatible base paths +on the *same* host, and there is **no discovery API** β€” the path is a per-model +fact published only in each model's AWS model card: + +| Model family | Mantle base path | +|---|---| +| `openai.gpt-oss-*`, `qwen.*` | `/v1` | +| `google.gemma-3-*` | `/v1` | +| `google.gemma-4-*` | `/openai/v1` | +| `openai.gpt-5.*` | `/openai/v1` | + +Today the base path is derived *inside the Strands SDK* from a hardcoded +prefix table (`strands.models._openai_bedrock._OPENAI_PATH_MODEL_PREFIXES`, +currently just `("openai.gpt-5.",)`). Any model whose id isn't in that table +falls through to `/v1`. When that's wrong (Gemma 4), inference 401s with +`access_denied` / "... is not enabled for this account". + +We've now hit this **twice** (gpt-5, then Gemma 4). Each occurrence requires a +dependency bump or a build-time monkeypatch of the SDK's private tuple +(`_ensure_gemma4_openai_v1_routing`, added 2026-07-13 as the stopgap). The path +is the one per-model Mantle fact we do **not** model as admin data β€” `apiMode` +(`chat` vs `responses`) and `region` already are. + +## Why it's SDK-derived today + +The builder delegates the whole Mantle wire-up to the SDK's +`bedrock_mantle_config`, which bundles three things: + +1. Region resolution +2. **Per-request bearer-token minting + rotation** (`provide_token`) +3. `base_url` derivation from the model id (the prefix table) + +`resolve_bedrock_client_args` sets `base_url` itself and the model classes +**reject** a caller-supplied `base_url` when `bedrock_mantle_config` is set +(fail-fast `ValueError`). So #3 is not separable from #1/#2 β€” to control the +path you must stop using `bedrock_mantle_config` entirely. + +There is a now-`[DEPRECATED]` field `mantleEndpointPath` (`ManagedModel*` in +`apis/shared/models/models.py`) and a helper `get_mantle_base_url(region, +endpoint_path)` that already builds arbitrary paths. The plumbing is half +present; it was retired, not removed. + +## Proposal + +Make the base path an **admin-set, per-model** value again β€” data, not code β€” +by building the OpenAI client ourselves instead of via `bedrock_mantle_config`. + +### Backend (`build_mantle_model`) + +- Un-deprecate `mantleEndpointPath`. Default `/v1`; admin sets `/openai/v1` for + Gemma 4 / gpt-5 (documented on the model card, so it's a copy-paste fact). +- When building the model, construct `client_args` ourselves: + - `base_url = get_mantle_base_url(region, endpoint_path)` + - `api_key = generate_bedrock_bearer_token(region)` (already exists) + - Do **not** pass `bedrock_mantle_config`. +- Delete `_ensure_gemma4_openai_v1_routing` and the SDK-tuple monkeypatch. + +### Frontend + +- Add an "Endpoint path" field to the Mantle model form (default `/v1`), with a + hint linking to "the path shown on the model's AWS model card." +- Curated cards carry the correct `mantleEndpointPath` per model. + +### The cost we take back: token lifecycle + +`bedrock_mantle_config` mints a **fresh token per request**; a self-set +`api_key` is fixed at model construction. Mitigations: + +- The agent model is rebuilt per turn (`AgentFactory`), so a token minted at + build time is fresh for the turn's duration β€” the common case is covered. +- The minted token's lifetime (~12h) far exceeds a single turn. +- **Risk:** a very long-lived / resumed agent could outlive the token. Needs a + check of the longest-lived `build_mantle_model` consumer + (`/chat/api-converse`, agent resume) before committing. If any consumer holds + one model across the token lifetime, add a small refresh shim (mint via a + callable the client invokes per request) β€” reproducing what the SDK does. + +## Decision + +| | Revive admin path (this note) | Keep SDK-derived + patch (current) | +|---|---|---| +| New `/openai/v1` family | data change, zero code | dep bump or monkeypatch | +| Token minting/rotation | **we own it** | free from SDK | +| Consistency with `apiMode`/`region` | matches | inconsistent | +| Blast radius | medium (touches token path) | tiny (one tuple) | + +**Recommendation:** ship the monkeypatch now (done) to unblock Gemma 4. Adopt +this note **only if a third family** forces the issue, or if we want to stop +tracking upstream SDK releases for a routing table we can trivially own. The +one open question β€” token lifetime vs. longest-lived consumer β€” must be +answered before implementation. + +## Open questions + +1. What is the longest-lived single `OpenAIModel`/`OpenAIResponsesModel` + instance across all `build_mantle_model` callers? (Determines whether a + static build-time token is safe or a refresh shim is required.) +2. Does `provide_token` from `aws-bedrock-token-generator` expose the token + TTL, so we could set an explicit shorter expiry and know when to refresh? +3. Any Mantle model served on a path *other* than `/v1` or `/openai/v1`? (If + the space stays two-valued, a boolean toggle may beat a free-text path.) diff --git a/docs/specs/quota-cooldown-one-pager.md b/docs/specs/quota-cooldown-one-pager.md new file mode 100644 index 000000000..eeefafe19 --- /dev/null +++ b/docs/specs/quota-cooldown-one-pager.md @@ -0,0 +1,135 @@ +# AI Cost Controls: From Monthly Cutoffs to Paced Budgets + +*One-page rationale for committee review β€” companion to the technical spec +(`quota-cooldown-windows.md`).* + +## The problem with what we have today + +Our only cost control is a per-user monthly dollar limit with two behaviors: a +warning at a set percentage, and a **hard cutoff at 100% that lasts until the +1st of the next month**. This creates four problems, and they compound: + +1. **It punishes exactly the wrong people.** The users who hit the limit are + our most engaged adopters β€” the people getting the most value from the + platform. Their reward for productivity is a lockout measured in weeks, + often mid-project. That is the single worst adoption signal we can send. +2. **It strands most of the budget.** AI usage is extremely uneven: most users + spend pennies, a modest group spends a few dollars, and a small group of + power users drives real value. Equal per-person allowances guarantee that + light users strand their budget unspent *while* heavy users are cut off β€” + waste and frustration from the same mechanism. +3. **It guarantees nothing about total spend.** Today's aggregate cost is + simply "whatever the sum of individual limits allows." Setting limits is + guesswork, and the institution has no enforced total β€” only a hoped-for one. +4. **It forces conservative limits.** Because the failure mode (a weeks-long + lockout) is so harsh, limits get set low to avoid it β€” shrinking the value + of the platform for everyone to protect against the behavior of a few. + +## The proposed model: three controls, each with one job + +**1. Cooldown windows β€” pace, don't punish.** Every user gets a healthy budget +for each 5-hour stretch of work β€” enough for a genuinely heavy session. Exceed +it and you wait **hours, with an exact reset time shown ("resets at 2:30 PM")** +β€” not weeks. This is the same mechanism Anthropic uses for Claude's own +subscribers: it caps the *rate* of spending without capping ambition. +*Alleviates problem 1:* a heavy user's worst day becomes "take a break until +mid-afternoon." + +**2. A pooled budget with one hard, adjustable ceiling β€” the fiscal +guarantee.** Leadership sets a single number: target average cost per user Γ— +user count. The system **enforces** that total β€” spend cannot exceed it. +Within the pool, budget flows to whoever is producing with it rather than +being reserved per person. +*Alleviates problems 2 and 3:* no stranded allowances, and institutional +exposure becomes one deliberately set number instead of an emergent sum of +guesses. + +**3. A generous individual monthly backstop β€” where the premium ends, not +where the month ends.** Each user keeps a personal monthly cap that normal +work never reaches; only sustained heavy use lands there, and only after +early warnings. Reaching it is not designed to end anyone's month: the target +behavior is an **automatic switch to a much cheaper model for the remainder** +(with warnings plus a documented exception path in the interim). It stops +runaways and bounds sustained heavy use; it does not ration normal work. +*Alleviates problem 4:* with the window bounding burn *rate* and the ceiling +bounding the *total*, individual caps can finally be generous. + +**Why the three compose:** the windows keep any individual or small group from +draining the shared pool quickly, which is what makes a hard shared ceiling +safe and fair. Each control does one job; none has to compensate for another. + +## Proposed opening configuration + +| Knob | Value | Why | +|---|---|---| +| Target average cost per user | **$5/month** | The committee's investment decision; fixes the ceiling (Γ— user count) | +| Session budget (per 5-hour window) | **$2.00** | A genuinely heavy work session β€” sized for real bursts (long documents, multi-step tasks), and for the reality that users have only 2–3 usable sessions in a waking day | +| Individual monthly backstop | **$30** (6Γ— target) | Where premium-model use ends for the month β€” normal months never reach it; sustained heavy use gets early warnings, then (per roadmap) an economy model rather than a stop; genuine exceptions get an override | +| Backstop horizon | **Monthly** (weekly optional per group) | Monthly suits deadline-clustered semester work (one crunch week fits inside it); a weekly cap β€” the model Claude itself uses β€” suits steadily heavy groups and safely permits even larger session budgets. Configurable per group; the pilot data shows which fits whom | + +Both pacing values are re-tuned from the first month's real usage +distribution; the dashboard is built to show exactly that. + +## Flexibility built in + +- **Exceptions without side doors.** Designated roles (e.g. grant-funded + research) or individuals can be exempted β€” permanently via policy tied to + the university's own identity roles, or temporarily via self-expiring + overrides ("capstone week"). Exempt usage is **always still measured and + reported**; exemption is an explicit, auditable grant, so the fiscal + guarantee stays honest: *spend cannot exceed the ceiling except for + designated users, whose spend is fully visible.* +- **Budget changes take effect within a minute.** Nothing is ever "marked + blocked" β€” every request is evaluated against the current numbers. If + leadership raises the ceiling as it approaches, affected users are working + again on their next message. No deployment, no reset, no ticket. +- **Policy follows the org chart automatically.** Tiers are assigned through + the university's identity system β€” when someone becomes faculty, their + treatment upgrades with zero administrative action. +- **Users see their own status.** A personal meter shows session and + monthly/weekly usage with exact reset times β€” no surprise cutoffs, and + fewer "how much do I have left?" support questions. +- **Every number is configuration, not code.** As an open-source platform, + the session budget, window length, caps, horizon (monthly vs. weekly), and + ceiling are all settings β€” each adopting institution, and each user group + within it, sets its own policy. + +## Safeguards at the ceiling + +- **Alerts at 80% and 90%**, with a live dashboard: month-to-date spend vs. + ceiling, burn rate, projected month-end total, and where the money goes β€” + problems surface weeks out, not overnight. +- **A built-in ~10% reserve:** enforcement triggers below the stated budget, + so "hitting the limit" still leaves leadership deciding with headroom. +- **Scheduled upgrade β€” degrade, don't stop:** replace pauses with automatic + fallback to a much cheaper model β€” first for individuals at their monthly + cap, later near the ceiling β€” so people keep working at a fraction of the + cost rather than stopping. + +Reaching the ceiling is not a failure β€” it means the budget was fully +utilized, and the decision is simply whether demonstrated demand justifies +next month's number. + +## At a glance + +| | Today | Proposed | +|---|---|---| +| Heavy user hits a limit | Locked out until the 1st | Waits until a stated time, e.g. 2:30 PM | +| Institutional total spend | Unenforced; sum of guesses | One hard, adjustable ceiling | +| Unused light-user budget | Stranded | Pooled β€” flows to productive use | +| Setting limits | Guesswork, tuned by fear of lockouts | Users Γ— target cost/user, plus two pacing values | +| Exceptions | Ad hoc | Auditable exemptions + self-expiring overrides, spend still visible | +| Raising the budget | β€” | Effective within a minute, mid-month | +| Visibility | Per-user usage only | Live spend vs. ceiling, burn rate, projection, distribution | + +## The decision requested + +1. Approve the paced-budget model (windows + pooled ceiling + backstop). +2. Set the initial target average cost per user (proposed: **$5/month**), + which fixes the ceiling. +3. Approve a phased rollout: pacing windows first, then ceiling enforcement + and the budget dashboard β€” each phase reversible and adjustable without + rework. + +*Engineering cost is modest: the model builds on cost tracking the platform +already performs; no new infrastructure of consequence.* diff --git a/docs/specs/quota-cooldown-windows.md b/docs/specs/quota-cooldown-windows.md new file mode 100644 index 000000000..9bb57f13b --- /dev/null +++ b/docs/specs/quota-cooldown-windows.md @@ -0,0 +1,689 @@ +# Quota Cooldown Windows + Platform Ceiling β€” pacing instead of a monthly cliff + +> Status: DRAFT (spec only, nothing built). Revised 2026-07-10 to add the +> **universal platform ceiling** as the fiscal-control layer (superseding the +> earlier statistical-overbooking policy). Replaces the current all-or-nothing +> enforcement UX β€” warn at a percent, hard-block until the 1st of next month β€” +> with a three-layer model: **anchored cooldown windows** (Claude-style +> "resets at 2:30 PM") pace individuals; a **hard, adjustable, platform-wide +> monthly ceiling** guarantees the institutional total; the per-user monthly +> limit is demoted to a generous **anti-runaway backstop**. + +## 1. Summary + +Three enforcement layers, each with one job, plus an observability surface: + +1. **Anchored cooldown window (pacing).** The platform defines one window + length β€” admin-adjustable via the platform-budget setting (`windowHours`, + default 5, env `QUOTA_WINDOW_HOURS` as fallback default); each tier + optionally gets a `window_cost_limit` (recommended default $2.00). A + user's window anchors at + their first turn; spend accumulates; exhausting it blocks until + `windowStart + QUOTA_WINDOW_HOURS` β€” an **exact, user-visible reset + timestamp**, hours away instead of weeks. Null `window_cost_limit` = no + window for that tier; per-tier opt-in is the rollout gate, no new global + flag. (Why the window *length* is platform-wide, not per-tier: Β§4.5.) +2. **Platform ceiling (fiscal guarantee).** One admin-set number β€” target + average cost per user Γ— user count (e.g. $5 Γ— 5,000 = $25,000/month) β€” + **enforced**, not aspirational. Fleet MTD spend is compared against an + enforcement line set below the stated ceiling by a reserve margin (default + 10%), with admin alert events at 80%/90%. Because enforcement is computed + per turn against live config (nothing is ever "marked blocked"), raising + the ceiling mid-month takes effect within the config cache TTL (~1 min) β€” + no deploy, no reset. Tiers/overrides can be explicitly **ceiling-exempt** + (spend still counted and reported; only enforcement is bypassed). +3. **Per-user monthly backstop ($30).** The existing `monthly_cost_limit`, + re-purposed: it marks **where the month's premium-model subsidy ends**, not + a guessed budget. Target behavior on reaching it is **degrade to an + economy model** (PR-7), with block-plus-early-warning as the bridge until + degrade lands; the ceiling protects the total (Β§12). + +Plus: **fleet budget-health admin view** β€” MTD spend vs. ceiling, burn rate, +projected exhaustion date, per-user spend distribution β€” from the existing +`SystemCostRollup` table and `PeriodCostIndex` GSI. Read-only; no +auto-actuation in v1. + +`action_on_limit: "degrade"` (continue on a cheap model instead of blocking) +is **PR-7 β€” scheduled, not optional**: with a burst-sized $2 window, honest +sustained use *can* reach the $30 backstop, so degrade is what keeps the +no-cliff promise there (decisions 14–15, Β§15). Until it lands, the backstop +blocks with early warning + the override lane as the documented bridge. +Extending degrade to ceiling behavior ("everyone rides the economy model near +the ceiling") stays roadmap. + +## 2. Why an *anchored* window + +Claude's own mechanic is not a rolling sum β€” it's a window that **starts at your +first message** and resets at a stated time. That shape is deliberately what we +copy, because it wins on all three axes that matter here: + +- **Explainable:** "You've used your 5-hour allowance β€” it resets at 2:30 PM" + beats both "your budget refills at $0.02/hour" (token bucket) and "your usage + over the trailing 5 hours exceeds…" (rolling sum, where the reset time is + unknowable in advance). +- **Cheap to store:** one small item per user, overwritten in place. A rolling + sum needs time-bucketed items and a multi-item read per check; a token bucket + is equally cheap but loses on explainability. +- **Cheap to enforce:** one extra comparison in `QuotaChecker`, one extra atomic + `ADD` in the existing per-turn write path. + +Don't derive the window budget by dividing the monthly budget into windows +($5 Γ· ~144 five-hour windows β‰ˆ 3Β’ β€” useless). The knobs solve different +problems: the **window** is sized to a genuinely heavy working session β€” +accounting for the fact that real users only have 2–3 usable windows in a +waking day (Β§12); the **backstop** marks where the month's premium subsidy +ends; the **ceiling** is sized to the institutional budget. The windows are also what make a shared +ceiling safe and fair: they bound every individual's burn rate, so no small +cohort can drain the pool quickly and the fleet trajectory stays smooth and +predictable. + +## 3. Current state (verified 2026-07-10) + +| Piece | Where | State | +|---|---|---| +| Tier model | `agents/main_agent/quota/models.py:26` (`QuotaTier`) | `monthly_cost_limit`, optional `daily_cost_limit`, `period_type: Literal["daily","monthly"]`, `soft_limit_percentage` (default 80), `action_on_limit: Literal["block","warn"]` | +| Check logic | `agents/main_agent/quota/checker.py:28` (`check_quota`) | Resolves tier β†’ reads monthly/daily aggregate β†’ warn at soft % (90% **hardcoded** at `checker.py:103`) β†’ block at 100% if `action_on_limit=="block"`. No-tier fails **closed** (`:47`); cost-read error fails **open** (`:78`) | +| Usage store | `apis/shared/storage/dynamodb_storage.py:312` (`update_user_cost_summary`) | `user-cost-summary` table, `PK=USER#{id}` / `SK=PERIOD#{YYYY-MM}`, atomic `ADD`, `GSI2 PeriodCostIndex` (`PERIOD#` / `COST#{padded-cents}`). **No TTL configured on this table** (`cost-tracking-tables-construct.ts:86` β€” only `sessions-metadata` has `timeToLiveAttribute`, `:49`) | +| Usage write (per turn) | `apis/app_api/sessions/services/metadata.py:190` β†’ `_update_cost_summary_async` (`:201`) | Derives `period` from message timestamp (`:293`), calls `update_user_cost_summary`. Also writes per-message `C#` records with 365-day TTL (`:151`) | +| Usage read | `apis/shared/costs/aggregator.py` (`CostAggregator.get_user_cost_summary`) | 30s in-process cache keyed `user+period` (`:20-27`) | +| Reset cadence | implicit | No reset job β€” the `PERIOD#` key rolls over at month boundary. Monthly cutoff lives in exactly two places: the storage key and `_get_current_period()` (`checker.py:188`) | +| Chat enforcement | `apis/inference_api/chat/routes.py:1116-1154` | Gated on `is_quota_enforcement_enabled()` + not resume/continuation; exceeded β†’ **conversational assistant message** (`stop_reason="quota_exceeded"`, `build_quota_exceeded_event`), not an HTTP error. Warning injected first-thing into the SSE stream by `stream_with_quota_warning()` (`:1729`) | +| API-key enforcement | `apis/inference_api/chat/converse_routes.py:369-393` | Same `check_quota` but raises `HTTPException(429)`. No-tier fails **open** here (`:376`) β€” pre-existing inconsistency with the chat path. `Retry-After` convention already used at `:354` | +| SSE events | `apis/shared/quota.py` (`QuotaWarningEvent:102`, `QuotaExceededEvent:120`, builders `:144-270`) | `reset_info` is a human string ("Quota resets in N day(s)") computed as days-to-month-end | +| Events audit | `agents/main_agent/quota/event_recorder.py`; `QuotaEvent.event_type: Literal["warning","block","reset","override_applied"]` (`models.py:136`) | Warning dedup 60 min; `record_reset` exists but has no caller | +| Unlimited tier | `checker.py:59` | `monthly_cost_limit == inf or >= 999999` β†’ skip all checks | +| Overrides | `QuotaOverride` (`models.py:206`) | Time-bounded custom/unlimited per-user override, resolved first β€” the standing mechanism for super users and temporary increases; kept and extended (Β§5) | +| Tier resolution | `agents/main_agent/quota/resolver.py:47` | Priority: override β†’ direct user β†’ app role β†’ JWT role β†’ email domain β†’ default tier; 5-min TTL cache | +| Admin API/UI | `apis/app_api/admin/quota/` ; SPA `admin/quota-tiers/` (tier-detail, assignment-list, override-list, event-viewer, quota-inspector) | Full tier/assignment/override CRUD exists β€” new fields slot into existing forms | +| Fleet rollups | `system-cost-rollup` table (`cost-tracking-tables-construct.ts:116`), written by `_update_system_rollups_async` (`dynamodb_storage.py`) | Daily/monthly/per-model system-wide aggregates. **Best-effort async write today** β€” becomes an enforcement input under the ceiling, needs hardening (Β§8). Inference-runtime read wiring (env + IAM) for this table **needs verification** β€” may currently be app-api-only | +| SPA quota surfaces | `components/quota-warning-banner/` + `services/quota/quota-warning.service.ts`; SSE dispatch `stream-parser-core.ts:659` | Banner shows severity, `formattedUsage`, `resetInfo` string | +| Frontend models | `admin/quota-tiers/models/quota.models.ts` | `WarningLevel = 'none'\|'80%'\|'90%'` mirrors backend | + +## 4. Key design decisions + +1. **Anchored window, not rolling sum, not token bucket.** See Β§2. *Rejected: + rolling sum over the per-message `C#` records β€” those are TTL'd and reading + them is a per-check range scan; also no predictable reset time. Rejected: + token bucket (continuous drip) β€” mathematically cleaner (no cliff, cooldown + exactly proportional to overdraft) but unexplainable to a student audience + and harder for admins to parameterize. Revisit if anchored windows prove too + coarse.* +2. **Window state is one reusable item per user in the `user-cost-summary` + table**: `PK=USER#{user_id}` / `SK=WINDOW#CURRENT`, attributes + `{windowStart: ISO, windowCost: Decimal, updatedAt}`. Overwritten in place + when a stale window is re-anchored β€” so no TTL needed (the table has none + configured) and no unbounded growth. *Rejected: hourly TTL buckets modeled on + `rate_limit.py` β€” right pattern for a rolling sum we aren't building, and + would require enabling TTL on the table (CDK change). Rejected: a new table β€” + both app-api (write) and the inference runtime (read via `CostAggregator`) + already have IAM + env wiring for `user-cost-summary`; zero CDK changes this + way.* +3. **The anchor is set by the write path, checked by the read path.** A turn's + cost lands after the turn finishes (`_update_cost_summary_async`), so the + window anchors at the first *completed* turn. The checker treats a missing or + expired item as "fresh window, usage 0". Consequence, accepted: enforcement + is check-before-turn against the balance at turn start β€” a single expensive + turn can overdraw the window. The overdraft just makes the cooldown feel + fuller; identical tolerance to today's monthly check. +4. **Concurrency via condition-expression two-step.** Live window: `UpdateItem` + `ADD windowCost :cost` with condition `windowStart >= :cutoff` + (`cutoff = now - QUOTA_WINDOW_HOURS`). On `ConditionalCheckFailed` (item + missing or stale): `PutItem` a fresh `{windowStart: now, windowCost: cost}` + with condition `attribute_not_exists(SK) OR windowStart < :cutoff`; if + *that* races and fails, retry the `ADD` once. Same atomic-increment + discipline as the monthly summary. +5. **Window length is platform-wide; only the dollar budget is per-tier.** The + write path must stay tier-agnostic (resolving a quota tier inside the cost + write would drag `QuotaResolver` into `metadata.py` and add a resolve per + turn), so the writer re-anchors on a fixed horizon. If tiers could set their + own `window_hours`, any tier window shorter than that horizon would get + enforcement holes: the accumulator keeps growing past the tier's window + while the checker already reads it as stale/zero, so the user is only ever + enforced during the first `window_hours` of each horizon period. Rather than + ship subtly-broken flexibility, v1 pins one platform-wide window length, + **stored in the platform-budget sentinel (`windowHours`, default 5; env + `QUOTA_WINDOW_HOURS` as fallback default)** and read identically by writer + (app-api) and checker (inference-api) through the same short-TTL cached + settings helper in `apis.shared` β€” tiers configure only + `window_cost_limit`. Being a stored setting rather than env makes the + length admin-adjustable in minutes, a deliberate pilot requirement + (Β§12 playbook). Per-tier lengths are a follow-on with a known path + (Β§16.1). Changing the length mid-flight causes a transient + under/over-count on existing anchors for at most one window; acceptable. +6. **Per-tier opt-in is the rollout gate; no new global flag.** + `window_cost_limit` is nullable β€” set β†’ enforced, null β†’ skipped. The admin + turns it on tier-by-tier; `ENABLE_QUOTA_ENFORCEMENT` remains the existing + global kill switch above it. The ceiling has its own independent gate: no + ceiling setting stored β†’ no ceiling check. Unlimited tiers (`checker.py:59`) + continue to skip per-user checks (window + backstop) β€” but **not** the + ceiling, unless also explicitly ceiling-exempt (decision 8). +7. **All three checks are independent; block reports the binding one.** Order: + resolve tier β†’ ceiling check (unless exempt) β†’ window check β†’ monthly + backstop check. `QuotaCheckResult` gains + `blocked_by: Literal["ceiling","window","monthly"] | None` plus window + fields (Β§5). A window block carries `resets_at` (exact ISO timestamp); a + monthly block keeps days-to-month-end copy; a ceiling block gets apologetic + it's-not-you copy (Β§9). +8. **Ceiling exemption is an explicit flag, separate from "unlimited".** An + unlimited personal quota and permission-to-spend-past-the-institutional-cap + are different grants; bundling them would let every super user silently + weaken the fiscal guarantee. `exempt_from_ceiling: bool = False` on + `QuotaTier` and `QuotaOverride`. Exempt users' spend is **always still + counted** in summaries, rollups, and the dashboard β€” exemption bypasses + enforcement only. The guarantee stays honest and auditable: "spend cannot + exceed the ceiling except for explicitly designated users, whose spend is + fully visible." +9. **Enforcement is computed per turn, never stored.** There is no + blocked-state to set or clear anywhere β€” every check compares live usage + against current config. Consequence, and a headline property: raising the + ceiling (or any limit/override) mid-month unblocks affected users on their + next message, within the config cache TTL (~1 min). No deploy, no restart, + no reset job. +10. **Ceiling enforcement line = ceiling Γ— (1 βˆ’ reserve).** Default reserve + 10%, admin-adjustable. Alerts fire at 80% and 90% of the *stated* ceiling + (admin alert events via `QuotaEventRecorder`, new system-scoped event + type); enforcement triggers at the reserve line. The gap is deliberate + headroom for a human decision ("raise it or ride it out") made during a + slow, visible approach rather than after an overrun. +11. **The ceiling reads the monthly `system-cost-rollup` item through a + 30–60s cache.** Fleet spend moves slowly relative to one turn; this is the + same staleness tolerance the per-user checks already run with, and adds no + per-turn Dynamo read beyond one cached fetch. The stored setting + (`monthlyCeiling`, `reservePercent`) is a `SYSTEM_SETTINGS#` sentinel item + in the auth-providers table (the established first-boot/skills-mode + pattern; zero CDK) read through the same short cache. +12. **Fleet policy is enforcement + observability, not a control algorithm.** + The ceiling enforces; the dashboard informs; the admin decides. *Rejected + for v1 (unchanged from the overbooking draft): any scheduled job that + auto-tightens limits β€” a silent mid-month budget cut is a support-ticket + generator.* +13. **Unify the warning thresholds while we're in the file.** The 90% warning + is hardcoded (`checker.py:103`) while the soft threshold is tier-config. + Add `critical_warning_percentage` (default 90) next to + `soft_limit_percentage` so monthly and window warnings derive from tier + config, and widen the frontend `WarningLevel` union from the hardcoded + `'80%'|'90%'` literals to a string. Low-risk, additive, kills a footgun. +14. **Window sized for real bursts, using waking-hours math.** The scary + arithmetic against a bigger window ("$2 Γ— 4.8 windows/day = $9.60") assumes + 24/7 usage; real students and staff have 2–3 usable windows in a waking + day, so the realistic worst honest day at a $2 window is $4–6. + Premium-model work (long documents, RAG, agentic tool loops with several + model calls per user turn) runs $0.30–1.00+ per task β€” $1 is a *chat* budget, + not a *work* budget; $2 is the burst size. Consequence, accepted + deliberately: an enthusiastic daily user can now reach the $30 backstop + (~15 maxed windows) β€” which is why the backstop's action must become + degrade (decision 15). *Rejected: a $1.50 "invariant-preserving" middle β€” + only half-fixes bursts and leaves no margin.* +15. **Degrade-at-backstop is the target behavior; block-with-warning is the + bridge.** Reaching $30 should drop the user to an economy model for the + rest of the month, not lock them out β€” otherwise the $2 window recreates a + miniature monthly cliff for the heaviest honest users. PR-7 is therefore + scheduled, not open-endedly deferred; until it lands, the 80% warning + ($24, days of notice) + override lane are the documented path. + *Rejected as an ADDITIONAL cap stacked under the $30 monthly: a weekly + limit β€” tight enough to matter (~$8) it blocks a legitimate finals-crunch + week (~$16–20 at two maxed windows/day), and stacking horizons multiplies + knobs without adding protection the ceiling doesn't already provide. A + weekly backstop **instead of** monthly is a different matter β€” a + legitimate alternative horizon (Claude's own session+weekly pairing), + supported per tier via `period_type: "weekly"`; see decision 16. + Rejected: dynamic windows scaled to remaining-backstop Γ· + days-left β€” it treats the backstop as an entitlement (resurrecting the + per-person-allowance mental model this design kills), collapses + explainability (the limit changes daily), and has inverted fleet timing: + everyone's burn rate scales UP at month end, exactly when fleet spend is + closest to the ceiling, destroying the smooth-trajectory property that + makes a hard shared ceiling safe. The fleet-safe dynamic direction β€” + windows tightening as the pool depletes β€” is the deferred + auto-actuation/degrade territory, not per-user scaling.* +16. **The backstop horizon is a per-tier choice β€” monthly or weekly; the + institution decides.** With the ceiling owning fiscal protection and + degrade removing lockout severity, weekly-vs-monthly is purely a question + of which burst *shape* a population needs. **Monthly** permits week-scale + bursts β€” one crunch week (~$18) plus three quiet ones β€” matching + deadline-clustered academic use. **Weekly** is the safe partner for a + bigger session window ($3–4): it bounds the integral so the rate can + rise, a blown week costs days not weeks, and every Monday is a fresh + start β€” Claude's own session+weekly pairing exists for exactly this + reason (the 5h window caps burst rate; the weekly cap bounds 24/7-style + sustained use that per-session caps can't see). `period_type` gains + `"weekly"` and the write path maintains a weekly aggregate + unconditionally alongside the monthly one (one extra atomic ADD β€” the + same tier-agnostic-writer argument as the window accumulator), so + flipping a tier's horizon needs no backfill and takes effect in minutes + like every other knob. As an open-source platform this stays + configuration, not policy: each adopting institution picks the horizon β€” + and every other number β€” per tier. The observe-only pilot month reveals + which shape each population actually has: deadline-clustered β†’ monthly; + steady-heavy β†’ weekly + bigger window. + +## 5. Data model changes + +### `QuotaTier` (`agents/main_agent/quota/models.py`) β€” additive + +```python +window_cost_limit: Optional[Decimal] = Field(None, alias="windowCostLimit", gt=0) +weekly_cost_limit: Optional[Decimal] = Field(None, alias="weeklyCostLimit", gt=0) +exempt_from_ceiling: bool = Field(default=False, alias="exemptFromCeiling") +critical_warning_percentage: Decimal = Field(default=Decimal("90.0"), + alias="criticalWarningPercentage", ge=0, le=100) +``` + +`period_type` widens from `Literal["daily","monthly"]` to +`Literal["daily","weekly","monthly"]` (decision 16). Limit resolution follows +the existing daily pattern (`checker.py:91`): `weekly` β†’ `weekly_cost_limit`, +`daily` β†’ `daily_cost_limit`, fallback `monthly_cost_limit`. + +Existing tier items deserialize unchanged (new fields optional or defaulted). + +`QuotaOverride` gains optional `window_cost_limit` and `exempt_from_ceiling` +too β€” a time-bounded override (already resolved ahead of everything, already +self-expiring via `valid_from`/`valid_until`) is the standing instrument for +super users and temporary increases ("capstone crunch week"), no cleanup +required. + +### Ceiling setting (sentinel item, auth-providers table) + +``` +PK = SK = SYSTEM_SETTINGS#platform-budget +monthlyCeiling = Decimal dollars (absent β‡’ ceiling disabled) +reservePercent = Decimal, default 10.0 +targetAvgCostPerUser = Decimal (informational; drives dashboard context) +windowHours = Decimal, default 5 (absent β‡’ env QUOTA_WINDOW_HOURS β‡’ 5) +updatedAt / updatedBy +``` + +### Window state item (`user-cost-summary` table) + +``` +PK = USER#{user_id} +SK = WINDOW#CURRENT +windowStart = ISO 8601 UTC of the anchoring turn +windowCost = Decimal dollars (atomic ADD) +updatedAt = ISO 8601 UTC +``` + +One item per user, ever. `WINDOW#CURRENT` sorts outside the `PERIOD#` prefix so +existing period queries are unaffected; the item carries no GSI2 attributes so +it never appears in `PeriodCostIndex`. + +### Weekly aggregate item (`user-cost-summary` table) + +``` +PK = USER#{user_id} +SK = PERIOD#{ISO week} e.g. PERIOD#2026-W28 (strftime "%G-W%V", UTC) +totalCost (atomic ADD), totalRequests, lastUpdated +``` + +A slim sibling of the monthly summary: totals only β€” no GSI2 attributes (the +dashboard's distribution queries stay monthly-keyed) and no per-model +breakdown. Writer and checker must derive the week key identically +(`%G-W%V`, UTC β€” ISO week, resets Monday 00:00 UTC). + +### `QuotaCheckResult` (`models.py:162`) β€” additive + +```python +blocked_by: Optional[Literal["ceiling", "window", "monthly"]] = Field(None, alias="blockedBy") +window_usage: Optional[Decimal] = Field(None, alias="windowUsage") +window_limit: Optional[Decimal] = Field(None, alias="windowLimit") +window_resets_at: Optional[str] = Field(None, alias="windowResetsAt") # ISO 8601 UTC +``` + +`warning_level` loosens from `Literal["none","80%","90%"]` to `str` (values +now derived from tier config β€” decision 13). + +### `QuotaEvent` (`models.py:136`) + +`event_type` Literal gains `"window_block"`, `"window_warning"`, +`"ceiling_block"`, and `"ceiling_alert"` (the last recorded system-scoped β€” +`PK=USER#__SYSTEM__` β€” at the 80%/90% thresholds, deduped like warnings). +`metadata` carries `{windowStart, windowResetsAt}` / `{fleetSpend, ceiling}` +as applicable. `QuotaEventRecorder` gets the corresponding record methods. + +## 6. Write path + +In `_update_cost_summary_async` (`metadata.py:201`), after the existing +`update_user_cost_summary` call, add: + +```python +await storage.update_user_window_cost(user_id=user_id, cost_delta=cost, + timestamp=timestamp) +await storage.update_user_weekly_cost(user_id=user_id, cost_delta=cost, + timestamp=timestamp) # PERIOD#%G-W%V +``` + +`DynamoDBStorage.update_user_window_cost` implements the two-step conditional +write from decision 4, with `cutoff = timestamp - QUOTA_WINDOW_HOURS`. It is +**tier-agnostic and unconditional** β€” it always accumulates, even for users on +window-less or unlimited tiers (a handful of wasted `ADD`s beats a resolver +dependency in the write path, and it means flipping `window_cost_limit` on a +tier takes effect against already-accumulated state β€” no "first window is +free" gap). Failures log and swallow, exactly like the monthly-summary write β€” +cost accounting must never fail a turn. + +Writer and checker MUST share the same window-length source: a +`get_quota_window_hours()` helper in `apis.shared` β€” resolution order +**sentinel `windowHours` (through a ~60s cache) β†’ env `QUOTA_WINDOW_HOURS` β†’ +`5`** β€” consumed by `dynamodb_storage.py` (app-api side) and `QuotaChecker` +(inference side). Cache skew between the two processes is bounded by the TTL +and costs at most one transiently mis-windowed anchor, same tolerance as a +mid-flight length change (decision 5). The sentinel lives in the +auth-providers table, whose name + IAM read are already wired to both +containers, so no CDK change. + +## 7. Check flow (`QuotaChecker.check_quota`) + +``` +resolve tier (unchanged; fail-closed on none) +exempt = tier.exempt_from_ceiling or (active override with exempt_from_ceiling) + +ceiling check (skip if exempt, or no ceiling setting stored): + fleet = rollup_reader.get_monthly_fleet_spend() # 30–60s cached + line = ceiling.monthlyCeiling * (1 - reservePercent/100) + record ceiling_alert events at 80%/90% of stated ceiling (deduped) + if fleet >= line: + β†’ allowed=False, blocked_by="ceiling" + +if unlimited tier: skip window + monthly (existing behavior, checker.py:59) + +window check (skip if tier.window_cost_limit is None): + item = aggregator.get_user_window(user_id) # 30s cached + fresh = item and item.windowStart + QUOTA_WINDOW_HOURS > now + window_usage = item.windowCost if fresh else 0 + resets_at = item.windowStart + QUOTA_WINDOW_HOURS if fresh else None + if window_usage >= tier.window_cost_limit and tier.action_on_limit == "block": + β†’ allowed=False, blocked_by="window", window_resets_at=resets_at + elif window pct >= soft/critical thresholds: + warning_level set from window (window warnings win ties over monthly β€” + they're the more actionable signal) + +monthly/weekly/daily backstop: same logic (checker.py:89-165); the period key +and limit resolve from tier.period_type (weekly β†’ PERIOD#%G-W%V + +weekly_cost_limit β€” decision 16); critical threshold now from +tier.critical_warning_percentage instead of hardcoded 90; reset copy for +weekly tiers = "resets Monday" with the exact timestamp +``` + +Failure semantics unchanged: any usage-read error (window, monthly, or fleet) +fails open with a logged warning, mirroring the existing monthly path +(`checker.py:78`). + +## 8. Hardening the rollup path (ceiling prerequisite) + +The ceiling turns `system-cost-rollup` from an observability aggregate into an +**enforcement input**. Two items before ceiling enforcement ships: + +1. `_update_system_rollups_async` is best-effort today β€” silent failures would + under-count fleet spend against the ceiling. Minimum bar: failures must log + at ERROR with a metric/alarm, and the budget-health view shows a + reconciliation check (sum of `PeriodCostIndex` user totals vs. the rollup + item) so drift is visible. Full fix (retry queue) only if drift is observed. +2. Verify the inference runtime has env + IAM read access to the + `system-cost-rollup` table β€” the per-user summary table is wired to both + sides, but rollups may be app-api-only today. If not, a small CDK/env + addition (thread through `PlatformComputeRefs`, per house convention). + +## 9. SSE / messaging + +- `QuotaWarningEvent` / `QuotaExceededEvent` (`apis/shared/quota.py`): add + optional `resetsAt` (ISO UTC) + `blockedBy`; `formattedUsage` shows the + binding limit's numbers (`"$1.72 / $2.00"` for a window event). +- `build_quota_exceeded_event` gains three branches: + - **window:** *"You've used your 5-hour allowance. It resets at {time} β€” + your conversation and history are untouched, just check back then."* + - **monthly:** existing copy (days to month end), plus a pointer to the + exception/override request path β€” bridge behavior until PR-7 replaces + the block with degrade. + - **ceiling:** apologetic, explicitly not-the-user's-fault: *"The + platform's shared monthly budget has been reached, so responses are + paused for everyone β€” this isn't about your usage. Service resumes when + the budget is increased or the month resets."* No usage table (the user's + own numbers are irrelevant and showing them implies blame). +- Delivery unchanged β€” blocks still stream as conversational assistant + messages (`routes.py:1141-1154`); warnings still injected by + `stream_with_quota_warning` (`:1729`) with the new fields riding along. +- SPA: `stream-parser-types.ts` validators + `quota-warning.service.ts` accept + the new optional fields; banner renders `resetsAt` in local time with a + countdown for window blocks ("Resets in 2h 14m Β· 2:30 PM"); ceiling blocks + render a distinct platform-wide-notice style, not the personal-usage style. + Additive β€” events without the new fields render exactly as today. + +### Always-visible quota status (user-facing endpoint) + +Everything a personal usage meter needs already exists per user by +construction: the `WINDOW#CURRENT` item (window usage + anchor β†’ percentage + +reset time), the `PERIOD#` aggregates (monthly or weekly usage), and the +resolved tier's limits. But SSE events only surface at warning/exceeded +thresholds β€” an always-visible meter needs one thin read endpoint: + +`GET /quota/status` on app-api (`Depends(get_current_user_from_session)` per +the auth rule), returning the caller's own state: + +```json +{ + "window": { "usage": 0.80, "limit": 2.00, "percentage": 40, + "resetsAt": "2026-07-10T21:30:00Z" }, // null if no window + "period": { "type": "monthly", "usage": 11.20, "limit": 30.00, + "percentage": 37, "resetsAt": "2026-08-01T00:00:00Z" }, + "unlimited": false, + "exemptFromCeiling": false +} +``` + +This is a self-scoped variant of what the admin quota inspector already +computes per user (`service.py:294`) β€” same resolution + reads the checker +uses, no new data. Ships in PR-4; the usage settings page renders both meters +("$0.80 of $2.00 this window Β· resets 2:30 PM"; "$11.20 of $30.00 this +month"), and unlimited tiers render as "no limits apply". A compact meter +near the composer is a possible later add once the endpoint exists. + +## 10. API-key / Converse path + +`converse_routes.py:369-393`: on `blocked_by == "window"`, the 429's +`Retry-After` becomes the real seconds until `window_resets_at` (today a quota +block sends no retry hint). Ceiling and monthly blocks keep plain 429 +semantics. Also fix the no-tier fail-open (`:376`) to fail closed, matching +the chat path β€” a one-line consistency fix riding along. + +## 11. Admin API + UI + +- **Tier CRUD** (`apis/app_api/admin/quota/`): pass through + `window_cost_limit`, `exempt_from_ceiling`, `critical_warning_percentage`; + tier-detail form gains a "Cooldown window" section (single dollar field; + helper text states the platform window length, read from a config echo on + the tier-list response so the SPA doesn't hardcode "5 hours") and an + "Exempt from platform ceiling" toggle with a warning-styled description. +- **Overrides**: form gains optional `window_cost_limit` + + `exempt_from_ceiling` β€” the super-user / temporary-increase lane (Β§5). +- **Quota inspector** (`GET /admin/quota/users/{user_id}`, `service.py:294`): + adds live window state (`windowUsage`, `windowLimit`, `windowStart`, + `windowResetsAt`, in-cooldown flag) and ceiling exemption status. +- **Event viewer**: renders `window_block` / `window_warning` / + `ceiling_block` / `ceiling_alert`. + +### Tier catalog guidance (documentation, not code) + +Fewer tiers, not more β€” with the ceiling holding the fiscal line, tiers only +encode *pacing*, so two or three cover the institution: + +| Tier | Window / 5h | Monthly backstop | Ceiling | +|---|---|---|---| +| Default (students, staff) | $2.00 | $30 | counted | +| Faculty / power users | $3.00–4.00 | $40–60 monthly **or** $15–18 weekly | counted | +| Research / grant-funded | none | very high | exempt (own funding) | +| Service accounts / headless agents | tight | tight | counted | + +Assignments carry over untouched (resolver ladder, Β§3): the default tier makes +the whole population paced on day one; JWT-role/email-domain rungs mean tier +membership tracks Entra automatically. + +## 12. Recommended initial configuration + sizing rationale + +| Knob | Value | Rationale | +|---|---|---| +| Window length (`windowHours` sentinel; env fallback) | 5 | Claude parity; matches a real work session | +| Default-tier `window_cost_limit` | **$2.00** | A genuinely heavy 5-hour *work* session: premium-model tasks (long documents, RAG, agentic tool loops) run $0.30–1.00+ each, so $1 covers chat but not bursts. Waking-hours math bounds the realistic worst honest day at 2–3 windows = $4–6 (decision 14) | +| Default-tier `monthly_cost_limit` (backstop) | **$30** (6Γ— target) | Where the month's premium-model subsidy ends. With a $2 window an enthusiastic daily user *can* reach it (~15 maxed windows) β€” by design its action becomes degrade-to-economy-model (PR-7, decision 15), not lockout; on bridge behavior, the 80% warning at $24 gives days of notice and the override lane is the exception path. $30 naturally fits the academic rhythm: one crunch week (~$16–20 at two maxed windows/day) + three normal weeks. Even in absurd scenarios it barely moves the average (2% of users at $30 adds $0.60 to the mean) β€” the ceiling, not the backstop, protects the budget | +| Backstop horizon (`period_type`) | **monthly** (default) | Per-tier choice (decision 16): monthly fits deadline-clustered student use; **weekly ($15–18) is the safe partner for a $3–4 power-user window** β€” pilot data decides which shape each population has | +| `targetAvgCostPerUser` | **$5** | Institutional target; informational context on the dashboard | +| `monthlyCeiling` | users Γ— $5 | The enforced total | +| `reservePercent` | 10 | Enforcement at 90% of stated budget = decision headroom | + +**Retuning after month one:** set the backstop at ~p99.5 of the observed +user-spend distribution (budget-health view provides it). **The health metric +for "are we punishing power users" is window-block events per week** β€” the +window is the limit honest heavy users can actually feel. Backstop events +should be rare; under bridge behavior each one deserves a look in the event +log, and their frequency is the direct measure of how urgently PR-7 is +needed. + +### Pilot tuning playbook + +Every number in the system is admin-adjustable at runtime, and because +enforcement is computed per turn (decision 9), a change applies to everyone's +next message once the relevant cache rolls β€” no deploy, no reset, nobody to +unblock. The symptomβ†’knob mapping, for responding to pilot feedback same-day: + +| Observed during pilot | Knob to turn | Where | Effective within | +|---|---|---|---| +| Too many users hitting cooldowns | Tier `window_cost_limit` ↑ | Tier form | ≀5 min (resolver cache) | +| Cooldowns feel too long to wait out | `windowHours` ↓ (budget ↓ proportionally) | Platform-budget editor | ~1 min | +| One cohort systematically constrained | New tier / reassignment | Tier + assignment CRUD | ≀5 min | +| One person in a legitimate crunch | Self-expiring override | Override form | ≀5 min | +| Fleet trending hot/cold vs. target | `monthlyCeiling` / `reservePercent` | Platform-budget editor | ~1 min | +| Honest users reaching the backstop | Tier `monthly_cost_limit` ↑; accelerate PR-7 | Tier form | ≀5 min | +| Power users want bigger bursts than a raised window safely allows | Switch tier to `period_type: "weekly"` + raise the window (decision 16) | Tier form | ≀5 min | +| Warnings too naggy / too late | Tier `soft_limit_percentage` / `critical_warning_percentage` | Tier form | ≀5 min | +| Any enforcement feels too harsh mid-pilot | Tier `action_on_limit` β†’ `"warn"` | Tier form | ≀5 min | + +**Recommended pilot phase 1: observe-only.** Set the pilot tier's +`action_on_limit` to `"warn"` β€” the entire mechanism runs and records +(window/backstop events, warnings, fleet telemetry, dashboard) but blocks +nobody. One month of that yields the would-have-been cooldown rate and the +real spend distribution at zero user risk; then flip to `"block"` with +numbers the data has already validated. Phase 1 provably cannot harm a user, +which is also the cleanest possible answer to pilot-risk questions. + +Diagnosis reads straight off the telemetry: window-block (or would-have- +blocked warning) events β†’ window budget; backstop events β†’ backstop size or +PR-7 urgency; ceiling gauge slope β†’ ceiling/reserve; a skewed per-tier +distribution β†’ tier design. Each failing pilot criterion points at exactly +one dial. + +The observe-only month also answers the **horizon question** (decision 16): +because the write path maintains monthly and weekly aggregates for everyone, +the pilot data shows each population's spend shape directly β€” +deadline-clustered (a few hot weeks, quiet otherwise β†’ monthly backstop fits) +vs. steady-heavy (consistent weeks β†’ weekly backstop + a bigger window fits). +Choosing the horizon per tier is then a data read, not a debate. + +## 13. Fleet budget health (dashboard) + +New read-only admin surface: `GET /admin/costs/budget-health?period=YYYY-MM` +(lives in `apis/app_api/admin/costs/` β€” it reads cost data, not quota config). + +| Field | Source | +|---|---| +| `totalSpendMTD`, `perModelBreakdown` | `system-cost-rollup` monthly item | +| `ceiling`, `reservePercent`, `enforcementLine`, `targetAvgCostPerUser` | sentinel setting | +| `activeUsers`, `avgCostPerUser` | `PeriodCostIndex` partition count; quotient | +| `burnRate`, `projectedEOM`, `projectedExhaustionDate` | daily rollups; linear extrapolation | +| `distribution` (p50/p90/p95/p99, top-N spenders) | `PeriodCostIndex` sorted by `COST#` | +| `reconciliation` (rollup vs. summed user totals) | drift check per Β§8 | +| `exemptSpendMTD` | sum over users on exempt tiers/overrides β€” keeps the guarantee auditable | + +SPA page (`admin/costs/`): spend-vs-ceiling gauge with the 80/90/enforcement +markers, burn-rate trend, projected exhaustion date, distribution curve, +exempt-spend callout, and the platform-budget editor +(ceiling/reserve/target/window-length, admin-gated). + +## 14. PR breakdown + +All PRs target `develop`, additive schemas, safe to ship dark. + +- **PR-1 β€” window data + write path (dark).** `QuotaTier`/`QuotaOverride`/ + `QuotaEvent` model fields; `get_quota_window_hours()` shared helper + (sentinel β†’ env β†’ default resolution from day one, so the length is + admin-adjustable as soon as an editor exists); + `update_user_window_cost` two-step conditional write + + `update_user_weekly_cost` slim atomic ADD (`PERIOD#%G-W%V`); hook both into + `_update_cost_summary_async`; `CostAggregator.get_user_window` with its own + cache key. Unit tests incl. the re-anchor race and week-key derivation. + Nothing reads the new tier fields yet. +- **PR-2 β€” window enforcement.** `QuotaChecker` window check + threshold + unification (decision 13) + weekly `period_type` resolution (decision 16); + `QuotaCheckResult` fields; event recorder types; SSE event fields + window + copy; chat path passthrough; converse path `Retry-After` + fail-closed fix. + Tier config remains the gate β€” behavior unchanged until an admin sets a + `window_cost_limit` or flips a `period_type`. +- **PR-3 β€” admin API + UI for windows.** Tier/override CRUD passthrough, tier + form section, exemption toggle (field only β€” ceiling not yet enforced), + quota inspector window state, event viewer types, and a minimal + window-length setting editor (the sentinel's `windowHours` only β€” the full + platform-budget editor arrives with PR-6, but pilot tuning needs the length + dial before then). Windows fully operable β€” and pilot-tunable β€” after this. +- **PR-4 β€” SPA user surfaces.** `GET /quota/status` endpoint (Β§9) + + stream-parser validators, banner countdown + local-time reset rendering, + usage settings page renders the window and period meters + ("$0.80 of $2.00 this window Β· resets 2:30 PM"). +- **PR-5 β€” ceiling enforcement.** Sentinel setting + admin CRUD; rollup-read + helper with cache; checker ceiling step + exemption resolution; `ceiling_*` + events; ceiling SSE copy + SPA notice style; rollup hardening + IAM/env + verification (Β§8). Absent setting = disabled, so ships dark. +- **PR-6 β€” budget-health dashboard.** Endpoint + SPA page per Β§13, including + the ceiling editor. +- **PR-7 β€” degrade at the backstop (`action_on_limit: "degrade"`).** + Scheduled, not optional (decision 15): completes the no-cliff promise for + the $30 backstop. See Β§15. Extending degrade to ceiling and window blocks + is a natural follow-on once the mechanism exists. + +Rollout order is deliberate: windows land and pace the population *before* +ceiling enforcement exists, so by the time the ceiling is live the fleet +trajectory is already smoothed β€” the ceiling should be boring on day one. + +## 15. PR-7: degrade instead of block + +Limit reached β†’ continue on a cheap model (Haiku/Nova-class) instead of +stopping, messaged as "you're on the economy model until {reset}." + +**Committed scope: the monthly backstop.** With a $2 window, honest sustained +use can reach $30, and degrade is what keeps that from being a miniature +monthly cliff (decision 15) β€” the heavy user's month becomes "work hard all +month, maybe finish it on the economy model, full reset on the 1st." It is +sequenced last because it's the one piece that isn't additive: model +selection isn't quota-aware today, and forcing an override touches agent +construction and the per-session agent cache key +(`inference_api/chat/service.py`). Until it lands, the backstop blocks with +the 80% warning + override lane as the bridge, and backstop-event frequency +(Β§12) measures the urgency. + +Prereqs from earlier PRs: `blocked_by` on the check result (PR-2/5) and a +tier-configured degrade model id; pricing for the degraded model already +lives in `ManagedModels`. Natural follow-ons once the mechanism exists: +degrade at the ceiling ("everyone rides the economy model for the rest of the +month" beats a platform pause) and optionally at the window. + +## 16. Open questions + +1. **Per-tier window lengths.** v1 deliberately pins one platform-wide length + (decision 5). If heterogeneous lengths are ever wanted, the clean path is + making the *write* tier-aware without a resolver call: the enforcement path + already resolved the tier that turn, so stamp the resolved window length + into the turn's message metadata and let `_update_cost_summary_async` pass + it through to the anchor arithmetic. Small, but touches the metadata + contract β€” build it when someone actually asks for a non-5h window. +2. **Per-unit ceilings.** If a college funds its own power users, that's the + seed of per-unit (per-tier or per-department) ceilings with their own + budgets. Needs tier/unit attribution on cost summaries β€” explicitly out of + v1; the `exempt_from_ceiling` flag is the v1 answer for separately-funded + cohorts. +3. **Should a window block suppress the monthly warning banner?** Proposed: + yes β€” one banner, binding constraint only. SPA-side concern; decide in + PR-4. +4. **Daily `period_type` tiers + windows.** Nothing conflicts (the window is + orthogonal to the aggregate period), but the admin form should discourage + configuring both a daily limit and a window β€” they overlap in purpose. Copy + question, not a code question. +5. **Ceiling-approach notification routing.** `ceiling_alert` events land in + the event log + dashboard in v1; wiring them to email/Teams for admins is + an obvious later add once a platform notification channel exists. diff --git a/frontend/ai.client/angular.json b/frontend/ai.client/angular.json index e7490d831..a035b0eb4 100644 --- a/frontend/ai.client/angular.json +++ b/frontend/ai.client/angular.json @@ -102,7 +102,10 @@ } }, "test": { - "builder": "@angular/build:unit-test" + "builder": "@angular/build:unit-test", + "options": { + "setupFiles": ["src/test-setup.ts"] + } } } } diff --git a/frontend/ai.client/package-lock.json b/frontend/ai.client/package-lock.json index 8562e0af9..e9c3db137 100644 --- a/frontend/ai.client/package-lock.json +++ b/frontend/ai.client/package-lock.json @@ -1,12 +1,12 @@ { "name": "ai.client", - "version": "1.4.0", + "version": "1.5.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "ai.client", - "version": "1.4.0", + "version": "1.5.0", "dependencies": { "@angular/cdk": "21.2.14", "@angular/common": "21.2.17", diff --git a/frontend/ai.client/package.json b/frontend/ai.client/package.json index 4a31510c0..2351021cb 100644 --- a/frontend/ai.client/package.json +++ b/frontend/ai.client/package.json @@ -1,6 +1,6 @@ { "name": "ai.client", - "version": "1.4.0", + "version": "1.5.0", "scripts": { "ng": "ng", "start": "ng serve", diff --git a/frontend/ai.client/src/app/admin/admin.layout.ts b/frontend/ai.client/src/app/admin/admin.layout.ts index 3b1db09ef..035a290be 100644 --- a/frontend/ai.client/src/app/admin/admin.layout.ts +++ b/frontend/ai.client/src/app/admin/admin.layout.ts @@ -79,7 +79,7 @@ interface NavGroup {
-