Skip to content

usage hook: partial paid tokens go unmetered when a stream is rescued mid-answer by a free rung #76

Description

@sre-helmcode

Follow-up required by the Security review of helmcode/nan-devops#358 (approved on the condition that this issue exists before merge).

The gap

PR #358 gives every fallback group a terminal rung on qwen3.6, so a stream that dies after a failover gets rescued instead of returning 502. That is a clear win for the member, but it moves one accounting case from "metered" to "metered nowhere".

Verified in scripts/litellm-usage-hook/usage_hook.py:

  • Before #358. The stream dies. async_post_call_streaming_iterator_hook never sets finished, so its finally calls _schedule_stream_abort_track(model=_model_name_request(request_data)). That name is the REQUESTED group, which is gated, so the OpenRouter tokens already streamed are posted against the member's cap. The member gets a 502 and is charged for it.
  • After #358. The rescue happens inside LiteLLM's stream wrapper, below the proxy hook. The hook sees one continuous stream ending cleanly, finished becomes True, and the abort meter never fires. async_log_success_event then resolves the model through _model_name_event, gets qwen3.6, which is not in GATED_MODELS, and returns before /track. The partial paid tokens are metered nowhere.

Measured volume on community, 24h to 2026-09-15: 67 requests out of roughly 426,000, i.e. 0.016%, and only the partial tokens before the cut. Enterprise was never separately counted, so its number is unknown rather than small.

Why it was not fixed in #358

The obvious one-liner ("if the served group is un-gated but the requested group is gated, meter against the requested group") would also change long-standing deliberate behaviour: a primary that falls straight to qwen3.6 without burning a single paid token is not metered today, on purpose, because that traffic costs GPU time we already rent. Applying the one-liner would start charging those members, on a volume far larger than 67/day. It would trade a 0.016% visibility gap for over-charging at scale.

Closing this properly needs information the hook does not read today: something that distinguishes "fell to free AFTER burning paid tokens" from "went to free immediately". Candidates are the router's failover trail (previous_models) or the cost on the standard_logging_object.

The part that is a product decision, not an accounting one

Raised by the UX reviewer and worth deciding explicitly rather than inheriting:

Fixing this takes away from the member something #358 gives them. Once the hook can tell the two cases apart, those partial tokens start being charged against the gated cap again, and the member sees consumption for a stream whose model was swapped mid-answer without telling them (the model field changes mid-SSE and nothing else does). Charging for that may well be right, but it should be a deliberate call, not a side effect of the metering fix.

Two sub-questions worth answering at the same time:

  1. Should the caller be told a fallback served the request at all? A response header (x-litellm-fallback-served-by or similar) is cheap and turns "the member cannot know" into "the member can know". Today the only signal is the mid-stream model change, which practically no OpenAI-compatible client reads.
  2. LiteLLM's continuation prompt is hardcoded English ("You are a helpful assistant... Your response should be in continuation of this text:"). On a Spanish-speaking community that can flip the language of an answer mid-sentence.

Also worth folding in (non-blocking findings from the same review)

  • The new invariant in tests/usage-hook/test_litellm_community_usage_hook.py only checks one hop, so a cycle of depth greater than 1 (A: [B] plus B: [A, qwen3.6]) would pass. No current config does this and there is no metering consequence, but walking the chain would close the whole class.
  • _SELF_RETRY_OK exempts whisper as a target in any position, not just in its own entry. glm5.3: ["whisper"] would skip that check, though the existing "canonicalised or free" invariant still catches it. Anchoring the exemption to the (primary, target) pair would make it explicit.

Related

  • helmcode/nan-devops#358 (the change that exposed this)
  • The Novita server overload 429s that pushed the traffic onto the fallback groups in the first place, currently open with the provider.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions