Skip to content

Fix Fast mode provider retries and reporting - #1466

Draft
mrubens wants to merge 3 commits into
developfrom
codex/fast-mode-provider-retries
Draft

Fix Fast mode provider retries and reporting#1466
mrubens wants to merge 3 commits into
developfrom
codex/fast-mode-provider-retries

Conversation

@mrubens

@mrubens mrubens commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

What changed

  • subscribe non-task OpenCode prompts to live provider retry and session-error events
  • report Fast Mode provider retries visibly instead of appearing stalled
  • retry transient Fast Mode inference failures up to three times with bounded exponential backoff
  • use longer 5/10/20-second backoff for rate limits and shorter 1/2/4-second backoff for other transient provider failures
  • abort failed OpenCode prompts before retrying so a reused conversation session (or leased server) is not still running the old turn
  • replace the generic failure reply with provider-specific terminal reporting for rate limits, timeouts, connectivity, credentials, quota, and unavailable models

Why

Fast Mode shared the deployment's inference provider with standard tasks but did not share their provider recovery behavior. OpenCode could retry a 429 internally while the control-plane request appeared hung, then Roomote would hit its 120-second timeout and post a generic apology. Standard tasks remained alive and recovered, making Fast Mode appear uniquely broken.

This gives Fast Mode the same essential behavior: observe live retry state, tell the user what is happening, apply bounded recovery, and preserve a useful terminal cause when recovery is exhausted.

Update (Aug 19)

Rebased and re-ported onto the native OpenCode tools/sessions architecture from #1460, which had rewritten the Fast agent turn loop and conflicted with the original branch:

  • the retry loop now wraps the native-session prompt inside fastAgentOpenCodeSessionManager.run's execute, and NonTaskOpenCodeSessionNotFoundError still propagates so the manager can bootstrap a fresh session with its own retry budget
  • retry notices post through the adapter directly so a system-posted notice never satisfies the model's acknowledgement gate, marks the turn closed, or becomes the recorded answer
  • the event subscription and session-abort behavior live in runNonTaskSdkPrompt, which now serves both ephemeral and persistent (Fast conversation) sessions

Validation

  • pnpm --filter @roomote/cloud-agents exec vitest run (781 tests, includes new retry/backoff/notice/terminal-classification coverage)
  • pnpm lint, pnpm check-types, pnpm knip

@roomote-community

roomote-community Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

2 issues outstanding. See task

  • packages/cloud-agents/src/server/non-task-provider-usage.ts:1105 Do not retry non-transient provider 4xx errors.
  • packages/cloud-agents/src/server/non-task-provider-usage.ts:1107 Classify HTTP 408 responses as timeouts for Fast Mode reporting.
  • packages/cloud-agents/src/server/fast-agent/fast-agent-service.ts:945 Invalidate a failed live session before queued turns can reuse it.

Reviewed 3bd0826

Comment thread packages/cloud-agents/src/server/fast-agent/fast-agent-service.ts
@mrubens mrubens closed this Aug 19, 2026
@mrubens mrubens reopened this Aug 19, 2026
Ported onto the native OpenCode session architecture (#1460): the retry
loop now wraps generateTrackedNonTaskTextInOpenCodeSession inside the
session manager's execute, session-not-found still propagates so the
manager can bootstrap a fresh session, and retry notices post directly
through the adapter so they never satisfy the model's acknowledgement
gate or close the turn.

- subscribe non-task OpenCode prompts to live provider retry and
  session-error events
- report Fast Mode provider retries visibly instead of appearing stalled
- retry transient Fast Mode inference failures up to three times with
  bounded exponential backoff (5/10/20s for rate limits, 1/2/4s for
  other transient provider failures)
- abort failed OpenCode prompts before retrying so a reused session or
  leased server is not still running the old turn
- replace the generic failure reply with provider-specific terminal
  reporting for rate limits, timeouts, connectivity, credentials,
  quota, and unavailable models
@mrubens
mrubens force-pushed the codex/fast-mode-provider-retries branch from 3ac6fa8 to d2c893c Compare August 19, 2026 18:34
A structured 400/413/422 client error cannot recover by resending the
same request, so the Fast retry loop (and validation classification)
now reports it terminally instead of retrying three times. 408 stays
retryable as a timeout; 401/402/403/404/429 keep their specific
classifications; 5xx stays retryable.
statusCode !== undefined &&
statusCode >= 400 &&
statusCode < 500 &&
statusCode !== 408

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HTTP 408 is excluded from the non-retryable 4xx branch but is never mapped to reason: 'timeout'. A structured { statusCode: 408 } whose provider text does not contain timeout falls through as provider_error, so Fast Mode uses the generic retry and terminal messages instead of its timeout-specific reporting. Classify 408 by status and add a status-only regression case.

@daniel-lxs

Copy link
Copy Markdown
Member

I think we should make one side clearly own retries before merging this.

This PR adds a new Roomote retry loop around session.prompt, but OpenCode already retries provider failures internally. For a normal retryable 429, OpenCode keeps the request pending while it retries, so Roomote never receives the 429 needed to enter the new 5/10/20s path. Roomote eventually hits its 120s timeout, classifies that as a timeout, and resubmits after 1/2/4s instead.

That means the new rate-limit backoff generally isn't controlling the case it was added for, and "three retries" can become four separate 120-second OpenCode retry windows.

The resubmission also uses the same persistent session. OpenCode saves the user message before inference starts, and aborting doesn't remove it, so each outer retry can add another copy of the same user turn to the conversation.

I think we should either let OpenCode own provider retries while Roomote observes them and applies one total deadline, or have Roomote own retries and disable/bound OpenCode's retrying. If Roomote resubmits, it also needs to revert the failed turn or use a clean session.

There are also named terminal errors, including context overflow, content filtering, output length, and some auth failures, that currently fall through to retryable: true, so I think those need explicit classification too.

// The system-posted closeout below is mirrored to compatibility history,
// not to OpenCode's live transcript. Force the next turn to bootstrap so
// the model can see the failure the user saw in Slack or Discord.
fastAgentOpenCodeSessionManager.invalidate(canonicalConversationId);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

invalidate runs only after fastAgentOpenCodeSessionManager.run() has rejected. That manager releases entry.tail in its finally before this catch executes, so a queued turn for the same conversation can wake up, see the stale session.id, and send a delta to the failed transcript before it is cleared. Invalidate the session before releasing queued work (or make the manager own that failure transition) so queued turns bootstrap from compatibility history.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants