Skip to content

Keep Codex threads alive with shared 429 backoff #507

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
alicon wants to merge 6 commits into from
Closed

Keep Codex threads alive with shared 429 backoff #507

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
c94c76f
feat: add shared Codex FMAPI rate limiter
alicon Sep 5, 2026
aeae89c
fix Codex FMAPI rate limits and model replay
alicon Sep 5, 2026
9c6f00a
add shared Qwen FMAPI throttling
alicon Sep 5, 2026
493564c
fix Codex nested config serialization
alicon Sep 5, 2026
6ad481e
fix Codex 429s with shared backoff
alicon Sep 5, 2026
33119e5
fix Codex managed config bypass
alicon Sep 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 54 additions & 6 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,53 @@ passed again. Smart routing uses the `task_v1` router by default. Power users ca
router for a launch by setting `SMART_ROUTER_NAME`, for example
`SMART_ROUTER_NAME=task_v2 ug codex --enable-smart-routing`.

### Codex shared rate limiting

Codex requests launched by `ug codex` share a local rolling 60-second input-token budget. This
prevents several concurrent Codex sessions from independently exhausting the same Databricks
Foundation Model API quota. Budgets are isolated by workspace and model, so traffic for one model
does not consume another model's allowance.

| Codex model | Published input tokens/minute | Local target |
|---|---:|---:|
| GPT-6 Astra | 200,000 | 180,000 |
| GPT-5.6 Sol | 2,000,000 | 1,800,000 |
| GPT-5.6 Terra | 2,000,000 | 1,800,000 |
| GPT-5.6 Luna | 2,000,000 | 1,800,000 |
| Kimi K3 | 200,000 | 180,000 |
| Qwen3.5 122B A10B | 1,000,000 | 900,000 |
| Qwen3-Next 80B A3B Instruct | 1,000,000 | 900,000 |

The 90% target leaves headroom for estimation error and requests made outside Unity Gateway.
Unity Gateway estimates input conservatively from the uncompressed JSON request size and records
only timestamps, model keys, and estimates in `~/.ucode/codex-rate-limit-state.json`; prompts and
credentials are never stored. When capacity is unavailable, the request waits and one short notice
is written to stderr. Unknown/new models pass through the proactive token-budget check until their
published quota is added.

Every upstream `429 Too Many Requests` response is also handled inside Unity Gateway instead of
consuming Codex's finite retry budget. Unity Gateway honors `Retry-After` when present; otherwise it
uses capped exponential backoff with jitter. The resulting cooldown is shared across local Codex
processes for the same workspace and model. If the request body is unreadable or the model is new,
the cooldown safely applies to the whole workspace. The original model and reasoning effort never
change, and the thread keeps retrying until capacity returns or the user cancels it.

The proxy address is different for each launch, so Codex's provider must remain in the ucode profile
and launch-time overrides. During configuration, current versions remove provider values that older
ucode versions wrote to `/etc/codex/managed_config.toml`; Codex gives that file higher precedence
and would otherwise silently bypass the limiter.

When a Codex thread switches from a model that returns visible reasoning, such as Kimi K3, to an
OpenAI reasoning model, Unity Gateway removes the nonportable `reasoning.content` field from the
outbound replay request. The saved Codex transcript is not changed. OpenAI's replayable
`encrypted_content` and all messages and tool calls remain intact.

The limiter applies to normal, app, and smart-routed Codex launches. Caller-managed server commands
such as `codex app-server` and `codex mcp-server` keep their existing launch path. To bypass the
limiter for one launch, use `UCODE_CODEX_RATE_LIMITER=0 ug codex`; that restores direct requests and
may expose the session to 429 responses. See the current
[Databricks Foundation Model API limits](https://docs.databricks.com/aws/en/machine-learning/foundation-model-apis/limits).

To configure all tools at once:

```bash
Expand Down Expand Up @@ -263,11 +310,12 @@ ug publish # publish it to the workspace

`ug setup` walks through the agents to enable and which one bare `ug` launches, then per agent:
Databricks-hosted models or an external Model Provider Service and the models to expose. Interactive
Claude Code and Codex configuration installs gateway-critical values in the OS-managed settings
scope so enterprise settings cannot silently override Unity Gateway. Non-interactive and CI runs use local
files without invoking `sudo`, and stop with an actionable error if an existing managed value
conflicts. Claude subscription relay is local-only because its loopback proxy exists only for that
session.
Claude Code configuration installs gateway-critical values in the OS-managed settings scope so
enterprise settings cannot silently override Unity Gateway. Codex keeps its provider launch-scoped
because its shared limiter uses a per-session loopback proxy. Codex configuration retires provider
values written there by older ucode versions and stops if unrelated machine policy still overrides
the launch-scoped provider. Non-interactive and CI runs never invoke `sudo`. Claude subscription
relay is also local-only because its loopback proxy exists only for that session.
Claude Code is asked one model per family (opus/sonnet/haiku/fable), since it selects models by family
alias; any family can be skipped.

Expand Down Expand Up @@ -407,7 +455,7 @@ control the installation.
| `~/.codex/ucode.config.toml` (or legacy `~/.codex/config.toml`) | Codex |
| `~/.claude/ucode-settings.json` | Claude Code settings generated by ug |
| `/etc/claude-code/managed-settings.json` (Linux) or `/Library/Application Support/ClaudeCode/managed-settings.json` (macOS) | Claude Code OS-managed settings |
| `/etc/codex/managed_config.toml` | Codex OS-managed settings |
| `/etc/codex/managed_config.toml` | Codex OS-managed settings inspected for provider conflicts; older ucode-owned values are retired |
| `~/.gemini/.env` | Gemini CLI |
| `~/.config/opencode/opencode.json` | OpenCode |
| `~/.copilot/.env` | GitHub Copilot CLI |
Expand Down
83 changes: 55 additions & 28 deletions docs/os-managed-settings-design.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,20 @@

## Summary

Claude Code and Codex give OS-managed settings higher precedence than user settings. Previously,
ucode could write only its local configuration while an existing machine-managed file silently
overrode the gateway endpoint, authentication helper, provider headers, or model.
Claude Code and Codex give OS-managed settings higher precedence than user settings. That is useful
for Claude Code's stable gateway configuration, but a Codex provider at that scope overrides the
per-launch loopback address used for shared throttling and 429 retries.

The two stacked PRs make precedence handling deterministic:
The shared managed-file lifecycle makes precedence handling deterministic:

1. The Claude PR adds the shared managed-file lifecycle and applies it to Claude Code.
2. The Codex PR reuses that lifecycle for TOML, applies it to Codex, and removes the old optional
managed-settings path.
2. Codex reuses that lifecycle to retire provider settings older ucode versions installed and to
preserve unrelated machine policy.

After both PRs merge, interactive configuration reconciles the agent's OS-managed file by default.
Non-interactive and CI execution never elevates privileges and instead uses local settings when the
managed file is compatible.
Interactive Claude configuration reconciles its OS-managed file by default. Codex configuration
keeps its provider in `~/.codex/ucode.config.toml` plus launch-time overrides. It restores the
recorded pre-ucode managed baseline, then verifies that remaining machine policy cannot bypass that
provider. Non-interactive and CI execution never elevates privileges.

## Configuration Files

Expand All @@ -23,8 +24,9 @@ managed file is compatible.
| Claude Code | `~/.claude/ucode-settings.json` | Linux: `/etc/claude-code/managed-settings.json`; macOS: `/Library/Application Support/ClaudeCode/managed-settings.json` |
| Codex | `~/.codex/ucode.config.toml` | `/etc/codex/managed_config.toml` |

The local file is always written. The OS-managed file is additionally reconciled during interactive
configuration, except for Claude subscription relay.
The local file is always written. Claude's OS-managed file is additionally reconciled during
interactive configuration, except for subscription relay. Codex's OS-managed file is inspected and
fingerprinted but is not populated by current ucode versions.

## Interactive Detection

Expand All @@ -38,24 +40,23 @@ command shape in some flows and did not guard managed-file writes consistently.

## Behavior Matrix

| Invocation | Managed file | Behavior |
| Agent and invocation | Managed file | Behavior |
| --- | --- | --- |
| Interactive | Absent | Create it from the ucode configuration after recording an absent baseline. |
| Interactive | Unrelated or partially populated | Preserve unrelated values and add or update all ucode-owned values. |
| Interactive | Conflicting | Back up the baseline, replace the conflicting ucode-owned values, and verify. |
| Interactive | Already identical | Continue without a backup, write, or `sudo` invocation. |
| Non-interactive | Absent | Use the local ucode file. Do not create the managed file. |
| Non-interactive | Ucode-owned values absent or equal | Use the local ucode file. Do not modify the managed file. |
| Non-interactive | Ucode-owned value conflicts | Stop before launching because the higher-precedence value would override ucode. |
| Claude, interactive | Absent or compatible | Record the baseline, add the stable gateway values, and verify. |
| Claude, non-interactive | Absent or compatible | Use the local ucode file without invoking `sudo`. |
| Codex, interactive | Contains tracked ucode provider values | Restore the pre-ucode baseline, preserving later external changes. |
| Codex, non-interactive | Contains tracked ucode provider values | Stop and require one interactive migration; never invoke `sudo`. |
| Codex, any | Absent or unrelated | Use the local profile and launch-scoped provider; record a compatibility fingerprint. |
| Codex, any | Selects another provider or overrides `ucode-databricks.base_url` | Stop because the managed value would bypass the launch-scoped proxy. |
| Any | Invalid, unreadable, or symlinked | Stop without modifying the file because precedence cannot be established safely. |

`ucode configure`, first-time `ucode claude` or `ucode codex`, and later launches all use the same
agent-specific reconciliation path. A first-time launch from an interactive terminal can therefore
request administrator permission. A first-time non-interactive launch remains local-only.
`ucode configure`, first-time agent launches, and later launches all use the same agent-specific
compatibility path. A Codex launch requests administrator permission only when it must retire a
tracked provider installed by an older ucode version.

## Interactive Reconciliation
## Claude Interactive Reconciliation

For each agent, ucode:
For Claude Code, ucode:

1. Strictly parses the existing managed JSON or TOML document.
2. Produces the desired document by applying the same gateway overlay used for the local ucode file.
Expand All @@ -66,9 +67,28 @@ For each agent, ucode:
7. Reads the installed file back and verifies its exact contents.
8. Records the last-applied snapshot, owned paths, and a launch fingerprint.

An existing managed file is reconciled even when it does not currently conflict. This ensures every
ucode-required value exists at the highest-precedence scope and avoids separate behavior for absent,
partial, and conflicting files.
An existing Claude managed file is reconciled even when it does not currently conflict. This
ensures every stable gateway value exists at the highest-precedence scope.

## Codex Launch-Scoped Provider

Every normal `ug codex` launch starts a loopback proxy on an ephemeral port. The launch overlay
replaces only `model_providers.ucode-databricks.base_url` with that address and disables request
compression so the proxy can identify the model and estimate input tokens.

Codex's system `managed_config.toml` has higher precedence than both the named ucode profile and
launch overrides. Persisting the direct Databricks endpoint there therefore bypasses the proxy even
when the child command visibly receives a loopback URL. Configuration now:

1. Uses the managed-backup manifest to restore the exact pre-ucode baseline or perform a three-way
revert when external policy changed later.
2. Preserves unrelated managed keys, including approval and model defaults.
3. Rejects a remaining external `model_provider` selection or managed
`model_providers.ucode-databricks.base_url`.
4. Fingerprints the compatible result so unchanged launches need only one `stat()` call.

This means a bare `codex` command is not routed through Unity Gateway by current ucode versions. Use
`ug codex` when the shared limiter, gateway authentication, and automatic 429 recovery are required.

## Privileged Write Transaction

Expand Down Expand Up @@ -215,7 +235,7 @@ administrator help.
- Add Claude managed status and revert output.
- Remove Claude's old managed-settings scope choice.

### PR 2: Codex
### PR 2: Codex (historical behavior)

- Stack on the Claude PR and reuse the shared lifecycle with strict TOML parsing and serialization.
- Make interactive Codex configuration reconcile OS-managed TOML by default.
Expand All @@ -224,3 +244,10 @@ administrator help.
- Keep opt-in smart-routing hooks in the local Codex config rather than adding them by default to
machine-managed policy.
- Remove the remaining managed-settings scope schema, resolution, setup prompt, summary, and tests.

### Codex launch-proxy amendment

- Retire the direct managed provider using the existing baseline and three-way restore lifecycle.
- Keep the provider in the named profile and launch overrides so every request reaches the
per-launch proxy.
- Preserve unrelated system policy and block only values that would bypass the proxy.
Loading