Skip to content

Add Stream Deck Plus dials, touch controls, and model presets - #13

Open
tlk3 wants to merge 153 commits into
dazer1234:mainfrom
tlk3:agent/stream-deck-plus-controls
Open

tlk3 wants to merge 153 commits into
dazer1234:mainfrom
tlk3:agent/stream-deck-plus-controls

Conversation

@tlk3

@tlk3 tlk3 commented Aug 12, 2026 •

Copy link
Copy Markdown

Summary

This expands Codex Deck into a first-class Stream Deck Plus integration. A new Encoder-only Codex Dial action lets each knob independently configure rotation, press, touch-strip tap, and LCD feedback while preserving all existing keypad behavior.

The default four-knob layout covers:

  • Model Presets — immediately cycle through user-defined model/reasoning pairs with continuous wrap and confirmed-state feedback
  • Agents — select occupied agents, show live status/context/host feedback, and focus on press
  • Actions — select and run configured Codex Micro actions
  • Usage — select Auto/5-hour/Weekly views, toggle the overview, and refresh from touch

What changed

  • Added the Stream Deck Plus Codex Dial manifest action, encoder layout, icons, and per-action property inspector.
  • Added per-knob presets plus fully independent rotation, press, touch, and feedback settings.
  • Added accessible live model/reasoning editors with add, remove, drag, keyboard reorder controls, unavailable-entry preservation, and an opt-in Ultra policy.
  • Added native paired model/reasoning switching through Codex's own application callback. The plugin does not type, synthesize keyboard navigation, or optimistically claim a selection before Codex confirms it.
  • Added immediate confirmed reasoning feedback, ULTRA OFF fail-closed behavior, and compact day/hour/minute usage-reset labels.
  • Added controller-wide ordering, bounded backpressure, generation/lifecycle fencing, and truthful SWITCHING, UNLISTED, UNAVAILABLE, degraded, and offline states.
  • Extended the authenticated Mac/Windows relay with additive capabilities for confirmed reasoning feedback and Model Presets. Older peers are refused before a partial command is sent.
  • Added non-destructive macOS bridge recovery: the watcher now degrades and retries without terminating or relaunching Codex, including when legacy Statsig is unavailable.
  • Added macOS launcher hardening, Stream Deck Plus documentation, release-audit coverage, and extensive domain/controller/bridge/relay/property-inspector tests.
  • Restored standalone command keycaps such as Terminal against the current Codex Micro runner layout while keeping dedicated reasoning mutations on their guarded path.

User impact

Stream Deck Plus owners can use the touch screen and four knobs as live Codex controls instead of mapping the device as keypad-only. The most common workflow—jumping between a few model/reasoning combinations—now takes one detent and updates the LCD from confirmed Codex state.

Existing Stream Deck keypad actions and single-host behavior remain unchanged. Stream Deck Plus support is optional and configured per knob.

Compatibility and safety

  • Uses allow-listed typed actions only; no arbitrary commands, URLs, keyboard focus movement, or global hotkeys.
  • Keeps the Chrome DevTools endpoint loopback-only.
  • Bounds and validates model catalogs, relay payloads, pending work, and renderer traversal.
  • Preserves independent Windows-only, macOS-only, and optional multi-host operation.
  • Does not include Codex/Elgato protected artwork, local profiles, logs, tokens, or generated release bundles.

Validation

  • npm ci
  • npm run check
  • npm test — 435 passed, 1 expected skip, 0 failed
  • npm run validate
  • npm run audit:release — 3 artifact roots passed
  • git diff --check origin/main...HEAD

Manual/live verification:

  • macOS 26.5.1
  • Codex for macOS 26.810.52044 (live renderer and native Micro activation verification)
  • Stream Deck 7.4.2 (22730)
  • physical Stream Deck Plus
  • live Codex macOS renderer integration, including persisted per-knob settings, touch actions, immediate dial feedback, Model Presets, restart persistence, and local bridge recovery

The cross-platform Stream Deck SDK, Windows launcher, and authenticated relay paths have automated coverage. This PR does not claim new physical Stream Deck Plus hardware verification on Windows.

tlk3 added 30 commits August 9, 2026 12:54
@tlk3

tlk3 commented Aug 16, 2026

Copy link
Copy Markdown
Author

Resolved a macOS recovery issue found during live testing on Codex 26.810.52044.

  • The background watcher can no longer terminate or relaunch Codex under any bridge-loss condition; it now remains degraded and retries in place.
  • Native Micro activation now works when the legacy Statsig client is unavailable, while preserving the older Statsig path and failing closed if native HID/joystick handlers are absent.
  • Added RED→GREEN regression coverage for previous-healthy bridge loss, process-generation replacement, no-Statsig activation, legacy activation, and missing-handler refusal.

Verification: 435 tests passed, 1 expected skip, TypeScript check, build, Stream Deck validation, release audit (3 roots), and diff check. The patched artifacts were also installed on macOS 26.5.1 with Codex 26.810.52044; live logs show local=ready and Codex Micro layout synchronized, and the installed watcher has no automatic restart path.

@tlk3

tlk3 commented Sep 3, 2026

Copy link
Copy Markdown
Author

Fixed Model Presets showing UNAVAILABLE on current Codex builds in 8c62d73. A live read-only probe traced the rejection to a non-enumerable, own-data Symbol.dispose cleanup function on the models/list response envelope. The validator now permits only that exact well-known cleanup symbol on the envelope, never invokes it, and continues rejecting accessors, proxies, unrelated/lookalike symbols, enumerable hooks, and symbols on model records. Original input cloning retains proxy rejection after descriptor validation. Regression reproduced the failure before the change; 75 bridge tests and the full suite (436 passed, 1 expected skip), TypeScript, build, plugin validation, and release audit pass. The patched serialized metadata reader was verified read-only against live Codex: gpt-5.6-sol / medium, nine catalog records. Saved presets and model selection were not modified. This commit is limited to the bridge and its regression test.

@tlk3

tlk3 commented Sep 5, 2026

Copy link
Copy Markdown
Author

Fixed current Codex model-preset detection in b2b6fff. The composer now displays GPT-prefixed names such as GPT-6 Astra, while catalog display names are normalized to 6 Astra. Normalize the optional GPT prefix on visible labels too, retaining exact catalog matching and ambiguity rejection. Regression reproduced before the fix; 436 tests pass (one expected skip), TypeScript, build, plugin validation, and release audit pass. A read-only probe against the running Codex app now resolves gpt-6-astra / low and all nine catalog models. No Codex restart was needed for verification.

@tlk3

tlk3 commented Sep 8, 2026

Copy link
Copy Markdown
Author

Added normal-launch macOS task-status recovery in bfa27e8.

After Codex opens normally or an update relaunches it without debug flags, the plugin can connect to the existing desktop IPC socket. It reads the six recent non-archived local tasks from the read-only catalog, subscribes to live status, and opens exact task IDs through Codex deep links. It does not modify or restart Codex, install another launcher, or create a router.

The adapter verifies socket ownership, bounds frames, projects only status metadata, and clears stale state on disconnect, owner loss, unsupported versions, missed revisions and read-state changes. Reconnection resubscribes. The renderer bridge remains preferred when available.

Scope limitation: this is task-status/navigation recovery, not full normal-launch control parity. Model presets, native actions and usage still need the renderer bridge. IPC snapshots omit composer authority, expose an unavailable action layout, and leave tasks without live snapshots marked unknown. Recent ordering is used in fallback mode. Details are in docs/DESKTOP_IPC.md.

Validation: full suite passed (440 passed, one expected skip), followed by seven additional independent IPC safety/relay tests, all passing. TypeScript, build, Stream Deck validation and release audit passed. A live read-only test forced only the renderer connection attempt to fail and recovered four live task statuses through IPC; two tasks without snapshots remained explicitly unknown. No CDP call or Codex restart was used for that test. The plugin build was installed locally with only a Stream Deck reload.

@tlk3

tlk3 commented Sep 10, 2026

Copy link
Copy Markdown
Author

Added normal-launch account Usage recovery in commit 10efad9.

  • Reads account rate limits through the bundled macOS Codex CLI (app-server --stdio), using only initialization and account/rateLimits/read. No debug flags, alternate launcher, desktop restart, or settings changes.
  • Background reads are coalesced/cached for 30 seconds and do not stall task status. Explicit Usage refresh forces a read. Helper output/time are bounded, errors are redacted, and only the spawned helper is terminated.
  • Uses the Codex account bucket without borrowing Spark windows or inventing missing limits. Reset-credit counts are display-only in this fallback; native reset applicability/consumption remains unavailable.
  • Usage-only failures clear usage without degrading connected task controls. A review-discovered disconnect race is covered by a regression: actual IPC loss still reports a transport failure.

Validation: 459 tests passed, one expected skip; TypeScript, build, Stream Deck package validation, release audit, and diff checks passed. Independent code review is clear. A live production-bridge probe returned account usage through desktop IPC on Codex 26.903.61454 without CDP. Installed the built plugin locally and reloaded only Stream Deck; its log reports local=ready and live task updates. Codex's process/start time remained unchanged.

Remaining limitation: this does not resolve normal-launch Model Preset or native Action controls. The inspected desktop IPC provides task owner/follower operations, not focused-composer state or a general desktop-command dispatcher. Task next-turn settings are not equivalent to the current composer's selection. Full support needs a desktop interface for focused composer state, validated model/effort updates, and native action dispatch. The renderer path remains preferred when available. Details are in docs/DESKTOP_IPC.md.

@tlk3

tlk3 commented Sep 10, 2026

Copy link
Copy Markdown
Author

Follow-up fc0b134 adds deduplicated, redacted Usage availability diagnostics and corrects stale macOS troubleshooting text that implied automatic recovery restarts. Actual installed Stream Deck process now logs account usage available (weekly); the standalone probe was not sufficient display verification. Missing five-hour windows still show unavailable. Model Preset/native Actions remain unsupported by the normal-launch fallback. 460 tests passed, one expected skip; check/build/package validation/release audit passed, with independent review clear. Reloaded Stream Deck only; Codex remained running.

@tlk3

tlk3 commented Sep 13, 2026

Copy link
Copy Markdown
Author

Fixed a fallback disconnect loop in c86701e. Codex 26.908.40834 emitted a 37,577,454-byte task snapshot, exceeding the plugin's previous 32 MiB receive limit. The plugin disconnected, cooled down for 30 seconds, retried the same snapshot, and degraded all controls repeatedly.

Incoming frames now permit up to 64 MiB; outgoing messages retain the 32 MiB limit. Invalid/oversized frames still disconnect with cooldown, and diagnostics report only numeric frame sizes rather than task content. The new regression delivers a fragmented 36 MiB payload followed by another valid message. This raises the bounded compatibility limit; it is not an unlimited streaming parser.

Validation: 461 tests passed, one expected skip; TypeScript, build, Stream Deck validation, release audit, and independent QA passed. Live checks against the running updated Codex processed the formerly rejected snapshot and continued receiving activity and weekly usage. Installed the verified plugin and reloaded Stream Deck only.

Separate unresolved issue: the Codex update also relaunched without renderer debug flags. This patch restores the task/usage fallback; Model Preset and native Action controls still require the renderer connection and a separately authorized bridge-enabled restart.

@tlk3

tlk3 commented Sep 13, 2026

Copy link
Copy Markdown
Author

Replaced buffered desktop-status decoding with incremental metadata projection in 3a943f4. The prior limit increase was insufficient: live task snapshots grew from ~36 MiB to ~72 MiB, again disconnecting the entire task/Usage fallback.

The live IPC path now streams JSON with socket backpressure and retains only bounded task-status metadata. Task history, tool output, and request bodies are skipped without assembling whole snapshots. Frame length no longer determines a buffer allocation. UTF-8 and split/coalesced frames, revision-checked patches, useful status/request updates, cancellation, and reconnect behavior are preserved. Unsupported patch values invalidate the affected task instead of inventing status. Frame deadlines and metadata/depth limits remain enforced; errors never include task content.

Validation: 472 tests passed, one expected skip; TypeScript, build, Stream Deck package validation, release audit, and independent agent QA passed. Streaming 70/80 MiB fixtures pass bounded-memory tests, including on Stream Deck's Node 20.20.0 runtime. Independent real-socket QA also passed an 80 MiB snapshot followed by reset/reconnect. Live probes against the previously failing running Codex continued receiving task status and weekly usage, with sampled heap around 9–14 MiB. Installed the matching bundle and reloaded Stream Deck only.

Dependency note: pinned stream-json 1.9.1 for Node 20 compatibility. The bundle source map confirms only Parser.js and Utf8Stream.js are included. npm reports a moderate advisory for its filter modules, which this integration neither imports nor bundles; the tokenizer also has an independent depth cap. Three existing high findings belong to the Elgato development CLI dependency tree. These audit findings are disclosed rather than described as a clean npm audit.

This resolves the snapshot-size disconnect mechanism. It does not restore the renderer connection removed by a Codex update: Model Preset and native Actions still need the previously documented bridge-enabled launch.

@tlk3

tlk3 commented Sep 17, 2026

Copy link
Copy Markdown
Author

Resolved another normal-launch degradation path in 7ecc6c3.

The Stream Deck logs showed recurring /usr/bin/sqlite3 -readonly failures with unable to open database file (14). A single transient catalog read was marking the whole local host degraded even though the desktop IPC stream, task status, navigation, and usage remained healthy.

The IPC fallback now retains the last validated six-task catalog only for that exact SQLite open-error condition, continues applying live task-status updates, and replaces the cache immediately after the next successful read. Other execution failures, malformed JSON, and invalid/unsupported catalog data still fail closed. No task messages are persisted or logged, and this does not invent model/composer/action authority.

Verification: TypeScript check; 475 tests passed with 1 expected skip; build; Stream Deck validation; release audit; independent code review; installed bundle checksum match; live Stream Deck-only reload connected to desktop IPC with local=ready, current task status, and weekly usage. Codex was not restarted.

@tlk3

tlk3 commented Sep 18, 2026

Copy link
Copy Markdown
Author

Resolved the macOS integrated-terminal restart failure in 0930763.

Root cause: start --restart waited inside the calling terminal while terminating Codex. When invoked from Codex's integrated terminal, that terminal and the launcher died before the bridge-enabled relaunch could occur, leaving Model Presets and Actions unavailable after what looked like a restart.

The explicit restart path now:

  • hands off to a detached helper before Codex exits;
  • pins the authorized process generation and exact app/executable paths;
  • revalidates immediately before termination and again before launch;
  • serializes concurrent restart requests; and
  • persists accepted/running/completed/rejected/failed outcomes under Application Support.

Regression coverage includes a real detached subprocess surviving its parent terminal, stale-generation/install rejection, concurrency serialization, and persisted outcomes. Fresh verification: 480 tests passed, 1 skipped; TypeScript check, build, Stream Deck validation, and release audit all passed. Independent review found no remaining P1/P2 issues.

The updated launcher runtime is installed locally without restarting the current Codex process. One explicit bridge-enabled restart is still required to verify the live Stream Deck recovery on Codex 26.915.31945 (9922).

@tlk3

tlk3 commented Sep 20, 2026

Copy link
Copy Markdown
Author

Resolved the recurring post-reboot/post-update degraded state in 3057ce1.

Root causes:

  • The macOS LaunchAgent could stall in shell startup or be delayed by Background process scheduling, so the watcher never reached Node after login.
  • Desktop IPC re-ran process verification on every refresh, allowing a transient ps failure to mark otherwise healthy controls degraded.
  • A normal Codex launch could remain without the renderer bridge indefinitely after the watcher came online.

Fixes:

  • Launch the installed watcher directly with a stable Node executable and no shell or Background deferral.
  • Verify the Codex process once per IPC connection/reconnection instead of once per refresh.
  • Add a bounded one-time startup recovery after the watcher has observed Codex closed, with exact process/generation checks, a stable-start window, one attempt per generation, cooldown, and repeated deadline checks before termination and relaunch. The installer deliberately leaves an already-running normal Codex session untouched.

Verification:

  • 487 tests passed, 1 expected skip
  • build and TypeScript check passed
  • Stream Deck validation passed
  • three-root release audit passed
  • macOS launcher self-test passed
  • independent review found no P0-P2 issues
  • installed LaunchAgent is running direct Node, the installed watcher and plugin hashes match the reviewed build, and the existing Codex PID remained unchanged during installation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant