[eas-cli] Stream workflow run logs in real time - #4228
Conversation
|
Subscribed to pull request
Generated by CodeMention Warning: The preamble and epilogue options in commentConfiguration are deprecated. Use template instead. |
Log files are still polled and remain authoritative. On top of that, each in-progress job now subscribes to its EAS Logs Centrifugo channel, so the tail shown by `eas workflow:run --wait` and `eas workflow:status --wait` updates as lines are produced rather than once per poll. Published lines are buffered separately and stay hidden until a logId appears in both sources, so the displayed log never shows a gap. Any failure to connect or subscribe falls back to polling alone. This mirrors how the website combines the two sources.
3261c8f to
40d7114
Compare
There was a problem hiding this comment.
Pull request overview
This PR updates eas-cli’s workflow run log UX to stream logs in near real time via Centrifugo/WebSocket, while keeping the existing polling-based log file fetching as a fallback/source of truth. It introduces a persistent per-job log state that merges file and realtime publications, and refactors log parsing so it can be reused by both sources.
Changes:
- Add a Centrifugo-based realtime logs client plus GraphQL mutations to obtain connection/subscription tokens.
- Refactor workflow log fetching/parsing into dedicated modules and introduce a
WorkflowRunLogsWatcherto persist/merge log state across polling iterations. - Add unit tests covering log parsing, watcher behavior, and
uniqBycall behavior.
Reviewed changes
Copilot reviewed 19 out of 20 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| yarn.lock | Locks new deps needed for realtime logs (Centrifugo, ws, protobufjs transitive deps). |
| packages/eas-cli/package.json | Adds centrifuge, ws, and @types/ws dependencies. |
| packages/eas-cli/src/api.ts | Adds getEASLogsWebsocketUrl() for staging/local/prod websocket endpoints. |
| packages/eas-cli/src/utils/expodash/uniqBy.ts | Improves typing (Set<K>) and avoids double getKey() evaluation. |
| packages/eas-cli/src/utils/expodash/tests/uniqBy-test.ts | Adds test ensuring getKey is called once per item. |
| packages/eas-cli/src/graphql/mutations/RealtimeLogsMutation.ts | Adds mutations for Centrifugo connection/subscription tokens. |
| packages/eas-cli/src/graphql/generated.ts | Adds generated GraphQL types for the new realtime logs mutations. |
| packages/eas-cli/src/utils/centrifuge.ts | Implements a Centrifugo + ws client wrapper with proxy support and subscription lifecycle. |
| packages/eas-cli/src/utils/tests/centrifuge-test.ts | Adds basic tests for realtime logs client construction behavior. |
| packages/eas-cli/src/commandUtils/workflow/types.ts | Introduces WorkflowRawLogLine and adjusts log line typing for parsing/streaming. |
| packages/eas-cli/src/commandUtils/workflow/logs/fetchLogs.ts | Moves/cleans up raw log fetching used by parsing/watcher. |
| packages/eas-cli/src/commandUtils/workflow/logs/parseLogs.ts | New reusable parsing + merging + grouping utilities for workflow logs. |
| packages/eas-cli/src/commandUtils/workflow/logs/watcher.ts | Adds persistent watcher/state that merges file logs with realtime publications per job. |
| packages/eas-cli/src/commandUtils/workflow/logs/tests/parseLogs-test.ts | Adds tests for JSONL parsing, merging, and step grouping. |
| packages/eas-cli/src/commandUtils/workflow/logs/tests/watcher-test.ts | Adds tests for watcher subscription/fetch behavior and log merge gating logic. |
| packages/eas-cli/src/commandUtils/workflow/utils.ts | Refactors status rendering to reuse parsing + watcher state and support realtime updates. |
| packages/eas-cli/src/commandUtils/workflow/stateMachine.ts | Updates imports to use the refactored log parsing function location. |
| packages/eas-cli/src/commandUtils/workflow/tests/utils-test.ts | Updates tests to cover new formatting/parsing structure for active workflow runs. |
| packages/eas-cli/src/tests/commands/workflow-logs-test.ts | Updates imports/mocks due to fetchLogs module relocation. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #4228 +/- ##
==========================================
+ Coverage 63.53% 63.77% +0.24%
==========================================
Files 1028 1034 +6
Lines 47033 47649 +616
Branches 9884 10003 +119
==========================================
+ Hits 29879 30384 +505
- Misses 17053 17164 +111
Partials 101 101 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
✅ Thank you for adding the changelog entry! |
Why
We can increase the responsiveness of workflow logs by using the new centrifugo realtime logs, received over websocket, instead of polling every 10 seconds.
How
Added
centrifugeandwsto deps.wsadded for two reasons: node 20 doesn't have the native client and the native client doesn't support proxiesAdded
centrifuge.ts, which wraps the centrifuge client + subscriptionsAdded mutations to obtain connection and subscription tokens for the realtime logs
Refactored the workflow log polling loop to persist state between polls.
Split up log fetching and parsing to reuse parsing in the realtime logs. The state consists of the workflow run state and a
WorkflowRunLogsWatcher, which updates the logs of each workflow job. The logs are merged from 2 sources: from the old polling endpoint and the new centrifuge subscriberTest Plan
workflow:runagainst staging with a workflow that produces a log every second and verified that the logs appear 1 by 1 instead of batches of 10