Skip to content

fix: Keep a failing background watcher from breaking a successful actor call - #1027

Merged
vdusek merged 1 commit into
masterfrom
fix/background-watchers-break-successful-call
Sep 16, 2026
Merged

vdusek merged 1 commit into
masterfrom
fix/background-watchers-break-successful-call

Conversation

@vdusek

@vdusek vdusek commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Description

actor.call() with the default logger='default' could raise on a run that had already succeeded. A status poll that failed after retries left the exception on the watcher task; stop() then cancelled an already-finished task and awaited it, re-raising past except asyncio.CancelledError. The sync twin leaked the same failure out of its thread.

  • Both watcher loops now use the handler StreamedLog._stream_log already had: warn on is_timeout_error, logger.exception otherwise. A failed poll gets reported and the run's own result stands.
  • The sync watcher's thread is a daemon, so a watcher whose loop never terminates cannot hold up interpreter shutdown.
  • StreamedLog.stop() closes the stream response and caps its join at _stop_timeout_s (5s), where it used to join unbounded on a stream that may not speak for hours.
  • start() on both sync classes checks is_alive(), so a handle left by a thread that outlived its stop does not block a restart.

The close alone is not enough. On Impit 0.13.2 Response.close() returns immediately and sets is_closed=True, yet a reader blocked in iter_bytes() stays blocked, and closing the Client does not release it either. The close is kept because it is correct for a custom HTTP client, and the capped join is what makes stop() return.

Why stop() is bounded this way

Trade-off: a thread outliving the 5s cap keeps running as a daemon, so stop() can return before the tail is flushed. On the actor.call() path the run's EOF ends the thread first, measured at 0.000s over repeated apify/hello-world runs.

Test plan

  • 8 unit tests and an integration test that fails a real run's background status polls, each red before the fix.

✍️ Drafted by Claude Code

@vdusek vdusek added adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. labels Aug 21, 2026
@vdusek vdusek self-assigned this Aug 21, 2026
@github-actions github-actions Bot added this to the 147th sprint - Tooling team milestone Aug 21, 2026
@github-actions github-actions Bot added the tested Temporary label used only programatically for some analytics. label Aug 21, 2026
@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.15385% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.25%. Comparing base (10ace92) to head (2020ac1).

Files with missing lines Patch % Lines
src/apify_client/_status_message_watcher.py 91.30% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1027      +/-   ##
==========================================
+ Coverage   95.24%   95.25%   +0.01%     
==========================================
  Files          59       59              
  Lines        5514     5547      +33     
==========================================
+ Hits         5252     5284      +32     
- Misses        262      263       +1     
Flag Coverage Δ
integration 92.06% <67.30%> (-0.14%) ⬇️
unit 87.41% <96.15%> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vdusek
vdusek force-pushed the fix/background-watchers-break-successful-call branch from 76a93b0 to 2020ac1 Compare September 16, 2026 13:01
@vdusek
vdusek requested a review from Pijukatel September 16, 2026 13:14
@vdusek
vdusek marked this pull request as ready for review September 16, 2026 13:14
@vdusek
vdusek merged commit efc009d into master Sep 16, 2026
33 of 34 checks passed
@vdusek
vdusek deleted the fix/background-watchers-break-successful-call branch September 16, 2026 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants