fix: Keep a failing background watcher from breaking a successful actor call - #1027
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1027 +/- ##
==========================================
+ Coverage 95.24% 95.25% +0.01%
==========================================
Files 59 59
Lines 5514 5547 +33
==========================================
+ Hits 5252 5284 +32
- Misses 262 263 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
vdusek
force-pushed
the
fix/background-watchers-break-successful-call
branch
from
September 16, 2026 13:01
76a93b0 to
2020ac1
Compare
vdusek
marked this pull request as ready for review
September 16, 2026 13:14
Pijukatel
approved these changes
Sep 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
actor.call()with the defaultlogger='default'could raise on a run that had already succeeded. A status poll that failed after retries left the exception on the watcher task;stop()then cancelled an already-finished task andawaited it, re-raising pastexcept asyncio.CancelledError. The sync twin leaked the same failure out of its thread.StreamedLog._stream_logalready had: warn onis_timeout_error,logger.exceptionotherwise. A failed poll gets reported and the run's own result stands.StreamedLog.stop()closes the stream response and caps its join at_stop_timeout_s(5s), where it used to join unbounded on a stream that may not speak for hours.start()on both sync classes checksis_alive(), so a handle left by a thread that outlived its stop does not block a restart.The close alone is not enough. On Impit 0.13.2
Response.close()returns immediately and setsis_closed=True, yet a reader blocked initer_bytes()stays blocked, and closing theClientdoes not release it either. The close is kept because it is correct for a custom HTTP client, and the capped join is what makesstop()return.Why
stop()is bounded this waystop()by requesting the log stream with a 30s read timeout.timeoutto the whole request, streamed body included, so a run logging past the bound was cut off mid-stream withimpit.TimeoutException(impit.TimeoutException in _streamed_log.py after successful actor completion #945). The stream has usedno_timeoutsince, with the unboundedstop()recorded as a known limitation._stream_timeoutstaysno_timeout, still asserted bytest_streamed_log_sync_requests_stream_with_no_timeout, so impit.TimeoutException in _streamed_log.py after successful actor completion #945 cannot regress. What it fixes is the case fix: prevent Actor log-streaming thread from crashing on stream timeout #944 left open: a manualstop()on a stream gone quiet.Trade-off: a thread outliving the 5s cap keeps running as a daemon, so
stop()can return before the tail is flushed. On theactor.call()path the run's EOF ends the thread first, measured at 0.000s over repeatedapify/hello-worldruns.Test plan
✍️ Drafted by Claude Code