_cas: avoid blocking calls to long grpc methods - #2160
Merged
Conversation
juergbi
reviewed
Aug 3, 2026
juergbi
reviewed
Aug 3, 2026
abderrahim
force-pushed
the
abderrahim/pull-push-hang
branch
3 times, most recently
from
August 4, 2026 15:13
c5f3504 to
fbe5aeb
Compare
abderrahim
changed the title
abderrahim
force-pushed
the
abderrahim/pull-push-hang
branch
from
August 4, 2026 15:15
fbe5aeb to
b1b8fa0
Compare
abderrahim
marked this pull request as ready for review
August 4, 2026 15:15
abderrahim
requested review from
BenjaminSchubert,
cs-shadow and
gtristan
as code owners
August 4, 2026 15:15
abderrahim
force-pushed
the
abderrahim/pull-push-hang
branch
from
August 5, 2026 10:30
b1b8fa0 to
50f97b8
Compare
Long-running synchronous grpc calls block the thread where they are
called, which can cause the scheduler to be unable to terminate the
job.
This commit replaces them with usage of the "asynchronous" future
API. The result() method of a grpc future will not block the thread
while it's waiting, and thus allows terminating the job without
waiting for the blocking call to return. This covers the LocalCAS
methods {Fetch,Upload}{Tree,MissingBlobs}, and the CAS method
FindMissingBlobs.
As an added benefit, we try to cancel the grpc calls when
terminated. This depends on the server side (buildbox-casd) promptly
cancelling the remote calls, which it doesn't consistently do at the
moment. But that can be fixed independently.
Fixes #2157
abderrahim
force-pushed
the
abderrahim/pull-push-hang
branch
from
August 6, 2026 14:56
50f97b8 to
7a79c78
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Long-running synchronous grpc calls block the thread where they are
called, which can cause the scheduler to be unable to terminate the
job.
This commit replaces them with usage of the "asynchronous" future
API. The result() method of a grpc future will not block the thread
while it's waiting, and thus allows terminating the job without
waiting for the blocking call to return. This covers the LocalCAS
methods {Fetch,Upload}{Tree,MissingBlobs}.
As an added benefit, we try to cancel the grpc calls when
terminated. This depends on the server side (buildbox-casd) promptly
cancelling the remote calls, which it doesn't consistently do at the
moment. But that can be fixed independently.
Fixes #2157