Skip to content

Gather a SyncWrapper's pending tasks inside its own loop (#70226) - #70227

Open
ggiesen wants to merge 1 commit into
saltstack:3008.xfrom
ggiesen:fix-asyncio-gather-loop-py314
Open

Gather a SyncWrapper's pending tasks inside its own loop (#70226)#70227
ggiesen wants to merge 1 commit into
saltstack:3008.xfrom
ggiesen:fix-asyncio-gather-loop-py314

Conversation

@ggiesen

@ggiesen ggiesen commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Stops proxy minions logging a wall of Error during asyncio shutdown on Python 3.14, and drains the pending tasks that error was preventing.

What issues does this PR fix?

Fixes #70226

Root cause

asyncio.gather has taken no loop argument since 3.10, so it resolves the loop from the calling context. SyncWrapper.close() runs outside the loop it is tearing down, and the pending tasks belong to self.asyncio_loop, so on Python 3.14 ensure_future rejects the mismatch:

ValueError: The future belongs to a different loop than the one specified as the loop argument

Earlier versions took the loop from the first future and let it through. Instrumenting the call site shows the loops never match (same=False on every call).

The broad except below caught it, so nothing crashed -- but it was logged every time, and the pending tasks were never drained, which is the work close() was doing.

Building the gather inside the loop fixes it on every version, and mirrors how the shutdown_asyncgens and shutdown_default_executor calls just below already construct their coroutine and close it if the loop cannot run it.

Previous Behavior

Every proxy minion on Python 3.14 logged this repeatedly at startup. On a freshly started single proxy it was the entire log:

occurrences
salt-call --local 0
salt-master 0
single salt-proxy 20
deltaproxy, 2 sub-proxies 58
any of the above on Python 3.10 0

New Behavior

None, on both. Verified on Python 3.14.7 with salt 3008.2: the single proxy's log went from 20 lines to empty, and the deltaproxy's from 58 to 2, with both still fully functional.

Merge requirements satisfied?

  • Docs
  • Changelog
  • Tests written/updated

Tests written?

Yes, one added to tests/pytests/unit/utils/test_asynchronous.py. It leaves a task pending on a wrapper's own loop and then calls close() from inside a different running loop, which is how the failure is reached, and asserts nothing was logged.

Worth noting for review: the test asserts on the swallowed log call rather than on the tasks. Asserting the tasks were drained does not work as a guard -- they are cancelled before the gather, so they report done() either way, and a loop cannot be driven to completion from inside another running loop. Against unmodified 3008.x on Python 3.14 the test fails with exactly the ValueError above. The whole file passes on both 3.14 and 3.10 with the fix.

Commits signed with GPG?

No

…0226)

asyncio.gather has taken no loop argument since 3.10, so it resolves the
loop from the calling context. SyncWrapper.close() runs outside the loop
it is tearing down and the pending tasks belong to that loop, so on
Python 3.14 ensure_future rejects the mismatch with "The future belongs
to a different loop than the one specified as the loop argument".
Earlier versions took the loop from the first future and let it through.

The broad except below caught it, so nothing crashed, but every proxy
minion logged it repeatedly at startup -- 20 lines for a single proxy,
58 for a deltaproxy with two sub-proxies, and on a fresh single proxy
that was the entire log. The pending tasks were also never drained,
which is the work close() was doing.

Build the gather inside the loop instead, where the running loop is the
right one on every version.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:full Run the full test suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: proxy minions log "Error during asyncio shutdown: The future belongs to a different loop" on Python 3.14

2 participants