Skip to content

Give deltaproxy sub-proxies a token for the master (#70071) - #70228

Open
ggiesen wants to merge 1 commit into
saltstack:3008.xfrom
ggiesen:fix-70071-subproxy-tok
Open

Give deltaproxy sub-proxies a token for the master (#70071)#70228
ggiesen wants to merge 1 commit into
saltstack:3008.xfrom
ggiesen:fix-70071-subproxy-tok

Conversation

@ggiesen

@ggiesen ggiesen commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Gives deltaproxy sub-proxies the token they need to talk to the master.

Credit where it is due: @waterWang diagnosed this correctly in #70073 and identified the right cause. That PR has had changes requested since 18 August with no response, and the fix it proposes sets the attribute to None rather than to a token. This takes the work forward with a real token, a test, a changelog, and the 3008.x branch.

What issues does this PR fix?

Fixes #70071

Root cause

A sub-proxy is constructed directly rather than going through connect_master, which is where an ordinary minion picks up self.tok. The attribute was therefore simply missing, and every read of it raised AttributeError.

The path it is actually reached by is:

handle_event -> ProxyMinion.handle_event
  -> pillar_refresh
    -> _register_resources_with_master
       "tok": self.tok  ->  AttributeError

So every saltutil.refresh_pillar and saltutil.refresh_grains failed part way through. The sub-proxy's resources were never registered with the master, and since an empty resource dict is sent deliberately to clear stale entries, stale entries were never cleared either. Everything earlier in pillar_refresh completes first, which is why the pillar refresh still appears to work and the breakage is easy to miss.

Worth recording what is not broken, since the issue is broader than the symptom: pillar refresh itself, event.send, mine.update, status.master and -I pillar targeting all behave correctly without this fix. In testing, only saltutil.refresh_pillar and saltutil.refresh_grains triggered it, once per sub-proxy.

The approach

post_master_init already hands each sub-proxy a req_channel, and AsyncReqChannel.factory builds its auth eagerly, so the token can be derived there the same way connect_master does it:

self.tok = pub_channel.auth.gen_token(b"salt")

That gives the sub-proxy a real token rather than a placeholder. Setting it to None also stops the AttributeError, because AsyncReqChannel._package_load overwrites load["tok"] unconditionally when crypt == "aes" -- I confirmed that -- but it only holds for that one path. self.tok is also read by _fire_master_prepare and _mine_send, and the overwrite does not happen on a clear channel, so a None would silently go out on the wire instead of raising.

Both the parallel and serial startup branches were assigning the channel with identical code, so this folds that into one helper and they cannot drift apart.

Previous Behavior

Every saltutil.refresh_pillar on a deltaproxy logged one AttributeError: 'ProxyMinion' object has no attribute 'tok' per sub-proxy, and no sub-proxy's resources ever reached the master.

New Behavior

No errors, and resource registration completes.

Merge requirements satisfied?

  • Docs
  • Changelog
  • Tests written/updated

Tests written?

Yes, two in tests/pytests/unit/metaproxy/test_deltaproxy.py: the helper builds the channel as before and derives the token from its auth, and the inverse -- a channel with no auth must leave the sub-proxy usable rather than raising while wiring it in. Both fail against unmodified 3008.x.

Verified against a live deltaproxy on 3008.2 with three sub-proxies. saltutil.refresh_pillar before the change produced 3 AttributeErrors, one per sub-proxy; after it produces none, with no "Unable to register resources" warnings and no errors of any kind in the log.

One unrelated tidy-up in the same function

_register_resources_with_master built its load outside the try that already exists to tolerate registration failures, which is why a missing attribute escaped into pillar_refresh and aborted the rest of it instead of being reported like any other registration problem. Moved inside. Happy to drop this if you would rather keep the PR to one thing.

Commits signed with GPG?

No

A sub-proxy is constructed directly rather than going through
connect_master, which is where an ordinary minion picks up self.tok, so
the attribute was simply missing and every read of it raised
AttributeError.

In practice that is _register_resources_with_master, reached from
pillar_refresh, so each saltutil.refresh_pillar and
saltutil.refresh_grains failed part way through: the sub-proxy's
resources were never registered with the master, and because an empty
resource dict is sent deliberately to clear stale entries, stale entries
were never cleared either. The pillar refresh itself completed first, so
the breakage was easy to miss.

post_master_init already hands each sub-proxy a req_channel, so derive
the token from that channel's auth the way connect_master does. Both the
parallel and serial startup branches were assigning the channel with
identical code; fold that into one helper so they cannot drift.

Also build the _register_resources_with_master load inside the try that
already guards the send. It is best-effort, so a problem assembling the
request should be reported like a failure to send it rather than
escaping into pillar_refresh and aborting the rest.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:full Run the full test suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

# [BUG] deltaproxy: sub-proxies raise AttributeError 'tok' when dispatching events (3008 asyncio port)

2 participants