Skip to content

Tear down deltaproxy sub-proxies on shutdown (#70221) - #70222

Open
ggiesen wants to merge 1 commit into
saltstack:3008.xfrom
ggiesen:fix-deltaproxy-subproxy-destroy
Open

Tear down deltaproxy sub-proxies on shutdown (#70221)#70222
ggiesen wants to merge 1 commit into
saltstack:3008.xfrom
ggiesen:fix-deltaproxy-subproxy-destroy

Conversation

@ggiesen

@ggiesen ggiesen commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Makes a deltaproxy tear down its sub-proxies when it stops.

What issues does this PR fix?

Fixes #70221

Root cause

post_master_init builds the sub-proxies into self.deltaproxy_objs, and nothing in the shutdown path ever touches that dict again. salt.minion.ProxyMinion had no destroy() of its own, so it inherited Minion.destroy(), which tears down only the instance it is called on. Each sub-proxy is a full ProxyMinion with its own req_channel, schedule, beacons and periodic callbacks, so all of those were abandoned on every stop or restart.

Previous Behavior

A SIGTERM to a deltaproxy hosting three sub-proxies produced exactly one teardown:

destroy dpcontrol

New Behavior

destroy minion1
destroy minion2
destroy minion3
destroy dpcontrol

Merge requirements satisfied?

  • Docs
  • Changelog
  • Tests written/updated

Tests written?

Yes, three in tests/pytests/unit/test_minion.py: sub-proxies are torn down and the control proxy still tears itself down afterwards; one sub-proxy raising does not stop the others or the control proxy; and a single proxy with no deltaproxy_objs is unaffected. The first two fail against unmodified 3008.x.

Verified against a live deltaproxy on 3008.2 with three sub-proxies, where destroy() calls went from 1 to 4 in the order above.

One note on reproducing: this is only observable once the proxy actually reaches its teardown. On an unpatched 3008.2 the daemon exits before the scheduled graceful shutdown runs at all, which is #70217 / #70218, so that fix needs to be in place first or nothing is torn down regardless. The two are independent changes.

A related question, deliberately not addressed here

While tracing this I noticed the proxymodule's own shutdown() is never invoked on daemon stop for any proxy, not just sub-proxies -- the only callers anywhere in salt/ are salt/modules/junos.py and salt/modules/status.py. So a proxymodule that opens a device session has no teardown hook salt itself calls when the daemon stops.

That may be deliberate, and changing it would alter behaviour for every proxy deployment rather than just deltaproxy, so this PR does not touch it. Raised in #70221 in case it is not intentional, and happy to follow up separately if you want it changed.

Commits signed with GPG?

No

A deltaproxy builds its sub-proxies into self.deltaproxy_objs during
post_master_init and nothing in the shutdown path ever touched that dict
again. ProxyMinion had no destroy() of its own, so it inherited
Minion.destroy(), which tears down only the instance it is called on.
Each sub-proxy is a full ProxyMinion with its own req_channel, schedule,
beacons and periodic callbacks, and all of those were abandoned on every
stop or restart.

Give ProxyMinion a destroy() that tears down the sub-proxies first and
then itself. One sub-proxy failing does not stop the others or the
control proxy's own teardown, and a single proxy with no sub-proxies is
unaffected.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:full Run the full test suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: deltaproxy never tears down its sub-proxies, abandoning their channels, schedules and beacons on every stop

2 participants