Description
salt.client.ssh.Single.__init__ sets self.minion_opts["__master_opts__"] = self.context["master_opts"] for relenv targets, embedding the master's entire own config (395 keys in a typical test master) into the minion config file that gets serialized (self.minion_config) and shipped to and read by salt-call on the remote target.
__master_opts__ is a master-side-only convention: every other reader of it (salt/client/ssh/wrapper/cmdmod.py, cp.py, publish.py, salt/client/ssh/state.py, salt/roster/__init__.py) pulls it from the Python wrapper opts dict while running on the master; nothing on the remote target ever reads it back out of its own minion config.
Impact
self.context["master_opts"] is an alias for the master's own opts, which gets mutated as nested Single/wrapper calls restore/adjust the master cachedir (#69605, #68458). Embedding it in the relenv minion config meant that config grew, unbounded, with every nested Single created during a single state run -- measured at ~20 KB -> ~120 KB -> ~207 KB in one reproduction -- until it exceeded the kernel's per-argument limit (MAX_ARG_STRLEN, 128 KB) and the ssh invocation failed with:
An Exception occurred while executing state.apply: Failed to spawn the VT.
Error: [Errno 7] Argument list too long: '/usr/bin/ssh'
This reproduced reliably in CI on tests/pytests/integration/ssh/state/test_pillar_override*.py (relenv parametrization) on Photon OS, Debian Arm64, and Ubuntu Arm64 runners.
Fix
Fixed in #70194: stop setting self.minion_opts["__master_opts__"] in Single.__init__'s relenv branch. No other code reads __master_opts__ from the minion's own config, so this is pure removal, no replacement needed.
History note
This issue was originally filed describing a different, now-superseded theory (that Single.run_wfunc() never dispatches to the existing but unused _run_wfunc_relenv() method, and that fixing that dispatch gap would resolve the ARG_MAX bug). That turned out to be incomplete/risky to implement (no deploy-bootstrap fallback, a config-path mismatch) and, more importantly, not the actual cause -- the __master_opts__ embedding above is. The dispatch gap is still real dead code, just not the cause of this bug; tracked separately in #70225.
Related
Description
salt.client.ssh.Single.__init__setsself.minion_opts["__master_opts__"] = self.context["master_opts"]for relenv targets, embedding the master's entire own config (395 keys in a typical test master) into the minion config file that gets serialized (self.minion_config) and shipped to and read bysalt-callon the remote target.__master_opts__is a master-side-only convention: every other reader of it (salt/client/ssh/wrapper/cmdmod.py,cp.py,publish.py,salt/client/ssh/state.py,salt/roster/__init__.py) pulls it from the Python wrapper opts dict while running on the master; nothing on the remote target ever reads it back out of its own minion config.Impact
self.context["master_opts"]is an alias for the master's own opts, which gets mutated as nestedSingle/wrapper calls restore/adjust the master cachedir (#69605, #68458). Embedding it in the relenv minion config meant that config grew, unbounded, with every nestedSinglecreated during a single state run -- measured at ~20 KB -> ~120 KB -> ~207 KB in one reproduction -- until it exceeded the kernel's per-argument limit (MAX_ARG_STRLEN, 128 KB) and the ssh invocation failed with:This reproduced reliably in CI on
tests/pytests/integration/ssh/state/test_pillar_override*.py(relenvparametrization) on Photon OS, Debian Arm64, and Ubuntu Arm64 runners.Fix
Fixed in #70194: stop setting
self.minion_opts["__master_opts__"]inSingle.__init__'s relenv branch. No other code reads__master_opts__from the minion's own config, so this is pure removal, no replacement needed.History note
This issue was originally filed describing a different, now-superseded theory (that
Single.run_wfunc()never dispatches to the existing but unused_run_wfunc_relenv()method, and that fixing that dispatch gap would resolve the ARG_MAX bug). That turned out to be incomplete/risky to implement (no deploy-bootstrap fallback, a config-path mismatch) and, more importantly, not the actual cause -- the__master_opts__embedding above is. The dispatch gap is still real dead code, just not the cause of this bug; tracked separately in #70225.Related