Skip to content

test: resolve the test harness repo root with fileURLToPath and cover the fixture-sandbox contract - #354

Merged
mrbobbytables merged 2 commits into
mainfrom
quality/test-harness-repo-root
Sep 21, 2026
Merged

mrbobbytables merged 2 commits into
mainfrom
quality/test-harness-repo-root

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

Claims exactly three files: tests/helpers.mjs (the module-level repoRoot
constant only), tests/validators-smoke.test.mjs, and a new
tests/helpers.test.mjs. Cluster: the fixture-sandbox harness contract.

The break

Both tests/helpers.mjs:6 and tests/validators-smoke.test.mjs:6 derived the
repository root from URL.prototype.pathname, which stays percent-encoded.
On a checkout whose path contains a space the root resolved to
/tmp/space%20dir/repo/ — a directory that does not exist — so every sandbox
run died in cpSync with ENOENT before the script under test was ever
spawned:

$ cp -r endusers "/tmp/space dir/repo" && cd "/tmp/space dir/repo" && node --test
ℹ tests 55
ℹ pass 0
ℹ fail 55

~/My Documents/, /Users/jane doe/ and C:\Users\...\My Repos\ are all
ordinary developer layouts. This resolves the root with fileURLToPath
instead, and exports the resolution so it can be asserted directly rather than
only observed through a downstream ENOENT.

The missing coverage

runScriptWithFixtures is the single point every fixture-based validator test
runs through, and nothing exercised it. A regression in the harness degrades
those suites rather than failing them — a sandbox that quietly read the real
repository would make every "rejects X" assertion pass for the wrong reason,
and no assertion anywhere would notice.

tests/helpers.test.mjs pins the contract in both directions: percent-decoded
root resolution, scripts/lib mirroring (so an ERR_MODULE_NOT_FOUND can
never masquerade as a rejected fixture), nested fixture placement, isolation
from the real repository, isolation between successive runs, temp-directory
cleanup, and distinct reporting of status/stdout/stderr.

Verification

Revision 00b44df, node v26.8.2, 2026-09-20.

checkout path before after
space-free 55 tests, 55 pass 63 tests, 63 pass
/tmp/space dir/ 55 tests, 0 pass 63 tests, 62 pass

The one test still failing under a spaced path is
validate-architectures.mjs passes against current repo data, which is the
production half of the same defect — six scripts/*.mjs files carry the
identical .pathname line. That half is not in this PR: it is production
code under scripts/, which the quality lane cannot push. It is filed with
exact replacement text as #353 and needs a human or an agent whose lane can
touch scripts/ to land it.

Falsification check: reverting resolveRepoRoot to .pathname fails exactly
the new percent-decoding assertion (8 tests, 7 pass, 1 fail), so the new test
cannot pass vacuously. prettier --check clean on all three files.

Disjointness from open PRs

Related Issue

Closes #352 — merging this leaves nothing for #352 to track: it scopes only the
tests/ half, and both of its boxes land here. The scripts/ half is tracked
separately as #353 and is deliberately out of scope for this lane.


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5

tests/helpers.mjs and tests/validators-smoke.test.mjs both derived the
repository root from URL.prototype.pathname, which stays percent-encoded.
On a checkout whose path contains a space the root resolved to a directory
that does not exist and all 55 tests failed in cpSync with ENOENT before
any script under test was spawned.

Resolve the root with fileURLToPath instead, and export the resolution so
it can be asserted directly.

Add tests/helpers.test.mjs covering the fixture-sandbox contract, which
had no tests of its own even though every fixture-based validator test
runs through it: percent-decoded root resolution, scripts/lib mirroring,
nested fixture placement, isolation from the real repository, isolation
between runs, temp-directory cleanup, and distinct reporting of status,
stdout and stderr.

Signed-off-by: kubestellar-hive[bot] <kubestellar-hive@hive.kubestellar.io>
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the outreach agent is always held because it publishes project-facing communication.

Hive will automatically remove the hold label once current policy no longer requires a level hold for "quality". If this is an outreach PR, a human must review it and remove the label.

The cleanup assertion counted endusers-test-* entries in the shared temp
directory before and after a run. node --test executes test files in
parallel processes, so a sandbox created and removed by another file
changes that count mid-assertion; CI failed with 1 !== 2 against a
harness that cleans up correctly.

Recover the sandbox path from the ENOENT the fixture-less run emits and
assert that exact directory is gone, which is independent of concurrent
runs.

Signed-off-by: kubestellar-hive[bot] <kubestellar-hive@hive.kubestellar.io>
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

CI triage: the Validate repository failure on this PR was PR-local, not a shared incident — hive-baseline-check.sh cncf/endusers 'Validate repository' exited 2 (could not read open PR checks), and manual diagnosis showed sibling quality PRs #360, #350, #347 and #343 green on the same check.

Root cause was in this PR's own new assertion, not in the harness under test. the sandbox directory is removed once the run returns counted endusers-test-* entries in the shared temp directory before and after a run. node --test runs test files in parallel processes, so a sandbox created and removed by another test file changes that count mid-assertion — hence 1 !== 2 against a harness that cleans up correctly.

Fixed in c96210b by pinning the assertion to this run's own sandbox: the fixture-less run dies with an ENOENT naming its sandbox path, and the test asserts that exact directory no longer exists. That is independent of concurrent runs.

Verified at node v26.8.2: npm run test:unit 63/63 pass, node --test tests/helpers.test.mjs 8/8 pass across repeated runs, prettier --check clean.

hold label stays on; human review still required.

🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: c96210b

— hive: agent=quality backend=copilot model=claude-opus-5

@mrbobbytables
mrbobbytables added this pull request to the merge queue Sep 21, 2026
Merged via the queue into main with commit f3f5894 Sep 21, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

1 participant