Read MCP registration sites in the zero-install detector (#485)

[pengfei-threemoonslab]

Closes #485.

The gap

tools/shipgate-detect.py is the documented first command an agent runs against a repository that has not adopted Shipgate — which is 100% of the population #431 is about. #431 taught the installed CLI to read a tool's name out of a TypeScript or Go registration site; the script did not gain it, so the two disagreed on the one question the script exists to answer.

Measured on live checkouts, before and after:

workspace	detect	script (before)	script (after)
mongodb-js/mongodb-mcp-server	true, 61 tools, route packages	false — "Stop"	true, 61 tools, route packages
grafana/mcp-grafana	true, 114 tools, route tools	false — "Stop"	true, 114 tools, route tools
github/github-mcp-server	true, 114 tools, route pkg/github	false — "Stop"	true, 114 tools, route pkg/github

On all three, suggested_sources, excluded_sources and the framework's evidence lines are now identical between the two detectors, character for character.

What is ported

Into the stdlib-only single file: the masking lexer (both language grammars, both escape decoders, the regex/division heuristic), the five idioms, is_scannable_path, the containment rule, the declared-dependency gate, the common-ancestor route, export precedence, and the scoring. FRAMEWORKS gains mcp_server_source; CONVENTIONAL_DIR_FRAMEWORKS is mirrored so it stays out of the conventional-directory bonus. SCRIPT_VERSION → 0.5.0.

Of the 40 reader functions shared with agents_shipgate.inputs.mcp_idioms, all 40 have identical bodies; of the 9 shared discovery functions, 7 do. The two that differ do so because they must, and the script says so where a reviewer diffing the files will read it: the export probe uses json instead of the pydantic-backed load_mcp_tools (the same accept rule _probe_mcp already mirrors), and the source read is factored into one function so the strict-decode contract is visible in one place.

Why the duplication is affordable

A second implementation of the load-bearing matcher is the recurring bug class here, so the two are not allowed to become different implementations.

tests/mcp_idiom_corpus.py now holds every case either reader has ever been asked about — each idiom's positive sample, the whole adversarial sweep, the path predicate's cases, both escape grammars, the masking failures, and the review regressions. tests/test_mcp_idioms.py checks the package reader against the expected names and omissions; tests/test_zero_install_detector.py checks the two readers against each other on the same inputs, comparing every field of every site including its byte span — the span is what decides whether a wrapper call reports a second omission for a tool its own argument already named. The whole corpus runs a second time with CRLF line endings, because a Windows checkout is a supported one and it moves every offset.

Around that:

• samples/mcp_source_only_server — a TypeScript MCP server with no export, so the existing samples/ parity sweep covers the route. Two tools named, one registration deliberately unenumerated.
• Thirteen constructed workspaces pin the branches a sample cannot reach: TypeScript, Go, both languages, a declared language that registers nothing, no declared dependency, no resolved registration, test-only registrations, covering export, partial export, wildcard export, an unresolved site outside the route, two registration directories, and the workspace root. A companion test pins the CLI's own answer for each, because a fixture set that never fires the route would agree perfectly and prove nothing.
• Parity on the strict read (undecodable bytes in a comment, placed so a lenient decode would widen the route directory), on the source-file cap, and on the export size bound.
• mcp_server_source is the one framework whose score, confidence, evidence lines and exclusion reason are pinned, not just its presence — the conjunction scoring, the "N registrations name themselves at runtime" line, and the sentence a reader gets when the route is withheld are all invisible to every other assertion.
• test_framework_vocabulary_names_every_cli_omission passes with an empty known_omissions; the idiom ids and omission reasons are pinned from both sides, against the CLI registry and against what the script's reader actually emits.

Six review rounds ran a perturbation sweep over the port, the discovery layer and the masker. Everything reachable is now caught by the test that claims to guard it. Rounds five and six audit the original reader as much as the port: a Go raw string decoded as though it were escape-processed invents raw_name, a / inside a regex character class ends the pattern early and swallows the line after it, three word-boundary lookbehinds admit constructs that merely end in MustTool / NewTool / Tool, and operationType is safe only because its pattern requires static — which the module says and nothing checked, while its twin description had both the fix and a regression. All six are corpus cases now, so they pin both readers.

Two perturbations stay invisible, provably, and the code says why at the line: the wildcard branch in the ported export reader cannot be distinguished from its fall-through for any input the probe accepts, and the ) early exit in _call_sites is redundant with the second-argument check beneath it.

One defect fixed in both

With no MCP export in the workspace at all, _covering_export returned every resolved name as "uncovered", and the caller renders a shortfall as "An MCP tool export is also present and does not name N of these registrations". A server whose surface exists only as source is exactly the population this input was built for, so that claim about a file that does not exist was published into the adoption evidence for every one of them — MongoDB, Grafana and GitHub included. Guarded in cli/discovery/mcp_source.py and in the port, with a regression test.

Also refreshed one stale prose count in cli/scope_routing.py (samples/ was 23 of 25 adopted, not 21 of 22, before this PR added a sample).

Left out, deliberately

_probe_suggested in the script reads every glob-matched JSON candidate with no size bound, while probe_suggested_source refuses inputs over 10 MB — a pre-existing divergence in suggested_sources and an unbounded read in a script that is curled onto unknown repositories. The new export reader here is bounded; the probe is filed separately rather than widened into this PR.

Verification

• ruff check . clean; full suite green (pytest -n auto -m "not perf", tests/test_adapter_static_only.py, and the perf run, each separately).
• The script runs standalone on a Python with no agents_shipgate importable, confirming it stayed stdlib-only.
• Acceptance criteria in The zero-install detector does not read MCP registration sites, so it still reports vendor MCP servers as non-agent projects #485: all three checked, on the live repositories.

🤖 Generated with Claude Code