fix(extensions): wire up list --available catalog query + harden add --from path traversal by darion-yaphet · Pull Request #3051 · github/spec-kit

darion-yaphet · 2026-06-18T05:19:47Z

What

Wires up the long-advertised extension list --available / --all catalog listing, and hardens extension add --from against path traversal. Split out of the PR-7 structural refactor (#3014) so the behavior change is reviewed on its own, with tests.

Depends on #3014. This branch is built on top of refactor/split-init-pr7 because the handler lives in extensions/_commands.py, which that refactor introduces. Until #3014 merges, its structural-move commit will also show in this diff; it disappears once #3014 lands in main.

Why

The --available / --all flags have existed since the original extension system (#1551), and their help text has always advertised "Show available extensions from catalog." But the implementation was a stub — it printed a static specify extension add <name> hint and never queried the catalog, even though ExtensionCatalog already existed. This is a long-standing dead/misleading-flag fix, not a new feature, and it's orthogonal to the refactor — which is why it's pulled out of #3014.

Changes

extension list --available / --all: query the catalog and list uninstalled extensions (filtering out installed IDs). --available lists catalog-only; --all lists installed + available; discovery-only entries are marked; a clear error is surfaced and the command exits non-zero when the catalog is unavailable.
extension add --from <url>: sanitize the extension label before building the download filename, so ../-style separators can no longer escape the downloads cache dir (path traversal).

Tests

test_extension_list_available.py — catalog query, installed-ID filtering, discovery-only entries, empty catalog, catalog-error exit, --all showing both sections.
test_extension_add_path_traversal.py — a traversal label is sanitized so the download stays inside the downloads dir.

Both suites verified red against the pre-fix behavior and green after.

Copilot

Pull request overview

This PR wires up the previously-stubbed specify extension list --available/--all behavior to actually query the extension catalog (including filtering out installed extensions and marking discovery-only entries), and hardens specify extension add --from <url> by sanitizing the user-provided label before forming the downloaded ZIP filename (preventing path traversal).

Changes:

Implement catalog-backed listing for extension list --available and --all, including installed-ID filtering and clearer catalog-unavailable failures.
Sanitize the extension label used for --from URL download filenames to prevent ../-style path traversal.
Add targeted tests for catalog-backed listing behavior and the path traversal hardening.

Show a summary per file

File	Description
`tests/test_extension_list_available.py`	Adds behavioral tests for `extension list --available/--all` catalog querying, filtering, and error cases.
`tests/test_extension_add_path_traversal.py`	Adds a security regression test ensuring `--from` label sanitization keeps downloads within the cache directory.
`src/specify_cli/extensions/_commands.py`	Introduces the extension/catalog Typer command handlers (moved from `__init__.py`) and implements the new `list --available/--all` + `add --from` hardening.
`src/specify_cli/extensions/__init__.py`	Updates imports for the new `extensions` package structure (part of the handler move).
`src/specify_cli/__init__.py`	Registers the new `extensions/_commands.py` command group to preserve the CLI surface after the move.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 5/5 changed files
Comments generated: 2

mnriem

Please address Copilot feedback

Copilot

Copilot's findings

Files reviewed: 3/3 changed files
Comments generated: 2

mnriem · 2026-06-23T12:40:34Z

Please address Copilot feedback

Copilot

Copilot's findings

Files reviewed: 4/4 changed files
Comments generated: 1

                        # Download extension ZIP (use resolved ID, not original argument which may be display name)
                        extension_id = ext_info['id']
-                        console.print(f"Downloading {_escape_markup(str(ext_info['name']))} v{_escape_markup(str(ext_info.get('version', 'unknown')))}...")
+                        console.print(f"Downloading {_escape_markup(str(ext_info.get('name', extension_id)))} v{_escape_markup(str(ext_info.get('version', 'unknown')))}...")
                        zip_path = catalog.download_extension(extension_id)


mnriem · 2026-06-23T14:15:54Z

Please address Copilot feedback

Copilot

Copilot's findings

Files reviewed: 5/5 changed files
Comments generated: 5

+            for ext in available_exts:
+                # Catalog fields are untrusted (remote/community catalogs); escape
+                # before embedding in Rich markup to prevent markup injection.
+                safe_id = _escape_markup(str(ext.get("id", "")))
+                verified_badge = " [green]✓ Verified[/green]" if ext.get("verified") else ""
+                safe_name = _escape_markup(str(ext.get("name", "(unnamed)")))
+                safe_version = _escape_markup(str(ext.get("version", "?")))
+                console.print(f"  [bold]{safe_name}[/bold] (v{safe_version}){verified_badge}")
+                console.print(f"     [dim]{safe_id}[/dim]")
+                console.print(f"     {_escape_markup(str(ext.get('description', '')))}")
+                install_allowed = ext.get("_install_allowed", True)
+                if install_allowed:
+                    console.print(f"     [cyan]Install:[/cyan] specify extension add {safe_id}")
+                else:
+                    catalog_name = _escape_markup(str(ext.get("_catalog_name", "")))
+                    console.print(f"     [yellow]Discovery only — not installable from '{catalog_name}'[/yellow]")
+                console.print()


+            console.print(f"[bold]{_escape_markup(str(ext.get('name', '(unnamed)')))}[/bold] (v{_escape_markup(str(ext.get('version', '?')))}){verified_badge}")
+            console.print(f"  {_escape_markup(str(ext.get('description', '')))}")


    # Header
    verified_badge = " [green]✓ Verified[/green]" if ext_info.get("verified") else ""
-    console.print(f"\n[bold]{_escape_markup(str(ext_info['name']))}[/bold] (v{_escape_markup(str(ext_info['version']))}){verified_badge}")
+    console.print(f"\n[bold]{_escape_markup(str(ext_info.get('name', '(unnamed)')))}[/bold] (v{_escape_markup(str(ext_info.get('version', '?')))}){verified_badge}")



    # Description
-    console.print(f"{_escape_markup(str(ext_info['description']))}")
+    console.print(f"{_escape_markup(str(ext_info.get('description', '')))}")


        version = ext_info.get("version", "unknown")
        zip_filename = f"{extension_id}-{version}.zip"
        zip_path = target_dir / zip_filename
+        try:
+            zip_path.resolve().relative_to(target_dir.resolve())


mnriem · 2026-06-23T18:20:51Z

Please address Copilot feedback

…ersion Address remaining Copilot review on PR github#3051: - list --available/--all and search now skip catalog entries without a valid id and coalesce JSON null name/version/description to placeholders instead of rendering the literal "None". - extension info gets the same null coalescing for name/version/ description/author/license via a shared _catalog_str helper. - download_extension sanitizes the catalog-provided version into a filename-safe token so a version with separators cannot produce a nested path or write failure, rather than relying on the post-hoc resolve() containment check alone. - Fix a stale mock in test_add_from_url_sanitizes_traversal_label (missing extra_headers kwarg, non-ZIP body) so the suite is green. New tests cover null-field rendering, id-skipping, the version token sanitizer, and the download filename path.

Copilot

Review details

Files reviewed: 5/5 changed files
Comments generated: 3
Review effort level: Low

+        available_exts = [ext for ext in results if ext.get("id") not in installed_ids]
+


        console.print(f"\n[green]Found {len(results)} extension(s):[/green]\n")

        for ext in results:
+            # Catalog entries are untrusted; a missing/blank id cannot be
+            # installed or referenced, so skip it rather than emit a bogus


                        # Download extension ZIP (use resolved ID, not original argument which may be display name)
                        extension_id = ext_info['id']
-                        console.print(f"Downloading {_escape_markup(str(ext_info['name']))} v{_escape_markup(str(ext_info.get('version', 'unknown')))}...")
+                        console.print(f"Downloading {_escape_markup(str(ext_info.get('name', extension_id)))} v{_escape_markup(str(ext_info.get('version', 'unknown')))}...")


mnriem · 2026-07-06T20:56:27Z

Please address Copilot feedback

Copilot

Review details

Files reviewed: 5/5 changed files
Comments generated: 1
Review effort level: Low

darion-yaphet · 2026-07-08T02:50:12Z

+    monkeypatch.setattr(
+        ExtensionCatalog,
+        "search",
+        lambda self, **kwargs: [{"id": "broken-ext"}],  # only id present
+    )


I’ll make the narrow fix: add a real ExtensionCatalog.search() regression test by stubbing _get_merged_extensions, then harden the search filtering so malformed catalog fields are treated as empty rather than crashing.

…ry the catalog - extension add --from: sanitize the extension label before building the download filename so "../" path separators can no longer escape the downloads dir and overwrite arbitrary files - extension list --available/--all: actually query the catalog and list uninstalled extensions (filtering out installed IDs), instead of only printing a static install hint that contradicted the CLI help and docs

… path traversal Add regression coverage for the two behaviors wired up in the preceding fix: - list --available/--all: queries the catalog, filters installed IDs, marks discovery-only entries, reports an empty catalog, and exits 1 on catalog failure. - add --from <url>: a label containing path separators is sanitized so the download cannot escape the downloads cache dir. Both suites were verified red against the pre-fix behavior and green after.

…ction - Escape untrusted catalog fields (name, version, id, description, catalog name) and the catalog-query error before embedding them in Rich markup, preventing markup injection in `extension list --available/--all` output - Always print the "Installed Extensions:" header when the installed section is shown, with a "No extensions installed." note when empty, so `--all` output is no longer missing the section on an empty project - Update the command docstring to reflect --available/--all catalog listing so `--help` is accurate

…ries Catalog entries are untrusted (remote/community catalogs) and only guaranteed to carry an injected `id`; every other field comes straight from `**ext_data`. Hard subscripts and unguarded `.get()` chains on that data could crash `search`, `info`, `add`, and `update`, and a couple of catalog-controlled values were printed without Rich-markup escaping. - Use `.get()` fallbacks for name/version/description across search/info/add resolution and the download message - Guard `requires`/`provides` with isinstance(dict) before `.get()`, and skip non-dict tool entries - Catch KeyError alongside InvalidVersion in `update` version parsing - Escape catalog-controlled `stars` before printing - Correct the add --from path-traversal test docstring to describe the real mitigation (generated tempfile in the downloads dir, not label sanitization) - Add regression tests for malformed catalog entries in list/search/info

Remote extension catalogs are untrusted, and catalog entry IDs feed into the cached ZIP filename used by extension installation. The download path now validates IDs against the manifest ID rule before catalog lookup and verifies the final ZIP path stays under the downloads directory before writing. The issue-template agent list is also synchronized with runtime integrations so consistency checks pass for newly registered agents. Constraint: Remote catalog metadata controls extension IDs and versions before local manifest validation can run Rejected: Sanitize path separators in the filename | silently rewriting remote IDs could install an extension under an unexpected identity Confidence: high Scope-risk: narrow Directive: Keep catalog-sourced identifiers validated before using them in filesystem paths Tested: uv run pytest tests/test_extension_catalog_robustness.py tests/test_extension_add_path_traversal.py tests/test_extension_update_hardening.py tests/test_extension_list_available.py -q Tested: uv run pytest tests/test_agent_config_consistency.py -q Tested: uv run python -m compileall -q src/specify_cli/extensions tests/test_extension_catalog_robustness.py Tested: git diff --check Not-tested: Full uv run pytest Assisted-by: Codex (model: GPT-5, autonomous) Co-authored-by: OmX <omx@oh-my-codex.dev>

…ersion Address remaining Copilot review on PR github#3051: - list --available/--all and search now skip catalog entries without a valid id and coalesce JSON null name/version/description to placeholders instead of rendering the literal "None". - extension info gets the same null coalescing for name/version/ description/author/license via a shared _catalog_str helper. - download_extension sanitizes the catalog-provided version into a filename-safe token so a version with separators cannot produce a nested path or write failure, rather than relying on the post-hoc resolve() containment check alone. - Fix a stale mock in test_add_from_url_sanitizes_traversal_label (missing extra_headers kwarg, non-ZIP body) so the suite is green. New tests cover null-field rendering, id-skipping, the version token sanitizer, and the download filename path.

…lbacks - list --available: filter with _catalog_id() before the empty check so an all-invalid-id catalog shows the "no additional extensions" fallback - search: count/iterate only valid-id entries and render _catalog_name via _catalog_str() so an explicit JSON null no longer prints "Catalog: None" - add: render download status name/version via _catalog_str() so null fields fall back to the resolved id / "unknown" instead of the literal "None" - add regression tests for each path

Remote/community catalog entries may contain null values or malformed tag data. Search filtering runs before CLI rendering fallbacks, so it must normalize unsafe field shapes before matching query, author, and tag filters. Coerce non-string search fields to empty text, ignore non-list tags, and drop non-string tag entries during matching. Add a regression test that exercises the real ExtensionCatalog.search path by stubbing _get_merged_extensions instead of replacing search itself.

darion-yaphet requested a review from mnriem as a code owner June 18, 2026 05:19

darion-yaphet marked this pull request as draft June 18, 2026 05:21

darion-yaphet mentioned this pull request Jun 18, 2026

refactor: move extension command handlers to extensions/_commands.py (PR-7/8) #3014

Merged

mnriem requested a review from Copilot June 18, 2026 13:32

Copilot started reviewing on behalf of mnriem June 18, 2026 13:32 View session

Copilot AI reviewed Jun 18, 2026

View reviewed changes

Comment thread src/specify_cli/extensions/_commands.py Outdated

Comment thread src/specify_cli/extensions/_commands.py

mnriem requested changes Jun 19, 2026

View reviewed changes

darion-yaphet force-pushed the fix/extension-list-available branch from 094f5ee to d468c02 Compare June 23, 2026 01:23

darion-yaphet marked this pull request as ready for review June 23, 2026 01:42

mnriem requested a review from Copilot June 23, 2026 11:58

Copilot started reviewing on behalf of mnriem June 23, 2026 11:59 View session

Copilot AI reviewed Jun 23, 2026

View reviewed changes

Comment thread src/specify_cli/extensions/_commands.py Outdated

Comment thread tests/test_extension_add_path_traversal.py Outdated

mnriem requested a review from Copilot June 23, 2026 14:03

Copilot started reviewing on behalf of mnriem June 23, 2026 14:04 View session

Copilot AI reviewed Jun 23, 2026

View reviewed changes

darion-yaphet force-pushed the fix/extension-list-available branch from f81d1ef to a7da3b9 Compare June 23, 2026 17:19

mnriem requested a review from Copilot June 23, 2026 17:38

Copilot started reviewing on behalf of mnriem June 23, 2026 17:39 View session

Copilot AI reviewed Jun 23, 2026

View reviewed changes

darion-yaphet force-pushed the fix/extension-list-available branch from a7da3b9 to 40c0bfd Compare July 3, 2026 08:48

mnriem requested a review from Copilot July 6, 2026 13:56

Copilot started reviewing on behalf of mnriem July 6, 2026 13:57 View session

Copilot AI reviewed Jul 6, 2026

View reviewed changes

mnriem requested a review from Copilot July 7, 2026 19:35

Copilot started reviewing on behalf of mnriem July 7, 2026 19:36 View session

Copilot AI reviewed Jul 7, 2026

View reviewed changes

darion-yaphet and others added 8 commits July 8, 2026 10:55

darion-yaphet force-pushed the fix/extension-list-available branch from ba7a6c7 to ac236ad Compare July 8, 2026 02:55

		console.print(f"[bold]{_escape_markup(str(ext.get('name', '(unnamed)')))}[/bold] (v{_escape_markup(str(ext.get('version', '?')))}){verified_badge}")
		console.print(f" {_escape_markup(str(ext.get('description', '')))}")

		available_exts = [ext for ext in results if ext.get("id") not in installed_ids]

Uh oh!

Conversation

darion-yaphet commented Jun 18, 2026

What

Why

Changes

Tests

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

mnriem left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

mnriem commented Jun 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

mnriem commented Jun 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

mnriem commented Jun 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Review details

Uh oh!

mnriem commented Jul 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Review details

Uh oh!

darion-yaphet Jul 8, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants