diff --git a/CHANGELOG.md b/CHANGELOG.md index 5a0daf515f..014ea897d7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,11 @@ Full release notes with details on each version: [GitHub Releases](https://githu ## 0.9.63 (2026-09-16) +- Fix: `$(cat graphify-out/.graphify_python)`, the `/graphify` skill's interpreter substitution used at 100+ call sites, was spliced in unquoted everywhere, unlike the already-quoted `"$PYTHON"` form Step 1 itself uses — an interpreter path containing a space (a venv under a user directory with a space in its name) word-split into multiple arguments and failed to exec. Every occurrence is now quoted (#1619, thanks @edtrackai). +- Fix: the `/graphify` skill's `--update` runbook told the agent to back up the old graph for the post-update diff AFTER the merge step and the diff block that reads the backup, so an agent following the runbook top to bottom never created it before the merge ran and the diff silently no-opped on every single update. The instruction now appears before the merge block it belongs with (#1619, thanks @edtrackai). +- Fix: `/graphify` skill Step 1 now stops with an actionable error when the interpreter install still fails after the retry, instead of silently writing a broken interpreter path that every later step then failed against with a cryptic error far removed from the real cause (#1619, thanks @edtrackai). +- Fix: a Windows path substituted into the `/graphify` skill's `INPUT_PATH` placeholder with backslashes corrupted the Python string literal it lands in (a lone `\t` becomes a tab, `\U` raises a `SyntaxError`). Every host now tells the agent to substitute forward slashes instead, and the PowerShell `Resolve-Path` call that saves the scan root is quoted so a path containing a space survives too (#1619, thanks @edtrackai). +- Fix: the `/graphify` skill's `--no-cluster` and `--force` flags are now actually implemented instead of being referenced but never wired through. Step 2 suggested `--no-cluster` for a flat corpus and the shrink-guard error message suggested `--force`, but Step 4 called `cluster()` unconditionally and `to_json()` never received `force=`, so neither flag could do anything. Step 4 now builds a single "Full Corpus" community instead of clustering when `--no-cluster` is given, both `to_json()` calls honor `force=`, and Step 5's labeling is skipped when there is nothing to label. Step 4's write also now passes through the placeholder "Full Corpus" community label it already computes, since it is the only write for that path and every node was silently ending up with no `community_name` at all without it (#1619, thanks @edtrackai). - Feature: Elixir `alias`/`import`/`require`/`use` targets now resolve onto the module's `defmodule` node across files, so the internal module dependency graph is no longer dropped as dangling. Only top-level modules are indexed (a nested `defmodule`, labeled with its bare inner name, cannot capture an unrelated `use ` from another file), and a same-file reference is left unresolved so it cannot clobber the structural `contains` edge (#3603, thanks @ayushcodes10). - Feature: a Rust `self.method()` call now resolves to a method defined on the same type in another file (the common split-`impl`-block layout), pooling methods across every `impl` of one type and refusing to link when two unrelated types share a bare name (#3602, thanks @ayushcodes10). - Feature: a Ruby member call `obj.foo` on a known-type receiver now resolves to a method `foo` inherited from a superclass, including across files, using the same conservative promotion as the implicit-self resolver — a single owning class, matching method kind, and one unambiguous ancestry chain, or it stays dangling (#3585, thanks @oleksii-tumanov). diff --git a/graphify/skill-agents.md b/graphify/skill-agents.md index 190827d9ac..d5457c16c5 100644 --- a/graphify/skill-agents.md +++ b/graphify/skill-agents.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -285,7 +300,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -309,7 +324,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -323,7 +338,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -356,7 +371,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -391,9 +406,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -414,20 +431,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -455,7 +481,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -478,12 +504,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -514,7 +542,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -554,7 +582,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/graphify/skill-aider.md b/graphify/skill-aider.md index 4996beb787..f920ea703d 100644 --- a/graphify/skill-aider.md +++ b/graphify/skill-aider.md @@ -15,6 +15,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --mode deep # thorough extraction, richer INFERRED edges /graphify --update # incremental - re-extract only new/changed files /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) - a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + Follow these steps in order. Do not skip steps. ### Step 1 - Ensure graphify is installed @@ -86,20 +90,31 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out "$PYTHON" -c "import sys; open('graphify-out/.graphify_python', 'w', encoding='utf-8').write(sys.executable)" ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -149,7 +164,7 @@ Set it as `GRAPHIFY_WHISPER_PROMPT` in the environment before running the transc **Step 2 - Transcribe:** ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os from pathlib import Path from graphify.transcribe import transcribe_all @@ -188,7 +203,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -222,7 +237,7 @@ Print: `"Semantic extraction: N files (sequential — Aider)"` Before dispatching any subagents, check which files already have cached extraction results: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -283,7 +298,7 @@ If more than half the chunks failed, stop and tell the user. Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -307,7 +322,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -321,7 +336,7 @@ print(f'Cached {saved} files') Merge cached + new results into `.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -354,7 +369,7 @@ Clean up temp files: `rm -f .graphify_cached.json .graphify_uncached.txt .graphi #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -389,9 +404,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source->target), otherwise `False` (the default undirected `Graph`). Substitute it everywhere it appears, the same way you substitute `INPUT_PATH` - do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given, otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -410,19 +427,26 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Persist the graph first and only write the report/analysis if it actually # persisted - to_json refuses to shrink an existing graph.json (#479), and a # report describing a graph we did not write would be a lie (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely - this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (fewer nodes than the existing graph). Run a full rebuild to be safe.') raise SystemExit(1) @@ -447,12 +471,14 @@ Replace INPUT_PATH with the actual path. ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) - there is only one placeholder community ("Full Corpus"), nothing to label. + Read `.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -482,7 +508,7 @@ Path('.graphify_labels.json').write_text(json.dumps({str(k): v for k, v in label # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (fewer nodes than the existing graph). Run a full rebuild to be safe.') print('Report updated with community labels') @@ -499,7 +525,7 @@ Replace INPUT_PATH with the actual path. If `--obsidian` was given: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_obsidian, to_canvas @@ -530,7 +556,7 @@ print(' _COMMUNITY_* - overview notes with cohesion scores and dataview queries Generate the HTML graph (always, unless `--no-viz`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_html @@ -557,7 +583,7 @@ else: **If `--neo4j`** - generate a Cypher file for manual import: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_cypher @@ -572,7 +598,7 @@ print('cypher.txt written - import with: cypher-shell < graphify-out/cypher.txt' **If `--neo4j-push `** - push directly to a running Neo4j instance. Ask the user for credentials if not provided: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster @@ -594,7 +620,7 @@ Replace `NEO4J_URI`, `NEO4J_USER`, `NEO4J_PASSWORD` with actual values. Default ### Step 7b - SVG export (only if --svg flag) ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_svg @@ -616,7 +642,7 @@ print('graph.svg written - embeds in Obsidian, Notion, GitHub READMEs') ### Step 7c - GraphML export (only if --graphml flag) ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.build import build_from_json from graphify.export import to_graphml @@ -658,7 +684,7 @@ To configure in Claude Desktop, add to `claude_desktop_config.json`: If `total_words` from `.graphify_detect.json` is greater than 5,000, run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.benchmark import run_benchmark, print_benchmark from pathlib import Path @@ -676,7 +702,7 @@ Print the output directly in chat. If `total_words <= 5000`, skip silently - the ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone @@ -759,7 +785,7 @@ The graph is the map. Your job after the pipeline is to be the guide. Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -782,7 +808,7 @@ if new_total > 0: If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -805,7 +831,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -813,11 +839,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json .graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_json @@ -844,7 +871,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -868,8 +895,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json .graphify_old.json` -Clean up after: `rm -f .graphify_old.json` +Clean up the backup after: `rm -f .graphify_old.json` --- @@ -878,7 +904,7 @@ Clean up after: `rm -f .graphify_old.json` Skip Steps 1–3. Load the existing graph from `graphify-out/graph.json` and re-run clustering: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.cluster import cluster, score_all from graphify.analyze import god_nodes, surprising_connections @@ -933,7 +959,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -951,7 +977,7 @@ Load `graphify-out/graph.json`, then: 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -1042,7 +1068,7 @@ Replace `QUESTION` with the user's actual question, `MODE` with `bfs` or `dfs`, After writing the answer, save it back into the graph so it improves future queries: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `QUESTION` with the question, `ANSWER` with your full answer text, `SOURCE_NODES` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -1055,7 +1081,7 @@ Find the shortest path between two named concepts in the graph. First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -1065,7 +1091,7 @@ if not Path('graphify-out/graph.json').exists(): If it fails, stop and tell the user to run `/graphify ` first. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -1117,7 +1143,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -1128,7 +1154,7 @@ Give a plain-language explanation of a single node - everything connected to it. First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -1138,7 +1164,7 @@ if not Path('graphify-out/graph.json').exists(): If it fails, stop and tell the user to run `/graphify ` first. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -1183,7 +1209,7 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` --- @@ -1193,7 +1219,7 @@ $(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path diff --git a/graphify/skill-amp.md b/graphify/skill-amp.md index 190827d9ac..d5457c16c5 100644 --- a/graphify/skill-amp.md +++ b/graphify/skill-amp.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -285,7 +300,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -309,7 +324,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -323,7 +338,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -356,7 +371,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -391,9 +406,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -414,20 +431,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -455,7 +481,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -478,12 +504,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -514,7 +542,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -554,7 +582,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/graphify/skill-claw.md b/graphify/skill-claw.md index abd2811d23..2c584f2f60 100644 --- a/graphify/skill-claw.md +++ b/graphify/skill-claw.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -288,7 +303,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -312,7 +327,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -326,7 +341,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -359,7 +374,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -394,9 +409,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -417,20 +434,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -458,7 +484,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -481,12 +507,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -517,7 +545,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -557,7 +585,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/graphify/skill-codex.md b/graphify/skill-codex.md index af3f723c78..d7616138ad 100644 --- a/graphify/skill-codex.md +++ b/graphify/skill-codex.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -285,7 +300,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -309,7 +324,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -323,7 +338,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -356,7 +371,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -391,9 +406,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -414,20 +431,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -455,7 +481,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -478,12 +504,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -514,7 +542,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -554,7 +582,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/graphify/skill-copilot.md b/graphify/skill-copilot.md index abd2811d23..2c584f2f60 100644 --- a/graphify/skill-copilot.md +++ b/graphify/skill-copilot.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -288,7 +303,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -312,7 +327,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -326,7 +341,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -359,7 +374,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -394,9 +409,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -417,20 +434,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -458,7 +484,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -481,12 +507,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -517,7 +545,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -557,7 +585,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/graphify/skill-devin.md b/graphify/skill-devin.md index f9be846cbf..a5c1a989af 100644 --- a/graphify/skill-devin.md +++ b/graphify/skill-devin.md @@ -25,6 +25,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --mode deep # thorough extraction, richer INFERRED edges /graphify --update # incremental - re-extract only new/changed files /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -65,6 +67,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) - a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + Follow these steps in order. Do not skip steps. ### Step 1 - Ensure graphify is installed @@ -97,6 +101,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -105,14 +120,14 @@ mkdir -p graphify-out export PYTHONUTF8=1 ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -162,7 +177,7 @@ Set it as `GRAPHIFY_WHISPER_PROMPT` in the environment before running the transc **Step 2 - Transcribe:** ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os from pathlib import Path from graphify.transcribe import transcribe_all @@ -201,7 +216,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -239,7 +254,7 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -339,7 +354,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re After each subagent call completes, write its result to `graphify-out/.graphify_chunk_N.json`. **After each subagent call completes, read the real token counts from the subagent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then merge: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path from graphify.semantic_cleanup import load_validated_semantic_fragment, sanitize_semantic_fragment @@ -368,7 +383,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -382,7 +397,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.semantic_cleanup import sanitize_semantic_fragment @@ -417,7 +432,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path from graphify.semantic_cleanup import sanitize_semantic_fragment @@ -454,9 +469,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source->target), otherwise `False` (the default undirected `Graph`). Substitute it everywhere it appears, the same way you substitute `INPUT_PATH` - do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given, otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -475,19 +492,26 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Persist the graph first and only write the report/analysis if it actually # persisted - to_json refuses to shrink an existing graph.json (#479), and a # report describing a graph we did not write would be a lie (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely - this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (fewer nodes than the existing graph). Run a full rebuild to be safe.') raise SystemExit(1) @@ -512,12 +536,14 @@ Replace INPUT_PATH with the actual path. ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) - there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -547,7 +573,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (fewer nodes than the existing graph). Run a full rebuild to be safe.') print('Report updated with community labels') @@ -564,7 +590,7 @@ Replace INPUT_PATH with the actual path. If `--obsidian` was given: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_obsidian, to_canvas @@ -595,7 +621,7 @@ print(' _COMMUNITY_* - overview notes with cohesion scores and dataview queries Generate the HTML graph (always, unless `--no-viz`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_html @@ -648,7 +674,7 @@ The wiki is an agent-crawlable export — `index.md` plus one article per commun Run this before Step 9 (cleanup) so `graphify-out/.graphify_labels.json` is still available. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.build import build_from_json from graphify.wiki import to_wiki @@ -676,7 +702,7 @@ print(' graphify-out/wiki/index.md -> agent entry point') **If `--neo4j`** - generate a Cypher file for manual import: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_cypher @@ -691,7 +717,7 @@ print('cypher.txt written - import with: cypher-shell < graphify-out/cypher.txt' **If `--neo4j-push `** - push directly to a running Neo4j instance. Ask the user for credentials if not provided: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import push_to_neo4j @@ -712,7 +738,7 @@ Replace `NEO4J_URI`, `NEO4J_USER`, `NEO4J_PASSWORD` with actual values. Default ### Step 7b - SVG export (only if --svg flag) ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_svg @@ -734,7 +760,7 @@ print('graph.svg written - embeds in Obsidian, Notion, GitHub READMEs') ### Step 7c - GraphML export (only if --graphml flag) ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.build import build_from_json from graphify.export import to_graphml @@ -776,7 +802,7 @@ To configure in Claude Desktop, add to `claude_desktop_config.json`: If `total_words` from `graphify-out/.graphify_detect.json` is greater than 5,000, run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.benchmark import run_benchmark, print_benchmark from pathlib import Path @@ -794,7 +820,7 @@ Print the output directly in chat. If `total_words <= 5000`, skip silently - the ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone @@ -898,7 +924,7 @@ fi Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -921,7 +947,7 @@ if new_total > 0: If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -943,7 +969,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -951,10 +977,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` + Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_json @@ -981,7 +1009,7 @@ Then run Steps 4-8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -1004,8 +1032,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- @@ -1014,7 +1041,7 @@ Clean up after: `rm -f graphify-out/.graphify_old.json` Skip Steps 1-3. Load the existing graph from `graphify-out/graph.json` and re-run clustering: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.cluster import cluster, score_all from graphify.analyze import god_nodes, surprising_connections @@ -1069,7 +1096,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -1087,7 +1114,7 @@ Load `graphify-out/graph.json`, then: 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -1178,7 +1205,7 @@ Replace `QUESTION` with the user's actual question, `MODE` with `bfs` or `dfs`, After writing the answer, save it back into the graph so it improves future queries: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` --- @@ -1189,7 +1216,7 @@ Find the shortest path between two named concepts in the graph. First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -1198,7 +1225,7 @@ if not Path('graphify-out/graph.json').exists(): ``` ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -1250,7 +1277,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names. Then explain the pa After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -1261,7 +1288,7 @@ Give a plain-language explanation of a single node - everything connected to it. First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -1270,7 +1297,7 @@ if not Path('graphify-out/graph.json').exists(): ``` ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -1315,7 +1342,7 @@ Replace `NODE_NAME` with the concept. Then write a 3-5 sentence explanation usin After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` --- @@ -1325,7 +1352,7 @@ $(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path diff --git a/graphify/skill-droid.md b/graphify/skill-droid.md index fd148d485d..7f436de524 100644 --- a/graphify/skill-droid.md +++ b/graphify/skill-droid.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -285,7 +300,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -309,7 +324,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -323,7 +338,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -356,7 +371,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -391,9 +406,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -414,20 +431,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -455,7 +481,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -478,12 +504,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -514,7 +542,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -554,7 +582,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/graphify/skill-kilo.md b/graphify/skill-kilo.md index 3e70b050a4..a517457a83 100644 --- a/graphify/skill-kilo.md +++ b/graphify/skill-kilo.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -288,7 +303,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -312,7 +327,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -326,7 +341,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -359,7 +374,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -394,9 +409,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -417,20 +434,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -458,7 +484,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -481,12 +507,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -517,7 +545,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -557,7 +585,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/graphify/skill-kiro.md b/graphify/skill-kiro.md index abd2811d23..2c584f2f60 100644 --- a/graphify/skill-kiro.md +++ b/graphify/skill-kiro.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -288,7 +303,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -312,7 +327,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -326,7 +341,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -359,7 +374,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -394,9 +409,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -417,20 +434,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -458,7 +484,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -481,12 +507,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -517,7 +545,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -557,7 +585,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/graphify/skill-opencode.md b/graphify/skill-opencode.md index 91ced60675..6616bb92df 100644 --- a/graphify/skill-opencode.md +++ b/graphify/skill-opencode.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -280,7 +295,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -304,7 +319,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -318,7 +333,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -351,7 +366,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -386,9 +401,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -409,20 +426,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -450,7 +476,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -473,12 +499,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -509,7 +537,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -549,7 +577,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/graphify/skill-pi.md b/graphify/skill-pi.md index abd2811d23..2c584f2f60 100644 --- a/graphify/skill-pi.md +++ b/graphify/skill-pi.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -288,7 +303,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -312,7 +327,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -326,7 +341,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -359,7 +374,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -394,9 +409,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -417,20 +434,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -458,7 +484,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -481,12 +507,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -517,7 +545,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -557,7 +585,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/graphify/skill-trae.md b/graphify/skill-trae.md index 050667bc20..a4f07e8b2a 100644 --- a/graphify/skill-trae.md +++ b/graphify/skill-trae.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -286,7 +301,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -310,7 +325,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -324,7 +339,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -357,7 +372,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -392,9 +407,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -415,20 +432,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -456,7 +482,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -479,12 +505,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -515,7 +543,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -555,7 +583,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/graphify/skill-vscode.md b/graphify/skill-vscode.md index 20c7c0835c..776d9623fb 100644 --- a/graphify/skill-vscode.md +++ b/graphify/skill-vscode.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -284,7 +299,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -308,7 +323,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -322,7 +337,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -355,7 +370,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -390,9 +405,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -413,20 +430,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -454,7 +480,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -477,12 +503,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -513,7 +541,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -553,7 +581,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/graphify/skill-windows.md b/graphify/skill-windows.md index 764c1914d9..4b3f8a3eb3 100644 --- a/graphify/skill-windows.md +++ b/graphify/skill-windows.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -116,6 +120,17 @@ if (-not $GRAPHIFY_PYTHON) { $GRAPHIFY_PYTHON = Find-GraphifyPython } +# #1619 B4: without this gate, a failed install left $GRAPHIFY_PYTHON $null, +# an empty .graphify_python got written anyway, and every later step then +# failed with a cryptic error far from the real cause instead of a clear one +# here. +if (-not $GRAPHIFY_PYTHON) { + Write-Host "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" + Write-Host " uv tool install graphifyy" + Write-Host " pip install graphifyy" + exit 1 +} + # Save interpreter path — all subsequent steps read this. # `Out-File -Encoding utf8` always writes a BOM on Windows PowerShell 5.1 (utf8NoBOM # only exists from PowerShell 6), and that BOM rides into the saved path, so the hook @@ -124,10 +139,10 @@ if (-not $GRAPHIFY_PYTHON) { $Utf8NoBom = New-Object System.Text.UTF8Encoding $false [System.IO.File]::WriteAllText((Join-Path $PWD 'graphify-out\.graphify_python'), [string]$GRAPHIFY_PYTHON, $Utf8NoBom) # Save scan root so `graphify update` (no args) knows where to look next time -[System.IO.File]::WriteAllText((Join-Path $PWD 'graphify-out\.graphify_root'), (Resolve-Path INPUT_PATH).Path, $Utf8NoBom) +[System.IO.File]::WriteAllText((Join-Path $PWD 'graphify-out\.graphify_root'), (Resolve-Path 'INPUT_PATH').Path, $Utf8NoBom) ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. **In every subsequent block, run Python through the saved interpreter — `& (Get-Content graphify-out\.graphify_python)` in place of a bare `python3` — so every step uses the interpreter that actually has graphify.** @@ -421,6 +436,8 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast["nodes"])} AST + {len(sem **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```powershell New-Item -ItemType Directory -Force -Path graphify-out | Out-Null @' @@ -444,20 +461,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -508,6 +534,8 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: @@ -544,7 +572,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') diff --git a/graphify/skill.md b/graphify/skill.md index abd2811d23..2c584f2f60 100644 --- a/graphify/skill.md +++ b/graphify/skill.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -288,7 +303,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -312,7 +327,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -326,7 +341,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -359,7 +374,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -394,9 +409,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -417,20 +434,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -458,7 +484,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -481,12 +507,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -517,7 +545,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -557,7 +585,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/graphify/skills/agents/references/add-watch.md b/graphify/skills/agents/references/add-watch.md index 77844343e1..937edf327c 100644 --- a/graphify/skills/agents/references/add-watch.md +++ b/graphify/skills/agents/references/add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/graphify/skills/agents/references/exports.md b/graphify/skills/agents/references/exports.md index 242ff868e0..27b031e58c 100644 --- a/graphify/skills/agents/references/exports.md +++ b/graphify/skills/agents/references/exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/graphify/skills/agents/references/query.md b/graphify/skills/agents/references/query.md index 56565eb782..fdc9c68d5f 100644 --- a/graphify/skills/agents/references/query.md +++ b/graphify/skills/agents/references/query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/graphify/skills/agents/references/transcribe.md b/graphify/skills/agents/references/transcribe.md index b967f83799..27064bfcdb 100644 --- a/graphify/skills/agents/references/transcribe.md +++ b/graphify/skills/agents/references/transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/graphify/skills/agents/references/update.md b/graphify/skills/agents/references/update.md index 3632fd4126..a0ca7dd58a 100644 --- a/graphify/skills/agents/references/update.md +++ b/graphify/skills/agents/references/update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/graphify/skills/amp/references/add-watch.md b/graphify/skills/amp/references/add-watch.md index 77844343e1..937edf327c 100644 --- a/graphify/skills/amp/references/add-watch.md +++ b/graphify/skills/amp/references/add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/graphify/skills/amp/references/exports.md b/graphify/skills/amp/references/exports.md index 242ff868e0..27b031e58c 100644 --- a/graphify/skills/amp/references/exports.md +++ b/graphify/skills/amp/references/exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/graphify/skills/amp/references/query.md b/graphify/skills/amp/references/query.md index 56565eb782..fdc9c68d5f 100644 --- a/graphify/skills/amp/references/query.md +++ b/graphify/skills/amp/references/query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/graphify/skills/amp/references/transcribe.md b/graphify/skills/amp/references/transcribe.md index b967f83799..27064bfcdb 100644 --- a/graphify/skills/amp/references/transcribe.md +++ b/graphify/skills/amp/references/transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/graphify/skills/amp/references/update.md b/graphify/skills/amp/references/update.md index 3632fd4126..a0ca7dd58a 100644 --- a/graphify/skills/amp/references/update.md +++ b/graphify/skills/amp/references/update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/graphify/skills/claude/references/add-watch.md b/graphify/skills/claude/references/add-watch.md index 77844343e1..937edf327c 100644 --- a/graphify/skills/claude/references/add-watch.md +++ b/graphify/skills/claude/references/add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/graphify/skills/claude/references/exports.md b/graphify/skills/claude/references/exports.md index 242ff868e0..27b031e58c 100644 --- a/graphify/skills/claude/references/exports.md +++ b/graphify/skills/claude/references/exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/graphify/skills/claude/references/query.md b/graphify/skills/claude/references/query.md index 56565eb782..fdc9c68d5f 100644 --- a/graphify/skills/claude/references/query.md +++ b/graphify/skills/claude/references/query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/graphify/skills/claude/references/transcribe.md b/graphify/skills/claude/references/transcribe.md index b967f83799..27064bfcdb 100644 --- a/graphify/skills/claude/references/transcribe.md +++ b/graphify/skills/claude/references/transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/graphify/skills/claude/references/update.md b/graphify/skills/claude/references/update.md index 3632fd4126..a0ca7dd58a 100644 --- a/graphify/skills/claude/references/update.md +++ b/graphify/skills/claude/references/update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/graphify/skills/claw/references/add-watch.md b/graphify/skills/claw/references/add-watch.md index 77844343e1..937edf327c 100644 --- a/graphify/skills/claw/references/add-watch.md +++ b/graphify/skills/claw/references/add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/graphify/skills/claw/references/exports.md b/graphify/skills/claw/references/exports.md index 242ff868e0..27b031e58c 100644 --- a/graphify/skills/claw/references/exports.md +++ b/graphify/skills/claw/references/exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/graphify/skills/claw/references/query.md b/graphify/skills/claw/references/query.md index 56565eb782..fdc9c68d5f 100644 --- a/graphify/skills/claw/references/query.md +++ b/graphify/skills/claw/references/query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/graphify/skills/claw/references/transcribe.md b/graphify/skills/claw/references/transcribe.md index b967f83799..27064bfcdb 100644 --- a/graphify/skills/claw/references/transcribe.md +++ b/graphify/skills/claw/references/transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/graphify/skills/claw/references/update.md b/graphify/skills/claw/references/update.md index 3632fd4126..a0ca7dd58a 100644 --- a/graphify/skills/claw/references/update.md +++ b/graphify/skills/claw/references/update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/graphify/skills/codex/references/add-watch.md b/graphify/skills/codex/references/add-watch.md index 77844343e1..937edf327c 100644 --- a/graphify/skills/codex/references/add-watch.md +++ b/graphify/skills/codex/references/add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/graphify/skills/codex/references/exports.md b/graphify/skills/codex/references/exports.md index 242ff868e0..27b031e58c 100644 --- a/graphify/skills/codex/references/exports.md +++ b/graphify/skills/codex/references/exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/graphify/skills/codex/references/query.md b/graphify/skills/codex/references/query.md index 56565eb782..fdc9c68d5f 100644 --- a/graphify/skills/codex/references/query.md +++ b/graphify/skills/codex/references/query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/graphify/skills/codex/references/transcribe.md b/graphify/skills/codex/references/transcribe.md index b967f83799..27064bfcdb 100644 --- a/graphify/skills/codex/references/transcribe.md +++ b/graphify/skills/codex/references/transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/graphify/skills/codex/references/update.md b/graphify/skills/codex/references/update.md index 3632fd4126..a0ca7dd58a 100644 --- a/graphify/skills/codex/references/update.md +++ b/graphify/skills/codex/references/update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/graphify/skills/copilot/references/add-watch.md b/graphify/skills/copilot/references/add-watch.md index 77844343e1..937edf327c 100644 --- a/graphify/skills/copilot/references/add-watch.md +++ b/graphify/skills/copilot/references/add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/graphify/skills/copilot/references/exports.md b/graphify/skills/copilot/references/exports.md index 242ff868e0..27b031e58c 100644 --- a/graphify/skills/copilot/references/exports.md +++ b/graphify/skills/copilot/references/exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/graphify/skills/copilot/references/query.md b/graphify/skills/copilot/references/query.md index 56565eb782..fdc9c68d5f 100644 --- a/graphify/skills/copilot/references/query.md +++ b/graphify/skills/copilot/references/query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/graphify/skills/copilot/references/transcribe.md b/graphify/skills/copilot/references/transcribe.md index b967f83799..27064bfcdb 100644 --- a/graphify/skills/copilot/references/transcribe.md +++ b/graphify/skills/copilot/references/transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/graphify/skills/copilot/references/update.md b/graphify/skills/copilot/references/update.md index 3632fd4126..a0ca7dd58a 100644 --- a/graphify/skills/copilot/references/update.md +++ b/graphify/skills/copilot/references/update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/graphify/skills/droid/references/add-watch.md b/graphify/skills/droid/references/add-watch.md index 77844343e1..937edf327c 100644 --- a/graphify/skills/droid/references/add-watch.md +++ b/graphify/skills/droid/references/add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/graphify/skills/droid/references/exports.md b/graphify/skills/droid/references/exports.md index 242ff868e0..27b031e58c 100644 --- a/graphify/skills/droid/references/exports.md +++ b/graphify/skills/droid/references/exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/graphify/skills/droid/references/query.md b/graphify/skills/droid/references/query.md index 56565eb782..fdc9c68d5f 100644 --- a/graphify/skills/droid/references/query.md +++ b/graphify/skills/droid/references/query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/graphify/skills/droid/references/transcribe.md b/graphify/skills/droid/references/transcribe.md index b967f83799..27064bfcdb 100644 --- a/graphify/skills/droid/references/transcribe.md +++ b/graphify/skills/droid/references/transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/graphify/skills/droid/references/update.md b/graphify/skills/droid/references/update.md index 3632fd4126..a0ca7dd58a 100644 --- a/graphify/skills/droid/references/update.md +++ b/graphify/skills/droid/references/update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/graphify/skills/kilo/references/add-watch.md b/graphify/skills/kilo/references/add-watch.md index 77844343e1..937edf327c 100644 --- a/graphify/skills/kilo/references/add-watch.md +++ b/graphify/skills/kilo/references/add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/graphify/skills/kilo/references/exports.md b/graphify/skills/kilo/references/exports.md index 242ff868e0..27b031e58c 100644 --- a/graphify/skills/kilo/references/exports.md +++ b/graphify/skills/kilo/references/exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/graphify/skills/kilo/references/query.md b/graphify/skills/kilo/references/query.md index 56565eb782..fdc9c68d5f 100644 --- a/graphify/skills/kilo/references/query.md +++ b/graphify/skills/kilo/references/query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/graphify/skills/kilo/references/transcribe.md b/graphify/skills/kilo/references/transcribe.md index b967f83799..27064bfcdb 100644 --- a/graphify/skills/kilo/references/transcribe.md +++ b/graphify/skills/kilo/references/transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/graphify/skills/kilo/references/update.md b/graphify/skills/kilo/references/update.md index 3632fd4126..a0ca7dd58a 100644 --- a/graphify/skills/kilo/references/update.md +++ b/graphify/skills/kilo/references/update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/graphify/skills/kiro/references/add-watch.md b/graphify/skills/kiro/references/add-watch.md index 77844343e1..937edf327c 100644 --- a/graphify/skills/kiro/references/add-watch.md +++ b/graphify/skills/kiro/references/add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/graphify/skills/kiro/references/exports.md b/graphify/skills/kiro/references/exports.md index 242ff868e0..27b031e58c 100644 --- a/graphify/skills/kiro/references/exports.md +++ b/graphify/skills/kiro/references/exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/graphify/skills/kiro/references/query.md b/graphify/skills/kiro/references/query.md index 56565eb782..fdc9c68d5f 100644 --- a/graphify/skills/kiro/references/query.md +++ b/graphify/skills/kiro/references/query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/graphify/skills/kiro/references/transcribe.md b/graphify/skills/kiro/references/transcribe.md index b967f83799..27064bfcdb 100644 --- a/graphify/skills/kiro/references/transcribe.md +++ b/graphify/skills/kiro/references/transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/graphify/skills/kiro/references/update.md b/graphify/skills/kiro/references/update.md index 3632fd4126..a0ca7dd58a 100644 --- a/graphify/skills/kiro/references/update.md +++ b/graphify/skills/kiro/references/update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/graphify/skills/opencode/references/add-watch.md b/graphify/skills/opencode/references/add-watch.md index 77844343e1..937edf327c 100644 --- a/graphify/skills/opencode/references/add-watch.md +++ b/graphify/skills/opencode/references/add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/graphify/skills/opencode/references/exports.md b/graphify/skills/opencode/references/exports.md index 242ff868e0..27b031e58c 100644 --- a/graphify/skills/opencode/references/exports.md +++ b/graphify/skills/opencode/references/exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/graphify/skills/opencode/references/query.md b/graphify/skills/opencode/references/query.md index 56565eb782..fdc9c68d5f 100644 --- a/graphify/skills/opencode/references/query.md +++ b/graphify/skills/opencode/references/query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/graphify/skills/opencode/references/transcribe.md b/graphify/skills/opencode/references/transcribe.md index b967f83799..27064bfcdb 100644 --- a/graphify/skills/opencode/references/transcribe.md +++ b/graphify/skills/opencode/references/transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/graphify/skills/opencode/references/update.md b/graphify/skills/opencode/references/update.md index 3632fd4126..a0ca7dd58a 100644 --- a/graphify/skills/opencode/references/update.md +++ b/graphify/skills/opencode/references/update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/graphify/skills/pi/references/add-watch.md b/graphify/skills/pi/references/add-watch.md index 77844343e1..937edf327c 100644 --- a/graphify/skills/pi/references/add-watch.md +++ b/graphify/skills/pi/references/add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/graphify/skills/pi/references/exports.md b/graphify/skills/pi/references/exports.md index 242ff868e0..27b031e58c 100644 --- a/graphify/skills/pi/references/exports.md +++ b/graphify/skills/pi/references/exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/graphify/skills/pi/references/query.md b/graphify/skills/pi/references/query.md index 56565eb782..fdc9c68d5f 100644 --- a/graphify/skills/pi/references/query.md +++ b/graphify/skills/pi/references/query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/graphify/skills/pi/references/transcribe.md b/graphify/skills/pi/references/transcribe.md index b967f83799..27064bfcdb 100644 --- a/graphify/skills/pi/references/transcribe.md +++ b/graphify/skills/pi/references/transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/graphify/skills/pi/references/update.md b/graphify/skills/pi/references/update.md index 3632fd4126..a0ca7dd58a 100644 --- a/graphify/skills/pi/references/update.md +++ b/graphify/skills/pi/references/update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/graphify/skills/trae/references/add-watch.md b/graphify/skills/trae/references/add-watch.md index 77844343e1..937edf327c 100644 --- a/graphify/skills/trae/references/add-watch.md +++ b/graphify/skills/trae/references/add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/graphify/skills/trae/references/exports.md b/graphify/skills/trae/references/exports.md index 242ff868e0..27b031e58c 100644 --- a/graphify/skills/trae/references/exports.md +++ b/graphify/skills/trae/references/exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/graphify/skills/trae/references/query.md b/graphify/skills/trae/references/query.md index 56565eb782..fdc9c68d5f 100644 --- a/graphify/skills/trae/references/query.md +++ b/graphify/skills/trae/references/query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/graphify/skills/trae/references/transcribe.md b/graphify/skills/trae/references/transcribe.md index b967f83799..27064bfcdb 100644 --- a/graphify/skills/trae/references/transcribe.md +++ b/graphify/skills/trae/references/transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/graphify/skills/trae/references/update.md b/graphify/skills/trae/references/update.md index 3632fd4126..a0ca7dd58a 100644 --- a/graphify/skills/trae/references/update.md +++ b/graphify/skills/trae/references/update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/graphify/skills/vscode/references/add-watch.md b/graphify/skills/vscode/references/add-watch.md index 77844343e1..937edf327c 100644 --- a/graphify/skills/vscode/references/add-watch.md +++ b/graphify/skills/vscode/references/add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/graphify/skills/vscode/references/exports.md b/graphify/skills/vscode/references/exports.md index 242ff868e0..27b031e58c 100644 --- a/graphify/skills/vscode/references/exports.md +++ b/graphify/skills/vscode/references/exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/graphify/skills/vscode/references/query.md b/graphify/skills/vscode/references/query.md index 56565eb782..fdc9c68d5f 100644 --- a/graphify/skills/vscode/references/query.md +++ b/graphify/skills/vscode/references/query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/graphify/skills/vscode/references/transcribe.md b/graphify/skills/vscode/references/transcribe.md index b967f83799..27064bfcdb 100644 --- a/graphify/skills/vscode/references/transcribe.md +++ b/graphify/skills/vscode/references/transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/graphify/skills/vscode/references/update.md b/graphify/skills/vscode/references/update.md index 3632fd4126..a0ca7dd58a 100644 --- a/graphify/skills/vscode/references/update.md +++ b/graphify/skills/vscode/references/update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/graphify/skills/windows/references/add-watch.md b/graphify/skills/windows/references/add-watch.md index 77844343e1..937edf327c 100644 --- a/graphify/skills/windows/references/add-watch.md +++ b/graphify/skills/windows/references/add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/graphify/skills/windows/references/exports.md b/graphify/skills/windows/references/exports.md index 242ff868e0..27b031e58c 100644 --- a/graphify/skills/windows/references/exports.md +++ b/graphify/skills/windows/references/exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/graphify/skills/windows/references/query.md b/graphify/skills/windows/references/query.md index 56565eb782..fdc9c68d5f 100644 --- a/graphify/skills/windows/references/query.md +++ b/graphify/skills/windows/references/query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/graphify/skills/windows/references/transcribe.md b/graphify/skills/windows/references/transcribe.md index b967f83799..27064bfcdb 100644 --- a/graphify/skills/windows/references/transcribe.md +++ b/graphify/skills/windows/references/transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/graphify/skills/windows/references/update.md b/graphify/skills/windows/references/update.md index 3632fd4126..a0ca7dd58a 100644 --- a/graphify/skills/windows/references/update.md +++ b/graphify/skills/windows/references/update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/tests/test_community_labels_skill.py b/tests/test_community_labels_skill.py index 5184190e94..dc4b702171 100644 --- a/tests/test_community_labels_skill.py +++ b/tests/test_community_labels_skill.py @@ -101,7 +101,7 @@ def test_skill_step5_reexports_graph_json_with_curated_labels(path: Path): ] assert post_labels_blocks, f"{path.name}: no Step-5 (LABELS_DICT) code block found" for block in post_labels_blocks: - assert "to_json(G, communities, 'graphify-out/graph.json', community_labels=labels)" in block, ( + assert "to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE)" in block, ( f"{path.name}: the post-labels Step-5 block must re-export " f"graphify-out/graph.json with community_labels=labels (#2490)" ) @@ -125,7 +125,7 @@ def test_core_fragments_step5_reexport_with_curated_labels(fragment: Path): ] assert post_labels_blocks, f"{fragment.name}: no Step-5 (LABELS_DICT) code block found" for block in post_labels_blocks: - assert "to_json(G, communities, 'graphify-out/graph.json', community_labels=labels)" in block, ( + assert "to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE)" in block, ( f"{fragment.name}: the post-labels Step-5 block must re-export " f"graphify-out/graph.json with community_labels=labels (#2490)" ) diff --git a/tests/test_devin.py b/tests/test_devin.py index a3bca5d05b..e73e650db2 100644 --- a/tests/test_devin.py +++ b/tests/test_devin.py @@ -240,14 +240,15 @@ def test_devin_skill_file_uses_python_c_syntax(): """Devin skill must use inline python -c syntax (cross-platform, no bash heredocs). All mature graphify skills use the interpreter-detection pattern - ``$(cat graphify-out/.graphify_python) -c "..."`` rather than bare - ``python -c "..."`` so they work in pipx / venv environments. + ``"$(cat graphify-out/.graphify_python)" -c "..."`` rather than bare + ``python -c "..."`` so they work in pipx / venv environments. Quoted (#1619 + B5) so an interpreter path containing a space does not word-split. """ import graphify skill = (Path(graphify.__file__).parent / "skill-devin.md").read_text() - assert '.graphify_python) -c "' in skill, ( + assert '.graphify_python)" -c "' in skill, ( "skill-devin.md must use the interpreter-detection pattern " - "'$(cat graphify-out/.graphify_python) -c \"...\"'" + "'\"$(cat graphify-out/.graphify_python)\" -c \"...\"'" ) assert "#!/bin/bash" not in skill diff --git a/tests/test_skillgen.py b/tests/test_skillgen.py index ab6f94474c..0d748b40b4 100644 --- a/tests/test_skillgen.py +++ b/tests/test_skillgen.py @@ -416,7 +416,7 @@ def test_windows_and_posix_cores_have_step_and_2490_parity(): for heading in _STEP_HEADINGS: assert heading in claude_core, f"skill.md lost step heading: {heading!r}" assert heading in windows_core, f"skill-windows.md lost step heading: {heading!r}" - line_2490 = "to_json(G, communities, 'graphify-out/graph.json', community_labels=labels)" + line_2490 = "to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE)" assert line_2490 in claude_core, "skill.md lost the #2490 Step-5 re-export" assert line_2490 in windows_core, "skill-windows.md lost the #2490 Step-5 re-export" @@ -651,6 +651,181 @@ def test_monoliths_carry_the_1392_runbook_fixes(): assert "if not wrote:" in body +def test_no_cluster_and_force_flags_are_wired_through(): + """#1619 C1/C2: --no-cluster and --force were documented/suggested but never + implemented -- cluster() ran unconditionally and to_json() never received + force=. Both are now wired through Step 4/5, for the core render and both + hand-maintained monoliths. + """ + claude_core, _ = _platform_artifacts("claude") + platforms = gen.load_platforms() + bodies = {"claude": claude_core} + for key in ("aider", "devin"): + bodies[key] = gen.render(platforms[key])[0].content + + for key, body in bodies.items(): + assert "--no-cluster" in body.split("## Usage")[1].split("```")[1], ( + f"[{key}] --no-cluster missing from Usage" + ) + assert "--force " in body.split("## Usage")[1].split("```")[1], ( + f"[{key}] --force missing from Usage" + ) + assert "if IS_NO_CLUSTER:" in body, f"[{key}] Step 4 does not branch on IS_NO_CLUSTER" + assert "communities = {0: list(G.nodes())}" in body, ( + f"[{key}] Step 4 missing the flat --no-cluster community shortcut" + ) + assert "force=IS_FORCE" in body, f"[{key}] to_json is not wired to IS_FORCE" + # Both to_json(...graph.json...) calls in the main build path take it - + # Step 4's first write and Step 5's curated-labels re-export. + assert body.count("force=IS_FORCE") >= 2, ( + f"[{key}] expected force=IS_FORCE on both Step 4 and Step 5 to_json calls" + ) + assert "Skip this step entirely if `--no-cluster` was given in Step 4" in body, ( + f"[{key}] Step 5 does not document skipping labeling for --no-cluster" + ) + + +def test_no_cluster_step_4_write_carries_the_full_corpus_label(): + """Review finding on #1619: Step 5 (which normally supplies real community + labels) is skipped entirely for --no-cluster, making Step 4's write the + only one -- so it must pass community_labels=labels itself, or every node + in a --no-cluster build silently loses its community_name even though the + 'Full Corpus' label was computed right above it.""" + claude_core, _ = _platform_artifacts("claude") + platforms = gen.load_platforms() + bodies = {"claude": claude_core} + for key in ("aider", "devin"): + bodies[key] = gen.render(platforms[key])[0].content + + for key, body in bodies.items(): + # Step 4 and Step 5 now both use this exact call shape -- Step 5's + # own occurrence alone would make a bare "in body" check pass even + # if Step 4's copy were still missing community_labels, so this + # counts occurrences the same way the neighboring force=IS_FORCE + # assertion above does. + step4_or_5_write = "to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE)" + assert body.count(step4_or_5_write) >= 2, ( + f"[{key}] expected community_labels=labels on both Step 4 and Step 5 " + "to_json calls -- Step 4's is the only write for --no-cluster, since " + "Step 5 is skipped for that path" + ) + + +def test_input_path_forward_slash_guidance_is_present(): + """#1619 B1: a Windows INPUT_PATH substitution with backslashes corrupts + the Python string literal it's spliced into. Every host must tell the + agent to substitute forward slashes instead, and the PowerShell + Resolve-Path call must be quoted so a path with a space in it survives.""" + claude_core, _ = _platform_artifacts("claude") + windows_core, _ = _platform_artifacts("windows") + platforms = gen.load_platforms() + bodies = {"claude": claude_core, "windows": windows_core} + for key in ("aider", "devin"): + bodies[key] = gen.render(platforms[key])[0].content + + for key, body in bodies.items(): + assert "substitute it with forward slashes" in body.lower(), ( + f"[{key}] missing the forward-slash INPUT_PATH guidance" + ) + + assert "(Resolve-Path 'INPUT_PATH')" in bodies["windows"], ( + "skill-windows.md's Resolve-Path call must be quoted (#1619 B1)" + ) + assert "(Resolve-Path INPUT_PATH)" not in bodies["windows"], ( + "the unquoted Resolve-Path call must not survive" + ) + + +def test_step1_gates_on_a_still_failed_install(): + """#1619 B4: a failed install must stop with an actionable error instead of + silently writing a broken interpreter path that fails every later step + with a cryptic error far from the real cause. + """ + claude_core, _ = _platform_artifacts("claude") + windows_core, _ = _platform_artifacts("windows") + platforms = gen.load_platforms() + bodies = {"claude": claude_core, "windows": windows_core} + for key in ("aider", "devin"): + bodies[key] = gen.render(platforms[key])[0].content + + for key, body in bodies.items(): + assert "could not install or locate a Python interpreter with graphify" in body, ( + f"[{key}] missing the Step 1 failure-gate error message" + ) + lines = body.splitlines() + gate_i = next( + i for i, l in enumerate(lines) + if "could not install or locate a Python interpreter" in l + ) + write_i = next( + i for i, l in enumerate(lines) + if ".graphify_python" in l and ("write(" in l or "WriteAllText" in l) + ) + # the gate fires after the install attempt, before the (possibly still + # broken) interpreter path is persisted for every later step to read. + assert gate_i < write_i, f"[{key}] Step 1 gate does not precede the interpreter-path write" + + +def test_update_backup_instruction_precedes_the_merge_it_backs_up(): + """#1619 C4: "save the old graph" must appear BEFORE the merge block and + the diff block that consumes it. An agent reading top to bottom that + reaches the merge before being told to back up never creates + .graphify_old.json, so the post-update diff's `if old_data:` silently + no-ops on every run instead of ever showing a diff. + """ + _, claude_refs = _platform_artifacts("claude") + update_body = claude_refs["update.md"] + platforms = gen.load_platforms() + bodies = {"claude (references/update.md)": update_body} + for key in ("aider", "devin"): + bodies[key] = gen.render(platforms[key])[0].content + + for key, body in bodies.items(): + lines = body.splitlines() + backup_i = next( + i for i, l in enumerate(lines) if l.strip().startswith("Before the merge step") + ) + merge_i = next( + i for i, l in enumerate(lines) + if "build_merge(" in l or "G_existing.update(G_new)" in l + ) + diff_i = next(i for i, l in enumerate(lines) if "old_data" in l and "Path(" in l) + assert backup_i < merge_i < diff_i, ( + f"[{key}] backup instruction must precede both the merge and the diff " + f"that reads the backup (backup={backup_i}, merge={merge_i}, diff={diff_i})" + ) + + +def test_interpreter_cat_substitution_is_quoted_everywhere(): + """#1619 B5: `$(cat graphify-out/.graphify_python)` names the interpreter to + run and must be quoted, like the `"$PYTHON"` form Step 1 already uses -- + an unquoted interpreter path containing a space (a venv under + `C:\\Users\\First Last\\...`) would otherwise word-split into multiple + arguments and fail to exec. + """ + claude_core, claude_refs = _platform_artifacts("claude") + platforms = gen.load_platforms() + bodies = {"claude": claude_core, **{f"claude:{k}": v for k, v in claude_refs.items()}} + for key in ("aider", "devin"): + bodies[key] = gen.render(platforms[key])[0].content + + found_any = False + for key, body in bodies.items(): + if "$(cat graphify-out/.graphify_python)" not in body: + continue + found_any = True + assert '"$(cat graphify-out/.graphify_python)"' in body, ( + f"[{key}] has the interpreter substitution but not the quoted form" + ) + # every occurrence must be the quoted form -- an unquoted survivor would + # still contain the bare substring outside any quoted instance. + unquoted = body.replace('"$(cat graphify-out/.graphify_python)"', "") + assert "$(cat graphify-out/.graphify_python)" not in unquoted, ( + f"[{key}] an unquoted $(cat ...) interpreter substitution survived" + ) + assert found_any, "no host's rendered output referenced the interpreter substitution at all" + + def test_monoliths_scope_semantic_cache_writes_to_uncached_files(): """#1757: generated monoliths pass the dispatched-file allowlist when replacing semantic cache entries.""" diff --git a/tools/skillgen/expected/graphify__skill-agents.md b/tools/skillgen/expected/graphify__skill-agents.md index 190827d9ac..d5457c16c5 100644 --- a/tools/skillgen/expected/graphify__skill-agents.md +++ b/tools/skillgen/expected/graphify__skill-agents.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -285,7 +300,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -309,7 +324,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -323,7 +338,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -356,7 +371,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -391,9 +406,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -414,20 +431,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -455,7 +481,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -478,12 +504,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -514,7 +542,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -554,7 +582,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/tools/skillgen/expected/graphify__skill-aider.md b/tools/skillgen/expected/graphify__skill-aider.md index 4996beb787..f920ea703d 100644 --- a/tools/skillgen/expected/graphify__skill-aider.md +++ b/tools/skillgen/expected/graphify__skill-aider.md @@ -15,6 +15,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --mode deep # thorough extraction, richer INFERRED edges /graphify --update # incremental - re-extract only new/changed files /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) - a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + Follow these steps in order. Do not skip steps. ### Step 1 - Ensure graphify is installed @@ -86,20 +90,31 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out "$PYTHON" -c "import sys; open('graphify-out/.graphify_python', 'w', encoding='utf-8').write(sys.executable)" ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -149,7 +164,7 @@ Set it as `GRAPHIFY_WHISPER_PROMPT` in the environment before running the transc **Step 2 - Transcribe:** ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os from pathlib import Path from graphify.transcribe import transcribe_all @@ -188,7 +203,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -222,7 +237,7 @@ Print: `"Semantic extraction: N files (sequential — Aider)"` Before dispatching any subagents, check which files already have cached extraction results: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -283,7 +298,7 @@ If more than half the chunks failed, stop and tell the user. Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -307,7 +322,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -321,7 +336,7 @@ print(f'Cached {saved} files') Merge cached + new results into `.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -354,7 +369,7 @@ Clean up temp files: `rm -f .graphify_cached.json .graphify_uncached.txt .graphi #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -389,9 +404,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source->target), otherwise `False` (the default undirected `Graph`). Substitute it everywhere it appears, the same way you substitute `INPUT_PATH` - do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given, otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -410,19 +427,26 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Persist the graph first and only write the report/analysis if it actually # persisted - to_json refuses to shrink an existing graph.json (#479), and a # report describing a graph we did not write would be a lie (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely - this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (fewer nodes than the existing graph). Run a full rebuild to be safe.') raise SystemExit(1) @@ -447,12 +471,14 @@ Replace INPUT_PATH with the actual path. ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) - there is only one placeholder community ("Full Corpus"), nothing to label. + Read `.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -482,7 +508,7 @@ Path('.graphify_labels.json').write_text(json.dumps({str(k): v for k, v in label # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (fewer nodes than the existing graph). Run a full rebuild to be safe.') print('Report updated with community labels') @@ -499,7 +525,7 @@ Replace INPUT_PATH with the actual path. If `--obsidian` was given: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_obsidian, to_canvas @@ -530,7 +556,7 @@ print(' _COMMUNITY_* - overview notes with cohesion scores and dataview queries Generate the HTML graph (always, unless `--no-viz`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_html @@ -557,7 +583,7 @@ else: **If `--neo4j`** - generate a Cypher file for manual import: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_cypher @@ -572,7 +598,7 @@ print('cypher.txt written - import with: cypher-shell < graphify-out/cypher.txt' **If `--neo4j-push `** - push directly to a running Neo4j instance. Ask the user for credentials if not provided: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster @@ -594,7 +620,7 @@ Replace `NEO4J_URI`, `NEO4J_USER`, `NEO4J_PASSWORD` with actual values. Default ### Step 7b - SVG export (only if --svg flag) ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_svg @@ -616,7 +642,7 @@ print('graph.svg written - embeds in Obsidian, Notion, GitHub READMEs') ### Step 7c - GraphML export (only if --graphml flag) ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.build import build_from_json from graphify.export import to_graphml @@ -658,7 +684,7 @@ To configure in Claude Desktop, add to `claude_desktop_config.json`: If `total_words` from `.graphify_detect.json` is greater than 5,000, run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.benchmark import run_benchmark, print_benchmark from pathlib import Path @@ -676,7 +702,7 @@ Print the output directly in chat. If `total_words <= 5000`, skip silently - the ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone @@ -759,7 +785,7 @@ The graph is the map. Your job after the pipeline is to be the guide. Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -782,7 +808,7 @@ if new_total > 0: If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -805,7 +831,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -813,11 +839,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json .graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_json @@ -844,7 +871,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -868,8 +895,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json .graphify_old.json` -Clean up after: `rm -f .graphify_old.json` +Clean up the backup after: `rm -f .graphify_old.json` --- @@ -878,7 +904,7 @@ Clean up after: `rm -f .graphify_old.json` Skip Steps 1–3. Load the existing graph from `graphify-out/graph.json` and re-run clustering: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.cluster import cluster, score_all from graphify.analyze import god_nodes, surprising_connections @@ -933,7 +959,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -951,7 +977,7 @@ Load `graphify-out/graph.json`, then: 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -1042,7 +1068,7 @@ Replace `QUESTION` with the user's actual question, `MODE` with `bfs` or `dfs`, After writing the answer, save it back into the graph so it improves future queries: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `QUESTION` with the question, `ANSWER` with your full answer text, `SOURCE_NODES` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -1055,7 +1081,7 @@ Find the shortest path between two named concepts in the graph. First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -1065,7 +1091,7 @@ if not Path('graphify-out/graph.json').exists(): If it fails, stop and tell the user to run `/graphify ` first. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -1117,7 +1143,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -1128,7 +1154,7 @@ Give a plain-language explanation of a single node - everything connected to it. First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -1138,7 +1164,7 @@ if not Path('graphify-out/graph.json').exists(): If it fails, stop and tell the user to run `/graphify ` first. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -1183,7 +1209,7 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` --- @@ -1193,7 +1219,7 @@ $(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path diff --git a/tools/skillgen/expected/graphify__skill-amp.md b/tools/skillgen/expected/graphify__skill-amp.md index 190827d9ac..d5457c16c5 100644 --- a/tools/skillgen/expected/graphify__skill-amp.md +++ b/tools/skillgen/expected/graphify__skill-amp.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -285,7 +300,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -309,7 +324,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -323,7 +338,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -356,7 +371,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -391,9 +406,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -414,20 +431,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -455,7 +481,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -478,12 +504,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -514,7 +542,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -554,7 +582,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/tools/skillgen/expected/graphify__skill-claw.md b/tools/skillgen/expected/graphify__skill-claw.md index abd2811d23..2c584f2f60 100644 --- a/tools/skillgen/expected/graphify__skill-claw.md +++ b/tools/skillgen/expected/graphify__skill-claw.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -288,7 +303,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -312,7 +327,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -326,7 +341,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -359,7 +374,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -394,9 +409,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -417,20 +434,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -458,7 +484,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -481,12 +507,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -517,7 +545,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -557,7 +585,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/tools/skillgen/expected/graphify__skill-codex.md b/tools/skillgen/expected/graphify__skill-codex.md index af3f723c78..d7616138ad 100644 --- a/tools/skillgen/expected/graphify__skill-codex.md +++ b/tools/skillgen/expected/graphify__skill-codex.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -285,7 +300,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -309,7 +324,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -323,7 +338,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -356,7 +371,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -391,9 +406,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -414,20 +431,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -455,7 +481,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -478,12 +504,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -514,7 +542,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -554,7 +582,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/tools/skillgen/expected/graphify__skill-copilot.md b/tools/skillgen/expected/graphify__skill-copilot.md index abd2811d23..2c584f2f60 100644 --- a/tools/skillgen/expected/graphify__skill-copilot.md +++ b/tools/skillgen/expected/graphify__skill-copilot.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -288,7 +303,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -312,7 +327,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -326,7 +341,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -359,7 +374,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -394,9 +409,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -417,20 +434,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -458,7 +484,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -481,12 +507,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -517,7 +545,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -557,7 +585,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/tools/skillgen/expected/graphify__skill-devin.md b/tools/skillgen/expected/graphify__skill-devin.md index f9be846cbf..a5c1a989af 100644 --- a/tools/skillgen/expected/graphify__skill-devin.md +++ b/tools/skillgen/expected/graphify__skill-devin.md @@ -25,6 +25,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --mode deep # thorough extraction, richer INFERRED edges /graphify --update # incremental - re-extract only new/changed files /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -65,6 +67,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) - a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + Follow these steps in order. Do not skip steps. ### Step 1 - Ensure graphify is installed @@ -97,6 +101,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -105,14 +120,14 @@ mkdir -p graphify-out export PYTHONUTF8=1 ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -162,7 +177,7 @@ Set it as `GRAPHIFY_WHISPER_PROMPT` in the environment before running the transc **Step 2 - Transcribe:** ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os from pathlib import Path from graphify.transcribe import transcribe_all @@ -201,7 +216,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -239,7 +254,7 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -339,7 +354,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re After each subagent call completes, write its result to `graphify-out/.graphify_chunk_N.json`. **After each subagent call completes, read the real token counts from the subagent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then merge: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path from graphify.semantic_cleanup import load_validated_semantic_fragment, sanitize_semantic_fragment @@ -368,7 +383,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -382,7 +397,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.semantic_cleanup import sanitize_semantic_fragment @@ -417,7 +432,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path from graphify.semantic_cleanup import sanitize_semantic_fragment @@ -454,9 +469,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source->target), otherwise `False` (the default undirected `Graph`). Substitute it everywhere it appears, the same way you substitute `INPUT_PATH` - do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given, otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -475,19 +492,26 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Persist the graph first and only write the report/analysis if it actually # persisted - to_json refuses to shrink an existing graph.json (#479), and a # report describing a graph we did not write would be a lie (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely - this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (fewer nodes than the existing graph). Run a full rebuild to be safe.') raise SystemExit(1) @@ -512,12 +536,14 @@ Replace INPUT_PATH with the actual path. ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) - there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -547,7 +573,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (fewer nodes than the existing graph). Run a full rebuild to be safe.') print('Report updated with community labels') @@ -564,7 +590,7 @@ Replace INPUT_PATH with the actual path. If `--obsidian` was given: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_obsidian, to_canvas @@ -595,7 +621,7 @@ print(' _COMMUNITY_* - overview notes with cohesion scores and dataview queries Generate the HTML graph (always, unless `--no-viz`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_html @@ -648,7 +674,7 @@ The wiki is an agent-crawlable export — `index.md` plus one article per commun Run this before Step 9 (cleanup) so `graphify-out/.graphify_labels.json` is still available. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.build import build_from_json from graphify.wiki import to_wiki @@ -676,7 +702,7 @@ print(' graphify-out/wiki/index.md -> agent entry point') **If `--neo4j`** - generate a Cypher file for manual import: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_cypher @@ -691,7 +717,7 @@ print('cypher.txt written - import with: cypher-shell < graphify-out/cypher.txt' **If `--neo4j-push `** - push directly to a running Neo4j instance. Ask the user for credentials if not provided: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import push_to_neo4j @@ -712,7 +738,7 @@ Replace `NEO4J_URI`, `NEO4J_USER`, `NEO4J_PASSWORD` with actual values. Default ### Step 7b - SVG export (only if --svg flag) ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_svg @@ -734,7 +760,7 @@ print('graph.svg written - embeds in Obsidian, Notion, GitHub READMEs') ### Step 7c - GraphML export (only if --graphml flag) ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.build import build_from_json from graphify.export import to_graphml @@ -776,7 +802,7 @@ To configure in Claude Desktop, add to `claude_desktop_config.json`: If `total_words` from `graphify-out/.graphify_detect.json` is greater than 5,000, run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.benchmark import run_benchmark, print_benchmark from pathlib import Path @@ -794,7 +820,7 @@ Print the output directly in chat. If `total_words <= 5000`, skip silently - the ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone @@ -898,7 +924,7 @@ fi Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -921,7 +947,7 @@ if new_total > 0: If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -943,7 +969,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -951,10 +977,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` + Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_json @@ -981,7 +1009,7 @@ Then run Steps 4-8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -1004,8 +1032,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- @@ -1014,7 +1041,7 @@ Clean up after: `rm -f graphify-out/.graphify_old.json` Skip Steps 1-3. Load the existing graph from `graphify-out/graph.json` and re-run clustering: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.cluster import cluster, score_all from graphify.analyze import god_nodes, surprising_connections @@ -1069,7 +1096,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -1087,7 +1114,7 @@ Load `graphify-out/graph.json`, then: 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -1178,7 +1205,7 @@ Replace `QUESTION` with the user's actual question, `MODE` with `bfs` or `dfs`, After writing the answer, save it back into the graph so it improves future queries: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` --- @@ -1189,7 +1216,7 @@ Find the shortest path between two named concepts in the graph. First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -1198,7 +1225,7 @@ if not Path('graphify-out/graph.json').exists(): ``` ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -1250,7 +1277,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names. Then explain the pa After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -1261,7 +1288,7 @@ Give a plain-language explanation of a single node - everything connected to it. First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -1270,7 +1297,7 @@ if not Path('graphify-out/graph.json').exists(): ``` ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -1315,7 +1342,7 @@ Replace `NODE_NAME` with the concept. Then write a 3-5 sentence explanation usin After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` --- @@ -1325,7 +1352,7 @@ $(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path diff --git a/tools/skillgen/expected/graphify__skill-droid.md b/tools/skillgen/expected/graphify__skill-droid.md index fd148d485d..7f436de524 100644 --- a/tools/skillgen/expected/graphify__skill-droid.md +++ b/tools/skillgen/expected/graphify__skill-droid.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -285,7 +300,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -309,7 +324,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -323,7 +338,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -356,7 +371,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -391,9 +406,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -414,20 +431,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -455,7 +481,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -478,12 +504,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -514,7 +542,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -554,7 +582,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/tools/skillgen/expected/graphify__skill-kilo.md b/tools/skillgen/expected/graphify__skill-kilo.md index 3e70b050a4..a517457a83 100644 --- a/tools/skillgen/expected/graphify__skill-kilo.md +++ b/tools/skillgen/expected/graphify__skill-kilo.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -288,7 +303,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -312,7 +327,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -326,7 +341,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -359,7 +374,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -394,9 +409,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -417,20 +434,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -458,7 +484,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -481,12 +507,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -517,7 +545,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -557,7 +585,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/tools/skillgen/expected/graphify__skill-kiro.md b/tools/skillgen/expected/graphify__skill-kiro.md index abd2811d23..2c584f2f60 100644 --- a/tools/skillgen/expected/graphify__skill-kiro.md +++ b/tools/skillgen/expected/graphify__skill-kiro.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -288,7 +303,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -312,7 +327,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -326,7 +341,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -359,7 +374,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -394,9 +409,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -417,20 +434,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -458,7 +484,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -481,12 +507,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -517,7 +545,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -557,7 +585,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/tools/skillgen/expected/graphify__skill-opencode.md b/tools/skillgen/expected/graphify__skill-opencode.md index 91ced60675..6616bb92df 100644 --- a/tools/skillgen/expected/graphify__skill-opencode.md +++ b/tools/skillgen/expected/graphify__skill-opencode.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -280,7 +295,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -304,7 +319,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -318,7 +333,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -351,7 +366,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -386,9 +401,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -409,20 +426,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -450,7 +476,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -473,12 +499,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -509,7 +537,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -549,7 +577,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/tools/skillgen/expected/graphify__skill-pi.md b/tools/skillgen/expected/graphify__skill-pi.md index abd2811d23..2c584f2f60 100644 --- a/tools/skillgen/expected/graphify__skill-pi.md +++ b/tools/skillgen/expected/graphify__skill-pi.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -288,7 +303,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -312,7 +327,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -326,7 +341,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -359,7 +374,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -394,9 +409,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -417,20 +434,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -458,7 +484,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -481,12 +507,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -517,7 +545,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -557,7 +585,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/tools/skillgen/expected/graphify__skill-trae.md b/tools/skillgen/expected/graphify__skill-trae.md index 050667bc20..a4f07e8b2a 100644 --- a/tools/skillgen/expected/graphify__skill-trae.md +++ b/tools/skillgen/expected/graphify__skill-trae.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -286,7 +301,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -310,7 +325,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -324,7 +339,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -357,7 +372,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -392,9 +407,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -415,20 +432,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -456,7 +482,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -479,12 +505,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -515,7 +543,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -555,7 +583,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/tools/skillgen/expected/graphify__skill-vscode.md b/tools/skillgen/expected/graphify__skill-vscode.md index 20c7c0835c..776d9623fb 100644 --- a/tools/skillgen/expected/graphify__skill-vscode.md +++ b/tools/skillgen/expected/graphify__skill-vscode.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -284,7 +299,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -308,7 +323,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -322,7 +337,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -355,7 +370,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -390,9 +405,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -413,20 +430,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -454,7 +480,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -477,12 +503,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -513,7 +541,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -553,7 +581,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/tools/skillgen/expected/graphify__skill-windows.md b/tools/skillgen/expected/graphify__skill-windows.md index 764c1914d9..4b3f8a3eb3 100644 --- a/tools/skillgen/expected/graphify__skill-windows.md +++ b/tools/skillgen/expected/graphify__skill-windows.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -116,6 +120,17 @@ if (-not $GRAPHIFY_PYTHON) { $GRAPHIFY_PYTHON = Find-GraphifyPython } +# #1619 B4: without this gate, a failed install left $GRAPHIFY_PYTHON $null, +# an empty .graphify_python got written anyway, and every later step then +# failed with a cryptic error far from the real cause instead of a clear one +# here. +if (-not $GRAPHIFY_PYTHON) { + Write-Host "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" + Write-Host " uv tool install graphifyy" + Write-Host " pip install graphifyy" + exit 1 +} + # Save interpreter path — all subsequent steps read this. # `Out-File -Encoding utf8` always writes a BOM on Windows PowerShell 5.1 (utf8NoBOM # only exists from PowerShell 6), and that BOM rides into the saved path, so the hook @@ -124,10 +139,10 @@ if (-not $GRAPHIFY_PYTHON) { $Utf8NoBom = New-Object System.Text.UTF8Encoding $false [System.IO.File]::WriteAllText((Join-Path $PWD 'graphify-out\.graphify_python'), [string]$GRAPHIFY_PYTHON, $Utf8NoBom) # Save scan root so `graphify update` (no args) knows where to look next time -[System.IO.File]::WriteAllText((Join-Path $PWD 'graphify-out\.graphify_root'), (Resolve-Path INPUT_PATH).Path, $Utf8NoBom) +[System.IO.File]::WriteAllText((Join-Path $PWD 'graphify-out\.graphify_root'), (Resolve-Path 'INPUT_PATH').Path, $Utf8NoBom) ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. **In every subsequent block, run Python through the saved interpreter — `& (Get-Content graphify-out\.graphify_python)` in place of a bare `python3` — so every step uses the interpreter that actually has graphify.** @@ -421,6 +436,8 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast["nodes"])} AST + {len(sem **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```powershell New-Item -ItemType Directory -Force -Path graphify-out | Out-Null @' @@ -444,20 +461,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -508,6 +534,8 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: @@ -544,7 +572,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') diff --git a/tools/skillgen/expected/graphify__skill.md b/tools/skillgen/expected/graphify__skill.md index abd2811d23..2c584f2f60 100644 --- a/tools/skillgen/expected/graphify__skill.md +++ b/tools/skillgen/expected/graphify__skill.md @@ -20,6 +20,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -92,6 +96,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -100,14 +115,14 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -172,7 +187,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -198,7 +213,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -220,7 +235,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -288,7 +303,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -312,7 +327,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -326,7 +341,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -359,7 +374,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -394,9 +409,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -417,20 +434,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -458,7 +484,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -481,12 +507,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -517,7 +545,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -557,7 +585,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/tools/skillgen/expected/graphify__skills__agents__references__add-watch.md b/tools/skillgen/expected/graphify__skills__agents__references__add-watch.md index 77844343e1..937edf327c 100644 --- a/tools/skillgen/expected/graphify__skills__agents__references__add-watch.md +++ b/tools/skillgen/expected/graphify__skills__agents__references__add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/tools/skillgen/expected/graphify__skills__agents__references__exports.md b/tools/skillgen/expected/graphify__skills__agents__references__exports.md index 242ff868e0..27b031e58c 100644 --- a/tools/skillgen/expected/graphify__skills__agents__references__exports.md +++ b/tools/skillgen/expected/graphify__skills__agents__references__exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/tools/skillgen/expected/graphify__skills__agents__references__query.md b/tools/skillgen/expected/graphify__skills__agents__references__query.md index 56565eb782..fdc9c68d5f 100644 --- a/tools/skillgen/expected/graphify__skills__agents__references__query.md +++ b/tools/skillgen/expected/graphify__skills__agents__references__query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/tools/skillgen/expected/graphify__skills__agents__references__transcribe.md b/tools/skillgen/expected/graphify__skills__agents__references__transcribe.md index b967f83799..27064bfcdb 100644 --- a/tools/skillgen/expected/graphify__skills__agents__references__transcribe.md +++ b/tools/skillgen/expected/graphify__skills__agents__references__transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/tools/skillgen/expected/graphify__skills__agents__references__update.md b/tools/skillgen/expected/graphify__skills__agents__references__update.md index 3632fd4126..a0ca7dd58a 100644 --- a/tools/skillgen/expected/graphify__skills__agents__references__update.md +++ b/tools/skillgen/expected/graphify__skills__agents__references__update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/tools/skillgen/expected/graphify__skills__amp__references__add-watch.md b/tools/skillgen/expected/graphify__skills__amp__references__add-watch.md index 77844343e1..937edf327c 100644 --- a/tools/skillgen/expected/graphify__skills__amp__references__add-watch.md +++ b/tools/skillgen/expected/graphify__skills__amp__references__add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/tools/skillgen/expected/graphify__skills__amp__references__exports.md b/tools/skillgen/expected/graphify__skills__amp__references__exports.md index 242ff868e0..27b031e58c 100644 --- a/tools/skillgen/expected/graphify__skills__amp__references__exports.md +++ b/tools/skillgen/expected/graphify__skills__amp__references__exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/tools/skillgen/expected/graphify__skills__amp__references__query.md b/tools/skillgen/expected/graphify__skills__amp__references__query.md index 56565eb782..fdc9c68d5f 100644 --- a/tools/skillgen/expected/graphify__skills__amp__references__query.md +++ b/tools/skillgen/expected/graphify__skills__amp__references__query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/tools/skillgen/expected/graphify__skills__amp__references__transcribe.md b/tools/skillgen/expected/graphify__skills__amp__references__transcribe.md index b967f83799..27064bfcdb 100644 --- a/tools/skillgen/expected/graphify__skills__amp__references__transcribe.md +++ b/tools/skillgen/expected/graphify__skills__amp__references__transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/tools/skillgen/expected/graphify__skills__amp__references__update.md b/tools/skillgen/expected/graphify__skills__amp__references__update.md index 3632fd4126..a0ca7dd58a 100644 --- a/tools/skillgen/expected/graphify__skills__amp__references__update.md +++ b/tools/skillgen/expected/graphify__skills__amp__references__update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/tools/skillgen/expected/graphify__skills__claude__references__add-watch.md b/tools/skillgen/expected/graphify__skills__claude__references__add-watch.md index 77844343e1..937edf327c 100644 --- a/tools/skillgen/expected/graphify__skills__claude__references__add-watch.md +++ b/tools/skillgen/expected/graphify__skills__claude__references__add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/tools/skillgen/expected/graphify__skills__claude__references__exports.md b/tools/skillgen/expected/graphify__skills__claude__references__exports.md index 242ff868e0..27b031e58c 100644 --- a/tools/skillgen/expected/graphify__skills__claude__references__exports.md +++ b/tools/skillgen/expected/graphify__skills__claude__references__exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/tools/skillgen/expected/graphify__skills__claude__references__query.md b/tools/skillgen/expected/graphify__skills__claude__references__query.md index 56565eb782..fdc9c68d5f 100644 --- a/tools/skillgen/expected/graphify__skills__claude__references__query.md +++ b/tools/skillgen/expected/graphify__skills__claude__references__query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/tools/skillgen/expected/graphify__skills__claude__references__transcribe.md b/tools/skillgen/expected/graphify__skills__claude__references__transcribe.md index b967f83799..27064bfcdb 100644 --- a/tools/skillgen/expected/graphify__skills__claude__references__transcribe.md +++ b/tools/skillgen/expected/graphify__skills__claude__references__transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/tools/skillgen/expected/graphify__skills__claude__references__update.md b/tools/skillgen/expected/graphify__skills__claude__references__update.md index 3632fd4126..a0ca7dd58a 100644 --- a/tools/skillgen/expected/graphify__skills__claude__references__update.md +++ b/tools/skillgen/expected/graphify__skills__claude__references__update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/tools/skillgen/expected/graphify__skills__claw__references__add-watch.md b/tools/skillgen/expected/graphify__skills__claw__references__add-watch.md index 77844343e1..937edf327c 100644 --- a/tools/skillgen/expected/graphify__skills__claw__references__add-watch.md +++ b/tools/skillgen/expected/graphify__skills__claw__references__add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/tools/skillgen/expected/graphify__skills__claw__references__exports.md b/tools/skillgen/expected/graphify__skills__claw__references__exports.md index 242ff868e0..27b031e58c 100644 --- a/tools/skillgen/expected/graphify__skills__claw__references__exports.md +++ b/tools/skillgen/expected/graphify__skills__claw__references__exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/tools/skillgen/expected/graphify__skills__claw__references__query.md b/tools/skillgen/expected/graphify__skills__claw__references__query.md index 56565eb782..fdc9c68d5f 100644 --- a/tools/skillgen/expected/graphify__skills__claw__references__query.md +++ b/tools/skillgen/expected/graphify__skills__claw__references__query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/tools/skillgen/expected/graphify__skills__claw__references__transcribe.md b/tools/skillgen/expected/graphify__skills__claw__references__transcribe.md index b967f83799..27064bfcdb 100644 --- a/tools/skillgen/expected/graphify__skills__claw__references__transcribe.md +++ b/tools/skillgen/expected/graphify__skills__claw__references__transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/tools/skillgen/expected/graphify__skills__claw__references__update.md b/tools/skillgen/expected/graphify__skills__claw__references__update.md index 3632fd4126..a0ca7dd58a 100644 --- a/tools/skillgen/expected/graphify__skills__claw__references__update.md +++ b/tools/skillgen/expected/graphify__skills__claw__references__update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/tools/skillgen/expected/graphify__skills__codex__references__add-watch.md b/tools/skillgen/expected/graphify__skills__codex__references__add-watch.md index 77844343e1..937edf327c 100644 --- a/tools/skillgen/expected/graphify__skills__codex__references__add-watch.md +++ b/tools/skillgen/expected/graphify__skills__codex__references__add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/tools/skillgen/expected/graphify__skills__codex__references__exports.md b/tools/skillgen/expected/graphify__skills__codex__references__exports.md index 242ff868e0..27b031e58c 100644 --- a/tools/skillgen/expected/graphify__skills__codex__references__exports.md +++ b/tools/skillgen/expected/graphify__skills__codex__references__exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/tools/skillgen/expected/graphify__skills__codex__references__query.md b/tools/skillgen/expected/graphify__skills__codex__references__query.md index 56565eb782..fdc9c68d5f 100644 --- a/tools/skillgen/expected/graphify__skills__codex__references__query.md +++ b/tools/skillgen/expected/graphify__skills__codex__references__query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/tools/skillgen/expected/graphify__skills__codex__references__transcribe.md b/tools/skillgen/expected/graphify__skills__codex__references__transcribe.md index b967f83799..27064bfcdb 100644 --- a/tools/skillgen/expected/graphify__skills__codex__references__transcribe.md +++ b/tools/skillgen/expected/graphify__skills__codex__references__transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/tools/skillgen/expected/graphify__skills__codex__references__update.md b/tools/skillgen/expected/graphify__skills__codex__references__update.md index 3632fd4126..a0ca7dd58a 100644 --- a/tools/skillgen/expected/graphify__skills__codex__references__update.md +++ b/tools/skillgen/expected/graphify__skills__codex__references__update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/tools/skillgen/expected/graphify__skills__copilot__references__add-watch.md b/tools/skillgen/expected/graphify__skills__copilot__references__add-watch.md index 77844343e1..937edf327c 100644 --- a/tools/skillgen/expected/graphify__skills__copilot__references__add-watch.md +++ b/tools/skillgen/expected/graphify__skills__copilot__references__add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/tools/skillgen/expected/graphify__skills__copilot__references__exports.md b/tools/skillgen/expected/graphify__skills__copilot__references__exports.md index 242ff868e0..27b031e58c 100644 --- a/tools/skillgen/expected/graphify__skills__copilot__references__exports.md +++ b/tools/skillgen/expected/graphify__skills__copilot__references__exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/tools/skillgen/expected/graphify__skills__copilot__references__query.md b/tools/skillgen/expected/graphify__skills__copilot__references__query.md index 56565eb782..fdc9c68d5f 100644 --- a/tools/skillgen/expected/graphify__skills__copilot__references__query.md +++ b/tools/skillgen/expected/graphify__skills__copilot__references__query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/tools/skillgen/expected/graphify__skills__copilot__references__transcribe.md b/tools/skillgen/expected/graphify__skills__copilot__references__transcribe.md index b967f83799..27064bfcdb 100644 --- a/tools/skillgen/expected/graphify__skills__copilot__references__transcribe.md +++ b/tools/skillgen/expected/graphify__skills__copilot__references__transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/tools/skillgen/expected/graphify__skills__copilot__references__update.md b/tools/skillgen/expected/graphify__skills__copilot__references__update.md index 3632fd4126..a0ca7dd58a 100644 --- a/tools/skillgen/expected/graphify__skills__copilot__references__update.md +++ b/tools/skillgen/expected/graphify__skills__copilot__references__update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/tools/skillgen/expected/graphify__skills__droid__references__add-watch.md b/tools/skillgen/expected/graphify__skills__droid__references__add-watch.md index 77844343e1..937edf327c 100644 --- a/tools/skillgen/expected/graphify__skills__droid__references__add-watch.md +++ b/tools/skillgen/expected/graphify__skills__droid__references__add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/tools/skillgen/expected/graphify__skills__droid__references__exports.md b/tools/skillgen/expected/graphify__skills__droid__references__exports.md index 242ff868e0..27b031e58c 100644 --- a/tools/skillgen/expected/graphify__skills__droid__references__exports.md +++ b/tools/skillgen/expected/graphify__skills__droid__references__exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/tools/skillgen/expected/graphify__skills__droid__references__query.md b/tools/skillgen/expected/graphify__skills__droid__references__query.md index 56565eb782..fdc9c68d5f 100644 --- a/tools/skillgen/expected/graphify__skills__droid__references__query.md +++ b/tools/skillgen/expected/graphify__skills__droid__references__query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/tools/skillgen/expected/graphify__skills__droid__references__transcribe.md b/tools/skillgen/expected/graphify__skills__droid__references__transcribe.md index b967f83799..27064bfcdb 100644 --- a/tools/skillgen/expected/graphify__skills__droid__references__transcribe.md +++ b/tools/skillgen/expected/graphify__skills__droid__references__transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/tools/skillgen/expected/graphify__skills__droid__references__update.md b/tools/skillgen/expected/graphify__skills__droid__references__update.md index 3632fd4126..a0ca7dd58a 100644 --- a/tools/skillgen/expected/graphify__skills__droid__references__update.md +++ b/tools/skillgen/expected/graphify__skills__droid__references__update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/tools/skillgen/expected/graphify__skills__kilo__references__add-watch.md b/tools/skillgen/expected/graphify__skills__kilo__references__add-watch.md index 77844343e1..937edf327c 100644 --- a/tools/skillgen/expected/graphify__skills__kilo__references__add-watch.md +++ b/tools/skillgen/expected/graphify__skills__kilo__references__add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/tools/skillgen/expected/graphify__skills__kilo__references__exports.md b/tools/skillgen/expected/graphify__skills__kilo__references__exports.md index 242ff868e0..27b031e58c 100644 --- a/tools/skillgen/expected/graphify__skills__kilo__references__exports.md +++ b/tools/skillgen/expected/graphify__skills__kilo__references__exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/tools/skillgen/expected/graphify__skills__kilo__references__query.md b/tools/skillgen/expected/graphify__skills__kilo__references__query.md index 56565eb782..fdc9c68d5f 100644 --- a/tools/skillgen/expected/graphify__skills__kilo__references__query.md +++ b/tools/skillgen/expected/graphify__skills__kilo__references__query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/tools/skillgen/expected/graphify__skills__kilo__references__transcribe.md b/tools/skillgen/expected/graphify__skills__kilo__references__transcribe.md index b967f83799..27064bfcdb 100644 --- a/tools/skillgen/expected/graphify__skills__kilo__references__transcribe.md +++ b/tools/skillgen/expected/graphify__skills__kilo__references__transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/tools/skillgen/expected/graphify__skills__kilo__references__update.md b/tools/skillgen/expected/graphify__skills__kilo__references__update.md index 3632fd4126..a0ca7dd58a 100644 --- a/tools/skillgen/expected/graphify__skills__kilo__references__update.md +++ b/tools/skillgen/expected/graphify__skills__kilo__references__update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/tools/skillgen/expected/graphify__skills__kiro__references__add-watch.md b/tools/skillgen/expected/graphify__skills__kiro__references__add-watch.md index 77844343e1..937edf327c 100644 --- a/tools/skillgen/expected/graphify__skills__kiro__references__add-watch.md +++ b/tools/skillgen/expected/graphify__skills__kiro__references__add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/tools/skillgen/expected/graphify__skills__kiro__references__exports.md b/tools/skillgen/expected/graphify__skills__kiro__references__exports.md index 242ff868e0..27b031e58c 100644 --- a/tools/skillgen/expected/graphify__skills__kiro__references__exports.md +++ b/tools/skillgen/expected/graphify__skills__kiro__references__exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/tools/skillgen/expected/graphify__skills__kiro__references__query.md b/tools/skillgen/expected/graphify__skills__kiro__references__query.md index 56565eb782..fdc9c68d5f 100644 --- a/tools/skillgen/expected/graphify__skills__kiro__references__query.md +++ b/tools/skillgen/expected/graphify__skills__kiro__references__query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/tools/skillgen/expected/graphify__skills__kiro__references__transcribe.md b/tools/skillgen/expected/graphify__skills__kiro__references__transcribe.md index b967f83799..27064bfcdb 100644 --- a/tools/skillgen/expected/graphify__skills__kiro__references__transcribe.md +++ b/tools/skillgen/expected/graphify__skills__kiro__references__transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/tools/skillgen/expected/graphify__skills__kiro__references__update.md b/tools/skillgen/expected/graphify__skills__kiro__references__update.md index 3632fd4126..a0ca7dd58a 100644 --- a/tools/skillgen/expected/graphify__skills__kiro__references__update.md +++ b/tools/skillgen/expected/graphify__skills__kiro__references__update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/tools/skillgen/expected/graphify__skills__opencode__references__add-watch.md b/tools/skillgen/expected/graphify__skills__opencode__references__add-watch.md index 77844343e1..937edf327c 100644 --- a/tools/skillgen/expected/graphify__skills__opencode__references__add-watch.md +++ b/tools/skillgen/expected/graphify__skills__opencode__references__add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/tools/skillgen/expected/graphify__skills__opencode__references__exports.md b/tools/skillgen/expected/graphify__skills__opencode__references__exports.md index 242ff868e0..27b031e58c 100644 --- a/tools/skillgen/expected/graphify__skills__opencode__references__exports.md +++ b/tools/skillgen/expected/graphify__skills__opencode__references__exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/tools/skillgen/expected/graphify__skills__opencode__references__query.md b/tools/skillgen/expected/graphify__skills__opencode__references__query.md index 56565eb782..fdc9c68d5f 100644 --- a/tools/skillgen/expected/graphify__skills__opencode__references__query.md +++ b/tools/skillgen/expected/graphify__skills__opencode__references__query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/tools/skillgen/expected/graphify__skills__opencode__references__transcribe.md b/tools/skillgen/expected/graphify__skills__opencode__references__transcribe.md index b967f83799..27064bfcdb 100644 --- a/tools/skillgen/expected/graphify__skills__opencode__references__transcribe.md +++ b/tools/skillgen/expected/graphify__skills__opencode__references__transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/tools/skillgen/expected/graphify__skills__opencode__references__update.md b/tools/skillgen/expected/graphify__skills__opencode__references__update.md index 3632fd4126..a0ca7dd58a 100644 --- a/tools/skillgen/expected/graphify__skills__opencode__references__update.md +++ b/tools/skillgen/expected/graphify__skills__opencode__references__update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/tools/skillgen/expected/graphify__skills__pi__references__add-watch.md b/tools/skillgen/expected/graphify__skills__pi__references__add-watch.md index 77844343e1..937edf327c 100644 --- a/tools/skillgen/expected/graphify__skills__pi__references__add-watch.md +++ b/tools/skillgen/expected/graphify__skills__pi__references__add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/tools/skillgen/expected/graphify__skills__pi__references__exports.md b/tools/skillgen/expected/graphify__skills__pi__references__exports.md index 242ff868e0..27b031e58c 100644 --- a/tools/skillgen/expected/graphify__skills__pi__references__exports.md +++ b/tools/skillgen/expected/graphify__skills__pi__references__exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/tools/skillgen/expected/graphify__skills__pi__references__query.md b/tools/skillgen/expected/graphify__skills__pi__references__query.md index 56565eb782..fdc9c68d5f 100644 --- a/tools/skillgen/expected/graphify__skills__pi__references__query.md +++ b/tools/skillgen/expected/graphify__skills__pi__references__query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/tools/skillgen/expected/graphify__skills__pi__references__transcribe.md b/tools/skillgen/expected/graphify__skills__pi__references__transcribe.md index b967f83799..27064bfcdb 100644 --- a/tools/skillgen/expected/graphify__skills__pi__references__transcribe.md +++ b/tools/skillgen/expected/graphify__skills__pi__references__transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/tools/skillgen/expected/graphify__skills__pi__references__update.md b/tools/skillgen/expected/graphify__skills__pi__references__update.md index 3632fd4126..a0ca7dd58a 100644 --- a/tools/skillgen/expected/graphify__skills__pi__references__update.md +++ b/tools/skillgen/expected/graphify__skills__pi__references__update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/tools/skillgen/expected/graphify__skills__trae__references__add-watch.md b/tools/skillgen/expected/graphify__skills__trae__references__add-watch.md index 77844343e1..937edf327c 100644 --- a/tools/skillgen/expected/graphify__skills__trae__references__add-watch.md +++ b/tools/skillgen/expected/graphify__skills__trae__references__add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/tools/skillgen/expected/graphify__skills__trae__references__exports.md b/tools/skillgen/expected/graphify__skills__trae__references__exports.md index 242ff868e0..27b031e58c 100644 --- a/tools/skillgen/expected/graphify__skills__trae__references__exports.md +++ b/tools/skillgen/expected/graphify__skills__trae__references__exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/tools/skillgen/expected/graphify__skills__trae__references__query.md b/tools/skillgen/expected/graphify__skills__trae__references__query.md index 56565eb782..fdc9c68d5f 100644 --- a/tools/skillgen/expected/graphify__skills__trae__references__query.md +++ b/tools/skillgen/expected/graphify__skills__trae__references__query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/tools/skillgen/expected/graphify__skills__trae__references__transcribe.md b/tools/skillgen/expected/graphify__skills__trae__references__transcribe.md index b967f83799..27064bfcdb 100644 --- a/tools/skillgen/expected/graphify__skills__trae__references__transcribe.md +++ b/tools/skillgen/expected/graphify__skills__trae__references__transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/tools/skillgen/expected/graphify__skills__trae__references__update.md b/tools/skillgen/expected/graphify__skills__trae__references__update.md index 3632fd4126..a0ca7dd58a 100644 --- a/tools/skillgen/expected/graphify__skills__trae__references__update.md +++ b/tools/skillgen/expected/graphify__skills__trae__references__update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/tools/skillgen/expected/graphify__skills__vscode__references__add-watch.md b/tools/skillgen/expected/graphify__skills__vscode__references__add-watch.md index 77844343e1..937edf327c 100644 --- a/tools/skillgen/expected/graphify__skills__vscode__references__add-watch.md +++ b/tools/skillgen/expected/graphify__skills__vscode__references__add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/tools/skillgen/expected/graphify__skills__vscode__references__exports.md b/tools/skillgen/expected/graphify__skills__vscode__references__exports.md index 242ff868e0..27b031e58c 100644 --- a/tools/skillgen/expected/graphify__skills__vscode__references__exports.md +++ b/tools/skillgen/expected/graphify__skills__vscode__references__exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/tools/skillgen/expected/graphify__skills__vscode__references__query.md b/tools/skillgen/expected/graphify__skills__vscode__references__query.md index 56565eb782..fdc9c68d5f 100644 --- a/tools/skillgen/expected/graphify__skills__vscode__references__query.md +++ b/tools/skillgen/expected/graphify__skills__vscode__references__query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/tools/skillgen/expected/graphify__skills__vscode__references__transcribe.md b/tools/skillgen/expected/graphify__skills__vscode__references__transcribe.md index b967f83799..27064bfcdb 100644 --- a/tools/skillgen/expected/graphify__skills__vscode__references__transcribe.md +++ b/tools/skillgen/expected/graphify__skills__vscode__references__transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/tools/skillgen/expected/graphify__skills__vscode__references__update.md b/tools/skillgen/expected/graphify__skills__vscode__references__update.md index 3632fd4126..a0ca7dd58a 100644 --- a/tools/skillgen/expected/graphify__skills__vscode__references__update.md +++ b/tools/skillgen/expected/graphify__skills__vscode__references__update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/tools/skillgen/expected/graphify__skills__windows__references__add-watch.md b/tools/skillgen/expected/graphify__skills__windows__references__add-watch.md index 77844343e1..937edf327c 100644 --- a/tools/skillgen/expected/graphify__skills__windows__references__add-watch.md +++ b/tools/skillgen/expected/graphify__skills__windows__references__add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/tools/skillgen/expected/graphify__skills__windows__references__exports.md b/tools/skillgen/expected/graphify__skills__windows__references__exports.md index 242ff868e0..27b031e58c 100644 --- a/tools/skillgen/expected/graphify__skills__windows__references__exports.md +++ b/tools/skillgen/expected/graphify__skills__windows__references__exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/tools/skillgen/expected/graphify__skills__windows__references__query.md b/tools/skillgen/expected/graphify__skills__windows__references__query.md index 56565eb782..fdc9c68d5f 100644 --- a/tools/skillgen/expected/graphify__skills__windows__references__query.md +++ b/tools/skillgen/expected/graphify__skills__windows__references__query.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/tools/skillgen/expected/graphify__skills__windows__references__transcribe.md b/tools/skillgen/expected/graphify__skills__windows__references__transcribe.md index b967f83799..27064bfcdb 100644 --- a/tools/skillgen/expected/graphify__skills__windows__references__transcribe.md +++ b/tools/skillgen/expected/graphify__skills__windows__references__transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/tools/skillgen/expected/graphify__skills__windows__references__update.md b/tools/skillgen/expected/graphify__skills__windows__references__update.md index 3632fd4126..a0ca7dd58a 100644 --- a/tools/skillgen/expected/graphify__skills__windows__references__update.md +++ b/tools/skillgen/expected/graphify__skills__windows__references__update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/tools/skillgen/fragments/core/aider.md b/tools/skillgen/fragments/core/aider.md index 4996beb787..f920ea703d 100644 --- a/tools/skillgen/fragments/core/aider.md +++ b/tools/skillgen/fragments/core/aider.md @@ -15,6 +15,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --mode deep # thorough extraction, richer INFERRED edges /graphify --update # incremental - re-extract only new/changed files /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -54,6 +56,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) - a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + Follow these steps in order. Do not skip steps. ### Step 1 - Ensure graphify is installed @@ -86,20 +90,31 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out "$PYTHON" -c "import sys; open('graphify-out/.graphify_python', 'w', encoding='utf-8').write(sys.executable)" ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -149,7 +164,7 @@ Set it as `GRAPHIFY_WHISPER_PROMPT` in the environment before running the transc **Step 2 - Transcribe:** ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os from pathlib import Path from graphify.transcribe import transcribe_all @@ -188,7 +203,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -222,7 +237,7 @@ Print: `"Semantic extraction: N files (sequential — Aider)"` Before dispatching any subagents, check which files already have cached extraction results: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -283,7 +298,7 @@ If more than half the chunks failed, stop and tell the user. Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -307,7 +322,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -321,7 +336,7 @@ print(f'Cached {saved} files') Merge cached + new results into `.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -354,7 +369,7 @@ Clean up temp files: `rm -f .graphify_cached.json .graphify_uncached.txt .graphi #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -389,9 +404,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source->target), otherwise `False` (the default undirected `Graph`). Substitute it everywhere it appears, the same way you substitute `INPUT_PATH` - do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given, otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -410,19 +427,26 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Persist the graph first and only write the report/analysis if it actually # persisted - to_json refuses to shrink an existing graph.json (#479), and a # report describing a graph we did not write would be a lie (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely - this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (fewer nodes than the existing graph). Run a full rebuild to be safe.') raise SystemExit(1) @@ -447,12 +471,14 @@ Replace INPUT_PATH with the actual path. ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) - there is only one placeholder community ("Full Corpus"), nothing to label. + Read `.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -482,7 +508,7 @@ Path('.graphify_labels.json').write_text(json.dumps({str(k): v for k, v in label # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (fewer nodes than the existing graph). Run a full rebuild to be safe.') print('Report updated with community labels') @@ -499,7 +525,7 @@ Replace INPUT_PATH with the actual path. If `--obsidian` was given: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_obsidian, to_canvas @@ -530,7 +556,7 @@ print(' _COMMUNITY_* - overview notes with cohesion scores and dataview queries Generate the HTML graph (always, unless `--no-viz`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_html @@ -557,7 +583,7 @@ else: **If `--neo4j`** - generate a Cypher file for manual import: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_cypher @@ -572,7 +598,7 @@ print('cypher.txt written - import with: cypher-shell < graphify-out/cypher.txt' **If `--neo4j-push `** - push directly to a running Neo4j instance. Ask the user for credentials if not provided: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster @@ -594,7 +620,7 @@ Replace `NEO4J_URI`, `NEO4J_USER`, `NEO4J_PASSWORD` with actual values. Default ### Step 7b - SVG export (only if --svg flag) ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_svg @@ -616,7 +642,7 @@ print('graph.svg written - embeds in Obsidian, Notion, GitHub READMEs') ### Step 7c - GraphML export (only if --graphml flag) ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.build import build_from_json from graphify.export import to_graphml @@ -658,7 +684,7 @@ To configure in Claude Desktop, add to `claude_desktop_config.json`: If `total_words` from `.graphify_detect.json` is greater than 5,000, run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.benchmark import run_benchmark, print_benchmark from pathlib import Path @@ -676,7 +702,7 @@ Print the output directly in chat. If `total_words <= 5000`, skip silently - the ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone @@ -759,7 +785,7 @@ The graph is the map. Your job after the pipeline is to be the guide. Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -782,7 +808,7 @@ if new_total > 0: If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -805,7 +831,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -813,11 +839,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json .graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_json @@ -844,7 +871,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -868,8 +895,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json .graphify_old.json` -Clean up after: `rm -f .graphify_old.json` +Clean up the backup after: `rm -f .graphify_old.json` --- @@ -878,7 +904,7 @@ Clean up after: `rm -f .graphify_old.json` Skip Steps 1–3. Load the existing graph from `graphify-out/graph.json` and re-run clustering: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.cluster import cluster, score_all from graphify.analyze import god_nodes, surprising_connections @@ -933,7 +959,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -951,7 +977,7 @@ Load `graphify-out/graph.json`, then: 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -1042,7 +1068,7 @@ Replace `QUESTION` with the user's actual question, `MODE` with `bfs` or `dfs`, After writing the answer, save it back into the graph so it improves future queries: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `QUESTION` with the question, `ANSWER` with your full answer text, `SOURCE_NODES` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -1055,7 +1081,7 @@ Find the shortest path between two named concepts in the graph. First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -1065,7 +1091,7 @@ if not Path('graphify-out/graph.json').exists(): If it fails, stop and tell the user to run `/graphify ` first. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -1117,7 +1143,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -1128,7 +1154,7 @@ Give a plain-language explanation of a single node - everything connected to it. First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -1138,7 +1164,7 @@ if not Path('graphify-out/graph.json').exists(): If it fails, stop and tell the user to run `/graphify ` first. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -1183,7 +1209,7 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` --- @@ -1193,7 +1219,7 @@ $(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path diff --git a/tools/skillgen/fragments/core/core.md b/tools/skillgen/fragments/core/core.md index c527a12563..890e2972e1 100644 --- a/tools/skillgen/fragments/core/core.md +++ b/tools/skillgen/fragments/core/core.md @@ -17,6 +17,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --directed # build directed graph (preserves edge direction: source→target) /graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -51,6 +53,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) — a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. Follow these steps in order. Do not skip steps. @@ -66,7 +70,7 @@ Only when the path is one or more `https://github.com/...` URLs, or several loca ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -131,7 +135,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -157,7 +161,7 @@ else: **Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -179,7 +183,7 @@ Before dispatching any subagents, check which files already have cached extracti SPEC_PATH below is the **absolute** path of the `references/extraction-spec.md` that ships beside this SKILL.md — the same file Step B2 loads and hands to every subagent. It is the extraction prompt, so cache entries are attributed to it: when a graphify upgrade changes the prompt, entries produced by the old one are re-extracted instead of replayed, and unchanged prompts keep their entries (#1939). Substitute the real path in both Step B0 and Step B3 — pass the same one to each, and do not drop the argument. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -223,7 +227,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path @@ -247,7 +251,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache. Pass the same SPEC_PATH as Step B0 — it stamps each entry with the prompt that produced it, and a write under a different prompt than the read lands where the next run won't look (#1939): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -261,7 +265,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -294,7 +298,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path @@ -329,9 +333,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given (explicitly, or accepted after Step 2 suggested it for a flat corpus), otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -352,20 +358,29 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + # Skip the expensive clustering step entirely - one placeholder community + # covering every node, per Step 2's flat-corpus suggestion. + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Export FIRST and honor the #479 shrink-guard: to_json returns False (writing # nothing) when the new graph is smaller than the existing graph.json. Only write # GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so # they never describe a graph that graph.json doesn't contain (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely -- this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -393,7 +408,7 @@ Replace INPUT_PATH with the actual path. A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.diagnostics import diagnose_extraction, format_diagnostic_report @@ -416,12 +431,14 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) — there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -452,7 +469,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') print('If this shrink is intentional (you deleted files), re-run a full build with --force.') @@ -492,7 +509,7 @@ These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, ` ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone diff --git a/tools/skillgen/fragments/core/devin.md b/tools/skillgen/fragments/core/devin.md index f9be846cbf..a5c1a989af 100644 --- a/tools/skillgen/fragments/core/devin.md +++ b/tools/skillgen/fragments/core/devin.md @@ -25,6 +25,8 @@ Turn any folder of files into a navigable knowledge graph with community detecti /graphify --mode deep # thorough extraction, richer INFERRED edges /graphify --update # incremental - re-extract only new/changed files /graphify --cluster-only # rerun clustering on existing graph +/graphify --no-cluster # skip clustering, one "Full Corpus" community (flat/small corpora) +/graphify --force # allow the rebuild to shrink graph.json (e.g. after deleting files) /graphify --no-viz # skip visualization, just report + JSON /graphify --html # (HTML is generated by default - this flag is a no-op) /graphify --svg # also export graph.svg (embeds in Notion, GitHub) @@ -65,6 +67,8 @@ If the user invoked `/graphify --help` or `/graphify -h` (with no other argument If no path was given, use `.` (current directory). Do not ask the user for a path. +Every occurrence of `INPUT_PATH` below is a placeholder substituted with this resolved path, inside a Python string literal. On Windows, substitute it with forward slashes (`C:/Users/me/project`, not `C:\Users\me\project`) - a literal backslash in a Windows path splices a stray escape into the Python source (`\t` becomes a tab, `\U` raises a `SyntaxError`), silently or loudly corrupting every block that uses it. + Follow these steps in order. Do not skip steps. ### Step 1 - Ensure graphify is installed @@ -97,6 +101,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -105,14 +120,14 @@ mkdir -p graphify-out export PYTHONUTF8=1 ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** ### Step 2 - Detect files ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.detect import detect from pathlib import Path @@ -162,7 +177,7 @@ Set it as `GRAPHIFY_WHISPER_PROMPT` in the environment before running the transc **Step 2 - Transcribe:** ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os from pathlib import Path from graphify.transcribe import transcribe_all @@ -201,7 +216,7 @@ Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is determin For any code files detected, run AST extraction in parallel with Part B subagents: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.extract import collect_files, extract from pathlib import Path @@ -239,7 +254,7 @@ Before dispatching subagents, print a timing estimate: Before dispatching any subagents, check which files already have cached extraction results: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import check_semantic_cache from pathlib import Path @@ -339,7 +354,7 @@ If more than half the chunks failed or are missing, stop and tell the user to re After each subagent call completes, write its result to `graphify-out/.graphify_chunk_N.json`. **After each subagent call completes, read the real token counts from the subagent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then merge: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, glob from pathlib import Path from graphify.semantic_cleanup import load_validated_semantic_fragment, sanitize_semantic_fragment @@ -368,7 +383,7 @@ print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens' Save new results to cache: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.cache import save_semantic_cache from pathlib import Path @@ -382,7 +397,7 @@ print(f'Cached {saved} files') Merge cached + new results into `graphify-out/.graphify_semantic.json`: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.semantic_cleanup import sanitize_semantic_fragment @@ -417,7 +432,7 @@ Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.gra #### Part C - Merge AST + semantic into final extraction ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from pathlib import Path from graphify.semantic_cleanup import sanitize_semantic_fragment @@ -454,9 +469,11 @@ print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(s **Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source->target), otherwise `False` (the default undirected `Graph`). Substitute it everywhere it appears, the same way you substitute `INPUT_PATH` - do not leave the literal `IS_DIRECTED` in the code. +Two more substitutions, in this step's block and Step 5's: replace `IS_NO_CLUSTER` with `True` if `--no-cluster` was given, otherwise `False`. Replace `IS_FORCE` with `True` if `--force` was given, otherwise `False`. + ```bash mkdir -p graphify-out -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import cluster, score_all @@ -475,19 +492,26 @@ if G.number_of_nodes() == 0: print('ERROR: Graph is empty - extraction produced no nodes.') print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') raise SystemExit(1) -communities = cluster(G) +if IS_NO_CLUSTER: + communities = {0: list(G.nodes())} +else: + communities = cluster(G) cohesion = score_all(G, communities) tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} gods = god_nodes(G) surprises = surprising_connections(G, communities) -labels = {cid: 'Community ' + str(cid) for cid in communities} -# Placeholder questions - regenerated with real labels in Step 5 +labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 (skipped for --no-cluster) questions = suggest_questions(G, communities, labels) # Persist the graph first and only write the report/analysis if it actually # persisted - to_json refuses to shrink an existing graph.json (#479), and a # report describing a graph we did not write would be a lie (#1392). -wrote = to_json(G, communities, 'graphify-out/graph.json') +# community_labels=labels is passed here too, not just in Step 5's rewrite, +# because --no-cluster skips Step 5 entirely - this is the only write for +# that path, so the 'Full Corpus' label computed above must reach graph.json +# now or every node silently loses its community_name. +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (fewer nodes than the existing graph). Run a full rebuild to be safe.') raise SystemExit(1) @@ -512,12 +536,14 @@ Replace INPUT_PATH with the actual path. ### Step 5 - Label communities +Skip this step entirely if `--no-cluster` was given in Step 4 (`IS_NO_CLUSTER` was `True`) - there is only one placeholder community ("Full Corpus"), nothing to label. + Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). Then regenerate the report and save the labels for the visualizer: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.cluster import score_all @@ -547,7 +573,7 @@ Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for # Re-export so graph.json nodes carry the curated community_name (#2490). # Same extraction as Step 4, so the #479 shrink-guard passes on node count; # if it still refuses, surface the guard message - do not force past it. -wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels) +wrote = to_json(G, communities, 'graphify-out/graph.json', community_labels=labels, force=IS_FORCE) if not wrote: print('ERROR: refused to shrink graphify-out/graph.json (fewer nodes than the existing graph). Run a full rebuild to be safe.') print('Report updated with community labels') @@ -564,7 +590,7 @@ Replace INPUT_PATH with the actual path. If `--obsidian` was given: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_obsidian, to_canvas @@ -595,7 +621,7 @@ print(' _COMMUNITY_* - overview notes with cohesion scores and dataview queries Generate the HTML graph (always, unless `--no-viz`): ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_html @@ -648,7 +674,7 @@ The wiki is an agent-crawlable export — `index.md` plus one article per commun Run this before Step 9 (cleanup) so `graphify-out/.graphify_labels.json` is still available. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.build import build_from_json from graphify.wiki import to_wiki @@ -676,7 +702,7 @@ print(' graphify-out/wiki/index.md -> agent entry point') **If `--neo4j`** - generate a Cypher file for manual import: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_cypher @@ -691,7 +717,7 @@ print('cypher.txt written - import with: cypher-shell < graphify-out/cypher.txt' **If `--neo4j-push `** - push directly to a running Neo4j instance. Ask the user for credentials if not provided: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import push_to_neo4j @@ -712,7 +738,7 @@ Replace `NEO4J_URI`, `NEO4J_USER`, `NEO4J_PASSWORD` with actual values. Default ### Step 7b - SVG export (only if --svg flag) ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_svg @@ -734,7 +760,7 @@ print('graph.svg written - embeds in Obsidian, Notion, GitHub READMEs') ### Step 7c - GraphML export (only if --graphml flag) ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.build import build_from_json from graphify.export import to_graphml @@ -776,7 +802,7 @@ To configure in Claude Desktop, add to `claude_desktop_config.json`: If `total_words` from `graphify-out/.graphify_detect.json` is greater than 5,000, run: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.benchmark import run_benchmark, print_benchmark from pathlib import Path @@ -794,7 +820,7 @@ Print the output directly in chat. If `total_words <= 5000`, skip silently - the ### Step 9 - Save manifest, update cost tracker, clean up, and report ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from datetime import datetime, timezone @@ -898,7 +924,7 @@ fi Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -921,7 +947,7 @@ if new_total > 0: If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -943,7 +969,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -951,10 +977,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` + Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.build import build_from_json from graphify.export import to_json @@ -981,7 +1009,7 @@ Then run Steps 4-8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -1004,8 +1032,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- @@ -1014,7 +1041,7 @@ Clean up after: `rm -f graphify-out/.graphify_old.json` Skip Steps 1-3. Load the existing graph from `graphify-out/graph.json` and re-run clustering: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.cluster import cluster, score_all from graphify.analyze import god_nodes, surprising_connections @@ -1069,7 +1096,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -1087,7 +1114,7 @@ Load `graphify-out/graph.json`, then: 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -1178,7 +1205,7 @@ Replace `QUESTION` with the user's actual question, `MODE` with `bfs` or `dfs`, After writing the answer, save it back into the graph so it improves future queries: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` --- @@ -1189,7 +1216,7 @@ Find the shortest path between two named concepts in the graph. First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -1198,7 +1225,7 @@ if not Path('graphify-out/graph.json').exists(): ``` ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -1250,7 +1277,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names. Then explain the pa After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -1261,7 +1288,7 @@ Give a plain-language explanation of a single node - everything connected to it. First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -1270,7 +1297,7 @@ if not Path('graphify-out/graph.json').exists(): ``` ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -1315,7 +1342,7 @@ Replace `NODE_NAME` with the concept. Then write a 3-5 sentence explanation usin After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` --- @@ -1325,7 +1352,7 @@ $(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path diff --git a/tools/skillgen/fragments/references/query/default.md b/tools/skillgen/fragments/references/query/default.md index 56565eb782..fdc9c68d5f 100644 --- a/tools/skillgen/fragments/references/query/default.md +++ b/tools/skillgen/fragments/references/query/default.md @@ -11,7 +11,7 @@ Two traversal modes - choose based on the question: First check the graph exists: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " from pathlib import Path if not Path('graphify-out/graph.json').exists(): print('ERROR: No graph found. Run /graphify first to build the graph.') @@ -28,7 +28,7 @@ Fix this **without inventing tokens** by expanding the query against the actual 1. Extract the token vocabulary from node labels: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, re from pathlib import Path data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8')) @@ -77,7 +77,7 @@ If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal 5. If the graph lacks enough information, say so - do not hallucinate edges. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from networkx.readwrite import json_graph import networkx as nx @@ -168,7 +168,7 @@ Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 ``` Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. @@ -194,7 +194,7 @@ graphify path "NODE_A" "NODE_B" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -246,7 +246,7 @@ Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B ``` --- @@ -262,7 +262,7 @@ graphify explain "NODE_NAME" If the CLI is unavailable, run it inline: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, sys import networkx as nx from networkx.readwrite import json_graph @@ -307,5 +307,5 @@ Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sent After writing the explanation, save it back: ```bash -$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +"$(cat graphify-out/.graphify_python)" -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME ``` diff --git a/tools/skillgen/fragments/references/shared/add-watch.md b/tools/skillgen/fragments/references/shared/add-watch.md index 77844343e1..937edf327c 100644 --- a/tools/skillgen/fragments/references/shared/add-watch.md +++ b/tools/skillgen/fragments/references/shared/add-watch.md @@ -7,7 +7,7 @@ Load this when the user ran `/graphify add ` or passed `--watch`. Neither i Fetch a URL and add it to the corpus, then update the graph. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys from graphify.ingest import ingest from pathlib import Path @@ -41,7 +41,7 @@ Supported URL types (auto-detected): Start a background watcher that monitors a folder and auto-updates the graph when files change. ```bash -$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +"$(cat graphify-out/.graphify_python)" -m graphify.watch INPUT_PATH --debounce 3 ``` Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: diff --git a/tools/skillgen/fragments/references/shared/exports.md b/tools/skillgen/fragments/references/shared/exports.md index 242ff868e0..27b031e58c 100644 --- a/tools/skillgen/fragments/references/shared/exports.md +++ b/tools/skillgen/fragments/references/shared/exports.md @@ -59,7 +59,7 @@ graphify export graphml ### Step 7d - MCP server (only if --mcp flag) ```bash -$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +"$(cat graphify-out/.graphify_python)" -m graphify.serve graphify-out/graph.json ``` This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. diff --git a/tools/skillgen/fragments/references/shared/transcribe.md b/tools/skillgen/fragments/references/shared/transcribe.md index b967f83799..27064bfcdb 100644 --- a/tools/skillgen/fragments/references/shared/transcribe.md +++ b/tools/skillgen/fragments/references/shared/transcribe.md @@ -26,7 +26,7 @@ Read the top god node labels from detect output or analysis, then compose a shor ```bash export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) export GRAPHIFY_WHISPER_PROMPT="" -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json, os, sys from pathlib import Path from graphify.transcribe import transcribe_all diff --git a/tools/skillgen/fragments/references/shared/update.md b/tools/skillgen/fragments/references/shared/update.md index 3632fd4126..a0ca7dd58a 100644 --- a/tools/skillgen/fragments/references/shared/update.md +++ b/tools/skillgen/fragments/references/shared/update.md @@ -7,7 +7,7 @@ Load this only when the user passed `--update` or `--cluster-only`. A first-time Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path @@ -30,7 +30,7 @@ if new_total > 0: Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) @@ -48,7 +48,7 @@ Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ If new files exist, first check whether all changed files are code files: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path @@ -71,7 +71,7 @@ If no new files exist (only deletions), create an empty extraction so the merge ```bash if [ ! -f graphify-out/.graphify_extract.json ]; then echo '[graphify update] Only deletions -- creating empty extraction for merge.' - $(cat graphify-out/.graphify_python) -c " + "$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') @@ -79,11 +79,12 @@ Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'e fi ``` +Before the merge step below, save the old graph so the post-update diff has something to compare against: `cp graphify-out/graph.json graphify-out/.graphify_old.json` Then: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from pathlib import Path from graphify.build import build_merge @@ -170,7 +171,7 @@ Then run Steps 4–8 on the merged graph as normal. After Step 4, show the graph diff: ```bash -$(cat graphify-out/.graphify_python) -c " +"$(cat graphify-out/.graphify_python)" -c " import json from graphify.analyze import graph_diff from graphify.build import build_from_json @@ -194,8 +195,7 @@ if old_data: " ``` -Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` -Clean up after: `rm -f graphify-out/.graphify_old.json` +Clean up the backup after: `rm -f graphify-out/.graphify_old.json` --- diff --git a/tools/skillgen/fragments/shell/posix.md b/tools/skillgen/fragments/shell/posix.md index 3534417d23..0297ca0d18 100644 --- a/tools/skillgen/fragments/shell/posix.md +++ b/tools/skillgen/fragments/shell/posix.md @@ -26,6 +26,17 @@ if ! "$PYTHON" -c "import graphify" 2>/dev/null; then "$PYTHON" -m pip install graphifyy -q 2>/dev/null \ || "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>&1 | tail -3 fi + # #1619 B4: without this gate, a failed install left PYTHON pointing at an + # interpreter that still cannot import graphify. The step fell through + # silently, writing that interpreter's path anyway, and every later step + # then failed with a cryptic "-c: command not found" far from the real + # cause instead of a clear error here. + if ! "$PYTHON" -c "import graphify" 2>/dev/null; then + echo "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" >&2 + echo " uv tool install graphifyy" >&2 + echo " python3 -m pip install graphifyy" >&2 + exit 1 + fi fi # Write interpreter path for all subsequent steps (persists across invocations) mkdir -p graphify-out @@ -34,6 +45,6 @@ mkdir -p graphify-out echo "$(cd INPUT_PATH && pwd)" > graphify-out/.graphify_root ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. -**In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.** +**In every subsequent bash block, replace `python3` with `"$(cat graphify-out/.graphify_python)"` to use the correct interpreter.** diff --git a/tools/skillgen/fragments/shell/powershell.md b/tools/skillgen/fragments/shell/powershell.md index 71e493cf8d..8cad678689 100644 --- a/tools/skillgen/fragments/shell/powershell.md +++ b/tools/skillgen/fragments/shell/powershell.md @@ -50,6 +50,17 @@ if (-not $GRAPHIFY_PYTHON) { $GRAPHIFY_PYTHON = Find-GraphifyPython } +# #1619 B4: without this gate, a failed install left $GRAPHIFY_PYTHON $null, +# an empty .graphify_python got written anyway, and every later step then +# failed with a cryptic error far from the real cause instead of a clear one +# here. +if (-not $GRAPHIFY_PYTHON) { + Write-Host "ERROR: could not install or locate a Python interpreter with graphify. Try one of:" + Write-Host " uv tool install graphifyy" + Write-Host " pip install graphifyy" + exit 1 +} + # Save interpreter path — all subsequent steps read this. # `Out-File -Encoding utf8` always writes a BOM on Windows PowerShell 5.1 (utf8NoBOM # only exists from PowerShell 6), and that BOM rides into the saved path, so the hook @@ -58,9 +69,9 @@ if (-not $GRAPHIFY_PYTHON) { $Utf8NoBom = New-Object System.Text.UTF8Encoding $false [System.IO.File]::WriteAllText((Join-Path $PWD 'graphify-out\.graphify_python'), [string]$GRAPHIFY_PYTHON, $Utf8NoBom) # Save scan root so `graphify update` (no args) knows where to look next time -[System.IO.File]::WriteAllText((Join-Path $PWD 'graphify-out\.graphify_root'), (Resolve-Path INPUT_PATH).Path, $Utf8NoBom) +[System.IO.File]::WriteAllText((Join-Path $PWD 'graphify-out\.graphify_root'), (Resolve-Path 'INPUT_PATH').Path, $Utf8NoBom) ``` -If the import succeeds, print nothing and move straight to Step 2. +If the import succeeds, print nothing and move straight to Step 2. If it prints the ERROR above, stop and tell the user what happened - do not proceed to Step 2. **In every subsequent block, run Python through the saved interpreter — `& (Get-Content graphify-out\.graphify_python)` in place of a bare `python3` — so every step uses the interpreter that actually has graphify.** diff --git a/tools/skillgen/gen.py b/tools/skillgen/gen.py index 09e19ede00..16683fd92e 100644 --- a/tools/skillgen/gen.py +++ b/tools/skillgen/gen.py @@ -381,7 +381,7 @@ def _render_frontmatter(platform: Platform) -> str: # untranslated bash to the Windows variant. ``_POWERSHELL_BANNED_TOKENS`` is the # belt-and-braces post-check on the final body. -_PY_INVOKE_POSIX = '$(cat graphify-out/.graphify_python) -c "' +_PY_INVOKE_POSIX = '"$(cat graphify-out/.graphify_python)" -c "' _PY_INVOKE_PS_OPEN = "@'" _PY_INVOKE_PS_CLOSE = "'@ | & (Get-Content graphify-out\\.graphify_python) -" _MKDIR_POSIX = "mkdir -p graphify-out" @@ -1142,6 +1142,117 @@ def _is_community_label_export_fix_line(line: str) -> bool: ) +def _is_no_cluster_force_fix_line(line: str) -> bool: + """Whether a line is part of the --no-cluster / --force wiring fix (#1619 C1/C2). + + Step 2 suggested --no-cluster and the #479 shrink-guard error message + suggested --force, but neither flag was ever implemented: Step 4 called + cluster() unconditionally, and to_json() never received force=. Both are + now wired through -- Step 4 builds a single "Full Corpus" community + instead of calling cluster() when --no-cluster was given, both to_json() + calls take force=IS_FORCE, and Step 5 is skipped when there is nothing to + label. These are the added Usage lines, the substitution instructions, + and the changed Step 4/5 body lines (old and new forms both sanctioned, + matched trimmed so the added lines' extra indentation inside the new + if/else does not matter). + """ + stripped = line.strip() + return ( + stripped.startswith('/graphify --no-cluster ') + or stripped.startswith('/graphify --force ') + or stripped.startswith("Two more substitutions, in this step's block and Step 5's:") + or stripped == "if IS_NO_CLUSTER:" + or stripped == "communities = {0: list(G.nodes())}" + or stripped == "communities = cluster(G)" + or stripped + == "labels = {0: 'Full Corpus'} if IS_NO_CLUSTER else {cid: 'Community ' + str(cid) for cid in communities}" + or stripped == "labels = {cid: 'Community ' + str(cid) for cid in communities}" + or "regenerated with real labels in Step 5" in line + or "Skip this step entirely if `--no-cluster` was given in Step 4" in line + # A review finding on #1619 pointed out the Step 4 to_json() call + # (sanctioned above via the #1392 predicate's generic "to_json(G, + # communities," match) never received community_labels=labels, so a + # --no-cluster build silently lost every node's community_name -- + # Step 5, which normally supplies it, is skipped for that path. The + # explanatory comment added alongside that fix is sanctioned here. + or "because --no-cluster skips Step 5 entirely" in line + or "the 'Full Corpus' label computed above must reach graph.json" in line + or "now or every node silently loses its community_name" in line + ) + + +def _is_install_failure_gate_fix_line(line: str) -> bool: + """Whether a line is part of the #1619 B4 Step 1 failure gate. + + A failed install used to leave PYTHON pointing at an interpreter that + still could not import graphify, with nothing checking for it: the step + fell through silently, wrote that interpreter's path anyway, and every + later step then failed with a cryptic "-c: command not found" far from + the real cause. An explicit re-check after the install attempt now stops + with an actionable error instead. The extra `fi` (added) and the changed + "If the import succeeds..." sentence (both old and new forms) are + sanctioned here too. + """ + stripped = line.strip() + return ( + stripped == "fi" + or "#1619 B4" in line + or "interpreter that still cannot import graphify" in line + or "silently, writing that interpreter's path anyway" in line + or 'then failed with a cryptic "-c: command not found"' in line + or "cause instead of a clear error here" in line + or stripped == 'if ! "$PYTHON" -c "import graphify" 2>/dev/null; then' + or "could not install or locate a Python interpreter with graphify" in line + or "uv tool install graphifyy" in line + or "python3 -m pip install graphifyy" in line + or stripped == "exit 1" + or stripped.startswith("If the import succeeds, print nothing and move straight to Step 2") + ) + + +def _is_update_backup_reorder_fix_line(line: str) -> bool: + """Whether a line is part of the #1619 C4 backup-ordering fix. + + "Before the merge step, save the old graph" appeared AFTER the merge + block and the diff block that consumes the backup, so an agent executing + top to bottom reached it too late and the post-update diff's `if + old_data:` silently no-opped every time. The instruction now appears + before the merge block it belongs with; the trailing cleanup line stays + where it was, reworded to stop implying a backup step just above it. + """ + stripped = line.strip() + return ( + stripped.startswith("Before the merge step") + or stripped.startswith("Clean up after:") + or stripped.startswith("Clean up the backup after:") + ) + + +def _is_quoted_interpreter_cat_fix_line(line: str) -> bool: + """Whether a line is part of the #1619 B5 interpreter-substitution quoting fix. + + `$(cat graphify-out/.graphify_python)` was spliced in unquoted everywhere + it names the interpreter to run, so an interpreter path containing a + space (a venv under `C:\\Users\\First Last\\...`) word-split into + multiple arguments. Every occurrence, and the prose line describing the + substitution, is now quoted, matching the `"$PYTHON"` form Step 1 already + used. Both the old (removed) and new (added) forms match here. + """ + return "$(cat graphify-out/.graphify_python)" in line + + +def _is_input_path_slash_guidance_line(line: str) -> bool: + """Whether a line is the #1619 B1 forward-slash INPUT_PATH guidance. + + A Windows path substituted into INPUT_PATH splices raw backslashes into a + Python string literal (`\\t` becomes a tab, `\\U` raises a SyntaxError), + silently or loudly corrupting the block. One rule near the top of each + monolith, right after the path-resolution paragraph it belongs with, + tells the agent to substitute with forward slashes instead. + """ + return "substitute it with forward slashes" in line.lower() + + # Every line that may differ between a rendered monolith and its pristine v8 # baseline. Each predicate documents one sanctioned change-class; a blank line is # allowed because the multi-line fix blocks insert spacing. Anything else failing @@ -1163,6 +1274,11 @@ def _is_community_label_export_fix_line(line: str) -> bool: _is_uv_from_interpreter_fix_line, _is_semantic_cache_scope_fix_line, _is_community_label_export_fix_line, + _is_no_cluster_force_fix_line, + _is_input_path_slash_guidance_line, + _is_install_failure_gate_fix_line, + _is_update_backup_reorder_fix_line, + _is_quoted_interpreter_cat_fix_line, )