Add a body attribute tier so query seeding can reach non-extraction content (#3313) - #3636
ayushcodes10 wants to merge 3 commits into
Conversation
_score_query only matched a node's label, source path, and the curated rationale attribute. A node emitted by something other than graphify's own extraction, a Confluence page, a Jira issue, has nowhere else to put its content, so storing it bought nothing on its own: the node could never become a seed and was therefore never traversed, no matter how relevant. Everything about the node looked correct in that situation, the data was in the graph and the ranker was fine, the node was simply never visited. body is now the documented convention for that free text, matched at its own tier well below the rationale tier rather than sharing it, since raw scraped or pasted prose is noisier than curated why text and an unweighted body match was measured elsewhere to make retrieval worse. Like rationale, a body hit adds to the score but never counts toward term coverage, so a long body adds recall without winning back an exact label tier it did not earn. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Mirrors the existing rationale tier test coverage exactly: reads a body match when the label does not match at all, sits below rationale and source in tier ordering, never restores term coverage, tolerates a list valued body, and feeds the trigram search text prefilter with the field count check preserved for when body is absent. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Graphify reviewed this change.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Formal verification. 1 change(s) tested, no difference found (not proven).
Graphify review — findings
Adds a body attribute tier to query seeding so nodes emitted by non-extraction sources (Confluence pages, Jira issues) can become seeds and be traversed instead of sitting invisible in the graph. _score_query and _node_search_text now match against body via the new _node_body_text helper, weighted at _BODY_MATCH_BONUS (0.35) — well below the curated rationale tier and adding recall only, without counting toward term coverage — since raw prose matches many query terms and an unweighted match made retrieval worse.
No blocking issues surfaced.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 825 functions depend on the 567 functions this change touches.
Health — this change adds coupling hotspots:
- new:
dispatch_command()— 2 callers, 125 callees - new:
_query_graph_text()— 25 callers, 10 callees - new:
_score_query()— 15 callers, 7 callees - new:
_query_terms()— 20 callers, 3 callees - new:
run_benchmark()— 16 callers, 3 callees - new:
_load_graph()— 14 callers, 3 callees - new:
_build_server()— 2 callers, 16 callees - new:
_node_search_text()— 7 callers, 4 callees - …and 9 more — each is listed as a finding
Verification — 825 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 640 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
286 of 286 test file(s) selected (100%) via static blast radius.
Escalated to a full run for safety — the selection is not trustworthy on its own (see below). CI should run the whole suite.
tests/test_affected_cli.py— full-run-safetytests/test_affected_member_seed.py— full-run-safetytests/test_agents_platform.py— full-run-safetytests/test_analyze.py— full-run-safetytests/test_anthropic_custom_endpoint.py— full-run-safetytests/test_antigravity_install.py— full-run-safetytests/test_apm_fallback_version.py— full-run-safetytests/test_architecture_doc.py— full-run-safetytests/test_astro_extraction.py— full-run-safetytests/test_astro_import_ids.py— full-run-safetytests/test_atomic_canvas_export.py— full-run-safetytests/test_atomic_version_stamp.py— full-run-safetytests/test_atomic_writes.py— full-run-safetytests/test_backend_env_isolation.py— full-run-safetytests/test_backend_extras.py— full-run-safetytests/test_benchmark.py— impact, full-run-safetytests/test_benchmark_raw_graph.py— impact, full-run-safetytests/test_build.py— full-run-safetytests/test_build_merge_dedup_scope.py— full-run-safetytests/test_build_merge_hyperedges_and_prune.py— full-run-safetytests/test_build_merge_shrink_guard.py— full-run-safetytests/test_builtin_global_type_refs.py— full-run-safetytests/test_cache.py— full-run-safetytests/test_callflow_html.py— full-run-safetytests/test_cargo_introspect.py— full-run-safetytests/test_carried_hyperedge_remap.py— full-run-safetytests/test_case_sensitive_resolution.py— full-run-safetytests/test_charmap_encoding.py— full-run-safetytests/test_chunking.py— full-run-safetytests/test_cjs_module_extension.py— full-run-safetytests/test_claude_cli_backend.py— full-run-safetytests/test_claude_md.py— full-run-safetytests/test_cli_broken_pipe.py— full-run-safetytests/test_cli_export.py— full-run-safetytests/test_cli_help.py— full-run-safetytests/test_cluster.py— full-run-safetytests/test_codebuddy.py— full-run-safetytests/test_community_hub_labels.py— full-run-safetytests/test_community_labels_skill.py— full-run-safetytests/test_confidence.py— full-run-safetytests/test_corrupt_graph_json.py— full-run-safetytests/test_cpp_nested_and_cli.py— full-run-safetytests/test_cpp_objc_cross_file_calls.py— full-run-safetytests/test_cpp_preprocess.py— full-run-safetytests/test_cross_extension_reexport_self_cycle.py— full-run-safetytests/test_cross_language_call_resolution.py— full-run-safetytests/test_cross_repo_external_call_guards.py— full-run-safetytests/test_cross_repo_member_calls.py— full-run-safetytests/test_cross_repo_shared_types.py— full-run-safetytests/test_csharp_call_site_generic_args.py— full-run-safety- … and 236 more
non-code file(s) changed (
CHANGELOG.md) → running the full suite for safety (a code graph can't see config/fixture/data deps)
changed code file(s) with no mapped test (
CHANGELOG.md) — a coverage gap or a missing link — running the full suite rather than only the selected tests
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
Formal verification
No difference found (not proven): No behavior difference found in \_node\_search\_text (not a proof).
The verifier ran both versions of \_node\_search\_text on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify \_score\_query.
The verifier did not have enough to check \_score\_query, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: no capturable inputs from the test suite; property tier: not verifiable: all 45 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly TypeError — names the real obstacle, not a sampling gap)
· 1 grounded finding(s) anchored inline below; 16 more finding(s) on lines outside this diff (see the check run).
| return _strip_diacritics(str(raw)).lower() | ||
|
|
||
|
|
||
| def _node_search_text(data: dict, nid: str) -> str: |
There was a problem hiding this comment.
_node_search_text()
7 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
1 participant
Summary
Fixes #3313. `_score_query`/`_pick_seeds` scored a node's label, source path, and the curated `rationale` attribute (#2293) — nothing else. A node emitted by something other than graphify's own extraction pipeline (a Confluence page, a Jira issue) has nowhere else to put its content, so storing text on an arbitrary attribute bought nothing on its own: the node could never become a seed and was therefore never traversed, no matter how relevant its content was to the query. Everything about the node looked correct in that failure mode — the data is in the graph, the ranker is fine — the node is just never visited, which the issue notes is easy to hit and expensive to diagnose.
Reproduced the issue's exact motivating shape (a page titled "Masterlist/Certificate controls" whose body contains "CSCA Master List", with "CSCA" appearing nowhere in its label) — before this fix the page is unreachable at any budget; after, it's the top-ranked result.
Fix
`body` is now the documented convention for this free text, following the exact pattern the existing `rationale` tier already established (same helper shape, same append-to-search-text treatment, same "doesn't count toward term coverage" rule so a long body adds recall without winning an exact-label tier it didn't earn) — but at its own, lower weight (`_BODY_MATCH_BONUS = 0.35`, vs. rationale's `0.75`), since raw scraped/pasted prose is noisier than curated why-text. This matches the issue's own finding: an unweighted body match was measured to make retrieval worse, since a few hundred characters of prose matches many query terms and a page that merely mentions the words ties with the page that's actually about them.
Did not adopt the issue's proposal 1 verbatim (an opt-in `GRAPHIFY_SEED_ATTRS` env var / build-config field) — a single documented attribute name, mirroring the already-shipped `rationale` convention, is simpler and needs no new configuration surface. Happy to add the configurable version if a single fixed name proves too narrow in practice.
Test plan
🤖 Generated with Claude Code