Skip to content

Add portable query-plan-analysis skill#385

Merged
erikdarlingdata merged 1 commit into
devfrom
skill/query-plan-analysis
Jul 10, 2026
Merged

Add portable query-plan-analysis skill#385
erikdarlingdata merged 1 commit into
devfrom
skill/query-plan-analysis

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

What

A self-contained Agent Skill at skills/query-plan-analysis/ that teaches any Claude agent to read a SQL Server execution plan without PerformanceStudio installed. Nothing in it references this repo, so the directory copies out cleanly for anyone.

Why this shape

The first instinct was to extract PlanViewer's 37 RuleNN_ detection methods into a rule catalog. That was wrong. An LLM spots LogicalOp == "Eager Spool" in XML unaided — detection is the disposable part. What a model can't do is know the things that make plan analysis sound authoritative and be wrong:

  • Cost percentages are estimates in every plan, including actual plans. The 97%-cost operator is routinely not the slow one.
  • Row-mode operator times are cumulative, so ranking by raw ActualElapsedms ranks by depth and always crowns the root node.
  • EstimateRows is per-execution; ActualRows is a total. Without dividing by ActualExecutions, the inner side of every nested loop looks catastrophically underestimated.
  • Missing-index requests emit equality columns in arbitrary order and ignore every index that already exists.

So the skill is built out of negative knowledge — the conclusions that are unsafe.

Contents

Path Purpose
SKILL.md Seven-step triage, ordered to establish ground truth before forming an opinion. Includes a "things that are not evidence" list.
scripts/extract.py Flattens a .sqlplan into a digest. --node N drills into one operator; --sql recovers full statement text. Stdlib only.
references/ timing, cardinality, warnings, parallelism, indexes, operators, plus a no-Python fallback.

Why an extractor is mandatory, not a nicety

  • .sqlplan files are UTF-16. grep PlanAffectingConvert plan.sqlplan matches nothing and reports no error, so a negative result is worthless.
  • Some plans are UTF-8 bytes still declaring encoding="utf-16" (opened and re-saved). Strict XML parsers reject them.
  • A trivial two-table join is 120 KB. The largest fixture in this repo is 7 MB.

extract.py mirrors PlanAnalyzer.Timing.cs and NodeTimeAttribution.cs for self-time attribution rather than trusting anyone's memory of the semantics: batch mode reports standalone times, row mode cumulative, exchange operators accumulate downstream wait time and their thread 0 is a coordinator, and parallel self-time must subtract within a thread rather than across threads.

Test plan

Seven agents with no prior context were given only the skill directory and a plan file, then graded against ground truth. All seven reached correct conclusions.

Three were deliberate traps:

  • A plan with nothing wrong. It refused to invent a bottleneck, and independently caught that HAVING COUNT_BIG(*) > POWER(2., 31) - 1 can never fire on a 2.46M-row table.
  • A false-alarm warning. Correctly identified transitive-predicate elimination rather than crying cross join.
  • An estimated-only plan, asked "which operator took the most time?" It correctly declined and asked for an actual plan.

A fourth had decoys: a trivial 26-write sort spill and a missing-index request sitting next to a scalar UDF consuming 13,693 of the query's 13,694 seconds. The agent named the UDF and explicitly dismissed both decoys.

extract.py runs clean on 298 of 299 example plans. The holdout is garbage.sqlplan, which is deliberately invalid and now fails with a readable message instead of a stack trace.

What testing found

Agent feedback fixed real defects, not cosmetics:

  • A bug. Eager-spool detection was "Spool" in physical and "Eager" in logical, which also matches Table Spool (Eager Spool) — ordinary Halloween protection in any update plan. The tool told update plans they needed an index, while indexes.md next to it said those spools are not defects.
  • A documentation hole that produced a wrong answer. warnings.md gave positive tells for two of three no-join-predicate cases. The missing one — both inputs pinned to the same constant, no outer references, no predicate — is the most common, and a reader applying the two stated tests literally would have diagnosed an accidental cross join.
  • Two factual errors. TSQLUserDefinedFunctionsNotParallelizable is not the only scalar-UDF serial-plan tell (CouldNotGenerateValidParallelPlan is at least as common), and the "do self times sum to the total?" sanity check is valid only for serial plans — in a parallel plan they legitimately exceed it, so the check as written would have talked an agent out of a correct answer.
  • An answer key. SKILL.md's worked example quoted the exact numbers of one of the test fixtures.

Writing the extractor caught two more before any agent saw it: self-elapsed printed next to cumulative CPU, and row-mode child subtraction applied to batch-mode CPU, underflowing a 41-second hash aggregate to "0 ms self CPU." Both are exactly the failures the skill exists to prevent, and neither surfaced until it ran against real plans.

One suggestion was declined: having the digest auto-classify warnings as benign or genuine. It has the inputs, but a confident wrong verdict is worse than none and is the precise failure mode this skill targets. It prints the facts and says INCONCLUSIVE when the row counts cannot discriminate.

Note

references/timing.md is transcribed from PlanAnalyzer.Timing.cs. If that file's semantics change, the skill drifts.

🤖 Generated with Claude Code

A self-contained Agent Skill that teaches any Claude agent to read a SQL Server
execution plan without needing PerformanceStudio installed. Nothing in it refers
to this repo, so the directory copies out cleanly.

The valuable content is negative knowledge -- the confident-sounding conclusions
that are wrong. Cost percentages are estimates even in actual plans. Row-mode
operator times are cumulative, so ranking by raw ActualElapsedms ranks by depth
and always crowns the root. Estimated vs actual rows must be normalized by
ActualExecutions or the inner side of every nested loop looks catastrophically
underestimated. Missing-index requests emit equality columns in arbitrary order
and ignore existing indexes.

Contents:
  SKILL.md         seven-step triage, ordered to establish ground truth before
                   forming an opinion, plus a "not evidence" list
  scripts/         extract.py flattens a .sqlplan into a digest; --node drills
                   into one operator; --sql recovers full statement text
  references/      timing, cardinality, warnings, parallelism, indexes, operators,
                   and a no-Python fallback

extract.py exists because .sqlplan files are UTF-16 (so grep silently matches
nothing), some declare an encoding they don't use, and a trivial plan is 120 KB
while the largest fixture here is 7 MB. It mirrors PlanAnalyzer.Timing.cs and
NodeTimeAttribution.cs for self-time attribution: batch mode reports standalone
times, row mode cumulative, exchange operators accumulate downstream wait time,
and parallel self-time must subtract within a thread rather than across threads.

Validated by seven agents with no prior context, run against the 299 example
plans. All reached correct conclusions, including on three traps: a plan with
nothing wrong, a no-join-predicate warning that was a false alarm, and an
estimated-only plan where the correct answer is to decline and ask for an actual
plan. Their findings fixed a real bug (Table Spool matched the Index Spool
detection, so update plans were told they needed an index), a documentation hole
that would have produced a wrong cross-join diagnosis, two factual errors about
scalar-UDF parallelism reasons and parallel self-time arithmetic, and an answer
key that had leaked from a test fixture into SKILL.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@erikdarlingdata erikdarlingdata merged commit af7b6e8 into dev Jul 10, 2026
2 checks passed
@erikdarlingdata erikdarlingdata deleted the skill/query-plan-analysis branch July 10, 2026 01:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant