diff --git a/.agents/skills/openspec-apply-change/SKILL.md b/.agents/skills/openspec-apply-change/SKILL.md index d23f2b3..5baf142 100644 --- a/.agents/skills/openspec-apply-change/SKILL.md +++ b/.agents/skills/openspec-apply-change/SKILL.md @@ -12,9 +12,23 @@ metadata: Implement tasks from an OpenSpec change. -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -**Input**: Optionally specify a change name (e.g., `$openspec-apply-change (Codex) or /openspec-apply-change (other agents) add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +**Input**: Optionally specify a change name (e.g., +`$openspec-apply-change (Codex) or /openspec-apply-change (other agents) add-auth`). +If omitted, check if it can be inferred from conversation context. If vague or +ambiguous you MUST prompt for available changes. **Steps** @@ -23,18 +37,24 @@ Implement tasks from an OpenSpec change. If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes and ask + the user to select one - Always announce: "Using change: " and how to override (e.g., `$openspec-apply-change (Codex) or /openspec-apply-change (other agents) `). + Always announce: "Using change: " and how to override (e.g., + `$openspec-apply-change (Codex) or /openspec-apply-change (other agents) `). 2. **Check status to understand the schema** + ```bash openspec status --change "" --json ``` + Parse the JSON to understand: - `schemaName`: The workflow being used (e.g., "spec-driven") - - `planningHome`, `changeRoot`, and `actionContext`: planning scope and edit constraints - - Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others) + - `planningHome`, `changeRoot`, and `actionContext`: planning scope and edit + constraints + - Which artifact contains the tasks (typically "tasks" for spec-driven, check + status for others) 3. **Get apply instructions** @@ -43,23 +63,31 @@ Implement tasks from an OpenSpec change. ``` This returns: - - `contextFiles`: artifact ID -> array of concrete file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs) + - `contextFiles`: artifact ID -> array of concrete file paths (varies by + schema - could be proposal/specs/design/tasks or + spec/tests/implementation/docs) - Progress (total, complete, remaining) - Task list with status - Dynamic instruction based on current state - - Optional `context`: current required project instruction input from the selected root + - Optional `context`: current required project instruction input from the + selected root - Optional `operationGuidance`: current advisory guidance for apply **Handle states:** - - If `state: "blocked"` (missing artifacts): show message, suggest using `$openspec-continue-change (Codex) or /openspec-continue-change (other agents)` (if it is not installed, run `openspec status --change "" --json` to see the next artifact and `openspec instructions --change "" --json` for how to create it) + - If `state: "blocked"` (missing artifacts): show message, suggest using + `$openspec-continue-change (Codex) or /openspec-continue-change (other agents)` + (if it is not installed, run `openspec status --change "" --json` to + see the next artifact and + `openspec instructions --change "" --json` for how to + create it) - If `state: "all_done"`: congratulate, suggest archive - Otherwise: proceed to implementation Treat `context` as a required prompt-level input. Read and consider it, and - apply relevant project facts, conventions, and constraints while implementing. - Treat `operationGuidance` as optional additive advice. Read and consider every - entry, and follow entries that are applicable and compatible with the built-in - workflow. + apply relevant project facts, conventions, and constraints while + implementing. Treat `operationGuidance` as optional additive advice. Read and + consider every entry, and follow entries that are applicable and compatible + with the built-in workflow. Keep both fields separate from CLI-returned state, missing artifacts, tasks, progress, `contextFiles`, and the built-in `instruction`. They are not @@ -72,8 +100,8 @@ Implement tasks from an OpenSpec change. 4. **Read context files** - Read every file path listed under `contextFiles` from the apply instructions output. - The files depend on the schema being used: + Read every file path listed under `contextFiles` from the apply instructions + output. The files depend on the schema being used: - **spec-driven**: proposal, specs, design, tasks - Other schemas: follow the contextFiles from CLI output @@ -100,7 +128,9 @@ Implement tasks from an OpenSpec change. **Pause if:** - Task is unclear → ask for clarification - Implementation reveals a design issue → suggest updating artifacts - - A task needs work beyond what the spec and tasks describe, or you are tempted to drop, narrow, defer, or accept exceptions to specified behavior to make it fit → surface the added scope and ask; do not absorb it silently + - A task needs work beyond what the spec and tasks describe, or you are + tempted to drop, narrow, defer, or accept exceptions to specified behavior + to make it fit → surface the added scope and ask; do not absorb it silently - Error or blocker encountered → report and wait for guidance - User interrupts @@ -114,7 +144,7 @@ Implement tasks from an OpenSpec change. **Output During Implementation** -``` +```text ## Implementing: (schema: ) Working on task 3/7: @@ -128,7 +158,7 @@ Working on task 4/7: **Output On Completion** -``` +```text ## Implementation Complete **Change:** @@ -145,7 +175,7 @@ All tasks complete! You can archive this change with `$openspec-archive-change ( **Output On Pause (Issue Encountered)** -``` +```text ## Implementation Paused **Change:** @@ -164,6 +194,7 @@ What would you like to do? ``` **Guardrails** + - Keep going through tasks until done or blocked - Always read context files before starting (from the apply instructions output) - If task is ambiguous, pause and ask before implementing @@ -171,18 +202,25 @@ What would you like to do? - Keep code changes minimal and scoped to each task - Update task checkbox immediately after completing each task - Pause on errors, blockers, or unclear requirements - don't guess -- When a task needs work beyond what the spec describes, surface the added scope and pause - never silently narrow, defer, or simplify away specified behavior -- Only mark a task `- [x]` when its specified behavior is fully implemented, not when it is partially done or deferred +- When a task needs work beyond what the spec describes, surface the added scope + and pause - never silently narrow, defer, or simplify away specified behavior +- Only mark a task `- [x]` when its specified behavior is fully implemented, not + when it is partially done or deferred - Use contextFiles from CLI output, don't assume specific file names - Do not use context or operation guidance as proof that a task is complete -- Apply relevant project context; report conflicts with controlling workflow inputs +- Apply relevant project context; report conflicts with controlling workflow + inputs - Consider every guidance entry; explain any inapplicable or conflicting advice -- Do not copy runtime context or operation guidance into implementation files or planning artifacts -- Preserve CLI-controlled blocked/ready/all-done behavior and completion criteria +- Do not copy runtime context or operation guidance into implementation files or + planning artifacts +- Preserve CLI-controlled blocked/ready/all-done behavior and completion + criteria **Fluid Workflow Integration** This skill supports the "actions on a change" model: -- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions -- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly +- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), + after partial implementation, interleaved with other actions +- **Allows artifact updates**: If implementation reveals design issues, suggest + updating artifacts - not phase-locked, work fluidly diff --git a/.agents/skills/openspec-archive-change/SKILL.md b/.agents/skills/openspec-archive-change/SKILL.md index b83ab87..30ed54c 100644 --- a/.agents/skills/openspec-archive-change/SKILL.md +++ b/.agents/skills/openspec-archive-change/SKILL.md @@ -12,11 +12,26 @@ metadata: Archive a completed change in the experimental workflow. -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -`` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve the full path from each delta spec when resolving its main spec. - -**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +`` is the spec directory relative to `specs/` (for example, +`user-auth` or `identity/user-auth`). Preserve the full path from each delta +spec when resolving its main spec. + +**Input**: Optionally specify a change name. If omitted, check if it can be +inferred from conversation context. If vague or ambiguous you MUST prompt for +available changes. **Steps** @@ -25,24 +40,29 @@ Archive a completed change in the experimental workflow. If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes and ask + the user to select one When prompting, show only active changes (not already archived). Include the schema used for each change if available. - Always announce: "Using change: " and how to override (e.g., `$openspec-archive-change (Codex) or /openspec-archive-change (other agents) `). + Always announce: "Using change: " and how to override (e.g., + `$openspec-archive-change (Codex) or /openspec-archive-change (other agents) `). **Load current archive inputs before the existing archive checks:** After resolving the selected change and planning root, run: + ```bash openspec instructions archive --change "" --json ``` - Keep the same selected-root flags on this command. This lookup is advisory and - optional: it only supplies extra prompt inputs, so it must never block archiving. - If it exits non-zero or returns invalid JSON — for example on an older CLI that - does not support this command yet — continue the archive workflow with no - context and no operation guidance. Do not report an error and do not stop. + + Keep the same selected-root flags on this command. This lookup is advisory + and optional: it only supplies extra prompt inputs, so it must never block + archiving. If it exits non-zero or returns invalid JSON — for example on an + older CLI that does not support this command yet — continue the archive + workflow with no context and no operation guidance. Do not report an error + and do not stop. A successful response may omit both optional fields. Treat `context` as a required prompt-level input: read and consider it, and apply relevant project @@ -50,14 +70,15 @@ Archive a completed change in the experimental workflow. additive advice: read and consider every entry, and follow entries that are applicable and compatible with the built-in archive workflow. - Keep both fields separate from built-in steps, explicit user choices, resolved - paths, CLI checks, and command contracts. If context conflicts with one of those - controlling inputs, report the conflict and preserve the controlling value. If - guidance is inapplicable or conflicts with a controlling input, do not follow it - and explain why. Do not infer replacement paths, skipped prompts, or flags from - either field, and do not copy their text verbatim into specs, change artifacts, - or archive summaries unless the user separately asks for it. These are - prompt-level behavior contracts, not enforceable checks. + Keep both fields separate from built-in steps, explicit user choices, + resolved paths, CLI checks, and command contracts. If context conflicts with + one of those controlling inputs, report the conflict and preserve the + controlling value. If guidance is inapplicable or conflicts with a + controlling input, do not follow it and explain why. Do not infer replacement + paths, skipped prompts, or flags from either field, and do not copy their + text verbatim into specs, change artifacts, or archive summaries unless the + user separately asks for it. These are prompt-level behavior contracts, not + enforceable checks. 2. **Check artifact completion status** @@ -65,10 +86,13 @@ Archive a completed change in the experimental workflow. Parse the JSON to understand: - `schemaName`: The workflow being used - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context - - `artifacts`: List of artifacts with their status (`done`, `skipped`, or other) + - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path + and scope context + - `artifacts`: List of artifacts with their status (`done`, `skipped`, or + other) - **If any artifacts are neither `done` nor `skipped`** (skipped artifacts satisfy the requirement - the change declares skip_specs): + **If any artifacts are neither `done` nor `skipped`** (skipped artifacts + satisfy the requirement - the change declares skip_specs): - Display warning listing incomplete artifacts - Ask the user to confirm they want to proceed - Proceed if user confirms @@ -89,13 +113,16 @@ Archive a completed change in the experimental workflow. 4. **Assess delta spec sync state** Use `artifactPaths.specs.existingOutputPaths` from status JSON as the only - delta-spec source. If the `specs` entry is missing or - `existingOutputPaths` is empty, proceed without a sync prompt and do not infer - delta specs from other artifacts. + delta-spec source. If the `specs` entry is missing or `existingOutputPaths` + is empty, proceed without a sync prompt and do not infer delta specs from + other artifacts. **If delta specs exist:** - - Compare each delta spec with its corresponding main spec at `/openspec/specs//spec.md` (use the store-aware `planningHome.root` from step 2, not a hardcoded repo path) - - Determine what changes would be applied (adds, modifications, removals, renames) + - Compare each delta spec with its corresponding main spec at + `/openspec/specs//spec.md` (use the + store-aware `planningHome.root` from step 2, not a hardcoded repo path) + - Determine what changes would be applied (adds, modifications, removals, + renames) - Show a combined summary before prompting **Prompt options:** @@ -110,34 +137,58 @@ Archive a completed change in the experimental workflow. Before a selected sync writes any main spec, run `openspec instructions specs --change "" --json` once with the same - selected-root flags. Require a zero exit status and valid artifact-instruction - JSON. If the lookup fails or returns invalid JSON, report the error and stop - before writing any main spec or moving the change. A valid response with omitted - `rules` is the no-rules case. Apply returned `rules` only to the content and - form of main specs produced by this merge; do not use them as archive guidance, - change CLI behavior, or copy the rule text into any output file. - - Then run the `openspec-sync-specs` workflow inline (agent-driven intelligent merge) for change '', passing the delta spec analysis and the fetched specs-rule snapshot from above, and wait for it to finish. The inline sync must reuse that snapshot without fetching `specs` instructions again. Do not delegate it to a background task — step 5 would move `changeRoot` out from under a sync that is still reading it, leaving the change archived and the main specs never updated. If your agent can only run it by delegation, delegate synchronously and wait for the result. - - Then re-run the comparison from the top of this step against every capability that has a delta spec in `artifactPaths.specs.existingOutputPaths` — not only the ones the sync reports it touched. A successful sync leaves nothing left to apply, so each capability must now read as already synced: + selected-root flags. Require a zero exit status and valid + artifact-instruction JSON. If the lookup fails or returns invalid JSON, + report the error and stop before writing any main spec or moving the change. + A valid response with omitted `rules` is the no-rules case. Apply returned + `rules` only to the content and form of main specs produced by this merge; do + not use them as archive guidance, change CLI behavior, or copy the rule text + into any output file. + + Then run the `openspec-sync-specs` workflow inline (agent-driven intelligent + merge) for change '', passing the delta spec analysis and the fetched + specs-rule snapshot from above, and wait for it to finish. The inline sync + must reuse that snapshot without fetching `specs` instructions again. Do not + delegate it to a background task — step 5 would move `changeRoot` out from + under a sync that is still reading it, leaving the change archived and the + main specs never updated. If your agent can only run it by delegation, + delegate synchronously and wait for the result. + + Then re-run the comparison from the top of this step against every capability + that has a delta spec in `artifactPaths.specs.existingOutputPaths` — not only + the ones the sync reports it touched. A successful sync leaves nothing left + to apply, so each capability must now read as already synced: - ADDED requirements present - - MODIFIED requirements carrying the scenario and description changes named in the delta, with their other scenarios intact - - REMOVED requirements gone — and where this sync retired a capability (removed its last requirement, leaving `## Requirements` empty), its main spec deleted rather than left empty; a spec the sync deliberately kept and reported is also a match - - RENAMED requirements present under the new name and absent under the old one - - If the sync failed, or any capability does not match, report what differs and stop — do not archive. Nothing has moved and `changeRoot` is intact, so the user can fix the mismatch or re-run the sync and start the archive again. + - MODIFIED requirements carrying the scenario and description changes named + in the delta, with their other scenarios intact + - REMOVED requirements gone — and where this sync retired a capability + (removed its last requirement, leaving `## Requirements` empty), its main + spec deleted rather than left empty; a spec the sync deliberately kept and + reported is also a match + - RENAMED requirements present under the new name and absent under the old + one + + If the sync failed, or any capability does not match, report what differs and + stop — do not archive. Nothing has moved and `changeRoot` is intact, so the + user can fix the mismatch or re-run the sync and start the archive again. 5. **Perform the archive** - Create an `archive` directory under `planningHome.changesDir` if it doesn't exist: + Create an `archive` directory under `planningHome.changesDir` if it doesn't + exist: + ```bash mkdir -p "/archive" ``` - Generate the target name: use the change name as-is when it already starts with a `YYYY-MM-DD-` prefix; otherwise prepend the current date as `YYYY-MM-DD-`. Never stack a second date (same rule as `openspec archive`). + Generate the target name: use the change name as-is when it already starts + with a `YYYY-MM-DD-` prefix; otherwise prepend the current date as + `YYYY-MM-DD-`. Never stack a second date (same rule as + `openspec archive`). **Check if target already exists:** - - If yes: Fail with error, suggest renaming existing archive or using different date + - If yes: Fail with error, suggest renaming existing archive or using + different date - If no: Move `changeRoot` to the archive directory ```bash @@ -167,16 +218,25 @@ Archive a completed change in the experimental workflow. ``` **Guardrails** + - Announce the selected change; prompt for selection when it is ambiguous - Use artifact graph (openspec status --json) for completion checking - Don't block archive on warnings - just inform and confirm - Preserve .openspec.yaml when moving to archive (it moves with the directory) - Show clear summary of what happened -- If sync is requested, run the `openspec-sync-specs` workflow inline (agent-driven) -- Never archive while a spec sync is still in flight — run the sync inline and verify the main specs before moving `changeRoot` -- If delta specs exist, always run the sync assessment and show the combined summary before prompting -- Apply relevant runtime context and report conflicts; operation guidance remains advisory -- Consider every guidance entry and explain any inapplicable or conflicting advice -- Existing CLI checks, resolved paths, prompts, and command contracts are unchanged -- Artifact rules constrain only the specs being written and are never operation guidance -- Never copy runtime context, operation guidance, or artifact-rule text verbatim into output files +- If sync is requested, run the `openspec-sync-specs` workflow inline + (agent-driven) +- Never archive while a spec sync is still in flight — run the sync inline and + verify the main specs before moving `changeRoot` +- If delta specs exist, always run the sync assessment and show the combined + summary before prompting +- Apply relevant runtime context and report conflicts; operation guidance + remains advisory +- Consider every guidance entry and explain any inapplicable or conflicting + advice +- Existing CLI checks, resolved paths, prompts, and command contracts are + unchanged +- Artifact rules constrain only the specs being written and are never operation + guidance +- Never copy runtime context, operation guidance, or artifact-rule text verbatim + into output files diff --git a/.agents/skills/openspec-explore/SKILL.md b/.agents/skills/openspec-explore/SKILL.md index 28bb0ee..fd999c8 100644 --- a/.agents/skills/openspec-explore/SKILL.md +++ b/.agents/skills/openspec-explore/SKILL.md @@ -10,20 +10,48 @@ metadata: generatedBy: "1.12.0" --- -Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes. - -**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, investigate the codebase, and run read-only commands or tools without confirmation, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create or update OpenSpec change artifacts (proposals, designs, specs) within a confirmed scope—that's capturing thinking, not implementing. Answering design or clarifying questions is never consent to write. Before the first write-capable action, name the artifacts or files you would change and what you would do, ask a direct yes/no question, and wait for the user's confirmation in a separate message. Confirmation covers only the scope you described; ask again before expanding it. For a new change, scaffold it first as described below. - -**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore. - -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. +Enter explore mode. Think deeply. Visualize freely. Follow the conversation +wherever it goes. + +**IMPORTANT: Explore mode is for thinking, not implementing.** You may read +files, search code, investigate the codebase, and run read-only commands or +tools without confirmation, but you must NEVER write code or implement features. +If the user asks you to implement something, remind them to exit explore mode +first and create a change proposal. You MAY create or update OpenSpec change +artifacts (proposals, designs, specs) within a confirmed scope—that's capturing +thinking, not implementing. Answering design or clarifying questions is never +consent to write. Before the first write-capable action, name the artifacts or +files you would change and what you would do, ask a direct yes/no question, and +wait for the user's confirmation in a separate message. Confirmation covers only +the scope you described; ask again before expanding it. For a new change, +scaffold it first as described below. + +**This is a stance, not a workflow.** There are no fixed steps, no required +sequence, no mandatory outputs. You're a thinking partner helping the user +explore. + +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. --- ## The Stance -- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script -- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions. +- **Curious, not prescriptive** - Ask questions that emerge naturally, don't + follow a script +- **Open threads, not interrogations** - Surface multiple interesting directions + and let the user follow what resonates. Don't funnel them through a single + path of questions. - **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking - **Adaptive** - Follow interesting threads, pivot when new information emerges - **Patient** - Don't rush to conclusions, let the shape of the problem emerge @@ -33,16 +61,37 @@ Enter explore mode. Think deeply. Visualize freely. Follow the conversation wher ## Planning a Change -When the user is planning a change, guide them toward shared understanding with focused discovery questions. For open-ended discussion, follow the conversation without imposing an interview or a required output. - -Before asking a factual question, follow the context discovery below and inspect relevant OpenSpec artifacts, source, tests, docs, and configuration. Do not ask the user to repeat facts you can verify. Summarize relevant findings without reproducing private context or rules. If evidence is missing, conflicting, or inaccessible, state that limitation and ask only for the clarification needed to proceed. - -- **Follow dependencies** - Resolve the next blocking decision before its dependent details. For example, clarify the user's outcome and scope before choosing an API or data model. Revisit downstream assumptions when an earlier answer changes. Skip branches that do not matter to this goal. -- **Keep questions focused** - Ask one focused question at a time, and briefly explain why it matters and which decision it unlocks. Batch questions only if the user asks for a batch; keep them small and group related decisions. -- **Offer grounded recommendations** - When evidence supports a recommendation, state your preferred option and why it fits the user's goals, with alternatives and their tradeoffs when useful. Do not invent intent, priorities, or external constraints: ask the user when only they can answer. Avoid a fixed question format. -- **Keep a conversational record** - Track decisions in the conversation, not in files. Separate confirmed decisions from proposed defaults and unresolved questions. Silence is not acceptance. Accepting an answer or a batch of recommendations is not permission to write. Keep file-write confirmation separate from discovery questions and follow the guardrails below. - -Stop asking when the user has enough clarity. Let them pause, pivot, or defer a decision; do not exhaust every branch or force a proposal. +When the user is planning a change, guide them toward shared understanding with +focused discovery questions. For open-ended discussion, follow the conversation +without imposing an interview or a required output. + +Before asking a factual question, follow the context discovery below and inspect +relevant OpenSpec artifacts, source, tests, docs, and configuration. Do not ask +the user to repeat facts you can verify. Summarize relevant findings without +reproducing private context or rules. If evidence is missing, conflicting, or +inaccessible, state that limitation and ask only for the clarification needed to +proceed. + +- **Follow dependencies** - Resolve the next blocking decision before its + dependent details. For example, clarify the user's outcome and scope before + choosing an API or data model. Revisit downstream assumptions when an earlier + answer changes. Skip branches that do not matter to this goal. +- **Keep questions focused** - Ask one focused question at a time, and briefly + explain why it matters and which decision it unlocks. Batch questions only if + the user asks for a batch; keep them small and group related decisions. +- **Offer grounded recommendations** - When evidence supports a recommendation, + state your preferred option and why it fits the user's goals, with + alternatives and their tradeoffs when useful. Do not invent intent, + priorities, or external constraints: ask the user when only they can answer. + Avoid a fixed question format. +- **Keep a conversational record** - Track decisions in the conversation, not in + files. Separate confirmed decisions from proposed defaults and unresolved + questions. Silence is not acceptance. Accepting an answer or a batch of + recommendations is not permission to write. Keep file-write confirmation + separate from discovery questions and follow the guardrails below. + +Stop asking when the user has enough clarity. Let them pause, pivot, or defer a +decision; do not exhaust every branch or force a proposal. For example, after inspecting the relevant code: @@ -60,25 +109,29 @@ adding a service to operate; shared state would need a separate sync design. Depending on what the user brings, you might: **Explore the problem space** + - Ask clarifying questions that emerge from what they said - Challenge assumptions - Reframe the problem - Find analogies **Investigate the codebase** + - Map existing architecture relevant to the discussion - Find integration points - Identify patterns already in use - Surface hidden complexity **Compare options** + - Brainstorm multiple approaches - Build comparison tables - Sketch tradeoffs - Recommend a path (if asked) **Visualize** -``` + +```text +------------------------------------------+ | Use ASCII diagrams liberally | +------------------------------------------+ @@ -95,10 +148,13 @@ Depending on what the user brings, you might: +------------------------------------------+ ``` -**Draw with plain ASCII only** — borders `+` `-` `|`, arrows `-->` `<--` `^` `v`, markers `*` `x`. -Unicode diagram glyphs can render at different widths across terminals, fonts, and locales, so padded boxes and aligned tables can drift. Keep every diagram character ASCII. +**Draw with plain ASCII only** — borders `+` `-` `|`, arrows `-->` `<--` `^` +`v`, markers `*` `x`. Unicode diagram glyphs can render at different widths +across terminals, fonts, and locales, so padded boxes and aligned tables can +drift. Keep every diagram character ASCII. **Surface risks and unknowns** + - Identify what could go wrong - Find gaps in understanding - Suggest spikes or investigations @@ -112,20 +168,28 @@ You have full context of the OpenSpec system. Use it naturally, don't force it. ### Check for context At the start, quickly check what exists: + ```bash openspec list --json ``` This tells you: + - If there are active changes - Their names, schemas, and status - What the user might be working on -Then read the project's own context from the resolved root - `/openspec/config.yaml` (or `config.yml`). Use the `root.path` returned above, and skip this if neither file exists: +Then read the project's own context from the resolved root - +`/openspec/config.yaml` (or `config.yml`). Use the `root.path` +returned above, and skip this if neither file exists: + - `context`: project background - tech stack, conventions, constraints -- `rules`: keyed by artifact id - the entries for an artifact apply only when you write that artifact +- `rules`: keyed by artifact id - the entries for an artifact apply only when + you write that artifact -Ground your thinking in these. They are constraints for you to follow, not content to reproduce: do NOT copy them into the conversation or into any artifact you create. +Ground your thinking in these. They are constraints for you to follow, not +content to reproduce: do NOT copy them into the conversation or into any +artifact you create. ### When no change exists @@ -134,14 +198,57 @@ Think freely. When insights crystallize, you might offer: - "This feels solid enough to start a change. Want me to create a proposal?" - Or keep exploring - no pressure to formalize -If the user asks you to capture the exploration as a new change, transition seamlessly into the requested capture: - -1. Run `openspec new change ""` (with `--store ` when applicable) before creating any artifacts. Never create a new change directory under `openspec/changes/` by hand; the CLI scaffold creates required metadata such as `.openspec.yaml`. Keep the selected `--store ` on every applicable follow-up `status` and `instructions` command. -2. Run `openspec status --change "" --json` (append the confirmed `--store ""` only for a registered standalone store), then process the requested artifacts in dependency order. For each requested artifact that is `ready`, run `openspec instructions "" --change "" --json` (append the confirmed `--store ""` only for a registered standalone store). Before creating a requested artifact, evaluate any condition in its own `instruction` against the explored change; record a deliberate skip instead when the condition does not apply. If a requested artifact is blocked by a direct prerequisite the user did not request, run `openspec instructions "" --change "" --json` (append the confirmed `--store ""` only for a registered standalone store) for that prerequisite whether it is `ready` or `blocked`. If its own `instruction` states a condition, evaluate that condition against the explored change and record a deliberate skip only when the condition does not apply. If the condition applies, or the prerequisite is not conditional, treat it as a normal prerequisite and ask before expanding the capture. Do not create an unrequested prerequisite unless the user approves. -3. Follow the returned `template` and `instruction` fields. Read completed dependency files listed in `dependencies`, and apply `context` and `rules` as constraints without copying them into the artifact. If the instruction delegates creation to a specific skill or command, invoke it; otherwise write the artifact to `resolvedOutputPath`, using the instruction to choose a concrete path when it is a glob. Verify that the selected concrete output exists. -4. After creating each artifact, re-run `openspec status --change "" --json` (append the confirmed `--store ""` only for a registered standalone store) and continue until every requested artifact is `done`, `skipped`, or was deliberately skipped because its own `instruction` stated a condition that did not apply. Tell the user about a deliberate conditional skip, remember it, and do not reconsider it. Dependencies are enablers, not gates: if a requested artifact is still `blocked` only because you deliberately skipped a conditional prerequisite, run `openspec instructions "" --change "" --json` (append the confirmed `--store ""` only for a registered standalone store) despite the blocked status, then create it using step 3 only when those recorded conditional skips are its sole missing dependencies. If a requested artifact is blocked by a prerequisite the user did not ask to capture and cannot be conditionally skipped, explain that dependency and ask before expanding the capture. - -Capture the artifact(s) the user requested without asking them to invoke another workflow command. If they asked only to start a change, stop after scaffolding and show its status. +If the user asks you to capture the exploration as a new change, transition +seamlessly into the requested capture: + +1. Run `openspec new change ""` (with `--store ` when applicable) + before creating any artifacts. Never create a new change directory under + `openspec/changes/` by hand; the CLI scaffold creates required metadata such + as `.openspec.yaml`. Keep the selected `--store ` on every applicable + follow-up `status` and `instructions` command. +2. Run `openspec status --change "" --json` (append the confirmed + `--store ""` only for a registered standalone store), then process the + requested artifacts in dependency order. For each requested artifact that is + `ready`, run `openspec instructions "" --change "" --json` + (append the confirmed `--store ""` only for a registered standalone + store). Before creating a requested artifact, evaluate any condition in its + own `instruction` against the explored change; record a deliberate skip + instead when the condition does not apply. If a requested artifact is blocked + by a direct prerequisite the user did not request, run + `openspec instructions "" --change "" --json` (append + the confirmed `--store ""` only for a registered standalone store) for + that prerequisite whether it is `ready` or `blocked`. If its own + `instruction` states a condition, evaluate that condition against the + explored change and record a deliberate skip only when the condition does not + apply. If the condition applies, or the prerequisite is not conditional, + treat it as a normal prerequisite and ask before expanding the capture. Do + not create an unrequested prerequisite unless the user approves. +3. Follow the returned `template` and `instruction` fields. Read completed + dependency files listed in `dependencies`, and apply `context` and `rules` as + constraints without copying them into the artifact. If the instruction + delegates creation to a specific skill or command, invoke it; otherwise write + the artifact to `resolvedOutputPath`, using the instruction to choose a + concrete path when it is a glob. Verify that the selected concrete output + exists. +4. After creating each artifact, re-run + `openspec status --change "" --json` (append the confirmed + `--store ""` only for a registered standalone store) and continue until + every requested artifact is `done`, `skipped`, or was deliberately skipped + because its own `instruction` stated a condition that did not apply. Tell the + user about a deliberate conditional skip, remember it, and do not reconsider + it. Dependencies are enablers, not gates: if a requested artifact is still + `blocked` only because you deliberately skipped a conditional prerequisite, + run `openspec instructions "" --change "" --json` (append + the confirmed `--store ""` only for a registered standalone store) + despite the blocked status, then create it using step 3 only when those + recorded conditional skips are its sole missing dependencies. If a requested + artifact is blocked by a prerequisite the user did not ask to capture and + cannot be conditionally skipped, explain that dependency and ask before + expanding the capture. + +Capture the artifact(s) the user requested without asking them to invoke another +workflow command. If they asked only to start a change, stop after scaffolding +and show its status. ### When a change exists @@ -149,25 +256,30 @@ If the user mentions a change or you detect one is relevant: 1. **Resolve and read existing artifacts for context** - Run `openspec status --change "" --json`. - - Use `changeRoot`, `artifactPaths`, and `actionContext` from the status JSON. + - Use `changeRoot`, `artifactPaths`, and `actionContext` from the status + JSON. - Read existing files from `artifactPaths..existingOutputPaths`. 2. **Reference them naturally in conversation** - - "Your design mentions using Redis, but we just realized SQLite fits better..." - - "The proposal scopes this to premium users, but we're now thinking everyone..." + - "Your design mentions using Redis, but we just realized SQLite fits + better..." + - "The proposal scopes this to premium users, but we're now thinking + everyone..." 3. **Offer to capture when decisions are made** - `` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve an existing capability's full path and follow the project's established organization for new capabilities. + `` is the spec directory relative to `specs/` (for example, + `user-auth` or `identity/user-auth`). Preserve an existing capability's full + path and follow the project's established organization for new capabilities. - | Insight Type | Where to Capture | - |----------------------------|-------------------------------------| - | New requirement discovered | `specs//spec.md` | - | Requirement changed | `specs//spec.md` | - | Design decision made | `design.md` | - | Scope changed | `proposal.md` | - | New work identified | `tasks.md` | - | Assumption invalidated | Relevant artifact | + | Insight Type | Where to Capture | + |----------------------------|-------------------------------------| + | New requirement discovered | `specs//spec.md` | + | Requirement changed | `specs//spec.md` | + | Design decision made | `design.md` | + | Scope changed | `proposal.md` | + | New work identified | `tasks.md` | + | Assumption invalidated | Relevant artifact | Example offers: - "That's a design decision. Capture it in design.md?" @@ -192,7 +304,8 @@ If the user mentions a change or you detect one is relevant: ## Handling Different Entry Points **User brings a vague idea:** -``` + +```text User: I'm thinking about adding real-time collaboration You: Real-time collab is a big space. Let me think about this... @@ -215,7 +328,8 @@ You: Real-time collab is a big space. Let me think about this... ``` **User brings a specific problem:** -``` + +```text User: The auth system is a mess You: [reads codebase] @@ -246,7 +360,8 @@ You: [reads codebase] ``` **User is stuck mid-implementation:** -``` + +```text User: $openspec-explore (Codex) or /openspec-explore (other agents) add-auth-system The OAuth integration is more complex than expected @@ -263,7 +378,8 @@ You: [reads change artifacts] ``` **User wants to compare options:** -``` + +```text User: Should we use Postgres or SQLite? You: Generic answer is boring. What's the context? @@ -304,7 +420,7 @@ There's no required ending. Discovery might: When it feels like things are crystallizing, you might summarize: -``` +```text ## What We Figured Out **The problem**: [crystallized understanding] @@ -324,12 +440,25 @@ But this summary is optional. Sometimes the thinking IS the value. ## Guardrails -- **Don't implement** - Never write code or implement features. Workflow configuration counts too: creating or editing schemas, templates, or `openspec/config.yaml` is a change, not thinking. Creating or updating OpenSpec change artifacts within the confirmed scope is fine, writing anything else is not. +- **Don't implement** - Never write code or implement features. Workflow + configuration counts too: creating or editing schemas, templates, or + `openspec/config.yaml` is a change, not thinking. Creating or updating + OpenSpec change artifacts within the confirmed scope is fine, writing anything + else is not. - **Don't fake understanding** - If something is unclear, dig deeper - **Don't rush** - Discovery is thinking time, not task time - **Don't force structure** - Let patterns emerge naturally -- **Don't auto-capture** - Offer to save insights, don't just do it. Read-only commands and tools need no confirmation. Before the first write-capable action—including `openspec new change` or another command that writes files—name the artifacts or files and proposed changes, ask a direct yes/no question, and wait for explicit confirmation in a separate user message. That confirmation covers only the described scope; ask again before expanding it. Answers to design or clarifying questions are never consent to write. -- **Don't manually scaffold changes** - Never create a new change directory under `openspec/changes/` by hand. Always use `openspec new change ""` (with `--store ` when applicable) so required metadata such as `.openspec.yaml` is created before writing artifacts. +- **Don't auto-capture** - Offer to save insights, don't just do it. Read-only + commands and tools need no confirmation. Before the first write-capable + action—including `openspec new change` or another command that writes + files—name the artifacts or files and proposed changes, ask a direct yes/no + question, and wait for explicit confirmation in a separate user message. That + confirmation covers only the described scope; ask again before expanding it. + Answers to design or clarifying questions are never consent to write. +- **Don't manually scaffold changes** - Never create a new change directory + under `openspec/changes/` by hand. Always use `openspec new change ""` + (with `--store ` when applicable) so required metadata such as + `.openspec.yaml` is created before writing artifacts. - **Do visualize** - A good diagram is worth many paragraphs - **Do explore the codebase** - Ground discussions in reality - **Do question assumptions** - Including the user's and your own diff --git a/.agents/skills/openspec-propose/SKILL.md b/.agents/skills/openspec-propose/SKILL.md index 0e4d83d..5a709d1 100644 --- a/.agents/skills/openspec-propose/SKILL.md +++ b/.agents/skills/openspec-propose/SKILL.md @@ -12,115 +12,216 @@ metadata: Propose a new change - create the change and generate all artifacts in one step. -**Planning boundary**: This workflow creates planning artifacts only. The user request that selected or triggered this workflow authorizes planning only, even if it asks to build or fix something. Do not edit project code. After the planning artifacts are complete, stop. Do not start implementation in the same response, even if the initial request asks for it. Wait for a new user request after the artifacts are presented; then start the apply workflow. +**Planning boundary**: This workflow creates planning artifacts only. The user +request that selected or triggered this workflow authorizes planning only, even +if it asks to build or fix something. Do not edit project code. After the +planning artifacts are complete, stop. Do not start implementation in the same +response, even if the initial request asks for it. Wait for a new user request +after the artifacts are presented; then start the apply workflow. + +I'll create a change with the artifacts your schema defines. With the default +spec-driven schema that is: -I'll create a change with the artifacts your schema defines. With the default spec-driven schema that is: - proposal.md (what & why) -- `specs//spec.md` (what the system must do - a delta, not the main spec) +- `specs//spec.md` (what the system must do - a delta, not the + main spec) - design.md (how) - tasks.md (implementation steps) -`` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve an existing capability's full path and follow the project's established organization for new capabilities. +`` is the spec directory relative to `specs/` (for example, +`user-auth` or `identity/user-auth`). Preserve an existing capability's full +path and follow the project's established organization for new capabilities. -When the user is ready to implement, they must start the apply workflow explicitly. +When the user is ready to implement, they must start the apply workflow +explicitly. --- -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -**Input**: The user's request should include a change name (kebab-case) OR a description of what they want to build. +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +**Input**: The user's request should include a change name (kebab-case) OR a +description of what they want to build. **Steps** 1. **Understand the request and clarify material ambiguity** If no clear input is provided, ask the user (open-ended, no preset options): - > "What change do you want to work on? Describe what you want to build or fix." + > "What change do you want to work on? Describe what you want to build or + > fix." - From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`). + From their description, derive a kebab-case name (e.g., "add user + authentication" → `add-user-auth`). - **IMPORTANT**: Do NOT proceed without understanding what the user wants to build. + **IMPORTANT**: Do NOT proceed without understanding what the user wants to + build. - If the request contains ambiguity that would materially affect scope, externally observable behavior, compatibility, or acceptance criteria, ask the user before creating the change. For minor details, make a reasonable assumption and record it in the planning artifacts. + If the request contains ambiguity that would materially affect scope, + externally observable behavior, compatibility, or acceptance criteria, ask + the user before creating the change. For minor details, make a reasonable + assumption and record it in the planning artifacts. 2. **Determine the workflow schema** - Use the configured default schema unless the user explicitly requests a different workflow. + Use the configured default schema unless the user explicitly requests a + different workflow. **Use a different schema only if the user:** - - Explicitly requests a specific schema by name → use `--schema ` - - Asks to "show workflows" or asks "what workflows" exist → resolve the authoritative root by running `openspec context --json` from the current working directory. If the user explicitly selected a registered store, use `openspec context --json --store ""`. Then run `openspec schemas --json` with its working directory set to the returned `root.path` and let them choose. This preserves roots selected by a local `store:` pointer or the global `defaultStore`; when a registered store was explicitly selected, append `--store ""` to `openspec schemas --json` as well. If context reports only `no_openspec_root`, run `openspec schemas --json` from the current working directory instead. Do not use this fallback for invalid or unavailable stores. + - Explicitly requests a specific schema by name → use + `--schema ` + - Asks to "show workflows" or asks "what workflows" exist → resolve the + authoritative root by running `openspec context --json` from the current + working directory. If the user explicitly selected a registered store, use + `openspec context --json --store ""`. Then run + `openspec schemas --json` with its working directory set to the returned + `root.path` and let them choose. This preserves roots selected by a local + `store:` pointer or the global `defaultStore`; when a registered store was + explicitly selected, append `--store ""` to + `openspec schemas --json` as well. If context reports only + `no_openspec_root`, run `openspec schemas --json` from the current working + directory instead. Do not use this fallback for invalid or unavailable + stores. Otherwise, omit `--schema` to preserve the configured default. 3. **Create the change directory** - Choose one schema form below. If a registered store is selected, append `--store ""` to that command and each later OpenSpec command shown below that accepts `--store`. + Choose one schema form below. If a registered store is selected, append + `--store ""` to that command and each later OpenSpec command shown + below that accepts `--store`. Using the configured default: + ```bash openspec new change "" ``` Using an explicitly requested schema: + ```bash openspec new change "" --schema "" ``` - This creates a scaffolded change in the planning home resolved by the CLI with `.openspec.yaml`. + + This creates a scaffolded change in the planning home resolved by the CLI + with `.openspec.yaml`. 4. **Get the artifact build order** + ```bash openspec status --change "" --json ``` + Parse the JSON to get: - - `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`) - - `artifacts`: list of all artifacts, each with its `status` and its `requires` edges (the artifact IDs it directly depends on) - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context. Use these instead of assuming repo-local paths. + - `applyRequires`: array of artifact IDs needed before implementation (e.g., + `["tasks"]`) + - `artifacts`: list of all artifacts, each with its `status` and its + `requires` edges (the artifact IDs it directly depends on) + - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path + and scope context. Use these instead of assuming repo-local paths. 5. **Create every artifact in the required set** Use a todo list to track progress through the artifacts. - Loop through artifacts in dependency order (artifacts with no pending dependencies first): + Loop through artifacts in dependency order (artifacts with no pending + dependencies first): a. **For each artifact that is `ready` (dependencies satisfied)**: - Get instructions: + ```bash openspec instructions --change "" --json ``` + - The instructions JSON includes: - - `context`: Project background (constraints for you - do NOT include in output) - - `rules`: Artifact-specific rules (constraints for you - do NOT include in output) + - `context`: Project background (constraints for you - do NOT include in + output) + - `rules`: Artifact-specific rules (constraints for you - do NOT include + in output) - `template`: The structure to use for your output file - `instruction`: Schema-specific guidance for this artifact type - - `skipped`/`warning`: present when the change declares skip_specs and this artifact must NOT be created - stop and pick another artifact + - `skipped`/`warning`: present when the change declares skip_specs and + this artifact must NOT be created - stop and pick another artifact - `resolvedOutputPath`: Resolved path or pattern to write the artifact - `dependencies`: Completed artifacts to read for context - - Read any completed dependency files for context - always re-read them from disk, even if you saw them earlier in the conversation (the user may have edited them) - - **Inspect the relevant project before drafting**: Read `context` and `rules` first, then inspect relevant implementation, nearby tests, configuration, and documentation outside `openspec/`. Keep inspection read-only and proportional to the change; reuse findings for later artifacts and inspect more only as needed. - - Identify the target project from the request and project context; the planning home may be separate from the code. If the target is unclear, ask. For greenfield or non-code changes, inspect the available structure and relevant documents. If source is unavailable, state the limitation and ask when it materially affects the plan. - - Ground scope, approach, and tasks in what you find. Distinguish observed behavior from assumptions and proposed additions; surface conflicts with existing specs instead of silently deciding which is correct. - - Do this discovery now, rather than leaving generic "explore the codebase" or "make a plan" tasks for implementation. Keep any necessary follow-up investigation specific to an unresolved question. - - If the `instruction` field delegates creation to a specific skill or command, invoke it to produce the artifact instead of writing the file yourself, then verify the artifact file exists at `resolvedOutputPath` - - Otherwise create the artifact file using `template` as the structure and write it to `resolvedOutputPath`. If `resolvedOutputPath` is a glob, follow `instruction` to choose the concrete file path - - Apply `context` and `rules` as constraints - but do NOT copy them into the file + - Read any completed dependency files for context - always re-read them + from disk, even if you saw them earlier in the conversation (the user + may have edited them) + - **Inspect the relevant project before drafting**: Read `context` and + `rules` first, then inspect relevant implementation, nearby tests, + configuration, and documentation outside `openspec/`. Keep inspection + read-only and proportional to the change; reuse findings for later + artifacts and inspect more only as needed. + - Identify the target project from the request and project context; the + planning home may be separate from the code. If the target is unclear, + ask. For greenfield or non-code changes, inspect the available + structure and relevant documents. If source is unavailable, state the + limitation and ask when it materially affects the plan. + - Ground scope, approach, and tasks in what you find. Distinguish + observed behavior from assumptions and proposed additions; surface + conflicts with existing specs instead of silently deciding which is + correct. + - Do this discovery now, rather than leaving generic "explore the + codebase" or "make a plan" tasks for implementation. Keep any + necessary follow-up investigation specific to an unresolved question. + - If the `instruction` field delegates creation to a specific skill or + command, invoke it to produce the artifact instead of writing the file + yourself, then verify the artifact file exists at `resolvedOutputPath` + - Otherwise create the artifact file using `template` as the structure and + write it to `resolvedOutputPath`. If `resolvedOutputPath` is a glob, + follow `instruction` to choose the concrete file path + - Apply `context` and `rules` as constraints - but do NOT copy them into + the file - Show brief progress: "Created " - b. **Continue until every artifact in the required set exists (not just `apply.requires`)** - - After creating each artifact, re-run `openspec status --change "" --json` - - The required set is `applyRequires` plus every artifact reachable from those by following the `requires` edges in `status --json` - walk them transitively (spec-driven closes over proposal, specs, design, tasks). Leave artifacts outside that set alone - - `status` is file-existence only, so an `applyRequires` artifact reading `done` does NOT mean its dependencies exist - writing `tasks.md` early marks `tasks` done while `specs` was never written. Use each artifact's `requires` edges, not its `status`, to build the required set: a `done` artifact still lists what it depends on - - An artifact already reading `status: "skipped"` is satisfied: the change declares `skip_specs` in `.openspec.yaml`, so its files must NOT exist. Never try to create one - - Create every artifact in the required set that is missing, then re-check - creating one can unblock others - - Skip one only when `status` already reports it `skipped`, or when its own `instruction` says it is conditional: run `openspec instructions --change "" --json` and skip only if its `instruction` field marks it optional (e.g. "create only if..."). Spec-driven's `design.md` qualifies; `specs` qualifies only via the `skipped` status above, never by your own judgment. Tell the user, and do not reconsider it - - Dependencies are enablers, not gates: if a required artifact is still `blocked` only because you skipped a conditional dependency, write it anyway - - Stop when every artifact in the required set is `done`, `skipped`, or was deliberately skipped + b. **Continue until every artifact in the required set exists (not just + `apply.requires`)** + - After creating each artifact, re-run + `openspec status --change "" --json` + - The required set is `applyRequires` plus every artifact reachable from + those by following the `requires` edges in `status --json` - walk them + transitively (spec-driven closes over proposal, specs, design, tasks). + Leave artifacts outside that set alone + - `status` is file-existence only, so an `applyRequires` artifact reading + `done` does NOT mean its dependencies exist - writing `tasks.md` early + marks `tasks` done while `specs` was never written. Use each artifact's + `requires` edges, not its `status`, to build the required set: a `done` + artifact still lists what it depends on + - An artifact already reading `status: "skipped"` is satisfied: the change + declares `skip_specs` in `.openspec.yaml`, so its files must NOT exist. + Never try to create one + - Create every artifact in the required set that is missing, then + re-check - creating one can unblock others + - Skip one only when `status` already reports it `skipped`, or when its + own `instruction` says it is conditional: run + `openspec instructions --change "" --json` and skip + only if its `instruction` field marks it optional (e.g. "create only + if..."). Spec-driven's `design.md` qualifies; `specs` qualifies only via + the `skipped` status above, never by your own judgment. Tell the user, + and do not reconsider it + - Dependencies are enablers, not gates: if a required artifact is still + `blocked` only because you skipped a conditional dependency, write it + anyway + - Stop when every artifact in the required set is `done`, `skipped`, or + was deliberately skipped c. **If an artifact requires user input** (unclear context): - Ask the user to clarify - Then continue with creation 6. **Show final status** + ```bash openspec status --change "" ``` @@ -128,26 +229,45 @@ When the user is ready to implement, they must start the apply workflow explicit **Output** After completing all artifacts, summarize: + - Change name and location -- List of artifacts created with brief descriptions, plus any conditional artifact you skipped and why +- List of artifacts created with brief descriptions, plus any conditional + artifact you skipped and why - What's ready: "All artifacts needed for implementation are ready." -- Prompt: "The artifacts are ready for review. When you are ready, run `$openspec-apply-change (Codex) or /openspec-apply-change (other agents)` or ask me to apply this change." +- Prompt: "The artifacts are ready for review. When you are ready, run + `$openspec-apply-change (Codex) or /openspec-apply-change (other agents)` or + ask me to apply this change." **Artifact Creation Guidelines** -- Follow the `instruction` field from `openspec instructions` for each artifact type - it is the authoritative guidance, even for familiar artifact names -- If the `instruction` field directs you to use a specific skill or command to create the artifact, invoke it instead of writing the artifact directly +- Follow the `instruction` field from `openspec instructions` for each artifact + type - it is the authoritative guidance, even for familiar artifact names +- If the `instruction` field directs you to use a specific skill or command to + create the artifact, invoke it instead of writing the artifact directly - The schema defines what each artifact should contain - follow it - Read dependency artifacts for context before creating new ones - Use `template` as the structure for your output file - fill in its sections -- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file - - Do NOT copy ``, ``, `` blocks into the artifact +- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for + the file + - Do NOT copy ``, ``, `` blocks into the + artifact - These guide what you write, but should never appear in the output **Guardrails** -- The request that invoked this workflow authorizes planning only. Any implementation or apply instruction in that request does not carry forward. Do NOT implement the change, start the apply workflow, or edit project code during this workflow. After presenting the artifacts, stop and wait for a new user request to start the apply workflow -- Create every artifact the apply phase transitively depends on, not just the ids listed in `apply.requires` -- Always read dependency artifacts before creating a new one - re-read from disk, not from conversation memory (files may have changed since you last saw them) -- Ask about ambiguities that would materially change scope, externally observable behavior, compatibility, or acceptance criteria; for minor details, make reasonable assumptions and record them -- If a change with that name already exists, ask if user wants to continue it or create a new one + +- The request that invoked this workflow authorizes planning only. Any + implementation or apply instruction in that request does not carry forward. Do + NOT implement the change, start the apply workflow, or edit project code + during this workflow. After presenting the artifacts, stop and wait for a new + user request to start the apply workflow +- Create every artifact the apply phase transitively depends on, not just the + ids listed in `apply.requires` +- Always read dependency artifacts before creating a new one - re-read from + disk, not from conversation memory (files may have changed since you last saw + them) +- Ask about ambiguities that would materially change scope, externally + observable behavior, compatibility, or acceptance criteria; for minor details, + make reasonable assumptions and record them +- If a change with that name already exists, ask if user wants to continue it or + create a new one - Verify each artifact file exists after writing before proceeding to next diff --git a/.agents/skills/openspec-sync-specs/SKILL.md b/.agents/skills/openspec-sync-specs/SKILL.md index d399e02..ab1f35b 100644 --- a/.agents/skills/openspec-sync-specs/SKILL.md +++ b/.agents/skills/openspec-sync-specs/SKILL.md @@ -12,13 +12,30 @@ metadata: Sync delta specs from a change to main specs. -This is an **agent-driven** operation - you will read delta specs and directly edit main specs to apply the changes. This allows intelligent merging (e.g., adding a scenario without copying the entire requirement). - -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -`` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve the full path from each delta spec when resolving its main spec. - -**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. +This is an **agent-driven** operation - you will read delta specs and directly +edit main specs to apply the changes. This allows intelligent merging (e.g., +adding a scenario without copying the entire requirement). + +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +`` is the spec directory relative to `specs/` (for example, +`user-auth` or `identity/user-auth`). Preserve the full path from each delta +spec when resolving its main spec. + +**Input**: Optionally specify a change name. If omitted, check if it can be +inferred from conversation context. If vague or ambiguous you MUST prompt for +available changes. **Steps** @@ -27,20 +44,27 @@ This is an **agent-driven** operation - you will read delta specs and directly e If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes and ask + the user to select one - When prompting, show changes that have delta specs (under `specs/` directory). + When prompting, show changes that have delta specs (under `specs/` + directory). - Always announce: "Using change: " and how to override (e.g., `$openspec-sync-specs (Codex) or /openspec-sync-specs (other agents) `). + Always announce: "Using change: " and how to override (e.g., + `$openspec-sync-specs (Codex) or /openspec-sync-specs (other agents) `). 2. **Resolve change context** Run: + ```bash openspec status --change "" --json ``` - The JSON includes `planningHome.root`. Main specs live under `/openspec/specs/` — use that (store-aware) root for every main-spec path below, not a hardcoded repo path. When a store is selected it points at the store, not the current repository. + The JSON includes `planningHome.root`. Main specs live under + `/openspec/specs/` — use that (store-aware) root for every + main-spec path below, not a hardcoded repo path. When a store is selected it + points at the store, not the current repository. 3. **Find delta specs** @@ -80,29 +104,34 @@ This is an **agent-driven** operation - you will read delta specs and directly e fetch the same instructions again. - Otherwise run that command once now with the same selected-root flags. - If the direct lookup exits non-zero or returns invalid artifact-instruction - JSON, report the error and stop before writing any main spec. Do not treat the - failure as an absent rule set. - - A valid response with omitted `rules` means no artifact rules are configured - and the existing semantic merge continues. + JSON, report the error and stop before writing any main spec. Do not treat + the failure as an absent rule set. + - A valid response with omitted `rules` means no artifact rules are + configured and the existing semantic merge continues. - Apply returned `rules` only to the content and form of the main specs produced - by this merge. Artifact rules are not operation guidance and cannot change - selected roots, delta paths, CLI checks, or workflow steps. Use their text as - constraints without copying it verbatim into a main spec or summary. + Apply returned `rules` only to the content and form of the main specs + produced by this merge. Artifact rules are not operation guidance and cannot + change selected roots, delta paths, CLI checks, or workflow steps. Use their + text as constraints without copying it verbatim into a main spec or summary. - For each capability delta spec path selected in step 3 — the full `existingOutputPaths` list, or the narrowed subset when a caller supplied one (these may belong to a selected store, not the repo): + For each capability delta spec path selected in step 3 — the full + `existingOutputPaths` list, or the narrowed subset when a caller supplied one + (these may belong to a selected store, not the repo): a. **Read the delta spec** to understand the intended changes - b. **Read the main spec** at `/openspec/specs//spec.md` (may not exist yet) + b. **Read the main spec** at + `/openspec/specs//spec.md` (may not exist + yet) c. **Apply changes intelligently**: - **ADDED Requirements:** + **ADDED Requirements:** - If requirement doesn't exist in main spec → add it - - If requirement already exists → update it to match (treat as implicit MODIFIED) + - If requirement already exists → update it to match (treat as implicit + MODIFIED) - **MODIFIED Requirements:** + **MODIFIED Requirements:** - Find the requirement in main spec - Apply the changes - this can be: - Adding new scenarios the main spec does not have yet @@ -110,10 +139,10 @@ This is an **agent-driven** operation - you will read delta specs and directly e - Changing the requirement description - Preserve scenarios/content not mentioned in the delta - **REMOVED Requirements:** + **REMOVED Requirements:** - Remove the entire requirement block from main spec - - Retiring the capability. Delete the whole `spec.md` - and the directory once - nothing else is left in it - only when ALL of these hold: + - Retiring the capability. Delete the whole `spec.md` - and the directory + once nothing else is left in it - only when ALL of these hold: 1. removing the requirements *this run* left no requirement blocks; 2. the rest of the spec is well-formed (it still has a `## Purpose`); 3. the main spec was not already empty before this sync - if you removed @@ -124,35 +153,38 @@ This is an **agent-driven** operation - you will read delta specs and directly e 5. the change's `.openspec.yaml` declares `retire_capabilities: true`; 6. the `spec.md` resolves inside the real specs root (do not follow a capability-directory symlink to delete an external file). - If removing the selected requirements would leave no requirement blocks and - any retirement condition is not satisfied, do not modify the main spec. Stop - the sync for that capability, report the blocking condition, and tell the user - how to resolve it. Never write or leave an empty `## Requirements` section. - When only the marker is missing, say that too - it is the one thing the user - can add to make the retirement go through. - - Deleting the file also deletes its `## Purpose`; any other section blocks - retirement. Name Purpose when you report the retirement. Include a pasteable - `git checkout` only when the spec lived in the caller's checkout; - otherwise give checkout-scoped recovery guidance. - - **RENAMED Requirements:** + If removing the selected requirements would leave no requirement blocks + and any retirement condition is not satisfied, do not modify the main + spec. Stop the sync for that capability, report the blocking condition, + and tell the user how to resolve it. Never write or leave an empty + `## Requirements` section. When only the marker is missing, say that + too - it is the one thing the user can add to make the retirement go + through. + - Deleting the file also deletes its `## Purpose`; any other section + blocks retirement. Name Purpose when you report the retirement. Include + a pasteable `git checkout` only when the spec lived in the caller's + checkout; otherwise give checkout-scoped recovery guidance. + + **RENAMED Requirements:** - Find the FROM requirement, rename to TO - **`## Purpose` in the delta:** + **`## Purpose` in the delta:** - The main spec already has one and it is authoritative - leave it alone (this is what `openspec archive` does; it warns and moves on) d. **Create new main spec** if capability doesn't exist yet: - Create `/openspec/specs//spec.md` - - Add Purpose section: copy the delta's `## Purpose` body verbatim when it has one - (this is what `openspec archive` does); only write a brief TBD placeholder when it does not + - Add Purpose section: copy the delta's `## Purpose` body verbatim when it + has one (this is what `openspec archive` does); only write a brief TBD + placeholder when it does not - Add Requirements section with the ADDED requirements - Follow the **Main Spec Format Reference** below 5. **Validate updated main specs** - Run `openspec validate --specs` with the same selected-root flags used earlier. - If validation fails, report the problems and do not claim the sync succeeded. + Run `openspec validate --specs` with the same selected-root flags used + earlier. If validation fails, report the problems and do not claim the sync + succeeded. 6. **Show summary** @@ -205,7 +237,9 @@ The system SHALL keep doing the existing thing, now also handling A. **Main Spec Format Reference** -Main specs are what the delta merges INTO. They must never contain delta operation headers (`## ADDED/MODIFIED/REMOVED/RENAMED Requirements`) - after syncing, every requirement lives under a single `## Requirements` section: +Main specs are what the delta merges INTO. They must never contain delta +operation headers (`## ADDED/MODIFIED/REMOVED/RENAMED Requirements`) - after +syncing, every requirement lives under a single `## Requirements` section: ```markdown # Specification @@ -226,7 +260,10 @@ The system SHALL do something new. **Key Principle: Intelligent Merging** Unlike programmatic merging, you merge rather than overwrite: -- A MODIFIED block carries the whole requirement - body plus every scenario that survives the change. `openspec validate` and `openspec archive` both reject one that drops a scenario the main spec still has. + +- A MODIFIED block carries the whole requirement - body plus every scenario that + survives the change. `openspec validate` and `openspec archive` both reject + one that drops a scenario the main spec still has. - Keep anything the delta does not mention, in the main spec's existing order - Use your judgment to merge changes sensibly @@ -249,14 +286,22 @@ Main specs are now updated. The change remains active - archive when implementat ``` **Guardrails** + - Read both delta and main specs before making changes - Preserve existing content not mentioned in delta -- Never copy a delta file into a main spec as-is - merge its content so the main spec keeps the Main Spec Format Reference structure, with no delta operation headers +- Never copy a delta file into a main spec as-is - merge its content so the main + spec keeps the Main Spec Format Reference structure, with no delta operation + headers - If something is unclear, ask for clarification - Show what you're changing as you go - The operation should be idempotent - running twice should give same result -- Use only `artifactPaths.specs.existingOutputPaths`; never infer delta specs from unrelated artifacts -- Honor a caller-supplied subset of `existingOutputPaths`; never widen it back to the full list -- Fetch specs instructions once for direct sync, or reuse the archive-supplied snapshot inline -- Stop before every main-spec write on a non-zero or invalid JSON specs-instruction response -- Artifact rules constrain only the specs being written and are never copied into output files +- Use only `artifactPaths.specs.existingOutputPaths`; never infer delta specs + from unrelated artifacts +- Honor a caller-supplied subset of `existingOutputPaths`; never widen it back + to the full list +- Fetch specs instructions once for direct sync, or reuse the archive-supplied + snapshot inline +- Stop before every main-spec write on a non-zero or invalid JSON + specs-instruction response +- Artifact rules constrain only the specs being written and are never copied + into output files diff --git a/.agents/skills/openspec-update-change/SKILL.md b/.agents/skills/openspec-update-change/SKILL.md index e4a5dab..fd24923 100644 --- a/.agents/skills/openspec-update-change/SKILL.md +++ b/.agents/skills/openspec-update-change/SKILL.md @@ -10,13 +10,32 @@ metadata: generatedBy: "1.12.0" --- -Revise a change's existing planning artifacts and keep them coherent. Never edit code. - -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. - -`$openspec-continue-change (Codex) or /openspec-continue-change (other agents)` is an optional workflow and may not be installed. Before suggesting it anywhere below, verify that it is available. If it is unavailable, `openspec status --change "" --json` shows the next artifact and `openspec instructions "" --change "" --json` explains how to create it. +Revise a change's existing planning artifacts and keep them coherent. Never edit +code. + +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +**Input**: Optionally specify a change name. If omitted, check if it can be +inferred from conversation context. If vague or ambiguous you MUST prompt for +available changes. + +`$openspec-continue-change (Codex) or /openspec-continue-change (other agents)` +is an optional workflow and may not be installed. Before suggesting it anywhere +below, verify that it is available. If it is unavailable, +`openspec status --change "" --json` shows the next artifact and +`openspec instructions "" --change "" --json` explains how to +create it. **Steps** @@ -25,67 +44,119 @@ Revise a change's existing planning artifacts and keep them coherent. Never edit If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes sorted by most recently modified, and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes sorted by + most recently modified, and ask the user to select one - When prompting, present the top 3-4 most recently modified changes as options, showing: + When prompting, present the top 3-4 most recently modified changes as + options, showing: - Change name - Schema (from `schema` field if present, otherwise "spec-driven") - Status (e.g., "0/5 tasks", "complete", "no tasks") - How recently it was modified (from `lastModified` field) - Mark the most recently modified change as "(Recommended)" since it's likely what the user wants to update. + Mark the most recently modified change as "(Recommended)" since it's likely + what the user wants to update. - Always announce: "Using change: " and how to override (e.g., `$openspec-update-change (Codex) or /openspec-update-change (other agents) `). + Always announce: "Using change: " and how to override (e.g., + `$openspec-update-change (Codex) or /openspec-update-change (other agents) `). 2. **Get the change's artifacts** + ```bash openspec status --change "" --json ``` + Parse the JSON to understand current state. The response includes: - `schemaName`: The workflow schema being used (e.g., "spec-driven") - - `artifacts`: Array of artifacts with their status ("done", "skipped", "ready", "blocked") - - `isPlanningComplete`: Boolean indicating if all planning artifacts are complete. Older CLI versions expose the same value as `isComplete`. - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context. Use these instead of assuming repo-local paths. - - The artifact ids and paths come from the active schema - do NOT assume them, and do NOT branch on hardcoded artifact names. Custom schemas must work unchanged. - - The files to edit are `artifactPaths..existingOutputPaths` - the concrete files that exist on disk, already glob-expanded for glob artifacts (e.g. `specs/**/*.md`). Do NOT write to `resolvedOutputPath`: for a glob artifact it is still the glob pattern, not a real file. + - `artifacts`: Array of artifacts with their status ("done", "skipped", + "ready", "blocked") + - `isPlanningComplete`: Boolean indicating if all planning artifacts are + complete. Older CLI versions expose the same value as `isComplete`. + - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path + and scope context. Use these instead of assuming repo-local paths. + + The artifact ids and paths come from the active schema - do NOT assume them, + and do NOT branch on hardcoded artifact names. Custom schemas must work + unchanged. + + The files to edit are `artifactPaths..existingOutputPaths` - the concrete + files that exist on disk, already glob-expanded for glob artifacts (e.g. + `specs/**/*.md`). Do NOT write to `resolvedOutputPath`: for a glob artifact + it is still the glob pattern, not a real file. 3. **Understand the request** - - If the user asked for a specific revision ("the design now uses X"), that is the starting edit. - - If they only said "update" / "make this coherent", treat it as a coherence review: read the existing artifacts and check them against each other for contradictions, gaps, and duplication. + - If the user asked for a specific revision ("the design now uses X"), that + is the starting edit. + - If they only said "update" / "make this coherent", treat it as a coherence + review: read the existing artifacts and check them against each other for + contradictions, gaps, and duplication. 4. **Read and reconcile** - - Read the artifact(s) the request touches and the change's other existing artifacts. - - Apply the requested edit. Then check every other existing artifact against it - in ANY direction: an edit to a later artifact may require revising an earlier one, not only the other way around. Build order is a useful reading order, not a constraint on which artifacts may be revised. + - Read the artifact(s) the request touches and the change's other existing + artifacts. + - Apply the requested edit. Then check every other existing artifact against + it - in ANY direction: an edit to a later artifact may require revising an + earlier one, not only the other way around. Build order is a useful reading + order, not a constraint on which artifacts may be revised. - Note everything that is now inconsistent, missing, or contradictory. - - Revise only files that already exist (`existingOutputPaths`). Do NOT create artifacts that don't exist yet, and do NOT invent new files under a glob artifact - note them and point the user to `$openspec-continue-change (Codex) or /openspec-continue-change (other agents)` to create them. + - Revise only files that already exist (`existingOutputPaths`). Do NOT create + artifacts that don't exist yet, and do NOT invent new files under a glob + artifact - note them and point the user to + `$openspec-continue-change (Codex) or /openspec-continue-change (other agents)` + to create them. - If the change is already coherent, say so and make no edits. 5. **Confirm and apply, one artifact at a time** - Show each proposed revision and why. Write only after the user confirms. - - If the user rejects a revision, do not write it - leave that artifact unchanged. - - When a substantial rewrite is needed, get that artifact's rules and template first: + - If the user rejects a revision, do not write it - leave that artifact + unchanged. + - When a substantial rewrite is needed, get that artifact's rules and + template first: + ```bash openspec instructions "" --change "" --json ``` 6. **Point to the next step (guidance only - NEVER act on it)** - - Artifacts still missing -> suggest `$openspec-continue-change (Codex) or /openspec-continue-change (other agents)` to create them. - - Change already implemented (tasks checked off / already applied) -> the code may no longer match the revised plan; suggest `$openspec-apply-change (Codex) or /openspec-apply-change (other agents)` to carry the delta into code. - - Everything done and implemented -> suggest `$openspec-archive-change (Codex) or /openspec-archive-change (other agents)`. + - Artifacts still missing -> suggest + `$openspec-continue-change (Codex) or /openspec-continue-change (other agents)` + to create them. + - Change already implemented (tasks checked off / already applied) -> the + code may no longer match the revised plan; suggest + `$openspec-apply-change (Codex) or /openspec-apply-change (other agents)` + to carry the delta into code. + - Everything done and implemented -> suggest + `$openspec-archive-change (Codex) or /openspec-archive-change (other agents)`. **Output** After each invocation, show: + - Which artifacts were revised (and which proposed revisions were rejected) -- Anything deferred to `$openspec-continue-change (Codex) or /openspec-continue-change (other agents)` (not-yet-created artifacts or files) +- Anything deferred to + `$openspec-continue-change (Codex) or /openspec-continue-change (other agents)` + (not-yet-created artifacts or files) - Where the change stands and the recommended next command **Guardrails** -- Planning artifacts only - NEVER edit implementation code. If the revised plan implies code changes, stop and point to `$openspec-apply-change (Codex) or /openspec-apply-change (other agents)`. -- Use the artifact ids and paths reported by `openspec status`; never branch on hardcoded artifact names. -- Edit only the concrete files in `existingOutputPaths`; never write to a glob `resolvedOutputPath`. -- Do not advance the build frontier: no new artifacts, no new files under glob artifacts - that is `$openspec-continue-change (Codex) or /openspec-continue-change (other agents)`'s job. + +- Planning artifacts only - NEVER edit implementation code. If the revised plan + implies code changes, stop and point to + `$openspec-apply-change (Codex) or /openspec-apply-change (other agents)`. +- Use the artifact ids and paths reported by `openspec status`; never branch on + hardcoded artifact names. +- Edit only the concrete files in `existingOutputPaths`; never write to a glob + `resolvedOutputPath`. +- Do not advance the build frontier: no new artifacts, no new files under glob + artifacts - that is + `$openspec-continue-change (Codex) or /openspec-continue-change (other agents)`'s + job. - Confirm every edit with the user before writing. -- If the request changes the change's *intent* rather than refining it, first verify whether the optional `$openspec-new-change (Codex) or /openspec-new-change (other agents)` workflow is available. If it is, recommend starting fresh with `$openspec-new-change (Codex) or /openspec-new-change (other agents)` (the "Update vs. Start Fresh" heuristic). If it is unavailable, ask for a distinct unused change name and recommend `openspec new change ""` instead. +- If the request changes the change's *intent* rather than refining it, first + verify whether the optional + `$openspec-new-change (Codex) or /openspec-new-change (other agents)` workflow + is available. If it is, recommend starting fresh with + `$openspec-new-change (Codex) or /openspec-new-change (other agents)` (the + "Update vs. Start Fresh" heuristic). If it is unavailable, ask for a distinct + unused change name and recommend `openspec new change ""` + instead. diff --git a/.agents/workflows/opsx-apply.md b/.agents/workflows/opsx-apply.md index b3d57f3..3c6a32b 100644 --- a/.agents/workflows/opsx-apply.md +++ b/.agents/workflows/opsx-apply.md @@ -4,9 +4,22 @@ description: "Implement tasks from an OpenSpec change (Experimental)" Implement tasks from an OpenSpec change. -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -**Input**: Optionally specify a change name (e.g., `/opsx-apply add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +**Input**: Optionally specify a change name (e.g., `/opsx-apply add-auth`). If +omitted, check if it can be inferred from conversation context. If vague or +ambiguous you MUST prompt for available changes. **Steps** @@ -15,18 +28,24 @@ Implement tasks from an OpenSpec change. If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes and ask + the user to select one - Always announce: "Using change: " and how to override (e.g., `/opsx-apply `). + Always announce: "Using change: " and how to override (e.g., + `/opsx-apply `). 2. **Check status to understand the schema** + ```bash openspec status --change "" --json ``` + Parse the JSON to understand: - `schemaName`: The workflow being used (e.g., "spec-driven") - - `planningHome`, `changeRoot`, and `actionContext`: planning scope and edit constraints - - Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others) + - `planningHome`, `changeRoot`, and `actionContext`: planning scope and edit + constraints + - Which artifact contains the tasks (typically "tasks" for spec-driven, check + status for others) 3. **Get apply instructions** @@ -35,23 +54,30 @@ Implement tasks from an OpenSpec change. ``` This returns: - - `contextFiles`: artifact ID -> array of concrete file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs) + - `contextFiles`: artifact ID -> array of concrete file paths (varies by + schema - could be proposal/specs/design/tasks or + spec/tests/implementation/docs) - Progress (total, complete, remaining) - Task list with status - Dynamic instruction based on current state - - Optional `context`: current required project instruction input from the selected root + - Optional `context`: current required project instruction input from the + selected root - Optional `operationGuidance`: current advisory guidance for apply **Handle states:** - - If `state: "blocked"` (missing artifacts): show message, suggest using `/opsx-continue` (if it is not installed, run `openspec status --change "" --json` to see the next artifact and `openspec instructions --change "" --json` for how to create it) + - If `state: "blocked"` (missing artifacts): show message, suggest using + `/opsx-continue` (if it is not installed, run + `openspec status --change "" --json` to see the next artifact and + `openspec instructions --change "" --json` for how to + create it) - If `state: "all_done"`: congratulate, suggest archive - Otherwise: proceed to implementation Treat `context` as a required prompt-level input. Read and consider it, and - apply relevant project facts, conventions, and constraints while implementing. - Treat `operationGuidance` as optional additive advice. Read and consider every - entry, and follow entries that are applicable and compatible with the built-in - workflow. + apply relevant project facts, conventions, and constraints while + implementing. Treat `operationGuidance` as optional additive advice. Read and + consider every entry, and follow entries that are applicable and compatible + with the built-in workflow. Keep both fields separate from CLI-returned state, missing artifacts, tasks, progress, `contextFiles`, and the built-in `instruction`. They are not @@ -64,8 +90,8 @@ Implement tasks from an OpenSpec change. 4. **Read context files** - Read every file path listed under `contextFiles` from the apply instructions output. - The files depend on the schema being used: + Read every file path listed under `contextFiles` from the apply instructions + output. The files depend on the schema being used: - **spec-driven**: proposal, specs, design, tasks - Other schemas: follow the contextFiles from CLI output @@ -92,7 +118,9 @@ Implement tasks from an OpenSpec change. **Pause if:** - Task is unclear → ask for clarification - Implementation reveals a design issue → suggest updating artifacts - - A task needs work beyond what the spec and tasks describe, or you are tempted to drop, narrow, defer, or accept exceptions to specified behavior to make it fit → surface the added scope and ask; do not absorb it silently + - A task needs work beyond what the spec and tasks describe, or you are + tempted to drop, narrow, defer, or accept exceptions to specified behavior + to make it fit → surface the added scope and ask; do not absorb it silently - Error or blocker encountered → report and wait for guidance - User interrupts @@ -106,7 +134,7 @@ Implement tasks from an OpenSpec change. **Output During Implementation** -``` +```text ## Implementing: (schema: ) Working on task 3/7: @@ -120,7 +148,7 @@ Working on task 4/7: **Output On Completion** -``` +```text ## Implementation Complete **Change:** @@ -137,7 +165,7 @@ All tasks complete! You can archive this change with `/opsx-archive`. **Output On Pause (Issue Encountered)** -``` +```text ## Implementation Paused **Change:** @@ -156,6 +184,7 @@ What would you like to do? ``` **Guardrails** + - Keep going through tasks until done or blocked - Always read context files before starting (from the apply instructions output) - If task is ambiguous, pause and ask before implementing @@ -163,18 +192,25 @@ What would you like to do? - Keep code changes minimal and scoped to each task - Update task checkbox immediately after completing each task - Pause on errors, blockers, or unclear requirements - don't guess -- When a task needs work beyond what the spec describes, surface the added scope and pause - never silently narrow, defer, or simplify away specified behavior -- Only mark a task `- [x]` when its specified behavior is fully implemented, not when it is partially done or deferred +- When a task needs work beyond what the spec describes, surface the added scope + and pause - never silently narrow, defer, or simplify away specified behavior +- Only mark a task `- [x]` when its specified behavior is fully implemented, not + when it is partially done or deferred - Use contextFiles from CLI output, don't assume specific file names - Do not use context or operation guidance as proof that a task is complete -- Apply relevant project context; report conflicts with controlling workflow inputs +- Apply relevant project context; report conflicts with controlling workflow + inputs - Consider every guidance entry; explain any inapplicable or conflicting advice -- Do not copy runtime context or operation guidance into implementation files or planning artifacts -- Preserve CLI-controlled blocked/ready/all-done behavior and completion criteria +- Do not copy runtime context or operation guidance into implementation files or + planning artifacts +- Preserve CLI-controlled blocked/ready/all-done behavior and completion + criteria **Fluid Workflow Integration** This skill supports the "actions on a change" model: -- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions -- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly +- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), + after partial implementation, interleaved with other actions +- **Allows artifact updates**: If implementation reveals design issues, suggest + updating artifacts - not phase-locked, work fluidly diff --git a/.agents/workflows/opsx-archive.md b/.agents/workflows/opsx-archive.md index a409d82..03ae235 100644 --- a/.agents/workflows/opsx-archive.md +++ b/.agents/workflows/opsx-archive.md @@ -4,11 +4,27 @@ description: "Archive a completed change in the experimental workflow" Archive a completed change in the experimental workflow. -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -`` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve the full path from each delta spec when resolving its main spec. - -**Input**: Optionally specify a change name after `/opsx-archive` (e.g., `/opsx-archive add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +`` is the spec directory relative to `specs/` (for example, +`user-auth` or `identity/user-auth`). Preserve the full path from each delta +spec when resolving its main spec. + +**Input**: Optionally specify a change name after `/opsx-archive` (e.g., +`/opsx-archive add-auth`). If omitted, check if it can be inferred from +conversation context. If vague or ambiguous you MUST prompt for available +changes. **Steps** @@ -17,24 +33,29 @@ Archive a completed change in the experimental workflow. If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes and ask + the user to select one When prompting, show only active changes (not already archived). Include the schema used for each change if available. - Always announce: "Using change: " and how to override (e.g., `/opsx-archive `). + Always announce: "Using change: " and how to override (e.g., + `/opsx-archive `). **Load current archive inputs before the existing archive checks:** After resolving the selected change and planning root, run: + ```bash openspec instructions archive --change "" --json ``` - Keep the same selected-root flags on this command. This lookup is advisory and - optional: it only supplies extra prompt inputs, so it must never block archiving. - If it exits non-zero or returns invalid JSON — for example on an older CLI that - does not support this command yet — continue the archive workflow with no - context and no operation guidance. Do not report an error and do not stop. + + Keep the same selected-root flags on this command. This lookup is advisory + and optional: it only supplies extra prompt inputs, so it must never block + archiving. If it exits non-zero or returns invalid JSON — for example on an + older CLI that does not support this command yet — continue the archive + workflow with no context and no operation guidance. Do not report an error + and do not stop. A successful response may omit both optional fields. Treat `context` as a required prompt-level input: read and consider it, and apply relevant project @@ -42,14 +63,15 @@ Archive a completed change in the experimental workflow. additive advice: read and consider every entry, and follow entries that are applicable and compatible with the built-in archive workflow. - Keep both fields separate from built-in steps, explicit user choices, resolved - paths, CLI checks, and command contracts. If context conflicts with one of those - controlling inputs, report the conflict and preserve the controlling value. If - guidance is inapplicable or conflicts with a controlling input, do not follow it - and explain why. Do not infer replacement paths, skipped prompts, or flags from - either field, and do not copy their text verbatim into specs, change artifacts, - or archive summaries unless the user separately asks for it. These are - prompt-level behavior contracts, not enforceable checks. + Keep both fields separate from built-in steps, explicit user choices, + resolved paths, CLI checks, and command contracts. If context conflicts with + one of those controlling inputs, report the conflict and preserve the + controlling value. If guidance is inapplicable or conflicts with a + controlling input, do not follow it and explain why. Do not infer replacement + paths, skipped prompts, or flags from either field, and do not copy their + text verbatim into specs, change artifacts, or archive summaries unless the + user separately asks for it. These are prompt-level behavior contracts, not + enforceable checks. 2. **Check artifact completion status** @@ -57,10 +79,13 @@ Archive a completed change in the experimental workflow. Parse the JSON to understand: - `schemaName`: The workflow being used - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context - - `artifacts`: List of artifacts with their status (`done`, `skipped`, or other) + - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path + and scope context + - `artifacts`: List of artifacts with their status (`done`, `skipped`, or + other) - **If any artifacts are neither `done` nor `skipped`** (skipped artifacts satisfy the requirement - the change declares skip_specs): + **If any artifacts are neither `done` nor `skipped`** (skipped artifacts + satisfy the requirement - the change declares skip_specs): - Display warning listing incomplete artifacts - Prompt user for confirmation to continue - Proceed if user confirms @@ -81,13 +106,16 @@ Archive a completed change in the experimental workflow. 4. **Assess delta spec sync state** Use `artifactPaths.specs.existingOutputPaths` from status JSON as the only - delta-spec source. If the `specs` entry is missing or - `existingOutputPaths` is empty, proceed without a sync prompt and do not infer - delta specs from other artifacts. + delta-spec source. If the `specs` entry is missing or `existingOutputPaths` + is empty, proceed without a sync prompt and do not infer delta specs from + other artifacts. **If delta specs exist:** - - Compare each delta spec with its corresponding main spec at `/openspec/specs//spec.md` (use the store-aware `planningHome.root` from step 2, not a hardcoded repo path) - - Determine what changes would be applied (adds, modifications, removals, renames) + - Compare each delta spec with its corresponding main spec at + `/openspec/specs//spec.md` (use the + store-aware `planningHome.root` from step 2, not a hardcoded repo path) + - Determine what changes would be applied (adds, modifications, removals, + renames) - Show a combined summary before prompting **Prompt options:** @@ -102,34 +130,58 @@ Archive a completed change in the experimental workflow. Before a selected sync writes any main spec, run `openspec instructions specs --change "" --json` once with the same - selected-root flags. Require a zero exit status and valid artifact-instruction - JSON. If the lookup fails or returns invalid JSON, report the error and stop - before writing any main spec or moving the change. A valid response with omitted - `rules` is the no-rules case. Apply returned `rules` only to the content and - form of main specs produced by this merge; do not use them as archive guidance, - change CLI behavior, or copy the rule text into any output file. - - Then run the `/opsx-sync` workflow inline (agent-driven intelligent merge) for change '', passing the delta spec analysis and the fetched specs-rule snapshot from above, and wait for it to finish. The inline sync must reuse that snapshot without fetching `specs` instructions again. Do not delegate it to a background task — step 5 would move `changeRoot` out from under a sync that is still reading it, leaving the change archived and the main specs never updated. If your agent can only run it by delegation, delegate synchronously and wait for the result. - - Then re-run the comparison from the top of this step against every capability that has a delta spec in `artifactPaths.specs.existingOutputPaths` — not only the ones the sync reports it touched. A successful sync leaves nothing left to apply, so each capability must now read as already synced: + selected-root flags. Require a zero exit status and valid + artifact-instruction JSON. If the lookup fails or returns invalid JSON, + report the error and stop before writing any main spec or moving the change. + A valid response with omitted `rules` is the no-rules case. Apply returned + `rules` only to the content and form of main specs produced by this merge; do + not use them as archive guidance, change CLI behavior, or copy the rule text + into any output file. + + Then run the `/opsx-sync` workflow inline (agent-driven intelligent merge) + for change '', passing the delta spec analysis and the fetched + specs-rule snapshot from above, and wait for it to finish. The inline sync + must reuse that snapshot without fetching `specs` instructions again. Do not + delegate it to a background task — step 5 would move `changeRoot` out from + under a sync that is still reading it, leaving the change archived and the + main specs never updated. If your agent can only run it by delegation, + delegate synchronously and wait for the result. + + Then re-run the comparison from the top of this step against every capability + that has a delta spec in `artifactPaths.specs.existingOutputPaths` — not only + the ones the sync reports it touched. A successful sync leaves nothing left + to apply, so each capability must now read as already synced: - ADDED requirements present - - MODIFIED requirements carrying the scenario and description changes named in the delta, with their other scenarios intact - - REMOVED requirements gone — and where this sync retired a capability (removed its last requirement, leaving `## Requirements` empty), its main spec deleted rather than left empty; a spec the sync deliberately kept and reported is also a match - - RENAMED requirements present under the new name and absent under the old one - - If the sync failed, or any capability does not match, report what differs and stop — do not archive. Nothing has moved and `changeRoot` is intact, so the user can fix the mismatch or re-run the sync and start the archive again. + - MODIFIED requirements carrying the scenario and description changes named + in the delta, with their other scenarios intact + - REMOVED requirements gone — and where this sync retired a capability + (removed its last requirement, leaving `## Requirements` empty), its main + spec deleted rather than left empty; a spec the sync deliberately kept and + reported is also a match + - RENAMED requirements present under the new name and absent under the old + one + + If the sync failed, or any capability does not match, report what differs and + stop — do not archive. Nothing has moved and `changeRoot` is intact, so the + user can fix the mismatch or re-run the sync and start the archive again. 5. **Perform the archive** - Create an `archive` directory under `planningHome.changesDir` if it doesn't exist: + Create an `archive` directory under `planningHome.changesDir` if it doesn't + exist: + ```bash mkdir -p "/archive" ``` - Generate the target name: use the change name as-is when it already starts with a `YYYY-MM-DD-` prefix; otherwise prepend the current date as `YYYY-MM-DD-`. Never stack a second date (same rule as `openspec archive`). + Generate the target name: use the change name as-is when it already starts + with a `YYYY-MM-DD-` prefix; otherwise prepend the current date as + `YYYY-MM-DD-`. Never stack a second date (same rule as + `openspec archive`). **Check if target already exists:** - - If yes: Fail with error, suggest renaming existing archive or using different date + - If yes: Fail with error, suggest renaming existing archive or using + different date - If no: Move `changeRoot` to the archive directory ```bash @@ -206,16 +258,24 @@ Target archive directory already exists. ``` **Guardrails** + - Announce the selected change; prompt for selection when it is ambiguous - Use artifact graph (openspec status --json) for completion checking - Don't block archive on warnings - just inform and confirm - Preserve .openspec.yaml when moving to archive (it moves with the directory) - Show clear summary of what happened - If sync is requested, run the `/opsx-sync` workflow inline (agent-driven) -- Never archive while a spec sync is still in flight — run the sync inline and verify the main specs before moving `changeRoot` -- If delta specs exist, always run the sync assessment and show the combined summary before prompting -- Apply relevant runtime context and report conflicts; operation guidance remains advisory -- Consider every guidance entry and explain any inapplicable or conflicting advice -- Existing CLI checks, resolved paths, prompts, and command contracts are unchanged -- Artifact rules constrain only the specs being written and are never operation guidance -- Never copy runtime context, operation guidance, or artifact-rule text verbatim into output files +- Never archive while a spec sync is still in flight — run the sync inline and + verify the main specs before moving `changeRoot` +- If delta specs exist, always run the sync assessment and show the combined + summary before prompting +- Apply relevant runtime context and report conflicts; operation guidance + remains advisory +- Consider every guidance entry and explain any inapplicable or conflicting + advice +- Existing CLI checks, resolved paths, prompts, and command contracts are + unchanged +- Artifact rules constrain only the specs being written and are never operation + guidance +- Never copy runtime context, operation guidance, or artifact-rule text verbatim + into output files diff --git a/.agents/workflows/opsx-explore.md b/.agents/workflows/opsx-explore.md index 2343d19..8311fca 100644 --- a/.agents/workflows/opsx-explore.md +++ b/.agents/workflows/opsx-explore.md @@ -2,15 +2,42 @@ description: "Enter explore mode - think through ideas, investigate problems, clarify requirements" --- -Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes. +Enter explore mode. Think deeply. Visualize freely. Follow the conversation +wherever it goes. + +**IMPORTANT: Explore mode is for thinking, not implementing.** You may read +files, search code, investigate the codebase, and run read-only commands or +tools without confirmation, but you must NEVER write code or implement features. +If the user asks you to implement something, remind them to exit explore mode +first and create a change proposal. You MAY create or update OpenSpec change +artifacts (proposals, designs, specs) within a confirmed scope—that's capturing +thinking, not implementing. Answering design or clarifying questions is never +consent to write. Before the first write-capable action, name the artifacts or +files you would change and what you would do, ask a direct yes/no question, and +wait for the user's confirmation in a separate message. Confirmation covers only +the scope you described; ask again before expanding it. For a new change, +scaffold it first as described below. + +**This is a stance, not a workflow.** There are no fixed steps, no required +sequence, no mandatory outputs. You're a thinking partner helping the user +explore. + +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +**Input**: The argument after `/opsx-explore` is whatever the user wants to +think about. Could be: -**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, investigate the codebase, and run read-only commands or tools without confirmation, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create or update OpenSpec change artifacts (proposals, designs, specs) within a confirmed scope—that's capturing thinking, not implementing. Answering design or clarifying questions is never consent to write. Before the first write-capable action, name the artifacts or files you would change and what you would do, ask a direct yes/no question, and wait for the user's confirmation in a separate message. Confirmation covers only the scope you described; ask again before expanding it. For a new change, scaffold it first as described below. - -**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore. - -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -**Input**: The argument after `/opsx-explore` is whatever the user wants to think about. Could be: - A vague idea: "real-time collaboration" - A specific problem: "the auth system is getting unwieldy" - A change name: "add-dark-mode" (to explore in context of that change) @@ -21,8 +48,11 @@ Enter explore mode. Think deeply. Visualize freely. Follow the conversation wher ## The Stance -- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script -- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions. +- **Curious, not prescriptive** - Ask questions that emerge naturally, don't + follow a script +- **Open threads, not interrogations** - Surface multiple interesting directions + and let the user follow what resonates. Don't funnel them through a single + path of questions. - **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking - **Adaptive** - Follow interesting threads, pivot when new information emerges - **Patient** - Don't rush to conclusions, let the shape of the problem emerge @@ -32,16 +62,37 @@ Enter explore mode. Think deeply. Visualize freely. Follow the conversation wher ## Planning a Change -When the user is planning a change, guide them toward shared understanding with focused discovery questions. For open-ended discussion, follow the conversation without imposing an interview or a required output. - -Before asking a factual question, follow the context discovery below and inspect relevant OpenSpec artifacts, source, tests, docs, and configuration. Do not ask the user to repeat facts you can verify. Summarize relevant findings without reproducing private context or rules. If evidence is missing, conflicting, or inaccessible, state that limitation and ask only for the clarification needed to proceed. - -- **Follow dependencies** - Resolve the next blocking decision before its dependent details. For example, clarify the user's outcome and scope before choosing an API or data model. Revisit downstream assumptions when an earlier answer changes. Skip branches that do not matter to this goal. -- **Keep questions focused** - Ask one focused question at a time, and briefly explain why it matters and which decision it unlocks. Batch questions only if the user asks for a batch; keep them small and group related decisions. -- **Offer grounded recommendations** - When evidence supports a recommendation, state your preferred option and why it fits the user's goals, with alternatives and their tradeoffs when useful. Do not invent intent, priorities, or external constraints: ask the user when only they can answer. Avoid a fixed question format. -- **Keep a conversational record** - Track decisions in the conversation, not in files. Separate confirmed decisions from proposed defaults and unresolved questions. Silence is not acceptance. Accepting an answer or a batch of recommendations is not permission to write. Keep file-write confirmation separate from discovery questions and follow the guardrails below. - -Stop asking when the user has enough clarity. Let them pause, pivot, or defer a decision; do not exhaust every branch or force a proposal. +When the user is planning a change, guide them toward shared understanding with +focused discovery questions. For open-ended discussion, follow the conversation +without imposing an interview or a required output. + +Before asking a factual question, follow the context discovery below and inspect +relevant OpenSpec artifacts, source, tests, docs, and configuration. Do not ask +the user to repeat facts you can verify. Summarize relevant findings without +reproducing private context or rules. If evidence is missing, conflicting, or +inaccessible, state that limitation and ask only for the clarification needed to +proceed. + +- **Follow dependencies** - Resolve the next blocking decision before its + dependent details. For example, clarify the user's outcome and scope before + choosing an API or data model. Revisit downstream assumptions when an earlier + answer changes. Skip branches that do not matter to this goal. +- **Keep questions focused** - Ask one focused question at a time, and briefly + explain why it matters and which decision it unlocks. Batch questions only if + the user asks for a batch; keep them small and group related decisions. +- **Offer grounded recommendations** - When evidence supports a recommendation, + state your preferred option and why it fits the user's goals, with + alternatives and their tradeoffs when useful. Do not invent intent, + priorities, or external constraints: ask the user when only they can answer. + Avoid a fixed question format. +- **Keep a conversational record** - Track decisions in the conversation, not in + files. Separate confirmed decisions from proposed defaults and unresolved + questions. Silence is not acceptance. Accepting an answer or a batch of + recommendations is not permission to write. Keep file-write confirmation + separate from discovery questions and follow the guardrails below. + +Stop asking when the user has enough clarity. Let them pause, pivot, or defer a +decision; do not exhaust every branch or force a proposal. For example, after inspecting the relevant code: @@ -59,25 +110,29 @@ adding a service to operate; shared state would need a separate sync design. Depending on what the user brings, you might: **Explore the problem space** + - Ask clarifying questions that emerge from what they said - Challenge assumptions - Reframe the problem - Find analogies **Investigate the codebase** + - Map existing architecture relevant to the discussion - Find integration points - Identify patterns already in use - Surface hidden complexity **Compare options** + - Brainstorm multiple approaches - Build comparison tables - Sketch tradeoffs - Recommend a path (if asked) **Visualize** -``` + +```text +------------------------------------------+ | Use ASCII diagrams liberally | +------------------------------------------+ @@ -94,10 +149,13 @@ Depending on what the user brings, you might: +------------------------------------------+ ``` -**Draw with plain ASCII only** — borders `+` `-` `|`, arrows `-->` `<--` `^` `v`, markers `*` `x`. -Unicode diagram glyphs can render at different widths across terminals, fonts, and locales, so padded boxes and aligned tables can drift. Keep every diagram character ASCII. +**Draw with plain ASCII only** — borders `+` `-` `|`, arrows `-->` `<--` `^` +`v`, markers `*` `x`. Unicode diagram glyphs can render at different widths +across terminals, fonts, and locales, so padded boxes and aligned tables can +drift. Keep every diagram character ASCII. **Surface risks and unknowns** + - Identify what could go wrong - Find gaps in understanding - Suggest spikes or investigations @@ -111,20 +169,28 @@ You have full context of the OpenSpec system. Use it naturally, don't force it. ### Check for context At the start, quickly check what exists: + ```bash openspec list --json ``` This tells you: + - If there are active changes - Their names, schemas, and status - What the user might be working on -Then read the project's own context from the resolved root - `/openspec/config.yaml` (or `config.yml`). Use the `root.path` returned above, and skip this if neither file exists: +Then read the project's own context from the resolved root - +`/openspec/config.yaml` (or `config.yml`). Use the `root.path` +returned above, and skip this if neither file exists: + - `context`: project background - tech stack, conventions, constraints -- `rules`: keyed by artifact id - the entries for an artifact apply only when you write that artifact +- `rules`: keyed by artifact id - the entries for an artifact apply only when + you write that artifact -Ground your thinking in these. They are constraints for you to follow, not content to reproduce: do NOT copy them into the conversation or into any artifact you create. +Ground your thinking in these. They are constraints for you to follow, not +content to reproduce: do NOT copy them into the conversation or into any +artifact you create. If the user mentioned a specific change name, read its artifacts for context. @@ -135,14 +201,57 @@ Think freely. When insights crystallize, you might offer: - "This feels solid enough to start a change. Want me to create a proposal?" - Or keep exploring - no pressure to formalize -If the user asks you to capture the exploration as a new change, transition seamlessly into the requested capture: - -1. Run `openspec new change ""` (with `--store ` when applicable) before creating any artifacts. Never create a new change directory under `openspec/changes/` by hand; the CLI scaffold creates required metadata such as `.openspec.yaml`. Keep the selected `--store ` on every applicable follow-up `status` and `instructions` command. -2. Run `openspec status --change "" --json` (append the confirmed `--store ""` only for a registered standalone store), then process the requested artifacts in dependency order. For each requested artifact that is `ready`, run `openspec instructions "" --change "" --json` (append the confirmed `--store ""` only for a registered standalone store). Before creating a requested artifact, evaluate any condition in its own `instruction` against the explored change; record a deliberate skip instead when the condition does not apply. If a requested artifact is blocked by a direct prerequisite the user did not request, run `openspec instructions "" --change "" --json` (append the confirmed `--store ""` only for a registered standalone store) for that prerequisite whether it is `ready` or `blocked`. If its own `instruction` states a condition, evaluate that condition against the explored change and record a deliberate skip only when the condition does not apply. If the condition applies, or the prerequisite is not conditional, treat it as a normal prerequisite and ask before expanding the capture. Do not create an unrequested prerequisite unless the user approves. -3. Follow the returned `template` and `instruction` fields. Read completed dependency files listed in `dependencies`, and apply `context` and `rules` as constraints without copying them into the artifact. If the instruction delegates creation to a specific skill or command, invoke it; otherwise write the artifact to `resolvedOutputPath`, using the instruction to choose a concrete path when it is a glob. Verify that the selected concrete output exists. -4. After creating each artifact, re-run `openspec status --change "" --json` (append the confirmed `--store ""` only for a registered standalone store) and continue until every requested artifact is `done`, `skipped`, or was deliberately skipped because its own `instruction` stated a condition that did not apply. Tell the user about a deliberate conditional skip, remember it, and do not reconsider it. Dependencies are enablers, not gates: if a requested artifact is still `blocked` only because you deliberately skipped a conditional prerequisite, run `openspec instructions "" --change "" --json` (append the confirmed `--store ""` only for a registered standalone store) despite the blocked status, then create it using step 3 only when those recorded conditional skips are its sole missing dependencies. If a requested artifact is blocked by a prerequisite the user did not ask to capture and cannot be conditionally skipped, explain that dependency and ask before expanding the capture. - -Capture the artifact(s) the user requested without asking them to invoke another workflow command. If they asked only to start a change, stop after scaffolding and show its status. +If the user asks you to capture the exploration as a new change, transition +seamlessly into the requested capture: + +1. Run `openspec new change ""` (with `--store ` when applicable) + before creating any artifacts. Never create a new change directory under + `openspec/changes/` by hand; the CLI scaffold creates required metadata such + as `.openspec.yaml`. Keep the selected `--store ` on every applicable + follow-up `status` and `instructions` command. +2. Run `openspec status --change "" --json` (append the confirmed + `--store ""` only for a registered standalone store), then process the + requested artifacts in dependency order. For each requested artifact that is + `ready`, run `openspec instructions "" --change "" --json` + (append the confirmed `--store ""` only for a registered standalone + store). Before creating a requested artifact, evaluate any condition in its + own `instruction` against the explored change; record a deliberate skip + instead when the condition does not apply. If a requested artifact is blocked + by a direct prerequisite the user did not request, run + `openspec instructions "" --change "" --json` (append + the confirmed `--store ""` only for a registered standalone store) for + that prerequisite whether it is `ready` or `blocked`. If its own + `instruction` states a condition, evaluate that condition against the + explored change and record a deliberate skip only when the condition does not + apply. If the condition applies, or the prerequisite is not conditional, + treat it as a normal prerequisite and ask before expanding the capture. Do + not create an unrequested prerequisite unless the user approves. +3. Follow the returned `template` and `instruction` fields. Read completed + dependency files listed in `dependencies`, and apply `context` and `rules` as + constraints without copying them into the artifact. If the instruction + delegates creation to a specific skill or command, invoke it; otherwise write + the artifact to `resolvedOutputPath`, using the instruction to choose a + concrete path when it is a glob. Verify that the selected concrete output + exists. +4. After creating each artifact, re-run + `openspec status --change "" --json` (append the confirmed + `--store ""` only for a registered standalone store) and continue until + every requested artifact is `done`, `skipped`, or was deliberately skipped + because its own `instruction` stated a condition that did not apply. Tell the + user about a deliberate conditional skip, remember it, and do not reconsider + it. Dependencies are enablers, not gates: if a requested artifact is still + `blocked` only because you deliberately skipped a conditional prerequisite, + run `openspec instructions "" --change "" --json` (append + the confirmed `--store ""` only for a registered standalone store) + despite the blocked status, then create it using step 3 only when those + recorded conditional skips are its sole missing dependencies. If a requested + artifact is blocked by a prerequisite the user did not ask to capture and + cannot be conditionally skipped, explain that dependency and ask before + expanding the capture. + +Capture the artifact(s) the user requested without asking them to invoke another +workflow command. If they asked only to start a change, stop after scaffolding +and show its status. ### When a change exists @@ -150,25 +259,30 @@ If the user mentions a change or you detect one is relevant: 1. **Resolve and read existing artifacts for context** - Run `openspec status --change "" --json`. - - Use `changeRoot`, `artifactPaths`, and `actionContext` from the status JSON. + - Use `changeRoot`, `artifactPaths`, and `actionContext` from the status + JSON. - Read existing files from `artifactPaths..existingOutputPaths`. 2. **Reference them naturally in conversation** - - "Your design mentions using Redis, but we just realized SQLite fits better..." - - "The proposal scopes this to premium users, but we're now thinking everyone..." + - "Your design mentions using Redis, but we just realized SQLite fits + better..." + - "The proposal scopes this to premium users, but we're now thinking + everyone..." 3. **Offer to capture when decisions are made** - `` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve an existing capability's full path and follow the project's established organization for new capabilities. + `` is the spec directory relative to `specs/` (for example, + `user-auth` or `identity/user-auth`). Preserve an existing capability's full + path and follow the project's established organization for new capabilities. - | Insight Type | Where to Capture | - |----------------------------|-------------------------------------| - | New requirement discovered | `specs//spec.md` | - | Requirement changed | `specs//spec.md` | - | Design decision made | `design.md` | - | Scope changed | `proposal.md` | - | New work identified | `tasks.md` | - | Assumption invalidated | Relevant artifact | + | Insight Type | Where to Capture | + |----------------------------|-------------------------------------| + | New requirement discovered | `specs//spec.md` | + | Requirement changed | `specs//spec.md` | + | Design decision made | `design.md` | + | Scope changed | `proposal.md` | + | New work identified | `tasks.md` | + | Assumption invalidated | Relevant artifact | Example offers: - "That's a design decision. Capture it in design.md?" @@ -199,18 +313,32 @@ There's no required ending. Discovery might: - **Just provide clarity**: User has what they need, moves on - **Continue later**: "We can pick this up anytime" -When things crystallize, you might offer a summary - but it's optional. Sometimes the thinking IS the value. +When things crystallize, you might offer a summary - but it's optional. +Sometimes the thinking IS the value. --- ## Guardrails -- **Don't implement** - Never write code or implement features. Workflow configuration counts too: creating or editing schemas, templates, or `openspec/config.yaml` is a change, not thinking. Creating or updating OpenSpec change artifacts within the confirmed scope is fine, writing anything else is not. +- **Don't implement** - Never write code or implement features. Workflow + configuration counts too: creating or editing schemas, templates, or + `openspec/config.yaml` is a change, not thinking. Creating or updating + OpenSpec change artifacts within the confirmed scope is fine, writing anything + else is not. - **Don't fake understanding** - If something is unclear, dig deeper - **Don't rush** - Discovery is thinking time, not task time - **Don't force structure** - Let patterns emerge naturally -- **Don't auto-capture** - Offer to save insights, don't just do it. Read-only commands and tools need no confirmation. Before the first write-capable action—including `openspec new change` or another command that writes files—name the artifacts or files and proposed changes, ask a direct yes/no question, and wait for explicit confirmation in a separate user message. That confirmation covers only the described scope; ask again before expanding it. Answers to design or clarifying questions are never consent to write. -- **Don't manually scaffold changes** - Never create a new change directory under `openspec/changes/` by hand. Always use `openspec new change ""` (with `--store ` when applicable) so required metadata such as `.openspec.yaml` is created before writing artifacts. +- **Don't auto-capture** - Offer to save insights, don't just do it. Read-only + commands and tools need no confirmation. Before the first write-capable + action—including `openspec new change` or another command that writes + files—name the artifacts or files and proposed changes, ask a direct yes/no + question, and wait for explicit confirmation in a separate user message. That + confirmation covers only the described scope; ask again before expanding it. + Answers to design or clarifying questions are never consent to write. +- **Don't manually scaffold changes** - Never create a new change directory + under `openspec/changes/` by hand. Always use `openspec new change ""` + (with `--store ` when applicable) so required metadata such as + `.openspec.yaml` is created before writing artifacts. - **Do visualize** - A good diagram is worth many paragraphs - **Do explore the codebase** - Ground discussions in reality - **Do question assumptions** - Including the user's and your own diff --git a/.agents/workflows/opsx-propose.md b/.agents/workflows/opsx-propose.md index 983e348..32fd4c1 100644 --- a/.agents/workflows/opsx-propose.md +++ b/.agents/workflows/opsx-propose.md @@ -4,115 +4,216 @@ description: "Propose a new change - create it and generate all artifacts in one Propose a new change - create the change and generate all artifacts in one step. -**Planning boundary**: This workflow creates planning artifacts only. The user request that selected or triggered this workflow authorizes planning only, even if it asks to build or fix something. Do not edit project code. After the planning artifacts are complete, stop. Do not start implementation in the same response, even if the initial request asks for it. Wait for a new user request after the artifacts are presented; then start the apply workflow. +**Planning boundary**: This workflow creates planning artifacts only. The user +request that selected or triggered this workflow authorizes planning only, even +if it asks to build or fix something. Do not edit project code. After the +planning artifacts are complete, stop. Do not start implementation in the same +response, even if the initial request asks for it. Wait for a new user request +after the artifacts are presented; then start the apply workflow. + +I'll create a change with the artifacts your schema defines. With the default +spec-driven schema that is: -I'll create a change with the artifacts your schema defines. With the default spec-driven schema that is: - proposal.md (what & why) -- `specs//spec.md` (what the system must do - a delta, not the main spec) +- `specs//spec.md` (what the system must do - a delta, not the + main spec) - design.md (how) - tasks.md (implementation steps) -`` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve an existing capability's full path and follow the project's established organization for new capabilities. +`` is the spec directory relative to `specs/` (for example, +`user-auth` or `identity/user-auth`). Preserve an existing capability's full +path and follow the project's established organization for new capabilities. -When the user is ready to implement, they must start the apply workflow explicitly. +When the user is ready to implement, they must start the apply workflow +explicitly. --- -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -**Input**: The argument after `/opsx-propose` is the change name (kebab-case), OR a description of what the user wants to build. +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +**Input**: The argument after `/opsx-propose` is the change name (kebab-case), +OR a description of what the user wants to build. **Steps** 1. **Understand the request and clarify material ambiguity** If no input is provided, ask the user (open-ended, no preset options): - > "What change do you want to work on? Describe what you want to build or fix." + > "What change do you want to work on? Describe what you want to build or + > fix." - From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`). + From their description, derive a kebab-case name (e.g., "add user + authentication" → `add-user-auth`). - **IMPORTANT**: Do NOT proceed without understanding what the user wants to build. + **IMPORTANT**: Do NOT proceed without understanding what the user wants to + build. - If the request contains ambiguity that would materially affect scope, externally observable behavior, compatibility, or acceptance criteria, ask the user before creating the change. For minor details, make a reasonable assumption and record it in the planning artifacts. + If the request contains ambiguity that would materially affect scope, + externally observable behavior, compatibility, or acceptance criteria, ask + the user before creating the change. For minor details, make a reasonable + assumption and record it in the planning artifacts. 2. **Determine the workflow schema** - Use the configured default schema unless the user explicitly requests a different workflow. + Use the configured default schema unless the user explicitly requests a + different workflow. **Use a different schema only if the user:** - - Explicitly requests a specific schema by name → use `--schema ` - - Asks to "show workflows" or asks "what workflows" exist → resolve the authoritative root by running `openspec context --json` from the current working directory. If the user explicitly selected a registered store, use `openspec context --json --store ""`. Then run `openspec schemas --json` with its working directory set to the returned `root.path` and let them choose. This preserves roots selected by a local `store:` pointer or the global `defaultStore`; when a registered store was explicitly selected, append `--store ""` to `openspec schemas --json` as well. If context reports only `no_openspec_root`, run `openspec schemas --json` from the current working directory instead. Do not use this fallback for invalid or unavailable stores. + - Explicitly requests a specific schema by name → use + `--schema ` + - Asks to "show workflows" or asks "what workflows" exist → resolve the + authoritative root by running `openspec context --json` from the current + working directory. If the user explicitly selected a registered store, use + `openspec context --json --store ""`. Then run + `openspec schemas --json` with its working directory set to the returned + `root.path` and let them choose. This preserves roots selected by a local + `store:` pointer or the global `defaultStore`; when a registered store was + explicitly selected, append `--store ""` to + `openspec schemas --json` as well. If context reports only + `no_openspec_root`, run `openspec schemas --json` from the current working + directory instead. Do not use this fallback for invalid or unavailable + stores. Otherwise, omit `--schema` to preserve the configured default. 3. **Create the change directory** - Choose one schema form below. If a registered store is selected, append `--store ""` to that command and each later OpenSpec command shown below that accepts `--store`. + Choose one schema form below. If a registered store is selected, append + `--store ""` to that command and each later OpenSpec command shown + below that accepts `--store`. Using the configured default: + ```bash openspec new change "" ``` Using an explicitly requested schema: + ```bash openspec new change "" --schema "" ``` - This creates a scaffolded change in the planning home resolved by the CLI with `.openspec.yaml`. + + This creates a scaffolded change in the planning home resolved by the CLI + with `.openspec.yaml`. 4. **Get the artifact build order** + ```bash openspec status --change "" --json ``` + Parse the JSON to get: - - `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`) - - `artifacts`: list of all artifacts, each with its `status` and its `requires` edges (the artifact IDs it directly depends on) - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context. Use these instead of assuming repo-local paths. + - `applyRequires`: array of artifact IDs needed before implementation (e.g., + `["tasks"]`) + - `artifacts`: list of all artifacts, each with its `status` and its + `requires` edges (the artifact IDs it directly depends on) + - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path + and scope context. Use these instead of assuming repo-local paths. 5. **Create every artifact in the required set** Use a todo list to track progress through the artifacts. - Loop through artifacts in dependency order (artifacts with no pending dependencies first): + Loop through artifacts in dependency order (artifacts with no pending + dependencies first): a. **For each artifact that is `ready` (dependencies satisfied)**: - Get instructions: + ```bash openspec instructions --change "" --json ``` + - The instructions JSON includes: - - `context`: Project background (constraints for you - do NOT include in output) - - `rules`: Artifact-specific rules (constraints for you - do NOT include in output) + - `context`: Project background (constraints for you - do NOT include in + output) + - `rules`: Artifact-specific rules (constraints for you - do NOT include + in output) - `template`: The structure to use for your output file - `instruction`: Schema-specific guidance for this artifact type - - `skipped`/`warning`: present when the change declares skip_specs and this artifact must NOT be created - stop and pick another artifact + - `skipped`/`warning`: present when the change declares skip_specs and + this artifact must NOT be created - stop and pick another artifact - `resolvedOutputPath`: Resolved path or pattern to write the artifact - `dependencies`: Completed artifacts to read for context - - Read any completed dependency files for context - always re-read them from disk, even if you saw them earlier in the conversation (the user may have edited them) - - **Inspect the relevant project before drafting**: Read `context` and `rules` first, then inspect relevant implementation, nearby tests, configuration, and documentation outside `openspec/`. Keep inspection read-only and proportional to the change; reuse findings for later artifacts and inspect more only as needed. - - Identify the target project from the request and project context; the planning home may be separate from the code. If the target is unclear, ask. For greenfield or non-code changes, inspect the available structure and relevant documents. If source is unavailable, state the limitation and ask when it materially affects the plan. - - Ground scope, approach, and tasks in what you find. Distinguish observed behavior from assumptions and proposed additions; surface conflicts with existing specs instead of silently deciding which is correct. - - Do this discovery now, rather than leaving generic "explore the codebase" or "make a plan" tasks for implementation. Keep any necessary follow-up investigation specific to an unresolved question. - - If the `instruction` field delegates creation to a specific skill or command, invoke it to produce the artifact instead of writing the file yourself, then verify the artifact file exists at `resolvedOutputPath` - - Otherwise create the artifact file using `template` as the structure and write it to `resolvedOutputPath`. If `resolvedOutputPath` is a glob, follow `instruction` to choose the concrete file path - - Apply `context` and `rules` as constraints - but do NOT copy them into the file + - Read any completed dependency files for context - always re-read them + from disk, even if you saw them earlier in the conversation (the user + may have edited them) + - **Inspect the relevant project before drafting**: Read `context` and + `rules` first, then inspect relevant implementation, nearby tests, + configuration, and documentation outside `openspec/`. Keep inspection + read-only and proportional to the change; reuse findings for later + artifacts and inspect more only as needed. + - Identify the target project from the request and project context; the + planning home may be separate from the code. If the target is unclear, + ask. For greenfield or non-code changes, inspect the available + structure and relevant documents. If source is unavailable, state the + limitation and ask when it materially affects the plan. + - Ground scope, approach, and tasks in what you find. Distinguish + observed behavior from assumptions and proposed additions; surface + conflicts with existing specs instead of silently deciding which is + correct. + - Do this discovery now, rather than leaving generic "explore the + codebase" or "make a plan" tasks for implementation. Keep any + necessary follow-up investigation specific to an unresolved question. + - If the `instruction` field delegates creation to a specific skill or + command, invoke it to produce the artifact instead of writing the file + yourself, then verify the artifact file exists at `resolvedOutputPath` + - Otherwise create the artifact file using `template` as the structure and + write it to `resolvedOutputPath`. If `resolvedOutputPath` is a glob, + follow `instruction` to choose the concrete file path + - Apply `context` and `rules` as constraints - but do NOT copy them into + the file - Show brief progress: "Created " - b. **Continue until every artifact in the required set exists (not just `apply.requires`)** - - After creating each artifact, re-run `openspec status --change "" --json` - - The required set is `applyRequires` plus every artifact reachable from those by following the `requires` edges in `status --json` - walk them transitively (spec-driven closes over proposal, specs, design, tasks). Leave artifacts outside that set alone - - `status` is file-existence only, so an `applyRequires` artifact reading `done` does NOT mean its dependencies exist - writing `tasks.md` early marks `tasks` done while `specs` was never written. Use each artifact's `requires` edges, not its `status`, to build the required set: a `done` artifact still lists what it depends on - - An artifact already reading `status: "skipped"` is satisfied: the change declares `skip_specs` in `.openspec.yaml`, so its files must NOT exist. Never try to create one - - Create every artifact in the required set that is missing, then re-check - creating one can unblock others - - Skip one only when `status` already reports it `skipped`, or when its own `instruction` says it is conditional: run `openspec instructions --change "" --json` and skip only if its `instruction` field marks it optional (e.g. "create only if..."). Spec-driven's `design.md` qualifies; `specs` qualifies only via the `skipped` status above, never by your own judgment. Tell the user, and do not reconsider it - - Dependencies are enablers, not gates: if a required artifact is still `blocked` only because you skipped a conditional dependency, write it anyway - - Stop when every artifact in the required set is `done`, `skipped`, or was deliberately skipped + b. **Continue until every artifact in the required set exists (not just + `apply.requires`)** + - After creating each artifact, re-run + `openspec status --change "" --json` + - The required set is `applyRequires` plus every artifact reachable from + those by following the `requires` edges in `status --json` - walk them + transitively (spec-driven closes over proposal, specs, design, tasks). + Leave artifacts outside that set alone + - `status` is file-existence only, so an `applyRequires` artifact reading + `done` does NOT mean its dependencies exist - writing `tasks.md` early + marks `tasks` done while `specs` was never written. Use each artifact's + `requires` edges, not its `status`, to build the required set: a `done` + artifact still lists what it depends on + - An artifact already reading `status: "skipped"` is satisfied: the change + declares `skip_specs` in `.openspec.yaml`, so its files must NOT exist. + Never try to create one + - Create every artifact in the required set that is missing, then + re-check - creating one can unblock others + - Skip one only when `status` already reports it `skipped`, or when its + own `instruction` says it is conditional: run + `openspec instructions --change "" --json` and skip + only if its `instruction` field marks it optional (e.g. "create only + if..."). Spec-driven's `design.md` qualifies; `specs` qualifies only via + the `skipped` status above, never by your own judgment. Tell the user, + and do not reconsider it + - Dependencies are enablers, not gates: if a required artifact is still + `blocked` only because you skipped a conditional dependency, write it + anyway + - Stop when every artifact in the required set is `done`, `skipped`, or + was deliberately skipped c. **If an artifact requires user input** (unclear context): - Ask the user to clarify - Then continue with creation 6. **Show final status** + ```bash openspec status --change "" ``` @@ -120,26 +221,44 @@ When the user is ready to implement, they must start the apply workflow explicit **Output** After completing all artifacts, summarize: + - Change name and location -- List of artifacts created with brief descriptions, plus any conditional artifact you skipped and why +- List of artifacts created with brief descriptions, plus any conditional + artifact you skipped and why - What's ready: "All artifacts needed for implementation are ready." -- Prompt: "The artifacts are ready for review. When you are ready, run `/opsx-apply`." +- Prompt: "The artifacts are ready for review. When you are ready, run + `/opsx-apply`." **Artifact Creation Guidelines** -- Follow the `instruction` field from `openspec instructions` for each artifact type - it is the authoritative guidance, even for familiar artifact names -- If the `instruction` field directs you to use a specific skill or command to create the artifact, invoke it instead of writing the artifact directly +- Follow the `instruction` field from `openspec instructions` for each artifact + type - it is the authoritative guidance, even for familiar artifact names +- If the `instruction` field directs you to use a specific skill or command to + create the artifact, invoke it instead of writing the artifact directly - The schema defines what each artifact should contain - follow it - Read dependency artifacts for context before creating new ones - Use `template` as the structure for your output file - fill in its sections -- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file - - Do NOT copy ``, ``, `` blocks into the artifact +- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for + the file + - Do NOT copy ``, ``, `` blocks into the + artifact - These guide what you write, but should never appear in the output **Guardrails** -- The request that invoked this workflow authorizes planning only. Any implementation or apply instruction in that request does not carry forward. Do NOT implement the change, start the apply workflow, or edit project code during this workflow. After presenting the artifacts, stop and wait for a new user request to start the apply workflow -- Create every artifact the apply phase transitively depends on, not just the ids listed in `apply.requires` -- Always read dependency artifacts before creating a new one - re-read from disk, not from conversation memory (files may have changed since you last saw them) -- Ask about ambiguities that would materially change scope, externally observable behavior, compatibility, or acceptance criteria; for minor details, make reasonable assumptions and record them -- If a change with that name already exists, ask if user wants to continue it or create a new one + +- The request that invoked this workflow authorizes planning only. Any + implementation or apply instruction in that request does not carry forward. Do + NOT implement the change, start the apply workflow, or edit project code + during this workflow. After presenting the artifacts, stop and wait for a new + user request to start the apply workflow +- Create every artifact the apply phase transitively depends on, not just the + ids listed in `apply.requires` +- Always read dependency artifacts before creating a new one - re-read from + disk, not from conversation memory (files may have changed since you last saw + them) +- Ask about ambiguities that would materially change scope, externally + observable behavior, compatibility, or acceptance criteria; for minor details, + make reasonable assumptions and record them +- If a change with that name already exists, ask if user wants to continue it or + create a new one - Verify each artifact file exists after writing before proceeding to next diff --git a/.agents/workflows/opsx-sync.md b/.agents/workflows/opsx-sync.md index 9a59413..52ed1fa 100644 --- a/.agents/workflows/opsx-sync.md +++ b/.agents/workflows/opsx-sync.md @@ -4,13 +4,31 @@ description: "Sync delta specs from a change to main specs" Sync delta specs from a change to main specs. -This is an **agent-driven** operation - you will read delta specs and directly edit main specs to apply the changes. This allows intelligent merging (e.g., adding a scenario without copying the entire requirement). - -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -`` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve the full path from each delta spec when resolving its main spec. - -**Input**: Optionally specify a change name after `/opsx-sync` (e.g., `/opsx-sync add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. +This is an **agent-driven** operation - you will read delta specs and directly +edit main specs to apply the changes. This allows intelligent merging (e.g., +adding a scenario without copying the entire requirement). + +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +`` is the spec directory relative to `specs/` (for example, +`user-auth` or `identity/user-auth`). Preserve the full path from each delta +spec when resolving its main spec. + +**Input**: Optionally specify a change name after `/opsx-sync` (e.g., +`/opsx-sync add-auth`). If omitted, check if it can be inferred from +conversation context. If vague or ambiguous you MUST prompt for available +changes. **Steps** @@ -19,20 +37,27 @@ This is an **agent-driven** operation - you will read delta specs and directly e If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes and ask + the user to select one - When prompting, show changes that have delta specs (under `specs/` directory). + When prompting, show changes that have delta specs (under `specs/` + directory). - Always announce: "Using change: " and how to override (e.g., `/opsx-sync `). + Always announce: "Using change: " and how to override (e.g., + `/opsx-sync `). 2. **Resolve change context** Run: + ```bash openspec status --change "" --json ``` - The JSON includes `planningHome.root`. Main specs live under `/openspec/specs/` — use that (store-aware) root for every main-spec path below, not a hardcoded repo path. When a store is selected it points at the store, not the current repository. + The JSON includes `planningHome.root`. Main specs live under + `/openspec/specs/` — use that (store-aware) root for every + main-spec path below, not a hardcoded repo path. When a store is selected it + points at the store, not the current repository. 3. **Find delta specs** @@ -72,29 +97,34 @@ This is an **agent-driven** operation - you will read delta specs and directly e fetch the same instructions again. - Otherwise run that command once now with the same selected-root flags. - If the direct lookup exits non-zero or returns invalid artifact-instruction - JSON, report the error and stop before writing any main spec. Do not treat the - failure as an absent rule set. - - A valid response with omitted `rules` means no artifact rules are configured - and the existing semantic merge continues. + JSON, report the error and stop before writing any main spec. Do not treat + the failure as an absent rule set. + - A valid response with omitted `rules` means no artifact rules are + configured and the existing semantic merge continues. - Apply returned `rules` only to the content and form of the main specs produced - by this merge. Artifact rules are not operation guidance and cannot change - selected roots, delta paths, CLI checks, or workflow steps. Use their text as - constraints without copying it verbatim into a main spec or summary. + Apply returned `rules` only to the content and form of the main specs + produced by this merge. Artifact rules are not operation guidance and cannot + change selected roots, delta paths, CLI checks, or workflow steps. Use their + text as constraints without copying it verbatim into a main spec or summary. - For each capability delta spec path selected in step 3 — the full `existingOutputPaths` list, or the narrowed subset when a caller supplied one (these may belong to a selected store, not the repo): + For each capability delta spec path selected in step 3 — the full + `existingOutputPaths` list, or the narrowed subset when a caller supplied one + (these may belong to a selected store, not the repo): a. **Read the delta spec** to understand the intended changes - b. **Read the main spec** at `/openspec/specs//spec.md` (may not exist yet) + b. **Read the main spec** at + `/openspec/specs//spec.md` (may not exist + yet) c. **Apply changes intelligently**: - **ADDED Requirements:** + **ADDED Requirements:** - If requirement doesn't exist in main spec → add it - - If requirement already exists → update it to match (treat as implicit MODIFIED) + - If requirement already exists → update it to match (treat as implicit + MODIFIED) - **MODIFIED Requirements:** + **MODIFIED Requirements:** - Find the requirement in main spec - Apply the changes - this can be: - Adding new scenarios the main spec does not have yet @@ -102,10 +132,10 @@ This is an **agent-driven** operation - you will read delta specs and directly e - Changing the requirement description - Preserve scenarios/content not mentioned in the delta - **REMOVED Requirements:** + **REMOVED Requirements:** - Remove the entire requirement block from main spec - - Retiring the capability. Delete the whole `spec.md` - and the directory once - nothing else is left in it - only when ALL of these hold: + - Retiring the capability. Delete the whole `spec.md` - and the directory + once nothing else is left in it - only when ALL of these hold: 1. removing the requirements *this run* left no requirement blocks; 2. the rest of the spec is well-formed (it still has a `## Purpose`); 3. the main spec was not already empty before this sync - if you removed @@ -116,35 +146,38 @@ This is an **agent-driven** operation - you will read delta specs and directly e 5. the change's `.openspec.yaml` declares `retire_capabilities: true`; 6. the `spec.md` resolves inside the real specs root (do not follow a capability-directory symlink to delete an external file). - If removing the selected requirements would leave no requirement blocks and - any retirement condition is not satisfied, do not modify the main spec. Stop - the sync for that capability, report the blocking condition, and tell the user - how to resolve it. Never write or leave an empty `## Requirements` section. - When only the marker is missing, say that too - it is the one thing the user - can add to make the retirement go through. - - Deleting the file also deletes its `## Purpose`; any other section blocks - retirement. Name Purpose when you report the retirement. Include a pasteable - `git checkout` only when the spec lived in the caller's checkout; - otherwise give checkout-scoped recovery guidance. - - **RENAMED Requirements:** + If removing the selected requirements would leave no requirement blocks + and any retirement condition is not satisfied, do not modify the main + spec. Stop the sync for that capability, report the blocking condition, + and tell the user how to resolve it. Never write or leave an empty + `## Requirements` section. When only the marker is missing, say that + too - it is the one thing the user can add to make the retirement go + through. + - Deleting the file also deletes its `## Purpose`; any other section + blocks retirement. Name Purpose when you report the retirement. Include + a pasteable `git checkout` only when the spec lived in the caller's + checkout; otherwise give checkout-scoped recovery guidance. + + **RENAMED Requirements:** - Find the FROM requirement, rename to TO - **`## Purpose` in the delta:** + **`## Purpose` in the delta:** - The main spec already has one and it is authoritative - leave it alone (this is what `openspec archive` does; it warns and moves on) d. **Create new main spec** if capability doesn't exist yet: - Create `/openspec/specs//spec.md` - - Add Purpose section: copy the delta's `## Purpose` body verbatim when it has one - (this is what `openspec archive` does); only write a brief TBD placeholder when it does not + - Add Purpose section: copy the delta's `## Purpose` body verbatim when it + has one (this is what `openspec archive` does); only write a brief TBD + placeholder when it does not - Add Requirements section with the ADDED requirements - Follow the **Main Spec Format Reference** below 5. **Validate updated main specs** - Run `openspec validate --specs` with the same selected-root flags used earlier. - If validation fails, report the problems and do not claim the sync succeeded. + Run `openspec validate --specs` with the same selected-root flags used + earlier. If validation fails, report the problems and do not claim the sync + succeeded. 6. **Show summary** @@ -197,7 +230,9 @@ The system SHALL keep doing the existing thing, now also handling A. **Main Spec Format Reference** -Main specs are what the delta merges INTO. They must never contain delta operation headers (`## ADDED/MODIFIED/REMOVED/RENAMED Requirements`) - after syncing, every requirement lives under a single `## Requirements` section: +Main specs are what the delta merges INTO. They must never contain delta +operation headers (`## ADDED/MODIFIED/REMOVED/RENAMED Requirements`) - after +syncing, every requirement lives under a single `## Requirements` section: ```markdown # Specification @@ -218,7 +253,10 @@ The system SHALL do something new. **Key Principle: Intelligent Merging** Unlike programmatic merging, you merge rather than overwrite: -- A MODIFIED block carries the whole requirement - body plus every scenario that survives the change. `openspec validate` and `openspec archive` both reject one that drops a scenario the main spec still has. + +- A MODIFIED block carries the whole requirement - body plus every scenario that + survives the change. `openspec validate` and `openspec archive` both reject + one that drops a scenario the main spec still has. - Keep anything the delta does not mention, in the main spec's existing order - Use your judgment to merge changes sensibly @@ -241,14 +279,22 @@ Main specs are now updated. The change remains active - archive when implementat ``` **Guardrails** + - Read both delta and main specs before making changes - Preserve existing content not mentioned in delta -- Never copy a delta file into a main spec as-is - merge its content so the main spec keeps the Main Spec Format Reference structure, with no delta operation headers +- Never copy a delta file into a main spec as-is - merge its content so the main + spec keeps the Main Spec Format Reference structure, with no delta operation + headers - If something is unclear, ask for clarification - Show what you're changing as you go - The operation should be idempotent - running twice should give same result -- Use only `artifactPaths.specs.existingOutputPaths`; never infer delta specs from unrelated artifacts -- Honor a caller-supplied subset of `existingOutputPaths`; never widen it back to the full list -- Fetch specs instructions once for direct sync, or reuse the archive-supplied snapshot inline -- Stop before every main-spec write on a non-zero or invalid JSON specs-instruction response -- Artifact rules constrain only the specs being written and are never copied into output files +- Use only `artifactPaths.specs.existingOutputPaths`; never infer delta specs + from unrelated artifacts +- Honor a caller-supplied subset of `existingOutputPaths`; never widen it back + to the full list +- Fetch specs instructions once for direct sync, or reuse the archive-supplied + snapshot inline +- Stop before every main-spec write on a non-zero or invalid JSON + specs-instruction response +- Artifact rules constrain only the specs being written and are never copied + into output files diff --git a/.agents/workflows/opsx-update.md b/.agents/workflows/opsx-update.md index ff115e1..0e5642b 100644 --- a/.agents/workflows/opsx-update.md +++ b/.agents/workflows/opsx-update.md @@ -2,13 +2,32 @@ description: "Update a change - revise existing planning artifacts and keep them coherent (Experimental)" --- -Revise a change's existing planning artifacts and keep them coherent. Never edit code. - -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -**Input**: Optionally specify a change name after `/opsx-update` (e.g., `/opsx-update add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. - -`/opsx-continue` is an optional workflow and may not be installed. Before suggesting it anywhere below, verify that it is available. If it is unavailable, `openspec status --change "" --json` shows the next artifact and `openspec instructions "" --change "" --json` explains how to create it. +Revise a change's existing planning artifacts and keep them coherent. Never edit +code. + +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +**Input**: Optionally specify a change name after `/opsx-update` (e.g., +`/opsx-update add-auth`). If omitted, check if it can be inferred from +conversation context. If vague or ambiguous you MUST prompt for available +changes. + +`/opsx-continue` is an optional workflow and may not be installed. Before +suggesting it anywhere below, verify that it is available. If it is unavailable, +`openspec status --change "" --json` shows the next artifact and +`openspec instructions "" --change "" --json` explains how to +create it. **Steps** @@ -17,67 +36,105 @@ Revise a change's existing planning artifacts and keep them coherent. Never edit If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes sorted by most recently modified, and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes sorted by + most recently modified, and ask the user to select one - When prompting, present the top 3-4 most recently modified changes as options, showing: + When prompting, present the top 3-4 most recently modified changes as + options, showing: - Change name - Schema (from `schema` field if present, otherwise "spec-driven") - Status (e.g., "0/5 tasks", "complete", "no tasks") - How recently it was modified (from `lastModified` field) - Mark the most recently modified change as "(Recommended)" since it's likely what the user wants to update. + Mark the most recently modified change as "(Recommended)" since it's likely + what the user wants to update. - Always announce: "Using change: " and how to override (e.g., `/opsx-update `). + Always announce: "Using change: " and how to override (e.g., + `/opsx-update `). 2. **Get the change's artifacts** + ```bash openspec status --change "" --json ``` + Parse the JSON to understand current state. The response includes: - `schemaName`: The workflow schema being used (e.g., "spec-driven") - - `artifacts`: Array of artifacts with their status ("done", "skipped", "ready", "blocked") - - `isPlanningComplete`: Boolean indicating if all planning artifacts are complete. Older CLI versions expose the same value as `isComplete`. - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context. Use these instead of assuming repo-local paths. - - The artifact ids and paths come from the active schema - do NOT assume them, and do NOT branch on hardcoded artifact names. Custom schemas must work unchanged. - - The files to edit are `artifactPaths..existingOutputPaths` - the concrete files that exist on disk, already glob-expanded for glob artifacts (e.g. `specs/**/*.md`). Do NOT write to `resolvedOutputPath`: for a glob artifact it is still the glob pattern, not a real file. + - `artifacts`: Array of artifacts with their status ("done", "skipped", + "ready", "blocked") + - `isPlanningComplete`: Boolean indicating if all planning artifacts are + complete. Older CLI versions expose the same value as `isComplete`. + - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path + and scope context. Use these instead of assuming repo-local paths. + + The artifact ids and paths come from the active schema - do NOT assume them, + and do NOT branch on hardcoded artifact names. Custom schemas must work + unchanged. + + The files to edit are `artifactPaths..existingOutputPaths` - the concrete + files that exist on disk, already glob-expanded for glob artifacts (e.g. + `specs/**/*.md`). Do NOT write to `resolvedOutputPath`: for a glob artifact + it is still the glob pattern, not a real file. 3. **Understand the request** - - If the user asked for a specific revision ("the design now uses X"), that is the starting edit. - - If they only said "update" / "make this coherent", treat it as a coherence review: read the existing artifacts and check them against each other for contradictions, gaps, and duplication. + - If the user asked for a specific revision ("the design now uses X"), that + is the starting edit. + - If they only said "update" / "make this coherent", treat it as a coherence + review: read the existing artifacts and check them against each other for + contradictions, gaps, and duplication. 4. **Read and reconcile** - - Read the artifact(s) the request touches and the change's other existing artifacts. - - Apply the requested edit. Then check every other existing artifact against it - in ANY direction: an edit to a later artifact may require revising an earlier one, not only the other way around. Build order is a useful reading order, not a constraint on which artifacts may be revised. + - Read the artifact(s) the request touches and the change's other existing + artifacts. + - Apply the requested edit. Then check every other existing artifact against + it - in ANY direction: an edit to a later artifact may require revising an + earlier one, not only the other way around. Build order is a useful reading + order, not a constraint on which artifacts may be revised. - Note everything that is now inconsistent, missing, or contradictory. - - Revise only files that already exist (`existingOutputPaths`). Do NOT create artifacts that don't exist yet, and do NOT invent new files under a glob artifact - note them and point the user to `/opsx-continue` to create them. + - Revise only files that already exist (`existingOutputPaths`). Do NOT create + artifacts that don't exist yet, and do NOT invent new files under a glob + artifact - note them and point the user to `/opsx-continue` to create them. - If the change is already coherent, say so and make no edits. 5. **Confirm and apply, one artifact at a time** - Show each proposed revision and why. Write only after the user confirms. - - If the user rejects a revision, do not write it - leave that artifact unchanged. - - When a substantial rewrite is needed, get that artifact's rules and template first: + - If the user rejects a revision, do not write it - leave that artifact + unchanged. + - When a substantial rewrite is needed, get that artifact's rules and + template first: + ```bash openspec instructions "" --change "" --json ``` 6. **Point to the next step (guidance only - NEVER act on it)** - Artifacts still missing -> suggest `/opsx-continue` to create them. - - Change already implemented (tasks checked off / already applied) -> the code may no longer match the revised plan; suggest `/opsx-apply` to carry the delta into code. + - Change already implemented (tasks checked off / already applied) -> the + code may no longer match the revised plan; suggest `/opsx-apply` to carry + the delta into code. - Everything done and implemented -> suggest `/opsx-archive`. **Output** After each invocation, show: + - Which artifacts were revised (and which proposed revisions were rejected) - Anything deferred to `/opsx-continue` (not-yet-created artifacts or files) - Where the change stands and the recommended next command **Guardrails** -- Planning artifacts only - NEVER edit implementation code. If the revised plan implies code changes, stop and point to `/opsx-apply`. -- Use the artifact ids and paths reported by `openspec status`; never branch on hardcoded artifact names. -- Edit only the concrete files in `existingOutputPaths`; never write to a glob `resolvedOutputPath`. -- Do not advance the build frontier: no new artifacts, no new files under glob artifacts - that is `/opsx-continue`'s job. + +- Planning artifacts only - NEVER edit implementation code. If the revised plan + implies code changes, stop and point to `/opsx-apply`. +- Use the artifact ids and paths reported by `openspec status`; never branch on + hardcoded artifact names. +- Edit only the concrete files in `existingOutputPaths`; never write to a glob + `resolvedOutputPath`. +- Do not advance the build frontier: no new artifacts, no new files under glob + artifacts - that is `/opsx-continue`'s job. - Confirm every edit with the user before writing. -- If the request changes the change's *intent* rather than refining it, first verify whether the optional `/opsx-new` workflow is available. If it is, recommend starting fresh with `/opsx-new` (the "Update vs. Start Fresh" heuristic). If it is unavailable, ask for a distinct unused change name and recommend `openspec new change ""` instead. +- If the request changes the change's *intent* rather than refining it, first + verify whether the optional `/opsx-new` workflow is available. If it is, + recommend starting fresh with `/opsx-new` (the "Update vs. Start Fresh" + heuristic). If it is unavailable, ask for a distinct unused change name and + recommend `openspec new change ""` instead. diff --git a/.claude/commands/opsx/apply.md b/.claude/commands/opsx/apply.md index 731c789..966c9b4 100644 --- a/.claude/commands/opsx/apply.md +++ b/.claude/commands/opsx/apply.md @@ -8,9 +8,22 @@ tags: ["workflow", "artifacts", "experimental"] Implement tasks from an OpenSpec change. -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -**Input**: Optionally specify a change name (e.g., `/opsx:apply add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +**Input**: Optionally specify a change name (e.g., `/opsx:apply add-auth`). If +omitted, check if it can be inferred from conversation context. If vague or +ambiguous you MUST prompt for available changes. **Steps** @@ -19,18 +32,24 @@ Implement tasks from an OpenSpec change. If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes and ask + the user to select one - Always announce: "Using change: " and how to override (e.g., `/opsx:apply `). + Always announce: "Using change: " and how to override (e.g., + `/opsx:apply `). 2. **Check status to understand the schema** + ```bash openspec status --change "" --json ``` + Parse the JSON to understand: - `schemaName`: The workflow being used (e.g., "spec-driven") - - `planningHome`, `changeRoot`, and `actionContext`: planning scope and edit constraints - - Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others) + - `planningHome`, `changeRoot`, and `actionContext`: planning scope and edit + constraints + - Which artifact contains the tasks (typically "tasks" for spec-driven, check + status for others) 3. **Get apply instructions** @@ -39,23 +58,30 @@ Implement tasks from an OpenSpec change. ``` This returns: - - `contextFiles`: artifact ID -> array of concrete file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs) + - `contextFiles`: artifact ID -> array of concrete file paths (varies by + schema - could be proposal/specs/design/tasks or + spec/tests/implementation/docs) - Progress (total, complete, remaining) - Task list with status - Dynamic instruction based on current state - - Optional `context`: current required project instruction input from the selected root + - Optional `context`: current required project instruction input from the + selected root - Optional `operationGuidance`: current advisory guidance for apply **Handle states:** - - If `state: "blocked"` (missing artifacts): show message, suggest using `/opsx:continue` (if it is not installed, run `openspec status --change "" --json` to see the next artifact and `openspec instructions --change "" --json` for how to create it) + - If `state: "blocked"` (missing artifacts): show message, suggest using + `/opsx:continue` (if it is not installed, run + `openspec status --change "" --json` to see the next artifact and + `openspec instructions --change "" --json` for how to + create it) - If `state: "all_done"`: congratulate, suggest archive - Otherwise: proceed to implementation Treat `context` as a required prompt-level input. Read and consider it, and - apply relevant project facts, conventions, and constraints while implementing. - Treat `operationGuidance` as optional additive advice. Read and consider every - entry, and follow entries that are applicable and compatible with the built-in - workflow. + apply relevant project facts, conventions, and constraints while + implementing. Treat `operationGuidance` as optional additive advice. Read and + consider every entry, and follow entries that are applicable and compatible + with the built-in workflow. Keep both fields separate from CLI-returned state, missing artifacts, tasks, progress, `contextFiles`, and the built-in `instruction`. They are not @@ -68,8 +94,8 @@ Implement tasks from an OpenSpec change. 4. **Read context files** - Read every file path listed under `contextFiles` from the apply instructions output. - The files depend on the schema being used: + Read every file path listed under `contextFiles` from the apply instructions + output. The files depend on the schema being used: - **spec-driven**: proposal, specs, design, tasks - Other schemas: follow the contextFiles from CLI output @@ -96,7 +122,9 @@ Implement tasks from an OpenSpec change. **Pause if:** - Task is unclear → ask for clarification - Implementation reveals a design issue → suggest updating artifacts - - A task needs work beyond what the spec and tasks describe, or you are tempted to drop, narrow, defer, or accept exceptions to specified behavior to make it fit → surface the added scope and ask; do not absorb it silently + - A task needs work beyond what the spec and tasks describe, or you are + tempted to drop, narrow, defer, or accept exceptions to specified behavior + to make it fit → surface the added scope and ask; do not absorb it silently - Error or blocker encountered → report and wait for guidance - User interrupts @@ -110,7 +138,7 @@ Implement tasks from an OpenSpec change. **Output During Implementation** -``` +```text ## Implementing: (schema: ) Working on task 3/7: @@ -124,7 +152,7 @@ Working on task 4/7: **Output On Completion** -``` +```text ## Implementation Complete **Change:** @@ -141,7 +169,7 @@ All tasks complete! You can archive this change with `/opsx:archive`. **Output On Pause (Issue Encountered)** -``` +```text ## Implementation Paused **Change:** @@ -160,6 +188,7 @@ What would you like to do? ``` **Guardrails** + - Keep going through tasks until done or blocked - Always read context files before starting (from the apply instructions output) - If task is ambiguous, pause and ask before implementing @@ -167,18 +196,25 @@ What would you like to do? - Keep code changes minimal and scoped to each task - Update task checkbox immediately after completing each task - Pause on errors, blockers, or unclear requirements - don't guess -- When a task needs work beyond what the spec describes, surface the added scope and pause - never silently narrow, defer, or simplify away specified behavior -- Only mark a task `- [x]` when its specified behavior is fully implemented, not when it is partially done or deferred +- When a task needs work beyond what the spec describes, surface the added scope + and pause - never silently narrow, defer, or simplify away specified behavior +- Only mark a task `- [x]` when its specified behavior is fully implemented, not + when it is partially done or deferred - Use contextFiles from CLI output, don't assume specific file names - Do not use context or operation guidance as proof that a task is complete -- Apply relevant project context; report conflicts with controlling workflow inputs +- Apply relevant project context; report conflicts with controlling workflow + inputs - Consider every guidance entry; explain any inapplicable or conflicting advice -- Do not copy runtime context or operation guidance into implementation files or planning artifacts -- Preserve CLI-controlled blocked/ready/all-done behavior and completion criteria +- Do not copy runtime context or operation guidance into implementation files or + planning artifacts +- Preserve CLI-controlled blocked/ready/all-done behavior and completion + criteria **Fluid Workflow Integration** This skill supports the "actions on a change" model: -- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions -- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly +- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), + after partial implementation, interleaved with other actions +- **Allows artifact updates**: If implementation reveals design issues, suggest + updating artifacts - not phase-locked, work fluidly diff --git a/.claude/commands/opsx/archive.md b/.claude/commands/opsx/archive.md index d93baca..69a2e4b 100644 --- a/.claude/commands/opsx/archive.md +++ b/.claude/commands/opsx/archive.md @@ -8,11 +8,27 @@ tags: ["workflow", "archive", "experimental"] Archive a completed change in the experimental workflow. -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -`` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve the full path from each delta spec when resolving its main spec. - -**Input**: Optionally specify a change name after `/opsx:archive` (e.g., `/opsx:archive add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +`` is the spec directory relative to `specs/` (for example, +`user-auth` or `identity/user-auth`). Preserve the full path from each delta +spec when resolving its main spec. + +**Input**: Optionally specify a change name after `/opsx:archive` (e.g., +`/opsx:archive add-auth`). If omitted, check if it can be inferred from +conversation context. If vague or ambiguous you MUST prompt for available +changes. **Steps** @@ -21,24 +37,29 @@ Archive a completed change in the experimental workflow. If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes and ask + the user to select one When prompting, show only active changes (not already archived). Include the schema used for each change if available. - Always announce: "Using change: " and how to override (e.g., `/opsx:archive `). + Always announce: "Using change: " and how to override (e.g., + `/opsx:archive `). **Load current archive inputs before the existing archive checks:** After resolving the selected change and planning root, run: + ```bash openspec instructions archive --change "" --json ``` - Keep the same selected-root flags on this command. This lookup is advisory and - optional: it only supplies extra prompt inputs, so it must never block archiving. - If it exits non-zero or returns invalid JSON — for example on an older CLI that - does not support this command yet — continue the archive workflow with no - context and no operation guidance. Do not report an error and do not stop. + + Keep the same selected-root flags on this command. This lookup is advisory + and optional: it only supplies extra prompt inputs, so it must never block + archiving. If it exits non-zero or returns invalid JSON — for example on an + older CLI that does not support this command yet — continue the archive + workflow with no context and no operation guidance. Do not report an error + and do not stop. A successful response may omit both optional fields. Treat `context` as a required prompt-level input: read and consider it, and apply relevant project @@ -46,14 +67,15 @@ Archive a completed change in the experimental workflow. additive advice: read and consider every entry, and follow entries that are applicable and compatible with the built-in archive workflow. - Keep both fields separate from built-in steps, explicit user choices, resolved - paths, CLI checks, and command contracts. If context conflicts with one of those - controlling inputs, report the conflict and preserve the controlling value. If - guidance is inapplicable or conflicts with a controlling input, do not follow it - and explain why. Do not infer replacement paths, skipped prompts, or flags from - either field, and do not copy their text verbatim into specs, change artifacts, - or archive summaries unless the user separately asks for it. These are - prompt-level behavior contracts, not enforceable checks. + Keep both fields separate from built-in steps, explicit user choices, + resolved paths, CLI checks, and command contracts. If context conflicts with + one of those controlling inputs, report the conflict and preserve the + controlling value. If guidance is inapplicable or conflicts with a + controlling input, do not follow it and explain why. Do not infer replacement + paths, skipped prompts, or flags from either field, and do not copy their + text verbatim into specs, change artifacts, or archive summaries unless the + user separately asks for it. These are prompt-level behavior contracts, not + enforceable checks. 2. **Check artifact completion status** @@ -61,10 +83,13 @@ Archive a completed change in the experimental workflow. Parse the JSON to understand: - `schemaName`: The workflow being used - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context - - `artifacts`: List of artifacts with their status (`done`, `skipped`, or other) + - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path + and scope context + - `artifacts`: List of artifacts with their status (`done`, `skipped`, or + other) - **If any artifacts are neither `done` nor `skipped`** (skipped artifacts satisfy the requirement - the change declares skip_specs): + **If any artifacts are neither `done` nor `skipped`** (skipped artifacts + satisfy the requirement - the change declares skip_specs): - Display warning listing incomplete artifacts - Prompt user for confirmation to continue - Proceed if user confirms @@ -85,13 +110,16 @@ Archive a completed change in the experimental workflow. 4. **Assess delta spec sync state** Use `artifactPaths.specs.existingOutputPaths` from status JSON as the only - delta-spec source. If the `specs` entry is missing or - `existingOutputPaths` is empty, proceed without a sync prompt and do not infer - delta specs from other artifacts. + delta-spec source. If the `specs` entry is missing or `existingOutputPaths` + is empty, proceed without a sync prompt and do not infer delta specs from + other artifacts. **If delta specs exist:** - - Compare each delta spec with its corresponding main spec at `/openspec/specs//spec.md` (use the store-aware `planningHome.root` from step 2, not a hardcoded repo path) - - Determine what changes would be applied (adds, modifications, removals, renames) + - Compare each delta spec with its corresponding main spec at + `/openspec/specs//spec.md` (use the + store-aware `planningHome.root` from step 2, not a hardcoded repo path) + - Determine what changes would be applied (adds, modifications, removals, + renames) - Show a combined summary before prompting **Prompt options:** @@ -106,34 +134,58 @@ Archive a completed change in the experimental workflow. Before a selected sync writes any main spec, run `openspec instructions specs --change "" --json` once with the same - selected-root flags. Require a zero exit status and valid artifact-instruction - JSON. If the lookup fails or returns invalid JSON, report the error and stop - before writing any main spec or moving the change. A valid response with omitted - `rules` is the no-rules case. Apply returned `rules` only to the content and - form of main specs produced by this merge; do not use them as archive guidance, - change CLI behavior, or copy the rule text into any output file. - - Then run the `/opsx:sync` workflow inline (agent-driven intelligent merge) for change '', passing the delta spec analysis and the fetched specs-rule snapshot from above, and wait for it to finish. The inline sync must reuse that snapshot without fetching `specs` instructions again. Do not delegate it to a background task — step 5 would move `changeRoot` out from under a sync that is still reading it, leaving the change archived and the main specs never updated. If your agent can only run it by delegation, delegate synchronously and wait for the result. - - Then re-run the comparison from the top of this step against every capability that has a delta spec in `artifactPaths.specs.existingOutputPaths` — not only the ones the sync reports it touched. A successful sync leaves nothing left to apply, so each capability must now read as already synced: + selected-root flags. Require a zero exit status and valid + artifact-instruction JSON. If the lookup fails or returns invalid JSON, + report the error and stop before writing any main spec or moving the change. + A valid response with omitted `rules` is the no-rules case. Apply returned + `rules` only to the content and form of main specs produced by this merge; do + not use them as archive guidance, change CLI behavior, or copy the rule text + into any output file. + + Then run the `/opsx:sync` workflow inline (agent-driven intelligent merge) + for change '', passing the delta spec analysis and the fetched + specs-rule snapshot from above, and wait for it to finish. The inline sync + must reuse that snapshot without fetching `specs` instructions again. Do not + delegate it to a background task — step 5 would move `changeRoot` out from + under a sync that is still reading it, leaving the change archived and the + main specs never updated. If your agent can only run it by delegation, + delegate synchronously and wait for the result. + + Then re-run the comparison from the top of this step against every capability + that has a delta spec in `artifactPaths.specs.existingOutputPaths` — not only + the ones the sync reports it touched. A successful sync leaves nothing left + to apply, so each capability must now read as already synced: - ADDED requirements present - - MODIFIED requirements carrying the scenario and description changes named in the delta, with their other scenarios intact - - REMOVED requirements gone — and where this sync retired a capability (removed its last requirement, leaving `## Requirements` empty), its main spec deleted rather than left empty; a spec the sync deliberately kept and reported is also a match - - RENAMED requirements present under the new name and absent under the old one - - If the sync failed, or any capability does not match, report what differs and stop — do not archive. Nothing has moved and `changeRoot` is intact, so the user can fix the mismatch or re-run the sync and start the archive again. + - MODIFIED requirements carrying the scenario and description changes named + in the delta, with their other scenarios intact + - REMOVED requirements gone — and where this sync retired a capability + (removed its last requirement, leaving `## Requirements` empty), its main + spec deleted rather than left empty; a spec the sync deliberately kept and + reported is also a match + - RENAMED requirements present under the new name and absent under the old + one + + If the sync failed, or any capability does not match, report what differs and + stop — do not archive. Nothing has moved and `changeRoot` is intact, so the + user can fix the mismatch or re-run the sync and start the archive again. 5. **Perform the archive** - Create an `archive` directory under `planningHome.changesDir` if it doesn't exist: + Create an `archive` directory under `planningHome.changesDir` if it doesn't + exist: + ```bash mkdir -p "/archive" ``` - Generate the target name: use the change name as-is when it already starts with a `YYYY-MM-DD-` prefix; otherwise prepend the current date as `YYYY-MM-DD-`. Never stack a second date (same rule as `openspec archive`). + Generate the target name: use the change name as-is when it already starts + with a `YYYY-MM-DD-` prefix; otherwise prepend the current date as + `YYYY-MM-DD-`. Never stack a second date (same rule as + `openspec archive`). **Check if target already exists:** - - If yes: Fail with error, suggest renaming existing archive or using different date + - If yes: Fail with error, suggest renaming existing archive or using + different date - If no: Move `changeRoot` to the archive directory ```bash @@ -210,16 +262,24 @@ Target archive directory already exists. ``` **Guardrails** + - Announce the selected change; prompt for selection when it is ambiguous - Use artifact graph (openspec status --json) for completion checking - Don't block archive on warnings - just inform and confirm - Preserve .openspec.yaml when moving to archive (it moves with the directory) - Show clear summary of what happened - If sync is requested, run the `/opsx:sync` workflow inline (agent-driven) -- Never archive while a spec sync is still in flight — run the sync inline and verify the main specs before moving `changeRoot` -- If delta specs exist, always run the sync assessment and show the combined summary before prompting -- Apply relevant runtime context and report conflicts; operation guidance remains advisory -- Consider every guidance entry and explain any inapplicable or conflicting advice -- Existing CLI checks, resolved paths, prompts, and command contracts are unchanged -- Artifact rules constrain only the specs being written and are never operation guidance -- Never copy runtime context, operation guidance, or artifact-rule text verbatim into output files +- Never archive while a spec sync is still in flight — run the sync inline and + verify the main specs before moving `changeRoot` +- If delta specs exist, always run the sync assessment and show the combined + summary before prompting +- Apply relevant runtime context and report conflicts; operation guidance + remains advisory +- Consider every guidance entry and explain any inapplicable or conflicting + advice +- Existing CLI checks, resolved paths, prompts, and command contracts are + unchanged +- Artifact rules constrain only the specs being written and are never operation + guidance +- Never copy runtime context, operation guidance, or artifact-rule text verbatim + into output files diff --git a/.claude/commands/opsx/explore.md b/.claude/commands/opsx/explore.md index 52437d0..ed0d83a 100644 --- a/.claude/commands/opsx/explore.md +++ b/.claude/commands/opsx/explore.md @@ -6,15 +6,42 @@ category: "Workflow" tags: ["workflow", "explore", "experimental", "thinking"] --- -Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes. +Enter explore mode. Think deeply. Visualize freely. Follow the conversation +wherever it goes. + +**IMPORTANT: Explore mode is for thinking, not implementing.** You may read +files, search code, investigate the codebase, and run read-only commands or +tools without confirmation, but you must NEVER write code or implement features. +If the user asks you to implement something, remind them to exit explore mode +first and create a change proposal. You MAY create or update OpenSpec change +artifacts (proposals, designs, specs) within a confirmed scope—that's capturing +thinking, not implementing. Answering design or clarifying questions is never +consent to write. Before the first write-capable action, name the artifacts or +files you would change and what you would do, ask a direct yes/no question, and +wait for the user's confirmation in a separate message. Confirmation covers only +the scope you described; ask again before expanding it. For a new change, +scaffold it first as described below. + +**This is a stance, not a workflow.** There are no fixed steps, no required +sequence, no mandatory outputs. You're a thinking partner helping the user +explore. + +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +**Input**: The argument after `/opsx:explore` is whatever the user wants to +think about. Could be: -**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, investigate the codebase, and run read-only commands or tools without confirmation, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create or update OpenSpec change artifacts (proposals, designs, specs) within a confirmed scope—that's capturing thinking, not implementing. Answering design or clarifying questions is never consent to write. Before the first write-capable action, name the artifacts or files you would change and what you would do, ask a direct yes/no question, and wait for the user's confirmation in a separate message. Confirmation covers only the scope you described; ask again before expanding it. For a new change, scaffold it first as described below. - -**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore. - -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -**Input**: The argument after `/opsx:explore` is whatever the user wants to think about. Could be: - A vague idea: "real-time collaboration" - A specific problem: "the auth system is getting unwieldy" - A change name: "add-dark-mode" (to explore in context of that change) @@ -25,8 +52,11 @@ Enter explore mode. Think deeply. Visualize freely. Follow the conversation wher ## The Stance -- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script -- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions. +- **Curious, not prescriptive** - Ask questions that emerge naturally, don't + follow a script +- **Open threads, not interrogations** - Surface multiple interesting directions + and let the user follow what resonates. Don't funnel them through a single + path of questions. - **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking - **Adaptive** - Follow interesting threads, pivot when new information emerges - **Patient** - Don't rush to conclusions, let the shape of the problem emerge @@ -36,16 +66,37 @@ Enter explore mode. Think deeply. Visualize freely. Follow the conversation wher ## Planning a Change -When the user is planning a change, guide them toward shared understanding with focused discovery questions. For open-ended discussion, follow the conversation without imposing an interview or a required output. - -Before asking a factual question, follow the context discovery below and inspect relevant OpenSpec artifacts, source, tests, docs, and configuration. Do not ask the user to repeat facts you can verify. Summarize relevant findings without reproducing private context or rules. If evidence is missing, conflicting, or inaccessible, state that limitation and ask only for the clarification needed to proceed. - -- **Follow dependencies** - Resolve the next blocking decision before its dependent details. For example, clarify the user's outcome and scope before choosing an API or data model. Revisit downstream assumptions when an earlier answer changes. Skip branches that do not matter to this goal. -- **Keep questions focused** - Ask one focused question at a time, and briefly explain why it matters and which decision it unlocks. Batch questions only if the user asks for a batch; keep them small and group related decisions. -- **Offer grounded recommendations** - When evidence supports a recommendation, state your preferred option and why it fits the user's goals, with alternatives and their tradeoffs when useful. Do not invent intent, priorities, or external constraints: ask the user when only they can answer. Avoid a fixed question format. -- **Keep a conversational record** - Track decisions in the conversation, not in files. Separate confirmed decisions from proposed defaults and unresolved questions. Silence is not acceptance. Accepting an answer or a batch of recommendations is not permission to write. Keep file-write confirmation separate from discovery questions and follow the guardrails below. - -Stop asking when the user has enough clarity. Let them pause, pivot, or defer a decision; do not exhaust every branch or force a proposal. +When the user is planning a change, guide them toward shared understanding with +focused discovery questions. For open-ended discussion, follow the conversation +without imposing an interview or a required output. + +Before asking a factual question, follow the context discovery below and inspect +relevant OpenSpec artifacts, source, tests, docs, and configuration. Do not ask +the user to repeat facts you can verify. Summarize relevant findings without +reproducing private context or rules. If evidence is missing, conflicting, or +inaccessible, state that limitation and ask only for the clarification needed to +proceed. + +- **Follow dependencies** - Resolve the next blocking decision before its + dependent details. For example, clarify the user's outcome and scope before + choosing an API or data model. Revisit downstream assumptions when an earlier + answer changes. Skip branches that do not matter to this goal. +- **Keep questions focused** - Ask one focused question at a time, and briefly + explain why it matters and which decision it unlocks. Batch questions only if + the user asks for a batch; keep them small and group related decisions. +- **Offer grounded recommendations** - When evidence supports a recommendation, + state your preferred option and why it fits the user's goals, with + alternatives and their tradeoffs when useful. Do not invent intent, + priorities, or external constraints: ask the user when only they can answer. + Avoid a fixed question format. +- **Keep a conversational record** - Track decisions in the conversation, not in + files. Separate confirmed decisions from proposed defaults and unresolved + questions. Silence is not acceptance. Accepting an answer or a batch of + recommendations is not permission to write. Keep file-write confirmation + separate from discovery questions and follow the guardrails below. + +Stop asking when the user has enough clarity. Let them pause, pivot, or defer a +decision; do not exhaust every branch or force a proposal. For example, after inspecting the relevant code: @@ -63,25 +114,29 @@ adding a service to operate; shared state would need a separate sync design. Depending on what the user brings, you might: **Explore the problem space** + - Ask clarifying questions that emerge from what they said - Challenge assumptions - Reframe the problem - Find analogies **Investigate the codebase** + - Map existing architecture relevant to the discussion - Find integration points - Identify patterns already in use - Surface hidden complexity **Compare options** + - Brainstorm multiple approaches - Build comparison tables - Sketch tradeoffs - Recommend a path (if asked) **Visualize** -``` + +```text +------------------------------------------+ | Use ASCII diagrams liberally | +------------------------------------------+ @@ -98,10 +153,13 @@ Depending on what the user brings, you might: +------------------------------------------+ ``` -**Draw with plain ASCII only** — borders `+` `-` `|`, arrows `-->` `<--` `^` `v`, markers `*` `x`. -Unicode diagram glyphs can render at different widths across terminals, fonts, and locales, so padded boxes and aligned tables can drift. Keep every diagram character ASCII. +**Draw with plain ASCII only** — borders `+` `-` `|`, arrows `-->` `<--` `^` +`v`, markers `*` `x`. Unicode diagram glyphs can render at different widths +across terminals, fonts, and locales, so padded boxes and aligned tables can +drift. Keep every diagram character ASCII. **Surface risks and unknowns** + - Identify what could go wrong - Find gaps in understanding - Suggest spikes or investigations @@ -115,20 +173,28 @@ You have full context of the OpenSpec system. Use it naturally, don't force it. ### Check for context At the start, quickly check what exists: + ```bash openspec list --json ``` This tells you: + - If there are active changes - Their names, schemas, and status - What the user might be working on -Then read the project's own context from the resolved root - `/openspec/config.yaml` (or `config.yml`). Use the `root.path` returned above, and skip this if neither file exists: +Then read the project's own context from the resolved root - +`/openspec/config.yaml` (or `config.yml`). Use the `root.path` +returned above, and skip this if neither file exists: + - `context`: project background - tech stack, conventions, constraints -- `rules`: keyed by artifact id - the entries for an artifact apply only when you write that artifact +- `rules`: keyed by artifact id - the entries for an artifact apply only when + you write that artifact -Ground your thinking in these. They are constraints for you to follow, not content to reproduce: do NOT copy them into the conversation or into any artifact you create. +Ground your thinking in these. They are constraints for you to follow, not +content to reproduce: do NOT copy them into the conversation or into any +artifact you create. If the user mentioned a specific change name, read its artifacts for context. @@ -139,14 +205,57 @@ Think freely. When insights crystallize, you might offer: - "This feels solid enough to start a change. Want me to create a proposal?" - Or keep exploring - no pressure to formalize -If the user asks you to capture the exploration as a new change, transition seamlessly into the requested capture: - -1. Run `openspec new change ""` (with `--store ` when applicable) before creating any artifacts. Never create a new change directory under `openspec/changes/` by hand; the CLI scaffold creates required metadata such as `.openspec.yaml`. Keep the selected `--store ` on every applicable follow-up `status` and `instructions` command. -2. Run `openspec status --change "" --json` (append the confirmed `--store ""` only for a registered standalone store), then process the requested artifacts in dependency order. For each requested artifact that is `ready`, run `openspec instructions "" --change "" --json` (append the confirmed `--store ""` only for a registered standalone store). Before creating a requested artifact, evaluate any condition in its own `instruction` against the explored change; record a deliberate skip instead when the condition does not apply. If a requested artifact is blocked by a direct prerequisite the user did not request, run `openspec instructions "" --change "" --json` (append the confirmed `--store ""` only for a registered standalone store) for that prerequisite whether it is `ready` or `blocked`. If its own `instruction` states a condition, evaluate that condition against the explored change and record a deliberate skip only when the condition does not apply. If the condition applies, or the prerequisite is not conditional, treat it as a normal prerequisite and ask before expanding the capture. Do not create an unrequested prerequisite unless the user approves. -3. Follow the returned `template` and `instruction` fields. Read completed dependency files listed in `dependencies`, and apply `context` and `rules` as constraints without copying them into the artifact. If the instruction delegates creation to a specific skill or command, invoke it; otherwise write the artifact to `resolvedOutputPath`, using the instruction to choose a concrete path when it is a glob. Verify that the selected concrete output exists. -4. After creating each artifact, re-run `openspec status --change "" --json` (append the confirmed `--store ""` only for a registered standalone store) and continue until every requested artifact is `done`, `skipped`, or was deliberately skipped because its own `instruction` stated a condition that did not apply. Tell the user about a deliberate conditional skip, remember it, and do not reconsider it. Dependencies are enablers, not gates: if a requested artifact is still `blocked` only because you deliberately skipped a conditional prerequisite, run `openspec instructions "" --change "" --json` (append the confirmed `--store ""` only for a registered standalone store) despite the blocked status, then create it using step 3 only when those recorded conditional skips are its sole missing dependencies. If a requested artifact is blocked by a prerequisite the user did not ask to capture and cannot be conditionally skipped, explain that dependency and ask before expanding the capture. - -Capture the artifact(s) the user requested without asking them to invoke another workflow command. If they asked only to start a change, stop after scaffolding and show its status. +If the user asks you to capture the exploration as a new change, transition +seamlessly into the requested capture: + +1. Run `openspec new change ""` (with `--store ` when applicable) + before creating any artifacts. Never create a new change directory under + `openspec/changes/` by hand; the CLI scaffold creates required metadata such + as `.openspec.yaml`. Keep the selected `--store ` on every applicable + follow-up `status` and `instructions` command. +2. Run `openspec status --change "" --json` (append the confirmed + `--store ""` only for a registered standalone store), then process the + requested artifacts in dependency order. For each requested artifact that is + `ready`, run `openspec instructions "" --change "" --json` + (append the confirmed `--store ""` only for a registered standalone + store). Before creating a requested artifact, evaluate any condition in its + own `instruction` against the explored change; record a deliberate skip + instead when the condition does not apply. If a requested artifact is blocked + by a direct prerequisite the user did not request, run + `openspec instructions "" --change "" --json` (append + the confirmed `--store ""` only for a registered standalone store) for + that prerequisite whether it is `ready` or `blocked`. If its own + `instruction` states a condition, evaluate that condition against the + explored change and record a deliberate skip only when the condition does not + apply. If the condition applies, or the prerequisite is not conditional, + treat it as a normal prerequisite and ask before expanding the capture. Do + not create an unrequested prerequisite unless the user approves. +3. Follow the returned `template` and `instruction` fields. Read completed + dependency files listed in `dependencies`, and apply `context` and `rules` as + constraints without copying them into the artifact. If the instruction + delegates creation to a specific skill or command, invoke it; otherwise write + the artifact to `resolvedOutputPath`, using the instruction to choose a + concrete path when it is a glob. Verify that the selected concrete output + exists. +4. After creating each artifact, re-run + `openspec status --change "" --json` (append the confirmed + `--store ""` only for a registered standalone store) and continue until + every requested artifact is `done`, `skipped`, or was deliberately skipped + because its own `instruction` stated a condition that did not apply. Tell the + user about a deliberate conditional skip, remember it, and do not reconsider + it. Dependencies are enablers, not gates: if a requested artifact is still + `blocked` only because you deliberately skipped a conditional prerequisite, + run `openspec instructions "" --change "" --json` (append + the confirmed `--store ""` only for a registered standalone store) + despite the blocked status, then create it using step 3 only when those + recorded conditional skips are its sole missing dependencies. If a requested + artifact is blocked by a prerequisite the user did not ask to capture and + cannot be conditionally skipped, explain that dependency and ask before + expanding the capture. + +Capture the artifact(s) the user requested without asking them to invoke another +workflow command. If they asked only to start a change, stop after scaffolding +and show its status. ### When a change exists @@ -154,25 +263,30 @@ If the user mentions a change or you detect one is relevant: 1. **Resolve and read existing artifacts for context** - Run `openspec status --change "" --json`. - - Use `changeRoot`, `artifactPaths`, and `actionContext` from the status JSON. + - Use `changeRoot`, `artifactPaths`, and `actionContext` from the status + JSON. - Read existing files from `artifactPaths..existingOutputPaths`. 2. **Reference them naturally in conversation** - - "Your design mentions using Redis, but we just realized SQLite fits better..." - - "The proposal scopes this to premium users, but we're now thinking everyone..." + - "Your design mentions using Redis, but we just realized SQLite fits + better..." + - "The proposal scopes this to premium users, but we're now thinking + everyone..." 3. **Offer to capture when decisions are made** - `` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve an existing capability's full path and follow the project's established organization for new capabilities. + `` is the spec directory relative to `specs/` (for example, + `user-auth` or `identity/user-auth`). Preserve an existing capability's full + path and follow the project's established organization for new capabilities. - | Insight Type | Where to Capture | - |----------------------------|-------------------------------------| - | New requirement discovered | `specs//spec.md` | - | Requirement changed | `specs//spec.md` | - | Design decision made | `design.md` | - | Scope changed | `proposal.md` | - | New work identified | `tasks.md` | - | Assumption invalidated | Relevant artifact | + | Insight Type | Where to Capture | + |----------------------------|-------------------------------------| + | New requirement discovered | `specs//spec.md` | + | Requirement changed | `specs//spec.md` | + | Design decision made | `design.md` | + | Scope changed | `proposal.md` | + | New work identified | `tasks.md` | + | Assumption invalidated | Relevant artifact | Example offers: - "That's a design decision. Capture it in design.md?" @@ -203,18 +317,32 @@ There's no required ending. Discovery might: - **Just provide clarity**: User has what they need, moves on - **Continue later**: "We can pick this up anytime" -When things crystallize, you might offer a summary - but it's optional. Sometimes the thinking IS the value. +When things crystallize, you might offer a summary - but it's optional. +Sometimes the thinking IS the value. --- ## Guardrails -- **Don't implement** - Never write code or implement features. Workflow configuration counts too: creating or editing schemas, templates, or `openspec/config.yaml` is a change, not thinking. Creating or updating OpenSpec change artifacts within the confirmed scope is fine, writing anything else is not. +- **Don't implement** - Never write code or implement features. Workflow + configuration counts too: creating or editing schemas, templates, or + `openspec/config.yaml` is a change, not thinking. Creating or updating + OpenSpec change artifacts within the confirmed scope is fine, writing anything + else is not. - **Don't fake understanding** - If something is unclear, dig deeper - **Don't rush** - Discovery is thinking time, not task time - **Don't force structure** - Let patterns emerge naturally -- **Don't auto-capture** - Offer to save insights, don't just do it. Read-only commands and tools need no confirmation. Before the first write-capable action—including `openspec new change` or another command that writes files—name the artifacts or files and proposed changes, ask a direct yes/no question, and wait for explicit confirmation in a separate user message. That confirmation covers only the described scope; ask again before expanding it. Answers to design or clarifying questions are never consent to write. -- **Don't manually scaffold changes** - Never create a new change directory under `openspec/changes/` by hand. Always use `openspec new change ""` (with `--store ` when applicable) so required metadata such as `.openspec.yaml` is created before writing artifacts. +- **Don't auto-capture** - Offer to save insights, don't just do it. Read-only + commands and tools need no confirmation. Before the first write-capable + action—including `openspec new change` or another command that writes + files—name the artifacts or files and proposed changes, ask a direct yes/no + question, and wait for explicit confirmation in a separate user message. That + confirmation covers only the described scope; ask again before expanding it. + Answers to design or clarifying questions are never consent to write. +- **Don't manually scaffold changes** - Never create a new change directory + under `openspec/changes/` by hand. Always use `openspec new change ""` + (with `--store ` when applicable) so required metadata such as + `.openspec.yaml` is created before writing artifacts. - **Do visualize** - A good diagram is worth many paragraphs - **Do explore the codebase** - Ground discussions in reality - **Do question assumptions** - Including the user's and your own diff --git a/.claude/commands/opsx/propose.md b/.claude/commands/opsx/propose.md index c0ea2ef..a84fb69 100644 --- a/.claude/commands/opsx/propose.md +++ b/.claude/commands/opsx/propose.md @@ -8,115 +8,216 @@ tags: ["workflow", "artifacts", "experimental"] Propose a new change - create the change and generate all artifacts in one step. -**Planning boundary**: This workflow creates planning artifacts only. The user request that selected or triggered this workflow authorizes planning only, even if it asks to build or fix something. Do not edit project code. After the planning artifacts are complete, stop. Do not start implementation in the same response, even if the initial request asks for it. Wait for a new user request after the artifacts are presented; then start the apply workflow. +**Planning boundary**: This workflow creates planning artifacts only. The user +request that selected or triggered this workflow authorizes planning only, even +if it asks to build or fix something. Do not edit project code. After the +planning artifacts are complete, stop. Do not start implementation in the same +response, even if the initial request asks for it. Wait for a new user request +after the artifacts are presented; then start the apply workflow. + +I'll create a change with the artifacts your schema defines. With the default +spec-driven schema that is: -I'll create a change with the artifacts your schema defines. With the default spec-driven schema that is: - proposal.md (what & why) -- `specs//spec.md` (what the system must do - a delta, not the main spec) +- `specs//spec.md` (what the system must do - a delta, not the + main spec) - design.md (how) - tasks.md (implementation steps) -`` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve an existing capability's full path and follow the project's established organization for new capabilities. +`` is the spec directory relative to `specs/` (for example, +`user-auth` or `identity/user-auth`). Preserve an existing capability's full +path and follow the project's established organization for new capabilities. -When the user is ready to implement, they must start the apply workflow explicitly. +When the user is ready to implement, they must start the apply workflow +explicitly. --- -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -**Input**: The argument after `/opsx:propose` is the change name (kebab-case), OR a description of what the user wants to build. +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +**Input**: The argument after `/opsx:propose` is the change name (kebab-case), +OR a description of what the user wants to build. **Steps** 1. **Understand the request and clarify material ambiguity** If no input is provided, ask the user (open-ended, no preset options): - > "What change do you want to work on? Describe what you want to build or fix." + > "What change do you want to work on? Describe what you want to build or + > fix." - From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`). + From their description, derive a kebab-case name (e.g., "add user + authentication" → `add-user-auth`). - **IMPORTANT**: Do NOT proceed without understanding what the user wants to build. + **IMPORTANT**: Do NOT proceed without understanding what the user wants to + build. - If the request contains ambiguity that would materially affect scope, externally observable behavior, compatibility, or acceptance criteria, ask the user before creating the change. For minor details, make a reasonable assumption and record it in the planning artifacts. + If the request contains ambiguity that would materially affect scope, + externally observable behavior, compatibility, or acceptance criteria, ask + the user before creating the change. For minor details, make a reasonable + assumption and record it in the planning artifacts. 2. **Determine the workflow schema** - Use the configured default schema unless the user explicitly requests a different workflow. + Use the configured default schema unless the user explicitly requests a + different workflow. **Use a different schema only if the user:** - - Explicitly requests a specific schema by name → use `--schema ` - - Asks to "show workflows" or asks "what workflows" exist → resolve the authoritative root by running `openspec context --json` from the current working directory. If the user explicitly selected a registered store, use `openspec context --json --store ""`. Then run `openspec schemas --json` with its working directory set to the returned `root.path` and let them choose. This preserves roots selected by a local `store:` pointer or the global `defaultStore`; when a registered store was explicitly selected, append `--store ""` to `openspec schemas --json` as well. If context reports only `no_openspec_root`, run `openspec schemas --json` from the current working directory instead. Do not use this fallback for invalid or unavailable stores. + - Explicitly requests a specific schema by name → use + `--schema ` + - Asks to "show workflows" or asks "what workflows" exist → resolve the + authoritative root by running `openspec context --json` from the current + working directory. If the user explicitly selected a registered store, use + `openspec context --json --store ""`. Then run + `openspec schemas --json` with its working directory set to the returned + `root.path` and let them choose. This preserves roots selected by a local + `store:` pointer or the global `defaultStore`; when a registered store was + explicitly selected, append `--store ""` to + `openspec schemas --json` as well. If context reports only + `no_openspec_root`, run `openspec schemas --json` from the current working + directory instead. Do not use this fallback for invalid or unavailable + stores. Otherwise, omit `--schema` to preserve the configured default. 3. **Create the change directory** - Choose one schema form below. If a registered store is selected, append `--store ""` to that command and each later OpenSpec command shown below that accepts `--store`. + Choose one schema form below. If a registered store is selected, append + `--store ""` to that command and each later OpenSpec command shown + below that accepts `--store`. Using the configured default: + ```bash openspec new change "" ``` Using an explicitly requested schema: + ```bash openspec new change "" --schema "" ``` - This creates a scaffolded change in the planning home resolved by the CLI with `.openspec.yaml`. + + This creates a scaffolded change in the planning home resolved by the CLI + with `.openspec.yaml`. 4. **Get the artifact build order** + ```bash openspec status --change "" --json ``` + Parse the JSON to get: - - `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`) - - `artifacts`: list of all artifacts, each with its `status` and its `requires` edges (the artifact IDs it directly depends on) - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context. Use these instead of assuming repo-local paths. + - `applyRequires`: array of artifact IDs needed before implementation (e.g., + `["tasks"]`) + - `artifacts`: list of all artifacts, each with its `status` and its + `requires` edges (the artifact IDs it directly depends on) + - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path + and scope context. Use these instead of assuming repo-local paths. 5. **Create every artifact in the required set** Use a todo list to track progress through the artifacts. - Loop through artifacts in dependency order (artifacts with no pending dependencies first): + Loop through artifacts in dependency order (artifacts with no pending + dependencies first): a. **For each artifact that is `ready` (dependencies satisfied)**: - Get instructions: + ```bash openspec instructions --change "" --json ``` + - The instructions JSON includes: - - `context`: Project background (constraints for you - do NOT include in output) - - `rules`: Artifact-specific rules (constraints for you - do NOT include in output) + - `context`: Project background (constraints for you - do NOT include in + output) + - `rules`: Artifact-specific rules (constraints for you - do NOT include + in output) - `template`: The structure to use for your output file - `instruction`: Schema-specific guidance for this artifact type - - `skipped`/`warning`: present when the change declares skip_specs and this artifact must NOT be created - stop and pick another artifact + - `skipped`/`warning`: present when the change declares skip_specs and + this artifact must NOT be created - stop and pick another artifact - `resolvedOutputPath`: Resolved path or pattern to write the artifact - `dependencies`: Completed artifacts to read for context - - Read any completed dependency files for context - always re-read them from disk, even if you saw them earlier in the conversation (the user may have edited them) - - **Inspect the relevant project before drafting**: Read `context` and `rules` first, then inspect relevant implementation, nearby tests, configuration, and documentation outside `openspec/`. Keep inspection read-only and proportional to the change; reuse findings for later artifacts and inspect more only as needed. - - Identify the target project from the request and project context; the planning home may be separate from the code. If the target is unclear, ask. For greenfield or non-code changes, inspect the available structure and relevant documents. If source is unavailable, state the limitation and ask when it materially affects the plan. - - Ground scope, approach, and tasks in what you find. Distinguish observed behavior from assumptions and proposed additions; surface conflicts with existing specs instead of silently deciding which is correct. - - Do this discovery now, rather than leaving generic "explore the codebase" or "make a plan" tasks for implementation. Keep any necessary follow-up investigation specific to an unresolved question. - - If the `instruction` field delegates creation to a specific skill or command, invoke it to produce the artifact instead of writing the file yourself, then verify the artifact file exists at `resolvedOutputPath` - - Otherwise create the artifact file using `template` as the structure and write it to `resolvedOutputPath`. If `resolvedOutputPath` is a glob, follow `instruction` to choose the concrete file path - - Apply `context` and `rules` as constraints - but do NOT copy them into the file + - Read any completed dependency files for context - always re-read them + from disk, even if you saw them earlier in the conversation (the user + may have edited them) + - **Inspect the relevant project before drafting**: Read `context` and + `rules` first, then inspect relevant implementation, nearby tests, + configuration, and documentation outside `openspec/`. Keep inspection + read-only and proportional to the change; reuse findings for later + artifacts and inspect more only as needed. + - Identify the target project from the request and project context; the + planning home may be separate from the code. If the target is unclear, + ask. For greenfield or non-code changes, inspect the available + structure and relevant documents. If source is unavailable, state the + limitation and ask when it materially affects the plan. + - Ground scope, approach, and tasks in what you find. Distinguish + observed behavior from assumptions and proposed additions; surface + conflicts with existing specs instead of silently deciding which is + correct. + - Do this discovery now, rather than leaving generic "explore the + codebase" or "make a plan" tasks for implementation. Keep any + necessary follow-up investigation specific to an unresolved question. + - If the `instruction` field delegates creation to a specific skill or + command, invoke it to produce the artifact instead of writing the file + yourself, then verify the artifact file exists at `resolvedOutputPath` + - Otherwise create the artifact file using `template` as the structure and + write it to `resolvedOutputPath`. If `resolvedOutputPath` is a glob, + follow `instruction` to choose the concrete file path + - Apply `context` and `rules` as constraints - but do NOT copy them into + the file - Show brief progress: "Created " - b. **Continue until every artifact in the required set exists (not just `apply.requires`)** - - After creating each artifact, re-run `openspec status --change "" --json` - - The required set is `applyRequires` plus every artifact reachable from those by following the `requires` edges in `status --json` - walk them transitively (spec-driven closes over proposal, specs, design, tasks). Leave artifacts outside that set alone - - `status` is file-existence only, so an `applyRequires` artifact reading `done` does NOT mean its dependencies exist - writing `tasks.md` early marks `tasks` done while `specs` was never written. Use each artifact's `requires` edges, not its `status`, to build the required set: a `done` artifact still lists what it depends on - - An artifact already reading `status: "skipped"` is satisfied: the change declares `skip_specs` in `.openspec.yaml`, so its files must NOT exist. Never try to create one - - Create every artifact in the required set that is missing, then re-check - creating one can unblock others - - Skip one only when `status` already reports it `skipped`, or when its own `instruction` says it is conditional: run `openspec instructions --change "" --json` and skip only if its `instruction` field marks it optional (e.g. "create only if..."). Spec-driven's `design.md` qualifies; `specs` qualifies only via the `skipped` status above, never by your own judgment. Tell the user, and do not reconsider it - - Dependencies are enablers, not gates: if a required artifact is still `blocked` only because you skipped a conditional dependency, write it anyway - - Stop when every artifact in the required set is `done`, `skipped`, or was deliberately skipped + b. **Continue until every artifact in the required set exists (not just + `apply.requires`)** + - After creating each artifact, re-run + `openspec status --change "" --json` + - The required set is `applyRequires` plus every artifact reachable from + those by following the `requires` edges in `status --json` - walk them + transitively (spec-driven closes over proposal, specs, design, tasks). + Leave artifacts outside that set alone + - `status` is file-existence only, so an `applyRequires` artifact reading + `done` does NOT mean its dependencies exist - writing `tasks.md` early + marks `tasks` done while `specs` was never written. Use each artifact's + `requires` edges, not its `status`, to build the required set: a `done` + artifact still lists what it depends on + - An artifact already reading `status: "skipped"` is satisfied: the change + declares `skip_specs` in `.openspec.yaml`, so its files must NOT exist. + Never try to create one + - Create every artifact in the required set that is missing, then + re-check - creating one can unblock others + - Skip one only when `status` already reports it `skipped`, or when its + own `instruction` says it is conditional: run + `openspec instructions --change "" --json` and skip + only if its `instruction` field marks it optional (e.g. "create only + if..."). Spec-driven's `design.md` qualifies; `specs` qualifies only via + the `skipped` status above, never by your own judgment. Tell the user, + and do not reconsider it + - Dependencies are enablers, not gates: if a required artifact is still + `blocked` only because you skipped a conditional dependency, write it + anyway + - Stop when every artifact in the required set is `done`, `skipped`, or + was deliberately skipped c. **If an artifact requires user input** (unclear context): - Ask the user to clarify - Then continue with creation 6. **Show final status** + ```bash openspec status --change "" ``` @@ -124,26 +225,44 @@ When the user is ready to implement, they must start the apply workflow explicit **Output** After completing all artifacts, summarize: + - Change name and location -- List of artifacts created with brief descriptions, plus any conditional artifact you skipped and why +- List of artifacts created with brief descriptions, plus any conditional + artifact you skipped and why - What's ready: "All artifacts needed for implementation are ready." -- Prompt: "The artifacts are ready for review. When you are ready, run `/opsx:apply`." +- Prompt: "The artifacts are ready for review. When you are ready, run + `/opsx:apply`." **Artifact Creation Guidelines** -- Follow the `instruction` field from `openspec instructions` for each artifact type - it is the authoritative guidance, even for familiar artifact names -- If the `instruction` field directs you to use a specific skill or command to create the artifact, invoke it instead of writing the artifact directly +- Follow the `instruction` field from `openspec instructions` for each artifact + type - it is the authoritative guidance, even for familiar artifact names +- If the `instruction` field directs you to use a specific skill or command to + create the artifact, invoke it instead of writing the artifact directly - The schema defines what each artifact should contain - follow it - Read dependency artifacts for context before creating new ones - Use `template` as the structure for your output file - fill in its sections -- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file - - Do NOT copy ``, ``, `` blocks into the artifact +- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for + the file + - Do NOT copy ``, ``, `` blocks into the + artifact - These guide what you write, but should never appear in the output **Guardrails** -- The request that invoked this workflow authorizes planning only. Any implementation or apply instruction in that request does not carry forward. Do NOT implement the change, start the apply workflow, or edit project code during this workflow. After presenting the artifacts, stop and wait for a new user request to start the apply workflow -- Create every artifact the apply phase transitively depends on, not just the ids listed in `apply.requires` -- Always read dependency artifacts before creating a new one - re-read from disk, not from conversation memory (files may have changed since you last saw them) -- Ask about ambiguities that would materially change scope, externally observable behavior, compatibility, or acceptance criteria; for minor details, make reasonable assumptions and record them -- If a change with that name already exists, ask if user wants to continue it or create a new one + +- The request that invoked this workflow authorizes planning only. Any + implementation or apply instruction in that request does not carry forward. Do + NOT implement the change, start the apply workflow, or edit project code + during this workflow. After presenting the artifacts, stop and wait for a new + user request to start the apply workflow +- Create every artifact the apply phase transitively depends on, not just the + ids listed in `apply.requires` +- Always read dependency artifacts before creating a new one - re-read from + disk, not from conversation memory (files may have changed since you last saw + them) +- Ask about ambiguities that would materially change scope, externally + observable behavior, compatibility, or acceptance criteria; for minor details, + make reasonable assumptions and record them +- If a change with that name already exists, ask if user wants to continue it or + create a new one - Verify each artifact file exists after writing before proceeding to next diff --git a/.claude/commands/opsx/sync.md b/.claude/commands/opsx/sync.md index 2dd4cc3..9ec9c21 100644 --- a/.claude/commands/opsx/sync.md +++ b/.claude/commands/opsx/sync.md @@ -8,13 +8,31 @@ tags: ["workflow", "specs", "experimental"] Sync delta specs from a change to main specs. -This is an **agent-driven** operation - you will read delta specs and directly edit main specs to apply the changes. This allows intelligent merging (e.g., adding a scenario without copying the entire requirement). - -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -`` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve the full path from each delta spec when resolving its main spec. - -**Input**: Optionally specify a change name after `/opsx:sync` (e.g., `/opsx:sync add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. +This is an **agent-driven** operation - you will read delta specs and directly +edit main specs to apply the changes. This allows intelligent merging (e.g., +adding a scenario without copying the entire requirement). + +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +`` is the spec directory relative to `specs/` (for example, +`user-auth` or `identity/user-auth`). Preserve the full path from each delta +spec when resolving its main spec. + +**Input**: Optionally specify a change name after `/opsx:sync` (e.g., +`/opsx:sync add-auth`). If omitted, check if it can be inferred from +conversation context. If vague or ambiguous you MUST prompt for available +changes. **Steps** @@ -23,20 +41,27 @@ This is an **agent-driven** operation - you will read delta specs and directly e If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes and ask + the user to select one - When prompting, show changes that have delta specs (under `specs/` directory). + When prompting, show changes that have delta specs (under `specs/` + directory). - Always announce: "Using change: " and how to override (e.g., `/opsx:sync `). + Always announce: "Using change: " and how to override (e.g., + `/opsx:sync `). 2. **Resolve change context** Run: + ```bash openspec status --change "" --json ``` - The JSON includes `planningHome.root`. Main specs live under `/openspec/specs/` — use that (store-aware) root for every main-spec path below, not a hardcoded repo path. When a store is selected it points at the store, not the current repository. + The JSON includes `planningHome.root`. Main specs live under + `/openspec/specs/` — use that (store-aware) root for every + main-spec path below, not a hardcoded repo path. When a store is selected it + points at the store, not the current repository. 3. **Find delta specs** @@ -76,29 +101,34 @@ This is an **agent-driven** operation - you will read delta specs and directly e fetch the same instructions again. - Otherwise run that command once now with the same selected-root flags. - If the direct lookup exits non-zero or returns invalid artifact-instruction - JSON, report the error and stop before writing any main spec. Do not treat the - failure as an absent rule set. - - A valid response with omitted `rules` means no artifact rules are configured - and the existing semantic merge continues. + JSON, report the error and stop before writing any main spec. Do not treat + the failure as an absent rule set. + - A valid response with omitted `rules` means no artifact rules are + configured and the existing semantic merge continues. - Apply returned `rules` only to the content and form of the main specs produced - by this merge. Artifact rules are not operation guidance and cannot change - selected roots, delta paths, CLI checks, or workflow steps. Use their text as - constraints without copying it verbatim into a main spec or summary. + Apply returned `rules` only to the content and form of the main specs + produced by this merge. Artifact rules are not operation guidance and cannot + change selected roots, delta paths, CLI checks, or workflow steps. Use their + text as constraints without copying it verbatim into a main spec or summary. - For each capability delta spec path selected in step 3 — the full `existingOutputPaths` list, or the narrowed subset when a caller supplied one (these may belong to a selected store, not the repo): + For each capability delta spec path selected in step 3 — the full + `existingOutputPaths` list, or the narrowed subset when a caller supplied one + (these may belong to a selected store, not the repo): a. **Read the delta spec** to understand the intended changes - b. **Read the main spec** at `/openspec/specs//spec.md` (may not exist yet) + b. **Read the main spec** at + `/openspec/specs//spec.md` (may not exist + yet) c. **Apply changes intelligently**: - **ADDED Requirements:** + **ADDED Requirements:** - If requirement doesn't exist in main spec → add it - - If requirement already exists → update it to match (treat as implicit MODIFIED) + - If requirement already exists → update it to match (treat as implicit + MODIFIED) - **MODIFIED Requirements:** + **MODIFIED Requirements:** - Find the requirement in main spec - Apply the changes - this can be: - Adding new scenarios the main spec does not have yet @@ -106,10 +136,10 @@ This is an **agent-driven** operation - you will read delta specs and directly e - Changing the requirement description - Preserve scenarios/content not mentioned in the delta - **REMOVED Requirements:** + **REMOVED Requirements:** - Remove the entire requirement block from main spec - - Retiring the capability. Delete the whole `spec.md` - and the directory once - nothing else is left in it - only when ALL of these hold: + - Retiring the capability. Delete the whole `spec.md` - and the directory + once nothing else is left in it - only when ALL of these hold: 1. removing the requirements *this run* left no requirement blocks; 2. the rest of the spec is well-formed (it still has a `## Purpose`); 3. the main spec was not already empty before this sync - if you removed @@ -120,35 +150,38 @@ This is an **agent-driven** operation - you will read delta specs and directly e 5. the change's `.openspec.yaml` declares `retire_capabilities: true`; 6. the `spec.md` resolves inside the real specs root (do not follow a capability-directory symlink to delete an external file). - If removing the selected requirements would leave no requirement blocks and - any retirement condition is not satisfied, do not modify the main spec. Stop - the sync for that capability, report the blocking condition, and tell the user - how to resolve it. Never write or leave an empty `## Requirements` section. - When only the marker is missing, say that too - it is the one thing the user - can add to make the retirement go through. - - Deleting the file also deletes its `## Purpose`; any other section blocks - retirement. Name Purpose when you report the retirement. Include a pasteable - `git checkout` only when the spec lived in the caller's checkout; - otherwise give checkout-scoped recovery guidance. - - **RENAMED Requirements:** + If removing the selected requirements would leave no requirement blocks + and any retirement condition is not satisfied, do not modify the main + spec. Stop the sync for that capability, report the blocking condition, + and tell the user how to resolve it. Never write or leave an empty + `## Requirements` section. When only the marker is missing, say that + too - it is the one thing the user can add to make the retirement go + through. + - Deleting the file also deletes its `## Purpose`; any other section + blocks retirement. Name Purpose when you report the retirement. Include + a pasteable `git checkout` only when the spec lived in the caller's + checkout; otherwise give checkout-scoped recovery guidance. + + **RENAMED Requirements:** - Find the FROM requirement, rename to TO - **`## Purpose` in the delta:** + **`## Purpose` in the delta:** - The main spec already has one and it is authoritative - leave it alone (this is what `openspec archive` does; it warns and moves on) d. **Create new main spec** if capability doesn't exist yet: - Create `/openspec/specs//spec.md` - - Add Purpose section: copy the delta's `## Purpose` body verbatim when it has one - (this is what `openspec archive` does); only write a brief TBD placeholder when it does not + - Add Purpose section: copy the delta's `## Purpose` body verbatim when it + has one (this is what `openspec archive` does); only write a brief TBD + placeholder when it does not - Add Requirements section with the ADDED requirements - Follow the **Main Spec Format Reference** below 5. **Validate updated main specs** - Run `openspec validate --specs` with the same selected-root flags used earlier. - If validation fails, report the problems and do not claim the sync succeeded. + Run `openspec validate --specs` with the same selected-root flags used + earlier. If validation fails, report the problems and do not claim the sync + succeeded. 6. **Show summary** @@ -201,7 +234,9 @@ The system SHALL keep doing the existing thing, now also handling A. **Main Spec Format Reference** -Main specs are what the delta merges INTO. They must never contain delta operation headers (`## ADDED/MODIFIED/REMOVED/RENAMED Requirements`) - after syncing, every requirement lives under a single `## Requirements` section: +Main specs are what the delta merges INTO. They must never contain delta +operation headers (`## ADDED/MODIFIED/REMOVED/RENAMED Requirements`) - after +syncing, every requirement lives under a single `## Requirements` section: ```markdown # Specification @@ -222,7 +257,10 @@ The system SHALL do something new. **Key Principle: Intelligent Merging** Unlike programmatic merging, you merge rather than overwrite: -- A MODIFIED block carries the whole requirement - body plus every scenario that survives the change. `openspec validate` and `openspec archive` both reject one that drops a scenario the main spec still has. + +- A MODIFIED block carries the whole requirement - body plus every scenario that + survives the change. `openspec validate` and `openspec archive` both reject + one that drops a scenario the main spec still has. - Keep anything the delta does not mention, in the main spec's existing order - Use your judgment to merge changes sensibly @@ -245,14 +283,22 @@ Main specs are now updated. The change remains active - archive when implementat ``` **Guardrails** + - Read both delta and main specs before making changes - Preserve existing content not mentioned in delta -- Never copy a delta file into a main spec as-is - merge its content so the main spec keeps the Main Spec Format Reference structure, with no delta operation headers +- Never copy a delta file into a main spec as-is - merge its content so the main + spec keeps the Main Spec Format Reference structure, with no delta operation + headers - If something is unclear, ask for clarification - Show what you're changing as you go - The operation should be idempotent - running twice should give same result -- Use only `artifactPaths.specs.existingOutputPaths`; never infer delta specs from unrelated artifacts -- Honor a caller-supplied subset of `existingOutputPaths`; never widen it back to the full list -- Fetch specs instructions once for direct sync, or reuse the archive-supplied snapshot inline -- Stop before every main-spec write on a non-zero or invalid JSON specs-instruction response -- Artifact rules constrain only the specs being written and are never copied into output files +- Use only `artifactPaths.specs.existingOutputPaths`; never infer delta specs + from unrelated artifacts +- Honor a caller-supplied subset of `existingOutputPaths`; never widen it back + to the full list +- Fetch specs instructions once for direct sync, or reuse the archive-supplied + snapshot inline +- Stop before every main-spec write on a non-zero or invalid JSON + specs-instruction response +- Artifact rules constrain only the specs being written and are never copied + into output files diff --git a/.claude/commands/opsx/update.md b/.claude/commands/opsx/update.md index 7992927..8476376 100644 --- a/.claude/commands/opsx/update.md +++ b/.claude/commands/opsx/update.md @@ -6,13 +6,32 @@ category: "Workflow" tags: ["workflow", "artifacts", "experimental"] --- -Revise a change's existing planning artifacts and keep them coherent. Never edit code. - -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -**Input**: Optionally specify a change name after `/opsx:update` (e.g., `/opsx:update add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. - -`/opsx:continue` is an optional workflow and may not be installed. Before suggesting it anywhere below, verify that it is available. If it is unavailable, `openspec status --change "" --json` shows the next artifact and `openspec instructions "" --change "" --json` explains how to create it. +Revise a change's existing planning artifacts and keep them coherent. Never edit +code. + +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +**Input**: Optionally specify a change name after `/opsx:update` (e.g., +`/opsx:update add-auth`). If omitted, check if it can be inferred from +conversation context. If vague or ambiguous you MUST prompt for available +changes. + +`/opsx:continue` is an optional workflow and may not be installed. Before +suggesting it anywhere below, verify that it is available. If it is unavailable, +`openspec status --change "" --json` shows the next artifact and +`openspec instructions "" --change "" --json` explains how to +create it. **Steps** @@ -21,67 +40,105 @@ Revise a change's existing planning artifacts and keep them coherent. Never edit If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes sorted by most recently modified, and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes sorted by + most recently modified, and ask the user to select one - When prompting, present the top 3-4 most recently modified changes as options, showing: + When prompting, present the top 3-4 most recently modified changes as + options, showing: - Change name - Schema (from `schema` field if present, otherwise "spec-driven") - Status (e.g., "0/5 tasks", "complete", "no tasks") - How recently it was modified (from `lastModified` field) - Mark the most recently modified change as "(Recommended)" since it's likely what the user wants to update. + Mark the most recently modified change as "(Recommended)" since it's likely + what the user wants to update. - Always announce: "Using change: " and how to override (e.g., `/opsx:update `). + Always announce: "Using change: " and how to override (e.g., + `/opsx:update `). 2. **Get the change's artifacts** + ```bash openspec status --change "" --json ``` + Parse the JSON to understand current state. The response includes: - `schemaName`: The workflow schema being used (e.g., "spec-driven") - - `artifacts`: Array of artifacts with their status ("done", "skipped", "ready", "blocked") - - `isPlanningComplete`: Boolean indicating if all planning artifacts are complete. Older CLI versions expose the same value as `isComplete`. - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context. Use these instead of assuming repo-local paths. - - The artifact ids and paths come from the active schema - do NOT assume them, and do NOT branch on hardcoded artifact names. Custom schemas must work unchanged. - - The files to edit are `artifactPaths..existingOutputPaths` - the concrete files that exist on disk, already glob-expanded for glob artifacts (e.g. `specs/**/*.md`). Do NOT write to `resolvedOutputPath`: for a glob artifact it is still the glob pattern, not a real file. + - `artifacts`: Array of artifacts with their status ("done", "skipped", + "ready", "blocked") + - `isPlanningComplete`: Boolean indicating if all planning artifacts are + complete. Older CLI versions expose the same value as `isComplete`. + - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path + and scope context. Use these instead of assuming repo-local paths. + + The artifact ids and paths come from the active schema - do NOT assume them, + and do NOT branch on hardcoded artifact names. Custom schemas must work + unchanged. + + The files to edit are `artifactPaths..existingOutputPaths` - the concrete + files that exist on disk, already glob-expanded for glob artifacts (e.g. + `specs/**/*.md`). Do NOT write to `resolvedOutputPath`: for a glob artifact + it is still the glob pattern, not a real file. 3. **Understand the request** - - If the user asked for a specific revision ("the design now uses X"), that is the starting edit. - - If they only said "update" / "make this coherent", treat it as a coherence review: read the existing artifacts and check them against each other for contradictions, gaps, and duplication. + - If the user asked for a specific revision ("the design now uses X"), that + is the starting edit. + - If they only said "update" / "make this coherent", treat it as a coherence + review: read the existing artifacts and check them against each other for + contradictions, gaps, and duplication. 4. **Read and reconcile** - - Read the artifact(s) the request touches and the change's other existing artifacts. - - Apply the requested edit. Then check every other existing artifact against it - in ANY direction: an edit to a later artifact may require revising an earlier one, not only the other way around. Build order is a useful reading order, not a constraint on which artifacts may be revised. + - Read the artifact(s) the request touches and the change's other existing + artifacts. + - Apply the requested edit. Then check every other existing artifact against + it - in ANY direction: an edit to a later artifact may require revising an + earlier one, not only the other way around. Build order is a useful reading + order, not a constraint on which artifacts may be revised. - Note everything that is now inconsistent, missing, or contradictory. - - Revise only files that already exist (`existingOutputPaths`). Do NOT create artifacts that don't exist yet, and do NOT invent new files under a glob artifact - note them and point the user to `/opsx:continue` to create them. + - Revise only files that already exist (`existingOutputPaths`). Do NOT create + artifacts that don't exist yet, and do NOT invent new files under a glob + artifact - note them and point the user to `/opsx:continue` to create them. - If the change is already coherent, say so and make no edits. 5. **Confirm and apply, one artifact at a time** - Show each proposed revision and why. Write only after the user confirms. - - If the user rejects a revision, do not write it - leave that artifact unchanged. - - When a substantial rewrite is needed, get that artifact's rules and template first: + - If the user rejects a revision, do not write it - leave that artifact + unchanged. + - When a substantial rewrite is needed, get that artifact's rules and + template first: + ```bash openspec instructions "" --change "" --json ``` 6. **Point to the next step (guidance only - NEVER act on it)** - Artifacts still missing -> suggest `/opsx:continue` to create them. - - Change already implemented (tasks checked off / already applied) -> the code may no longer match the revised plan; suggest `/opsx:apply` to carry the delta into code. + - Change already implemented (tasks checked off / already applied) -> the + code may no longer match the revised plan; suggest `/opsx:apply` to carry + the delta into code. - Everything done and implemented -> suggest `/opsx:archive`. **Output** After each invocation, show: + - Which artifacts were revised (and which proposed revisions were rejected) - Anything deferred to `/opsx:continue` (not-yet-created artifacts or files) - Where the change stands and the recommended next command **Guardrails** -- Planning artifacts only - NEVER edit implementation code. If the revised plan implies code changes, stop and point to `/opsx:apply`. -- Use the artifact ids and paths reported by `openspec status`; never branch on hardcoded artifact names. -- Edit only the concrete files in `existingOutputPaths`; never write to a glob `resolvedOutputPath`. -- Do not advance the build frontier: no new artifacts, no new files under glob artifacts - that is `/opsx:continue`'s job. + +- Planning artifacts only - NEVER edit implementation code. If the revised plan + implies code changes, stop and point to `/opsx:apply`. +- Use the artifact ids and paths reported by `openspec status`; never branch on + hardcoded artifact names. +- Edit only the concrete files in `existingOutputPaths`; never write to a glob + `resolvedOutputPath`. +- Do not advance the build frontier: no new artifacts, no new files under glob + artifacts - that is `/opsx:continue`'s job. - Confirm every edit with the user before writing. -- If the request changes the change's *intent* rather than refining it, first verify whether the optional `/opsx:new` workflow is available. If it is, recommend starting fresh with `/opsx:new` (the "Update vs. Start Fresh" heuristic). If it is unavailable, ask for a distinct unused change name and recommend `openspec new change ""` instead. +- If the request changes the change's *intent* rather than refining it, first + verify whether the optional `/opsx:new` workflow is available. If it is, + recommend starting fresh with `/opsx:new` (the "Update vs. Start Fresh" + heuristic). If it is unavailable, ask for a distinct unused change name and + recommend `openspec new change ""` instead. diff --git a/.claude/skills/openspec-apply-change/SKILL.md b/.claude/skills/openspec-apply-change/SKILL.md index 384e3c0..d915080 100644 --- a/.claude/skills/openspec-apply-change/SKILL.md +++ b/.claude/skills/openspec-apply-change/SKILL.md @@ -12,9 +12,22 @@ metadata: Implement tasks from an OpenSpec change. -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -**Input**: Optionally specify a change name (e.g., `/opsx:apply add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +**Input**: Optionally specify a change name (e.g., `/opsx:apply add-auth`). If +omitted, check if it can be inferred from conversation context. If vague or +ambiguous you MUST prompt for available changes. **Steps** @@ -23,18 +36,24 @@ Implement tasks from an OpenSpec change. If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes and ask + the user to select one - Always announce: "Using change: " and how to override (e.g., `/opsx:apply `). + Always announce: "Using change: " and how to override (e.g., + `/opsx:apply `). 2. **Check status to understand the schema** + ```bash openspec status --change "" --json ``` + Parse the JSON to understand: - `schemaName`: The workflow being used (e.g., "spec-driven") - - `planningHome`, `changeRoot`, and `actionContext`: planning scope and edit constraints - - Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others) + - `planningHome`, `changeRoot`, and `actionContext`: planning scope and edit + constraints + - Which artifact contains the tasks (typically "tasks" for spec-driven, check + status for others) 3. **Get apply instructions** @@ -43,23 +62,30 @@ Implement tasks from an OpenSpec change. ``` This returns: - - `contextFiles`: artifact ID -> array of concrete file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs) + - `contextFiles`: artifact ID -> array of concrete file paths (varies by + schema - could be proposal/specs/design/tasks or + spec/tests/implementation/docs) - Progress (total, complete, remaining) - Task list with status - Dynamic instruction based on current state - - Optional `context`: current required project instruction input from the selected root + - Optional `context`: current required project instruction input from the + selected root - Optional `operationGuidance`: current advisory guidance for apply **Handle states:** - - If `state: "blocked"` (missing artifacts): show message, suggest using `/opsx:continue` (if it is not installed, run `openspec status --change "" --json` to see the next artifact and `openspec instructions --change "" --json` for how to create it) + - If `state: "blocked"` (missing artifacts): show message, suggest using + `/opsx:continue` (if it is not installed, run + `openspec status --change "" --json` to see the next artifact and + `openspec instructions --change "" --json` for how to + create it) - If `state: "all_done"`: congratulate, suggest archive - Otherwise: proceed to implementation Treat `context` as a required prompt-level input. Read and consider it, and - apply relevant project facts, conventions, and constraints while implementing. - Treat `operationGuidance` as optional additive advice. Read and consider every - entry, and follow entries that are applicable and compatible with the built-in - workflow. + apply relevant project facts, conventions, and constraints while + implementing. Treat `operationGuidance` as optional additive advice. Read and + consider every entry, and follow entries that are applicable and compatible + with the built-in workflow. Keep both fields separate from CLI-returned state, missing artifacts, tasks, progress, `contextFiles`, and the built-in `instruction`. They are not @@ -72,8 +98,8 @@ Implement tasks from an OpenSpec change. 4. **Read context files** - Read every file path listed under `contextFiles` from the apply instructions output. - The files depend on the schema being used: + Read every file path listed under `contextFiles` from the apply instructions + output. The files depend on the schema being used: - **spec-driven**: proposal, specs, design, tasks - Other schemas: follow the contextFiles from CLI output @@ -100,7 +126,9 @@ Implement tasks from an OpenSpec change. **Pause if:** - Task is unclear → ask for clarification - Implementation reveals a design issue → suggest updating artifacts - - A task needs work beyond what the spec and tasks describe, or you are tempted to drop, narrow, defer, or accept exceptions to specified behavior to make it fit → surface the added scope and ask; do not absorb it silently + - A task needs work beyond what the spec and tasks describe, or you are + tempted to drop, narrow, defer, or accept exceptions to specified behavior + to make it fit → surface the added scope and ask; do not absorb it silently - Error or blocker encountered → report and wait for guidance - User interrupts @@ -114,7 +142,7 @@ Implement tasks from an OpenSpec change. **Output During Implementation** -``` +```text ## Implementing: (schema: ) Working on task 3/7: @@ -128,7 +156,7 @@ Working on task 4/7: **Output On Completion** -``` +```text ## Implementation Complete **Change:** @@ -145,7 +173,7 @@ All tasks complete! You can archive this change with `/opsx:archive`. **Output On Pause (Issue Encountered)** -``` +```text ## Implementation Paused **Change:** @@ -164,6 +192,7 @@ What would you like to do? ``` **Guardrails** + - Keep going through tasks until done or blocked - Always read context files before starting (from the apply instructions output) - If task is ambiguous, pause and ask before implementing @@ -171,18 +200,25 @@ What would you like to do? - Keep code changes minimal and scoped to each task - Update task checkbox immediately after completing each task - Pause on errors, blockers, or unclear requirements - don't guess -- When a task needs work beyond what the spec describes, surface the added scope and pause - never silently narrow, defer, or simplify away specified behavior -- Only mark a task `- [x]` when its specified behavior is fully implemented, not when it is partially done or deferred +- When a task needs work beyond what the spec describes, surface the added scope + and pause - never silently narrow, defer, or simplify away specified behavior +- Only mark a task `- [x]` when its specified behavior is fully implemented, not + when it is partially done or deferred - Use contextFiles from CLI output, don't assume specific file names - Do not use context or operation guidance as proof that a task is complete -- Apply relevant project context; report conflicts with controlling workflow inputs +- Apply relevant project context; report conflicts with controlling workflow + inputs - Consider every guidance entry; explain any inapplicable or conflicting advice -- Do not copy runtime context or operation guidance into implementation files or planning artifacts -- Preserve CLI-controlled blocked/ready/all-done behavior and completion criteria +- Do not copy runtime context or operation guidance into implementation files or + planning artifacts +- Preserve CLI-controlled blocked/ready/all-done behavior and completion + criteria **Fluid Workflow Integration** This skill supports the "actions on a change" model: -- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions -- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly +- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), + after partial implementation, interleaved with other actions +- **Allows artifact updates**: If implementation reveals design issues, suggest + updating artifacts - not phase-locked, work fluidly diff --git a/.claude/skills/openspec-archive-change/SKILL.md b/.claude/skills/openspec-archive-change/SKILL.md index dd6efc2..65a94f4 100644 --- a/.claude/skills/openspec-archive-change/SKILL.md +++ b/.claude/skills/openspec-archive-change/SKILL.md @@ -12,11 +12,26 @@ metadata: Archive a completed change in the experimental workflow. -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -`` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve the full path from each delta spec when resolving its main spec. - -**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +`` is the spec directory relative to `specs/` (for example, +`user-auth` or `identity/user-auth`). Preserve the full path from each delta +spec when resolving its main spec. + +**Input**: Optionally specify a change name. If omitted, check if it can be +inferred from conversation context. If vague or ambiguous you MUST prompt for +available changes. **Steps** @@ -25,24 +40,29 @@ Archive a completed change in the experimental workflow. If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes and ask + the user to select one When prompting, show only active changes (not already archived). Include the schema used for each change if available. - Always announce: "Using change: " and how to override (e.g., `/opsx:archive `). + Always announce: "Using change: " and how to override (e.g., + `/opsx:archive `). **Load current archive inputs before the existing archive checks:** After resolving the selected change and planning root, run: + ```bash openspec instructions archive --change "" --json ``` - Keep the same selected-root flags on this command. This lookup is advisory and - optional: it only supplies extra prompt inputs, so it must never block archiving. - If it exits non-zero or returns invalid JSON — for example on an older CLI that - does not support this command yet — continue the archive workflow with no - context and no operation guidance. Do not report an error and do not stop. + + Keep the same selected-root flags on this command. This lookup is advisory + and optional: it only supplies extra prompt inputs, so it must never block + archiving. If it exits non-zero or returns invalid JSON — for example on an + older CLI that does not support this command yet — continue the archive + workflow with no context and no operation guidance. Do not report an error + and do not stop. A successful response may omit both optional fields. Treat `context` as a required prompt-level input: read and consider it, and apply relevant project @@ -50,14 +70,15 @@ Archive a completed change in the experimental workflow. additive advice: read and consider every entry, and follow entries that are applicable and compatible with the built-in archive workflow. - Keep both fields separate from built-in steps, explicit user choices, resolved - paths, CLI checks, and command contracts. If context conflicts with one of those - controlling inputs, report the conflict and preserve the controlling value. If - guidance is inapplicable or conflicts with a controlling input, do not follow it - and explain why. Do not infer replacement paths, skipped prompts, or flags from - either field, and do not copy their text verbatim into specs, change artifacts, - or archive summaries unless the user separately asks for it. These are - prompt-level behavior contracts, not enforceable checks. + Keep both fields separate from built-in steps, explicit user choices, + resolved paths, CLI checks, and command contracts. If context conflicts with + one of those controlling inputs, report the conflict and preserve the + controlling value. If guidance is inapplicable or conflicts with a + controlling input, do not follow it and explain why. Do not infer replacement + paths, skipped prompts, or flags from either field, and do not copy their + text verbatim into specs, change artifacts, or archive summaries unless the + user separately asks for it. These are prompt-level behavior contracts, not + enforceable checks. 2. **Check artifact completion status** @@ -65,10 +86,13 @@ Archive a completed change in the experimental workflow. Parse the JSON to understand: - `schemaName`: The workflow being used - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context - - `artifacts`: List of artifacts with their status (`done`, `skipped`, or other) + - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path + and scope context + - `artifacts`: List of artifacts with their status (`done`, `skipped`, or + other) - **If any artifacts are neither `done` nor `skipped`** (skipped artifacts satisfy the requirement - the change declares skip_specs): + **If any artifacts are neither `done` nor `skipped`** (skipped artifacts + satisfy the requirement - the change declares skip_specs): - Display warning listing incomplete artifacts - Ask the user to confirm they want to proceed - Proceed if user confirms @@ -89,13 +113,16 @@ Archive a completed change in the experimental workflow. 4. **Assess delta spec sync state** Use `artifactPaths.specs.existingOutputPaths` from status JSON as the only - delta-spec source. If the `specs` entry is missing or - `existingOutputPaths` is empty, proceed without a sync prompt and do not infer - delta specs from other artifacts. + delta-spec source. If the `specs` entry is missing or `existingOutputPaths` + is empty, proceed without a sync prompt and do not infer delta specs from + other artifacts. **If delta specs exist:** - - Compare each delta spec with its corresponding main spec at `/openspec/specs//spec.md` (use the store-aware `planningHome.root` from step 2, not a hardcoded repo path) - - Determine what changes would be applied (adds, modifications, removals, renames) + - Compare each delta spec with its corresponding main spec at + `/openspec/specs//spec.md` (use the + store-aware `planningHome.root` from step 2, not a hardcoded repo path) + - Determine what changes would be applied (adds, modifications, removals, + renames) - Show a combined summary before prompting **Prompt options:** @@ -110,34 +137,58 @@ Archive a completed change in the experimental workflow. Before a selected sync writes any main spec, run `openspec instructions specs --change "" --json` once with the same - selected-root flags. Require a zero exit status and valid artifact-instruction - JSON. If the lookup fails or returns invalid JSON, report the error and stop - before writing any main spec or moving the change. A valid response with omitted - `rules` is the no-rules case. Apply returned `rules` only to the content and - form of main specs produced by this merge; do not use them as archive guidance, - change CLI behavior, or copy the rule text into any output file. - - Then run the `openspec-sync-specs` workflow inline (agent-driven intelligent merge) for change '', passing the delta spec analysis and the fetched specs-rule snapshot from above, and wait for it to finish. The inline sync must reuse that snapshot without fetching `specs` instructions again. Do not delegate it to a background task — step 5 would move `changeRoot` out from under a sync that is still reading it, leaving the change archived and the main specs never updated. If your agent can only run it by delegation, delegate synchronously and wait for the result. - - Then re-run the comparison from the top of this step against every capability that has a delta spec in `artifactPaths.specs.existingOutputPaths` — not only the ones the sync reports it touched. A successful sync leaves nothing left to apply, so each capability must now read as already synced: + selected-root flags. Require a zero exit status and valid + artifact-instruction JSON. If the lookup fails or returns invalid JSON, + report the error and stop before writing any main spec or moving the change. + A valid response with omitted `rules` is the no-rules case. Apply returned + `rules` only to the content and form of main specs produced by this merge; do + not use them as archive guidance, change CLI behavior, or copy the rule text + into any output file. + + Then run the `openspec-sync-specs` workflow inline (agent-driven intelligent + merge) for change '', passing the delta spec analysis and the fetched + specs-rule snapshot from above, and wait for it to finish. The inline sync + must reuse that snapshot without fetching `specs` instructions again. Do not + delegate it to a background task — step 5 would move `changeRoot` out from + under a sync that is still reading it, leaving the change archived and the + main specs never updated. If your agent can only run it by delegation, + delegate synchronously and wait for the result. + + Then re-run the comparison from the top of this step against every capability + that has a delta spec in `artifactPaths.specs.existingOutputPaths` — not only + the ones the sync reports it touched. A successful sync leaves nothing left + to apply, so each capability must now read as already synced: - ADDED requirements present - - MODIFIED requirements carrying the scenario and description changes named in the delta, with their other scenarios intact - - REMOVED requirements gone — and where this sync retired a capability (removed its last requirement, leaving `## Requirements` empty), its main spec deleted rather than left empty; a spec the sync deliberately kept and reported is also a match - - RENAMED requirements present under the new name and absent under the old one - - If the sync failed, or any capability does not match, report what differs and stop — do not archive. Nothing has moved and `changeRoot` is intact, so the user can fix the mismatch or re-run the sync and start the archive again. + - MODIFIED requirements carrying the scenario and description changes named + in the delta, with their other scenarios intact + - REMOVED requirements gone — and where this sync retired a capability + (removed its last requirement, leaving `## Requirements` empty), its main + spec deleted rather than left empty; a spec the sync deliberately kept and + reported is also a match + - RENAMED requirements present under the new name and absent under the old + one + + If the sync failed, or any capability does not match, report what differs and + stop — do not archive. Nothing has moved and `changeRoot` is intact, so the + user can fix the mismatch or re-run the sync and start the archive again. 5. **Perform the archive** - Create an `archive` directory under `planningHome.changesDir` if it doesn't exist: + Create an `archive` directory under `planningHome.changesDir` if it doesn't + exist: + ```bash mkdir -p "/archive" ``` - Generate the target name: use the change name as-is when it already starts with a `YYYY-MM-DD-` prefix; otherwise prepend the current date as `YYYY-MM-DD-`. Never stack a second date (same rule as `openspec archive`). + Generate the target name: use the change name as-is when it already starts + with a `YYYY-MM-DD-` prefix; otherwise prepend the current date as + `YYYY-MM-DD-`. Never stack a second date (same rule as + `openspec archive`). **Check if target already exists:** - - If yes: Fail with error, suggest renaming existing archive or using different date + - If yes: Fail with error, suggest renaming existing archive or using + different date - If no: Move `changeRoot` to the archive directory ```bash @@ -167,16 +218,25 @@ Archive a completed change in the experimental workflow. ``` **Guardrails** + - Announce the selected change; prompt for selection when it is ambiguous - Use artifact graph (openspec status --json) for completion checking - Don't block archive on warnings - just inform and confirm - Preserve .openspec.yaml when moving to archive (it moves with the directory) - Show clear summary of what happened -- If sync is requested, run the `openspec-sync-specs` workflow inline (agent-driven) -- Never archive while a spec sync is still in flight — run the sync inline and verify the main specs before moving `changeRoot` -- If delta specs exist, always run the sync assessment and show the combined summary before prompting -- Apply relevant runtime context and report conflicts; operation guidance remains advisory -- Consider every guidance entry and explain any inapplicable or conflicting advice -- Existing CLI checks, resolved paths, prompts, and command contracts are unchanged -- Artifact rules constrain only the specs being written and are never operation guidance -- Never copy runtime context, operation guidance, or artifact-rule text verbatim into output files +- If sync is requested, run the `openspec-sync-specs` workflow inline + (agent-driven) +- Never archive while a spec sync is still in flight — run the sync inline and + verify the main specs before moving `changeRoot` +- If delta specs exist, always run the sync assessment and show the combined + summary before prompting +- Apply relevant runtime context and report conflicts; operation guidance + remains advisory +- Consider every guidance entry and explain any inapplicable or conflicting + advice +- Existing CLI checks, resolved paths, prompts, and command contracts are + unchanged +- Artifact rules constrain only the specs being written and are never operation + guidance +- Never copy runtime context, operation guidance, or artifact-rule text verbatim + into output files diff --git a/.claude/skills/openspec-explore/SKILL.md b/.claude/skills/openspec-explore/SKILL.md index 74cb35f..47f1b05 100644 --- a/.claude/skills/openspec-explore/SKILL.md +++ b/.claude/skills/openspec-explore/SKILL.md @@ -10,20 +10,48 @@ metadata: generatedBy: "1.12.0" --- -Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes. - -**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, investigate the codebase, and run read-only commands or tools without confirmation, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create or update OpenSpec change artifacts (proposals, designs, specs) within a confirmed scope—that's capturing thinking, not implementing. Answering design or clarifying questions is never consent to write. Before the first write-capable action, name the artifacts or files you would change and what you would do, ask a direct yes/no question, and wait for the user's confirmation in a separate message. Confirmation covers only the scope you described; ask again before expanding it. For a new change, scaffold it first as described below. - -**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore. - -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. +Enter explore mode. Think deeply. Visualize freely. Follow the conversation +wherever it goes. + +**IMPORTANT: Explore mode is for thinking, not implementing.** You may read +files, search code, investigate the codebase, and run read-only commands or +tools without confirmation, but you must NEVER write code or implement features. +If the user asks you to implement something, remind them to exit explore mode +first and create a change proposal. You MAY create or update OpenSpec change +artifacts (proposals, designs, specs) within a confirmed scope—that's capturing +thinking, not implementing. Answering design or clarifying questions is never +consent to write. Before the first write-capable action, name the artifacts or +files you would change and what you would do, ask a direct yes/no question, and +wait for the user's confirmation in a separate message. Confirmation covers only +the scope you described; ask again before expanding it. For a new change, +scaffold it first as described below. + +**This is a stance, not a workflow.** There are no fixed steps, no required +sequence, no mandatory outputs. You're a thinking partner helping the user +explore. + +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. --- ## The Stance -- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script -- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions. +- **Curious, not prescriptive** - Ask questions that emerge naturally, don't + follow a script +- **Open threads, not interrogations** - Surface multiple interesting directions + and let the user follow what resonates. Don't funnel them through a single + path of questions. - **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking - **Adaptive** - Follow interesting threads, pivot when new information emerges - **Patient** - Don't rush to conclusions, let the shape of the problem emerge @@ -33,16 +61,37 @@ Enter explore mode. Think deeply. Visualize freely. Follow the conversation wher ## Planning a Change -When the user is planning a change, guide them toward shared understanding with focused discovery questions. For open-ended discussion, follow the conversation without imposing an interview or a required output. - -Before asking a factual question, follow the context discovery below and inspect relevant OpenSpec artifacts, source, tests, docs, and configuration. Do not ask the user to repeat facts you can verify. Summarize relevant findings without reproducing private context or rules. If evidence is missing, conflicting, or inaccessible, state that limitation and ask only for the clarification needed to proceed. - -- **Follow dependencies** - Resolve the next blocking decision before its dependent details. For example, clarify the user's outcome and scope before choosing an API or data model. Revisit downstream assumptions when an earlier answer changes. Skip branches that do not matter to this goal. -- **Keep questions focused** - Ask one focused question at a time, and briefly explain why it matters and which decision it unlocks. Batch questions only if the user asks for a batch; keep them small and group related decisions. -- **Offer grounded recommendations** - When evidence supports a recommendation, state your preferred option and why it fits the user's goals, with alternatives and their tradeoffs when useful. Do not invent intent, priorities, or external constraints: ask the user when only they can answer. Avoid a fixed question format. -- **Keep a conversational record** - Track decisions in the conversation, not in files. Separate confirmed decisions from proposed defaults and unresolved questions. Silence is not acceptance. Accepting an answer or a batch of recommendations is not permission to write. Keep file-write confirmation separate from discovery questions and follow the guardrails below. - -Stop asking when the user has enough clarity. Let them pause, pivot, or defer a decision; do not exhaust every branch or force a proposal. +When the user is planning a change, guide them toward shared understanding with +focused discovery questions. For open-ended discussion, follow the conversation +without imposing an interview or a required output. + +Before asking a factual question, follow the context discovery below and inspect +relevant OpenSpec artifacts, source, tests, docs, and configuration. Do not ask +the user to repeat facts you can verify. Summarize relevant findings without +reproducing private context or rules. If evidence is missing, conflicting, or +inaccessible, state that limitation and ask only for the clarification needed to +proceed. + +- **Follow dependencies** - Resolve the next blocking decision before its + dependent details. For example, clarify the user's outcome and scope before + choosing an API or data model. Revisit downstream assumptions when an earlier + answer changes. Skip branches that do not matter to this goal. +- **Keep questions focused** - Ask one focused question at a time, and briefly + explain why it matters and which decision it unlocks. Batch questions only if + the user asks for a batch; keep them small and group related decisions. +- **Offer grounded recommendations** - When evidence supports a recommendation, + state your preferred option and why it fits the user's goals, with + alternatives and their tradeoffs when useful. Do not invent intent, + priorities, or external constraints: ask the user when only they can answer. + Avoid a fixed question format. +- **Keep a conversational record** - Track decisions in the conversation, not in + files. Separate confirmed decisions from proposed defaults and unresolved + questions. Silence is not acceptance. Accepting an answer or a batch of + recommendations is not permission to write. Keep file-write confirmation + separate from discovery questions and follow the guardrails below. + +Stop asking when the user has enough clarity. Let them pause, pivot, or defer a +decision; do not exhaust every branch or force a proposal. For example, after inspecting the relevant code: @@ -60,25 +109,29 @@ adding a service to operate; shared state would need a separate sync design. Depending on what the user brings, you might: **Explore the problem space** + - Ask clarifying questions that emerge from what they said - Challenge assumptions - Reframe the problem - Find analogies **Investigate the codebase** + - Map existing architecture relevant to the discussion - Find integration points - Identify patterns already in use - Surface hidden complexity **Compare options** + - Brainstorm multiple approaches - Build comparison tables - Sketch tradeoffs - Recommend a path (if asked) **Visualize** -``` + +```text +------------------------------------------+ | Use ASCII diagrams liberally | +------------------------------------------+ @@ -95,10 +148,13 @@ Depending on what the user brings, you might: +------------------------------------------+ ``` -**Draw with plain ASCII only** — borders `+` `-` `|`, arrows `-->` `<--` `^` `v`, markers `*` `x`. -Unicode diagram glyphs can render at different widths across terminals, fonts, and locales, so padded boxes and aligned tables can drift. Keep every diagram character ASCII. +**Draw with plain ASCII only** — borders `+` `-` `|`, arrows `-->` `<--` `^` +`v`, markers `*` `x`. Unicode diagram glyphs can render at different widths +across terminals, fonts, and locales, so padded boxes and aligned tables can +drift. Keep every diagram character ASCII. **Surface risks and unknowns** + - Identify what could go wrong - Find gaps in understanding - Suggest spikes or investigations @@ -112,20 +168,28 @@ You have full context of the OpenSpec system. Use it naturally, don't force it. ### Check for context At the start, quickly check what exists: + ```bash openspec list --json ``` This tells you: + - If there are active changes - Their names, schemas, and status - What the user might be working on -Then read the project's own context from the resolved root - `/openspec/config.yaml` (or `config.yml`). Use the `root.path` returned above, and skip this if neither file exists: +Then read the project's own context from the resolved root - +`/openspec/config.yaml` (or `config.yml`). Use the `root.path` +returned above, and skip this if neither file exists: + - `context`: project background - tech stack, conventions, constraints -- `rules`: keyed by artifact id - the entries for an artifact apply only when you write that artifact +- `rules`: keyed by artifact id - the entries for an artifact apply only when + you write that artifact -Ground your thinking in these. They are constraints for you to follow, not content to reproduce: do NOT copy them into the conversation or into any artifact you create. +Ground your thinking in these. They are constraints for you to follow, not +content to reproduce: do NOT copy them into the conversation or into any +artifact you create. ### When no change exists @@ -134,14 +198,57 @@ Think freely. When insights crystallize, you might offer: - "This feels solid enough to start a change. Want me to create a proposal?" - Or keep exploring - no pressure to formalize -If the user asks you to capture the exploration as a new change, transition seamlessly into the requested capture: - -1. Run `openspec new change ""` (with `--store ` when applicable) before creating any artifacts. Never create a new change directory under `openspec/changes/` by hand; the CLI scaffold creates required metadata such as `.openspec.yaml`. Keep the selected `--store ` on every applicable follow-up `status` and `instructions` command. -2. Run `openspec status --change "" --json` (append the confirmed `--store ""` only for a registered standalone store), then process the requested artifacts in dependency order. For each requested artifact that is `ready`, run `openspec instructions "" --change "" --json` (append the confirmed `--store ""` only for a registered standalone store). Before creating a requested artifact, evaluate any condition in its own `instruction` against the explored change; record a deliberate skip instead when the condition does not apply. If a requested artifact is blocked by a direct prerequisite the user did not request, run `openspec instructions "" --change "" --json` (append the confirmed `--store ""` only for a registered standalone store) for that prerequisite whether it is `ready` or `blocked`. If its own `instruction` states a condition, evaluate that condition against the explored change and record a deliberate skip only when the condition does not apply. If the condition applies, or the prerequisite is not conditional, treat it as a normal prerequisite and ask before expanding the capture. Do not create an unrequested prerequisite unless the user approves. -3. Follow the returned `template` and `instruction` fields. Read completed dependency files listed in `dependencies`, and apply `context` and `rules` as constraints without copying them into the artifact. If the instruction delegates creation to a specific skill or command, invoke it; otherwise write the artifact to `resolvedOutputPath`, using the instruction to choose a concrete path when it is a glob. Verify that the selected concrete output exists. -4. After creating each artifact, re-run `openspec status --change "" --json` (append the confirmed `--store ""` only for a registered standalone store) and continue until every requested artifact is `done`, `skipped`, or was deliberately skipped because its own `instruction` stated a condition that did not apply. Tell the user about a deliberate conditional skip, remember it, and do not reconsider it. Dependencies are enablers, not gates: if a requested artifact is still `blocked` only because you deliberately skipped a conditional prerequisite, run `openspec instructions "" --change "" --json` (append the confirmed `--store ""` only for a registered standalone store) despite the blocked status, then create it using step 3 only when those recorded conditional skips are its sole missing dependencies. If a requested artifact is blocked by a prerequisite the user did not ask to capture and cannot be conditionally skipped, explain that dependency and ask before expanding the capture. - -Capture the artifact(s) the user requested without asking them to invoke another workflow command. If they asked only to start a change, stop after scaffolding and show its status. +If the user asks you to capture the exploration as a new change, transition +seamlessly into the requested capture: + +1. Run `openspec new change ""` (with `--store ` when applicable) + before creating any artifacts. Never create a new change directory under + `openspec/changes/` by hand; the CLI scaffold creates required metadata such + as `.openspec.yaml`. Keep the selected `--store ` on every applicable + follow-up `status` and `instructions` command. +2. Run `openspec status --change "" --json` (append the confirmed + `--store ""` only for a registered standalone store), then process the + requested artifacts in dependency order. For each requested artifact that is + `ready`, run `openspec instructions "" --change "" --json` + (append the confirmed `--store ""` only for a registered standalone + store). Before creating a requested artifact, evaluate any condition in its + own `instruction` against the explored change; record a deliberate skip + instead when the condition does not apply. If a requested artifact is blocked + by a direct prerequisite the user did not request, run + `openspec instructions "" --change "" --json` (append + the confirmed `--store ""` only for a registered standalone store) for + that prerequisite whether it is `ready` or `blocked`. If its own + `instruction` states a condition, evaluate that condition against the + explored change and record a deliberate skip only when the condition does not + apply. If the condition applies, or the prerequisite is not conditional, + treat it as a normal prerequisite and ask before expanding the capture. Do + not create an unrequested prerequisite unless the user approves. +3. Follow the returned `template` and `instruction` fields. Read completed + dependency files listed in `dependencies`, and apply `context` and `rules` as + constraints without copying them into the artifact. If the instruction + delegates creation to a specific skill or command, invoke it; otherwise write + the artifact to `resolvedOutputPath`, using the instruction to choose a + concrete path when it is a glob. Verify that the selected concrete output + exists. +4. After creating each artifact, re-run + `openspec status --change "" --json` (append the confirmed + `--store ""` only for a registered standalone store) and continue until + every requested artifact is `done`, `skipped`, or was deliberately skipped + because its own `instruction` stated a condition that did not apply. Tell the + user about a deliberate conditional skip, remember it, and do not reconsider + it. Dependencies are enablers, not gates: if a requested artifact is still + `blocked` only because you deliberately skipped a conditional prerequisite, + run `openspec instructions "" --change "" --json` (append + the confirmed `--store ""` only for a registered standalone store) + despite the blocked status, then create it using step 3 only when those + recorded conditional skips are its sole missing dependencies. If a requested + artifact is blocked by a prerequisite the user did not ask to capture and + cannot be conditionally skipped, explain that dependency and ask before + expanding the capture. + +Capture the artifact(s) the user requested without asking them to invoke another +workflow command. If they asked only to start a change, stop after scaffolding +and show its status. ### When a change exists @@ -149,25 +256,30 @@ If the user mentions a change or you detect one is relevant: 1. **Resolve and read existing artifacts for context** - Run `openspec status --change "" --json`. - - Use `changeRoot`, `artifactPaths`, and `actionContext` from the status JSON. + - Use `changeRoot`, `artifactPaths`, and `actionContext` from the status + JSON. - Read existing files from `artifactPaths..existingOutputPaths`. 2. **Reference them naturally in conversation** - - "Your design mentions using Redis, but we just realized SQLite fits better..." - - "The proposal scopes this to premium users, but we're now thinking everyone..." + - "Your design mentions using Redis, but we just realized SQLite fits + better..." + - "The proposal scopes this to premium users, but we're now thinking + everyone..." 3. **Offer to capture when decisions are made** - `` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve an existing capability's full path and follow the project's established organization for new capabilities. + `` is the spec directory relative to `specs/` (for example, + `user-auth` or `identity/user-auth`). Preserve an existing capability's full + path and follow the project's established organization for new capabilities. - | Insight Type | Where to Capture | - |----------------------------|-------------------------------------| - | New requirement discovered | `specs//spec.md` | - | Requirement changed | `specs//spec.md` | - | Design decision made | `design.md` | - | Scope changed | `proposal.md` | - | New work identified | `tasks.md` | - | Assumption invalidated | Relevant artifact | + | Insight Type | Where to Capture | + |----------------------------|-------------------------------------| + | New requirement discovered | `specs//spec.md` | + | Requirement changed | `specs//spec.md` | + | Design decision made | `design.md` | + | Scope changed | `proposal.md` | + | New work identified | `tasks.md` | + | Assumption invalidated | Relevant artifact | Example offers: - "That's a design decision. Capture it in design.md?" @@ -192,7 +304,8 @@ If the user mentions a change or you detect one is relevant: ## Handling Different Entry Points **User brings a vague idea:** -``` + +```text User: I'm thinking about adding real-time collaboration You: Real-time collab is a big space. Let me think about this... @@ -215,7 +328,8 @@ You: Real-time collab is a big space. Let me think about this... ``` **User brings a specific problem:** -``` + +```text User: The auth system is a mess You: [reads codebase] @@ -246,7 +360,8 @@ You: [reads codebase] ``` **User is stuck mid-implementation:** -``` + +```text User: /opsx:explore add-auth-system The OAuth integration is more complex than expected @@ -263,7 +378,8 @@ You: [reads change artifacts] ``` **User wants to compare options:** -``` + +```text User: Should we use Postgres or SQLite? You: Generic answer is boring. What's the context? @@ -304,7 +420,7 @@ There's no required ending. Discovery might: When it feels like things are crystallizing, you might summarize: -``` +```text ## What We Figured Out **The problem**: [crystallized understanding] @@ -324,12 +440,25 @@ But this summary is optional. Sometimes the thinking IS the value. ## Guardrails -- **Don't implement** - Never write code or implement features. Workflow configuration counts too: creating or editing schemas, templates, or `openspec/config.yaml` is a change, not thinking. Creating or updating OpenSpec change artifacts within the confirmed scope is fine, writing anything else is not. +- **Don't implement** - Never write code or implement features. Workflow + configuration counts too: creating or editing schemas, templates, or + `openspec/config.yaml` is a change, not thinking. Creating or updating + OpenSpec change artifacts within the confirmed scope is fine, writing anything + else is not. - **Don't fake understanding** - If something is unclear, dig deeper - **Don't rush** - Discovery is thinking time, not task time - **Don't force structure** - Let patterns emerge naturally -- **Don't auto-capture** - Offer to save insights, don't just do it. Read-only commands and tools need no confirmation. Before the first write-capable action—including `openspec new change` or another command that writes files—name the artifacts or files and proposed changes, ask a direct yes/no question, and wait for explicit confirmation in a separate user message. That confirmation covers only the described scope; ask again before expanding it. Answers to design or clarifying questions are never consent to write. -- **Don't manually scaffold changes** - Never create a new change directory under `openspec/changes/` by hand. Always use `openspec new change ""` (with `--store ` when applicable) so required metadata such as `.openspec.yaml` is created before writing artifacts. +- **Don't auto-capture** - Offer to save insights, don't just do it. Read-only + commands and tools need no confirmation. Before the first write-capable + action—including `openspec new change` or another command that writes + files—name the artifacts or files and proposed changes, ask a direct yes/no + question, and wait for explicit confirmation in a separate user message. That + confirmation covers only the described scope; ask again before expanding it. + Answers to design or clarifying questions are never consent to write. +- **Don't manually scaffold changes** - Never create a new change directory + under `openspec/changes/` by hand. Always use `openspec new change ""` + (with `--store ` when applicable) so required metadata such as + `.openspec.yaml` is created before writing artifacts. - **Do visualize** - A good diagram is worth many paragraphs - **Do explore the codebase** - Ground discussions in reality - **Do question assumptions** - Including the user's and your own diff --git a/.claude/skills/openspec-propose/SKILL.md b/.claude/skills/openspec-propose/SKILL.md index 6d45c32..9cba6a2 100644 --- a/.claude/skills/openspec-propose/SKILL.md +++ b/.claude/skills/openspec-propose/SKILL.md @@ -12,115 +12,216 @@ metadata: Propose a new change - create the change and generate all artifacts in one step. -**Planning boundary**: This workflow creates planning artifacts only. The user request that selected or triggered this workflow authorizes planning only, even if it asks to build or fix something. Do not edit project code. After the planning artifacts are complete, stop. Do not start implementation in the same response, even if the initial request asks for it. Wait for a new user request after the artifacts are presented; then start the apply workflow. +**Planning boundary**: This workflow creates planning artifacts only. The user +request that selected or triggered this workflow authorizes planning only, even +if it asks to build or fix something. Do not edit project code. After the +planning artifacts are complete, stop. Do not start implementation in the same +response, even if the initial request asks for it. Wait for a new user request +after the artifacts are presented; then start the apply workflow. + +I'll create a change with the artifacts your schema defines. With the default +spec-driven schema that is: -I'll create a change with the artifacts your schema defines. With the default spec-driven schema that is: - proposal.md (what & why) -- `specs//spec.md` (what the system must do - a delta, not the main spec) +- `specs//spec.md` (what the system must do - a delta, not the + main spec) - design.md (how) - tasks.md (implementation steps) -`` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve an existing capability's full path and follow the project's established organization for new capabilities. +`` is the spec directory relative to `specs/` (for example, +`user-auth` or `identity/user-auth`). Preserve an existing capability's full +path and follow the project's established organization for new capabilities. -When the user is ready to implement, they must start the apply workflow explicitly. +When the user is ready to implement, they must start the apply workflow +explicitly. --- -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -**Input**: The user's request should include a change name (kebab-case) OR a description of what they want to build. +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +**Input**: The user's request should include a change name (kebab-case) OR a +description of what they want to build. **Steps** 1. **Understand the request and clarify material ambiguity** If no clear input is provided, ask the user (open-ended, no preset options): - > "What change do you want to work on? Describe what you want to build or fix." + > "What change do you want to work on? Describe what you want to build or + > fix." - From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`). + From their description, derive a kebab-case name (e.g., "add user + authentication" → `add-user-auth`). - **IMPORTANT**: Do NOT proceed without understanding what the user wants to build. + **IMPORTANT**: Do NOT proceed without understanding what the user wants to + build. - If the request contains ambiguity that would materially affect scope, externally observable behavior, compatibility, or acceptance criteria, ask the user before creating the change. For minor details, make a reasonable assumption and record it in the planning artifacts. + If the request contains ambiguity that would materially affect scope, + externally observable behavior, compatibility, or acceptance criteria, ask + the user before creating the change. For minor details, make a reasonable + assumption and record it in the planning artifacts. 2. **Determine the workflow schema** - Use the configured default schema unless the user explicitly requests a different workflow. + Use the configured default schema unless the user explicitly requests a + different workflow. **Use a different schema only if the user:** - - Explicitly requests a specific schema by name → use `--schema ` - - Asks to "show workflows" or asks "what workflows" exist → resolve the authoritative root by running `openspec context --json` from the current working directory. If the user explicitly selected a registered store, use `openspec context --json --store ""`. Then run `openspec schemas --json` with its working directory set to the returned `root.path` and let them choose. This preserves roots selected by a local `store:` pointer or the global `defaultStore`; when a registered store was explicitly selected, append `--store ""` to `openspec schemas --json` as well. If context reports only `no_openspec_root`, run `openspec schemas --json` from the current working directory instead. Do not use this fallback for invalid or unavailable stores. + - Explicitly requests a specific schema by name → use + `--schema ` + - Asks to "show workflows" or asks "what workflows" exist → resolve the + authoritative root by running `openspec context --json` from the current + working directory. If the user explicitly selected a registered store, use + `openspec context --json --store ""`. Then run + `openspec schemas --json` with its working directory set to the returned + `root.path` and let them choose. This preserves roots selected by a local + `store:` pointer or the global `defaultStore`; when a registered store was + explicitly selected, append `--store ""` to + `openspec schemas --json` as well. If context reports only + `no_openspec_root`, run `openspec schemas --json` from the current working + directory instead. Do not use this fallback for invalid or unavailable + stores. Otherwise, omit `--schema` to preserve the configured default. 3. **Create the change directory** - Choose one schema form below. If a registered store is selected, append `--store ""` to that command and each later OpenSpec command shown below that accepts `--store`. + Choose one schema form below. If a registered store is selected, append + `--store ""` to that command and each later OpenSpec command shown + below that accepts `--store`. Using the configured default: + ```bash openspec new change "" ``` Using an explicitly requested schema: + ```bash openspec new change "" --schema "" ``` - This creates a scaffolded change in the planning home resolved by the CLI with `.openspec.yaml`. + + This creates a scaffolded change in the planning home resolved by the CLI + with `.openspec.yaml`. 4. **Get the artifact build order** + ```bash openspec status --change "" --json ``` + Parse the JSON to get: - - `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`) - - `artifacts`: list of all artifacts, each with its `status` and its `requires` edges (the artifact IDs it directly depends on) - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context. Use these instead of assuming repo-local paths. + - `applyRequires`: array of artifact IDs needed before implementation (e.g., + `["tasks"]`) + - `artifacts`: list of all artifacts, each with its `status` and its + `requires` edges (the artifact IDs it directly depends on) + - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path + and scope context. Use these instead of assuming repo-local paths. 5. **Create every artifact in the required set** Use a todo list to track progress through the artifacts. - Loop through artifacts in dependency order (artifacts with no pending dependencies first): + Loop through artifacts in dependency order (artifacts with no pending + dependencies first): a. **For each artifact that is `ready` (dependencies satisfied)**: - Get instructions: + ```bash openspec instructions --change "" --json ``` + - The instructions JSON includes: - - `context`: Project background (constraints for you - do NOT include in output) - - `rules`: Artifact-specific rules (constraints for you - do NOT include in output) + - `context`: Project background (constraints for you - do NOT include in + output) + - `rules`: Artifact-specific rules (constraints for you - do NOT include + in output) - `template`: The structure to use for your output file - `instruction`: Schema-specific guidance for this artifact type - - `skipped`/`warning`: present when the change declares skip_specs and this artifact must NOT be created - stop and pick another artifact + - `skipped`/`warning`: present when the change declares skip_specs and + this artifact must NOT be created - stop and pick another artifact - `resolvedOutputPath`: Resolved path or pattern to write the artifact - `dependencies`: Completed artifacts to read for context - - Read any completed dependency files for context - always re-read them from disk, even if you saw them earlier in the conversation (the user may have edited them) - - **Inspect the relevant project before drafting**: Read `context` and `rules` first, then inspect relevant implementation, nearby tests, configuration, and documentation outside `openspec/`. Keep inspection read-only and proportional to the change; reuse findings for later artifacts and inspect more only as needed. - - Identify the target project from the request and project context; the planning home may be separate from the code. If the target is unclear, ask. For greenfield or non-code changes, inspect the available structure and relevant documents. If source is unavailable, state the limitation and ask when it materially affects the plan. - - Ground scope, approach, and tasks in what you find. Distinguish observed behavior from assumptions and proposed additions; surface conflicts with existing specs instead of silently deciding which is correct. - - Do this discovery now, rather than leaving generic "explore the codebase" or "make a plan" tasks for implementation. Keep any necessary follow-up investigation specific to an unresolved question. - - If the `instruction` field delegates creation to a specific skill or command, invoke it to produce the artifact instead of writing the file yourself, then verify the artifact file exists at `resolvedOutputPath` - - Otherwise create the artifact file using `template` as the structure and write it to `resolvedOutputPath`. If `resolvedOutputPath` is a glob, follow `instruction` to choose the concrete file path - - Apply `context` and `rules` as constraints - but do NOT copy them into the file + - Read any completed dependency files for context - always re-read them + from disk, even if you saw them earlier in the conversation (the user + may have edited them) + - **Inspect the relevant project before drafting**: Read `context` and + `rules` first, then inspect relevant implementation, nearby tests, + configuration, and documentation outside `openspec/`. Keep inspection + read-only and proportional to the change; reuse findings for later + artifacts and inspect more only as needed. + - Identify the target project from the request and project context; the + planning home may be separate from the code. If the target is unclear, + ask. For greenfield or non-code changes, inspect the available + structure and relevant documents. If source is unavailable, state the + limitation and ask when it materially affects the plan. + - Ground scope, approach, and tasks in what you find. Distinguish + observed behavior from assumptions and proposed additions; surface + conflicts with existing specs instead of silently deciding which is + correct. + - Do this discovery now, rather than leaving generic "explore the + codebase" or "make a plan" tasks for implementation. Keep any + necessary follow-up investigation specific to an unresolved question. + - If the `instruction` field delegates creation to a specific skill or + command, invoke it to produce the artifact instead of writing the file + yourself, then verify the artifact file exists at `resolvedOutputPath` + - Otherwise create the artifact file using `template` as the structure and + write it to `resolvedOutputPath`. If `resolvedOutputPath` is a glob, + follow `instruction` to choose the concrete file path + - Apply `context` and `rules` as constraints - but do NOT copy them into + the file - Show brief progress: "Created " - b. **Continue until every artifact in the required set exists (not just `apply.requires`)** - - After creating each artifact, re-run `openspec status --change "" --json` - - The required set is `applyRequires` plus every artifact reachable from those by following the `requires` edges in `status --json` - walk them transitively (spec-driven closes over proposal, specs, design, tasks). Leave artifacts outside that set alone - - `status` is file-existence only, so an `applyRequires` artifact reading `done` does NOT mean its dependencies exist - writing `tasks.md` early marks `tasks` done while `specs` was never written. Use each artifact's `requires` edges, not its `status`, to build the required set: a `done` artifact still lists what it depends on - - An artifact already reading `status: "skipped"` is satisfied: the change declares `skip_specs` in `.openspec.yaml`, so its files must NOT exist. Never try to create one - - Create every artifact in the required set that is missing, then re-check - creating one can unblock others - - Skip one only when `status` already reports it `skipped`, or when its own `instruction` says it is conditional: run `openspec instructions --change "" --json` and skip only if its `instruction` field marks it optional (e.g. "create only if..."). Spec-driven's `design.md` qualifies; `specs` qualifies only via the `skipped` status above, never by your own judgment. Tell the user, and do not reconsider it - - Dependencies are enablers, not gates: if a required artifact is still `blocked` only because you skipped a conditional dependency, write it anyway - - Stop when every artifact in the required set is `done`, `skipped`, or was deliberately skipped + b. **Continue until every artifact in the required set exists (not just + `apply.requires`)** + - After creating each artifact, re-run + `openspec status --change "" --json` + - The required set is `applyRequires` plus every artifact reachable from + those by following the `requires` edges in `status --json` - walk them + transitively (spec-driven closes over proposal, specs, design, tasks). + Leave artifacts outside that set alone + - `status` is file-existence only, so an `applyRequires` artifact reading + `done` does NOT mean its dependencies exist - writing `tasks.md` early + marks `tasks` done while `specs` was never written. Use each artifact's + `requires` edges, not its `status`, to build the required set: a `done` + artifact still lists what it depends on + - An artifact already reading `status: "skipped"` is satisfied: the change + declares `skip_specs` in `.openspec.yaml`, so its files must NOT exist. + Never try to create one + - Create every artifact in the required set that is missing, then + re-check - creating one can unblock others + - Skip one only when `status` already reports it `skipped`, or when its + own `instruction` says it is conditional: run + `openspec instructions --change "" --json` and skip + only if its `instruction` field marks it optional (e.g. "create only + if..."). Spec-driven's `design.md` qualifies; `specs` qualifies only via + the `skipped` status above, never by your own judgment. Tell the user, + and do not reconsider it + - Dependencies are enablers, not gates: if a required artifact is still + `blocked` only because you skipped a conditional dependency, write it + anyway + - Stop when every artifact in the required set is `done`, `skipped`, or + was deliberately skipped c. **If an artifact requires user input** (unclear context): - Ask the user to clarify - Then continue with creation 6. **Show final status** + ```bash openspec status --change "" ``` @@ -128,26 +229,44 @@ When the user is ready to implement, they must start the apply workflow explicit **Output** After completing all artifacts, summarize: + - Change name and location -- List of artifacts created with brief descriptions, plus any conditional artifact you skipped and why +- List of artifacts created with brief descriptions, plus any conditional + artifact you skipped and why - What's ready: "All artifacts needed for implementation are ready." -- Prompt: "The artifacts are ready for review. When you are ready, run `/opsx:apply` or ask me to apply this change." +- Prompt: "The artifacts are ready for review. When you are ready, run + `/opsx:apply` or ask me to apply this change." **Artifact Creation Guidelines** -- Follow the `instruction` field from `openspec instructions` for each artifact type - it is the authoritative guidance, even for familiar artifact names -- If the `instruction` field directs you to use a specific skill or command to create the artifact, invoke it instead of writing the artifact directly +- Follow the `instruction` field from `openspec instructions` for each artifact + type - it is the authoritative guidance, even for familiar artifact names +- If the `instruction` field directs you to use a specific skill or command to + create the artifact, invoke it instead of writing the artifact directly - The schema defines what each artifact should contain - follow it - Read dependency artifacts for context before creating new ones - Use `template` as the structure for your output file - fill in its sections -- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file - - Do NOT copy ``, ``, `` blocks into the artifact +- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for + the file + - Do NOT copy ``, ``, `` blocks into the + artifact - These guide what you write, but should never appear in the output **Guardrails** -- The request that invoked this workflow authorizes planning only. Any implementation or apply instruction in that request does not carry forward. Do NOT implement the change, start the apply workflow, or edit project code during this workflow. After presenting the artifacts, stop and wait for a new user request to start the apply workflow -- Create every artifact the apply phase transitively depends on, not just the ids listed in `apply.requires` -- Always read dependency artifacts before creating a new one - re-read from disk, not from conversation memory (files may have changed since you last saw them) -- Ask about ambiguities that would materially change scope, externally observable behavior, compatibility, or acceptance criteria; for minor details, make reasonable assumptions and record them -- If a change with that name already exists, ask if user wants to continue it or create a new one + +- The request that invoked this workflow authorizes planning only. Any + implementation or apply instruction in that request does not carry forward. Do + NOT implement the change, start the apply workflow, or edit project code + during this workflow. After presenting the artifacts, stop and wait for a new + user request to start the apply workflow +- Create every artifact the apply phase transitively depends on, not just the + ids listed in `apply.requires` +- Always read dependency artifacts before creating a new one - re-read from + disk, not from conversation memory (files may have changed since you last saw + them) +- Ask about ambiguities that would materially change scope, externally + observable behavior, compatibility, or acceptance criteria; for minor details, + make reasonable assumptions and record them +- If a change with that name already exists, ask if user wants to continue it or + create a new one - Verify each artifact file exists after writing before proceeding to next diff --git a/.claude/skills/openspec-sync-specs/SKILL.md b/.claude/skills/openspec-sync-specs/SKILL.md index b939242..a919efb 100644 --- a/.claude/skills/openspec-sync-specs/SKILL.md +++ b/.claude/skills/openspec-sync-specs/SKILL.md @@ -12,13 +12,30 @@ metadata: Sync delta specs from a change to main specs. -This is an **agent-driven** operation - you will read delta specs and directly edit main specs to apply the changes. This allows intelligent merging (e.g., adding a scenario without copying the entire requirement). - -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -`` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve the full path from each delta spec when resolving its main spec. - -**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. +This is an **agent-driven** operation - you will read delta specs and directly +edit main specs to apply the changes. This allows intelligent merging (e.g., +adding a scenario without copying the entire requirement). + +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +`` is the spec directory relative to `specs/` (for example, +`user-auth` or `identity/user-auth`). Preserve the full path from each delta +spec when resolving its main spec. + +**Input**: Optionally specify a change name. If omitted, check if it can be +inferred from conversation context. If vague or ambiguous you MUST prompt for +available changes. **Steps** @@ -27,20 +44,27 @@ This is an **agent-driven** operation - you will read delta specs and directly e If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes and ask + the user to select one - When prompting, show changes that have delta specs (under `specs/` directory). + When prompting, show changes that have delta specs (under `specs/` + directory). - Always announce: "Using change: " and how to override (e.g., `/opsx:sync `). + Always announce: "Using change: " and how to override (e.g., + `/opsx:sync `). 2. **Resolve change context** Run: + ```bash openspec status --change "" --json ``` - The JSON includes `planningHome.root`. Main specs live under `/openspec/specs/` — use that (store-aware) root for every main-spec path below, not a hardcoded repo path. When a store is selected it points at the store, not the current repository. + The JSON includes `planningHome.root`. Main specs live under + `/openspec/specs/` — use that (store-aware) root for every + main-spec path below, not a hardcoded repo path. When a store is selected it + points at the store, not the current repository. 3. **Find delta specs** @@ -80,29 +104,34 @@ This is an **agent-driven** operation - you will read delta specs and directly e fetch the same instructions again. - Otherwise run that command once now with the same selected-root flags. - If the direct lookup exits non-zero or returns invalid artifact-instruction - JSON, report the error and stop before writing any main spec. Do not treat the - failure as an absent rule set. - - A valid response with omitted `rules` means no artifact rules are configured - and the existing semantic merge continues. + JSON, report the error and stop before writing any main spec. Do not treat + the failure as an absent rule set. + - A valid response with omitted `rules` means no artifact rules are + configured and the existing semantic merge continues. - Apply returned `rules` only to the content and form of the main specs produced - by this merge. Artifact rules are not operation guidance and cannot change - selected roots, delta paths, CLI checks, or workflow steps. Use their text as - constraints without copying it verbatim into a main spec or summary. + Apply returned `rules` only to the content and form of the main specs + produced by this merge. Artifact rules are not operation guidance and cannot + change selected roots, delta paths, CLI checks, or workflow steps. Use their + text as constraints without copying it verbatim into a main spec or summary. - For each capability delta spec path selected in step 3 — the full `existingOutputPaths` list, or the narrowed subset when a caller supplied one (these may belong to a selected store, not the repo): + For each capability delta spec path selected in step 3 — the full + `existingOutputPaths` list, or the narrowed subset when a caller supplied one + (these may belong to a selected store, not the repo): a. **Read the delta spec** to understand the intended changes - b. **Read the main spec** at `/openspec/specs//spec.md` (may not exist yet) + b. **Read the main spec** at + `/openspec/specs//spec.md` (may not exist + yet) c. **Apply changes intelligently**: - **ADDED Requirements:** + **ADDED Requirements:** - If requirement doesn't exist in main spec → add it - - If requirement already exists → update it to match (treat as implicit MODIFIED) + - If requirement already exists → update it to match (treat as implicit + MODIFIED) - **MODIFIED Requirements:** + **MODIFIED Requirements:** - Find the requirement in main spec - Apply the changes - this can be: - Adding new scenarios the main spec does not have yet @@ -110,10 +139,10 @@ This is an **agent-driven** operation - you will read delta specs and directly e - Changing the requirement description - Preserve scenarios/content not mentioned in the delta - **REMOVED Requirements:** + **REMOVED Requirements:** - Remove the entire requirement block from main spec - - Retiring the capability. Delete the whole `spec.md` - and the directory once - nothing else is left in it - only when ALL of these hold: + - Retiring the capability. Delete the whole `spec.md` - and the directory + once nothing else is left in it - only when ALL of these hold: 1. removing the requirements *this run* left no requirement blocks; 2. the rest of the spec is well-formed (it still has a `## Purpose`); 3. the main spec was not already empty before this sync - if you removed @@ -124,35 +153,38 @@ This is an **agent-driven** operation - you will read delta specs and directly e 5. the change's `.openspec.yaml` declares `retire_capabilities: true`; 6. the `spec.md` resolves inside the real specs root (do not follow a capability-directory symlink to delete an external file). - If removing the selected requirements would leave no requirement blocks and - any retirement condition is not satisfied, do not modify the main spec. Stop - the sync for that capability, report the blocking condition, and tell the user - how to resolve it. Never write or leave an empty `## Requirements` section. - When only the marker is missing, say that too - it is the one thing the user - can add to make the retirement go through. - - Deleting the file also deletes its `## Purpose`; any other section blocks - retirement. Name Purpose when you report the retirement. Include a pasteable - `git checkout` only when the spec lived in the caller's checkout; - otherwise give checkout-scoped recovery guidance. - - **RENAMED Requirements:** + If removing the selected requirements would leave no requirement blocks + and any retirement condition is not satisfied, do not modify the main + spec. Stop the sync for that capability, report the blocking condition, + and tell the user how to resolve it. Never write or leave an empty + `## Requirements` section. When only the marker is missing, say that + too - it is the one thing the user can add to make the retirement go + through. + - Deleting the file also deletes its `## Purpose`; any other section + blocks retirement. Name Purpose when you report the retirement. Include + a pasteable `git checkout` only when the spec lived in the caller's + checkout; otherwise give checkout-scoped recovery guidance. + + **RENAMED Requirements:** - Find the FROM requirement, rename to TO - **`## Purpose` in the delta:** + **`## Purpose` in the delta:** - The main spec already has one and it is authoritative - leave it alone (this is what `openspec archive` does; it warns and moves on) d. **Create new main spec** if capability doesn't exist yet: - Create `/openspec/specs//spec.md` - - Add Purpose section: copy the delta's `## Purpose` body verbatim when it has one - (this is what `openspec archive` does); only write a brief TBD placeholder when it does not + - Add Purpose section: copy the delta's `## Purpose` body verbatim when it + has one (this is what `openspec archive` does); only write a brief TBD + placeholder when it does not - Add Requirements section with the ADDED requirements - Follow the **Main Spec Format Reference** below 5. **Validate updated main specs** - Run `openspec validate --specs` with the same selected-root flags used earlier. - If validation fails, report the problems and do not claim the sync succeeded. + Run `openspec validate --specs` with the same selected-root flags used + earlier. If validation fails, report the problems and do not claim the sync + succeeded. 6. **Show summary** @@ -205,7 +237,9 @@ The system SHALL keep doing the existing thing, now also handling A. **Main Spec Format Reference** -Main specs are what the delta merges INTO. They must never contain delta operation headers (`## ADDED/MODIFIED/REMOVED/RENAMED Requirements`) - after syncing, every requirement lives under a single `## Requirements` section: +Main specs are what the delta merges INTO. They must never contain delta +operation headers (`## ADDED/MODIFIED/REMOVED/RENAMED Requirements`) - after +syncing, every requirement lives under a single `## Requirements` section: ```markdown # Specification @@ -226,7 +260,10 @@ The system SHALL do something new. **Key Principle: Intelligent Merging** Unlike programmatic merging, you merge rather than overwrite: -- A MODIFIED block carries the whole requirement - body plus every scenario that survives the change. `openspec validate` and `openspec archive` both reject one that drops a scenario the main spec still has. + +- A MODIFIED block carries the whole requirement - body plus every scenario that + survives the change. `openspec validate` and `openspec archive` both reject + one that drops a scenario the main spec still has. - Keep anything the delta does not mention, in the main spec's existing order - Use your judgment to merge changes sensibly @@ -249,14 +286,22 @@ Main specs are now updated. The change remains active - archive when implementat ``` **Guardrails** + - Read both delta and main specs before making changes - Preserve existing content not mentioned in delta -- Never copy a delta file into a main spec as-is - merge its content so the main spec keeps the Main Spec Format Reference structure, with no delta operation headers +- Never copy a delta file into a main spec as-is - merge its content so the main + spec keeps the Main Spec Format Reference structure, with no delta operation + headers - If something is unclear, ask for clarification - Show what you're changing as you go - The operation should be idempotent - running twice should give same result -- Use only `artifactPaths.specs.existingOutputPaths`; never infer delta specs from unrelated artifacts -- Honor a caller-supplied subset of `existingOutputPaths`; never widen it back to the full list -- Fetch specs instructions once for direct sync, or reuse the archive-supplied snapshot inline -- Stop before every main-spec write on a non-zero or invalid JSON specs-instruction response -- Artifact rules constrain only the specs being written and are never copied into output files +- Use only `artifactPaths.specs.existingOutputPaths`; never infer delta specs + from unrelated artifacts +- Honor a caller-supplied subset of `existingOutputPaths`; never widen it back + to the full list +- Fetch specs instructions once for direct sync, or reuse the archive-supplied + snapshot inline +- Stop before every main-spec write on a non-zero or invalid JSON + specs-instruction response +- Artifact rules constrain only the specs being written and are never copied + into output files diff --git a/.claude/skills/openspec-update-change/SKILL.md b/.claude/skills/openspec-update-change/SKILL.md index 1db491a..3205311 100644 --- a/.claude/skills/openspec-update-change/SKILL.md +++ b/.claude/skills/openspec-update-change/SKILL.md @@ -10,13 +10,31 @@ metadata: generatedBy: "1.12.0" --- -Revise a change's existing planning artifacts and keep them coherent. Never edit code. - -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. - -`/opsx:continue` is an optional workflow and may not be installed. Before suggesting it anywhere below, verify that it is available. If it is unavailable, `openspec status --change "" --json` shows the next artifact and `openspec instructions "" --change "" --json` explains how to create it. +Revise a change's existing planning artifacts and keep them coherent. Never edit +code. + +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +**Input**: Optionally specify a change name. If omitted, check if it can be +inferred from conversation context. If vague or ambiguous you MUST prompt for +available changes. + +`/opsx:continue` is an optional workflow and may not be installed. Before +suggesting it anywhere below, verify that it is available. If it is unavailable, +`openspec status --change "" --json` shows the next artifact and +`openspec instructions "" --change "" --json` explains how to +create it. **Steps** @@ -25,67 +43,105 @@ Revise a change's existing planning artifacts and keep them coherent. Never edit If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes sorted by most recently modified, and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes sorted by + most recently modified, and ask the user to select one - When prompting, present the top 3-4 most recently modified changes as options, showing: + When prompting, present the top 3-4 most recently modified changes as + options, showing: - Change name - Schema (from `schema` field if present, otherwise "spec-driven") - Status (e.g., "0/5 tasks", "complete", "no tasks") - How recently it was modified (from `lastModified` field) - Mark the most recently modified change as "(Recommended)" since it's likely what the user wants to update. + Mark the most recently modified change as "(Recommended)" since it's likely + what the user wants to update. - Always announce: "Using change: " and how to override (e.g., `/opsx:update `). + Always announce: "Using change: " and how to override (e.g., + `/opsx:update `). 2. **Get the change's artifacts** + ```bash openspec status --change "" --json ``` + Parse the JSON to understand current state. The response includes: - `schemaName`: The workflow schema being used (e.g., "spec-driven") - - `artifacts`: Array of artifacts with their status ("done", "skipped", "ready", "blocked") - - `isPlanningComplete`: Boolean indicating if all planning artifacts are complete. Older CLI versions expose the same value as `isComplete`. - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context. Use these instead of assuming repo-local paths. - - The artifact ids and paths come from the active schema - do NOT assume them, and do NOT branch on hardcoded artifact names. Custom schemas must work unchanged. - - The files to edit are `artifactPaths..existingOutputPaths` - the concrete files that exist on disk, already glob-expanded for glob artifacts (e.g. `specs/**/*.md`). Do NOT write to `resolvedOutputPath`: for a glob artifact it is still the glob pattern, not a real file. + - `artifacts`: Array of artifacts with their status ("done", "skipped", + "ready", "blocked") + - `isPlanningComplete`: Boolean indicating if all planning artifacts are + complete. Older CLI versions expose the same value as `isComplete`. + - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path + and scope context. Use these instead of assuming repo-local paths. + + The artifact ids and paths come from the active schema - do NOT assume them, + and do NOT branch on hardcoded artifact names. Custom schemas must work + unchanged. + + The files to edit are `artifactPaths..existingOutputPaths` - the concrete + files that exist on disk, already glob-expanded for glob artifacts (e.g. + `specs/**/*.md`). Do NOT write to `resolvedOutputPath`: for a glob artifact + it is still the glob pattern, not a real file. 3. **Understand the request** - - If the user asked for a specific revision ("the design now uses X"), that is the starting edit. - - If they only said "update" / "make this coherent", treat it as a coherence review: read the existing artifacts and check them against each other for contradictions, gaps, and duplication. + - If the user asked for a specific revision ("the design now uses X"), that + is the starting edit. + - If they only said "update" / "make this coherent", treat it as a coherence + review: read the existing artifacts and check them against each other for + contradictions, gaps, and duplication. 4. **Read and reconcile** - - Read the artifact(s) the request touches and the change's other existing artifacts. - - Apply the requested edit. Then check every other existing artifact against it - in ANY direction: an edit to a later artifact may require revising an earlier one, not only the other way around. Build order is a useful reading order, not a constraint on which artifacts may be revised. + - Read the artifact(s) the request touches and the change's other existing + artifacts. + - Apply the requested edit. Then check every other existing artifact against + it - in ANY direction: an edit to a later artifact may require revising an + earlier one, not only the other way around. Build order is a useful reading + order, not a constraint on which artifacts may be revised. - Note everything that is now inconsistent, missing, or contradictory. - - Revise only files that already exist (`existingOutputPaths`). Do NOT create artifacts that don't exist yet, and do NOT invent new files under a glob artifact - note them and point the user to `/opsx:continue` to create them. + - Revise only files that already exist (`existingOutputPaths`). Do NOT create + artifacts that don't exist yet, and do NOT invent new files under a glob + artifact - note them and point the user to `/opsx:continue` to create them. - If the change is already coherent, say so and make no edits. 5. **Confirm and apply, one artifact at a time** - Show each proposed revision and why. Write only after the user confirms. - - If the user rejects a revision, do not write it - leave that artifact unchanged. - - When a substantial rewrite is needed, get that artifact's rules and template first: + - If the user rejects a revision, do not write it - leave that artifact + unchanged. + - When a substantial rewrite is needed, get that artifact's rules and + template first: + ```bash openspec instructions "" --change "" --json ``` 6. **Point to the next step (guidance only - NEVER act on it)** - Artifacts still missing -> suggest `/opsx:continue` to create them. - - Change already implemented (tasks checked off / already applied) -> the code may no longer match the revised plan; suggest `/opsx:apply` to carry the delta into code. + - Change already implemented (tasks checked off / already applied) -> the + code may no longer match the revised plan; suggest `/opsx:apply` to carry + the delta into code. - Everything done and implemented -> suggest `/opsx:archive`. **Output** After each invocation, show: + - Which artifacts were revised (and which proposed revisions were rejected) - Anything deferred to `/opsx:continue` (not-yet-created artifacts or files) - Where the change stands and the recommended next command **Guardrails** -- Planning artifacts only - NEVER edit implementation code. If the revised plan implies code changes, stop and point to `/opsx:apply`. -- Use the artifact ids and paths reported by `openspec status`; never branch on hardcoded artifact names. -- Edit only the concrete files in `existingOutputPaths`; never write to a glob `resolvedOutputPath`. -- Do not advance the build frontier: no new artifacts, no new files under glob artifacts - that is `/opsx:continue`'s job. + +- Planning artifacts only - NEVER edit implementation code. If the revised plan + implies code changes, stop and point to `/opsx:apply`. +- Use the artifact ids and paths reported by `openspec status`; never branch on + hardcoded artifact names. +- Edit only the concrete files in `existingOutputPaths`; never write to a glob + `resolvedOutputPath`. +- Do not advance the build frontier: no new artifacts, no new files under glob + artifacts - that is `/opsx:continue`'s job. - Confirm every edit with the user before writing. -- If the request changes the change's *intent* rather than refining it, first verify whether the optional `/opsx:new` workflow is available. If it is, recommend starting fresh with `/opsx:new` (the "Update vs. Start Fresh" heuristic). If it is unavailable, ask for a distinct unused change name and recommend `openspec new change ""` instead. +- If the request changes the change's *intent* rather than refining it, first + verify whether the optional `/opsx:new` workflow is available. If it is, + recommend starting fresh with `/opsx:new` (the "Update vs. Start Fresh" + heuristic). If it is unavailable, ask for a distinct unused change name and + recommend `openspec new change ""` instead. diff --git a/.gemini/skills/openspec-apply-change/SKILL.md b/.gemini/skills/openspec-apply-change/SKILL.md index 384e3c0..d915080 100644 --- a/.gemini/skills/openspec-apply-change/SKILL.md +++ b/.gemini/skills/openspec-apply-change/SKILL.md @@ -12,9 +12,22 @@ metadata: Implement tasks from an OpenSpec change. -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -**Input**: Optionally specify a change name (e.g., `/opsx:apply add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +**Input**: Optionally specify a change name (e.g., `/opsx:apply add-auth`). If +omitted, check if it can be inferred from conversation context. If vague or +ambiguous you MUST prompt for available changes. **Steps** @@ -23,18 +36,24 @@ Implement tasks from an OpenSpec change. If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes and ask + the user to select one - Always announce: "Using change: " and how to override (e.g., `/opsx:apply `). + Always announce: "Using change: " and how to override (e.g., + `/opsx:apply `). 2. **Check status to understand the schema** + ```bash openspec status --change "" --json ``` + Parse the JSON to understand: - `schemaName`: The workflow being used (e.g., "spec-driven") - - `planningHome`, `changeRoot`, and `actionContext`: planning scope and edit constraints - - Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others) + - `planningHome`, `changeRoot`, and `actionContext`: planning scope and edit + constraints + - Which artifact contains the tasks (typically "tasks" for spec-driven, check + status for others) 3. **Get apply instructions** @@ -43,23 +62,30 @@ Implement tasks from an OpenSpec change. ``` This returns: - - `contextFiles`: artifact ID -> array of concrete file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs) + - `contextFiles`: artifact ID -> array of concrete file paths (varies by + schema - could be proposal/specs/design/tasks or + spec/tests/implementation/docs) - Progress (total, complete, remaining) - Task list with status - Dynamic instruction based on current state - - Optional `context`: current required project instruction input from the selected root + - Optional `context`: current required project instruction input from the + selected root - Optional `operationGuidance`: current advisory guidance for apply **Handle states:** - - If `state: "blocked"` (missing artifacts): show message, suggest using `/opsx:continue` (if it is not installed, run `openspec status --change "" --json` to see the next artifact and `openspec instructions --change "" --json` for how to create it) + - If `state: "blocked"` (missing artifacts): show message, suggest using + `/opsx:continue` (if it is not installed, run + `openspec status --change "" --json` to see the next artifact and + `openspec instructions --change "" --json` for how to + create it) - If `state: "all_done"`: congratulate, suggest archive - Otherwise: proceed to implementation Treat `context` as a required prompt-level input. Read and consider it, and - apply relevant project facts, conventions, and constraints while implementing. - Treat `operationGuidance` as optional additive advice. Read and consider every - entry, and follow entries that are applicable and compatible with the built-in - workflow. + apply relevant project facts, conventions, and constraints while + implementing. Treat `operationGuidance` as optional additive advice. Read and + consider every entry, and follow entries that are applicable and compatible + with the built-in workflow. Keep both fields separate from CLI-returned state, missing artifacts, tasks, progress, `contextFiles`, and the built-in `instruction`. They are not @@ -72,8 +98,8 @@ Implement tasks from an OpenSpec change. 4. **Read context files** - Read every file path listed under `contextFiles` from the apply instructions output. - The files depend on the schema being used: + Read every file path listed under `contextFiles` from the apply instructions + output. The files depend on the schema being used: - **spec-driven**: proposal, specs, design, tasks - Other schemas: follow the contextFiles from CLI output @@ -100,7 +126,9 @@ Implement tasks from an OpenSpec change. **Pause if:** - Task is unclear → ask for clarification - Implementation reveals a design issue → suggest updating artifacts - - A task needs work beyond what the spec and tasks describe, or you are tempted to drop, narrow, defer, or accept exceptions to specified behavior to make it fit → surface the added scope and ask; do not absorb it silently + - A task needs work beyond what the spec and tasks describe, or you are + tempted to drop, narrow, defer, or accept exceptions to specified behavior + to make it fit → surface the added scope and ask; do not absorb it silently - Error or blocker encountered → report and wait for guidance - User interrupts @@ -114,7 +142,7 @@ Implement tasks from an OpenSpec change. **Output During Implementation** -``` +```text ## Implementing: (schema: ) Working on task 3/7: @@ -128,7 +156,7 @@ Working on task 4/7: **Output On Completion** -``` +```text ## Implementation Complete **Change:** @@ -145,7 +173,7 @@ All tasks complete! You can archive this change with `/opsx:archive`. **Output On Pause (Issue Encountered)** -``` +```text ## Implementation Paused **Change:** @@ -164,6 +192,7 @@ What would you like to do? ``` **Guardrails** + - Keep going through tasks until done or blocked - Always read context files before starting (from the apply instructions output) - If task is ambiguous, pause and ask before implementing @@ -171,18 +200,25 @@ What would you like to do? - Keep code changes minimal and scoped to each task - Update task checkbox immediately after completing each task - Pause on errors, blockers, or unclear requirements - don't guess -- When a task needs work beyond what the spec describes, surface the added scope and pause - never silently narrow, defer, or simplify away specified behavior -- Only mark a task `- [x]` when its specified behavior is fully implemented, not when it is partially done or deferred +- When a task needs work beyond what the spec describes, surface the added scope + and pause - never silently narrow, defer, or simplify away specified behavior +- Only mark a task `- [x]` when its specified behavior is fully implemented, not + when it is partially done or deferred - Use contextFiles from CLI output, don't assume specific file names - Do not use context or operation guidance as proof that a task is complete -- Apply relevant project context; report conflicts with controlling workflow inputs +- Apply relevant project context; report conflicts with controlling workflow + inputs - Consider every guidance entry; explain any inapplicable or conflicting advice -- Do not copy runtime context or operation guidance into implementation files or planning artifacts -- Preserve CLI-controlled blocked/ready/all-done behavior and completion criteria +- Do not copy runtime context or operation guidance into implementation files or + planning artifacts +- Preserve CLI-controlled blocked/ready/all-done behavior and completion + criteria **Fluid Workflow Integration** This skill supports the "actions on a change" model: -- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions -- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly +- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), + after partial implementation, interleaved with other actions +- **Allows artifact updates**: If implementation reveals design issues, suggest + updating artifacts - not phase-locked, work fluidly diff --git a/.gemini/skills/openspec-archive-change/SKILL.md b/.gemini/skills/openspec-archive-change/SKILL.md index dd6efc2..65a94f4 100644 --- a/.gemini/skills/openspec-archive-change/SKILL.md +++ b/.gemini/skills/openspec-archive-change/SKILL.md @@ -12,11 +12,26 @@ metadata: Archive a completed change in the experimental workflow. -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -`` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve the full path from each delta spec when resolving its main spec. - -**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +`` is the spec directory relative to `specs/` (for example, +`user-auth` or `identity/user-auth`). Preserve the full path from each delta +spec when resolving its main spec. + +**Input**: Optionally specify a change name. If omitted, check if it can be +inferred from conversation context. If vague or ambiguous you MUST prompt for +available changes. **Steps** @@ -25,24 +40,29 @@ Archive a completed change in the experimental workflow. If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes and ask + the user to select one When prompting, show only active changes (not already archived). Include the schema used for each change if available. - Always announce: "Using change: " and how to override (e.g., `/opsx:archive `). + Always announce: "Using change: " and how to override (e.g., + `/opsx:archive `). **Load current archive inputs before the existing archive checks:** After resolving the selected change and planning root, run: + ```bash openspec instructions archive --change "" --json ``` - Keep the same selected-root flags on this command. This lookup is advisory and - optional: it only supplies extra prompt inputs, so it must never block archiving. - If it exits non-zero or returns invalid JSON — for example on an older CLI that - does not support this command yet — continue the archive workflow with no - context and no operation guidance. Do not report an error and do not stop. + + Keep the same selected-root flags on this command. This lookup is advisory + and optional: it only supplies extra prompt inputs, so it must never block + archiving. If it exits non-zero or returns invalid JSON — for example on an + older CLI that does not support this command yet — continue the archive + workflow with no context and no operation guidance. Do not report an error + and do not stop. A successful response may omit both optional fields. Treat `context` as a required prompt-level input: read and consider it, and apply relevant project @@ -50,14 +70,15 @@ Archive a completed change in the experimental workflow. additive advice: read and consider every entry, and follow entries that are applicable and compatible with the built-in archive workflow. - Keep both fields separate from built-in steps, explicit user choices, resolved - paths, CLI checks, and command contracts. If context conflicts with one of those - controlling inputs, report the conflict and preserve the controlling value. If - guidance is inapplicable or conflicts with a controlling input, do not follow it - and explain why. Do not infer replacement paths, skipped prompts, or flags from - either field, and do not copy their text verbatim into specs, change artifacts, - or archive summaries unless the user separately asks for it. These are - prompt-level behavior contracts, not enforceable checks. + Keep both fields separate from built-in steps, explicit user choices, + resolved paths, CLI checks, and command contracts. If context conflicts with + one of those controlling inputs, report the conflict and preserve the + controlling value. If guidance is inapplicable or conflicts with a + controlling input, do not follow it and explain why. Do not infer replacement + paths, skipped prompts, or flags from either field, and do not copy their + text verbatim into specs, change artifacts, or archive summaries unless the + user separately asks for it. These are prompt-level behavior contracts, not + enforceable checks. 2. **Check artifact completion status** @@ -65,10 +86,13 @@ Archive a completed change in the experimental workflow. Parse the JSON to understand: - `schemaName`: The workflow being used - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context - - `artifacts`: List of artifacts with their status (`done`, `skipped`, or other) + - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path + and scope context + - `artifacts`: List of artifacts with their status (`done`, `skipped`, or + other) - **If any artifacts are neither `done` nor `skipped`** (skipped artifacts satisfy the requirement - the change declares skip_specs): + **If any artifacts are neither `done` nor `skipped`** (skipped artifacts + satisfy the requirement - the change declares skip_specs): - Display warning listing incomplete artifacts - Ask the user to confirm they want to proceed - Proceed if user confirms @@ -89,13 +113,16 @@ Archive a completed change in the experimental workflow. 4. **Assess delta spec sync state** Use `artifactPaths.specs.existingOutputPaths` from status JSON as the only - delta-spec source. If the `specs` entry is missing or - `existingOutputPaths` is empty, proceed without a sync prompt and do not infer - delta specs from other artifacts. + delta-spec source. If the `specs` entry is missing or `existingOutputPaths` + is empty, proceed without a sync prompt and do not infer delta specs from + other artifacts. **If delta specs exist:** - - Compare each delta spec with its corresponding main spec at `/openspec/specs//spec.md` (use the store-aware `planningHome.root` from step 2, not a hardcoded repo path) - - Determine what changes would be applied (adds, modifications, removals, renames) + - Compare each delta spec with its corresponding main spec at + `/openspec/specs//spec.md` (use the + store-aware `planningHome.root` from step 2, not a hardcoded repo path) + - Determine what changes would be applied (adds, modifications, removals, + renames) - Show a combined summary before prompting **Prompt options:** @@ -110,34 +137,58 @@ Archive a completed change in the experimental workflow. Before a selected sync writes any main spec, run `openspec instructions specs --change "" --json` once with the same - selected-root flags. Require a zero exit status and valid artifact-instruction - JSON. If the lookup fails or returns invalid JSON, report the error and stop - before writing any main spec or moving the change. A valid response with omitted - `rules` is the no-rules case. Apply returned `rules` only to the content and - form of main specs produced by this merge; do not use them as archive guidance, - change CLI behavior, or copy the rule text into any output file. - - Then run the `openspec-sync-specs` workflow inline (agent-driven intelligent merge) for change '', passing the delta spec analysis and the fetched specs-rule snapshot from above, and wait for it to finish. The inline sync must reuse that snapshot without fetching `specs` instructions again. Do not delegate it to a background task — step 5 would move `changeRoot` out from under a sync that is still reading it, leaving the change archived and the main specs never updated. If your agent can only run it by delegation, delegate synchronously and wait for the result. - - Then re-run the comparison from the top of this step against every capability that has a delta spec in `artifactPaths.specs.existingOutputPaths` — not only the ones the sync reports it touched. A successful sync leaves nothing left to apply, so each capability must now read as already synced: + selected-root flags. Require a zero exit status and valid + artifact-instruction JSON. If the lookup fails or returns invalid JSON, + report the error and stop before writing any main spec or moving the change. + A valid response with omitted `rules` is the no-rules case. Apply returned + `rules` only to the content and form of main specs produced by this merge; do + not use them as archive guidance, change CLI behavior, or copy the rule text + into any output file. + + Then run the `openspec-sync-specs` workflow inline (agent-driven intelligent + merge) for change '', passing the delta spec analysis and the fetched + specs-rule snapshot from above, and wait for it to finish. The inline sync + must reuse that snapshot without fetching `specs` instructions again. Do not + delegate it to a background task — step 5 would move `changeRoot` out from + under a sync that is still reading it, leaving the change archived and the + main specs never updated. If your agent can only run it by delegation, + delegate synchronously and wait for the result. + + Then re-run the comparison from the top of this step against every capability + that has a delta spec in `artifactPaths.specs.existingOutputPaths` — not only + the ones the sync reports it touched. A successful sync leaves nothing left + to apply, so each capability must now read as already synced: - ADDED requirements present - - MODIFIED requirements carrying the scenario and description changes named in the delta, with their other scenarios intact - - REMOVED requirements gone — and where this sync retired a capability (removed its last requirement, leaving `## Requirements` empty), its main spec deleted rather than left empty; a spec the sync deliberately kept and reported is also a match - - RENAMED requirements present under the new name and absent under the old one - - If the sync failed, or any capability does not match, report what differs and stop — do not archive. Nothing has moved and `changeRoot` is intact, so the user can fix the mismatch or re-run the sync and start the archive again. + - MODIFIED requirements carrying the scenario and description changes named + in the delta, with their other scenarios intact + - REMOVED requirements gone — and where this sync retired a capability + (removed its last requirement, leaving `## Requirements` empty), its main + spec deleted rather than left empty; a spec the sync deliberately kept and + reported is also a match + - RENAMED requirements present under the new name and absent under the old + one + + If the sync failed, or any capability does not match, report what differs and + stop — do not archive. Nothing has moved and `changeRoot` is intact, so the + user can fix the mismatch or re-run the sync and start the archive again. 5. **Perform the archive** - Create an `archive` directory under `planningHome.changesDir` if it doesn't exist: + Create an `archive` directory under `planningHome.changesDir` if it doesn't + exist: + ```bash mkdir -p "/archive" ``` - Generate the target name: use the change name as-is when it already starts with a `YYYY-MM-DD-` prefix; otherwise prepend the current date as `YYYY-MM-DD-`. Never stack a second date (same rule as `openspec archive`). + Generate the target name: use the change name as-is when it already starts + with a `YYYY-MM-DD-` prefix; otherwise prepend the current date as + `YYYY-MM-DD-`. Never stack a second date (same rule as + `openspec archive`). **Check if target already exists:** - - If yes: Fail with error, suggest renaming existing archive or using different date + - If yes: Fail with error, suggest renaming existing archive or using + different date - If no: Move `changeRoot` to the archive directory ```bash @@ -167,16 +218,25 @@ Archive a completed change in the experimental workflow. ``` **Guardrails** + - Announce the selected change; prompt for selection when it is ambiguous - Use artifact graph (openspec status --json) for completion checking - Don't block archive on warnings - just inform and confirm - Preserve .openspec.yaml when moving to archive (it moves with the directory) - Show clear summary of what happened -- If sync is requested, run the `openspec-sync-specs` workflow inline (agent-driven) -- Never archive while a spec sync is still in flight — run the sync inline and verify the main specs before moving `changeRoot` -- If delta specs exist, always run the sync assessment and show the combined summary before prompting -- Apply relevant runtime context and report conflicts; operation guidance remains advisory -- Consider every guidance entry and explain any inapplicable or conflicting advice -- Existing CLI checks, resolved paths, prompts, and command contracts are unchanged -- Artifact rules constrain only the specs being written and are never operation guidance -- Never copy runtime context, operation guidance, or artifact-rule text verbatim into output files +- If sync is requested, run the `openspec-sync-specs` workflow inline + (agent-driven) +- Never archive while a spec sync is still in flight — run the sync inline and + verify the main specs before moving `changeRoot` +- If delta specs exist, always run the sync assessment and show the combined + summary before prompting +- Apply relevant runtime context and report conflicts; operation guidance + remains advisory +- Consider every guidance entry and explain any inapplicable or conflicting + advice +- Existing CLI checks, resolved paths, prompts, and command contracts are + unchanged +- Artifact rules constrain only the specs being written and are never operation + guidance +- Never copy runtime context, operation guidance, or artifact-rule text verbatim + into output files diff --git a/.gemini/skills/openspec-explore/SKILL.md b/.gemini/skills/openspec-explore/SKILL.md index 74cb35f..47f1b05 100644 --- a/.gemini/skills/openspec-explore/SKILL.md +++ b/.gemini/skills/openspec-explore/SKILL.md @@ -10,20 +10,48 @@ metadata: generatedBy: "1.12.0" --- -Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes. - -**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, investigate the codebase, and run read-only commands or tools without confirmation, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create or update OpenSpec change artifacts (proposals, designs, specs) within a confirmed scope—that's capturing thinking, not implementing. Answering design or clarifying questions is never consent to write. Before the first write-capable action, name the artifacts or files you would change and what you would do, ask a direct yes/no question, and wait for the user's confirmation in a separate message. Confirmation covers only the scope you described; ask again before expanding it. For a new change, scaffold it first as described below. - -**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore. - -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. +Enter explore mode. Think deeply. Visualize freely. Follow the conversation +wherever it goes. + +**IMPORTANT: Explore mode is for thinking, not implementing.** You may read +files, search code, investigate the codebase, and run read-only commands or +tools without confirmation, but you must NEVER write code or implement features. +If the user asks you to implement something, remind them to exit explore mode +first and create a change proposal. You MAY create or update OpenSpec change +artifacts (proposals, designs, specs) within a confirmed scope—that's capturing +thinking, not implementing. Answering design or clarifying questions is never +consent to write. Before the first write-capable action, name the artifacts or +files you would change and what you would do, ask a direct yes/no question, and +wait for the user's confirmation in a separate message. Confirmation covers only +the scope you described; ask again before expanding it. For a new change, +scaffold it first as described below. + +**This is a stance, not a workflow.** There are no fixed steps, no required +sequence, no mandatory outputs. You're a thinking partner helping the user +explore. + +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. --- ## The Stance -- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script -- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions. +- **Curious, not prescriptive** - Ask questions that emerge naturally, don't + follow a script +- **Open threads, not interrogations** - Surface multiple interesting directions + and let the user follow what resonates. Don't funnel them through a single + path of questions. - **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking - **Adaptive** - Follow interesting threads, pivot when new information emerges - **Patient** - Don't rush to conclusions, let the shape of the problem emerge @@ -33,16 +61,37 @@ Enter explore mode. Think deeply. Visualize freely. Follow the conversation wher ## Planning a Change -When the user is planning a change, guide them toward shared understanding with focused discovery questions. For open-ended discussion, follow the conversation without imposing an interview or a required output. - -Before asking a factual question, follow the context discovery below and inspect relevant OpenSpec artifacts, source, tests, docs, and configuration. Do not ask the user to repeat facts you can verify. Summarize relevant findings without reproducing private context or rules. If evidence is missing, conflicting, or inaccessible, state that limitation and ask only for the clarification needed to proceed. - -- **Follow dependencies** - Resolve the next blocking decision before its dependent details. For example, clarify the user's outcome and scope before choosing an API or data model. Revisit downstream assumptions when an earlier answer changes. Skip branches that do not matter to this goal. -- **Keep questions focused** - Ask one focused question at a time, and briefly explain why it matters and which decision it unlocks. Batch questions only if the user asks for a batch; keep them small and group related decisions. -- **Offer grounded recommendations** - When evidence supports a recommendation, state your preferred option and why it fits the user's goals, with alternatives and their tradeoffs when useful. Do not invent intent, priorities, or external constraints: ask the user when only they can answer. Avoid a fixed question format. -- **Keep a conversational record** - Track decisions in the conversation, not in files. Separate confirmed decisions from proposed defaults and unresolved questions. Silence is not acceptance. Accepting an answer or a batch of recommendations is not permission to write. Keep file-write confirmation separate from discovery questions and follow the guardrails below. - -Stop asking when the user has enough clarity. Let them pause, pivot, or defer a decision; do not exhaust every branch or force a proposal. +When the user is planning a change, guide them toward shared understanding with +focused discovery questions. For open-ended discussion, follow the conversation +without imposing an interview or a required output. + +Before asking a factual question, follow the context discovery below and inspect +relevant OpenSpec artifacts, source, tests, docs, and configuration. Do not ask +the user to repeat facts you can verify. Summarize relevant findings without +reproducing private context or rules. If evidence is missing, conflicting, or +inaccessible, state that limitation and ask only for the clarification needed to +proceed. + +- **Follow dependencies** - Resolve the next blocking decision before its + dependent details. For example, clarify the user's outcome and scope before + choosing an API or data model. Revisit downstream assumptions when an earlier + answer changes. Skip branches that do not matter to this goal. +- **Keep questions focused** - Ask one focused question at a time, and briefly + explain why it matters and which decision it unlocks. Batch questions only if + the user asks for a batch; keep them small and group related decisions. +- **Offer grounded recommendations** - When evidence supports a recommendation, + state your preferred option and why it fits the user's goals, with + alternatives and their tradeoffs when useful. Do not invent intent, + priorities, or external constraints: ask the user when only they can answer. + Avoid a fixed question format. +- **Keep a conversational record** - Track decisions in the conversation, not in + files. Separate confirmed decisions from proposed defaults and unresolved + questions. Silence is not acceptance. Accepting an answer or a batch of + recommendations is not permission to write. Keep file-write confirmation + separate from discovery questions and follow the guardrails below. + +Stop asking when the user has enough clarity. Let them pause, pivot, or defer a +decision; do not exhaust every branch or force a proposal. For example, after inspecting the relevant code: @@ -60,25 +109,29 @@ adding a service to operate; shared state would need a separate sync design. Depending on what the user brings, you might: **Explore the problem space** + - Ask clarifying questions that emerge from what they said - Challenge assumptions - Reframe the problem - Find analogies **Investigate the codebase** + - Map existing architecture relevant to the discussion - Find integration points - Identify patterns already in use - Surface hidden complexity **Compare options** + - Brainstorm multiple approaches - Build comparison tables - Sketch tradeoffs - Recommend a path (if asked) **Visualize** -``` + +```text +------------------------------------------+ | Use ASCII diagrams liberally | +------------------------------------------+ @@ -95,10 +148,13 @@ Depending on what the user brings, you might: +------------------------------------------+ ``` -**Draw with plain ASCII only** — borders `+` `-` `|`, arrows `-->` `<--` `^` `v`, markers `*` `x`. -Unicode diagram glyphs can render at different widths across terminals, fonts, and locales, so padded boxes and aligned tables can drift. Keep every diagram character ASCII. +**Draw with plain ASCII only** — borders `+` `-` `|`, arrows `-->` `<--` `^` +`v`, markers `*` `x`. Unicode diagram glyphs can render at different widths +across terminals, fonts, and locales, so padded boxes and aligned tables can +drift. Keep every diagram character ASCII. **Surface risks and unknowns** + - Identify what could go wrong - Find gaps in understanding - Suggest spikes or investigations @@ -112,20 +168,28 @@ You have full context of the OpenSpec system. Use it naturally, don't force it. ### Check for context At the start, quickly check what exists: + ```bash openspec list --json ``` This tells you: + - If there are active changes - Their names, schemas, and status - What the user might be working on -Then read the project's own context from the resolved root - `/openspec/config.yaml` (or `config.yml`). Use the `root.path` returned above, and skip this if neither file exists: +Then read the project's own context from the resolved root - +`/openspec/config.yaml` (or `config.yml`). Use the `root.path` +returned above, and skip this if neither file exists: + - `context`: project background - tech stack, conventions, constraints -- `rules`: keyed by artifact id - the entries for an artifact apply only when you write that artifact +- `rules`: keyed by artifact id - the entries for an artifact apply only when + you write that artifact -Ground your thinking in these. They are constraints for you to follow, not content to reproduce: do NOT copy them into the conversation or into any artifact you create. +Ground your thinking in these. They are constraints for you to follow, not +content to reproduce: do NOT copy them into the conversation or into any +artifact you create. ### When no change exists @@ -134,14 +198,57 @@ Think freely. When insights crystallize, you might offer: - "This feels solid enough to start a change. Want me to create a proposal?" - Or keep exploring - no pressure to formalize -If the user asks you to capture the exploration as a new change, transition seamlessly into the requested capture: - -1. Run `openspec new change ""` (with `--store ` when applicable) before creating any artifacts. Never create a new change directory under `openspec/changes/` by hand; the CLI scaffold creates required metadata such as `.openspec.yaml`. Keep the selected `--store ` on every applicable follow-up `status` and `instructions` command. -2. Run `openspec status --change "" --json` (append the confirmed `--store ""` only for a registered standalone store), then process the requested artifacts in dependency order. For each requested artifact that is `ready`, run `openspec instructions "" --change "" --json` (append the confirmed `--store ""` only for a registered standalone store). Before creating a requested artifact, evaluate any condition in its own `instruction` against the explored change; record a deliberate skip instead when the condition does not apply. If a requested artifact is blocked by a direct prerequisite the user did not request, run `openspec instructions "" --change "" --json` (append the confirmed `--store ""` only for a registered standalone store) for that prerequisite whether it is `ready` or `blocked`. If its own `instruction` states a condition, evaluate that condition against the explored change and record a deliberate skip only when the condition does not apply. If the condition applies, or the prerequisite is not conditional, treat it as a normal prerequisite and ask before expanding the capture. Do not create an unrequested prerequisite unless the user approves. -3. Follow the returned `template` and `instruction` fields. Read completed dependency files listed in `dependencies`, and apply `context` and `rules` as constraints without copying them into the artifact. If the instruction delegates creation to a specific skill or command, invoke it; otherwise write the artifact to `resolvedOutputPath`, using the instruction to choose a concrete path when it is a glob. Verify that the selected concrete output exists. -4. After creating each artifact, re-run `openspec status --change "" --json` (append the confirmed `--store ""` only for a registered standalone store) and continue until every requested artifact is `done`, `skipped`, or was deliberately skipped because its own `instruction` stated a condition that did not apply. Tell the user about a deliberate conditional skip, remember it, and do not reconsider it. Dependencies are enablers, not gates: if a requested artifact is still `blocked` only because you deliberately skipped a conditional prerequisite, run `openspec instructions "" --change "" --json` (append the confirmed `--store ""` only for a registered standalone store) despite the blocked status, then create it using step 3 only when those recorded conditional skips are its sole missing dependencies. If a requested artifact is blocked by a prerequisite the user did not ask to capture and cannot be conditionally skipped, explain that dependency and ask before expanding the capture. - -Capture the artifact(s) the user requested without asking them to invoke another workflow command. If they asked only to start a change, stop after scaffolding and show its status. +If the user asks you to capture the exploration as a new change, transition +seamlessly into the requested capture: + +1. Run `openspec new change ""` (with `--store ` when applicable) + before creating any artifacts. Never create a new change directory under + `openspec/changes/` by hand; the CLI scaffold creates required metadata such + as `.openspec.yaml`. Keep the selected `--store ` on every applicable + follow-up `status` and `instructions` command. +2. Run `openspec status --change "" --json` (append the confirmed + `--store ""` only for a registered standalone store), then process the + requested artifacts in dependency order. For each requested artifact that is + `ready`, run `openspec instructions "" --change "" --json` + (append the confirmed `--store ""` only for a registered standalone + store). Before creating a requested artifact, evaluate any condition in its + own `instruction` against the explored change; record a deliberate skip + instead when the condition does not apply. If a requested artifact is blocked + by a direct prerequisite the user did not request, run + `openspec instructions "" --change "" --json` (append + the confirmed `--store ""` only for a registered standalone store) for + that prerequisite whether it is `ready` or `blocked`. If its own + `instruction` states a condition, evaluate that condition against the + explored change and record a deliberate skip only when the condition does not + apply. If the condition applies, or the prerequisite is not conditional, + treat it as a normal prerequisite and ask before expanding the capture. Do + not create an unrequested prerequisite unless the user approves. +3. Follow the returned `template` and `instruction` fields. Read completed + dependency files listed in `dependencies`, and apply `context` and `rules` as + constraints without copying them into the artifact. If the instruction + delegates creation to a specific skill or command, invoke it; otherwise write + the artifact to `resolvedOutputPath`, using the instruction to choose a + concrete path when it is a glob. Verify that the selected concrete output + exists. +4. After creating each artifact, re-run + `openspec status --change "" --json` (append the confirmed + `--store ""` only for a registered standalone store) and continue until + every requested artifact is `done`, `skipped`, or was deliberately skipped + because its own `instruction` stated a condition that did not apply. Tell the + user about a deliberate conditional skip, remember it, and do not reconsider + it. Dependencies are enablers, not gates: if a requested artifact is still + `blocked` only because you deliberately skipped a conditional prerequisite, + run `openspec instructions "" --change "" --json` (append + the confirmed `--store ""` only for a registered standalone store) + despite the blocked status, then create it using step 3 only when those + recorded conditional skips are its sole missing dependencies. If a requested + artifact is blocked by a prerequisite the user did not ask to capture and + cannot be conditionally skipped, explain that dependency and ask before + expanding the capture. + +Capture the artifact(s) the user requested without asking them to invoke another +workflow command. If they asked only to start a change, stop after scaffolding +and show its status. ### When a change exists @@ -149,25 +256,30 @@ If the user mentions a change or you detect one is relevant: 1. **Resolve and read existing artifacts for context** - Run `openspec status --change "" --json`. - - Use `changeRoot`, `artifactPaths`, and `actionContext` from the status JSON. + - Use `changeRoot`, `artifactPaths`, and `actionContext` from the status + JSON. - Read existing files from `artifactPaths..existingOutputPaths`. 2. **Reference them naturally in conversation** - - "Your design mentions using Redis, but we just realized SQLite fits better..." - - "The proposal scopes this to premium users, but we're now thinking everyone..." + - "Your design mentions using Redis, but we just realized SQLite fits + better..." + - "The proposal scopes this to premium users, but we're now thinking + everyone..." 3. **Offer to capture when decisions are made** - `` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve an existing capability's full path and follow the project's established organization for new capabilities. + `` is the spec directory relative to `specs/` (for example, + `user-auth` or `identity/user-auth`). Preserve an existing capability's full + path and follow the project's established organization for new capabilities. - | Insight Type | Where to Capture | - |----------------------------|-------------------------------------| - | New requirement discovered | `specs//spec.md` | - | Requirement changed | `specs//spec.md` | - | Design decision made | `design.md` | - | Scope changed | `proposal.md` | - | New work identified | `tasks.md` | - | Assumption invalidated | Relevant artifact | + | Insight Type | Where to Capture | + |----------------------------|-------------------------------------| + | New requirement discovered | `specs//spec.md` | + | Requirement changed | `specs//spec.md` | + | Design decision made | `design.md` | + | Scope changed | `proposal.md` | + | New work identified | `tasks.md` | + | Assumption invalidated | Relevant artifact | Example offers: - "That's a design decision. Capture it in design.md?" @@ -192,7 +304,8 @@ If the user mentions a change or you detect one is relevant: ## Handling Different Entry Points **User brings a vague idea:** -``` + +```text User: I'm thinking about adding real-time collaboration You: Real-time collab is a big space. Let me think about this... @@ -215,7 +328,8 @@ You: Real-time collab is a big space. Let me think about this... ``` **User brings a specific problem:** -``` + +```text User: The auth system is a mess You: [reads codebase] @@ -246,7 +360,8 @@ You: [reads codebase] ``` **User is stuck mid-implementation:** -``` + +```text User: /opsx:explore add-auth-system The OAuth integration is more complex than expected @@ -263,7 +378,8 @@ You: [reads change artifacts] ``` **User wants to compare options:** -``` + +```text User: Should we use Postgres or SQLite? You: Generic answer is boring. What's the context? @@ -304,7 +420,7 @@ There's no required ending. Discovery might: When it feels like things are crystallizing, you might summarize: -``` +```text ## What We Figured Out **The problem**: [crystallized understanding] @@ -324,12 +440,25 @@ But this summary is optional. Sometimes the thinking IS the value. ## Guardrails -- **Don't implement** - Never write code or implement features. Workflow configuration counts too: creating or editing schemas, templates, or `openspec/config.yaml` is a change, not thinking. Creating or updating OpenSpec change artifacts within the confirmed scope is fine, writing anything else is not. +- **Don't implement** - Never write code or implement features. Workflow + configuration counts too: creating or editing schemas, templates, or + `openspec/config.yaml` is a change, not thinking. Creating or updating + OpenSpec change artifacts within the confirmed scope is fine, writing anything + else is not. - **Don't fake understanding** - If something is unclear, dig deeper - **Don't rush** - Discovery is thinking time, not task time - **Don't force structure** - Let patterns emerge naturally -- **Don't auto-capture** - Offer to save insights, don't just do it. Read-only commands and tools need no confirmation. Before the first write-capable action—including `openspec new change` or another command that writes files—name the artifacts or files and proposed changes, ask a direct yes/no question, and wait for explicit confirmation in a separate user message. That confirmation covers only the described scope; ask again before expanding it. Answers to design or clarifying questions are never consent to write. -- **Don't manually scaffold changes** - Never create a new change directory under `openspec/changes/` by hand. Always use `openspec new change ""` (with `--store ` when applicable) so required metadata such as `.openspec.yaml` is created before writing artifacts. +- **Don't auto-capture** - Offer to save insights, don't just do it. Read-only + commands and tools need no confirmation. Before the first write-capable + action—including `openspec new change` or another command that writes + files—name the artifacts or files and proposed changes, ask a direct yes/no + question, and wait for explicit confirmation in a separate user message. That + confirmation covers only the described scope; ask again before expanding it. + Answers to design or clarifying questions are never consent to write. +- **Don't manually scaffold changes** - Never create a new change directory + under `openspec/changes/` by hand. Always use `openspec new change ""` + (with `--store ` when applicable) so required metadata such as + `.openspec.yaml` is created before writing artifacts. - **Do visualize** - A good diagram is worth many paragraphs - **Do explore the codebase** - Ground discussions in reality - **Do question assumptions** - Including the user's and your own diff --git a/.gemini/skills/openspec-propose/SKILL.md b/.gemini/skills/openspec-propose/SKILL.md index 6d45c32..9cba6a2 100644 --- a/.gemini/skills/openspec-propose/SKILL.md +++ b/.gemini/skills/openspec-propose/SKILL.md @@ -12,115 +12,216 @@ metadata: Propose a new change - create the change and generate all artifacts in one step. -**Planning boundary**: This workflow creates planning artifacts only. The user request that selected or triggered this workflow authorizes planning only, even if it asks to build or fix something. Do not edit project code. After the planning artifacts are complete, stop. Do not start implementation in the same response, even if the initial request asks for it. Wait for a new user request after the artifacts are presented; then start the apply workflow. +**Planning boundary**: This workflow creates planning artifacts only. The user +request that selected or triggered this workflow authorizes planning only, even +if it asks to build or fix something. Do not edit project code. After the +planning artifacts are complete, stop. Do not start implementation in the same +response, even if the initial request asks for it. Wait for a new user request +after the artifacts are presented; then start the apply workflow. + +I'll create a change with the artifacts your schema defines. With the default +spec-driven schema that is: -I'll create a change with the artifacts your schema defines. With the default spec-driven schema that is: - proposal.md (what & why) -- `specs//spec.md` (what the system must do - a delta, not the main spec) +- `specs//spec.md` (what the system must do - a delta, not the + main spec) - design.md (how) - tasks.md (implementation steps) -`` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve an existing capability's full path and follow the project's established organization for new capabilities. +`` is the spec directory relative to `specs/` (for example, +`user-auth` or `identity/user-auth`). Preserve an existing capability's full +path and follow the project's established organization for new capabilities. -When the user is ready to implement, they must start the apply workflow explicitly. +When the user is ready to implement, they must start the apply workflow +explicitly. --- -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -**Input**: The user's request should include a change name (kebab-case) OR a description of what they want to build. +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +**Input**: The user's request should include a change name (kebab-case) OR a +description of what they want to build. **Steps** 1. **Understand the request and clarify material ambiguity** If no clear input is provided, ask the user (open-ended, no preset options): - > "What change do you want to work on? Describe what you want to build or fix." + > "What change do you want to work on? Describe what you want to build or + > fix." - From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`). + From their description, derive a kebab-case name (e.g., "add user + authentication" → `add-user-auth`). - **IMPORTANT**: Do NOT proceed without understanding what the user wants to build. + **IMPORTANT**: Do NOT proceed without understanding what the user wants to + build. - If the request contains ambiguity that would materially affect scope, externally observable behavior, compatibility, or acceptance criteria, ask the user before creating the change. For minor details, make a reasonable assumption and record it in the planning artifacts. + If the request contains ambiguity that would materially affect scope, + externally observable behavior, compatibility, or acceptance criteria, ask + the user before creating the change. For minor details, make a reasonable + assumption and record it in the planning artifacts. 2. **Determine the workflow schema** - Use the configured default schema unless the user explicitly requests a different workflow. + Use the configured default schema unless the user explicitly requests a + different workflow. **Use a different schema only if the user:** - - Explicitly requests a specific schema by name → use `--schema ` - - Asks to "show workflows" or asks "what workflows" exist → resolve the authoritative root by running `openspec context --json` from the current working directory. If the user explicitly selected a registered store, use `openspec context --json --store ""`. Then run `openspec schemas --json` with its working directory set to the returned `root.path` and let them choose. This preserves roots selected by a local `store:` pointer or the global `defaultStore`; when a registered store was explicitly selected, append `--store ""` to `openspec schemas --json` as well. If context reports only `no_openspec_root`, run `openspec schemas --json` from the current working directory instead. Do not use this fallback for invalid or unavailable stores. + - Explicitly requests a specific schema by name → use + `--schema ` + - Asks to "show workflows" or asks "what workflows" exist → resolve the + authoritative root by running `openspec context --json` from the current + working directory. If the user explicitly selected a registered store, use + `openspec context --json --store ""`. Then run + `openspec schemas --json` with its working directory set to the returned + `root.path` and let them choose. This preserves roots selected by a local + `store:` pointer or the global `defaultStore`; when a registered store was + explicitly selected, append `--store ""` to + `openspec schemas --json` as well. If context reports only + `no_openspec_root`, run `openspec schemas --json` from the current working + directory instead. Do not use this fallback for invalid or unavailable + stores. Otherwise, omit `--schema` to preserve the configured default. 3. **Create the change directory** - Choose one schema form below. If a registered store is selected, append `--store ""` to that command and each later OpenSpec command shown below that accepts `--store`. + Choose one schema form below. If a registered store is selected, append + `--store ""` to that command and each later OpenSpec command shown + below that accepts `--store`. Using the configured default: + ```bash openspec new change "" ``` Using an explicitly requested schema: + ```bash openspec new change "" --schema "" ``` - This creates a scaffolded change in the planning home resolved by the CLI with `.openspec.yaml`. + + This creates a scaffolded change in the planning home resolved by the CLI + with `.openspec.yaml`. 4. **Get the artifact build order** + ```bash openspec status --change "" --json ``` + Parse the JSON to get: - - `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`) - - `artifacts`: list of all artifacts, each with its `status` and its `requires` edges (the artifact IDs it directly depends on) - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context. Use these instead of assuming repo-local paths. + - `applyRequires`: array of artifact IDs needed before implementation (e.g., + `["tasks"]`) + - `artifacts`: list of all artifacts, each with its `status` and its + `requires` edges (the artifact IDs it directly depends on) + - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path + and scope context. Use these instead of assuming repo-local paths. 5. **Create every artifact in the required set** Use a todo list to track progress through the artifacts. - Loop through artifacts in dependency order (artifacts with no pending dependencies first): + Loop through artifacts in dependency order (artifacts with no pending + dependencies first): a. **For each artifact that is `ready` (dependencies satisfied)**: - Get instructions: + ```bash openspec instructions --change "" --json ``` + - The instructions JSON includes: - - `context`: Project background (constraints for you - do NOT include in output) - - `rules`: Artifact-specific rules (constraints for you - do NOT include in output) + - `context`: Project background (constraints for you - do NOT include in + output) + - `rules`: Artifact-specific rules (constraints for you - do NOT include + in output) - `template`: The structure to use for your output file - `instruction`: Schema-specific guidance for this artifact type - - `skipped`/`warning`: present when the change declares skip_specs and this artifact must NOT be created - stop and pick another artifact + - `skipped`/`warning`: present when the change declares skip_specs and + this artifact must NOT be created - stop and pick another artifact - `resolvedOutputPath`: Resolved path or pattern to write the artifact - `dependencies`: Completed artifacts to read for context - - Read any completed dependency files for context - always re-read them from disk, even if you saw them earlier in the conversation (the user may have edited them) - - **Inspect the relevant project before drafting**: Read `context` and `rules` first, then inspect relevant implementation, nearby tests, configuration, and documentation outside `openspec/`. Keep inspection read-only and proportional to the change; reuse findings for later artifacts and inspect more only as needed. - - Identify the target project from the request and project context; the planning home may be separate from the code. If the target is unclear, ask. For greenfield or non-code changes, inspect the available structure and relevant documents. If source is unavailable, state the limitation and ask when it materially affects the plan. - - Ground scope, approach, and tasks in what you find. Distinguish observed behavior from assumptions and proposed additions; surface conflicts with existing specs instead of silently deciding which is correct. - - Do this discovery now, rather than leaving generic "explore the codebase" or "make a plan" tasks for implementation. Keep any necessary follow-up investigation specific to an unresolved question. - - If the `instruction` field delegates creation to a specific skill or command, invoke it to produce the artifact instead of writing the file yourself, then verify the artifact file exists at `resolvedOutputPath` - - Otherwise create the artifact file using `template` as the structure and write it to `resolvedOutputPath`. If `resolvedOutputPath` is a glob, follow `instruction` to choose the concrete file path - - Apply `context` and `rules` as constraints - but do NOT copy them into the file + - Read any completed dependency files for context - always re-read them + from disk, even if you saw them earlier in the conversation (the user + may have edited them) + - **Inspect the relevant project before drafting**: Read `context` and + `rules` first, then inspect relevant implementation, nearby tests, + configuration, and documentation outside `openspec/`. Keep inspection + read-only and proportional to the change; reuse findings for later + artifacts and inspect more only as needed. + - Identify the target project from the request and project context; the + planning home may be separate from the code. If the target is unclear, + ask. For greenfield or non-code changes, inspect the available + structure and relevant documents. If source is unavailable, state the + limitation and ask when it materially affects the plan. + - Ground scope, approach, and tasks in what you find. Distinguish + observed behavior from assumptions and proposed additions; surface + conflicts with existing specs instead of silently deciding which is + correct. + - Do this discovery now, rather than leaving generic "explore the + codebase" or "make a plan" tasks for implementation. Keep any + necessary follow-up investigation specific to an unresolved question. + - If the `instruction` field delegates creation to a specific skill or + command, invoke it to produce the artifact instead of writing the file + yourself, then verify the artifact file exists at `resolvedOutputPath` + - Otherwise create the artifact file using `template` as the structure and + write it to `resolvedOutputPath`. If `resolvedOutputPath` is a glob, + follow `instruction` to choose the concrete file path + - Apply `context` and `rules` as constraints - but do NOT copy them into + the file - Show brief progress: "Created " - b. **Continue until every artifact in the required set exists (not just `apply.requires`)** - - After creating each artifact, re-run `openspec status --change "" --json` - - The required set is `applyRequires` plus every artifact reachable from those by following the `requires` edges in `status --json` - walk them transitively (spec-driven closes over proposal, specs, design, tasks). Leave artifacts outside that set alone - - `status` is file-existence only, so an `applyRequires` artifact reading `done` does NOT mean its dependencies exist - writing `tasks.md` early marks `tasks` done while `specs` was never written. Use each artifact's `requires` edges, not its `status`, to build the required set: a `done` artifact still lists what it depends on - - An artifact already reading `status: "skipped"` is satisfied: the change declares `skip_specs` in `.openspec.yaml`, so its files must NOT exist. Never try to create one - - Create every artifact in the required set that is missing, then re-check - creating one can unblock others - - Skip one only when `status` already reports it `skipped`, or when its own `instruction` says it is conditional: run `openspec instructions --change "" --json` and skip only if its `instruction` field marks it optional (e.g. "create only if..."). Spec-driven's `design.md` qualifies; `specs` qualifies only via the `skipped` status above, never by your own judgment. Tell the user, and do not reconsider it - - Dependencies are enablers, not gates: if a required artifact is still `blocked` only because you skipped a conditional dependency, write it anyway - - Stop when every artifact in the required set is `done`, `skipped`, or was deliberately skipped + b. **Continue until every artifact in the required set exists (not just + `apply.requires`)** + - After creating each artifact, re-run + `openspec status --change "" --json` + - The required set is `applyRequires` plus every artifact reachable from + those by following the `requires` edges in `status --json` - walk them + transitively (spec-driven closes over proposal, specs, design, tasks). + Leave artifacts outside that set alone + - `status` is file-existence only, so an `applyRequires` artifact reading + `done` does NOT mean its dependencies exist - writing `tasks.md` early + marks `tasks` done while `specs` was never written. Use each artifact's + `requires` edges, not its `status`, to build the required set: a `done` + artifact still lists what it depends on + - An artifact already reading `status: "skipped"` is satisfied: the change + declares `skip_specs` in `.openspec.yaml`, so its files must NOT exist. + Never try to create one + - Create every artifact in the required set that is missing, then + re-check - creating one can unblock others + - Skip one only when `status` already reports it `skipped`, or when its + own `instruction` says it is conditional: run + `openspec instructions --change "" --json` and skip + only if its `instruction` field marks it optional (e.g. "create only + if..."). Spec-driven's `design.md` qualifies; `specs` qualifies only via + the `skipped` status above, never by your own judgment. Tell the user, + and do not reconsider it + - Dependencies are enablers, not gates: if a required artifact is still + `blocked` only because you skipped a conditional dependency, write it + anyway + - Stop when every artifact in the required set is `done`, `skipped`, or + was deliberately skipped c. **If an artifact requires user input** (unclear context): - Ask the user to clarify - Then continue with creation 6. **Show final status** + ```bash openspec status --change "" ``` @@ -128,26 +229,44 @@ When the user is ready to implement, they must start the apply workflow explicit **Output** After completing all artifacts, summarize: + - Change name and location -- List of artifacts created with brief descriptions, plus any conditional artifact you skipped and why +- List of artifacts created with brief descriptions, plus any conditional + artifact you skipped and why - What's ready: "All artifacts needed for implementation are ready." -- Prompt: "The artifacts are ready for review. When you are ready, run `/opsx:apply` or ask me to apply this change." +- Prompt: "The artifacts are ready for review. When you are ready, run + `/opsx:apply` or ask me to apply this change." **Artifact Creation Guidelines** -- Follow the `instruction` field from `openspec instructions` for each artifact type - it is the authoritative guidance, even for familiar artifact names -- If the `instruction` field directs you to use a specific skill or command to create the artifact, invoke it instead of writing the artifact directly +- Follow the `instruction` field from `openspec instructions` for each artifact + type - it is the authoritative guidance, even for familiar artifact names +- If the `instruction` field directs you to use a specific skill or command to + create the artifact, invoke it instead of writing the artifact directly - The schema defines what each artifact should contain - follow it - Read dependency artifacts for context before creating new ones - Use `template` as the structure for your output file - fill in its sections -- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file - - Do NOT copy ``, ``, `` blocks into the artifact +- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for + the file + - Do NOT copy ``, ``, `` blocks into the + artifact - These guide what you write, but should never appear in the output **Guardrails** -- The request that invoked this workflow authorizes planning only. Any implementation or apply instruction in that request does not carry forward. Do NOT implement the change, start the apply workflow, or edit project code during this workflow. After presenting the artifacts, stop and wait for a new user request to start the apply workflow -- Create every artifact the apply phase transitively depends on, not just the ids listed in `apply.requires` -- Always read dependency artifacts before creating a new one - re-read from disk, not from conversation memory (files may have changed since you last saw them) -- Ask about ambiguities that would materially change scope, externally observable behavior, compatibility, or acceptance criteria; for minor details, make reasonable assumptions and record them -- If a change with that name already exists, ask if user wants to continue it or create a new one + +- The request that invoked this workflow authorizes planning only. Any + implementation or apply instruction in that request does not carry forward. Do + NOT implement the change, start the apply workflow, or edit project code + during this workflow. After presenting the artifacts, stop and wait for a new + user request to start the apply workflow +- Create every artifact the apply phase transitively depends on, not just the + ids listed in `apply.requires` +- Always read dependency artifacts before creating a new one - re-read from + disk, not from conversation memory (files may have changed since you last saw + them) +- Ask about ambiguities that would materially change scope, externally + observable behavior, compatibility, or acceptance criteria; for minor details, + make reasonable assumptions and record them +- If a change with that name already exists, ask if user wants to continue it or + create a new one - Verify each artifact file exists after writing before proceeding to next diff --git a/.gemini/skills/openspec-sync-specs/SKILL.md b/.gemini/skills/openspec-sync-specs/SKILL.md index b939242..a919efb 100644 --- a/.gemini/skills/openspec-sync-specs/SKILL.md +++ b/.gemini/skills/openspec-sync-specs/SKILL.md @@ -12,13 +12,30 @@ metadata: Sync delta specs from a change to main specs. -This is an **agent-driven** operation - you will read delta specs and directly edit main specs to apply the changes. This allows intelligent merging (e.g., adding a scenario without copying the entire requirement). - -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -`` is the spec directory relative to `specs/` (for example, `user-auth` or `identity/user-auth`). Preserve the full path from each delta spec when resolving its main spec. - -**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. +This is an **agent-driven** operation - you will read delta specs and directly +edit main specs to apply the changes. This allows intelligent merging (e.g., +adding a scenario without copying the entire requirement). + +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +`` is the spec directory relative to `specs/` (for example, +`user-auth` or `identity/user-auth`). Preserve the full path from each delta +spec when resolving its main spec. + +**Input**: Optionally specify a change name. If omitted, check if it can be +inferred from conversation context. If vague or ambiguous you MUST prompt for +available changes. **Steps** @@ -27,20 +44,27 @@ This is an **agent-driven** operation - you will read delta specs and directly e If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes and ask + the user to select one - When prompting, show changes that have delta specs (under `specs/` directory). + When prompting, show changes that have delta specs (under `specs/` + directory). - Always announce: "Using change: " and how to override (e.g., `/opsx:sync `). + Always announce: "Using change: " and how to override (e.g., + `/opsx:sync `). 2. **Resolve change context** Run: + ```bash openspec status --change "" --json ``` - The JSON includes `planningHome.root`. Main specs live under `/openspec/specs/` — use that (store-aware) root for every main-spec path below, not a hardcoded repo path. When a store is selected it points at the store, not the current repository. + The JSON includes `planningHome.root`. Main specs live under + `/openspec/specs/` — use that (store-aware) root for every + main-spec path below, not a hardcoded repo path. When a store is selected it + points at the store, not the current repository. 3. **Find delta specs** @@ -80,29 +104,34 @@ This is an **agent-driven** operation - you will read delta specs and directly e fetch the same instructions again. - Otherwise run that command once now with the same selected-root flags. - If the direct lookup exits non-zero or returns invalid artifact-instruction - JSON, report the error and stop before writing any main spec. Do not treat the - failure as an absent rule set. - - A valid response with omitted `rules` means no artifact rules are configured - and the existing semantic merge continues. + JSON, report the error and stop before writing any main spec. Do not treat + the failure as an absent rule set. + - A valid response with omitted `rules` means no artifact rules are + configured and the existing semantic merge continues. - Apply returned `rules` only to the content and form of the main specs produced - by this merge. Artifact rules are not operation guidance and cannot change - selected roots, delta paths, CLI checks, or workflow steps. Use their text as - constraints without copying it verbatim into a main spec or summary. + Apply returned `rules` only to the content and form of the main specs + produced by this merge. Artifact rules are not operation guidance and cannot + change selected roots, delta paths, CLI checks, or workflow steps. Use their + text as constraints without copying it verbatim into a main spec or summary. - For each capability delta spec path selected in step 3 — the full `existingOutputPaths` list, or the narrowed subset when a caller supplied one (these may belong to a selected store, not the repo): + For each capability delta spec path selected in step 3 — the full + `existingOutputPaths` list, or the narrowed subset when a caller supplied one + (these may belong to a selected store, not the repo): a. **Read the delta spec** to understand the intended changes - b. **Read the main spec** at `/openspec/specs//spec.md` (may not exist yet) + b. **Read the main spec** at + `/openspec/specs//spec.md` (may not exist + yet) c. **Apply changes intelligently**: - **ADDED Requirements:** + **ADDED Requirements:** - If requirement doesn't exist in main spec → add it - - If requirement already exists → update it to match (treat as implicit MODIFIED) + - If requirement already exists → update it to match (treat as implicit + MODIFIED) - **MODIFIED Requirements:** + **MODIFIED Requirements:** - Find the requirement in main spec - Apply the changes - this can be: - Adding new scenarios the main spec does not have yet @@ -110,10 +139,10 @@ This is an **agent-driven** operation - you will read delta specs and directly e - Changing the requirement description - Preserve scenarios/content not mentioned in the delta - **REMOVED Requirements:** + **REMOVED Requirements:** - Remove the entire requirement block from main spec - - Retiring the capability. Delete the whole `spec.md` - and the directory once - nothing else is left in it - only when ALL of these hold: + - Retiring the capability. Delete the whole `spec.md` - and the directory + once nothing else is left in it - only when ALL of these hold: 1. removing the requirements *this run* left no requirement blocks; 2. the rest of the spec is well-formed (it still has a `## Purpose`); 3. the main spec was not already empty before this sync - if you removed @@ -124,35 +153,38 @@ This is an **agent-driven** operation - you will read delta specs and directly e 5. the change's `.openspec.yaml` declares `retire_capabilities: true`; 6. the `spec.md` resolves inside the real specs root (do not follow a capability-directory symlink to delete an external file). - If removing the selected requirements would leave no requirement blocks and - any retirement condition is not satisfied, do not modify the main spec. Stop - the sync for that capability, report the blocking condition, and tell the user - how to resolve it. Never write or leave an empty `## Requirements` section. - When only the marker is missing, say that too - it is the one thing the user - can add to make the retirement go through. - - Deleting the file also deletes its `## Purpose`; any other section blocks - retirement. Name Purpose when you report the retirement. Include a pasteable - `git checkout` only when the spec lived in the caller's checkout; - otherwise give checkout-scoped recovery guidance. - - **RENAMED Requirements:** + If removing the selected requirements would leave no requirement blocks + and any retirement condition is not satisfied, do not modify the main + spec. Stop the sync for that capability, report the blocking condition, + and tell the user how to resolve it. Never write or leave an empty + `## Requirements` section. When only the marker is missing, say that + too - it is the one thing the user can add to make the retirement go + through. + - Deleting the file also deletes its `## Purpose`; any other section + blocks retirement. Name Purpose when you report the retirement. Include + a pasteable `git checkout` only when the spec lived in the caller's + checkout; otherwise give checkout-scoped recovery guidance. + + **RENAMED Requirements:** - Find the FROM requirement, rename to TO - **`## Purpose` in the delta:** + **`## Purpose` in the delta:** - The main spec already has one and it is authoritative - leave it alone (this is what `openspec archive` does; it warns and moves on) d. **Create new main spec** if capability doesn't exist yet: - Create `/openspec/specs//spec.md` - - Add Purpose section: copy the delta's `## Purpose` body verbatim when it has one - (this is what `openspec archive` does); only write a brief TBD placeholder when it does not + - Add Purpose section: copy the delta's `## Purpose` body verbatim when it + has one (this is what `openspec archive` does); only write a brief TBD + placeholder when it does not - Add Requirements section with the ADDED requirements - Follow the **Main Spec Format Reference** below 5. **Validate updated main specs** - Run `openspec validate --specs` with the same selected-root flags used earlier. - If validation fails, report the problems and do not claim the sync succeeded. + Run `openspec validate --specs` with the same selected-root flags used + earlier. If validation fails, report the problems and do not claim the sync + succeeded. 6. **Show summary** @@ -205,7 +237,9 @@ The system SHALL keep doing the existing thing, now also handling A. **Main Spec Format Reference** -Main specs are what the delta merges INTO. They must never contain delta operation headers (`## ADDED/MODIFIED/REMOVED/RENAMED Requirements`) - after syncing, every requirement lives under a single `## Requirements` section: +Main specs are what the delta merges INTO. They must never contain delta +operation headers (`## ADDED/MODIFIED/REMOVED/RENAMED Requirements`) - after +syncing, every requirement lives under a single `## Requirements` section: ```markdown # Specification @@ -226,7 +260,10 @@ The system SHALL do something new. **Key Principle: Intelligent Merging** Unlike programmatic merging, you merge rather than overwrite: -- A MODIFIED block carries the whole requirement - body plus every scenario that survives the change. `openspec validate` and `openspec archive` both reject one that drops a scenario the main spec still has. + +- A MODIFIED block carries the whole requirement - body plus every scenario that + survives the change. `openspec validate` and `openspec archive` both reject + one that drops a scenario the main spec still has. - Keep anything the delta does not mention, in the main spec's existing order - Use your judgment to merge changes sensibly @@ -249,14 +286,22 @@ Main specs are now updated. The change remains active - archive when implementat ``` **Guardrails** + - Read both delta and main specs before making changes - Preserve existing content not mentioned in delta -- Never copy a delta file into a main spec as-is - merge its content so the main spec keeps the Main Spec Format Reference structure, with no delta operation headers +- Never copy a delta file into a main spec as-is - merge its content so the main + spec keeps the Main Spec Format Reference structure, with no delta operation + headers - If something is unclear, ask for clarification - Show what you're changing as you go - The operation should be idempotent - running twice should give same result -- Use only `artifactPaths.specs.existingOutputPaths`; never infer delta specs from unrelated artifacts -- Honor a caller-supplied subset of `existingOutputPaths`; never widen it back to the full list -- Fetch specs instructions once for direct sync, or reuse the archive-supplied snapshot inline -- Stop before every main-spec write on a non-zero or invalid JSON specs-instruction response -- Artifact rules constrain only the specs being written and are never copied into output files +- Use only `artifactPaths.specs.existingOutputPaths`; never infer delta specs + from unrelated artifacts +- Honor a caller-supplied subset of `existingOutputPaths`; never widen it back + to the full list +- Fetch specs instructions once for direct sync, or reuse the archive-supplied + snapshot inline +- Stop before every main-spec write on a non-zero or invalid JSON + specs-instruction response +- Artifact rules constrain only the specs being written and are never copied + into output files diff --git a/.gemini/skills/openspec-update-change/SKILL.md b/.gemini/skills/openspec-update-change/SKILL.md index 1db491a..3205311 100644 --- a/.gemini/skills/openspec-update-change/SKILL.md +++ b/.gemini/skills/openspec-update-change/SKILL.md @@ -10,13 +10,31 @@ metadata: generatedBy: "1.12.0" --- -Revise a change's existing planning artifacts and keep them coherent. Never edit code. - -**Store selection:** If the user names a store (a store is a standalone OpenSpec repo registered on this machine) or the work lives in one, run `openspec store list --json` to discover registered store ids, then pass `--store ` on the commands that read or write specs and changes (`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, `doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as sticky for the rest of the workflow. Every unscoped example of those commands below is shorthand: before running it, append the flag. For example, run `openspec status --change "" --json --store ""`, not the unscoped form shown below. Other commands do not take the flag. Hints printed by commands already carry the flag; keep it on follow-ups. Without a store, commands act on the nearest local `openspec/` root. - -**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. - -`/opsx:continue` is an optional workflow and may not be installed. Before suggesting it anywhere below, verify that it is available. If it is unavailable, `openspec status --change "" --json` shows the next artifact and `openspec instructions "" --change "" --json` explains how to create it. +Revise a change's existing planning artifacts and keep them coherent. Never edit +code. + +**Store selection:** If the user names a store (a store is a standalone OpenSpec +repo registered on this machine) or the work lives in one, run +`openspec store list --json` to discover registered store ids, then pass +`--store ` on the commands that read or write specs and changes +(`new change`, `status`, `instructions`, `list`, `show`, `validate`, `archive`, +`doctor`, `context`, `schemas`, `view`). Once selected, treat `--store ` as +sticky for the rest of the workflow. Every unscoped example of those commands +below is shorthand: before running it, append the flag. For example, run +`openspec status --change "" --json --store ""`, not the unscoped form +shown below. Other commands do not take the flag. Hints printed by commands +already carry the flag; keep it on follow-ups. Without a store, commands act on +the nearest local `openspec/` root. + +**Input**: Optionally specify a change name. If omitted, check if it can be +inferred from conversation context. If vague or ambiguous you MUST prompt for +available changes. + +`/opsx:continue` is an optional workflow and may not be installed. Before +suggesting it anywhere below, verify that it is available. If it is unavailable, +`openspec status --change "" --json` shows the next artifact and +`openspec instructions "" --change "" --json` explains how to +create it. **Steps** @@ -25,67 +43,105 @@ Revise a change's existing planning artifacts and keep them coherent. Never edit If a name is provided, use it. Otherwise: - Infer from conversation context if the user mentioned a change - Auto-select if only one active change exists - - If ambiguous, run `openspec list --json` to get available changes sorted by most recently modified, and ask the user to select one + - If ambiguous, run `openspec list --json` to get available changes sorted by + most recently modified, and ask the user to select one - When prompting, present the top 3-4 most recently modified changes as options, showing: + When prompting, present the top 3-4 most recently modified changes as + options, showing: - Change name - Schema (from `schema` field if present, otherwise "spec-driven") - Status (e.g., "0/5 tasks", "complete", "no tasks") - How recently it was modified (from `lastModified` field) - Mark the most recently modified change as "(Recommended)" since it's likely what the user wants to update. + Mark the most recently modified change as "(Recommended)" since it's likely + what the user wants to update. - Always announce: "Using change: " and how to override (e.g., `/opsx:update `). + Always announce: "Using change: " and how to override (e.g., + `/opsx:update `). 2. **Get the change's artifacts** + ```bash openspec status --change "" --json ``` + Parse the JSON to understand current state. The response includes: - `schemaName`: The workflow schema being used (e.g., "spec-driven") - - `artifacts`: Array of artifacts with their status ("done", "skipped", "ready", "blocked") - - `isPlanningComplete`: Boolean indicating if all planning artifacts are complete. Older CLI versions expose the same value as `isComplete`. - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context. Use these instead of assuming repo-local paths. - - The artifact ids and paths come from the active schema - do NOT assume them, and do NOT branch on hardcoded artifact names. Custom schemas must work unchanged. - - The files to edit are `artifactPaths..existingOutputPaths` - the concrete files that exist on disk, already glob-expanded for glob artifacts (e.g. `specs/**/*.md`). Do NOT write to `resolvedOutputPath`: for a glob artifact it is still the glob pattern, not a real file. + - `artifacts`: Array of artifacts with their status ("done", "skipped", + "ready", "blocked") + - `isPlanningComplete`: Boolean indicating if all planning artifacts are + complete. Older CLI versions expose the same value as `isComplete`. + - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path + and scope context. Use these instead of assuming repo-local paths. + + The artifact ids and paths come from the active schema - do NOT assume them, + and do NOT branch on hardcoded artifact names. Custom schemas must work + unchanged. + + The files to edit are `artifactPaths..existingOutputPaths` - the concrete + files that exist on disk, already glob-expanded for glob artifacts (e.g. + `specs/**/*.md`). Do NOT write to `resolvedOutputPath`: for a glob artifact + it is still the glob pattern, not a real file. 3. **Understand the request** - - If the user asked for a specific revision ("the design now uses X"), that is the starting edit. - - If they only said "update" / "make this coherent", treat it as a coherence review: read the existing artifacts and check them against each other for contradictions, gaps, and duplication. + - If the user asked for a specific revision ("the design now uses X"), that + is the starting edit. + - If they only said "update" / "make this coherent", treat it as a coherence + review: read the existing artifacts and check them against each other for + contradictions, gaps, and duplication. 4. **Read and reconcile** - - Read the artifact(s) the request touches and the change's other existing artifacts. - - Apply the requested edit. Then check every other existing artifact against it - in ANY direction: an edit to a later artifact may require revising an earlier one, not only the other way around. Build order is a useful reading order, not a constraint on which artifacts may be revised. + - Read the artifact(s) the request touches and the change's other existing + artifacts. + - Apply the requested edit. Then check every other existing artifact against + it - in ANY direction: an edit to a later artifact may require revising an + earlier one, not only the other way around. Build order is a useful reading + order, not a constraint on which artifacts may be revised. - Note everything that is now inconsistent, missing, or contradictory. - - Revise only files that already exist (`existingOutputPaths`). Do NOT create artifacts that don't exist yet, and do NOT invent new files under a glob artifact - note them and point the user to `/opsx:continue` to create them. + - Revise only files that already exist (`existingOutputPaths`). Do NOT create + artifacts that don't exist yet, and do NOT invent new files under a glob + artifact - note them and point the user to `/opsx:continue` to create them. - If the change is already coherent, say so and make no edits. 5. **Confirm and apply, one artifact at a time** - Show each proposed revision and why. Write only after the user confirms. - - If the user rejects a revision, do not write it - leave that artifact unchanged. - - When a substantial rewrite is needed, get that artifact's rules and template first: + - If the user rejects a revision, do not write it - leave that artifact + unchanged. + - When a substantial rewrite is needed, get that artifact's rules and + template first: + ```bash openspec instructions "" --change "" --json ``` 6. **Point to the next step (guidance only - NEVER act on it)** - Artifacts still missing -> suggest `/opsx:continue` to create them. - - Change already implemented (tasks checked off / already applied) -> the code may no longer match the revised plan; suggest `/opsx:apply` to carry the delta into code. + - Change already implemented (tasks checked off / already applied) -> the + code may no longer match the revised plan; suggest `/opsx:apply` to carry + the delta into code. - Everything done and implemented -> suggest `/opsx:archive`. **Output** After each invocation, show: + - Which artifacts were revised (and which proposed revisions were rejected) - Anything deferred to `/opsx:continue` (not-yet-created artifacts or files) - Where the change stands and the recommended next command **Guardrails** -- Planning artifacts only - NEVER edit implementation code. If the revised plan implies code changes, stop and point to `/opsx:apply`. -- Use the artifact ids and paths reported by `openspec status`; never branch on hardcoded artifact names. -- Edit only the concrete files in `existingOutputPaths`; never write to a glob `resolvedOutputPath`. -- Do not advance the build frontier: no new artifacts, no new files under glob artifacts - that is `/opsx:continue`'s job. + +- Planning artifacts only - NEVER edit implementation code. If the revised plan + implies code changes, stop and point to `/opsx:apply`. +- Use the artifact ids and paths reported by `openspec status`; never branch on + hardcoded artifact names. +- Edit only the concrete files in `existingOutputPaths`; never write to a glob + `resolvedOutputPath`. +- Do not advance the build frontier: no new artifacts, no new files under glob + artifacts - that is `/opsx:continue`'s job. - Confirm every edit with the user before writing. -- If the request changes the change's *intent* rather than refining it, first verify whether the optional `/opsx:new` workflow is available. If it is, recommend starting fresh with `/opsx:new` (the "Update vs. Start Fresh" heuristic). If it is unavailable, ask for a distinct unused change name and recommend `openspec new change ""` instead. +- If the request changes the change's *intent* rather than refining it, first + verify whether the optional `/opsx:new` workflow is available. If it is, + recommend starting fresh with `/opsx:new` (the "Update vs. Start Fresh" + heuristic). If it is unavailable, ask for a distinct unused change name and + recommend `openspec new change ""` instead. diff --git a/.github/dependabot.yml b/.github/dependabot.yml index d30f881..cc1431b 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -4,7 +4,11 @@ updates: directory: "/" schedule: interval: "daily" + cooldown: + default-days: 7 - package-ecosystem: "github-actions" directory: "/" schedule: interval: "daily" + cooldown: + default-days: 7 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 0788422..fd30148 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -4,19 +4,29 @@ on: branches: - main pull_request: + jobs: test: name: test runs-on: ubuntu-latest + permissions: + contents: read steps: - - uses: actions/checkout@v4 - - uses: actions/setup-go@v5 - with: - go-version: '1.23.5' - - uses: sqlc-dev/setup-sqlc@v4 - with: - sqlc-version: '1.28.0' - - run: make - - run: make test - - run: sqlc diff - working-directory: examples + - uses: actions/checkout@v7 + with: + persist-credentials: false + + - uses: actions/setup-go@v5 + with: + go-version-file: go.mod + + - uses: sqlc-dev/setup-sqlc@v4 + with: + sqlc-version: "1.31.1" + + - run: make + + - run: make test + + - run: sqlc diff + working-directory: examples diff --git a/.github/workflows/db.yml b/.github/workflows/db.yml index 46d0115..d70cdc7 100644 --- a/.github/workflows/db.yml +++ b/.github/workflows/db.yml @@ -4,15 +4,16 @@ on: branches: - main pull_request: -jobs: +jobs: build: name: test runs-on: ubuntu-latest - + permissions: + contents: read services: postgres: - image: postgres:11 + image: postgres:18 env: POSTGRES_USER: postgres POSTGRES_PASSWORD: postgres @@ -21,17 +22,31 @@ jobs: - 5432:5432 # needed because the postgres container does not provide a healthcheck options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5 - steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v7 + with: + persist-credentials: false + - uses: actions/setup-python@v5 + with: - python-version: 3.9 + python-version: "3.14" + - name: Install python dependencies working-directory: ./examples run: | python -m pip install --upgrade pip python -m pip install -r requirements.txt + + - name: Compile generated fixtures + run: python -m compileall -q internal/endtoend/testdata + + - name: Type-check feature matrix fixtures + run: | + for d in internal/endtoend/testdata/feature_matrix_*; do + (cd "$d" && mypy --strict --explicit-package-bases db _check.py) + done + - name: Test python code working-directory: ./examples env: diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000..e7b23a6 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,98 @@ +name: release + +on: + push: + branches: [main] + workflow_dispatch: + inputs: + bump: + description: "Version bump (auto = detect from PR title)" + required: false + default: auto + type: choice + options: + - auto + - patch + - minor + - major + +concurrency: + group: release + cancel-in-progress: false + +jobs: + release: + name: release + runs-on: ubuntu-latest + permissions: + contents: write + pull-requests: read + steps: + - uses: actions/checkout@v4 + with: + persist-credentials: false + fetch-depth: 0 + + - name: Compute version + id: version + run: | + bump=${INPUTS_BUMP:-auto} + if [[ "${bump}" == auto ]]; then + title=$(gh api "repos/${GITHUB_REPOSITORY}/commits/${GITHUB_SHA}/pulls" --jq '.[0].title // ""') + body=$(gh api "repos/${GITHUB_REPOSITORY}/commits/${GITHUB_SHA}/pulls" --jq '.[0].body // ""') + echo "PR title: ${title:-}" + if grep -qE '^[a-z]+(\(.+\))?!:' <<< "${title}" || grep -qE '^BREAKING[ -]CHANGE:' <<< "${body}"; then + bump=major + elif grep -qE '^feat(\(.+\))?:' <<< "${title}"; then + bump=minor + elif grep -qE '^(fix|hotfix)(\(.+\))?:' <<< "${title}"; then + bump=patch + else + echo "No releasable PR title; skipping release" + exit 0 + fi + fi + latest=$(git tag --list 'v*' --sort=-v:refname | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | head -1) + if [[ -z "${latest}" ]]; then + echo "version=v1.0.0" >> "${GITHUB_OUTPUT}" + exit 0 + fi + IFS=. read -r major minor patch <<< "${latest}" + major=${major#v} + case "${bump}" in + major) major=$((major + 1)); minor=0; patch=0 ;; + minor) minor=$((minor + 1)); patch=0 ;; + patch) patch=$((patch + 1)) ;; + esac + echo "version=v${major}.${minor}.${patch}" >> "${GITHUB_OUTPUT}" + env: + INPUTS_BUMP: ${{ inputs.bump }} + GH_TOKEN: ${{ github.token }} + + - uses: actions/setup-go@v5 + if: steps.version.outputs.version + with: + go-version-file: go.mod + + - name: Build WASM binary + if: steps.version.outputs.version + run: | + mkdir -p bin + cd plugin && GOOS=wasip1 GOARCH=wasm go build -o ../bin/alt-sqlc-gen-python.wasm main.go + + - name: Generate checksum + if: steps.version.outputs.version + run: sha256sum bin/alt-sqlc-gen-python.wasm > bin/alt-sqlc-gen-python.wasm.sha256 + + - name: Create GitHub release + if: steps.version.outputs.version + run: | + gh release create "${VERSION}" \ + --target "${GITHUB_SHA}" \ + --title "${VERSION}" \ + --generate-notes \ + bin/alt-sqlc-gen-python.wasm \ + bin/alt-sqlc-gen-python.wasm.sha256 + env: + VERSION: ${{ steps.version.outputs.version }} + GH_TOKEN: ${{ github.token }} diff --git a/.github/zizmor.yml b/.github/zizmor.yml new file mode 100644 index 0000000..00ea2bb --- /dev/null +++ b/.github/zizmor.yml @@ -0,0 +1,5 @@ +rules: + unpinned-uses: + config: + policies: + "*": ref-pin diff --git a/.gitignore b/.gitignore index 60b4f3d..1b2729e 100644 --- a/.gitignore +++ b/.gitignore @@ -5,3 +5,5 @@ bin .direnv .devenv* devenv.local.nix +__pycache__/ +.mypy_cache/ diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml new file mode 100644 index 0000000..42384d3 --- /dev/null +++ b/.pre-commit-config.yaml @@ -0,0 +1,30 @@ +repos: + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v6.0.0 + hooks: + - id: check-added-large-files + args: ["--maxkb=500"] + - id: trailing-whitespace + - id: end-of-file-fixer + - id: mixed-line-ending + args: ["--fix=lf"] + - id: check-merge-conflict + - id: check-yaml + args: ["--allow-multiple-documents"] + - id: check-json + - id: check-toml + - id: check-xml + - repo: https://github.com/rvben/rumdl-pre-commit + rev: v0.2.78 + hooks: + - id: rumdl-fmt + - repo: https://github.com/zricethezav/gitleaks + rev: v8.30.0 + hooks: + - id: gitleaks + args: ["--verbose"] + - repo: https://github.com/zizmorcore/zizmor-pre-commit + rev: v1.30.1 + hooks: + - id: zizmor + args: ["--fix=all"] diff --git a/.rumdl.toml b/.rumdl.toml new file mode 100644 index 0000000..3cd87d1 --- /dev/null +++ b/.rumdl.toml @@ -0,0 +1,5 @@ +[global] +disable = ["MD033"] + +[MD013] +reflow = true diff --git a/README.md b/README.md index c9f2531..1d1b15a 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,45 @@ +# sqlc-gen-python + +A [sqlc](https://sqlc.dev) plugin that generates typed Python (SQLAlchemy + +dataclasses or pydantic) from PostgreSQL queries. + +## Fork notes + +This is a fork of [sqlc-gen-python][upstream], brought up to parity with sqlc +v1.31.1. I started this fork because the official sqlc-gen-python and its forks +are no longer receiving active contributions at the time of writing. + +Compared to upstream it adds: + +- `sqlc.embed()`, query comments as docstrings, multi-dimensional arrays +- `:copyfrom`, `:batchexec`, `:batchone` and `:batchmany` +- Type overrides by database type or column, and renames, including from + sqlc's global `options` +- Queriers that accept SQLAlchemy `Session`/`AsyncSession`, with a typed + `_conn` and `typing.cast` on row values, so output passes `mypy --strict` +- Opt-in output styles: modern syntax (`X | None`, `list[X]`), PEP 695 + generic queriers, `pydantic.AwareDatetime`, `typing.Protocol` querier + interfaces, and typed errors for constraint violations +- Fixes: Python keywords are escaped (`from` → `from_`), enum member names + are always valid and unique, string literals and multi-line comments are + escaped, and `varchar`/`timestamp`/`time` and friends no longer map to `Any` + +Every new behaviour is behind an option and off by default, so generated code +for an existing config only changes where upstream produced invalid or untyped +output. Only PostgreSQL is supported. + ## Usage +Each GitHub release publishes `alt-sqlc-gen-python.wasm` and a matching +`alt-sqlc-gen-python.wasm.sha256`. Pin both in `sqlc.yaml`: + ```yaml version: "2" plugins: - name: py wasm: - url: https://downloads.sqlc.dev/plugin/sqlc-gen-python_1.3.0.wasm - sha256: fbedae96b5ecae2380a70fb5b925fd4bff58a6cfb1f3140375d098fbab7b3a3c + url: https://github.com/kvdomingo/sqlc-gen-python/releases/download/vX.Y.Z/alt-sqlc-gen-python.wasm + sha256: sql: - schema: "schema.sql" queries: "query.sql" @@ -20,27 +53,308 @@ sql: emit_async_querier: true ``` -### Emit Pydantic Models instead of `dataclasses` +Generated code needs SQLAlchemy and a PostgreSQL driver (psycopg 2/3 or +asyncpg). Some options raise the minimum Python version; see each option. -Option: `emit_pydantic_models` +### Options + +| Option | Default | Effect | +|---|---|---| +| `package` | | Package the query files import `models` (and `errors`) from | +| `emit_sync_querier` | `false` | Emit `Querier` | +| `emit_async_querier` | `false` | Emit `AsyncQuerier` | +| `emit_pydantic_models` | `false` | `pydantic.BaseModel` instead of dataclasses | +| `emit_str_enum` | `false` | `enum.StrEnum` (Python 3.11+) instead of `(str, enum.Enum)` | +| `emit_exact_table_names` | `false` | Don't singularize table names for model classes | +| `inflection_exclude_table_names` | `[]` | Table names never singularized | +| `query_parameter_limit` | `4` | Params above this become a params class; `0` always uses one | +| `emit_modern_types` | `false` | `X \| None`, `list[X]`, `collections.abc` (Python 3.10+) | +| `emit_generic_querier` | `false` | PEP 695 generic queriers (Python 3.12+) | +| `emit_aware_datetime` | `false` | `timestamptz` as `pydantic.AwareDatetime` (needs `emit_pydantic_models`) | +| `emit_querier_protocol` | `false` | `QuerierProtocol` / `AsyncQuerierProtocol` | +| `emit_query_errors` | `false` | Generate `errors.py` and raise typed errors | +| `overrides` | `[]` | Python types for database types or columns | +| `rename` | `{}` | Python names for database identifiers | +| `omit_unused_structs` | `false` | Drop models and enums no query uses | +| `omit_sqlc_version` | `false` | Leave the sqlc version out of file headers | + +### Sync and Async Queriers + +Options: `emit_sync_querier`, `emit_async_querier` + +These generate `Querier` and/or `AsyncQuerier` classes that wrap a SQLAlchemy +connection and expose a method for each query. + +- `Querier` accepts `sqlalchemy.engine.Connection` or `sqlalchemy.orm.Session` +- `AsyncQuerier` accepts `sqlalchemy.ext.asyncio.AsyncConnection` or + `sqlalchemy.ext.asyncio.AsyncSession` + +```py +with Session(engine) as session: + querier = query.Querier(session) + author = querier.get_author(id=1) +``` + +The query command determines the method signature: + +| Command | Sync return type | Async return type | +|---|---|---| +| `:one` | `Optional[Row]` | `Optional[Row]` | +| `:many` | `Iterator[Row]` | `AsyncIterator[Row]` | +| `:exec` | `None` | `None` | +| `:execrows` | `int` | `int` | +| `:execresult` | `sqlalchemy.engine.Result[Any]` | `sqlalchemy.engine.Result[Any]` | +| `:copyfrom` | `int` | `int` | +| `:batchexec` | `None` | `None` | +| `:batchone` | `Iterator[Optional[Row]]` | `AsyncIterator[Optional[Row]]` | +| `:batchmany` | `Iterator[List[Row]]` | `AsyncIterator[List[Row]]` | + +`Row` is a reused model, a generated row class, or a scalar type. Comments +above a query's `-- name:` line become the method's docstring. + +Generated code with both options enabled (default syntax): + +```py +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def get_user(self, *, id: int) -> Optional[models.User]: + """Fetch a user by id.""" + row = self._conn.execute(sqlalchemy.text(GET_USER), {"p1": id}).first() + if row is None: + return None + return models.User( + id=cast(int, row[0]), + name=cast(str, row[1]), + ) + + def list_users(self) -> Iterator[models.User]: + result = self._conn.execute(sqlalchemy.text(LIST_USERS)) + for row in result: + yield models.User( + id=cast(int, row[0]), + name=cast(str, row[1]), + ) + + +class AsyncQuerier: + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): + self._conn = conn + + async def list_users(self) -> AsyncIterator[models.User]: + result = await self._conn.stream(sqlalchemy.text(LIST_USERS)) + async for row in result: + yield models.User( + id=cast(int, row[0]), + name=cast(str, row[1]), + ) +``` + +### Generic Queriers + +Option: `emit_generic_querier` (Python 3.12+) + +Emits the queriers as [PEP 695](https://peps.python.org/pep-0695/) generics, +so the connection type you pass in is preserved: `Querier(session)._conn` is +typed as `Session`. Type-checking the output needs mypy 1.12+ or a recent +pyright. + +```py +class Querier[_ConnT: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]]: + _conn: _ConnT + + def __init__(self, conn: _ConnT): + self._conn = conn +``` + +### Bulk and Batch Commands + +Commands: `:copyfrom`, `:batchexec`, `:batchone`, `:batchmany` + +All four always generate a params class, whatever `query_parameter_limit` is, +and take one positional `Sequence` of it. An empty sequence runs nothing. + +- `:copyfrom` runs a single `executemany` and returns the row count, as + reported by the driver (some report `-1`). +- `:batchexec` runs a single `executemany` and returns `None`. +- `:batchone` and `:batchmany` return a generator that runs the statement + once per item, in order, as you iterate. `:batchone` yields the first row or + `None` per item; `:batchmany` yields a list of rows per item. That is one + round-trip per item; for bulk reads prefer `:many` with + `= ANY($1::bigint[])`. + +```sql +-- name: CreateAuthors :batchexec +INSERT INTO authors (name, bio) VALUES ($1, $2); +``` + +```py +@dataclasses.dataclass() +class CreateAuthorsParams: + name: str + bio: Optional[str] + + +class Querier: + # ... + + def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> None: + if not arg: + return None + self._conn.execute( + sqlalchemy.text(CREATE_AUTHORS), + [{"p1": a.name, "p2": a.bio} for a in arg], + ) +``` + +### Querier Protocols for Testability + +Option: `emit_querier_protocol` + +Generates `QuerierProtocol` and `AsyncQuerierProtocol` (`typing.Protocol`) +classes that declare every querier method. The queriers satisfy them +structurally, so application code can depend on the protocol and tests can +pass a simple fake. A protocol is emitted only for queriers that are enabled. + +```py +class QuerierProtocol(Protocol): + def get_author(self, *, id: int) -> Optional[models.Author]: ... + + def list_authors(self) -> Iterator[models.Author]: ... +``` + +```py +def get_author_bio(querier: QuerierProtocol, author_id: int) -> str: + author = querier.get_author(id=author_id) + return author.bio if author and author.bio else "Unknown" + + +class FakeQuerier: + def get_author(self, *, id: int) -> Optional[models.Author]: + return models.Author(id=id, name="Test", bio="A bio") + + def list_authors(self) -> Iterator[models.Author]: + yield from () +``` + +### Typed Error Wrapping + +Option: `emit_query_errors` + +Generates an `errors.py` module next to `models.py`. Every querier method body +is wrapped in `errors._wrap_errors(...)`, which re-raises +`sqlalchemy.exc.IntegrityError` and `sqlalchemy.exc.OperationalError` as a +subclass of `errors.QueryError`, chosen by SQLSTATE: -By default, `sqlc-gen-python` will emit `dataclasses` for the models. If you prefer to use [`pydantic`](https://docs.pydantic.dev/latest/) models, you can enable this option. +| Exception | SQLSTATE | From | +|---|---|---| +| `UniqueViolationError` | `23505` | `IntegrityError` | +| `ForeignKeyViolationError` | `23503` | `IntegrityError` | +| `CheckViolationError` | `23514` | `IntegrityError` | +| `NotNullViolationError` | `23502` | `IntegrityError` | +| `ExclusionViolationError` | `23P01` | `IntegrityError` | +| `StatementTimeoutError` | `57014` | `OperationalError` | +| `DeadlockError` | `40P01` | `OperationalError` | +| `SerializationError` | `40001` | `OperationalError` | -with `emit_pydantic_models` +Any other SQLSTATE raises `QueryError` itself. Each error has `query_name`, +`cause` (the original SQLAlchemy exception, also set as `__cause__`) and +`constraint_name` (when the driver reports it). This works with psycopg 2, +psycopg 3 and asyncpg. Errors raised while you iterate a `:many`, `:batchone` +or `:batchmany` result are wrapped too; all other exceptions pass through +unchanged. ```py -from pydantic import BaseModel +def create_author(self, *, name: str, bio: Optional[str]) -> Optional[models.Author]: + with errors._wrap_errors("create_author"): + row = self._conn.execute( + sqlalchemy.text(CREATE_AUTHOR), {"p1": name, "p2": bio} + ).first() + ... +``` + +```py +try: + querier.create_author(name="Ursula", bio=None) +except errors.UniqueViolationError as e: + print(e.constraint_name) # "authors_name_key" +``` + +A query file named `errors.sql` would overwrite the generated module, so it +fails generation. + +### Embedded Structs with `sqlc.embed()` + +When a query joins tables, `sqlc.embed()` nests whole models in the row +instead of flattening their columns. With a `LEFT JOIN` that finds no match, +the embedded model is built from `None` values even though its fields are +typed as non-null, as in sqlc's Go codegen. + +```sql +-- name: GetBookWithAuthor :one +SELECT sqlc.embed(books), sqlc.embed(authors) +FROM books +JOIN authors ON books.author_id = authors.id +WHERE books.id = $1; +``` + +```py +@dataclasses.dataclass() +class GetBookWithAuthorRow: + books: models.Book + authors: models.Author + + +def get_book_with_author(self, *, id: int) -> Optional[GetBookWithAuthorRow]: + row = self._conn.execute(sqlalchemy.text(GET_BOOK_WITH_AUTHOR), {"p1": id}).first() + if row is None: + return None + return GetBookWithAuthorRow( + books=models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + ), + authors=models.Author( + id=cast(int, row[3]), + name=cast(str, row[4]), + ), + ) +``` + +### Modern Type Syntax + +Option: `emit_modern_types` (Python 3.10+) + +Emits `X | None` and `list[X]` instead of `Optional[X]` and `List[X]`, and +imports `Iterator`, `AsyncIterator` and `Sequence` from `collections.abc` +instead of `typing`. +### Aware Datetimes + +Option: `emit_aware_datetime` (requires `emit_pydantic_models` and pydantic 2) + +Annotates `timestamptz` columns and parameters as `pydantic.AwareDatetime`, so +naive datetimes fail validation. `timestamp` stays `datetime.datetime`. + +### Emit Pydantic Models instead of `dataclasses` + +Option: `emit_pydantic_models` + +```py class Author(pydantic.BaseModel): id: int name: str ``` -without `emit_pydantic_models` +Without the option: ```py -import dataclasses - @dataclasses.dataclass() class Author: id: int @@ -49,17 +363,10 @@ class Author: ### Use `enum.StrEnum` for Enums -Option: `emit_str_enum` +Option: `emit_str_enum` (Python 3.11+) -`enum.StrEnum` was introduce in Python 3.11. - -`enum.StrEnum` is a subclass of `str` that is also a subclass of `Enum`. This allows for the use of `Enum` values as strings, compared to strings, or compared to other `enum.StrEnum` types. - -This is convenient for type checking and validation, as well as for serialization and deserialization. - -By default, `sqlc-gen-python` will emit `(str, enum.Enum)` for the enum classes. If you prefer to use `enum.StrEnum`, you can enable this option. - -with `emit_str_enum` +`enum.StrEnum` members are `str` instances, so they compare equal to strings +and serialize as strings. ```py class Status(enum.StrEnum): @@ -68,11 +375,138 @@ class Status(enum.StrEnum): CLOSED = "clo@sed" ``` -without `emit_str_enum` (current behavior) +Without the option, enums subclass `(str, enum.Enum)`. Enum values that don't +make valid names become `VALUE_`, get a `VALUE_` prefix, or a `_2` suffix. -```py -class Status(str, enum.Enum): - """Venues can be either open or closed""" - OPEN = "op!en" - CLOSED = "clo@sed" +### Type Overrides + +Option: `overrides` + +Maps a database type or a column to a Python type. `py_type` is either a +dotted path, imported as its module, or a bare name plus `py_import`, imported +with `from`: + +```yaml +options: + package: authors + overrides: + - db_type: jsonb + py_type: my_lib.types.Payload # import my_lib.types + - db_type: jsonb + nullable: true + py_type: my_lib.types.Payload + - column: "authors.id" # or schema.table.column; * globs + py_type: UUID + py_import: uuid # from uuid import UUID + - db_type: bytea + py_type: bytes # builtin, no import +``` + +- A `db_type` entry applies to non-null columns, or only to nullable ones with + `nullable: true`. +- `column` entries win over `db_type` entries, and also apply to parameters + compared against that column. +- Nullable and array columns keep their `Optional[...]`/`List[...]` wrappers. +- `db_type` accepts SQL spellings such as `bigint` or + `timestamp with time zone` as well as `int8` or `timestamptz`. +- A generic `py_type` such as `dict[str, decimal.Decimal]` imports the module + of every dotted name in it. Bare names other than builtins are not imported, + so write `typing.Any`, not `Any`. + +Overrides and `rename` can also be set once for every codegen block in sqlc's +top-level `options`. Codegen `overrides` are checked before global ones, but a +global `rename` entry replaces a codegen entry for the same key, as in sqlc's +Go codegen: + +```yaml +options: + py: + overrides: + - db_type: citext + py_type: my_lib.CIText +``` + +### Renames + +Option: `rename` + +Maps a database identifier to the Python name used for it: a singularized +table name (model class), an enum type name (enum class), a column name (field +and parameter) or an enum value (member). As in sqlc's Go codegen, it is one +flat map. + +```yaml +rename: + spotify_url: spotify_link + person: Human +``` + +Names that are Python keywords get a trailing underscore (`from` becomes +`from_`), whether they come from the schema or from `rename`. So do keyword +parameters that would shadow a name the method body uses, such as `errors`, +`models`, `cast` or an imported module, and a repeated parameter name gets a +numeric suffix (`id`, `id_2`). + +### Omit Unused Structs + +Option: `omit_unused_structs` + +Leaves out of `models.py` the enums and models that no query's parameters, +result or embedded tables refer to. + +### Omit sqlc Version + +Option: `omit_sqlc_version` + +Leaves the `# versions:` / `# sqlc vX.Y.Z` lines out of file headers. + +## Migrating from upstream or the original alt-sqlc-gen-python + +- From upstream sqlc-gen-python: swap the plugin URL and sha256. Options are + unchanged and new behaviour is opt-in. Expect diffs where upstream produced + invalid code (keyword names, empty classes, enum names) or `Any` + (`varchar`, `timestamp`, …), plus the annotation-only `cast()` and + `_conn: Union[...]` changes. +- From [asavoy/alt-sqlc-gen-python][asavoy]: + - Set `emit_modern_types`, `emit_aware_datetime` and `emit_generic_querier` + to get its output style; here they are opt-in. + - `emit_query_errors` keeps the same class names, constructor, `query_name` + and `cause`; `constraint_name` is new. + - `overrides` accepts its `py_type` + `py_import` form unchanged. + - `:batchexec` takes one positional `arg: Sequence[Params]` instead of a + keyword-only `args: list[Params]`. + +## Development + +```sh +mise install # Go, buf, Python +mise exec -- make all # bin/sqlc-gen-python.wasm +mise exec -- make test # unit tests + sqlc diff over internal/endtoend/testdata ``` + +`make test` needs sqlc on `PATH`. The examples in `examples/` are generated +from the local build; run `sqlc diff` there, and `pytest src/tests` against a +Postgres (see `examples/src/tests/conftest.py` for the `PG_*` variables). + +To try a local build in another project, point `sqlc.yaml` at it +(`url: file:///path/to/bin/sqlc-gen-python.wasm`, no `sha256` needed). + +## Release process + +Releases are automated by `.github/workflows/release.yml`. On every push to +`main`, the workflow reads the merged PR's title: + +| PR title | Bump | +|---|---| +| `feat: …` | minor | +| `fix: …` / `hotfix: …` | patch | +| `type!: …`, or `BREAKING CHANGE:` in the body | major | +| anything else | no release | + +It then builds `alt-sqlc-gen-python.wasm`, writes its sha256 to +`alt-sqlc-gen-python.wasm.sha256`, and creates a GitHub release tagged +`vX.Y.Z` with both files and generated notes. To force a bump, run the +workflow manually with `bump` set to `patch`, `minor` or `major`. + +[upstream]: https://github.com/sqlc-dev/sqlc-gen-python +[asavoy]: https://github.com/asavoy/alt-sqlc-gen-python diff --git a/examples/pytest.ini b/examples/pytest.ini new file mode 100644 index 0000000..7e40766 --- /dev/null +++ b/examples/pytest.ini @@ -0,0 +1,4 @@ +[pytest] +asyncio_mode = auto +asyncio_default_fixture_loop_scope = session +asyncio_default_test_loop_scope = session diff --git a/examples/requirements.txt b/examples/requirements.txt index 26f645a..49c7394 100644 --- a/examples/requirements.txt +++ b/examples/requirements.txt @@ -1,5 +1,7 @@ -pytest~=6.2.2 -pytest-asyncio~=0.14.0 -psycopg2-binary~=2.8.6 -asyncpg~=0.21.0 -sqlalchemy==1.4.0 +pytest>=8 +pytest-asyncio>=0.24 +psycopg[binary]>=3.2 +asyncpg>=0.30 +sqlalchemy[asyncio]>=2 +pydantic>=2 +mypy>=1.12 diff --git a/examples/sqlc.yaml b/examples/sqlc.yaml index 712c5db..630e3f7 100644 --- a/examples/sqlc.yaml +++ b/examples/sqlc.yaml @@ -2,8 +2,7 @@ version: '2' plugins: - name: py wasm: - url: https://downloads.sqlc.dev/plugin/sqlc-gen-python_1.2.0.wasm - sha256: a6c5d174c407007c3717eea36ff0882744346e6ba991f92f71d6ab2895204c0e + url: file://../bin/sqlc-gen-python.wasm sql: - schema: "src/authors/schema.sql" queries: "src/authors/query.sql" @@ -46,3 +45,32 @@ sql: package: ondeck emit_async_querier: true query_parameter_limit: 5 +- schema: "src/library/schema.sql" + queries: "src/library/query.sql" + engine: postgresql + codegen: + - out: src/library + plugin: py + options: + package: library + emit_sync_querier: true + emit_async_querier: true + emit_querier_protocol: true + emit_query_errors: true +- schema: "src/library_modern/schema.sql" + queries: "src/library_modern/query.sql" + engine: postgresql + codegen: + - out: src/library_modern + plugin: py + options: + package: library_modern + emit_sync_querier: true + emit_async_querier: true + emit_querier_protocol: true + emit_query_errors: true + emit_pydantic_models: true + emit_str_enum: true + emit_modern_types: true + emit_aware_datetime: true + emit_generic_querier: true diff --git a/examples/src/authors/models.py b/examples/src/authors/models.py index 96553a5..973a3fe 100644 --- a/examples/src/authors/models.py +++ b/examples/src/authors/models.py @@ -1,6 +1,6 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 import dataclasses from typing import Optional diff --git a/examples/src/authors/query.py b/examples/src/authors/query.py index 019f877..2c1175b 100644 --- a/examples/src/authors/query.py +++ b/examples/src/authors/query.py @@ -1,11 +1,12 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 # source: query.sql -from typing import AsyncIterator, Iterator, Optional +from typing import AsyncIterator, Iterator, Optional, Union, cast import sqlalchemy import sqlalchemy.ext.asyncio +import sqlalchemy.orm from authors import models @@ -39,7 +40,9 @@ class Querier: - def __init__(self, conn: sqlalchemy.engine.Connection): + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): self._conn = conn def create_author(self, *, name: str, bio: Optional[str]) -> Optional[models.Author]: @@ -47,9 +50,9 @@ def create_author(self, *, name: str, bio: Optional[str]) -> Optional[models.Aut if row is None: return None return models.Author( - id=row[0], - name=row[1], - bio=row[2], + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), ) def delete_author(self, *, id: int) -> None: @@ -60,23 +63,25 @@ def get_author(self, *, id: int) -> Optional[models.Author]: if row is None: return None return models.Author( - id=row[0], - name=row[1], - bio=row[2], + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), ) def list_authors(self) -> Iterator[models.Author]: result = self._conn.execute(sqlalchemy.text(LIST_AUTHORS)) for row in result: yield models.Author( - id=row[0], - name=row[1], - bio=row[2], + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), ) class AsyncQuerier: - def __init__(self, conn: sqlalchemy.ext.asyncio.AsyncConnection): + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): self._conn = conn async def create_author(self, *, name: str, bio: Optional[str]) -> Optional[models.Author]: @@ -84,9 +89,9 @@ async def create_author(self, *, name: str, bio: Optional[str]) -> Optional[mode if row is None: return None return models.Author( - id=row[0], - name=row[1], - bio=row[2], + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), ) async def delete_author(self, *, id: int) -> None: @@ -97,16 +102,16 @@ async def get_author(self, *, id: int) -> Optional[models.Author]: if row is None: return None return models.Author( - id=row[0], - name=row[1], - bio=row[2], + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), ) async def list_authors(self) -> AsyncIterator[models.Author]: result = await self._conn.stream(sqlalchemy.text(LIST_AUTHORS)) async for row in result: yield models.Author( - id=row[0], - name=row[1], - bio=row[2], + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), ) diff --git a/examples/src/booktest/models.py b/examples/src/booktest/models.py index d7ee131..0baeb70 100644 --- a/examples/src/booktest/models.py +++ b/examples/src/booktest/models.py @@ -1,6 +1,6 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 import dataclasses import datetime import enum diff --git a/examples/src/booktest/query.py b/examples/src/booktest/query.py index bc71f22..21ce54a 100644 --- a/examples/src/booktest/query.py +++ b/examples/src/booktest/query.py @@ -1,10 +1,10 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 # source: query.sql import dataclasses import datetime -from typing import AsyncIterator, List, Optional +from typing import AsyncIterator, List, Optional, Union, cast import sqlalchemy import sqlalchemy.ext.asyncio @@ -13,7 +13,7 @@ BOOKS_BY_TAGS = """-- name: books_by_tags \\:many -SELECT +SELECT book_id, title, name, @@ -112,32 +112,34 @@ class CreateBookParams: class AsyncQuerier: - def __init__(self, conn: sqlalchemy.ext.asyncio.AsyncConnection): + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): self._conn = conn async def books_by_tags(self, *, dollar_1: List[str]) -> AsyncIterator[BooksByTagsRow]: result = await self._conn.stream(sqlalchemy.text(BOOKS_BY_TAGS), {"p1": dollar_1}) async for row in result: yield BooksByTagsRow( - book_id=row[0], - title=row[1], - name=row[2], - isbn=row[3], - tags=row[4], + book_id=cast(int, row[0]), + title=cast(str, row[1]), + name=cast(Optional[str], row[2]), + isbn=cast(str, row[3]), + tags=cast(List[str], row[4]), ) async def books_by_title_year(self, *, title: str, year: int) -> AsyncIterator[models.Book]: result = await self._conn.stream(sqlalchemy.text(BOOKS_BY_TITLE_YEAR), {"p1": title, "p2": year}) async for row in result: yield models.Book( - book_id=row[0], - author_id=row[1], - isbn=row[2], - book_type=row[3], - title=row[4], - year=row[5], - available=row[6], - tags=row[7], + book_id=cast(int, row[0]), + author_id=cast(int, row[1]), + isbn=cast(str, row[2]), + book_type=cast(models.BookType, row[3]), + title=cast(str, row[4]), + year=cast(int, row[5]), + available=cast(datetime.datetime, row[6]), + tags=cast(List[str], row[7]), ) async def create_author(self, *, name: str) -> Optional[models.Author]: @@ -145,8 +147,8 @@ async def create_author(self, *, name: str) -> Optional[models.Author]: if row is None: return None return models.Author( - author_id=row[0], - name=row[1], + author_id=cast(int, row[0]), + name=cast(str, row[1]), ) async def create_book(self, arg: CreateBookParams) -> Optional[models.Book]: @@ -162,14 +164,14 @@ async def create_book(self, arg: CreateBookParams) -> Optional[models.Book]: if row is None: return None return models.Book( - book_id=row[0], - author_id=row[1], - isbn=row[2], - book_type=row[3], - title=row[4], - year=row[5], - available=row[6], - tags=row[7], + book_id=cast(int, row[0]), + author_id=cast(int, row[1]), + isbn=cast(str, row[2]), + book_type=cast(models.BookType, row[3]), + title=cast(str, row[4]), + year=cast(int, row[5]), + available=cast(datetime.datetime, row[6]), + tags=cast(List[str], row[7]), ) async def delete_book(self, *, book_id: int) -> None: @@ -180,8 +182,8 @@ async def get_author(self, *, author_id: int) -> Optional[models.Author]: if row is None: return None return models.Author( - author_id=row[0], - name=row[1], + author_id=cast(int, row[0]), + name=cast(str, row[1]), ) async def get_book(self, *, book_id: int) -> Optional[models.Book]: @@ -189,14 +191,14 @@ async def get_book(self, *, book_id: int) -> Optional[models.Book]: if row is None: return None return models.Book( - book_id=row[0], - author_id=row[1], - isbn=row[2], - book_type=row[3], - title=row[4], - year=row[5], - available=row[6], - tags=row[7], + book_id=cast(int, row[0]), + author_id=cast(int, row[1]), + isbn=cast(str, row[2]), + book_type=cast(models.BookType, row[3]), + title=cast(str, row[4]), + year=cast(int, row[5]), + available=cast(datetime.datetime, row[6]), + tags=cast(List[str], row[7]), ) async def update_book(self, *, title: str, tags: List[str], book_id: int) -> None: diff --git a/examples/src/booktest/query.sql b/examples/src/booktest/query.sql index 194897a..bf27473 100644 --- a/examples/src/booktest/query.sql +++ b/examples/src/booktest/query.sql @@ -15,7 +15,7 @@ SELECT * FROM books WHERE title = $1 AND year = $2; -- name: BooksByTags :many -SELECT +SELECT book_id, title, name, diff --git a/examples/src/jets/models.py b/examples/src/jets/models.py index 0d4eb5d..1fbd6cc 100644 --- a/examples/src/jets/models.py +++ b/examples/src/jets/models.py @@ -1,6 +1,6 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 import dataclasses diff --git a/examples/src/jets/query-building.py b/examples/src/jets/query-building.py index 7651116..658492d 100644 --- a/examples/src/jets/query-building.py +++ b/examples/src/jets/query-building.py @@ -1,8 +1,8 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 # source: query-building.sql -from typing import AsyncIterator, Optional +from typing import AsyncIterator, Optional, Union, cast import sqlalchemy import sqlalchemy.ext.asyncio @@ -26,14 +26,16 @@ class AsyncQuerier: - def __init__(self, conn: sqlalchemy.ext.asyncio.AsyncConnection): + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): self._conn = conn async def count_pilots(self) -> Optional[int]: row = (await self._conn.execute(sqlalchemy.text(COUNT_PILOTS))).first() if row is None: return None - return row[0] + return cast(int, row[0]) async def delete_pilot(self, *, id: int) -> None: await self._conn.execute(sqlalchemy.text(DELETE_PILOT), {"p1": id}) @@ -42,6 +44,6 @@ async def list_pilots(self) -> AsyncIterator[models.Pilot]: result = await self._conn.stream(sqlalchemy.text(LIST_PILOTS)) async for row in result: yield models.Pilot( - id=row[0], - name=row[1], + id=cast(int, row[0]), + name=cast(str, row[1]), ) diff --git a/examples/src/library/__init__.py b/examples/src/library/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/examples/src/library/errors.py b/examples/src/library/errors.py new file mode 100644 index 0000000..b5998e9 --- /dev/null +++ b/examples/src/library/errors.py @@ -0,0 +1,108 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import contextlib +from typing import Iterator, Optional + +import sqlalchemy.exc + + +class QueryError(Exception): + """Base class for all query errors.""" + + def __init__(self, message: str, query_name: str, cause: Optional[Exception] = None): + super().__init__(message) + self.query_name = query_name + self.cause = cause + self.constraint_name: Optional[str] = _constraint_name(cause) + + +class UniqueViolationError(QueryError): + """Raised on unique constraint violation (PG 23505).""" + + +class ForeignKeyViolationError(QueryError): + """Raised on foreign key constraint violation (PG 23503).""" + + +class CheckViolationError(QueryError): + """Raised on check constraint violation (PG 23514).""" + + +class NotNullViolationError(QueryError): + """Raised on not-null constraint violation (PG 23502).""" + + +class ExclusionViolationError(QueryError): + """Raised on exclusion constraint violation (PG 23P01).""" + + +class StatementTimeoutError(QueryError): + """Raised on statement timeout (PG 57014).""" + + +class DeadlockError(QueryError): + """Raised on deadlock detected (PG 40P01).""" + + +class SerializationError(QueryError): + """Raised on serialization failure (PG 40001).""" + + +def _sqlstate(e: sqlalchemy.exc.DBAPIError) -> Optional[str]: + # psycopg2 exposes pgcode, psycopg 3 sqlstate; SQLAlchemy's asyncpg + # adapter keeps the asyncpg error, which has sqlstate, as __cause__. + for source in (e.orig, getattr(e.orig, "__cause__", None)): + code = getattr(source, "pgcode", None) or getattr(source, "sqlstate", None) + if isinstance(code, str): + return code + return None + + +def _constraint_name(cause: Optional[Exception]) -> Optional[str]: + orig = getattr(cause, "orig", None) + name = getattr(getattr(orig, "diag", None), "constraint_name", None) + if name is None: + name = getattr(getattr(orig, "__cause__", None), "constraint_name", None) + return name if isinstance(name, str) else None + + +def _wrap_integrity_error(e: sqlalchemy.exc.IntegrityError, query_name: str) -> QueryError: + pgcode = _sqlstate(e) + if pgcode == "23505": + return UniqueViolationError(str(e), query_name, cause=e) + if pgcode == "23503": + return ForeignKeyViolationError(str(e), query_name, cause=e) + if pgcode == "23514": + return CheckViolationError(str(e), query_name, cause=e) + if pgcode == "23502": + return NotNullViolationError(str(e), query_name, cause=e) + if pgcode == "23P01": + return ExclusionViolationError(str(e), query_name, cause=e) + return QueryError(str(e), query_name, cause=e) + + +def _wrap_operational_error(e: sqlalchemy.exc.OperationalError, query_name: str) -> QueryError: + pgcode = _sqlstate(e) + if pgcode == "57014": + return StatementTimeoutError(str(e), query_name, cause=e) + if pgcode == "40P01": + return DeadlockError(str(e), query_name, cause=e) + if pgcode == "40001": + return SerializationError(str(e), query_name, cause=e) + return QueryError(str(e), query_name, cause=e) + + +@contextlib.contextmanager +def _wrap_errors(query_name: str) -> Iterator[None]: + """Re-raise integrity and operational errors as typed QueryErrors. + + Only those two SQLAlchemy exceptions are caught, so GeneratorExit from a + consumer that stops iterating early passes through untouched. + """ + try: + yield + except sqlalchemy.exc.IntegrityError as e: + raise _wrap_integrity_error(e, query_name) from e + except sqlalchemy.exc.OperationalError as e: + raise _wrap_operational_error(e, query_name) from e diff --git a/examples/src/library/models.py b/examples/src/library/models.py new file mode 100644 index 0000000..837bd53 --- /dev/null +++ b/examples/src/library/models.py @@ -0,0 +1,29 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses +import datetime +import enum +from typing import Optional + + +class BookStatus(str, enum.Enum): + DRAFT = "draft" + IN_PRINT = "in-print" + OUT_OF_PRINT = "out_of_print" + + +@dataclasses.dataclass() +class Author: + id: int + name: str + bio: Optional[str] + created_at: datetime.datetime + + +@dataclasses.dataclass() +class Book: + id: int + author_id: int + title: str + status: BookStatus diff --git a/examples/src/library/query.py b/examples/src/library/query.py new file mode 100644 index 0000000..418c151 --- /dev/null +++ b/examples/src/library/query.py @@ -0,0 +1,412 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import dataclasses +import datetime +from typing import Any, AsyncIterator, Iterator, List, Optional, Protocol, Sequence, Union, cast + +import sqlalchemy +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from library import errors, models + + +CREATE_AUTHOR = """-- name: create_author \\:one +INSERT INTO authors (name, bio) VALUES (:p1, :p2) RETURNING id, name, bio, created_at +""" + + +CREATE_AUTHOR_AT = """-- name: create_author_at \\:one +INSERT INTO authors (name, created_at) VALUES (:p1, :p2) RETURNING id, name, bio, created_at +""" + + +CREATE_AUTHORS = """-- name: create_authors \\:copyfrom +INSERT INTO authors (name, bio) VALUES (:p1, :p2) +""" + + +@dataclasses.dataclass() +class CreateAuthorsParams: + name: str + bio: Optional[str] + + +CREATE_BOOK = """-- name: create_book \\:one +INSERT INTO books (author_id, title) VALUES (:p1, :p2) RETURNING id, author_id, title, status +""" + + +CREATE_BOOKS = """-- name: create_books \\:batchexec +INSERT INTO books (author_id, title, status) VALUES (:p1, :p2, :p3) +""" + + +@dataclasses.dataclass() +class CreateBooksParams: + author_id: int + title: str + status: models.BookStatus + + +GET_AUTHORS_BY_ID = """-- name: get_authors_by_id \\:batchone +SELECT id, name, bio, created_at FROM authors WHERE id = :p1 +""" + + +@dataclasses.dataclass() +class GetAuthorsByIDParams: + id: int + + +GET_BOOK_WITH_AUTHOR = """-- name: get_book_with_author \\:one +SELECT books.id, books.author_id, books.title, books.status, authors.id, authors.name, authors.bio, authors.created_at +FROM books JOIN authors ON authors.id = books.author_id +WHERE books.id = :p1 +""" + + +@dataclasses.dataclass() +class GetBookWithAuthorRow: + books: models.Book + authors: models.Author + + +INSERT_AUTHORS_RETURNING = """-- name: insert_authors_returning \\:batchone +INSERT INTO authors (name) VALUES (:p1) RETURNING id, name, bio, created_at +""" + + +@dataclasses.dataclass() +class InsertAuthorsReturningParams: + name: str + + +LIST_AUTHORS = """-- name: list_authors \\:many +SELECT id, name, bio, created_at FROM authors ORDER BY name +""" + + +LIST_BOOKS_BY_AUTHOR = """-- name: list_books_by_author \\:batchmany +SELECT id, author_id, title, status FROM books WHERE author_id = :p1 ORDER BY id +""" + + +@dataclasses.dataclass() +class ListBooksByAuthorParams: + author_id: int + + +SET_BOOK_TITLE = """-- name: set_book_title \\:exec +UPDATE books SET title = :p2 WHERE id = :p1 +""" + + +class QuerierProtocol(Protocol): + def create_author(self, *, name: str, bio: Optional[str]) -> Optional[models.Author]: ... + + def create_author_at(self, *, name: str, created_at: datetime.datetime) -> Optional[models.Author]: ... + + def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: ... + + def create_book(self, *, author_id: int, title: str) -> Optional[models.Book]: ... + + def create_books(self, arg: Sequence[CreateBooksParams]) -> None: ... + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> Iterator[Optional[models.Author]]: ... + + def get_book_with_author(self, *, id: int) -> Optional[GetBookWithAuthorRow]: ... + + def insert_authors_returning(self, arg: Sequence[InsertAuthorsReturningParams]) -> Iterator[Optional[models.Author]]: ... + + def list_authors(self) -> Iterator[models.Author]: ... + + def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> Iterator[List[models.Book]]: ... + + def set_book_title(self, *, id: int, title: str) -> None: ... + + +class AsyncQuerierProtocol(Protocol): + async def create_author(self, *, name: str, bio: Optional[str]) -> Optional[models.Author]: ... + + async def create_author_at(self, *, name: str, created_at: datetime.datetime) -> Optional[models.Author]: ... + + async def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: ... + + async def create_book(self, *, author_id: int, title: str) -> Optional[models.Book]: ... + + async def create_books(self, arg: Sequence[CreateBooksParams]) -> None: ... + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> AsyncIterator[Optional[models.Author]]: ... + + async def get_book_with_author(self, *, id: int) -> Optional[GetBookWithAuthorRow]: ... + + def insert_authors_returning(self, arg: Sequence[InsertAuthorsReturningParams]) -> AsyncIterator[Optional[models.Author]]: ... + + def list_authors(self) -> AsyncIterator[models.Author]: ... + + def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> AsyncIterator[List[models.Book]]: ... + + async def set_book_title(self, *, id: int, title: str) -> None: ... + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def create_author(self, *, name: str, bio: Optional[str]) -> Optional[models.Author]: + """Create one author.""" + with errors._wrap_errors("create_author"): + row = self._conn.execute(sqlalchemy.text(CREATE_AUTHOR), {"p1": name, "p2": bio}).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + created_at=cast(datetime.datetime, row[3]), + ) + + def create_author_at(self, *, name: str, created_at: datetime.datetime) -> Optional[models.Author]: + with errors._wrap_errors("create_author_at"): + row = self._conn.execute(sqlalchemy.text(CREATE_AUTHOR_AT), {"p1": name, "p2": created_at}).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + created_at=cast(datetime.datetime, row[3]), + ) + + def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: + with errors._wrap_errors("create_authors"): + if not arg: + return 0 + result = self._conn.execute(sqlalchemy.text(CREATE_AUTHORS), [{"p1": a.name, "p2": a.bio} for a in arg]) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + def create_book(self, *, author_id: int, title: str) -> Optional[models.Book]: + with errors._wrap_errors("create_book"): + row = self._conn.execute(sqlalchemy.text(CREATE_BOOK), {"p1": author_id, "p2": title}).first() + if row is None: + return None + return models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + ) + + def create_books(self, arg: Sequence[CreateBooksParams]) -> None: + with errors._wrap_errors("create_books"): + if not arg: + return None + self._conn.execute(sqlalchemy.text(CREATE_BOOKS), [{"p1": a.author_id, "p2": a.title, "p3": a.status} for a in arg]) + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> Iterator[Optional[models.Author]]: + with errors._wrap_errors("get_authors_by_id"): + for a in arg: + row = self._conn.execute(sqlalchemy.text(GET_AUTHORS_BY_ID), {"p1": a.id}).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + created_at=cast(datetime.datetime, row[3]), + ) + + def get_book_with_author(self, *, id: int) -> Optional[GetBookWithAuthorRow]: + with errors._wrap_errors("get_book_with_author"): + row = self._conn.execute(sqlalchemy.text(GET_BOOK_WITH_AUTHOR), {"p1": id}).first() + if row is None: + return None + return GetBookWithAuthorRow( + books=models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + ), + authors=models.Author( + id=cast(int, row[4]), + name=cast(str, row[5]), + bio=cast(Optional[str], row[6]), + created_at=cast(datetime.datetime, row[7]), + ), + ) + + def insert_authors_returning(self, arg: Sequence[InsertAuthorsReturningParams]) -> Iterator[Optional[models.Author]]: + with errors._wrap_errors("insert_authors_returning"): + for a in arg: + row = self._conn.execute(sqlalchemy.text(INSERT_AUTHORS_RETURNING), {"p1": a.name}).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + created_at=cast(datetime.datetime, row[3]), + ) + + def list_authors(self) -> Iterator[models.Author]: + with errors._wrap_errors("list_authors"): + result = self._conn.execute(sqlalchemy.text(LIST_AUTHORS)) + for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + created_at=cast(datetime.datetime, row[3]), + ) + + def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> Iterator[List[models.Book]]: + with errors._wrap_errors("list_books_by_author"): + for a in arg: + yield [models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + ) for row in self._conn.execute(sqlalchemy.text(LIST_BOOKS_BY_AUTHOR), {"p1": a.author_id})] + + def set_book_title(self, *, id: int, title: str) -> None: + with errors._wrap_errors("set_book_title"): + self._conn.execute(sqlalchemy.text(SET_BOOK_TITLE), {"p1": id, "p2": title}) + + +class AsyncQuerier: + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): + self._conn = conn + + async def create_author(self, *, name: str, bio: Optional[str]) -> Optional[models.Author]: + """Create one author.""" + with errors._wrap_errors("create_author"): + row = (await self._conn.execute(sqlalchemy.text(CREATE_AUTHOR), {"p1": name, "p2": bio})).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + created_at=cast(datetime.datetime, row[3]), + ) + + async def create_author_at(self, *, name: str, created_at: datetime.datetime) -> Optional[models.Author]: + with errors._wrap_errors("create_author_at"): + row = (await self._conn.execute(sqlalchemy.text(CREATE_AUTHOR_AT), {"p1": name, "p2": created_at})).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + created_at=cast(datetime.datetime, row[3]), + ) + + async def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: + with errors._wrap_errors("create_authors"): + if not arg: + return 0 + result = await self._conn.execute(sqlalchemy.text(CREATE_AUTHORS), [{"p1": a.name, "p2": a.bio} for a in arg]) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + async def create_book(self, *, author_id: int, title: str) -> Optional[models.Book]: + with errors._wrap_errors("create_book"): + row = (await self._conn.execute(sqlalchemy.text(CREATE_BOOK), {"p1": author_id, "p2": title})).first() + if row is None: + return None + return models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + ) + + async def create_books(self, arg: Sequence[CreateBooksParams]) -> None: + with errors._wrap_errors("create_books"): + if not arg: + return None + await self._conn.execute(sqlalchemy.text(CREATE_BOOKS), [{"p1": a.author_id, "p2": a.title, "p3": a.status} for a in arg]) + + async def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> AsyncIterator[Optional[models.Author]]: + with errors._wrap_errors("get_authors_by_id"): + for a in arg: + row = (await self._conn.execute(sqlalchemy.text(GET_AUTHORS_BY_ID), {"p1": a.id})).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + created_at=cast(datetime.datetime, row[3]), + ) + + async def get_book_with_author(self, *, id: int) -> Optional[GetBookWithAuthorRow]: + with errors._wrap_errors("get_book_with_author"): + row = (await self._conn.execute(sqlalchemy.text(GET_BOOK_WITH_AUTHOR), {"p1": id})).first() + if row is None: + return None + return GetBookWithAuthorRow( + books=models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + ), + authors=models.Author( + id=cast(int, row[4]), + name=cast(str, row[5]), + bio=cast(Optional[str], row[6]), + created_at=cast(datetime.datetime, row[7]), + ), + ) + + async def insert_authors_returning(self, arg: Sequence[InsertAuthorsReturningParams]) -> AsyncIterator[Optional[models.Author]]: + with errors._wrap_errors("insert_authors_returning"): + for a in arg: + row = (await self._conn.execute(sqlalchemy.text(INSERT_AUTHORS_RETURNING), {"p1": a.name})).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + created_at=cast(datetime.datetime, row[3]), + ) + + async def list_authors(self) -> AsyncIterator[models.Author]: + with errors._wrap_errors("list_authors"): + result = await self._conn.stream(sqlalchemy.text(LIST_AUTHORS)) + async for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + created_at=cast(datetime.datetime, row[3]), + ) + + async def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> AsyncIterator[List[models.Book]]: + with errors._wrap_errors("list_books_by_author"): + for a in arg: + yield [models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + ) for row in await self._conn.execute(sqlalchemy.text(LIST_BOOKS_BY_AUTHOR), {"p1": a.author_id})] + + async def set_book_title(self, *, id: int, title: str) -> None: + with errors._wrap_errors("set_book_title"): + await self._conn.execute(sqlalchemy.text(SET_BOOK_TITLE), {"p1": id, "p2": title}) diff --git a/examples/src/library/query.sql b/examples/src/library/query.sql new file mode 100644 index 0000000..e3c9579 --- /dev/null +++ b/examples/src/library/query.sql @@ -0,0 +1,35 @@ +-- Create one author. +-- name: CreateAuthor :one +INSERT INTO authors (name, bio) VALUES ($1, $2) RETURNING *; + +-- name: CreateAuthorAt :one +INSERT INTO authors (name, created_at) VALUES ($1, $2) RETURNING *; + +-- name: ListAuthors :many +SELECT * FROM authors ORDER BY name; + +-- name: CreateAuthors :copyfrom +INSERT INTO authors (name, bio) VALUES ($1, $2); + +-- name: CreateBooks :batchexec +INSERT INTO books (author_id, title, status) VALUES ($1, $2, $3); + +-- name: CreateBook :one +INSERT INTO books (author_id, title) VALUES ($1, $2) RETURNING *; + +-- name: SetBookTitle :exec +UPDATE books SET title = $2 WHERE id = $1; + +-- name: GetAuthorsByID :batchone +SELECT * FROM authors WHERE id = $1; + +-- name: ListBooksByAuthor :batchmany +SELECT * FROM books WHERE author_id = $1 ORDER BY id; + +-- name: GetBookWithAuthor :one +SELECT sqlc.embed(books), sqlc.embed(authors) +FROM books JOIN authors ON authors.id = books.author_id +WHERE books.id = $1; + +-- name: InsertAuthorsReturning :batchone +INSERT INTO authors (name) VALUES ($1) RETURNING *; diff --git a/examples/src/library/schema.sql b/examples/src/library/schema.sql new file mode 100644 index 0000000..4f97a67 --- /dev/null +++ b/examples/src/library/schema.sql @@ -0,0 +1,18 @@ +CREATE TYPE book_status AS ENUM ('draft', 'in-print', 'out_of_print'); + +CREATE TABLE authors ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL, + bio text, + created_at timestamptz NOT NULL DEFAULT now(), + CONSTRAINT authors_name_key UNIQUE (name) +); + +CREATE TABLE books ( + id BIGSERIAL PRIMARY KEY, + author_id bigint NOT NULL, + title text NOT NULL, + status book_status NOT NULL DEFAULT 'draft', + CONSTRAINT books_author_id_fkey FOREIGN KEY (author_id) REFERENCES authors (id), + CONSTRAINT books_title_check CHECK (title <> '') +); diff --git a/examples/src/library_modern/__init__.py b/examples/src/library_modern/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/examples/src/library_modern/errors.py b/examples/src/library_modern/errors.py new file mode 100644 index 0000000..bc58233 --- /dev/null +++ b/examples/src/library_modern/errors.py @@ -0,0 +1,108 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import contextlib +from collections.abc import Iterator + +import sqlalchemy.exc + + +class QueryError(Exception): + """Base class for all query errors.""" + + def __init__(self, message: str, query_name: str, cause: Exception | None = None): + super().__init__(message) + self.query_name = query_name + self.cause = cause + self.constraint_name: str | None = _constraint_name(cause) + + +class UniqueViolationError(QueryError): + """Raised on unique constraint violation (PG 23505).""" + + +class ForeignKeyViolationError(QueryError): + """Raised on foreign key constraint violation (PG 23503).""" + + +class CheckViolationError(QueryError): + """Raised on check constraint violation (PG 23514).""" + + +class NotNullViolationError(QueryError): + """Raised on not-null constraint violation (PG 23502).""" + + +class ExclusionViolationError(QueryError): + """Raised on exclusion constraint violation (PG 23P01).""" + + +class StatementTimeoutError(QueryError): + """Raised on statement timeout (PG 57014).""" + + +class DeadlockError(QueryError): + """Raised on deadlock detected (PG 40P01).""" + + +class SerializationError(QueryError): + """Raised on serialization failure (PG 40001).""" + + +def _sqlstate(e: sqlalchemy.exc.DBAPIError) -> str | None: + # psycopg2 exposes pgcode, psycopg 3 sqlstate; SQLAlchemy's asyncpg + # adapter keeps the asyncpg error, which has sqlstate, as __cause__. + for source in (e.orig, getattr(e.orig, "__cause__", None)): + code = getattr(source, "pgcode", None) or getattr(source, "sqlstate", None) + if isinstance(code, str): + return code + return None + + +def _constraint_name(cause: Exception | None) -> str | None: + orig = getattr(cause, "orig", None) + name = getattr(getattr(orig, "diag", None), "constraint_name", None) + if name is None: + name = getattr(getattr(orig, "__cause__", None), "constraint_name", None) + return name if isinstance(name, str) else None + + +def _wrap_integrity_error(e: sqlalchemy.exc.IntegrityError, query_name: str) -> QueryError: + pgcode = _sqlstate(e) + if pgcode == "23505": + return UniqueViolationError(str(e), query_name, cause=e) + if pgcode == "23503": + return ForeignKeyViolationError(str(e), query_name, cause=e) + if pgcode == "23514": + return CheckViolationError(str(e), query_name, cause=e) + if pgcode == "23502": + return NotNullViolationError(str(e), query_name, cause=e) + if pgcode == "23P01": + return ExclusionViolationError(str(e), query_name, cause=e) + return QueryError(str(e), query_name, cause=e) + + +def _wrap_operational_error(e: sqlalchemy.exc.OperationalError, query_name: str) -> QueryError: + pgcode = _sqlstate(e) + if pgcode == "57014": + return StatementTimeoutError(str(e), query_name, cause=e) + if pgcode == "40P01": + return DeadlockError(str(e), query_name, cause=e) + if pgcode == "40001": + return SerializationError(str(e), query_name, cause=e) + return QueryError(str(e), query_name, cause=e) + + +@contextlib.contextmanager +def _wrap_errors(query_name: str) -> Iterator[None]: + """Re-raise integrity and operational errors as typed QueryErrors. + + Only those two SQLAlchemy exceptions are caught, so GeneratorExit from a + consumer that stops iterating early passes through untouched. + """ + try: + yield + except sqlalchemy.exc.IntegrityError as e: + raise _wrap_integrity_error(e, query_name) from e + except sqlalchemy.exc.OperationalError as e: + raise _wrap_operational_error(e, query_name) from e diff --git a/examples/src/library_modern/models.py b/examples/src/library_modern/models.py new file mode 100644 index 0000000..ea48804 --- /dev/null +++ b/examples/src/library_modern/models.py @@ -0,0 +1,25 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import enum +import pydantic + + +class BookStatus(enum.StrEnum): + DRAFT = "draft" + IN_PRINT = "in-print" + OUT_OF_PRINT = "out_of_print" + + +class Author(pydantic.BaseModel): + id: int + name: str + bio: str | None + created_at: pydantic.AwareDatetime + + +class Book(pydantic.BaseModel): + id: int + author_id: int + title: str + status: BookStatus diff --git a/examples/src/library_modern/query.py b/examples/src/library_modern/query.py new file mode 100644 index 0000000..0985dee --- /dev/null +++ b/examples/src/library_modern/query.py @@ -0,0 +1,406 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import pydantic +from collections.abc import AsyncIterator, Iterator, Sequence +from typing import Any, Protocol, cast + +import sqlalchemy +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from library_modern import errors, models + + +CREATE_AUTHOR = """-- name: create_author \\:one +INSERT INTO authors (name, bio) VALUES (:p1, :p2) RETURNING id, name, bio, created_at +""" + + +CREATE_AUTHOR_AT = """-- name: create_author_at \\:one +INSERT INTO authors (name, created_at) VALUES (:p1, :p2) RETURNING id, name, bio, created_at +""" + + +CREATE_AUTHORS = """-- name: create_authors \\:copyfrom +INSERT INTO authors (name, bio) VALUES (:p1, :p2) +""" + + +class CreateAuthorsParams(pydantic.BaseModel): + name: str + bio: str | None + + +CREATE_BOOK = """-- name: create_book \\:one +INSERT INTO books (author_id, title) VALUES (:p1, :p2) RETURNING id, author_id, title, status +""" + + +CREATE_BOOKS = """-- name: create_books \\:batchexec +INSERT INTO books (author_id, title, status) VALUES (:p1, :p2, :p3) +""" + + +class CreateBooksParams(pydantic.BaseModel): + author_id: int + title: str + status: models.BookStatus + + +GET_AUTHORS_BY_ID = """-- name: get_authors_by_id \\:batchone +SELECT id, name, bio, created_at FROM authors WHERE id = :p1 +""" + + +class GetAuthorsByIDParams(pydantic.BaseModel): + id: int + + +GET_BOOK_WITH_AUTHOR = """-- name: get_book_with_author \\:one +SELECT books.id, books.author_id, books.title, books.status, authors.id, authors.name, authors.bio, authors.created_at +FROM books JOIN authors ON authors.id = books.author_id +WHERE books.id = :p1 +""" + + +class GetBookWithAuthorRow(pydantic.BaseModel): + books: models.Book + authors: models.Author + + +INSERT_AUTHORS_RETURNING = """-- name: insert_authors_returning \\:batchone +INSERT INTO authors (name) VALUES (:p1) RETURNING id, name, bio, created_at +""" + + +class InsertAuthorsReturningParams(pydantic.BaseModel): + name: str + + +LIST_AUTHORS = """-- name: list_authors \\:many +SELECT id, name, bio, created_at FROM authors ORDER BY name +""" + + +LIST_BOOKS_BY_AUTHOR = """-- name: list_books_by_author \\:batchmany +SELECT id, author_id, title, status FROM books WHERE author_id = :p1 ORDER BY id +""" + + +class ListBooksByAuthorParams(pydantic.BaseModel): + author_id: int + + +SET_BOOK_TITLE = """-- name: set_book_title \\:exec +UPDATE books SET title = :p2 WHERE id = :p1 +""" + + +class QuerierProtocol(Protocol): + def create_author(self, *, name: str, bio: str | None) -> models.Author | None: ... + + def create_author_at(self, *, name: str, created_at: pydantic.AwareDatetime) -> models.Author | None: ... + + def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: ... + + def create_book(self, *, author_id: int, title: str) -> models.Book | None: ... + + def create_books(self, arg: Sequence[CreateBooksParams]) -> None: ... + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> Iterator[models.Author | None]: ... + + def get_book_with_author(self, *, id: int) -> GetBookWithAuthorRow | None: ... + + def insert_authors_returning(self, arg: Sequence[InsertAuthorsReturningParams]) -> Iterator[models.Author | None]: ... + + def list_authors(self) -> Iterator[models.Author]: ... + + def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> Iterator[list[models.Book]]: ... + + def set_book_title(self, *, id: int, title: str) -> None: ... + + +class AsyncQuerierProtocol(Protocol): + async def create_author(self, *, name: str, bio: str | None) -> models.Author | None: ... + + async def create_author_at(self, *, name: str, created_at: pydantic.AwareDatetime) -> models.Author | None: ... + + async def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: ... + + async def create_book(self, *, author_id: int, title: str) -> models.Book | None: ... + + async def create_books(self, arg: Sequence[CreateBooksParams]) -> None: ... + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> AsyncIterator[models.Author | None]: ... + + async def get_book_with_author(self, *, id: int) -> GetBookWithAuthorRow | None: ... + + def insert_authors_returning(self, arg: Sequence[InsertAuthorsReturningParams]) -> AsyncIterator[models.Author | None]: ... + + def list_authors(self) -> AsyncIterator[models.Author]: ... + + def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> AsyncIterator[list[models.Book]]: ... + + async def set_book_title(self, *, id: int, title: str) -> None: ... + + +class Querier[_ConnT: sqlalchemy.engine.Connection | sqlalchemy.orm.Session]: + _conn: _ConnT + + def __init__(self, conn: _ConnT): + self._conn = conn + + def create_author(self, *, name: str, bio: str | None) -> models.Author | None: + """Create one author.""" + with errors._wrap_errors("create_author"): + row = self._conn.execute(sqlalchemy.text(CREATE_AUTHOR), {"p1": name, "p2": bio}).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + def create_author_at(self, *, name: str, created_at: pydantic.AwareDatetime) -> models.Author | None: + with errors._wrap_errors("create_author_at"): + row = self._conn.execute(sqlalchemy.text(CREATE_AUTHOR_AT), {"p1": name, "p2": created_at}).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: + with errors._wrap_errors("create_authors"): + if not arg: + return 0 + result = self._conn.execute(sqlalchemy.text(CREATE_AUTHORS), [{"p1": a.name, "p2": a.bio} for a in arg]) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + def create_book(self, *, author_id: int, title: str) -> models.Book | None: + with errors._wrap_errors("create_book"): + row = self._conn.execute(sqlalchemy.text(CREATE_BOOK), {"p1": author_id, "p2": title}).first() + if row is None: + return None + return models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + ) + + def create_books(self, arg: Sequence[CreateBooksParams]) -> None: + with errors._wrap_errors("create_books"): + if not arg: + return None + self._conn.execute(sqlalchemy.text(CREATE_BOOKS), [{"p1": a.author_id, "p2": a.title, "p3": a.status} for a in arg]) + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> Iterator[models.Author | None]: + with errors._wrap_errors("get_authors_by_id"): + for a in arg: + row = self._conn.execute(sqlalchemy.text(GET_AUTHORS_BY_ID), {"p1": a.id}).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + def get_book_with_author(self, *, id: int) -> GetBookWithAuthorRow | None: + with errors._wrap_errors("get_book_with_author"): + row = self._conn.execute(sqlalchemy.text(GET_BOOK_WITH_AUTHOR), {"p1": id}).first() + if row is None: + return None + return GetBookWithAuthorRow( + books=models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + ), + authors=models.Author( + id=cast(int, row[4]), + name=cast(str, row[5]), + bio=cast(str | None, row[6]), + created_at=cast(pydantic.AwareDatetime, row[7]), + ), + ) + + def insert_authors_returning(self, arg: Sequence[InsertAuthorsReturningParams]) -> Iterator[models.Author | None]: + with errors._wrap_errors("insert_authors_returning"): + for a in arg: + row = self._conn.execute(sqlalchemy.text(INSERT_AUTHORS_RETURNING), {"p1": a.name}).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + def list_authors(self) -> Iterator[models.Author]: + with errors._wrap_errors("list_authors"): + result = self._conn.execute(sqlalchemy.text(LIST_AUTHORS)) + for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> Iterator[list[models.Book]]: + with errors._wrap_errors("list_books_by_author"): + for a in arg: + yield [models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + ) for row in self._conn.execute(sqlalchemy.text(LIST_BOOKS_BY_AUTHOR), {"p1": a.author_id})] + + def set_book_title(self, *, id: int, title: str) -> None: + with errors._wrap_errors("set_book_title"): + self._conn.execute(sqlalchemy.text(SET_BOOK_TITLE), {"p1": id, "p2": title}) + + +class AsyncQuerier[_ConnT: sqlalchemy.ext.asyncio.AsyncConnection | sqlalchemy.ext.asyncio.AsyncSession]: + _conn: _ConnT + + def __init__(self, conn: _ConnT): + self._conn = conn + + async def create_author(self, *, name: str, bio: str | None) -> models.Author | None: + """Create one author.""" + with errors._wrap_errors("create_author"): + row = (await self._conn.execute(sqlalchemy.text(CREATE_AUTHOR), {"p1": name, "p2": bio})).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + async def create_author_at(self, *, name: str, created_at: pydantic.AwareDatetime) -> models.Author | None: + with errors._wrap_errors("create_author_at"): + row = (await self._conn.execute(sqlalchemy.text(CREATE_AUTHOR_AT), {"p1": name, "p2": created_at})).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + async def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: + with errors._wrap_errors("create_authors"): + if not arg: + return 0 + result = await self._conn.execute(sqlalchemy.text(CREATE_AUTHORS), [{"p1": a.name, "p2": a.bio} for a in arg]) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + async def create_book(self, *, author_id: int, title: str) -> models.Book | None: + with errors._wrap_errors("create_book"): + row = (await self._conn.execute(sqlalchemy.text(CREATE_BOOK), {"p1": author_id, "p2": title})).first() + if row is None: + return None + return models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + ) + + async def create_books(self, arg: Sequence[CreateBooksParams]) -> None: + with errors._wrap_errors("create_books"): + if not arg: + return None + await self._conn.execute(sqlalchemy.text(CREATE_BOOKS), [{"p1": a.author_id, "p2": a.title, "p3": a.status} for a in arg]) + + async def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> AsyncIterator[models.Author | None]: + with errors._wrap_errors("get_authors_by_id"): + for a in arg: + row = (await self._conn.execute(sqlalchemy.text(GET_AUTHORS_BY_ID), {"p1": a.id})).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + async def get_book_with_author(self, *, id: int) -> GetBookWithAuthorRow | None: + with errors._wrap_errors("get_book_with_author"): + row = (await self._conn.execute(sqlalchemy.text(GET_BOOK_WITH_AUTHOR), {"p1": id})).first() + if row is None: + return None + return GetBookWithAuthorRow( + books=models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + ), + authors=models.Author( + id=cast(int, row[4]), + name=cast(str, row[5]), + bio=cast(str | None, row[6]), + created_at=cast(pydantic.AwareDatetime, row[7]), + ), + ) + + async def insert_authors_returning(self, arg: Sequence[InsertAuthorsReturningParams]) -> AsyncIterator[models.Author | None]: + with errors._wrap_errors("insert_authors_returning"): + for a in arg: + row = (await self._conn.execute(sqlalchemy.text(INSERT_AUTHORS_RETURNING), {"p1": a.name})).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + async def list_authors(self) -> AsyncIterator[models.Author]: + with errors._wrap_errors("list_authors"): + result = await self._conn.stream(sqlalchemy.text(LIST_AUTHORS)) + async for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + async def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> AsyncIterator[list[models.Book]]: + with errors._wrap_errors("list_books_by_author"): + for a in arg: + yield [models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + ) for row in await self._conn.execute(sqlalchemy.text(LIST_BOOKS_BY_AUTHOR), {"p1": a.author_id})] + + async def set_book_title(self, *, id: int, title: str) -> None: + with errors._wrap_errors("set_book_title"): + await self._conn.execute(sqlalchemy.text(SET_BOOK_TITLE), {"p1": id, "p2": title}) diff --git a/examples/src/library_modern/query.sql b/examples/src/library_modern/query.sql new file mode 100644 index 0000000..e3c9579 --- /dev/null +++ b/examples/src/library_modern/query.sql @@ -0,0 +1,35 @@ +-- Create one author. +-- name: CreateAuthor :one +INSERT INTO authors (name, bio) VALUES ($1, $2) RETURNING *; + +-- name: CreateAuthorAt :one +INSERT INTO authors (name, created_at) VALUES ($1, $2) RETURNING *; + +-- name: ListAuthors :many +SELECT * FROM authors ORDER BY name; + +-- name: CreateAuthors :copyfrom +INSERT INTO authors (name, bio) VALUES ($1, $2); + +-- name: CreateBooks :batchexec +INSERT INTO books (author_id, title, status) VALUES ($1, $2, $3); + +-- name: CreateBook :one +INSERT INTO books (author_id, title) VALUES ($1, $2) RETURNING *; + +-- name: SetBookTitle :exec +UPDATE books SET title = $2 WHERE id = $1; + +-- name: GetAuthorsByID :batchone +SELECT * FROM authors WHERE id = $1; + +-- name: ListBooksByAuthor :batchmany +SELECT * FROM books WHERE author_id = $1 ORDER BY id; + +-- name: GetBookWithAuthor :one +SELECT sqlc.embed(books), sqlc.embed(authors) +FROM books JOIN authors ON authors.id = books.author_id +WHERE books.id = $1; + +-- name: InsertAuthorsReturning :batchone +INSERT INTO authors (name) VALUES ($1) RETURNING *; diff --git a/examples/src/library_modern/schema.sql b/examples/src/library_modern/schema.sql new file mode 100644 index 0000000..4f97a67 --- /dev/null +++ b/examples/src/library_modern/schema.sql @@ -0,0 +1,18 @@ +CREATE TYPE book_status AS ENUM ('draft', 'in-print', 'out_of_print'); + +CREATE TABLE authors ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL, + bio text, + created_at timestamptz NOT NULL DEFAULT now(), + CONSTRAINT authors_name_key UNIQUE (name) +); + +CREATE TABLE books ( + id BIGSERIAL PRIMARY KEY, + author_id bigint NOT NULL, + title text NOT NULL, + status book_status NOT NULL DEFAULT 'draft', + CONSTRAINT books_author_id_fkey FOREIGN KEY (author_id) REFERENCES authors (id), + CONSTRAINT books_title_check CHECK (title <> '') +); diff --git a/examples/src/ondeck/city.py b/examples/src/ondeck/city.py index 5af93e9..676b7ed 100644 --- a/examples/src/ondeck/city.py +++ b/examples/src/ondeck/city.py @@ -1,8 +1,8 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 # source: city.sql -from typing import AsyncIterator, Optional +from typing import AsyncIterator, Optional, Union, cast import sqlalchemy import sqlalchemy.ext.asyncio @@ -43,16 +43,21 @@ class AsyncQuerier: - def __init__(self, conn: sqlalchemy.ext.asyncio.AsyncConnection): + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): self._conn = conn async def create_city(self, *, name: str, slug: str) -> Optional[models.City]: + """Create a new city. The slug must be unique. +This is the second line of the comment +This is the third line""" row = (await self._conn.execute(sqlalchemy.text(CREATE_CITY), {"p1": name, "p2": slug})).first() if row is None: return None return models.City( - slug=row[0], - name=row[1], + slug=cast(str, row[0]), + name=cast(str, row[1]), ) async def get_city(self, *, slug: str) -> Optional[models.City]: @@ -60,16 +65,16 @@ async def get_city(self, *, slug: str) -> Optional[models.City]: if row is None: return None return models.City( - slug=row[0], - name=row[1], + slug=cast(str, row[0]), + name=cast(str, row[1]), ) async def list_cities(self) -> AsyncIterator[models.City]: result = await self._conn.stream(sqlalchemy.text(LIST_CITIES)) async for row in result: yield models.City( - slug=row[0], - name=row[1], + slug=cast(str, row[0]), + name=cast(str, row[1]), ) async def update_city_name(self, *, slug: str, name: str) -> None: diff --git a/examples/src/ondeck/models.py b/examples/src/ondeck/models.py index 1161408..cc4c159 100644 --- a/examples/src/ondeck/models.py +++ b/examples/src/ondeck/models.py @@ -1,6 +1,6 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 import dataclasses import datetime import enum diff --git a/examples/src/ondeck/schema/0002_venue.sql b/examples/src/ondeck/schema/0002_venue.sql index 940de7a..ca9d10d 100644 --- a/examples/src/ondeck/schema/0002_venue.sql +++ b/examples/src/ondeck/schema/0002_venue.sql @@ -15,4 +15,3 @@ CREATE TABLE venues ( ); COMMENT ON TABLE venues IS 'Venues are places where muisc happens'; COMMENT ON COLUMN venues.slug IS 'This value appears in public URLs'; - diff --git a/examples/src/ondeck/venue.py b/examples/src/ondeck/venue.py index 6159bf6..28157ef 100644 --- a/examples/src/ondeck/venue.py +++ b/examples/src/ondeck/venue.py @@ -1,9 +1,10 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 # source: venue.sql import dataclasses -from typing import AsyncIterator, List, Optional +import datetime +from typing import AsyncIterator, List, Optional, Union, cast import sqlalchemy import sqlalchemy.ext.asyncio @@ -91,7 +92,9 @@ class VenueCountByCityRow: class AsyncQuerier: - def __init__(self, conn: sqlalchemy.ext.asyncio.AsyncConnection): + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): self._conn = conn async def create_venue(self, arg: CreateVenueParams) -> Optional[int]: @@ -106,7 +109,7 @@ async def create_venue(self, arg: CreateVenueParams) -> Optional[int]: })).first() if row is None: return None - return row[0] + return cast(int, row[0]) async def delete_venue(self, *, slug: str) -> None: await self._conn.execute(sqlalchemy.text(DELETE_VENUE), {"p1": slug}) @@ -116,44 +119,44 @@ async def get_venue(self, *, slug: str, city: str) -> Optional[models.Venue]: if row is None: return None return models.Venue( - id=row[0], - status=row[1], - statuses=row[2], - slug=row[3], - name=row[4], - city=row[5], - spotify_playlist=row[6], - songkick_id=row[7], - tags=row[8], - created_at=row[9], + id=cast(int, row[0]), + status=cast(models.Status, row[1]), + statuses=cast(Optional[List[models.Status]], row[2]), + slug=cast(str, row[3]), + name=cast(str, row[4]), + city=cast(str, row[5]), + spotify_playlist=cast(str, row[6]), + songkick_id=cast(Optional[str], row[7]), + tags=cast(Optional[List[str]], row[8]), + created_at=cast(datetime.datetime, row[9]), ) async def list_venues(self, *, city: str) -> AsyncIterator[models.Venue]: result = await self._conn.stream(sqlalchemy.text(LIST_VENUES), {"p1": city}) async for row in result: yield models.Venue( - id=row[0], - status=row[1], - statuses=row[2], - slug=row[3], - name=row[4], - city=row[5], - spotify_playlist=row[6], - songkick_id=row[7], - tags=row[8], - created_at=row[9], + id=cast(int, row[0]), + status=cast(models.Status, row[1]), + statuses=cast(Optional[List[models.Status]], row[2]), + slug=cast(str, row[3]), + name=cast(str, row[4]), + city=cast(str, row[5]), + spotify_playlist=cast(str, row[6]), + songkick_id=cast(Optional[str], row[7]), + tags=cast(Optional[List[str]], row[8]), + created_at=cast(datetime.datetime, row[9]), ) async def update_venue_name(self, *, slug: str, name: str) -> Optional[int]: row = (await self._conn.execute(sqlalchemy.text(UPDATE_VENUE_NAME), {"p1": slug, "p2": name})).first() if row is None: return None - return row[0] + return cast(int, row[0]) async def venue_count_by_city(self) -> AsyncIterator[VenueCountByCityRow]: result = await self._conn.stream(sqlalchemy.text(VENUE_COUNT_BY_CITY)) async for row in result: yield VenueCountByCityRow( - city=row[0], - count=row[1], + city=cast(str, row[0]), + count=cast(int, row[1]), ) diff --git a/examples/src/tests/conftest.py b/examples/src/tests/conftest.py index f807209..0f94c28 100644 --- a/examples/src/tests/conftest.py +++ b/examples/src/tests/conftest.py @@ -1,4 +1,3 @@ -import asyncio import os import random @@ -15,7 +14,7 @@ def postgres_uri() -> str: pg_password = os.environ.get("PG_PASSWORD", "mysecretpassword") pg_db = os.environ.get("PG_DATABASE", "dinotest") - return f"postgresql://{pg_user}:{pg_password}@{pg_host}:{pg_port}/{pg_db}" + return f"postgresql+psycopg://{pg_user}:{pg_password}@{pg_host}:{pg_port}/{pg_db}" @pytest.fixture(scope="session") @@ -40,7 +39,7 @@ def db(sqlalchemy_connection: sqlalchemy.engine.Connection) -> sqlalchemy.engine @pytest.fixture(scope="session") async def async_sqlalchemy_connection(postgres_uri) -> sqlalchemy.ext.asyncio.AsyncConnection: - postgres_uri = postgres_uri.replace("postgresql", "postgresql+asyncpg") + postgres_uri = postgres_uri.replace("postgresql+psycopg", "postgresql+asyncpg") engine = sqlalchemy.ext.asyncio.create_async_engine(postgres_uri) async with engine.connect() as conn: yield conn @@ -57,11 +56,3 @@ async def async_db(async_sqlalchemy_connection: sqlalchemy.ext.asyncio.AsyncConn await conn.rollback() await conn.execute(sqlalchemy.text(f"DROP SCHEMA {schema_name} CASCADE")) await conn.execute(sqlalchemy.text("SET search_path TO public")) - - -@pytest.fixture(scope="session") -def event_loop(): - """Change event_loop fixture to session level.""" - loop = asyncio.get_event_loop_policy().new_event_loop() - yield loop - loop.close() diff --git a/examples/src/tests/test_library.py b/examples/src/tests/test_library.py new file mode 100644 index 0000000..ede6678 --- /dev/null +++ b/examples/src/tests/test_library.py @@ -0,0 +1,281 @@ +import datetime +import os +from typing import Iterator, List, Optional, Sequence + +import pydantic +import pytest +import sqlalchemy +import sqlalchemy.event +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from dbtest.migrations import apply_migrations, apply_migrations_async +from library import errors, models, query +from library_modern import errors as modern_errors +from library_modern import models as modern_models +from library_modern import query as modern_query + +SCHEMA = os.path.dirname(__file__) + "/../library/schema.sql" + + +class StatementLog: + def __init__(self, conn: sqlalchemy.engine.Connection): + self.statements: List[bool] = [] + sqlalchemy.event.listen(conn, "before_cursor_execute", self._record) + + def _record(self, conn, cursor, statement, parameters, context, executemany): + self.statements.append(executemany) + + +@pytest.fixture +def q(db: sqlalchemy.engine.Connection) -> query.Querier: + apply_migrations(db, [SCHEMA]) + return query.Querier(db) + + +@pytest.fixture +async def aq(async_db: sqlalchemy.ext.asyncio.AsyncConnection) -> query.AsyncQuerier: + await apply_migrations_async(async_db, [SCHEMA]) + return query.AsyncQuerier(async_db) + + +def test_copyfrom_and_batchexec_use_one_executemany(db, q): + log = StatementLog(db) + assert q.create_authors([]) == 0 + q.create_books([]) + assert log.statements == [] + + count = q.create_authors([ + query.CreateAuthorsParams(name="Ursula", bio=None), + query.CreateAuthorsParams(name="Octavia", bio="Kindred"), + ]) + assert count == 2 + authors = {a.name: a for a in q.list_authors()} + log.statements.clear() + q.create_books([ + query.CreateBooksParams(author_id=authors["Ursula"].id, title="The Dispossessed", status=models.BookStatus.IN_PRINT), + query.CreateBooksParams(author_id=authors["Octavia"].id, title="Kindred", status=models.BookStatus.DRAFT), + ]) + assert log.statements == [True] + + +def test_batchone_is_lazy_and_ordered(db, q): + a = q.create_author(name="Ursula", bio=None) + b = q.create_author(name="Octavia", bio=None) + assert a is not None and b is not None + log = StatementLog(db) + + results = q.get_authors_by_id([ + query.GetAuthorsByIDParams(id=a.id), + query.GetAuthorsByIDParams(id=-1), + query.GetAuthorsByIDParams(id=b.id), + ]) + assert log.statements == [] + assert next(results) == a + assert len(log.statements) == 1 + assert list(results) == [None, b] + assert len(log.statements) == 3 + + +def test_batchmany_yields_a_list_per_input(q): + a = q.create_author(name="Ursula", bio=None) + b = q.create_author(name="Octavia", bio=None) + assert a is not None and b is not None + first = q.create_book(author_id=a.id, title="The Dispossessed") + second = q.create_book(author_id=a.id, title="The Lathe of Heaven") + + results = list(q.list_books_by_author([ + query.ListBooksByAuthorParams(author_id=a.id), + query.ListBooksByAuthorParams(author_id=b.id), + ])) + assert results == [[first, second], []] + + +def test_embed(q): + a = q.create_author(name="Ursula", bio="Earthsea") + assert a is not None + book = q.create_book(author_id=a.id, title="The Dispossessed") + assert book is not None + row = q.get_book_with_author(id=book.id) + assert row == query.GetBookWithAuthorRow(books=book, authors=a) + + +def test_query_docstring(): + assert query.Querier.create_author.__doc__ == "Create one author." + + +def test_typed_errors(db, q): + a = q.create_author(name="Ursula", bio=None) + assert a is not None + + with pytest.raises(errors.UniqueViolationError) as unique: + with db.begin_nested(): + q.create_author(name="Ursula", bio=None) + assert unique.value.query_name == "create_author" + assert unique.value.constraint_name == "authors_name_key" + assert isinstance(unique.value.cause, sqlalchemy.exc.IntegrityError) + assert unique.value.__cause__ is unique.value.cause + + with pytest.raises(errors.ForeignKeyViolationError) as fk: + with db.begin_nested(): + q.create_book(author_id=-1, title="Orphan") + assert fk.value.constraint_name == "books_author_id_fkey" + + with pytest.raises(errors.NotNullViolationError): + with db.begin_nested(): + q.create_author(name=None, bio=None) # type: ignore[arg-type] + + book = q.create_book(author_id=a.id, title="The Dispossessed") + assert book is not None + with pytest.raises(errors.CheckViolationError) as check: + with db.begin_nested(): + q.set_book_title(id=book.id, title="") + assert check.value.constraint_name == "books_title_check" + + with pytest.raises(errors.QueryError): + with db.begin_nested(): + q.create_authors([query.CreateAuthorsParams(name="Ursula", bio=None)]) + + +def test_error_while_iterating_is_wrapped(db, q): + results = q.insert_authors_returning([ + query.InsertAuthorsReturningParams(name="Ursula"), + query.InsertAuthorsReturningParams(name="Ursula"), + ]) + with db.begin_nested(): + first = next(results) + assert first is not None and first.name == "Ursula" + with pytest.raises(errors.UniqueViolationError) as unique: + with db.begin_nested(): + next(results) + assert unique.value.query_name == "insert_authors_returning" + + +async def test_async_error_while_iterating_is_wrapped(async_db, aq): + names = [query.InsertAuthorsReturningParams(name="Ursula")] * 2 + with pytest.raises(errors.UniqueViolationError): + async with async_db.begin_nested(): + async for _ in aq.insert_authors_returning(names): + pass + + +def test_other_errors_pass_through(db, q): + with pytest.raises(sqlalchemy.exc.ProgrammingError): + with db.begin_nested(): + db.execute(sqlalchemy.text("DROP TABLE books")) + list(q.list_books_by_author([query.ListBooksByAuthorParams(author_id=1)])) + + +async def test_async_batches_and_errors(async_db, aq): + a = await aq.create_author(name="Ursula", bio=None) + assert a is not None + assert await aq.create_authors([query.CreateAuthorsParams(name="Octavia", bio=None)]) == 1 + + found = [x async for x in aq.get_authors_by_id([query.GetAuthorsByIDParams(id=a.id), query.GetAuthorsByIDParams(id=-1)])] + assert found == [a, None] + books = [x async for x in aq.list_books_by_author([query.ListBooksByAuthorParams(author_id=a.id)])] + assert books == [[]] + + with pytest.raises(errors.UniqueViolationError) as unique: + async with async_db.begin_nested(): + await aq.create_author(name="Ursula", bio=None) + assert unique.value.constraint_name == "authors_name_key" + + with pytest.raises(errors.ForeignKeyViolationError) as fk: + async with async_db.begin_nested(): + await aq.create_books([query.CreateBooksParams(author_id=-1, title="Orphan", status=models.BookStatus.DRAFT)]) + assert fk.value.constraint_name == "books_author_id_fkey" + + with pytest.raises(errors.NotNullViolationError): + async with async_db.begin_nested(): + await aq.create_author(name=None, bio=None) # type: ignore[arg-type] + + +def test_session_backed_querier(db): + apply_migrations(db, [SCHEMA]) + with sqlalchemy.orm.Session(bind=db) as session: + q = query.Querier(session) + a = q.create_author(name="Ursula", bio=None) + assert a is not None + assert q.create_authors([query.CreateAuthorsParams(name="Octavia", bio=None)]) == 1 + assert [x.name for x in q.list_authors()] == ["Octavia", "Ursula"] + assert list(q.get_authors_by_id([query.GetAuthorsByIDParams(id=a.id)])) == [a] + + +class FakeQuerier: + def __init__(self) -> None: + self.authors: dict = {} + + def create_author(self, *, name: str, bio: Optional[str]) -> Optional[models.Author]: + author = models.Author(id=len(self.authors) + 1, name=name, bio=bio, created_at=datetime.datetime.now(datetime.timezone.utc)) + self.authors[author.id] = author + return author + + def create_author_at(self, *, name: str, created_at: datetime.datetime) -> Optional[models.Author]: + raise NotImplementedError + + def list_authors(self) -> Iterator[models.Author]: + yield from sorted(self.authors.values(), key=lambda a: a.name) + + def create_authors(self, arg: Sequence[query.CreateAuthorsParams]) -> int: + for a in arg: + self.create_author(name=a.name, bio=a.bio) + return len(arg) + + def create_books(self, arg: Sequence[query.CreateBooksParams]) -> None: + raise NotImplementedError + + def create_book(self, *, author_id: int, title: str) -> Optional[models.Book]: + raise NotImplementedError + + def set_book_title(self, *, id: int, title: str) -> None: + raise NotImplementedError + + def get_authors_by_id(self, arg: Sequence[query.GetAuthorsByIDParams]) -> Iterator[Optional[models.Author]]: + for a in arg: + yield self.authors.get(a.id) + + def list_books_by_author(self, arg: Sequence[query.ListBooksByAuthorParams]) -> Iterator[List[models.Book]]: + for _ in arg: + yield [] + + def get_book_with_author(self, *, id: int) -> Optional[query.GetBookWithAuthorRow]: + return None + + def insert_authors_returning(self, arg: Sequence[query.InsertAuthorsReturningParams]) -> Iterator[Optional[models.Author]]: + raise NotImplementedError + + +def author_names(q: query.QuerierProtocol) -> List[str]: + return [a.name for a in q.list_authors()] + + +def test_fake_satisfies_protocol(): + fake = FakeQuerier() + fake.create_authors([query.CreateAuthorsParams(name="b", bio=None), query.CreateAuthorsParams(name="a", bio=None)]) + assert author_names(fake) == ["a", "b"] + + +def test_modern_generic_session_querier(db): + apply_migrations(db, [os.path.dirname(__file__) + "/../library_modern/schema.sql"]) + with sqlalchemy.orm.Session(bind=db) as session: + q = modern_query.Querier(session) + assert q._conn is session + a = q.create_author(name="Ursula", bio=None) + assert a is not None + assert a.created_at.tzinfo is not None + book = q.create_book(author_id=a.id, title="The Dispossessed") + assert book is not None + assert book.status is modern_models.BookStatus.DRAFT + row = q.get_book_with_author(id=book.id) + assert row is not None + assert isinstance(row, pydantic.BaseModel) + assert row.authors == a + with pytest.raises(modern_errors.UniqueViolationError): + with session.begin_nested(): + q.create_author(name="Ursula", bio=None) + + +def test_modern_aware_datetime_rejects_naive(): + with pytest.raises(pydantic.ValidationError): + modern_models.Author(id=1, name="a", bio=None, created_at=datetime.datetime(2024, 1, 1)) diff --git a/go.mod b/go.mod index f8363d2..fd73b0d 100644 --- a/go.mod +++ b/go.mod @@ -1,11 +1,12 @@ module github.com/sqlc-dev/sqlc-gen-python -go 1.19 +go 1.27 require ( github.com/google/go-cmp v0.6.0 github.com/jinzhu/inflection v1.0.0 github.com/sqlc-dev/plugin-sdk-go v1.23.0 + golang.org/x/text v0.12.0 google.golang.org/protobuf v1.34.2 ) @@ -13,7 +14,6 @@ require ( github.com/golang/protobuf v1.5.3 // indirect golang.org/x/net v0.14.0 // indirect golang.org/x/sys v0.11.0 // indirect - golang.org/x/text v0.12.0 // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20230822172742-b8732ec3820d // indirect google.golang.org/grpc v1.59.0 // indirect ) diff --git a/internal/ast/ast.pb.go b/internal/ast/ast.pb.go index b0b1d3a..e72e083 100644 --- a/internal/ast/ast.pb.go +++ b/internal/ast/ast.pb.go @@ -57,6 +57,13 @@ type Node struct { // *Node_Await // *Node_AsyncFor // *Node_ImportGroup + // *Node_ListComp + // *Node_With + // *Node_BinOp + // *Node_BitOr + // *Node_Tuple + // *Node_UnaryOp + // *Node_Not Node isNode_Node `protobuf_oneof:"node"` } @@ -309,6 +316,55 @@ func (x *Node) GetImportGroup() *ImportGroup { return nil } +func (x *Node) GetListComp() *ListComp { + if x, ok := x.GetNode().(*Node_ListComp); ok { + return x.ListComp + } + return nil +} + +func (x *Node) GetWith() *With { + if x, ok := x.GetNode().(*Node_With); ok { + return x.With + } + return nil +} + +func (x *Node) GetBinOp() *BinOp { + if x, ok := x.GetNode().(*Node_BinOp); ok { + return x.BinOp + } + return nil +} + +func (x *Node) GetBitOr() *BitOr { + if x, ok := x.GetNode().(*Node_BitOr); ok { + return x.BitOr + } + return nil +} + +func (x *Node) GetTuple() *Tuple { + if x, ok := x.GetNode().(*Node_Tuple); ok { + return x.Tuple + } + return nil +} + +func (x *Node) GetUnaryOp() *UnaryOp { + if x, ok := x.GetNode().(*Node_UnaryOp); ok { + return x.UnaryOp + } + return nil +} + +func (x *Node) GetNot() *Not { + if x, ok := x.GetNode().(*Node_Not); ok { + return x.Not + } + return nil +} + type isNode_Node interface { isNode_Node() } @@ -433,6 +489,34 @@ type Node_ImportGroup struct { ImportGroup *ImportGroup `protobuf:"bytes,30,opt,name=import_group,json=ImportGroup,proto3,oneof"` } +type Node_ListComp struct { + ListComp *ListComp `protobuf:"bytes,31,opt,name=list_comp,json=ListComp,proto3,oneof"` +} + +type Node_With struct { + With *With `protobuf:"bytes,32,opt,name=with,json=With,proto3,oneof"` +} + +type Node_BinOp struct { + BinOp *BinOp `protobuf:"bytes,33,opt,name=bin_op,json=BinOp,proto3,oneof"` +} + +type Node_BitOr struct { + BitOr *BitOr `protobuf:"bytes,34,opt,name=bit_or,json=BitOr,proto3,oneof"` +} + +type Node_Tuple struct { + Tuple *Tuple `protobuf:"bytes,35,opt,name=tuple,json=Tuple,proto3,oneof"` +} + +type Node_UnaryOp struct { + UnaryOp *UnaryOp `protobuf:"bytes,36,opt,name=unary_op,json=UnaryOp,proto3,oneof"` +} + +type Node_Not struct { + Not *Not `protobuf:"bytes,37,opt,name=not,json=Not,proto3,oneof"` +} + func (*Node_ClassDef) isNode_Node() {} func (*Node_Import) isNode_Node() {} @@ -493,6 +577,20 @@ func (*Node_AsyncFor) isNode_Node() {} func (*Node_ImportGroup) isNode_Node() {} +func (*Node_ListComp) isNode_Node() {} + +func (*Node_With) isNode_Node() {} + +func (*Node_BinOp) isNode_Node() {} + +func (*Node_BitOr) isNode_Node() {} + +func (*Node_Tuple) isNode_Node() {} + +func (*Node_UnaryOp) isNode_Node() {} + +func (*Node_Not) isNode_Node() {} + type Alias struct { state protoimpl.MessageState sizeCache protoimpl.SizeCache @@ -1088,11 +1186,12 @@ type ClassDef struct { sizeCache protoimpl.SizeCache unknownFields protoimpl.UnknownFields - Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"` - Bases []*Node `protobuf:"bytes,2,rep,name=bases,proto3" json:"bases,omitempty"` - Keywords []*Node `protobuf:"bytes,3,rep,name=keywords,proto3" json:"keywords,omitempty"` - Body []*Node `protobuf:"bytes,4,rep,name=body,proto3" json:"body,omitempty"` - DecoratorList []*Node `protobuf:"bytes,5,rep,name=decorator_list,proto3" json:"decorator_list,omitempty"` + Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"` + Bases []*Node `protobuf:"bytes,2,rep,name=bases,proto3" json:"bases,omitempty"` + Keywords []*Node `protobuf:"bytes,3,rep,name=keywords,proto3" json:"keywords,omitempty"` + Body []*Node `protobuf:"bytes,4,rep,name=body,proto3" json:"body,omitempty"` + DecoratorList []*Node `protobuf:"bytes,5,rep,name=decorator_list,proto3" json:"decorator_list,omitempty"` + TypeParams []*TypeVar `protobuf:"bytes,6,rep,name=type_params,proto3" json:"type_params,omitempty"` } func (x *ClassDef) Reset() { @@ -1162,6 +1261,13 @@ func (x *ClassDef) GetDecoratorList() []*Node { return nil } +func (x *ClassDef) GetTypeParams() []*TypeVar { + if x != nil { + return x.TypeParams + } + return nil +} + // The Python ast module does not parse comments. It's not clear if this is the // best way to support them in the AST type Comment struct { @@ -1284,6 +1390,7 @@ type Constant struct { // *Constant_Str // *Constant_Int // *Constant_None + // *Constant_Ellipsis Value isConstant_Value `protobuf_oneof:"value"` } @@ -1347,6 +1454,13 @@ func (x *Constant) GetNone() bool { return false } +func (x *Constant) GetEllipsis() bool { + if x, ok := x.GetValue().(*Constant_Ellipsis); ok { + return x.Ellipsis + } + return false +} + type isConstant_Value interface { isConstant_Value() } @@ -1363,12 +1477,18 @@ type Constant_None struct { None bool `protobuf:"varint,3,opt,name=none,proto3,oneof"` } +type Constant_Ellipsis struct { + Ellipsis bool `protobuf:"varint,4,opt,name=ellipsis,proto3,oneof"` +} + func (*Constant_Str) isConstant_Value() {} func (*Constant_Int) isConstant_Value() {} func (*Constant_None) isConstant_Value() {} +func (*Constant_Ellipsis) isConstant_Value() {} + type Dict struct { state protoimpl.MessageState sizeCache protoimpl.SizeCache @@ -2164,16 +2284,17 @@ func (x *Subscript) GetSlice() *Node { return nil } -type Yield struct { +type ListComp struct { state protoimpl.MessageState sizeCache protoimpl.SizeCache unknownFields protoimpl.UnknownFields - Value *Node `protobuf:"bytes,1,opt,name=value,proto3" json:"value,omitempty"` + Elt *Node `protobuf:"bytes,1,opt,name=elt,proto3" json:"elt,omitempty"` + Generators []*Comprehension `protobuf:"bytes,2,rep,name=generators,proto3" json:"generators,omitempty"` } -func (x *Yield) Reset() { - *x = Yield{} +func (x *ListComp) Reset() { + *x = ListComp{} if protoimpl.UnsafeEnabled { mi := &file_ast_ast_proto_msgTypes[30] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) @@ -2181,13 +2302,13 @@ func (x *Yield) Reset() { } } -func (x *Yield) String() string { +func (x *ListComp) String() string { return protoimpl.X.MessageStringOf(x) } -func (*Yield) ProtoMessage() {} +func (*ListComp) ProtoMessage() {} -func (x *Yield) ProtoReflect() protoreflect.Message { +func (x *ListComp) ProtoReflect() protoreflect.Message { mi := &file_ast_ast_proto_msgTypes[30] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) @@ -2199,397 +2320,1019 @@ func (x *Yield) ProtoReflect() protoreflect.Message { return mi.MessageOf(x) } -// Deprecated: Use Yield.ProtoReflect.Descriptor instead. -func (*Yield) Descriptor() ([]byte, []int) { +// Deprecated: Use ListComp.ProtoReflect.Descriptor instead. +func (*ListComp) Descriptor() ([]byte, []int) { return file_ast_ast_proto_rawDescGZIP(), []int{30} } -func (x *Yield) GetValue() *Node { +func (x *ListComp) GetElt() *Node { if x != nil { - return x.Value + return x.Elt } return nil } -var File_ast_ast_proto protoreflect.FileDescriptor +func (x *ListComp) GetGenerators() []*Comprehension { + if x != nil { + return x.Generators + } + return nil +} -var file_ast_ast_proto_rawDesc = []byte{ - 0x0a, 0x0d, 0x61, 0x73, 0x74, 0x2f, 0x61, 0x73, 0x74, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, - 0x03, 0x61, 0x73, 0x74, 0x22, 0xde, 0x09, 0x0a, 0x04, 0x4e, 0x6f, 0x64, 0x65, 0x12, 0x2c, 0x0a, - 0x09, 0x63, 0x6c, 0x61, 0x73, 0x73, 0x5f, 0x64, 0x65, 0x66, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, - 0x32, 0x0d, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x43, 0x6c, 0x61, 0x73, 0x73, 0x44, 0x65, 0x66, 0x48, - 0x00, 0x52, 0x08, 0x43, 0x6c, 0x61, 0x73, 0x73, 0x44, 0x65, 0x66, 0x12, 0x25, 0x0a, 0x06, 0x69, - 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0b, 0x2e, 0x61, 0x73, - 0x74, 0x2e, 0x49, 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x48, 0x00, 0x52, 0x06, 0x49, 0x6d, 0x70, 0x6f, - 0x72, 0x74, 0x12, 0x32, 0x0a, 0x0b, 0x69, 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x5f, 0x66, 0x72, 0x6f, - 0x6d, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0f, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x49, 0x6d, - 0x70, 0x6f, 0x72, 0x74, 0x46, 0x72, 0x6f, 0x6d, 0x48, 0x00, 0x52, 0x0a, 0x49, 0x6d, 0x70, 0x6f, - 0x72, 0x74, 0x46, 0x72, 0x6f, 0x6d, 0x12, 0x25, 0x0a, 0x06, 0x6d, 0x6f, 0x64, 0x75, 0x6c, 0x65, - 0x18, 0x04, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0b, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4d, 0x6f, 0x64, - 0x75, 0x6c, 0x65, 0x48, 0x00, 0x52, 0x06, 0x4d, 0x6f, 0x64, 0x75, 0x6c, 0x65, 0x12, 0x22, 0x0a, - 0x05, 0x61, 0x6c, 0x69, 0x61, 0x73, 0x18, 0x05, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0a, 0x2e, 0x61, - 0x73, 0x74, 0x2e, 0x41, 0x6c, 0x69, 0x61, 0x73, 0x48, 0x00, 0x52, 0x05, 0x41, 0x6c, 0x69, 0x61, - 0x73, 0x12, 0x2f, 0x0a, 0x0a, 0x61, 0x6e, 0x6e, 0x5f, 0x61, 0x73, 0x73, 0x69, 0x67, 0x6e, 0x18, - 0x06, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0e, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x41, 0x6e, 0x6e, 0x41, - 0x73, 0x73, 0x69, 0x67, 0x6e, 0x48, 0x00, 0x52, 0x09, 0x41, 0x6e, 0x6e, 0x41, 0x73, 0x73, 0x69, - 0x67, 0x6e, 0x12, 0x1f, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x07, 0x20, 0x01, 0x28, 0x0b, - 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x61, 0x6d, 0x65, 0x48, 0x00, 0x52, 0x04, 0x4e, - 0x61, 0x6d, 0x65, 0x12, 0x2e, 0x0a, 0x09, 0x73, 0x75, 0x62, 0x73, 0x63, 0x72, 0x69, 0x70, 0x74, - 0x18, 0x08, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0e, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x53, 0x75, 0x62, - 0x73, 0x63, 0x72, 0x69, 0x70, 0x74, 0x48, 0x00, 0x52, 0x09, 0x53, 0x75, 0x62, 0x73, 0x63, 0x72, - 0x69, 0x70, 0x74, 0x12, 0x2e, 0x0a, 0x09, 0x61, 0x74, 0x74, 0x72, 0x69, 0x62, 0x75, 0x74, 0x65, - 0x18, 0x09, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0e, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x41, 0x74, 0x74, - 0x72, 0x69, 0x62, 0x75, 0x74, 0x65, 0x48, 0x00, 0x52, 0x09, 0x41, 0x74, 0x74, 0x72, 0x69, 0x62, - 0x75, 0x74, 0x65, 0x12, 0x2b, 0x0a, 0x08, 0x63, 0x6f, 0x6e, 0x73, 0x74, 0x61, 0x6e, 0x74, 0x18, - 0x0a, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0d, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x43, 0x6f, 0x6e, 0x73, - 0x74, 0x61, 0x6e, 0x74, 0x48, 0x00, 0x52, 0x08, 0x43, 0x6f, 0x6e, 0x73, 0x74, 0x61, 0x6e, 0x74, - 0x12, 0x25, 0x0a, 0x06, 0x61, 0x73, 0x73, 0x69, 0x67, 0x6e, 0x18, 0x0b, 0x20, 0x01, 0x28, 0x0b, - 0x32, 0x0b, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x41, 0x73, 0x73, 0x69, 0x67, 0x6e, 0x48, 0x00, 0x52, - 0x06, 0x41, 0x73, 0x73, 0x69, 0x67, 0x6e, 0x12, 0x28, 0x0a, 0x07, 0x63, 0x6f, 0x6d, 0x6d, 0x65, - 0x6e, 0x74, 0x18, 0x0c, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0c, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x43, - 0x6f, 0x6d, 0x6d, 0x65, 0x6e, 0x74, 0x48, 0x00, 0x52, 0x07, 0x43, 0x6f, 0x6d, 0x6d, 0x65, 0x6e, - 0x74, 0x12, 0x1f, 0x0a, 0x04, 0x65, 0x78, 0x70, 0x72, 0x18, 0x0d, 0x20, 0x01, 0x28, 0x0b, 0x32, - 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x45, 0x78, 0x70, 0x72, 0x48, 0x00, 0x52, 0x04, 0x45, 0x78, - 0x70, 0x72, 0x12, 0x1f, 0x0a, 0x04, 0x63, 0x61, 0x6c, 0x6c, 0x18, 0x0e, 0x20, 0x01, 0x28, 0x0b, - 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x43, 0x61, 0x6c, 0x6c, 0x48, 0x00, 0x52, 0x04, 0x43, - 0x61, 0x6c, 0x6c, 0x12, 0x35, 0x0a, 0x0c, 0x66, 0x75, 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x5f, - 0x64, 0x65, 0x66, 0x18, 0x0f, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x10, 0x2e, 0x61, 0x73, 0x74, 0x2e, - 0x46, 0x75, 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x44, 0x65, 0x66, 0x48, 0x00, 0x52, 0x0b, 0x46, - 0x75, 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x44, 0x65, 0x66, 0x12, 0x1c, 0x0a, 0x03, 0x61, 0x72, - 0x67, 0x18, 0x10, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x08, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x41, 0x72, - 0x67, 0x48, 0x00, 0x52, 0x03, 0x41, 0x72, 0x67, 0x12, 0x2e, 0x0a, 0x09, 0x61, 0x72, 0x67, 0x75, - 0x6d, 0x65, 0x6e, 0x74, 0x73, 0x18, 0x11, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0e, 0x2e, 0x61, 0x73, - 0x74, 0x2e, 0x41, 0x72, 0x67, 0x75, 0x6d, 0x65, 0x6e, 0x74, 0x73, 0x48, 0x00, 0x52, 0x09, 0x41, - 0x72, 0x67, 0x75, 0x6d, 0x65, 0x6e, 0x74, 0x73, 0x12, 0x45, 0x0a, 0x12, 0x61, 0x73, 0x79, 0x6e, - 0x63, 0x5f, 0x66, 0x75, 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x5f, 0x64, 0x65, 0x66, 0x18, 0x12, - 0x20, 0x01, 0x28, 0x0b, 0x32, 0x15, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x41, 0x73, 0x79, 0x6e, 0x63, - 0x46, 0x75, 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x44, 0x65, 0x66, 0x48, 0x00, 0x52, 0x10, 0x41, - 0x73, 0x79, 0x6e, 0x63, 0x46, 0x75, 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x44, 0x65, 0x66, 0x12, - 0x1f, 0x0a, 0x04, 0x70, 0x61, 0x73, 0x73, 0x18, 0x13, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, - 0x61, 0x73, 0x74, 0x2e, 0x50, 0x61, 0x73, 0x73, 0x48, 0x00, 0x52, 0x04, 0x50, 0x61, 0x73, 0x73, - 0x12, 0x1f, 0x0a, 0x04, 0x64, 0x69, 0x63, 0x74, 0x18, 0x14, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, - 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x44, 0x69, 0x63, 0x74, 0x48, 0x00, 0x52, 0x04, 0x44, 0x69, 0x63, - 0x74, 0x12, 0x19, 0x0a, 0x02, 0x69, 0x66, 0x18, 0x15, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x07, 0x2e, - 0x61, 0x73, 0x74, 0x2e, 0x49, 0x66, 0x48, 0x00, 0x52, 0x02, 0x49, 0x66, 0x12, 0x28, 0x0a, 0x07, - 0x63, 0x6f, 0x6d, 0x70, 0x61, 0x72, 0x65, 0x18, 0x16, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0c, 0x2e, - 0x61, 0x73, 0x74, 0x2e, 0x43, 0x6f, 0x6d, 0x70, 0x61, 0x72, 0x65, 0x48, 0x00, 0x52, 0x07, 0x43, - 0x6f, 0x6d, 0x70, 0x61, 0x72, 0x65, 0x12, 0x25, 0x0a, 0x06, 0x72, 0x65, 0x74, 0x75, 0x72, 0x6e, - 0x18, 0x17, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0b, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x52, 0x65, 0x74, - 0x75, 0x72, 0x6e, 0x48, 0x00, 0x52, 0x06, 0x52, 0x65, 0x74, 0x75, 0x72, 0x6e, 0x12, 0x19, 0x0a, - 0x02, 0x69, 0x73, 0x18, 0x18, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x07, 0x2e, 0x61, 0x73, 0x74, 0x2e, - 0x49, 0x73, 0x48, 0x00, 0x52, 0x02, 0x49, 0x73, 0x12, 0x28, 0x0a, 0x07, 0x6b, 0x65, 0x79, 0x77, - 0x6f, 0x72, 0x64, 0x18, 0x19, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0c, 0x2e, 0x61, 0x73, 0x74, 0x2e, - 0x4b, 0x65, 0x79, 0x77, 0x6f, 0x72, 0x64, 0x48, 0x00, 0x52, 0x07, 0x4b, 0x65, 0x79, 0x77, 0x6f, - 0x72, 0x64, 0x12, 0x22, 0x0a, 0x05, 0x79, 0x69, 0x65, 0x6c, 0x64, 0x18, 0x1a, 0x20, 0x01, 0x28, - 0x0b, 0x32, 0x0a, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x59, 0x69, 0x65, 0x6c, 0x64, 0x48, 0x00, 0x52, - 0x05, 0x59, 0x69, 0x65, 0x6c, 0x64, 0x12, 0x1c, 0x0a, 0x03, 0x66, 0x6f, 0x72, 0x18, 0x1b, 0x20, - 0x01, 0x28, 0x0b, 0x32, 0x08, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x46, 0x6f, 0x72, 0x48, 0x00, 0x52, - 0x03, 0x46, 0x6f, 0x72, 0x12, 0x22, 0x0a, 0x05, 0x61, 0x77, 0x61, 0x69, 0x74, 0x18, 0x1c, 0x20, - 0x01, 0x28, 0x0b, 0x32, 0x0a, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x41, 0x77, 0x61, 0x69, 0x74, 0x48, - 0x00, 0x52, 0x05, 0x41, 0x77, 0x61, 0x69, 0x74, 0x12, 0x2c, 0x0a, 0x09, 0x61, 0x73, 0x79, 0x6e, - 0x63, 0x5f, 0x66, 0x6f, 0x72, 0x18, 0x1d, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0d, 0x2e, 0x61, 0x73, - 0x74, 0x2e, 0x41, 0x73, 0x79, 0x6e, 0x63, 0x46, 0x6f, 0x72, 0x48, 0x00, 0x52, 0x08, 0x41, 0x73, - 0x79, 0x6e, 0x63, 0x46, 0x6f, 0x72, 0x12, 0x35, 0x0a, 0x0c, 0x69, 0x6d, 0x70, 0x6f, 0x72, 0x74, - 0x5f, 0x67, 0x72, 0x6f, 0x75, 0x70, 0x18, 0x1e, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x10, 0x2e, 0x61, - 0x73, 0x74, 0x2e, 0x49, 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x47, 0x72, 0x6f, 0x75, 0x70, 0x48, 0x00, - 0x52, 0x0b, 0x49, 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x47, 0x72, 0x6f, 0x75, 0x70, 0x42, 0x06, 0x0a, - 0x04, 0x6e, 0x6f, 0x64, 0x65, 0x22, 0x1b, 0x0a, 0x05, 0x41, 0x6c, 0x69, 0x61, 0x73, 0x12, 0x12, - 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, - 0x6d, 0x65, 0x22, 0x28, 0x0a, 0x05, 0x41, 0x77, 0x61, 0x69, 0x74, 0x12, 0x1f, 0x0a, 0x05, 0x76, - 0x61, 0x6c, 0x75, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, - 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x22, 0x40, 0x0a, 0x09, - 0x41, 0x74, 0x74, 0x72, 0x69, 0x62, 0x75, 0x74, 0x65, 0x12, 0x1f, 0x0a, 0x05, 0x76, 0x61, 0x6c, - 0x75, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, - 0x6f, 0x64, 0x65, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x61, 0x74, - 0x74, 0x72, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x61, 0x74, 0x74, 0x72, 0x22, 0x8b, - 0x01, 0x0a, 0x09, 0x41, 0x6e, 0x6e, 0x41, 0x73, 0x73, 0x69, 0x67, 0x6e, 0x12, 0x21, 0x0a, 0x06, - 0x74, 0x61, 0x72, 0x67, 0x65, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, - 0x73, 0x74, 0x2e, 0x4e, 0x61, 0x6d, 0x65, 0x52, 0x06, 0x74, 0x61, 0x72, 0x67, 0x65, 0x74, 0x12, - 0x29, 0x0a, 0x0a, 0x61, 0x6e, 0x6e, 0x6f, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x18, 0x02, 0x20, - 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x0a, - 0x61, 0x6e, 0x6e, 0x6f, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x16, 0x0a, 0x06, 0x73, 0x69, - 0x6d, 0x70, 0x6c, 0x65, 0x18, 0x03, 0x20, 0x01, 0x28, 0x05, 0x52, 0x06, 0x73, 0x69, 0x6d, 0x70, - 0x6c, 0x65, 0x12, 0x18, 0x0a, 0x07, 0x43, 0x6f, 0x6d, 0x6d, 0x65, 0x6e, 0x74, 0x18, 0x04, 0x20, - 0x01, 0x28, 0x09, 0x52, 0x07, 0x63, 0x6f, 0x6d, 0x6d, 0x65, 0x6e, 0x74, 0x22, 0x42, 0x0a, 0x03, - 0x41, 0x72, 0x67, 0x12, 0x10, 0x0a, 0x03, 0x61, 0x72, 0x67, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, - 0x52, 0x03, 0x61, 0x72, 0x67, 0x12, 0x29, 0x0a, 0x0a, 0x61, 0x6e, 0x6e, 0x6f, 0x74, 0x61, 0x74, - 0x69, 0x6f, 0x6e, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, - 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x0a, 0x61, 0x6e, 0x6e, 0x6f, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, - 0x22, 0x55, 0x0a, 0x09, 0x41, 0x72, 0x67, 0x75, 0x6d, 0x65, 0x6e, 0x74, 0x73, 0x12, 0x1c, 0x0a, - 0x04, 0x61, 0x72, 0x67, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x08, 0x2e, 0x61, 0x73, - 0x74, 0x2e, 0x41, 0x72, 0x67, 0x52, 0x04, 0x61, 0x72, 0x67, 0x73, 0x12, 0x2a, 0x0a, 0x0c, 0x6b, - 0x77, 0x5f, 0x6f, 0x6e, 0x6c, 0x79, 0x5f, 0x61, 0x72, 0x67, 0x73, 0x18, 0x02, 0x20, 0x03, 0x28, - 0x0b, 0x32, 0x08, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x41, 0x72, 0x67, 0x52, 0x0a, 0x6b, 0x77, 0x6f, - 0x6e, 0x6c, 0x79, 0x61, 0x72, 0x67, 0x73, 0x22, 0x6b, 0x0a, 0x08, 0x41, 0x73, 0x79, 0x6e, 0x63, - 0x46, 0x6f, 0x72, 0x12, 0x21, 0x0a, 0x06, 0x74, 0x61, 0x72, 0x67, 0x65, 0x74, 0x18, 0x01, 0x20, - 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x06, - 0x74, 0x61, 0x72, 0x67, 0x65, 0x74, 0x12, 0x1d, 0x0a, 0x04, 0x69, 0x74, 0x65, 0x72, 0x18, 0x02, - 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, - 0x04, 0x69, 0x74, 0x65, 0x72, 0x12, 0x1d, 0x0a, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x18, 0x03, 0x20, - 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, - 0x62, 0x6f, 0x64, 0x79, 0x22, 0x8e, 0x01, 0x0a, 0x10, 0x41, 0x73, 0x79, 0x6e, 0x63, 0x46, 0x75, - 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x44, 0x65, 0x66, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, - 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x22, 0x0a, - 0x04, 0x41, 0x72, 0x67, 0x73, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0e, 0x2e, 0x61, 0x73, - 0x74, 0x2e, 0x41, 0x72, 0x67, 0x75, 0x6d, 0x65, 0x6e, 0x74, 0x73, 0x52, 0x04, 0x61, 0x72, 0x67, - 0x73, 0x12, 0x1d, 0x0a, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, - 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x62, 0x6f, 0x64, 0x79, - 0x12, 0x23, 0x0a, 0x07, 0x72, 0x65, 0x74, 0x75, 0x72, 0x6e, 0x73, 0x18, 0x04, 0x20, 0x01, 0x28, - 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x07, 0x72, 0x65, - 0x74, 0x75, 0x72, 0x6e, 0x73, 0x22, 0x68, 0x0a, 0x06, 0x41, 0x73, 0x73, 0x69, 0x67, 0x6e, 0x12, - 0x23, 0x0a, 0x07, 0x74, 0x61, 0x72, 0x67, 0x65, 0x74, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, - 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x07, 0x74, 0x61, 0x72, - 0x67, 0x65, 0x74, 0x73, 0x12, 0x1f, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02, 0x20, - 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x05, - 0x76, 0x61, 0x6c, 0x75, 0x65, 0x12, 0x18, 0x0a, 0x07, 0x43, 0x6f, 0x6d, 0x6d, 0x65, 0x6e, 0x74, - 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x63, 0x6f, 0x6d, 0x6d, 0x65, 0x6e, 0x74, 0x22, - 0x6e, 0x0a, 0x04, 0x43, 0x61, 0x6c, 0x6c, 0x12, 0x1d, 0x0a, 0x04, 0x66, 0x75, 0x6e, 0x63, 0x18, - 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, - 0x52, 0x04, 0x66, 0x75, 0x6e, 0x63, 0x12, 0x1d, 0x0a, 0x04, 0x61, 0x72, 0x67, 0x73, 0x18, 0x02, - 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, - 0x04, 0x61, 0x72, 0x67, 0x73, 0x12, 0x28, 0x0a, 0x08, 0x6b, 0x65, 0x79, 0x77, 0x6f, 0x72, 0x64, - 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x0c, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4b, 0x65, - 0x79, 0x77, 0x6f, 0x72, 0x64, 0x52, 0x08, 0x6b, 0x65, 0x79, 0x77, 0x6f, 0x72, 0x64, 0x73, 0x22, - 0xb8, 0x01, 0x0a, 0x08, 0x43, 0x6c, 0x61, 0x73, 0x73, 0x44, 0x65, 0x66, 0x12, 0x12, 0x0a, 0x04, - 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, - 0x12, 0x1f, 0x0a, 0x05, 0x62, 0x61, 0x73, 0x65, 0x73, 0x18, 0x02, 0x20, 0x03, 0x28, 0x0b, 0x32, - 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x05, 0x62, 0x61, 0x73, 0x65, - 0x73, 0x12, 0x25, 0x0a, 0x08, 0x6b, 0x65, 0x79, 0x77, 0x6f, 0x72, 0x64, 0x73, 0x18, 0x03, 0x20, - 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x08, - 0x6b, 0x65, 0x79, 0x77, 0x6f, 0x72, 0x64, 0x73, 0x12, 0x1d, 0x0a, 0x04, 0x62, 0x6f, 0x64, 0x79, - 0x18, 0x04, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, - 0x65, 0x52, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x12, 0x31, 0x0a, 0x0e, 0x64, 0x65, 0x63, 0x6f, 0x72, - 0x61, 0x74, 0x6f, 0x72, 0x5f, 0x6c, 0x69, 0x73, 0x74, 0x18, 0x05, 0x20, 0x03, 0x28, 0x0b, 0x32, - 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x0e, 0x64, 0x65, 0x63, 0x6f, - 0x72, 0x61, 0x74, 0x6f, 0x72, 0x5f, 0x6c, 0x69, 0x73, 0x74, 0x22, 0x1d, 0x0a, 0x07, 0x43, 0x6f, - 0x6d, 0x6d, 0x65, 0x6e, 0x74, 0x12, 0x12, 0x0a, 0x04, 0x74, 0x65, 0x78, 0x74, 0x18, 0x01, 0x20, - 0x01, 0x28, 0x09, 0x52, 0x04, 0x74, 0x65, 0x78, 0x74, 0x22, 0x72, 0x0a, 0x07, 0x43, 0x6f, 0x6d, - 0x70, 0x61, 0x72, 0x65, 0x12, 0x1d, 0x0a, 0x04, 0x6c, 0x65, 0x66, 0x74, 0x18, 0x01, 0x20, 0x01, - 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x6c, - 0x65, 0x66, 0x74, 0x12, 0x1b, 0x0a, 0x03, 0x6f, 0x70, 0x73, 0x18, 0x02, 0x20, 0x03, 0x28, 0x0b, - 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x03, 0x6f, 0x70, 0x73, - 0x12, 0x2b, 0x0a, 0x0b, 0x63, 0x6f, 0x6d, 0x70, 0x61, 0x72, 0x61, 0x74, 0x6f, 0x72, 0x73, 0x18, - 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, - 0x52, 0x0b, 0x63, 0x6f, 0x6d, 0x70, 0x61, 0x72, 0x61, 0x74, 0x6f, 0x72, 0x73, 0x22, 0x54, 0x0a, - 0x08, 0x43, 0x6f, 0x6e, 0x73, 0x74, 0x61, 0x6e, 0x74, 0x12, 0x15, 0x0a, 0x03, 0x73, 0x74, 0x72, - 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x48, 0x00, 0x52, 0x06, 0x73, 0x74, 0x72, 0x69, 0x6e, 0x67, - 0x12, 0x12, 0x0a, 0x03, 0x69, 0x6e, 0x74, 0x18, 0x02, 0x20, 0x01, 0x28, 0x05, 0x48, 0x00, 0x52, - 0x03, 0x69, 0x6e, 0x74, 0x12, 0x14, 0x0a, 0x04, 0x6e, 0x6f, 0x6e, 0x65, 0x18, 0x03, 0x20, 0x01, - 0x28, 0x08, 0x48, 0x00, 0x52, 0x04, 0x6e, 0x6f, 0x6e, 0x65, 0x42, 0x07, 0x0a, 0x05, 0x76, 0x61, - 0x6c, 0x75, 0x65, 0x22, 0x48, 0x0a, 0x04, 0x44, 0x69, 0x63, 0x74, 0x12, 0x1d, 0x0a, 0x04, 0x6b, - 0x65, 0x79, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, - 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x6b, 0x65, 0x79, 0x73, 0x12, 0x21, 0x0a, 0x06, 0x76, 0x61, - 0x6c, 0x75, 0x65, 0x73, 0x18, 0x02, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, - 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x06, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x22, 0x27, 0x0a, - 0x04, 0x45, 0x78, 0x70, 0x72, 0x12, 0x1f, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x01, - 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, - 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x22, 0x66, 0x0a, 0x03, 0x46, 0x6f, 0x72, 0x12, 0x21, 0x0a, - 0x06, 0x74, 0x61, 0x72, 0x67, 0x65, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, - 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x06, 0x74, 0x61, 0x72, 0x67, 0x65, 0x74, - 0x12, 0x1d, 0x0a, 0x04, 0x69, 0x74, 0x65, 0x72, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, - 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x69, 0x74, 0x65, 0x72, 0x12, - 0x1d, 0x0a, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, - 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x22, 0x89, - 0x01, 0x0a, 0x0b, 0x46, 0x75, 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x44, 0x65, 0x66, 0x12, 0x12, - 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, - 0x6d, 0x65, 0x12, 0x22, 0x0a, 0x04, 0x41, 0x72, 0x67, 0x73, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, - 0x32, 0x0e, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x41, 0x72, 0x67, 0x75, 0x6d, 0x65, 0x6e, 0x74, 0x73, - 0x52, 0x04, 0x61, 0x72, 0x67, 0x73, 0x12, 0x1d, 0x0a, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x18, 0x03, - 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, - 0x04, 0x62, 0x6f, 0x64, 0x79, 0x12, 0x23, 0x0a, 0x07, 0x72, 0x65, 0x74, 0x75, 0x72, 0x6e, 0x73, - 0x18, 0x04, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, - 0x65, 0x52, 0x07, 0x72, 0x65, 0x74, 0x75, 0x72, 0x6e, 0x73, 0x22, 0x66, 0x0a, 0x02, 0x49, 0x66, - 0x12, 0x1d, 0x0a, 0x04, 0x74, 0x65, 0x73, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, - 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x74, 0x65, 0x73, 0x74, 0x12, - 0x1d, 0x0a, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x18, 0x02, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, - 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x12, 0x22, - 0x0a, 0x07, 0x6f, 0x72, 0x5f, 0x65, 0x6c, 0x73, 0x65, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, - 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x06, 0x6f, 0x72, 0x65, 0x6c, - 0x73, 0x65, 0x22, 0x29, 0x0a, 0x06, 0x49, 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x12, 0x1f, 0x0a, 0x05, - 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, - 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x05, 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x22, 0x5b, 0x0a, - 0x0a, 0x49, 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x46, 0x72, 0x6f, 0x6d, 0x12, 0x16, 0x0a, 0x06, 0x6d, - 0x6f, 0x64, 0x75, 0x6c, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x6d, 0x6f, 0x64, - 0x75, 0x6c, 0x65, 0x12, 0x1f, 0x0a, 0x05, 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x18, 0x02, 0x20, 0x03, - 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x05, 0x6e, - 0x61, 0x6d, 0x65, 0x73, 0x12, 0x14, 0x0a, 0x05, 0x6c, 0x65, 0x76, 0x65, 0x6c, 0x18, 0x03, 0x20, - 0x01, 0x28, 0x05, 0x52, 0x05, 0x6c, 0x65, 0x76, 0x65, 0x6c, 0x22, 0x32, 0x0a, 0x0b, 0x49, 0x6d, - 0x70, 0x6f, 0x72, 0x74, 0x47, 0x72, 0x6f, 0x75, 0x70, 0x12, 0x23, 0x0a, 0x07, 0x69, 0x6d, 0x70, - 0x6f, 0x72, 0x74, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, - 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x07, 0x69, 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x73, 0x22, 0x04, - 0x0a, 0x02, 0x49, 0x73, 0x22, 0x3c, 0x0a, 0x07, 0x4b, 0x65, 0x79, 0x77, 0x6f, 0x72, 0x64, 0x12, - 0x10, 0x0a, 0x03, 0x61, 0x72, 0x67, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x61, 0x72, - 0x67, 0x12, 0x1f, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, - 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x05, 0x76, 0x61, 0x6c, - 0x75, 0x65, 0x22, 0x27, 0x0a, 0x06, 0x4d, 0x6f, 0x64, 0x75, 0x6c, 0x65, 0x12, 0x1d, 0x0a, 0x04, - 0x62, 0x6f, 0x64, 0x79, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, - 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x22, 0x16, 0x0a, 0x04, 0x4e, - 0x61, 0x6d, 0x65, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, - 0x02, 0x69, 0x64, 0x22, 0x06, 0x0a, 0x04, 0x50, 0x61, 0x73, 0x73, 0x22, 0x29, 0x0a, 0x06, 0x52, - 0x65, 0x74, 0x75, 0x72, 0x6e, 0x12, 0x1f, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x01, - 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, - 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x22, 0x4d, 0x0a, 0x09, 0x53, 0x75, 0x62, 0x73, 0x63, 0x72, - 0x69, 0x70, 0x74, 0x12, 0x1f, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x01, 0x20, 0x01, - 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x61, 0x6d, 0x65, 0x52, 0x05, 0x76, - 0x61, 0x6c, 0x75, 0x65, 0x12, 0x1f, 0x0a, 0x05, 0x73, 0x6c, 0x69, 0x63, 0x65, 0x18, 0x02, 0x20, - 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x05, - 0x73, 0x6c, 0x69, 0x63, 0x65, 0x22, 0x28, 0x0a, 0x05, 0x59, 0x69, 0x65, 0x6c, 0x64, 0x12, 0x1f, - 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, - 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x42, - 0x71, 0x0a, 0x07, 0x63, 0x6f, 0x6d, 0x2e, 0x61, 0x73, 0x74, 0x42, 0x08, 0x41, 0x73, 0x74, 0x50, - 0x72, 0x6f, 0x74, 0x6f, 0x50, 0x01, 0x5a, 0x30, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, - 0x6f, 0x6d, 0x2f, 0x73, 0x71, 0x6c, 0x63, 0x2d, 0x64, 0x65, 0x76, 0x2f, 0x73, 0x71, 0x6c, 0x63, - 0x2d, 0x67, 0x65, 0x6e, 0x2d, 0x70, 0x79, 0x74, 0x68, 0x6f, 0x6e, 0x2f, 0x69, 0x6e, 0x74, 0x65, - 0x72, 0x6e, 0x61, 0x6c, 0x2f, 0x61, 0x73, 0x74, 0xa2, 0x02, 0x03, 0x41, 0x58, 0x58, 0xaa, 0x02, - 0x03, 0x41, 0x73, 0x74, 0xca, 0x02, 0x03, 0x41, 0x73, 0x74, 0xe2, 0x02, 0x0f, 0x41, 0x73, 0x74, - 0x5c, 0x47, 0x50, 0x42, 0x4d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0xea, 0x02, 0x03, 0x41, - 0x73, 0x74, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, +type Comprehension struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + Target *Node `protobuf:"bytes,1,opt,name=target,proto3" json:"target,omitempty"` + Iter *Node `protobuf:"bytes,2,opt,name=iter,proto3" json:"iter,omitempty"` + Ifs []*Node `protobuf:"bytes,3,rep,name=ifs,proto3" json:"ifs,omitempty"` } -var ( - file_ast_ast_proto_rawDescOnce sync.Once - file_ast_ast_proto_rawDescData = file_ast_ast_proto_rawDesc -) +func (x *Comprehension) Reset() { + *x = Comprehension{} + if protoimpl.UnsafeEnabled { + mi := &file_ast_ast_proto_msgTypes[31] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} -func file_ast_ast_proto_rawDescGZIP() []byte { - file_ast_ast_proto_rawDescOnce.Do(func() { - file_ast_ast_proto_rawDescData = protoimpl.X.CompressGZIP(file_ast_ast_proto_rawDescData) - }) - return file_ast_ast_proto_rawDescData +func (x *Comprehension) String() string { + return protoimpl.X.MessageStringOf(x) } -var file_ast_ast_proto_msgTypes = make([]protoimpl.MessageInfo, 31) -var file_ast_ast_proto_goTypes = []interface{}{ - (*Node)(nil), // 0: ast.Node - (*Alias)(nil), // 1: ast.Alias - (*Await)(nil), // 2: ast.Await - (*Attribute)(nil), // 3: ast.Attribute - (*AnnAssign)(nil), // 4: ast.AnnAssign - (*Arg)(nil), // 5: ast.Arg - (*Arguments)(nil), // 6: ast.Arguments - (*AsyncFor)(nil), // 7: ast.AsyncFor - (*AsyncFunctionDef)(nil), // 8: ast.AsyncFunctionDef - (*Assign)(nil), // 9: ast.Assign - (*Call)(nil), // 10: ast.Call - (*ClassDef)(nil), // 11: ast.ClassDef - (*Comment)(nil), // 12: ast.Comment - (*Compare)(nil), // 13: ast.Compare - (*Constant)(nil), // 14: ast.Constant - (*Dict)(nil), // 15: ast.Dict - (*Expr)(nil), // 16: ast.Expr - (*For)(nil), // 17: ast.For - (*FunctionDef)(nil), // 18: ast.FunctionDef - (*If)(nil), // 19: ast.If - (*Import)(nil), // 20: ast.Import - (*ImportFrom)(nil), // 21: ast.ImportFrom - (*ImportGroup)(nil), // 22: ast.ImportGroup - (*Is)(nil), // 23: ast.Is - (*Keyword)(nil), // 24: ast.Keyword - (*Module)(nil), // 25: ast.Module - (*Name)(nil), // 26: ast.Name - (*Pass)(nil), // 27: ast.Pass - (*Return)(nil), // 28: ast.Return - (*Subscript)(nil), // 29: ast.Subscript - (*Yield)(nil), // 30: ast.Yield +func (*Comprehension) ProtoMessage() {} + +func (x *Comprehension) ProtoReflect() protoreflect.Message { + mi := &file_ast_ast_proto_msgTypes[31] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) } -var file_ast_ast_proto_depIdxs = []int32{ - 11, // 0: ast.Node.class_def:type_name -> ast.ClassDef - 20, // 1: ast.Node.import:type_name -> ast.Import - 21, // 2: ast.Node.import_from:type_name -> ast.ImportFrom - 25, // 3: ast.Node.module:type_name -> ast.Module - 1, // 4: ast.Node.alias:type_name -> ast.Alias - 4, // 5: ast.Node.ann_assign:type_name -> ast.AnnAssign - 26, // 6: ast.Node.name:type_name -> ast.Name - 29, // 7: ast.Node.subscript:type_name -> ast.Subscript - 3, // 8: ast.Node.attribute:type_name -> ast.Attribute - 14, // 9: ast.Node.constant:type_name -> ast.Constant - 9, // 10: ast.Node.assign:type_name -> ast.Assign - 12, // 11: ast.Node.comment:type_name -> ast.Comment - 16, // 12: ast.Node.expr:type_name -> ast.Expr - 10, // 13: ast.Node.call:type_name -> ast.Call - 18, // 14: ast.Node.function_def:type_name -> ast.FunctionDef - 5, // 15: ast.Node.arg:type_name -> ast.Arg - 6, // 16: ast.Node.arguments:type_name -> ast.Arguments - 8, // 17: ast.Node.async_function_def:type_name -> ast.AsyncFunctionDef - 27, // 18: ast.Node.pass:type_name -> ast.Pass - 15, // 19: ast.Node.dict:type_name -> ast.Dict - 19, // 20: ast.Node.if:type_name -> ast.If - 13, // 21: ast.Node.compare:type_name -> ast.Compare - 28, // 22: ast.Node.return:type_name -> ast.Return - 23, // 23: ast.Node.is:type_name -> ast.Is - 24, // 24: ast.Node.keyword:type_name -> ast.Keyword - 30, // 25: ast.Node.yield:type_name -> ast.Yield - 17, // 26: ast.Node.for:type_name -> ast.For - 2, // 27: ast.Node.await:type_name -> ast.Await - 7, // 28: ast.Node.async_for:type_name -> ast.AsyncFor - 22, // 29: ast.Node.import_group:type_name -> ast.ImportGroup - 0, // 30: ast.Await.value:type_name -> ast.Node - 0, // 31: ast.Attribute.value:type_name -> ast.Node - 26, // 32: ast.AnnAssign.target:type_name -> ast.Name - 0, // 33: ast.AnnAssign.annotation:type_name -> ast.Node - 0, // 34: ast.Arg.annotation:type_name -> ast.Node - 5, // 35: ast.Arguments.args:type_name -> ast.Arg - 5, // 36: ast.Arguments.kw_only_args:type_name -> ast.Arg - 0, // 37: ast.AsyncFor.target:type_name -> ast.Node - 0, // 38: ast.AsyncFor.iter:type_name -> ast.Node - 0, // 39: ast.AsyncFor.body:type_name -> ast.Node - 6, // 40: ast.AsyncFunctionDef.Args:type_name -> ast.Arguments - 0, // 41: ast.AsyncFunctionDef.body:type_name -> ast.Node - 0, // 42: ast.AsyncFunctionDef.returns:type_name -> ast.Node - 0, // 43: ast.Assign.targets:type_name -> ast.Node - 0, // 44: ast.Assign.value:type_name -> ast.Node - 0, // 45: ast.Call.func:type_name -> ast.Node - 0, // 46: ast.Call.args:type_name -> ast.Node - 24, // 47: ast.Call.keywords:type_name -> ast.Keyword - 0, // 48: ast.ClassDef.bases:type_name -> ast.Node - 0, // 49: ast.ClassDef.keywords:type_name -> ast.Node - 0, // 50: ast.ClassDef.body:type_name -> ast.Node - 0, // 51: ast.ClassDef.decorator_list:type_name -> ast.Node - 0, // 52: ast.Compare.left:type_name -> ast.Node - 0, // 53: ast.Compare.ops:type_name -> ast.Node - 0, // 54: ast.Compare.comparators:type_name -> ast.Node - 0, // 55: ast.Dict.keys:type_name -> ast.Node - 0, // 56: ast.Dict.values:type_name -> ast.Node - 0, // 57: ast.Expr.value:type_name -> ast.Node - 0, // 58: ast.For.target:type_name -> ast.Node - 0, // 59: ast.For.iter:type_name -> ast.Node - 0, // 60: ast.For.body:type_name -> ast.Node - 6, // 61: ast.FunctionDef.Args:type_name -> ast.Arguments - 0, // 62: ast.FunctionDef.body:type_name -> ast.Node - 0, // 63: ast.FunctionDef.returns:type_name -> ast.Node - 0, // 64: ast.If.test:type_name -> ast.Node - 0, // 65: ast.If.body:type_name -> ast.Node - 0, // 66: ast.If.or_else:type_name -> ast.Node - 0, // 67: ast.Import.names:type_name -> ast.Node - 0, // 68: ast.ImportFrom.names:type_name -> ast.Node - 0, // 69: ast.ImportGroup.imports:type_name -> ast.Node - 0, // 70: ast.Keyword.value:type_name -> ast.Node - 0, // 71: ast.Module.body:type_name -> ast.Node - 0, // 72: ast.Return.value:type_name -> ast.Node - 26, // 73: ast.Subscript.value:type_name -> ast.Name - 0, // 74: ast.Subscript.slice:type_name -> ast.Node - 0, // 75: ast.Yield.value:type_name -> ast.Node - 76, // [76:76] is the sub-list for method output_type - 76, // [76:76] is the sub-list for method input_type - 76, // [76:76] is the sub-list for extension type_name - 76, // [76:76] is the sub-list for extension extendee - 0, // [0:76] is the sub-list for field type_name + +// Deprecated: Use Comprehension.ProtoReflect.Descriptor instead. +func (*Comprehension) Descriptor() ([]byte, []int) { + return file_ast_ast_proto_rawDescGZIP(), []int{31} } -func init() { file_ast_ast_proto_init() } -func file_ast_ast_proto_init() { - if File_ast_ast_proto != nil { - return +func (x *Comprehension) GetTarget() *Node { + if x != nil { + return x.Target } - if !protoimpl.UnsafeEnabled { - file_ast_ast_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} { - switch v := v.(*Node); i { - case 0: - return &v.state - case 1: - return &v.sizeCache - case 2: - return &v.unknownFields + return nil +} + +func (x *Comprehension) GetIter() *Node { + if x != nil { + return x.Iter + } + return nil +} + +func (x *Comprehension) GetIfs() []*Node { + if x != nil { + return x.Ifs + } + return nil +} + +type With struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + Items []*WithItem `protobuf:"bytes,1,rep,name=items,proto3" json:"items,omitempty"` + Body []*Node `protobuf:"bytes,2,rep,name=body,proto3" json:"body,omitempty"` +} + +func (x *With) Reset() { + *x = With{} + if protoimpl.UnsafeEnabled { + mi := &file_ast_ast_proto_msgTypes[32] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *With) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*With) ProtoMessage() {} + +func (x *With) ProtoReflect() protoreflect.Message { + mi := &file_ast_ast_proto_msgTypes[32] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use With.ProtoReflect.Descriptor instead. +func (*With) Descriptor() ([]byte, []int) { + return file_ast_ast_proto_rawDescGZIP(), []int{32} +} + +func (x *With) GetItems() []*WithItem { + if x != nil { + return x.Items + } + return nil +} + +func (x *With) GetBody() []*Node { + if x != nil { + return x.Body + } + return nil +} + +type WithItem struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + ContextExpr *Node `protobuf:"bytes,1,opt,name=context_expr,proto3" json:"context_expr,omitempty"` + OptionalVars *Node `protobuf:"bytes,2,opt,name=optional_vars,proto3" json:"optional_vars,omitempty"` +} + +func (x *WithItem) Reset() { + *x = WithItem{} + if protoimpl.UnsafeEnabled { + mi := &file_ast_ast_proto_msgTypes[33] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *WithItem) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*WithItem) ProtoMessage() {} + +func (x *WithItem) ProtoReflect() protoreflect.Message { + mi := &file_ast_ast_proto_msgTypes[33] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use WithItem.ProtoReflect.Descriptor instead. +func (*WithItem) Descriptor() ([]byte, []int) { + return file_ast_ast_proto_rawDescGZIP(), []int{33} +} + +func (x *WithItem) GetContextExpr() *Node { + if x != nil { + return x.ContextExpr + } + return nil +} + +func (x *WithItem) GetOptionalVars() *Node { + if x != nil { + return x.OptionalVars + } + return nil +} + +type BinOp struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + Left *Node `protobuf:"bytes,1,opt,name=left,proto3" json:"left,omitempty"` + Op *Node `protobuf:"bytes,2,opt,name=op,proto3" json:"op,omitempty"` + Right *Node `protobuf:"bytes,3,opt,name=right,proto3" json:"right,omitempty"` +} + +func (x *BinOp) Reset() { + *x = BinOp{} + if protoimpl.UnsafeEnabled { + mi := &file_ast_ast_proto_msgTypes[34] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *BinOp) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*BinOp) ProtoMessage() {} + +func (x *BinOp) ProtoReflect() protoreflect.Message { + mi := &file_ast_ast_proto_msgTypes[34] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use BinOp.ProtoReflect.Descriptor instead. +func (*BinOp) Descriptor() ([]byte, []int) { + return file_ast_ast_proto_rawDescGZIP(), []int{34} +} + +func (x *BinOp) GetLeft() *Node { + if x != nil { + return x.Left + } + return nil +} + +func (x *BinOp) GetOp() *Node { + if x != nil { + return x.Op + } + return nil +} + +func (x *BinOp) GetRight() *Node { + if x != nil { + return x.Right + } + return nil +} + +type BitOr struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields +} + +func (x *BitOr) Reset() { + *x = BitOr{} + if protoimpl.UnsafeEnabled { + mi := &file_ast_ast_proto_msgTypes[35] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *BitOr) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*BitOr) ProtoMessage() {} + +func (x *BitOr) ProtoReflect() protoreflect.Message { + mi := &file_ast_ast_proto_msgTypes[35] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use BitOr.ProtoReflect.Descriptor instead. +func (*BitOr) Descriptor() ([]byte, []int) { + return file_ast_ast_proto_rawDescGZIP(), []int{35} +} + +type UnaryOp struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + Op *Node `protobuf:"bytes,1,opt,name=op,proto3" json:"op,omitempty"` + Operand *Node `protobuf:"bytes,2,opt,name=operand,proto3" json:"operand,omitempty"` +} + +func (x *UnaryOp) Reset() { + *x = UnaryOp{} + if protoimpl.UnsafeEnabled { + mi := &file_ast_ast_proto_msgTypes[36] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *UnaryOp) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*UnaryOp) ProtoMessage() {} + +func (x *UnaryOp) ProtoReflect() protoreflect.Message { + mi := &file_ast_ast_proto_msgTypes[36] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use UnaryOp.ProtoReflect.Descriptor instead. +func (*UnaryOp) Descriptor() ([]byte, []int) { + return file_ast_ast_proto_rawDescGZIP(), []int{36} +} + +func (x *UnaryOp) GetOp() *Node { + if x != nil { + return x.Op + } + return nil +} + +func (x *UnaryOp) GetOperand() *Node { + if x != nil { + return x.Operand + } + return nil +} + +type Not struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields +} + +func (x *Not) Reset() { + *x = Not{} + if protoimpl.UnsafeEnabled { + mi := &file_ast_ast_proto_msgTypes[37] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *Not) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*Not) ProtoMessage() {} + +func (x *Not) ProtoReflect() protoreflect.Message { + mi := &file_ast_ast_proto_msgTypes[37] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use Not.ProtoReflect.Descriptor instead. +func (*Not) Descriptor() ([]byte, []int) { + return file_ast_ast_proto_rawDescGZIP(), []int{37} +} + +type Tuple struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + Elts []*Node `protobuf:"bytes,1,rep,name=elts,proto3" json:"elts,omitempty"` +} + +func (x *Tuple) Reset() { + *x = Tuple{} + if protoimpl.UnsafeEnabled { + mi := &file_ast_ast_proto_msgTypes[38] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *Tuple) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*Tuple) ProtoMessage() {} + +func (x *Tuple) ProtoReflect() protoreflect.Message { + mi := &file_ast_ast_proto_msgTypes[38] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use Tuple.ProtoReflect.Descriptor instead. +func (*Tuple) Descriptor() ([]byte, []int) { + return file_ast_ast_proto_rawDescGZIP(), []int{38} +} + +func (x *Tuple) GetElts() []*Node { + if x != nil { + return x.Elts + } + return nil +} + +// PEP 695 type parameter, printed as `T: ` inside `class C[...]`. +type TypeVar struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"` + Bound *Node `protobuf:"bytes,2,opt,name=bound,proto3" json:"bound,omitempty"` +} + +func (x *TypeVar) Reset() { + *x = TypeVar{} + if protoimpl.UnsafeEnabled { + mi := &file_ast_ast_proto_msgTypes[39] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *TypeVar) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*TypeVar) ProtoMessage() {} + +func (x *TypeVar) ProtoReflect() protoreflect.Message { + mi := &file_ast_ast_proto_msgTypes[39] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use TypeVar.ProtoReflect.Descriptor instead. +func (*TypeVar) Descriptor() ([]byte, []int) { + return file_ast_ast_proto_rawDescGZIP(), []int{39} +} + +func (x *TypeVar) GetName() string { + if x != nil { + return x.Name + } + return "" +} + +func (x *TypeVar) GetBound() *Node { + if x != nil { + return x.Bound + } + return nil +} + +type Yield struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + Value *Node `protobuf:"bytes,1,opt,name=value,proto3" json:"value,omitempty"` +} + +func (x *Yield) Reset() { + *x = Yield{} + if protoimpl.UnsafeEnabled { + mi := &file_ast_ast_proto_msgTypes[40] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *Yield) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*Yield) ProtoMessage() {} + +func (x *Yield) ProtoReflect() protoreflect.Message { + mi := &file_ast_ast_proto_msgTypes[40] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use Yield.ProtoReflect.Descriptor instead. +func (*Yield) Descriptor() ([]byte, []int) { + return file_ast_ast_proto_rawDescGZIP(), []int{40} +} + +func (x *Yield) GetValue() *Node { + if x != nil { + return x.Value + } + return nil +} + +var File_ast_ast_proto protoreflect.FileDescriptor + +var file_ast_ast_proto_rawDesc = []byte{ + 0x0a, 0x0d, 0x61, 0x73, 0x74, 0x2f, 0x61, 0x73, 0x74, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, + 0x03, 0x61, 0x73, 0x74, 0x22, 0xe4, 0x0b, 0x0a, 0x04, 0x4e, 0x6f, 0x64, 0x65, 0x12, 0x2c, 0x0a, + 0x09, 0x63, 0x6c, 0x61, 0x73, 0x73, 0x5f, 0x64, 0x65, 0x66, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, + 0x32, 0x0d, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x43, 0x6c, 0x61, 0x73, 0x73, 0x44, 0x65, 0x66, 0x48, + 0x00, 0x52, 0x08, 0x43, 0x6c, 0x61, 0x73, 0x73, 0x44, 0x65, 0x66, 0x12, 0x25, 0x0a, 0x06, 0x69, + 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0b, 0x2e, 0x61, 0x73, + 0x74, 0x2e, 0x49, 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x48, 0x00, 0x52, 0x06, 0x49, 0x6d, 0x70, 0x6f, + 0x72, 0x74, 0x12, 0x32, 0x0a, 0x0b, 0x69, 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x5f, 0x66, 0x72, 0x6f, + 0x6d, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0f, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x49, 0x6d, + 0x70, 0x6f, 0x72, 0x74, 0x46, 0x72, 0x6f, 0x6d, 0x48, 0x00, 0x52, 0x0a, 0x49, 0x6d, 0x70, 0x6f, + 0x72, 0x74, 0x46, 0x72, 0x6f, 0x6d, 0x12, 0x25, 0x0a, 0x06, 0x6d, 0x6f, 0x64, 0x75, 0x6c, 0x65, + 0x18, 0x04, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0b, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4d, 0x6f, 0x64, + 0x75, 0x6c, 0x65, 0x48, 0x00, 0x52, 0x06, 0x4d, 0x6f, 0x64, 0x75, 0x6c, 0x65, 0x12, 0x22, 0x0a, + 0x05, 0x61, 0x6c, 0x69, 0x61, 0x73, 0x18, 0x05, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0a, 0x2e, 0x61, + 0x73, 0x74, 0x2e, 0x41, 0x6c, 0x69, 0x61, 0x73, 0x48, 0x00, 0x52, 0x05, 0x41, 0x6c, 0x69, 0x61, + 0x73, 0x12, 0x2f, 0x0a, 0x0a, 0x61, 0x6e, 0x6e, 0x5f, 0x61, 0x73, 0x73, 0x69, 0x67, 0x6e, 0x18, + 0x06, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0e, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x41, 0x6e, 0x6e, 0x41, + 0x73, 0x73, 0x69, 0x67, 0x6e, 0x48, 0x00, 0x52, 0x09, 0x41, 0x6e, 0x6e, 0x41, 0x73, 0x73, 0x69, + 0x67, 0x6e, 0x12, 0x1f, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x07, 0x20, 0x01, 0x28, 0x0b, + 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x61, 0x6d, 0x65, 0x48, 0x00, 0x52, 0x04, 0x4e, + 0x61, 0x6d, 0x65, 0x12, 0x2e, 0x0a, 0x09, 0x73, 0x75, 0x62, 0x73, 0x63, 0x72, 0x69, 0x70, 0x74, + 0x18, 0x08, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0e, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x53, 0x75, 0x62, + 0x73, 0x63, 0x72, 0x69, 0x70, 0x74, 0x48, 0x00, 0x52, 0x09, 0x53, 0x75, 0x62, 0x73, 0x63, 0x72, + 0x69, 0x70, 0x74, 0x12, 0x2e, 0x0a, 0x09, 0x61, 0x74, 0x74, 0x72, 0x69, 0x62, 0x75, 0x74, 0x65, + 0x18, 0x09, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0e, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x41, 0x74, 0x74, + 0x72, 0x69, 0x62, 0x75, 0x74, 0x65, 0x48, 0x00, 0x52, 0x09, 0x41, 0x74, 0x74, 0x72, 0x69, 0x62, + 0x75, 0x74, 0x65, 0x12, 0x2b, 0x0a, 0x08, 0x63, 0x6f, 0x6e, 0x73, 0x74, 0x61, 0x6e, 0x74, 0x18, + 0x0a, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0d, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x43, 0x6f, 0x6e, 0x73, + 0x74, 0x61, 0x6e, 0x74, 0x48, 0x00, 0x52, 0x08, 0x43, 0x6f, 0x6e, 0x73, 0x74, 0x61, 0x6e, 0x74, + 0x12, 0x25, 0x0a, 0x06, 0x61, 0x73, 0x73, 0x69, 0x67, 0x6e, 0x18, 0x0b, 0x20, 0x01, 0x28, 0x0b, + 0x32, 0x0b, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x41, 0x73, 0x73, 0x69, 0x67, 0x6e, 0x48, 0x00, 0x52, + 0x06, 0x41, 0x73, 0x73, 0x69, 0x67, 0x6e, 0x12, 0x28, 0x0a, 0x07, 0x63, 0x6f, 0x6d, 0x6d, 0x65, + 0x6e, 0x74, 0x18, 0x0c, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0c, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x43, + 0x6f, 0x6d, 0x6d, 0x65, 0x6e, 0x74, 0x48, 0x00, 0x52, 0x07, 0x43, 0x6f, 0x6d, 0x6d, 0x65, 0x6e, + 0x74, 0x12, 0x1f, 0x0a, 0x04, 0x65, 0x78, 0x70, 0x72, 0x18, 0x0d, 0x20, 0x01, 0x28, 0x0b, 0x32, + 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x45, 0x78, 0x70, 0x72, 0x48, 0x00, 0x52, 0x04, 0x45, 0x78, + 0x70, 0x72, 0x12, 0x1f, 0x0a, 0x04, 0x63, 0x61, 0x6c, 0x6c, 0x18, 0x0e, 0x20, 0x01, 0x28, 0x0b, + 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x43, 0x61, 0x6c, 0x6c, 0x48, 0x00, 0x52, 0x04, 0x43, + 0x61, 0x6c, 0x6c, 0x12, 0x35, 0x0a, 0x0c, 0x66, 0x75, 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x5f, + 0x64, 0x65, 0x66, 0x18, 0x0f, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x10, 0x2e, 0x61, 0x73, 0x74, 0x2e, + 0x46, 0x75, 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x44, 0x65, 0x66, 0x48, 0x00, 0x52, 0x0b, 0x46, + 0x75, 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x44, 0x65, 0x66, 0x12, 0x1c, 0x0a, 0x03, 0x61, 0x72, + 0x67, 0x18, 0x10, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x08, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x41, 0x72, + 0x67, 0x48, 0x00, 0x52, 0x03, 0x41, 0x72, 0x67, 0x12, 0x2e, 0x0a, 0x09, 0x61, 0x72, 0x67, 0x75, + 0x6d, 0x65, 0x6e, 0x74, 0x73, 0x18, 0x11, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0e, 0x2e, 0x61, 0x73, + 0x74, 0x2e, 0x41, 0x72, 0x67, 0x75, 0x6d, 0x65, 0x6e, 0x74, 0x73, 0x48, 0x00, 0x52, 0x09, 0x41, + 0x72, 0x67, 0x75, 0x6d, 0x65, 0x6e, 0x74, 0x73, 0x12, 0x45, 0x0a, 0x12, 0x61, 0x73, 0x79, 0x6e, + 0x63, 0x5f, 0x66, 0x75, 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x5f, 0x64, 0x65, 0x66, 0x18, 0x12, + 0x20, 0x01, 0x28, 0x0b, 0x32, 0x15, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x41, 0x73, 0x79, 0x6e, 0x63, + 0x46, 0x75, 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x44, 0x65, 0x66, 0x48, 0x00, 0x52, 0x10, 0x41, + 0x73, 0x79, 0x6e, 0x63, 0x46, 0x75, 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x44, 0x65, 0x66, 0x12, + 0x1f, 0x0a, 0x04, 0x70, 0x61, 0x73, 0x73, 0x18, 0x13, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, + 0x61, 0x73, 0x74, 0x2e, 0x50, 0x61, 0x73, 0x73, 0x48, 0x00, 0x52, 0x04, 0x50, 0x61, 0x73, 0x73, + 0x12, 0x1f, 0x0a, 0x04, 0x64, 0x69, 0x63, 0x74, 0x18, 0x14, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, + 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x44, 0x69, 0x63, 0x74, 0x48, 0x00, 0x52, 0x04, 0x44, 0x69, 0x63, + 0x74, 0x12, 0x19, 0x0a, 0x02, 0x69, 0x66, 0x18, 0x15, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x07, 0x2e, + 0x61, 0x73, 0x74, 0x2e, 0x49, 0x66, 0x48, 0x00, 0x52, 0x02, 0x49, 0x66, 0x12, 0x28, 0x0a, 0x07, + 0x63, 0x6f, 0x6d, 0x70, 0x61, 0x72, 0x65, 0x18, 0x16, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0c, 0x2e, + 0x61, 0x73, 0x74, 0x2e, 0x43, 0x6f, 0x6d, 0x70, 0x61, 0x72, 0x65, 0x48, 0x00, 0x52, 0x07, 0x43, + 0x6f, 0x6d, 0x70, 0x61, 0x72, 0x65, 0x12, 0x25, 0x0a, 0x06, 0x72, 0x65, 0x74, 0x75, 0x72, 0x6e, + 0x18, 0x17, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0b, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x52, 0x65, 0x74, + 0x75, 0x72, 0x6e, 0x48, 0x00, 0x52, 0x06, 0x52, 0x65, 0x74, 0x75, 0x72, 0x6e, 0x12, 0x19, 0x0a, + 0x02, 0x69, 0x73, 0x18, 0x18, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x07, 0x2e, 0x61, 0x73, 0x74, 0x2e, + 0x49, 0x73, 0x48, 0x00, 0x52, 0x02, 0x49, 0x73, 0x12, 0x28, 0x0a, 0x07, 0x6b, 0x65, 0x79, 0x77, + 0x6f, 0x72, 0x64, 0x18, 0x19, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0c, 0x2e, 0x61, 0x73, 0x74, 0x2e, + 0x4b, 0x65, 0x79, 0x77, 0x6f, 0x72, 0x64, 0x48, 0x00, 0x52, 0x07, 0x4b, 0x65, 0x79, 0x77, 0x6f, + 0x72, 0x64, 0x12, 0x22, 0x0a, 0x05, 0x79, 0x69, 0x65, 0x6c, 0x64, 0x18, 0x1a, 0x20, 0x01, 0x28, + 0x0b, 0x32, 0x0a, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x59, 0x69, 0x65, 0x6c, 0x64, 0x48, 0x00, 0x52, + 0x05, 0x59, 0x69, 0x65, 0x6c, 0x64, 0x12, 0x1c, 0x0a, 0x03, 0x66, 0x6f, 0x72, 0x18, 0x1b, 0x20, + 0x01, 0x28, 0x0b, 0x32, 0x08, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x46, 0x6f, 0x72, 0x48, 0x00, 0x52, + 0x03, 0x46, 0x6f, 0x72, 0x12, 0x22, 0x0a, 0x05, 0x61, 0x77, 0x61, 0x69, 0x74, 0x18, 0x1c, 0x20, + 0x01, 0x28, 0x0b, 0x32, 0x0a, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x41, 0x77, 0x61, 0x69, 0x74, 0x48, + 0x00, 0x52, 0x05, 0x41, 0x77, 0x61, 0x69, 0x74, 0x12, 0x2c, 0x0a, 0x09, 0x61, 0x73, 0x79, 0x6e, + 0x63, 0x5f, 0x66, 0x6f, 0x72, 0x18, 0x1d, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0d, 0x2e, 0x61, 0x73, + 0x74, 0x2e, 0x41, 0x73, 0x79, 0x6e, 0x63, 0x46, 0x6f, 0x72, 0x48, 0x00, 0x52, 0x08, 0x41, 0x73, + 0x79, 0x6e, 0x63, 0x46, 0x6f, 0x72, 0x12, 0x35, 0x0a, 0x0c, 0x69, 0x6d, 0x70, 0x6f, 0x72, 0x74, + 0x5f, 0x67, 0x72, 0x6f, 0x75, 0x70, 0x18, 0x1e, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x10, 0x2e, 0x61, + 0x73, 0x74, 0x2e, 0x49, 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x47, 0x72, 0x6f, 0x75, 0x70, 0x48, 0x00, + 0x52, 0x0b, 0x49, 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x47, 0x72, 0x6f, 0x75, 0x70, 0x12, 0x2c, 0x0a, + 0x09, 0x6c, 0x69, 0x73, 0x74, 0x5f, 0x63, 0x6f, 0x6d, 0x70, 0x18, 0x1f, 0x20, 0x01, 0x28, 0x0b, + 0x32, 0x0d, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x43, 0x6f, 0x6d, 0x70, 0x48, + 0x00, 0x52, 0x08, 0x4c, 0x69, 0x73, 0x74, 0x43, 0x6f, 0x6d, 0x70, 0x12, 0x1f, 0x0a, 0x04, 0x77, + 0x69, 0x74, 0x68, 0x18, 0x20, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, + 0x57, 0x69, 0x74, 0x68, 0x48, 0x00, 0x52, 0x04, 0x57, 0x69, 0x74, 0x68, 0x12, 0x23, 0x0a, 0x06, + 0x62, 0x69, 0x6e, 0x5f, 0x6f, 0x70, 0x18, 0x21, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0a, 0x2e, 0x61, + 0x73, 0x74, 0x2e, 0x42, 0x69, 0x6e, 0x4f, 0x70, 0x48, 0x00, 0x52, 0x05, 0x42, 0x69, 0x6e, 0x4f, + 0x70, 0x12, 0x23, 0x0a, 0x06, 0x62, 0x69, 0x74, 0x5f, 0x6f, 0x72, 0x18, 0x22, 0x20, 0x01, 0x28, + 0x0b, 0x32, 0x0a, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x42, 0x69, 0x74, 0x4f, 0x72, 0x48, 0x00, 0x52, + 0x05, 0x42, 0x69, 0x74, 0x4f, 0x72, 0x12, 0x22, 0x0a, 0x05, 0x74, 0x75, 0x70, 0x6c, 0x65, 0x18, + 0x23, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0a, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x54, 0x75, 0x70, 0x6c, + 0x65, 0x48, 0x00, 0x52, 0x05, 0x54, 0x75, 0x70, 0x6c, 0x65, 0x12, 0x29, 0x0a, 0x08, 0x75, 0x6e, + 0x61, 0x72, 0x79, 0x5f, 0x6f, 0x70, 0x18, 0x24, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0c, 0x2e, 0x61, + 0x73, 0x74, 0x2e, 0x55, 0x6e, 0x61, 0x72, 0x79, 0x4f, 0x70, 0x48, 0x00, 0x52, 0x07, 0x55, 0x6e, + 0x61, 0x72, 0x79, 0x4f, 0x70, 0x12, 0x1c, 0x0a, 0x03, 0x6e, 0x6f, 0x74, 0x18, 0x25, 0x20, 0x01, + 0x28, 0x0b, 0x32, 0x08, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x74, 0x48, 0x00, 0x52, 0x03, + 0x4e, 0x6f, 0x74, 0x42, 0x06, 0x0a, 0x04, 0x6e, 0x6f, 0x64, 0x65, 0x22, 0x1b, 0x0a, 0x05, 0x41, + 0x6c, 0x69, 0x61, 0x73, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, + 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x22, 0x28, 0x0a, 0x05, 0x41, 0x77, 0x61, 0x69, + 0x74, 0x12, 0x1f, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, + 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x05, 0x76, 0x61, 0x6c, + 0x75, 0x65, 0x22, 0x40, 0x0a, 0x09, 0x41, 0x74, 0x74, 0x72, 0x69, 0x62, 0x75, 0x74, 0x65, 0x12, + 0x1f, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, + 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, + 0x12, 0x12, 0x0a, 0x04, 0x61, 0x74, 0x74, 0x72, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, + 0x61, 0x74, 0x74, 0x72, 0x22, 0x8b, 0x01, 0x0a, 0x09, 0x41, 0x6e, 0x6e, 0x41, 0x73, 0x73, 0x69, + 0x67, 0x6e, 0x12, 0x21, 0x0a, 0x06, 0x74, 0x61, 0x72, 0x67, 0x65, 0x74, 0x18, 0x01, 0x20, 0x01, + 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x61, 0x6d, 0x65, 0x52, 0x06, 0x74, + 0x61, 0x72, 0x67, 0x65, 0x74, 0x12, 0x29, 0x0a, 0x0a, 0x61, 0x6e, 0x6e, 0x6f, 0x74, 0x61, 0x74, + 0x69, 0x6f, 0x6e, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, + 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x0a, 0x61, 0x6e, 0x6e, 0x6f, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, + 0x12, 0x16, 0x0a, 0x06, 0x73, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x18, 0x03, 0x20, 0x01, 0x28, 0x05, + 0x52, 0x06, 0x73, 0x69, 0x6d, 0x70, 0x6c, 0x65, 0x12, 0x18, 0x0a, 0x07, 0x43, 0x6f, 0x6d, 0x6d, + 0x65, 0x6e, 0x74, 0x18, 0x04, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x63, 0x6f, 0x6d, 0x6d, 0x65, + 0x6e, 0x74, 0x22, 0x42, 0x0a, 0x03, 0x41, 0x72, 0x67, 0x12, 0x10, 0x0a, 0x03, 0x61, 0x72, 0x67, + 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x61, 0x72, 0x67, 0x12, 0x29, 0x0a, 0x0a, 0x61, + 0x6e, 0x6e, 0x6f, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, + 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x0a, 0x61, 0x6e, 0x6e, 0x6f, + 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x22, 0x55, 0x0a, 0x09, 0x41, 0x72, 0x67, 0x75, 0x6d, 0x65, + 0x6e, 0x74, 0x73, 0x12, 0x1c, 0x0a, 0x04, 0x61, 0x72, 0x67, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, + 0x0b, 0x32, 0x08, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x41, 0x72, 0x67, 0x52, 0x04, 0x61, 0x72, 0x67, + 0x73, 0x12, 0x2a, 0x0a, 0x0c, 0x6b, 0x77, 0x5f, 0x6f, 0x6e, 0x6c, 0x79, 0x5f, 0x61, 0x72, 0x67, + 0x73, 0x18, 0x02, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x08, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x41, 0x72, + 0x67, 0x52, 0x0a, 0x6b, 0x77, 0x6f, 0x6e, 0x6c, 0x79, 0x61, 0x72, 0x67, 0x73, 0x22, 0x6b, 0x0a, + 0x08, 0x41, 0x73, 0x79, 0x6e, 0x63, 0x46, 0x6f, 0x72, 0x12, 0x21, 0x0a, 0x06, 0x74, 0x61, 0x72, + 0x67, 0x65, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, + 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x06, 0x74, 0x61, 0x72, 0x67, 0x65, 0x74, 0x12, 0x1d, 0x0a, 0x04, + 0x69, 0x74, 0x65, 0x72, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, + 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x69, 0x74, 0x65, 0x72, 0x12, 0x1d, 0x0a, 0x04, 0x62, + 0x6f, 0x64, 0x79, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, + 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x22, 0x8e, 0x01, 0x0a, 0x10, 0x41, + 0x73, 0x79, 0x6e, 0x63, 0x46, 0x75, 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x44, 0x65, 0x66, 0x12, + 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, + 0x61, 0x6d, 0x65, 0x12, 0x22, 0x0a, 0x04, 0x41, 0x72, 0x67, 0x73, 0x18, 0x02, 0x20, 0x01, 0x28, + 0x0b, 0x32, 0x0e, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x41, 0x72, 0x67, 0x75, 0x6d, 0x65, 0x6e, 0x74, + 0x73, 0x52, 0x04, 0x61, 0x72, 0x67, 0x73, 0x12, 0x1d, 0x0a, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x18, + 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, + 0x52, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x12, 0x23, 0x0a, 0x07, 0x72, 0x65, 0x74, 0x75, 0x72, 0x6e, + 0x73, 0x18, 0x04, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, + 0x64, 0x65, 0x52, 0x07, 0x72, 0x65, 0x74, 0x75, 0x72, 0x6e, 0x73, 0x22, 0x68, 0x0a, 0x06, 0x41, + 0x73, 0x73, 0x69, 0x67, 0x6e, 0x12, 0x23, 0x0a, 0x07, 0x74, 0x61, 0x72, 0x67, 0x65, 0x74, 0x73, + 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, + 0x65, 0x52, 0x07, 0x74, 0x61, 0x72, 0x67, 0x65, 0x74, 0x73, 0x12, 0x1f, 0x0a, 0x05, 0x76, 0x61, + 0x6c, 0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, + 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x12, 0x18, 0x0a, 0x07, 0x43, + 0x6f, 0x6d, 0x6d, 0x65, 0x6e, 0x74, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x63, 0x6f, + 0x6d, 0x6d, 0x65, 0x6e, 0x74, 0x22, 0x6e, 0x0a, 0x04, 0x43, 0x61, 0x6c, 0x6c, 0x12, 0x1d, 0x0a, + 0x04, 0x66, 0x75, 0x6e, 0x63, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, + 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x66, 0x75, 0x6e, 0x63, 0x12, 0x1d, 0x0a, 0x04, + 0x61, 0x72, 0x67, 0x73, 0x18, 0x02, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, + 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x61, 0x72, 0x67, 0x73, 0x12, 0x28, 0x0a, 0x08, 0x6b, + 0x65, 0x79, 0x77, 0x6f, 0x72, 0x64, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x0c, 0x2e, + 0x61, 0x73, 0x74, 0x2e, 0x4b, 0x65, 0x79, 0x77, 0x6f, 0x72, 0x64, 0x52, 0x08, 0x6b, 0x65, 0x79, + 0x77, 0x6f, 0x72, 0x64, 0x73, 0x22, 0xe8, 0x01, 0x0a, 0x08, 0x43, 0x6c, 0x61, 0x73, 0x73, 0x44, + 0x65, 0x66, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, + 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x1f, 0x0a, 0x05, 0x62, 0x61, 0x73, 0x65, 0x73, 0x18, + 0x02, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, + 0x52, 0x05, 0x62, 0x61, 0x73, 0x65, 0x73, 0x12, 0x25, 0x0a, 0x08, 0x6b, 0x65, 0x79, 0x77, 0x6f, + 0x72, 0x64, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, + 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x08, 0x6b, 0x65, 0x79, 0x77, 0x6f, 0x72, 0x64, 0x73, 0x12, 0x1d, + 0x0a, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x18, 0x04, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, + 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x12, 0x31, 0x0a, + 0x0e, 0x64, 0x65, 0x63, 0x6f, 0x72, 0x61, 0x74, 0x6f, 0x72, 0x5f, 0x6c, 0x69, 0x73, 0x74, 0x18, + 0x05, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, + 0x52, 0x0e, 0x64, 0x65, 0x63, 0x6f, 0x72, 0x61, 0x74, 0x6f, 0x72, 0x5f, 0x6c, 0x69, 0x73, 0x74, + 0x12, 0x2e, 0x0a, 0x0b, 0x74, 0x79, 0x70, 0x65, 0x5f, 0x70, 0x61, 0x72, 0x61, 0x6d, 0x73, 0x18, + 0x06, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x0c, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x54, 0x79, 0x70, 0x65, + 0x56, 0x61, 0x72, 0x52, 0x0b, 0x74, 0x79, 0x70, 0x65, 0x5f, 0x70, 0x61, 0x72, 0x61, 0x6d, 0x73, + 0x22, 0x1d, 0x0a, 0x07, 0x43, 0x6f, 0x6d, 0x6d, 0x65, 0x6e, 0x74, 0x12, 0x12, 0x0a, 0x04, 0x74, + 0x65, 0x78, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x74, 0x65, 0x78, 0x74, 0x22, + 0x72, 0x0a, 0x07, 0x43, 0x6f, 0x6d, 0x70, 0x61, 0x72, 0x65, 0x12, 0x1d, 0x0a, 0x04, 0x6c, 0x65, + 0x66, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, + 0x6f, 0x64, 0x65, 0x52, 0x04, 0x6c, 0x65, 0x66, 0x74, 0x12, 0x1b, 0x0a, 0x03, 0x6f, 0x70, 0x73, + 0x18, 0x02, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, + 0x65, 0x52, 0x03, 0x6f, 0x70, 0x73, 0x12, 0x2b, 0x0a, 0x0b, 0x63, 0x6f, 0x6d, 0x70, 0x61, 0x72, + 0x61, 0x74, 0x6f, 0x72, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, + 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x0b, 0x63, 0x6f, 0x6d, 0x70, 0x61, 0x72, 0x61, 0x74, + 0x6f, 0x72, 0x73, 0x22, 0x72, 0x0a, 0x08, 0x43, 0x6f, 0x6e, 0x73, 0x74, 0x61, 0x6e, 0x74, 0x12, + 0x15, 0x0a, 0x03, 0x73, 0x74, 0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x48, 0x00, 0x52, 0x06, + 0x73, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x12, 0x12, 0x0a, 0x03, 0x69, 0x6e, 0x74, 0x18, 0x02, 0x20, + 0x01, 0x28, 0x05, 0x48, 0x00, 0x52, 0x03, 0x69, 0x6e, 0x74, 0x12, 0x14, 0x0a, 0x04, 0x6e, 0x6f, + 0x6e, 0x65, 0x18, 0x03, 0x20, 0x01, 0x28, 0x08, 0x48, 0x00, 0x52, 0x04, 0x6e, 0x6f, 0x6e, 0x65, + 0x12, 0x1c, 0x0a, 0x08, 0x65, 0x6c, 0x6c, 0x69, 0x70, 0x73, 0x69, 0x73, 0x18, 0x04, 0x20, 0x01, + 0x28, 0x08, 0x48, 0x00, 0x52, 0x08, 0x65, 0x6c, 0x6c, 0x69, 0x70, 0x73, 0x69, 0x73, 0x42, 0x07, + 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x22, 0x48, 0x0a, 0x04, 0x44, 0x69, 0x63, 0x74, 0x12, + 0x1d, 0x0a, 0x04, 0x6b, 0x65, 0x79, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, + 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x6b, 0x65, 0x79, 0x73, 0x12, 0x21, + 0x0a, 0x06, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x18, 0x02, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, + 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x06, 0x76, 0x61, 0x6c, 0x75, 0x65, + 0x73, 0x22, 0x27, 0x0a, 0x04, 0x45, 0x78, 0x70, 0x72, 0x12, 0x1f, 0x0a, 0x05, 0x76, 0x61, 0x6c, + 0x75, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, + 0x6f, 0x64, 0x65, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x22, 0x66, 0x0a, 0x03, 0x46, 0x6f, + 0x72, 0x12, 0x21, 0x0a, 0x06, 0x74, 0x61, 0x72, 0x67, 0x65, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, + 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x06, 0x74, 0x61, + 0x72, 0x67, 0x65, 0x74, 0x12, 0x1d, 0x0a, 0x04, 0x69, 0x74, 0x65, 0x72, 0x18, 0x02, 0x20, 0x01, + 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x69, + 0x74, 0x65, 0x72, 0x12, 0x1d, 0x0a, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x18, 0x03, 0x20, 0x03, 0x28, + 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x62, 0x6f, + 0x64, 0x79, 0x22, 0x89, 0x01, 0x0a, 0x0b, 0x46, 0x75, 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x44, + 0x65, 0x66, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, + 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x22, 0x0a, 0x04, 0x41, 0x72, 0x67, 0x73, 0x18, 0x02, + 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0e, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x41, 0x72, 0x67, 0x75, 0x6d, + 0x65, 0x6e, 0x74, 0x73, 0x52, 0x04, 0x61, 0x72, 0x67, 0x73, 0x12, 0x1d, 0x0a, 0x04, 0x62, 0x6f, + 0x64, 0x79, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, + 0x6f, 0x64, 0x65, 0x52, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x12, 0x23, 0x0a, 0x07, 0x72, 0x65, 0x74, + 0x75, 0x72, 0x6e, 0x73, 0x18, 0x04, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, + 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x07, 0x72, 0x65, 0x74, 0x75, 0x72, 0x6e, 0x73, 0x22, 0x66, + 0x0a, 0x02, 0x49, 0x66, 0x12, 0x1d, 0x0a, 0x04, 0x74, 0x65, 0x73, 0x74, 0x18, 0x01, 0x20, 0x01, + 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x74, + 0x65, 0x73, 0x74, 0x12, 0x1d, 0x0a, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x18, 0x02, 0x20, 0x03, 0x28, + 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x62, 0x6f, + 0x64, 0x79, 0x12, 0x22, 0x0a, 0x07, 0x6f, 0x72, 0x5f, 0x65, 0x6c, 0x73, 0x65, 0x18, 0x03, 0x20, + 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x06, + 0x6f, 0x72, 0x65, 0x6c, 0x73, 0x65, 0x22, 0x29, 0x0a, 0x06, 0x49, 0x6d, 0x70, 0x6f, 0x72, 0x74, + 0x12, 0x1f, 0x0a, 0x05, 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, + 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x05, 0x6e, 0x61, 0x6d, 0x65, + 0x73, 0x22, 0x5b, 0x0a, 0x0a, 0x49, 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x46, 0x72, 0x6f, 0x6d, 0x12, + 0x16, 0x0a, 0x06, 0x6d, 0x6f, 0x64, 0x75, 0x6c, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, + 0x06, 0x6d, 0x6f, 0x64, 0x75, 0x6c, 0x65, 0x12, 0x1f, 0x0a, 0x05, 0x6e, 0x61, 0x6d, 0x65, 0x73, + 0x18, 0x02, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, + 0x65, 0x52, 0x05, 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x12, 0x14, 0x0a, 0x05, 0x6c, 0x65, 0x76, 0x65, + 0x6c, 0x18, 0x03, 0x20, 0x01, 0x28, 0x05, 0x52, 0x05, 0x6c, 0x65, 0x76, 0x65, 0x6c, 0x22, 0x32, + 0x0a, 0x0b, 0x49, 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x47, 0x72, 0x6f, 0x75, 0x70, 0x12, 0x23, 0x0a, + 0x07, 0x69, 0x6d, 0x70, 0x6f, 0x72, 0x74, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, + 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x07, 0x69, 0x6d, 0x70, 0x6f, 0x72, + 0x74, 0x73, 0x22, 0x04, 0x0a, 0x02, 0x49, 0x73, 0x22, 0x3c, 0x0a, 0x07, 0x4b, 0x65, 0x79, 0x77, + 0x6f, 0x72, 0x64, 0x12, 0x10, 0x0a, 0x03, 0x61, 0x72, 0x67, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, + 0x52, 0x03, 0x61, 0x72, 0x67, 0x12, 0x1f, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02, + 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, + 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x22, 0x27, 0x0a, 0x06, 0x4d, 0x6f, 0x64, 0x75, 0x6c, 0x65, + 0x12, 0x1d, 0x0a, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, + 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x22, + 0x16, 0x0a, 0x04, 0x4e, 0x61, 0x6d, 0x65, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x01, 0x20, + 0x01, 0x28, 0x09, 0x52, 0x02, 0x69, 0x64, 0x22, 0x06, 0x0a, 0x04, 0x50, 0x61, 0x73, 0x73, 0x22, + 0x29, 0x0a, 0x06, 0x52, 0x65, 0x74, 0x75, 0x72, 0x6e, 0x12, 0x1f, 0x0a, 0x05, 0x76, 0x61, 0x6c, + 0x75, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, + 0x6f, 0x64, 0x65, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x22, 0x4d, 0x0a, 0x09, 0x53, 0x75, + 0x62, 0x73, 0x63, 0x72, 0x69, 0x70, 0x74, 0x12, 0x1f, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, + 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x61, 0x6d, + 0x65, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x12, 0x1f, 0x0a, 0x05, 0x73, 0x6c, 0x69, 0x63, + 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, + 0x64, 0x65, 0x52, 0x05, 0x73, 0x6c, 0x69, 0x63, 0x65, 0x22, 0x5b, 0x0a, 0x08, 0x4c, 0x69, 0x73, + 0x74, 0x43, 0x6f, 0x6d, 0x70, 0x12, 0x1b, 0x0a, 0x03, 0x65, 0x6c, 0x74, 0x18, 0x01, 0x20, 0x01, + 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x03, 0x65, + 0x6c, 0x74, 0x12, 0x32, 0x0a, 0x0a, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x6f, 0x72, 0x73, + 0x18, 0x02, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x12, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x43, 0x6f, 0x6d, + 0x70, 0x72, 0x65, 0x68, 0x65, 0x6e, 0x73, 0x69, 0x6f, 0x6e, 0x52, 0x0a, 0x67, 0x65, 0x6e, 0x65, + 0x72, 0x61, 0x74, 0x6f, 0x72, 0x73, 0x22, 0x6e, 0x0a, 0x0d, 0x43, 0x6f, 0x6d, 0x70, 0x72, 0x65, + 0x68, 0x65, 0x6e, 0x73, 0x69, 0x6f, 0x6e, 0x12, 0x21, 0x0a, 0x06, 0x74, 0x61, 0x72, 0x67, 0x65, + 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, + 0x64, 0x65, 0x52, 0x06, 0x74, 0x61, 0x72, 0x67, 0x65, 0x74, 0x12, 0x1d, 0x0a, 0x04, 0x69, 0x74, + 0x65, 0x72, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, + 0x6f, 0x64, 0x65, 0x52, 0x04, 0x69, 0x74, 0x65, 0x72, 0x12, 0x1b, 0x0a, 0x03, 0x69, 0x66, 0x73, + 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, + 0x65, 0x52, 0x03, 0x69, 0x66, 0x73, 0x22, 0x4a, 0x0a, 0x04, 0x57, 0x69, 0x74, 0x68, 0x12, 0x23, + 0x0a, 0x05, 0x69, 0x74, 0x65, 0x6d, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x0d, 0x2e, + 0x61, 0x73, 0x74, 0x2e, 0x57, 0x69, 0x74, 0x68, 0x49, 0x74, 0x65, 0x6d, 0x52, 0x05, 0x69, 0x74, + 0x65, 0x6d, 0x73, 0x12, 0x1d, 0x0a, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x18, 0x02, 0x20, 0x03, 0x28, + 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x04, 0x62, 0x6f, + 0x64, 0x79, 0x22, 0x6a, 0x0a, 0x08, 0x57, 0x69, 0x74, 0x68, 0x49, 0x74, 0x65, 0x6d, 0x12, 0x2d, + 0x0a, 0x0c, 0x63, 0x6f, 0x6e, 0x74, 0x65, 0x78, 0x74, 0x5f, 0x65, 0x78, 0x70, 0x72, 0x18, 0x01, + 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, + 0x0c, 0x63, 0x6f, 0x6e, 0x74, 0x65, 0x78, 0x74, 0x5f, 0x65, 0x78, 0x70, 0x72, 0x12, 0x2f, 0x0a, + 0x0d, 0x6f, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x61, 0x6c, 0x5f, 0x76, 0x61, 0x72, 0x73, 0x18, 0x02, + 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, + 0x0d, 0x6f, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x61, 0x6c, 0x5f, 0x76, 0x61, 0x72, 0x73, 0x22, 0x62, + 0x0a, 0x05, 0x42, 0x69, 0x6e, 0x4f, 0x70, 0x12, 0x1d, 0x0a, 0x04, 0x6c, 0x65, 0x66, 0x74, 0x18, + 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, + 0x52, 0x04, 0x6c, 0x65, 0x66, 0x74, 0x12, 0x19, 0x0a, 0x02, 0x6f, 0x70, 0x18, 0x02, 0x20, 0x01, + 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x02, 0x6f, + 0x70, 0x12, 0x1f, 0x0a, 0x05, 0x72, 0x69, 0x67, 0x68, 0x74, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0b, + 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x05, 0x72, 0x69, 0x67, + 0x68, 0x74, 0x22, 0x07, 0x0a, 0x05, 0x42, 0x69, 0x74, 0x4f, 0x72, 0x22, 0x49, 0x0a, 0x07, 0x55, + 0x6e, 0x61, 0x72, 0x79, 0x4f, 0x70, 0x12, 0x19, 0x0a, 0x02, 0x6f, 0x70, 0x18, 0x01, 0x20, 0x01, + 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x02, 0x6f, + 0x70, 0x12, 0x23, 0x0a, 0x07, 0x6f, 0x70, 0x65, 0x72, 0x61, 0x6e, 0x64, 0x18, 0x02, 0x20, 0x01, + 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x07, 0x6f, + 0x70, 0x65, 0x72, 0x61, 0x6e, 0x64, 0x22, 0x05, 0x0a, 0x03, 0x4e, 0x6f, 0x74, 0x22, 0x26, 0x0a, + 0x05, 0x54, 0x75, 0x70, 0x6c, 0x65, 0x12, 0x1d, 0x0a, 0x04, 0x65, 0x6c, 0x74, 0x73, 0x18, 0x01, + 0x20, 0x03, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, + 0x04, 0x65, 0x6c, 0x74, 0x73, 0x22, 0x3e, 0x0a, 0x07, 0x54, 0x79, 0x70, 0x65, 0x56, 0x61, 0x72, + 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, + 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x1f, 0x0a, 0x05, 0x62, 0x6f, 0x75, 0x6e, 0x64, 0x18, 0x02, 0x20, + 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x05, + 0x62, 0x6f, 0x75, 0x6e, 0x64, 0x22, 0x28, 0x0a, 0x05, 0x59, 0x69, 0x65, 0x6c, 0x64, 0x12, 0x1f, + 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x09, 0x2e, + 0x61, 0x73, 0x74, 0x2e, 0x4e, 0x6f, 0x64, 0x65, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x42, + 0x71, 0x0a, 0x07, 0x63, 0x6f, 0x6d, 0x2e, 0x61, 0x73, 0x74, 0x42, 0x08, 0x41, 0x73, 0x74, 0x50, + 0x72, 0x6f, 0x74, 0x6f, 0x50, 0x01, 0x5a, 0x30, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, + 0x6f, 0x6d, 0x2f, 0x73, 0x71, 0x6c, 0x63, 0x2d, 0x64, 0x65, 0x76, 0x2f, 0x73, 0x71, 0x6c, 0x63, + 0x2d, 0x67, 0x65, 0x6e, 0x2d, 0x70, 0x79, 0x74, 0x68, 0x6f, 0x6e, 0x2f, 0x69, 0x6e, 0x74, 0x65, + 0x72, 0x6e, 0x61, 0x6c, 0x2f, 0x61, 0x73, 0x74, 0xa2, 0x02, 0x03, 0x41, 0x58, 0x58, 0xaa, 0x02, + 0x03, 0x41, 0x73, 0x74, 0xca, 0x02, 0x03, 0x41, 0x73, 0x74, 0xe2, 0x02, 0x0f, 0x41, 0x73, 0x74, + 0x5c, 0x47, 0x50, 0x42, 0x4d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0xea, 0x02, 0x03, 0x41, + 0x73, 0x74, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, +} + +var ( + file_ast_ast_proto_rawDescOnce sync.Once + file_ast_ast_proto_rawDescData = file_ast_ast_proto_rawDesc +) + +func file_ast_ast_proto_rawDescGZIP() []byte { + file_ast_ast_proto_rawDescOnce.Do(func() { + file_ast_ast_proto_rawDescData = protoimpl.X.CompressGZIP(file_ast_ast_proto_rawDescData) + }) + return file_ast_ast_proto_rawDescData +} + +var file_ast_ast_proto_msgTypes = make([]protoimpl.MessageInfo, 41) +var file_ast_ast_proto_goTypes = []interface{}{ + (*Node)(nil), // 0: ast.Node + (*Alias)(nil), // 1: ast.Alias + (*Await)(nil), // 2: ast.Await + (*Attribute)(nil), // 3: ast.Attribute + (*AnnAssign)(nil), // 4: ast.AnnAssign + (*Arg)(nil), // 5: ast.Arg + (*Arguments)(nil), // 6: ast.Arguments + (*AsyncFor)(nil), // 7: ast.AsyncFor + (*AsyncFunctionDef)(nil), // 8: ast.AsyncFunctionDef + (*Assign)(nil), // 9: ast.Assign + (*Call)(nil), // 10: ast.Call + (*ClassDef)(nil), // 11: ast.ClassDef + (*Comment)(nil), // 12: ast.Comment + (*Compare)(nil), // 13: ast.Compare + (*Constant)(nil), // 14: ast.Constant + (*Dict)(nil), // 15: ast.Dict + (*Expr)(nil), // 16: ast.Expr + (*For)(nil), // 17: ast.For + (*FunctionDef)(nil), // 18: ast.FunctionDef + (*If)(nil), // 19: ast.If + (*Import)(nil), // 20: ast.Import + (*ImportFrom)(nil), // 21: ast.ImportFrom + (*ImportGroup)(nil), // 22: ast.ImportGroup + (*Is)(nil), // 23: ast.Is + (*Keyword)(nil), // 24: ast.Keyword + (*Module)(nil), // 25: ast.Module + (*Name)(nil), // 26: ast.Name + (*Pass)(nil), // 27: ast.Pass + (*Return)(nil), // 28: ast.Return + (*Subscript)(nil), // 29: ast.Subscript + (*ListComp)(nil), // 30: ast.ListComp + (*Comprehension)(nil), // 31: ast.Comprehension + (*With)(nil), // 32: ast.With + (*WithItem)(nil), // 33: ast.WithItem + (*BinOp)(nil), // 34: ast.BinOp + (*BitOr)(nil), // 35: ast.BitOr + (*UnaryOp)(nil), // 36: ast.UnaryOp + (*Not)(nil), // 37: ast.Not + (*Tuple)(nil), // 38: ast.Tuple + (*TypeVar)(nil), // 39: ast.TypeVar + (*Yield)(nil), // 40: ast.Yield +} +var file_ast_ast_proto_depIdxs = []int32{ + 11, // 0: ast.Node.class_def:type_name -> ast.ClassDef + 20, // 1: ast.Node.import:type_name -> ast.Import + 21, // 2: ast.Node.import_from:type_name -> ast.ImportFrom + 25, // 3: ast.Node.module:type_name -> ast.Module + 1, // 4: ast.Node.alias:type_name -> ast.Alias + 4, // 5: ast.Node.ann_assign:type_name -> ast.AnnAssign + 26, // 6: ast.Node.name:type_name -> ast.Name + 29, // 7: ast.Node.subscript:type_name -> ast.Subscript + 3, // 8: ast.Node.attribute:type_name -> ast.Attribute + 14, // 9: ast.Node.constant:type_name -> ast.Constant + 9, // 10: ast.Node.assign:type_name -> ast.Assign + 12, // 11: ast.Node.comment:type_name -> ast.Comment + 16, // 12: ast.Node.expr:type_name -> ast.Expr + 10, // 13: ast.Node.call:type_name -> ast.Call + 18, // 14: ast.Node.function_def:type_name -> ast.FunctionDef + 5, // 15: ast.Node.arg:type_name -> ast.Arg + 6, // 16: ast.Node.arguments:type_name -> ast.Arguments + 8, // 17: ast.Node.async_function_def:type_name -> ast.AsyncFunctionDef + 27, // 18: ast.Node.pass:type_name -> ast.Pass + 15, // 19: ast.Node.dict:type_name -> ast.Dict + 19, // 20: ast.Node.if:type_name -> ast.If + 13, // 21: ast.Node.compare:type_name -> ast.Compare + 28, // 22: ast.Node.return:type_name -> ast.Return + 23, // 23: ast.Node.is:type_name -> ast.Is + 24, // 24: ast.Node.keyword:type_name -> ast.Keyword + 40, // 25: ast.Node.yield:type_name -> ast.Yield + 17, // 26: ast.Node.for:type_name -> ast.For + 2, // 27: ast.Node.await:type_name -> ast.Await + 7, // 28: ast.Node.async_for:type_name -> ast.AsyncFor + 22, // 29: ast.Node.import_group:type_name -> ast.ImportGroup + 30, // 30: ast.Node.list_comp:type_name -> ast.ListComp + 32, // 31: ast.Node.with:type_name -> ast.With + 34, // 32: ast.Node.bin_op:type_name -> ast.BinOp + 35, // 33: ast.Node.bit_or:type_name -> ast.BitOr + 38, // 34: ast.Node.tuple:type_name -> ast.Tuple + 36, // 35: ast.Node.unary_op:type_name -> ast.UnaryOp + 37, // 36: ast.Node.not:type_name -> ast.Not + 0, // 37: ast.Await.value:type_name -> ast.Node + 0, // 38: ast.Attribute.value:type_name -> ast.Node + 26, // 39: ast.AnnAssign.target:type_name -> ast.Name + 0, // 40: ast.AnnAssign.annotation:type_name -> ast.Node + 0, // 41: ast.Arg.annotation:type_name -> ast.Node + 5, // 42: ast.Arguments.args:type_name -> ast.Arg + 5, // 43: ast.Arguments.kw_only_args:type_name -> ast.Arg + 0, // 44: ast.AsyncFor.target:type_name -> ast.Node + 0, // 45: ast.AsyncFor.iter:type_name -> ast.Node + 0, // 46: ast.AsyncFor.body:type_name -> ast.Node + 6, // 47: ast.AsyncFunctionDef.Args:type_name -> ast.Arguments + 0, // 48: ast.AsyncFunctionDef.body:type_name -> ast.Node + 0, // 49: ast.AsyncFunctionDef.returns:type_name -> ast.Node + 0, // 50: ast.Assign.targets:type_name -> ast.Node + 0, // 51: ast.Assign.value:type_name -> ast.Node + 0, // 52: ast.Call.func:type_name -> ast.Node + 0, // 53: ast.Call.args:type_name -> ast.Node + 24, // 54: ast.Call.keywords:type_name -> ast.Keyword + 0, // 55: ast.ClassDef.bases:type_name -> ast.Node + 0, // 56: ast.ClassDef.keywords:type_name -> ast.Node + 0, // 57: ast.ClassDef.body:type_name -> ast.Node + 0, // 58: ast.ClassDef.decorator_list:type_name -> ast.Node + 39, // 59: ast.ClassDef.type_params:type_name -> ast.TypeVar + 0, // 60: ast.Compare.left:type_name -> ast.Node + 0, // 61: ast.Compare.ops:type_name -> ast.Node + 0, // 62: ast.Compare.comparators:type_name -> ast.Node + 0, // 63: ast.Dict.keys:type_name -> ast.Node + 0, // 64: ast.Dict.values:type_name -> ast.Node + 0, // 65: ast.Expr.value:type_name -> ast.Node + 0, // 66: ast.For.target:type_name -> ast.Node + 0, // 67: ast.For.iter:type_name -> ast.Node + 0, // 68: ast.For.body:type_name -> ast.Node + 6, // 69: ast.FunctionDef.Args:type_name -> ast.Arguments + 0, // 70: ast.FunctionDef.body:type_name -> ast.Node + 0, // 71: ast.FunctionDef.returns:type_name -> ast.Node + 0, // 72: ast.If.test:type_name -> ast.Node + 0, // 73: ast.If.body:type_name -> ast.Node + 0, // 74: ast.If.or_else:type_name -> ast.Node + 0, // 75: ast.Import.names:type_name -> ast.Node + 0, // 76: ast.ImportFrom.names:type_name -> ast.Node + 0, // 77: ast.ImportGroup.imports:type_name -> ast.Node + 0, // 78: ast.Keyword.value:type_name -> ast.Node + 0, // 79: ast.Module.body:type_name -> ast.Node + 0, // 80: ast.Return.value:type_name -> ast.Node + 26, // 81: ast.Subscript.value:type_name -> ast.Name + 0, // 82: ast.Subscript.slice:type_name -> ast.Node + 0, // 83: ast.ListComp.elt:type_name -> ast.Node + 31, // 84: ast.ListComp.generators:type_name -> ast.Comprehension + 0, // 85: ast.Comprehension.target:type_name -> ast.Node + 0, // 86: ast.Comprehension.iter:type_name -> ast.Node + 0, // 87: ast.Comprehension.ifs:type_name -> ast.Node + 33, // 88: ast.With.items:type_name -> ast.WithItem + 0, // 89: ast.With.body:type_name -> ast.Node + 0, // 90: ast.WithItem.context_expr:type_name -> ast.Node + 0, // 91: ast.WithItem.optional_vars:type_name -> ast.Node + 0, // 92: ast.BinOp.left:type_name -> ast.Node + 0, // 93: ast.BinOp.op:type_name -> ast.Node + 0, // 94: ast.BinOp.right:type_name -> ast.Node + 0, // 95: ast.UnaryOp.op:type_name -> ast.Node + 0, // 96: ast.UnaryOp.operand:type_name -> ast.Node + 0, // 97: ast.Tuple.elts:type_name -> ast.Node + 0, // 98: ast.TypeVar.bound:type_name -> ast.Node + 0, // 99: ast.Yield.value:type_name -> ast.Node + 100, // [100:100] is the sub-list for method output_type + 100, // [100:100] is the sub-list for method input_type + 100, // [100:100] is the sub-list for extension type_name + 100, // [100:100] is the sub-list for extension extendee + 0, // [0:100] is the sub-list for field type_name +} + +func init() { file_ast_ast_proto_init() } +func file_ast_ast_proto_init() { + if File_ast_ast_proto != nil { + return + } + if !protoimpl.UnsafeEnabled { + file_ast_ast_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*Node); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields default: return nil } @@ -2943,6 +3686,126 @@ func file_ast_ast_proto_init() { } } file_ast_ast_proto_msgTypes[30].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*ListComp); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_ast_ast_proto_msgTypes[31].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*Comprehension); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_ast_ast_proto_msgTypes[32].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*With); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_ast_ast_proto_msgTypes[33].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*WithItem); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_ast_ast_proto_msgTypes[34].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*BinOp); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_ast_ast_proto_msgTypes[35].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*BitOr); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_ast_ast_proto_msgTypes[36].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*UnaryOp); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_ast_ast_proto_msgTypes[37].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*Not); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_ast_ast_proto_msgTypes[38].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*Tuple); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_ast_ast_proto_msgTypes[39].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*TypeVar); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_ast_ast_proto_msgTypes[40].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Yield); i { case 0: return &v.state @@ -2986,11 +3849,19 @@ func file_ast_ast_proto_init() { (*Node_Await)(nil), (*Node_AsyncFor)(nil), (*Node_ImportGroup)(nil), + (*Node_ListComp)(nil), + (*Node_With)(nil), + (*Node_BinOp)(nil), + (*Node_BitOr)(nil), + (*Node_Tuple)(nil), + (*Node_UnaryOp)(nil), + (*Node_Not)(nil), } file_ast_ast_proto_msgTypes[14].OneofWrappers = []interface{}{ (*Constant_Str)(nil), (*Constant_Int)(nil), (*Constant_None)(nil), + (*Constant_Ellipsis)(nil), } type x struct{} out := protoimpl.TypeBuilder{ @@ -2998,7 +3869,7 @@ func file_ast_ast_proto_init() { GoPackagePath: reflect.TypeOf(x{}).PkgPath(), RawDescriptor: file_ast_ast_proto_rawDesc, NumEnums: 0, - NumMessages: 31, + NumMessages: 41, NumExtensions: 0, NumServices: 0, }, diff --git a/internal/config.go b/internal/config.go index 1a8a565..005e6d3 100644 --- a/internal/config.go +++ b/internal/config.go @@ -1,13 +1,222 @@ package python +import ( + "encoding/json" + "errors" + "fmt" + "path" + "regexp" + "strings" + + "github.com/sqlc-dev/plugin-sdk-go/plugin" + "github.com/sqlc-dev/plugin-sdk-go/sdk" +) + type Config struct { - EmitExactTableNames bool `json:"emit_exact_table_names"` - EmitSyncQuerier bool `json:"emit_sync_querier"` - EmitAsyncQuerier bool `json:"emit_async_querier"` - Package string `json:"package"` - Out string `json:"out"` - EmitPydanticModels bool `json:"emit_pydantic_models"` - EmitStrEnum bool `json:"emit_str_enum"` - QueryParameterLimit *int32 `json:"query_parameter_limit"` - InflectionExcludeTableNames []string `json:"inflection_exclude_table_names"` + EmitExactTableNames bool `json:"emit_exact_table_names"` + EmitSyncQuerier bool `json:"emit_sync_querier"` + EmitAsyncQuerier bool `json:"emit_async_querier"` + Package string `json:"package"` + Out string `json:"out"` + EmitPydanticModels bool `json:"emit_pydantic_models"` + EmitStrEnum bool `json:"emit_str_enum"` + QueryParameterLimit *int32 `json:"query_parameter_limit"` + InflectionExcludeTableNames []string `json:"inflection_exclude_table_names"` + Overrides []Override `json:"overrides"` + Rename map[string]string `json:"rename"` + OmitUnusedStructs bool `json:"omit_unused_structs"` + OmitSqlcVersion bool `json:"omit_sqlc_version"` + EmitModernTypes bool `json:"emit_modern_types"` + EmitGenericQuerier bool `json:"emit_generic_querier"` + EmitAwareDatetime bool `json:"emit_aware_datetime"` + EmitQuerierProtocol bool `json:"emit_querier_protocol"` + EmitQueryErrors bool `json:"emit_query_errors"` +} + +type Override struct { + DBType string `json:"db_type,omitempty"` + Column string `json:"column,omitempty"` + Nullable bool `json:"nullable,omitempty"` + PyType string `json:"py_type,omitempty"` + PyImport string `json:"py_import,omitempty"` + + // schema, table and column glob patterns, set when Column is used + columnPatterns []string +} + +// globalOptions is the subset of sqlc's top-level `options.` block +// that applies to every codegen entry. +type globalOptions struct { + Overrides []Override `json:"overrides"` + Rename map[string]string `json:"rename"` +} + +func parseConfig(req *plugin.GenerateRequest) (Config, error) { + var conf Config + if len(req.PluginOptions) > 0 { + if err := json.Unmarshal(req.PluginOptions, &conf); err != nil { + return conf, err + } + } + if len(req.GlobalOptions) > 0 { + var global globalOptions + if err := json.Unmarshal(req.GlobalOptions, &global); err != nil { + return conf, err + } + conf.Overrides = append(conf.Overrides, global.Overrides...) + if len(global.Rename) > 0 { + if conf.Rename == nil { + conf.Rename = map[string]string{} + } + for k, v := range global.Rename { + conf.Rename[k] = v + } + } + } + + defaultSchema := "public" + if req.Catalog != nil && req.Catalog.DefaultSchema != "" { + defaultSchema = req.Catalog.DefaultSchema + } + for i := range conf.Overrides { + if err := conf.Overrides[i].parse(defaultSchema); err != nil { + return conf, fmt.Errorf("invalid override %s: %w", conf.Overrides[i].describe(), err) + } + } + + if conf.EmitAwareDatetime && !conf.EmitPydanticModels { + return conf, errors.New("emit_aware_datetime requires emit_pydantic_models") + } + return conf, nil +} + +func (o *Override) describe() string { + b, _ := json.Marshal(o) + return string(b) +} + +func (o *Override) parse(defaultSchema string) error { + switch { + case o.Column != "" && o.DBType != "": + return errors.New("set either column or db_type, not both") + case o.Column == "" && o.DBType == "": + return errors.New("set one of column or db_type") + case strings.TrimSpace(o.PyType) == "": + return errors.New("py_type must not be empty") + case o.PyImport != "" && strings.Contains(o.PyType, "."): + return errors.New("py_type must be a bare name when py_import is set") + } + if o.Column == "" { + if canonical, ok := pgTypeAliases[o.DBType]; ok { + o.DBType = canonical + } + return nil + } + parts := strings.Split(o.Column, ".") + switch len(parts) { + case 2: + o.columnPatterns = []string{defaultSchema, parts[0], parts[1]} + case 3: + o.columnPatterns = parts + default: + return fmt.Errorf("column %q must have the form table.column or schema.table.column", o.Column) + } + for _, p := range o.columnPatterns { + if _, err := path.Match(p, ""); err != nil { + return fmt.Errorf("column %q: %w", o.Column, err) + } + } + return nil +} + +// pgTypeAliases maps the SQL spellings of built-in types to the names sqlc +// reports for columns. +var pgTypeAliases = map[string]string{ + "bigint": "int8", + "integer": "int4", + "int": "int4", + "smallint": "int2", + "boolean": "bool", + "real": "float4", + "double precision": "float8", + "decimal": "numeric", + "character varying": "varchar", + "character": "bpchar", + "char": "bpchar", + "timestamp without time zone": "timestamp", + "timestamp with time zone": "timestamptz", + "time without time zone": "time", + "time with time zone": "timetz", +} + +// moduleNames are the module-level names a querier method body may read. +// A parameter with one of these names would shadow it. +func (c Config) moduleNames() map[string]struct{} { + names := map[string]struct{}{ + "self": {}, "sqlalchemy": {}, "models": {}, "errors": {}, + "cast": {}, "Any": {}, "List": {}, "Optional": {}, + "list": {}, "int": {}, "float": {}, "bool": {}, "str": {}, "memoryview": {}, + } + for m := range stdlibModules { + names[m] = struct{}{} + } + for _, o := range c.Overrides { + for _, m := range rootNamePattern.FindAllStringSubmatch(o.PyType, -1) { + names[m[1]] = struct{}{} + } + } + return names +} + +// rootNamePattern matches each name in a type expression that is not an +// attribute, such as dict, str and uuid in "dict[str, uuid.UUID]". +var rootNamePattern = regexp.MustCompile(`(?:^|[^.\w])([A-Za-z_]\w*)`) + +func globMatch(pattern, name string) bool { + ok, _ := path.Match(pattern, name) + return ok +} + +func (o *Override) matchesColumn(col *plugin.Column, defaultSchema string) bool { + if o.columnPatterns == nil || col.Table == nil { + return false + } + schema := col.Table.Schema + if schema == "" { + schema = defaultSchema + } + name := col.Name + if col.OriginalName != "" { + name = col.OriginalName + } + return globMatch(o.columnPatterns[0], schema) && + globMatch(o.columnPatterns[1], col.Table.Name) && + globMatch(o.columnPatterns[2], name) +} + +func (o *Override) matchesType(col *plugin.Column) bool { + if o.DBType == "" { + return false + } + typ := sdk.DataType(col.Type) + if typ != o.DBType && typ != "pg_catalog."+o.DBType { + return false + } + return o.Nullable == !col.NotNull +} + +// findOverride returns the override for col: column entries first, then +// db_type entries, each in config order. +func (c Config) findOverride(col *plugin.Column, defaultSchema string) *Override { + for i := range c.Overrides { + if c.Overrides[i].matchesColumn(col, defaultSchema) { + return &c.Overrides[i] + } + } + for i := range c.Overrides { + if c.Overrides[i].matchesType(col) { + return &c.Overrides[i] + } + } + return nil } diff --git a/internal/config_test.go b/internal/config_test.go new file mode 100644 index 0000000..c97d4eb --- /dev/null +++ b/internal/config_test.go @@ -0,0 +1,115 @@ +package python + +import ( + "strings" + "testing" + + "github.com/sqlc-dev/plugin-sdk-go/plugin" +) + +func TestParseConfigGlobalMerge(t *testing.T) { + req := &plugin.GenerateRequest{ + Catalog: &plugin.Catalog{DefaultSchema: "public"}, + PluginOptions: []byte(`{ + "overrides": [{"db_type": "uuid", "py_type": "local.Id"}], + "rename": {"a": "local_a", "b": "local_b"} + }`), + GlobalOptions: []byte(`{ + "overrides": [{"db_type": "uuid", "py_type": "global.Id"}, {"db_type": "citext", "py_type": "my_lib.CIText"}], + "rename": {"b": "global_b", "c": "global_c"} + }`), + } + conf, err := parseConfig(req) + if err != nil { + t.Fatal(err) + } + var got []string + for _, o := range conf.Overrides { + got = append(got, o.PyType) + } + if strings.Join(got, ",") != "local.Id,global.Id,my_lib.CIText" { + t.Errorf("override order = %v", got) + } + col := &plugin.Column{Type: &plugin.Identifier{Name: "uuid"}, NotNull: true} + if o := conf.findOverride(col, "public"); o == nil || o.PyType != "local.Id" { + t.Errorf("local override should win, got %+v", o) + } + want := map[string]string{"a": "local_a", "b": "global_b", "c": "global_c"} + for k, v := range want { + if conf.Rename[k] != v { + t.Errorf("rename[%q] = %q, want %q", k, conf.Rename[k], v) + } + } +} + +func TestParseConfigInvalidOverrides(t *testing.T) { + for _, tc := range []struct { + override string + err string + }{ + {`{"py_type": "str"}`, "set one of column or db_type"}, + {`{"db_type": "uuid", "column": "a.b", "py_type": "str"}`, "not both"}, + {`{"db_type": "uuid", "py_type": " "}`, "py_type must not be empty"}, + {`{"db_type": "uuid", "py_type": "uuid.UUID", "py_import": "uuid"}`, "bare name"}, + {`{"column": "id", "py_type": "int"}`, "table.column"}, + {`{"column": "a.b.c.d", "py_type": "int"}`, "table.column"}, + {`{"column": "a.[b", "py_type": "int"}`, "syntax error"}, + } { + req := &plugin.GenerateRequest{ + Catalog: &plugin.Catalog{DefaultSchema: "public"}, + PluginOptions: []byte(`{"overrides": [` + tc.override + `]}`), + } + _, err := parseConfig(req) + if err == nil || !strings.Contains(err.Error(), tc.err) || !strings.Contains(err.Error(), "invalid override {") { + t.Errorf("override %s: err = %v, want it to name the entry and contain %q", tc.override, err, tc.err) + } + } +} + +func TestParseConfigAwareDatetimeRequiresPydantic(t *testing.T) { + req := &plugin.GenerateRequest{PluginOptions: []byte(`{"emit_aware_datetime": true}`)} + if _, err := parseConfig(req); err == nil || !strings.Contains(err.Error(), "emit_pydantic_models") { + t.Errorf("err = %v", err) + } + req.PluginOptions = []byte(`{"emit_aware_datetime": true, "emit_pydantic_models": true}`) + if _, err := parseConfig(req); err != nil { + t.Error(err) + } +} + +func TestFindOverride(t *testing.T) { + conf := Config{Overrides: []Override{ + {DBType: "uuid", PyType: "my_ids.Id"}, + {Column: "users.id", PyType: "my_ids.UserId"}, + {DBType: "jsonb", PyType: "my_lib.Payload", Nullable: true}, + {Column: "*.created_*", PyType: "my_lib.Stamp"}, + }} + for i := range conf.Overrides { + if err := conf.Overrides[i].parse("public"); err != nil { + t.Fatal(err) + } + } + users := &plugin.Identifier{Name: "users"} + for _, tc := range []struct { + name string + col *plugin.Column + want string + }{ + {"column wins", &plugin.Column{Name: "id", Table: users, Type: &plugin.Identifier{Name: "uuid"}, NotNull: true}, "my_ids.UserId"}, + {"db_type", &plugin.Column{Name: "org_id", Table: users, Type: &plugin.Identifier{Name: "uuid"}, NotNull: true}, "my_ids.Id"}, + {"pg_catalog alias", &plugin.Column{Name: "x", Type: &plugin.Identifier{Schema: "pg_catalog", Name: "uuid"}, NotNull: true}, "my_ids.Id"}, + {"nullable skipped by default", &plugin.Column{Name: "x", Type: &plugin.Identifier{Name: "uuid"}}, ""}, + {"nullable entry", &plugin.Column{Name: "x", Type: &plugin.Identifier{Name: "jsonb"}}, "my_lib.Payload"}, + {"nullable entry skips not null", &plugin.Column{Name: "x", Type: &plugin.Identifier{Name: "jsonb"}, NotNull: true}, ""}, + {"glob", &plugin.Column{Name: "created_at", Table: &plugin.Identifier{Name: "posts"}, Type: &plugin.Identifier{Name: "timestamptz"}}, "my_lib.Stamp"}, + {"other schema", &plugin.Column{Name: "id", Table: &plugin.Identifier{Schema: "audit", Name: "users"}, Type: &plugin.Identifier{Name: "int8"}, NotNull: true}, ""}, + } { + got := "" + if o := conf.findOverride(tc.col, "public"); o != nil { + got = o.PyType + } + if got != tc.want { + t.Errorf("%s: got %q, want %q", tc.name, got, tc.want) + } + } +} diff --git a/internal/endtoend/testdata/batch_exec/db/models.py b/internal/endtoend/testdata/batch_exec/db/models.py new file mode 100644 index 0000000..973a3fe --- /dev/null +++ b/internal/endtoend/testdata/batch_exec/db/models.py @@ -0,0 +1,12 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses +from typing import Optional + + +@dataclasses.dataclass() +class Author: + id: int + name: str + bio: Optional[str] diff --git a/internal/endtoend/testdata/batch_exec/db/query.py b/internal/endtoend/testdata/batch_exec/db/query.py new file mode 100644 index 0000000..06df908 --- /dev/null +++ b/internal/endtoend/testdata/batch_exec/db/query.py @@ -0,0 +1,67 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import dataclasses +from typing import Optional, Sequence, Union + +import sqlalchemy +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from db import models + + +CREATE_AUTHORS = """-- name: create_authors \\:batchexec +INSERT INTO authors (name, bio) VALUES (:p1, :p2) +""" + + +@dataclasses.dataclass() +class CreateAuthorsParams: + name: str + bio: Optional[str] + + +DELETE_AUTHORS = """-- name: delete_authors \\:batchexec +DELETE FROM authors WHERE id = :p1 +""" + + +@dataclasses.dataclass() +class DeleteAuthorsParams: + id: int + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> None: + if not arg: + return None + self._conn.execute(sqlalchemy.text(CREATE_AUTHORS), [{"p1": a.name, "p2": a.bio} for a in arg]) + + def delete_authors(self, arg: Sequence[DeleteAuthorsParams]) -> None: + if not arg: + return None + self._conn.execute(sqlalchemy.text(DELETE_AUTHORS), [{"p1": a.id} for a in arg]) + + +class AsyncQuerier: + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): + self._conn = conn + + async def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> None: + if not arg: + return None + await self._conn.execute(sqlalchemy.text(CREATE_AUTHORS), [{"p1": a.name, "p2": a.bio} for a in arg]) + + async def delete_authors(self, arg: Sequence[DeleteAuthorsParams]) -> None: + if not arg: + return None + await self._conn.execute(sqlalchemy.text(DELETE_AUTHORS), [{"p1": a.id} for a in arg]) diff --git a/internal/endtoend/testdata/batch_exec/query.sql b/internal/endtoend/testdata/batch_exec/query.sql new file mode 100644 index 0000000..e71af71 --- /dev/null +++ b/internal/endtoend/testdata/batch_exec/query.sql @@ -0,0 +1,5 @@ +-- name: CreateAuthors :batchexec +INSERT INTO authors (name, bio) VALUES ($1, $2); + +-- name: DeleteAuthors :batchexec +DELETE FROM authors WHERE id = $1; diff --git a/internal/endtoend/testdata/batch_exec/schema.sql b/internal/endtoend/testdata/batch_exec/schema.sql new file mode 100644 index 0000000..a7b9212 --- /dev/null +++ b/internal/endtoend/testdata/batch_exec/schema.sql @@ -0,0 +1,5 @@ +CREATE TABLE authors ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL, + bio text +); diff --git a/internal/endtoend/testdata/batch_exec/sqlc.yaml b/internal/endtoend/testdata/batch_exec/sqlc.yaml new file mode 100644 index 0000000..da10353 --- /dev/null +++ b/internal/endtoend/testdata/batch_exec/sqlc.yaml @@ -0,0 +1,17 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + emit_async_querier: true diff --git a/internal/endtoend/testdata/batch_one_many/classic/models.py b/internal/endtoend/testdata/batch_one_many/classic/models.py new file mode 100644 index 0000000..2a0c683 --- /dev/null +++ b/internal/endtoend/testdata/batch_one_many/classic/models.py @@ -0,0 +1,19 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses +from typing import Optional + + +@dataclasses.dataclass() +class Author: + id: int + name: str + bio: Optional[str] + + +@dataclasses.dataclass() +class Book: + id: int + author_id: int + title: str diff --git a/internal/endtoend/testdata/batch_one_many/classic/query.py b/internal/endtoend/testdata/batch_one_many/classic/query.py new file mode 100644 index 0000000..6bcff2c --- /dev/null +++ b/internal/endtoend/testdata/batch_one_many/classic/query.py @@ -0,0 +1,143 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import dataclasses +from typing import AsyncIterator, Iterator, List, Optional, Sequence, Union, cast + +import sqlalchemy +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from classic import models + + +GET_AUTHOR_NAMES = """-- name: get_author_names \\:batchone +SELECT name FROM authors WHERE id = :p1 +""" + + +@dataclasses.dataclass() +class GetAuthorNamesParams: + id: int + + +GET_AUTHORS_BY_ID = """-- name: get_authors_by_id \\:batchone +SELECT id, name, bio FROM authors WHERE id = :p1 +""" + + +@dataclasses.dataclass() +class GetAuthorsByIDParams: + id: int + + +LIST_BOOK_TITLES = """-- name: list_book_titles \\:batchmany +SELECT id, title FROM books WHERE author_id = :p1 AND title LIKE :p2 +""" + + +@dataclasses.dataclass() +class ListBookTitlesParams: + author_id: int + title: str + + +@dataclasses.dataclass() +class ListBookTitlesRow: + id: int + title: str + + +LIST_BOOKS_BY_AUTHOR = """-- name: list_books_by_author \\:batchmany +SELECT id, author_id, title FROM books WHERE author_id = :p1 +""" + + +@dataclasses.dataclass() +class ListBooksByAuthorParams: + author_id: int + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def get_author_names(self, arg: Sequence[GetAuthorNamesParams]) -> Iterator[Optional[str]]: + for a in arg: + row = self._conn.execute(sqlalchemy.text(GET_AUTHOR_NAMES), {"p1": a.id}).first() + if row is None: + yield None + else: + yield cast(str, row[0]) + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> Iterator[Optional[models.Author]]: + for a in arg: + row = self._conn.execute(sqlalchemy.text(GET_AUTHORS_BY_ID), {"p1": a.id}).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + def list_book_titles(self, arg: Sequence[ListBookTitlesParams]) -> Iterator[List[ListBookTitlesRow]]: + for a in arg: + yield [ListBookTitlesRow( + id=cast(int, row[0]), + title=cast(str, row[1]), + ) for row in self._conn.execute(sqlalchemy.text(LIST_BOOK_TITLES), {"p1": a.author_id, "p2": a.title})] + + def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> Iterator[List[models.Book]]: + for a in arg: + yield [models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + ) for row in self._conn.execute(sqlalchemy.text(LIST_BOOKS_BY_AUTHOR), {"p1": a.author_id})] + + +class AsyncQuerier: + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): + self._conn = conn + + async def get_author_names(self, arg: Sequence[GetAuthorNamesParams]) -> AsyncIterator[Optional[str]]: + for a in arg: + row = (await self._conn.execute(sqlalchemy.text(GET_AUTHOR_NAMES), {"p1": a.id})).first() + if row is None: + yield None + else: + yield cast(str, row[0]) + + async def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> AsyncIterator[Optional[models.Author]]: + for a in arg: + row = (await self._conn.execute(sqlalchemy.text(GET_AUTHORS_BY_ID), {"p1": a.id})).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + async def list_book_titles(self, arg: Sequence[ListBookTitlesParams]) -> AsyncIterator[List[ListBookTitlesRow]]: + for a in arg: + yield [ListBookTitlesRow( + id=cast(int, row[0]), + title=cast(str, row[1]), + ) for row in await self._conn.execute(sqlalchemy.text(LIST_BOOK_TITLES), {"p1": a.author_id, "p2": a.title})] + + async def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> AsyncIterator[List[models.Book]]: + for a in arg: + yield [models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + ) for row in await self._conn.execute(sqlalchemy.text(LIST_BOOKS_BY_AUTHOR), {"p1": a.author_id})] diff --git a/internal/endtoend/testdata/batch_one_many/modern/models.py b/internal/endtoend/testdata/batch_one_many/modern/models.py new file mode 100644 index 0000000..a1d8805 --- /dev/null +++ b/internal/endtoend/testdata/batch_one_many/modern/models.py @@ -0,0 +1,18 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses + + +@dataclasses.dataclass() +class Author: + id: int + name: str + bio: str | None + + +@dataclasses.dataclass() +class Book: + id: int + author_id: int + title: str diff --git a/internal/endtoend/testdata/batch_one_many/modern/query.py b/internal/endtoend/testdata/batch_one_many/modern/query.py new file mode 100644 index 0000000..a9ee1f5 --- /dev/null +++ b/internal/endtoend/testdata/batch_one_many/modern/query.py @@ -0,0 +1,144 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import dataclasses +from collections.abc import AsyncIterator, Iterator, Sequence +from typing import cast + +import sqlalchemy +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from modern import models + + +GET_AUTHOR_NAMES = """-- name: get_author_names \\:batchone +SELECT name FROM authors WHERE id = :p1 +""" + + +@dataclasses.dataclass() +class GetAuthorNamesParams: + id: int + + +GET_AUTHORS_BY_ID = """-- name: get_authors_by_id \\:batchone +SELECT id, name, bio FROM authors WHERE id = :p1 +""" + + +@dataclasses.dataclass() +class GetAuthorsByIDParams: + id: int + + +LIST_BOOK_TITLES = """-- name: list_book_titles \\:batchmany +SELECT id, title FROM books WHERE author_id = :p1 AND title LIKE :p2 +""" + + +@dataclasses.dataclass() +class ListBookTitlesParams: + author_id: int + title: str + + +@dataclasses.dataclass() +class ListBookTitlesRow: + id: int + title: str + + +LIST_BOOKS_BY_AUTHOR = """-- name: list_books_by_author \\:batchmany +SELECT id, author_id, title FROM books WHERE author_id = :p1 +""" + + +@dataclasses.dataclass() +class ListBooksByAuthorParams: + author_id: int + + +class Querier: + _conn: sqlalchemy.engine.Connection | sqlalchemy.orm.Session + + def __init__(self, conn: sqlalchemy.engine.Connection | sqlalchemy.orm.Session): + self._conn = conn + + def get_author_names(self, arg: Sequence[GetAuthorNamesParams]) -> Iterator[str | None]: + for a in arg: + row = self._conn.execute(sqlalchemy.text(GET_AUTHOR_NAMES), {"p1": a.id}).first() + if row is None: + yield None + else: + yield cast(str, row[0]) + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> Iterator[models.Author | None]: + for a in arg: + row = self._conn.execute(sqlalchemy.text(GET_AUTHORS_BY_ID), {"p1": a.id}).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + ) + + def list_book_titles(self, arg: Sequence[ListBookTitlesParams]) -> Iterator[list[ListBookTitlesRow]]: + for a in arg: + yield [ListBookTitlesRow( + id=cast(int, row[0]), + title=cast(str, row[1]), + ) for row in self._conn.execute(sqlalchemy.text(LIST_BOOK_TITLES), {"p1": a.author_id, "p2": a.title})] + + def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> Iterator[list[models.Book]]: + for a in arg: + yield [models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + ) for row in self._conn.execute(sqlalchemy.text(LIST_BOOKS_BY_AUTHOR), {"p1": a.author_id})] + + +class AsyncQuerier: + _conn: sqlalchemy.ext.asyncio.AsyncConnection | sqlalchemy.ext.asyncio.AsyncSession + + def __init__(self, conn: sqlalchemy.ext.asyncio.AsyncConnection | sqlalchemy.ext.asyncio.AsyncSession): + self._conn = conn + + async def get_author_names(self, arg: Sequence[GetAuthorNamesParams]) -> AsyncIterator[str | None]: + for a in arg: + row = (await self._conn.execute(sqlalchemy.text(GET_AUTHOR_NAMES), {"p1": a.id})).first() + if row is None: + yield None + else: + yield cast(str, row[0]) + + async def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> AsyncIterator[models.Author | None]: + for a in arg: + row = (await self._conn.execute(sqlalchemy.text(GET_AUTHORS_BY_ID), {"p1": a.id})).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + ) + + async def list_book_titles(self, arg: Sequence[ListBookTitlesParams]) -> AsyncIterator[list[ListBookTitlesRow]]: + for a in arg: + yield [ListBookTitlesRow( + id=cast(int, row[0]), + title=cast(str, row[1]), + ) for row in await self._conn.execute(sqlalchemy.text(LIST_BOOK_TITLES), {"p1": a.author_id, "p2": a.title})] + + async def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> AsyncIterator[list[models.Book]]: + for a in arg: + yield [models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + ) for row in await self._conn.execute(sqlalchemy.text(LIST_BOOKS_BY_AUTHOR), {"p1": a.author_id})] diff --git a/internal/endtoend/testdata/batch_one_many/query.sql b/internal/endtoend/testdata/batch_one_many/query.sql new file mode 100644 index 0000000..3589816 --- /dev/null +++ b/internal/endtoend/testdata/batch_one_many/query.sql @@ -0,0 +1,11 @@ +-- name: GetAuthorsByID :batchone +SELECT * FROM authors WHERE id = $1; + +-- name: GetAuthorNames :batchone +SELECT name FROM authors WHERE id = $1; + +-- name: ListBooksByAuthor :batchmany +SELECT * FROM books WHERE author_id = $1; + +-- name: ListBookTitles :batchmany +SELECT id, title FROM books WHERE author_id = $1 AND title LIKE $2; diff --git a/internal/endtoend/testdata/batch_one_many/schema.sql b/internal/endtoend/testdata/batch_one_many/schema.sql new file mode 100644 index 0000000..e54d7c5 --- /dev/null +++ b/internal/endtoend/testdata/batch_one_many/schema.sql @@ -0,0 +1,11 @@ +CREATE TABLE authors ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL, + bio text +); + +CREATE TABLE books ( + id BIGSERIAL PRIMARY KEY, + author_id bigint NOT NULL REFERENCES authors (id), + title text NOT NULL +); diff --git a/internal/endtoend/testdata/batch_one_many/sqlc.yaml b/internal/endtoend/testdata/batch_one_many/sqlc.yaml new file mode 100644 index 0000000..831d536 --- /dev/null +++ b/internal/endtoend/testdata/batch_one_many/sqlc.yaml @@ -0,0 +1,24 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: classic + options: + package: classic + emit_sync_querier: true + emit_async_querier: true + - plugin: py + out: modern + options: + package: modern + emit_sync_querier: true + emit_async_querier: true + emit_modern_types: true diff --git a/internal/endtoend/testdata/constant_name_conflict/query.sql b/internal/endtoend/testdata/constant_name_conflict/query.sql new file mode 100644 index 0000000..b5a11d9 --- /dev/null +++ b/internal/endtoend/testdata/constant_name_conflict/query.sql @@ -0,0 +1,5 @@ +-- name: GetIid :one +SELECT name FROM things WHERE id = $1; + +-- name: GetIıd :one +SELECT id FROM things WHERE id = $1; diff --git a/internal/endtoend/testdata/constant_name_conflict/schema.sql b/internal/endtoend/testdata/constant_name_conflict/schema.sql new file mode 100644 index 0000000..cf1ff81 --- /dev/null +++ b/internal/endtoend/testdata/constant_name_conflict/schema.sql @@ -0,0 +1 @@ +CREATE TABLE things (id integer NOT NULL, name text NOT NULL); diff --git a/internal/endtoend/testdata/constant_name_conflict/sqlc.yaml b/internal/endtoend/testdata/constant_name_conflict/sqlc.yaml new file mode 100644 index 0000000..b2cb463 --- /dev/null +++ b/internal/endtoend/testdata/constant_name_conflict/sqlc.yaml @@ -0,0 +1,16 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true diff --git a/internal/endtoend/testdata/constant_name_conflict/stderr.txt b/internal/endtoend/testdata/constant_name_conflict/stderr.txt new file mode 100644 index 0000000..9dfb268 --- /dev/null +++ b/internal/endtoend/testdata/constant_name_conflict/stderr.txt @@ -0,0 +1,2 @@ +# package py +error generating code: error generating output: queries GetIid and GetIıd both generate constant GET_IID diff --git a/internal/endtoend/testdata/copyfrom/db/models.py b/internal/endtoend/testdata/copyfrom/db/models.py new file mode 100644 index 0000000..973a3fe --- /dev/null +++ b/internal/endtoend/testdata/copyfrom/db/models.py @@ -0,0 +1,12 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses +from typing import Optional + + +@dataclasses.dataclass() +class Author: + id: int + name: str + bio: Optional[str] diff --git a/internal/endtoend/testdata/copyfrom/db/query.py b/internal/endtoend/testdata/copyfrom/db/query.py new file mode 100644 index 0000000..e31e6e3 --- /dev/null +++ b/internal/endtoend/testdata/copyfrom/db/query.py @@ -0,0 +1,49 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import dataclasses +from typing import Any, Optional, Sequence, Union, cast + +import sqlalchemy +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from db import models + + +CREATE_AUTHORS = """-- name: create_authors \\:copyfrom +INSERT INTO authors (name, bio) VALUES (:p1, :p2) +""" + + +@dataclasses.dataclass() +class CreateAuthorsParams: + name: str + bio: Optional[str] + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: + if not arg: + return 0 + result = self._conn.execute(sqlalchemy.text(CREATE_AUTHORS), [{"p1": a.name, "p2": a.bio} for a in arg]) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + +class AsyncQuerier: + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): + self._conn = conn + + async def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: + if not arg: + return 0 + result = await self._conn.execute(sqlalchemy.text(CREATE_AUTHORS), [{"p1": a.name, "p2": a.bio} for a in arg]) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount diff --git a/internal/endtoend/testdata/copyfrom/query.sql b/internal/endtoend/testdata/copyfrom/query.sql new file mode 100644 index 0000000..1619de8 --- /dev/null +++ b/internal/endtoend/testdata/copyfrom/query.sql @@ -0,0 +1,2 @@ +-- name: CreateAuthors :copyfrom +INSERT INTO authors (name, bio) VALUES ($1, $2); diff --git a/internal/endtoend/testdata/copyfrom/schema.sql b/internal/endtoend/testdata/copyfrom/schema.sql new file mode 100644 index 0000000..a7b9212 --- /dev/null +++ b/internal/endtoend/testdata/copyfrom/schema.sql @@ -0,0 +1,5 @@ +CREATE TABLE authors ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL, + bio text +); diff --git a/internal/endtoend/testdata/copyfrom/sqlc.yaml b/internal/endtoend/testdata/copyfrom/sqlc.yaml new file mode 100644 index 0000000..da10353 --- /dev/null +++ b/internal/endtoend/testdata/copyfrom/sqlc.yaml @@ -0,0 +1,17 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + emit_async_querier: true diff --git a/internal/endtoend/testdata/emit_aware_datetime/db/models.py b/internal/endtoend/testdata/emit_aware_datetime/db/models.py new file mode 100644 index 0000000..3f77125 --- /dev/null +++ b/internal/endtoend/testdata/emit_aware_datetime/db/models.py @@ -0,0 +1,14 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import datetime +import pydantic +from typing import List, Optional + + +class Event(pydantic.BaseModel): + id: int + a: pydantic.AwareDatetime + b: datetime.datetime + c: Optional[pydantic.AwareDatetime] + d: Optional[List[pydantic.AwareDatetime]] diff --git a/internal/endtoend/testdata/emit_aware_datetime/db/query.py b/internal/endtoend/testdata/emit_aware_datetime/db/query.py new file mode 100644 index 0000000..219dc3e --- /dev/null +++ b/internal/endtoend/testdata/emit_aware_datetime/db/query.py @@ -0,0 +1,46 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import datetime +import pydantic +from typing import Iterator, List, Optional, Union, cast + +import sqlalchemy +import sqlalchemy.orm + +from db import models + + +GET_LATEST = """-- name: get_latest \\:one +SELECT max(a)\\:\\:timestamptz AS latest FROM events +""" + + +LIST_EVENTS_SINCE = """-- name: list_events_since \\:many +SELECT id, a, b, c, d FROM events WHERE a > :p1 +""" + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def get_latest(self) -> Optional[pydantic.AwareDatetime]: + row = self._conn.execute(sqlalchemy.text(GET_LATEST)).first() + if row is None: + return None + return cast(pydantic.AwareDatetime, row[0]) + + def list_events_since(self, *, a: pydantic.AwareDatetime) -> Iterator[models.Event]: + result = self._conn.execute(sqlalchemy.text(LIST_EVENTS_SINCE), {"p1": a}) + for row in result: + yield models.Event( + id=cast(int, row[0]), + a=cast(pydantic.AwareDatetime, row[1]), + b=cast(datetime.datetime, row[2]), + c=cast(Optional[pydantic.AwareDatetime], row[3]), + d=cast(Optional[List[pydantic.AwareDatetime]], row[4]), + ) diff --git a/internal/endtoend/testdata/emit_aware_datetime/query.sql b/internal/endtoend/testdata/emit_aware_datetime/query.sql new file mode 100644 index 0000000..46bc245 --- /dev/null +++ b/internal/endtoend/testdata/emit_aware_datetime/query.sql @@ -0,0 +1,5 @@ +-- name: ListEventsSince :many +SELECT * FROM events WHERE a > $1; + +-- name: GetLatest :one +SELECT max(a)::timestamptz AS latest FROM events; diff --git a/internal/endtoend/testdata/emit_aware_datetime/schema.sql b/internal/endtoend/testdata/emit_aware_datetime/schema.sql new file mode 100644 index 0000000..219d033 --- /dev/null +++ b/internal/endtoend/testdata/emit_aware_datetime/schema.sql @@ -0,0 +1,7 @@ +CREATE TABLE events ( + id BIGSERIAL PRIMARY KEY, + a timestamptz NOT NULL, + b timestamp NOT NULL, + c timestamp with time zone, + d timestamptz[] +); diff --git a/internal/endtoend/testdata/emit_aware_datetime/sqlc.yaml b/internal/endtoend/testdata/emit_aware_datetime/sqlc.yaml new file mode 100644 index 0000000..f8e2ad2 --- /dev/null +++ b/internal/endtoend/testdata/emit_aware_datetime/sqlc.yaml @@ -0,0 +1,18 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + emit_pydantic_models: true + emit_aware_datetime: true diff --git a/internal/endtoend/testdata/emit_aware_datetime_no_pydantic/query.sql b/internal/endtoend/testdata/emit_aware_datetime_no_pydantic/query.sql new file mode 100644 index 0000000..46bc245 --- /dev/null +++ b/internal/endtoend/testdata/emit_aware_datetime_no_pydantic/query.sql @@ -0,0 +1,5 @@ +-- name: ListEventsSince :many +SELECT * FROM events WHERE a > $1; + +-- name: GetLatest :one +SELECT max(a)::timestamptz AS latest FROM events; diff --git a/internal/endtoend/testdata/emit_aware_datetime_no_pydantic/schema.sql b/internal/endtoend/testdata/emit_aware_datetime_no_pydantic/schema.sql new file mode 100644 index 0000000..219d033 --- /dev/null +++ b/internal/endtoend/testdata/emit_aware_datetime_no_pydantic/schema.sql @@ -0,0 +1,7 @@ +CREATE TABLE events ( + id BIGSERIAL PRIMARY KEY, + a timestamptz NOT NULL, + b timestamp NOT NULL, + c timestamp with time zone, + d timestamptz[] +); diff --git a/internal/endtoend/testdata/emit_aware_datetime_no_pydantic/sqlc.yaml b/internal/endtoend/testdata/emit_aware_datetime_no_pydantic/sqlc.yaml new file mode 100644 index 0000000..a0a4a54 --- /dev/null +++ b/internal/endtoend/testdata/emit_aware_datetime_no_pydantic/sqlc.yaml @@ -0,0 +1,17 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + emit_aware_datetime: true diff --git a/internal/endtoend/testdata/emit_aware_datetime_no_pydantic/stderr.txt b/internal/endtoend/testdata/emit_aware_datetime_no_pydantic/stderr.txt new file mode 100644 index 0000000..cbd7f22 --- /dev/null +++ b/internal/endtoend/testdata/emit_aware_datetime_no_pydantic/stderr.txt @@ -0,0 +1,2 @@ +# package py +error generating code: error generating output: emit_aware_datetime requires emit_pydantic_models diff --git a/internal/endtoend/testdata/emit_generic_querier/_check.py b/internal/endtoend/testdata/emit_generic_querier/_check.py new file mode 100644 index 0000000..2e5b7f9 --- /dev/null +++ b/internal/endtoend/testdata/emit_generic_querier/_check.py @@ -0,0 +1,29 @@ +from typing import assert_type + +import sqlalchemy.engine +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from classic import query as classic +from modern import query as modern + + +def sync(session: sqlalchemy.orm.Session, conn: sqlalchemy.engine.Connection) -> None: + q = classic.Querier(session) + assert_type(q, classic.Querier[sqlalchemy.orm.Session]) + assert_type(q._conn, sqlalchemy.orm.Session) + assert_type(classic.Querier(conn)._conn, sqlalchemy.engine.Connection) + assert_type(modern.Querier(session)._conn, sqlalchemy.orm.Session) + + p: classic.QuerierProtocol = classic.Querier(session) + mp: modern.QuerierProtocol = modern.Querier(conn) + + classic.Querier(object()) # type: ignore[type-var] + modern.Querier(object()) # type: ignore[type-var] + + +def asynchronous(session: sqlalchemy.ext.asyncio.AsyncSession) -> None: + q = classic.AsyncQuerier(session) + assert_type(q._conn, sqlalchemy.ext.asyncio.AsyncSession) + p: classic.AsyncQuerierProtocol = q + mp: modern.AsyncQuerierProtocol = modern.AsyncQuerier(session) diff --git a/internal/endtoend/testdata/emit_generic_querier/classic/models.py b/internal/endtoend/testdata/emit_generic_querier/classic/models.py new file mode 100644 index 0000000..973a3fe --- /dev/null +++ b/internal/endtoend/testdata/emit_generic_querier/classic/models.py @@ -0,0 +1,12 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses +from typing import Optional + + +@dataclasses.dataclass() +class Author: + id: int + name: str + bio: Optional[str] diff --git a/internal/endtoend/testdata/emit_generic_querier/classic/query.py b/internal/endtoend/testdata/emit_generic_querier/classic/query.py new file mode 100644 index 0000000..7851e55 --- /dev/null +++ b/internal/endtoend/testdata/emit_generic_querier/classic/query.py @@ -0,0 +1,123 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +from typing import Any, AsyncIterator, Iterator, Optional, Protocol, Union, cast + +import sqlalchemy +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from classic import models + + +DELETE_AUTHOR = """-- name: delete_author \\:execrows +DELETE FROM authors WHERE id = :p1 +""" + + +GET_AUTHOR = """-- name: get_author \\:one +SELECT id, name, bio FROM authors WHERE id = :p1 +""" + + +LIST_AUTHORS = """-- name: list_authors \\:many +SELECT id, name, bio FROM authors ORDER BY name +""" + + +T = """-- name: t \\:one +SELECT name FROM authors WHERE id = :p1 +""" + + +class QuerierProtocol(Protocol): + def delete_author(self, *, id: int) -> int: ... + + def get_author(self, *, id: int) -> Optional[models.Author]: ... + + def list_authors(self) -> Iterator[models.Author]: ... + + def t(self, *, id: int) -> Optional[str]: ... + + +class AsyncQuerierProtocol(Protocol): + async def delete_author(self, *, id: int) -> int: ... + + async def get_author(self, *, id: int) -> Optional[models.Author]: ... + + def list_authors(self) -> AsyncIterator[models.Author]: ... + + async def t(self, *, id: int) -> Optional[str]: ... + + +class Querier[_ConnT: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]]: + _conn: _ConnT + + def __init__(self, conn: _ConnT): + self._conn = conn + + def delete_author(self, *, id: int) -> int: + result = self._conn.execute(sqlalchemy.text(DELETE_AUTHOR), {"p1": id}) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + def get_author(self, *, id: int) -> Optional[models.Author]: + row = self._conn.execute(sqlalchemy.text(GET_AUTHOR), {"p1": id}).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + def list_authors(self) -> Iterator[models.Author]: + result = self._conn.execute(sqlalchemy.text(LIST_AUTHORS)) + for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + def t(self, *, id: int) -> Optional[str]: + row = self._conn.execute(sqlalchemy.text(T), {"p1": id}).first() + if row is None: + return None + return cast(str, row[0]) + + +class AsyncQuerier[_ConnT: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]]: + _conn: _ConnT + + def __init__(self, conn: _ConnT): + self._conn = conn + + async def delete_author(self, *, id: int) -> int: + result = await self._conn.execute(sqlalchemy.text(DELETE_AUTHOR), {"p1": id}) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + async def get_author(self, *, id: int) -> Optional[models.Author]: + row = (await self._conn.execute(sqlalchemy.text(GET_AUTHOR), {"p1": id})).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + async def list_authors(self) -> AsyncIterator[models.Author]: + result = await self._conn.stream(sqlalchemy.text(LIST_AUTHORS)) + async for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + async def t(self, *, id: int) -> Optional[str]: + row = (await self._conn.execute(sqlalchemy.text(T), {"p1": id})).first() + if row is None: + return None + return cast(str, row[0]) diff --git a/internal/endtoend/testdata/emit_generic_querier/modern/models.py b/internal/endtoend/testdata/emit_generic_querier/modern/models.py new file mode 100644 index 0000000..89d370d --- /dev/null +++ b/internal/endtoend/testdata/emit_generic_querier/modern/models.py @@ -0,0 +1,11 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses + + +@dataclasses.dataclass() +class Author: + id: int + name: str + bio: str | None diff --git a/internal/endtoend/testdata/emit_generic_querier/modern/query.py b/internal/endtoend/testdata/emit_generic_querier/modern/query.py new file mode 100644 index 0000000..5301b4e --- /dev/null +++ b/internal/endtoend/testdata/emit_generic_querier/modern/query.py @@ -0,0 +1,124 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +from collections.abc import AsyncIterator, Iterator +from typing import Any, Protocol, cast + +import sqlalchemy +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from modern import models + + +DELETE_AUTHOR = """-- name: delete_author \\:execrows +DELETE FROM authors WHERE id = :p1 +""" + + +GET_AUTHOR = """-- name: get_author \\:one +SELECT id, name, bio FROM authors WHERE id = :p1 +""" + + +LIST_AUTHORS = """-- name: list_authors \\:many +SELECT id, name, bio FROM authors ORDER BY name +""" + + +T = """-- name: t \\:one +SELECT name FROM authors WHERE id = :p1 +""" + + +class QuerierProtocol(Protocol): + def delete_author(self, *, id: int) -> int: ... + + def get_author(self, *, id: int) -> models.Author | None: ... + + def list_authors(self) -> Iterator[models.Author]: ... + + def t(self, *, id: int) -> str | None: ... + + +class AsyncQuerierProtocol(Protocol): + async def delete_author(self, *, id: int) -> int: ... + + async def get_author(self, *, id: int) -> models.Author | None: ... + + def list_authors(self) -> AsyncIterator[models.Author]: ... + + async def t(self, *, id: int) -> str | None: ... + + +class Querier[_ConnT: sqlalchemy.engine.Connection | sqlalchemy.orm.Session]: + _conn: _ConnT + + def __init__(self, conn: _ConnT): + self._conn = conn + + def delete_author(self, *, id: int) -> int: + result = self._conn.execute(sqlalchemy.text(DELETE_AUTHOR), {"p1": id}) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + def get_author(self, *, id: int) -> models.Author | None: + row = self._conn.execute(sqlalchemy.text(GET_AUTHOR), {"p1": id}).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + ) + + def list_authors(self) -> Iterator[models.Author]: + result = self._conn.execute(sqlalchemy.text(LIST_AUTHORS)) + for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + ) + + def t(self, *, id: int) -> str | None: + row = self._conn.execute(sqlalchemy.text(T), {"p1": id}).first() + if row is None: + return None + return cast(str, row[0]) + + +class AsyncQuerier[_ConnT: sqlalchemy.ext.asyncio.AsyncConnection | sqlalchemy.ext.asyncio.AsyncSession]: + _conn: _ConnT + + def __init__(self, conn: _ConnT): + self._conn = conn + + async def delete_author(self, *, id: int) -> int: + result = await self._conn.execute(sqlalchemy.text(DELETE_AUTHOR), {"p1": id}) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + async def get_author(self, *, id: int) -> models.Author | None: + row = (await self._conn.execute(sqlalchemy.text(GET_AUTHOR), {"p1": id})).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + ) + + async def list_authors(self) -> AsyncIterator[models.Author]: + result = await self._conn.stream(sqlalchemy.text(LIST_AUTHORS)) + async for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + ) + + async def t(self, *, id: int) -> str | None: + row = (await self._conn.execute(sqlalchemy.text(T), {"p1": id})).first() + if row is None: + return None + return cast(str, row[0]) diff --git a/internal/endtoend/testdata/emit_generic_querier/query.sql b/internal/endtoend/testdata/emit_generic_querier/query.sql new file mode 100644 index 0000000..45284f8 --- /dev/null +++ b/internal/endtoend/testdata/emit_generic_querier/query.sql @@ -0,0 +1,11 @@ +-- name: GetAuthor :one +SELECT * FROM authors WHERE id = $1; + +-- name: ListAuthors :many +SELECT * FROM authors ORDER BY name; + +-- name: DeleteAuthor :execrows +DELETE FROM authors WHERE id = $1; + +-- name: T :one +SELECT name FROM authors WHERE id = $1; diff --git a/internal/endtoend/testdata/emit_generic_querier/schema.sql b/internal/endtoend/testdata/emit_generic_querier/schema.sql new file mode 100644 index 0000000..a7b9212 --- /dev/null +++ b/internal/endtoend/testdata/emit_generic_querier/schema.sql @@ -0,0 +1,5 @@ +CREATE TABLE authors ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL, + bio text +); diff --git a/internal/endtoend/testdata/emit_generic_querier/sqlc.yaml b/internal/endtoend/testdata/emit_generic_querier/sqlc.yaml new file mode 100644 index 0000000..9a271c3 --- /dev/null +++ b/internal/endtoend/testdata/emit_generic_querier/sqlc.yaml @@ -0,0 +1,28 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: classic + options: + package: classic + emit_sync_querier: true + emit_async_querier: true + emit_generic_querier: true + emit_querier_protocol: true + - plugin: py + out: modern + options: + package: modern + emit_sync_querier: true + emit_async_querier: true + emit_generic_querier: true + emit_querier_protocol: true + emit_modern_types: true diff --git a/internal/endtoend/testdata/emit_modern_types/db/models.py b/internal/endtoend/testdata/emit_modern_types/db/models.py new file mode 100644 index 0000000..98305b3 --- /dev/null +++ b/internal/endtoend/testdata/emit_modern_types/db/models.py @@ -0,0 +1,22 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses +import datetime +import enum +from typing import Any + + +class Status(str, enum.Enum): + DRAFT = "draft" + PUBLISHED = "published" + + +@dataclasses.dataclass() +class Post: + id: int + title: str + tags: list[str] | None + status: Status | None + payload: Any | None + created_at: datetime.datetime diff --git a/internal/endtoend/testdata/emit_modern_types/db/query.py b/internal/endtoend/testdata/emit_modern_types/db/query.py new file mode 100644 index 0000000..988d639 --- /dev/null +++ b/internal/endtoend/testdata/emit_modern_types/db/query.py @@ -0,0 +1,157 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import dataclasses +import datetime +from collections.abc import AsyncIterator, Iterator +from typing import Any, cast + +import sqlalchemy +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from db import models + + +CREATE_POST = """-- name: create_post \\:one +INSERT INTO posts (title, tags, status, created_at) VALUES (:p1, :p2, :p3, :p4) +RETURNING id, title, tags, status, payload, created_at +""" + + +GET_POST = """-- name: get_post \\:one +SELECT id, title, tags, status, payload, created_at FROM posts WHERE id = :p1 +""" + + +LIST_POSTS = """-- name: list_posts \\:many +SELECT id, title, tags, status, payload, created_at FROM posts ORDER BY id +""" + + +LIST_TITLES = """-- name: list_titles \\:many +SELECT id, title FROM posts WHERE status = :p1 +""" + + +@dataclasses.dataclass() +class ListTitlesRow: + id: int + title: str + + +class Querier: + _conn: sqlalchemy.engine.Connection | sqlalchemy.orm.Session + + def __init__(self, conn: sqlalchemy.engine.Connection | sqlalchemy.orm.Session): + self._conn = conn + + def create_post(self, *, title: str, tags: list[str] | None, status: models.Status | None, created_at: datetime.datetime) -> models.Post | None: + row = self._conn.execute(sqlalchemy.text(CREATE_POST), { + "p1": title, + "p2": tags, + "p3": status, + "p4": created_at, + }).first() + if row is None: + return None + return models.Post( + id=cast(int, row[0]), + title=cast(str, row[1]), + tags=cast(list[str] | None, row[2]), + status=cast(models.Status | None, row[3]), + payload=cast(Any | None, row[4]), + created_at=cast(datetime.datetime, row[5]), + ) + + def get_post(self, *, id: int) -> models.Post | None: + row = self._conn.execute(sqlalchemy.text(GET_POST), {"p1": id}).first() + if row is None: + return None + return models.Post( + id=cast(int, row[0]), + title=cast(str, row[1]), + tags=cast(list[str] | None, row[2]), + status=cast(models.Status | None, row[3]), + payload=cast(Any | None, row[4]), + created_at=cast(datetime.datetime, row[5]), + ) + + def list_posts(self) -> Iterator[models.Post]: + result = self._conn.execute(sqlalchemy.text(LIST_POSTS)) + for row in result: + yield models.Post( + id=cast(int, row[0]), + title=cast(str, row[1]), + tags=cast(list[str] | None, row[2]), + status=cast(models.Status | None, row[3]), + payload=cast(Any | None, row[4]), + created_at=cast(datetime.datetime, row[5]), + ) + + def list_titles(self, *, status: models.Status | None) -> Iterator[ListTitlesRow]: + result = self._conn.execute(sqlalchemy.text(LIST_TITLES), {"p1": status}) + for row in result: + yield ListTitlesRow( + id=cast(int, row[0]), + title=cast(str, row[1]), + ) + + +class AsyncQuerier: + _conn: sqlalchemy.ext.asyncio.AsyncConnection | sqlalchemy.ext.asyncio.AsyncSession + + def __init__(self, conn: sqlalchemy.ext.asyncio.AsyncConnection | sqlalchemy.ext.asyncio.AsyncSession): + self._conn = conn + + async def create_post(self, *, title: str, tags: list[str] | None, status: models.Status | None, created_at: datetime.datetime) -> models.Post | None: + row = (await self._conn.execute(sqlalchemy.text(CREATE_POST), { + "p1": title, + "p2": tags, + "p3": status, + "p4": created_at, + })).first() + if row is None: + return None + return models.Post( + id=cast(int, row[0]), + title=cast(str, row[1]), + tags=cast(list[str] | None, row[2]), + status=cast(models.Status | None, row[3]), + payload=cast(Any | None, row[4]), + created_at=cast(datetime.datetime, row[5]), + ) + + async def get_post(self, *, id: int) -> models.Post | None: + row = (await self._conn.execute(sqlalchemy.text(GET_POST), {"p1": id})).first() + if row is None: + return None + return models.Post( + id=cast(int, row[0]), + title=cast(str, row[1]), + tags=cast(list[str] | None, row[2]), + status=cast(models.Status | None, row[3]), + payload=cast(Any | None, row[4]), + created_at=cast(datetime.datetime, row[5]), + ) + + async def list_posts(self) -> AsyncIterator[models.Post]: + result = await self._conn.stream(sqlalchemy.text(LIST_POSTS)) + async for row in result: + yield models.Post( + id=cast(int, row[0]), + title=cast(str, row[1]), + tags=cast(list[str] | None, row[2]), + status=cast(models.Status | None, row[3]), + payload=cast(Any | None, row[4]), + created_at=cast(datetime.datetime, row[5]), + ) + + async def list_titles(self, *, status: models.Status | None) -> AsyncIterator[ListTitlesRow]: + result = await self._conn.stream(sqlalchemy.text(LIST_TITLES), {"p1": status}) + async for row in result: + yield ListTitlesRow( + id=cast(int, row[0]), + title=cast(str, row[1]), + ) diff --git a/internal/endtoend/testdata/emit_modern_types/query.sql b/internal/endtoend/testdata/emit_modern_types/query.sql new file mode 100644 index 0000000..432a26f --- /dev/null +++ b/internal/endtoend/testdata/emit_modern_types/query.sql @@ -0,0 +1,12 @@ +-- name: GetPost :one +SELECT * FROM posts WHERE id = $1; + +-- name: ListPosts :many +SELECT * FROM posts ORDER BY id; + +-- name: ListTitles :many +SELECT id, title FROM posts WHERE status = $1; + +-- name: CreatePost :one +INSERT INTO posts (title, tags, status, created_at) VALUES ($1, $2, $3, $4) +RETURNING *; diff --git a/internal/endtoend/testdata/emit_modern_types/schema.sql b/internal/endtoend/testdata/emit_modern_types/schema.sql new file mode 100644 index 0000000..5ab043c --- /dev/null +++ b/internal/endtoend/testdata/emit_modern_types/schema.sql @@ -0,0 +1,10 @@ +CREATE TYPE status AS ENUM ('draft', 'published'); + +CREATE TABLE posts ( + id BIGSERIAL PRIMARY KEY, + title text NOT NULL, + tags text[], + status status, + payload jsonb, + created_at timestamptz NOT NULL +); diff --git a/internal/endtoend/testdata/emit_modern_types/sqlc.yaml b/internal/endtoend/testdata/emit_modern_types/sqlc.yaml new file mode 100644 index 0000000..6dd87b7 --- /dev/null +++ b/internal/endtoend/testdata/emit_modern_types/sqlc.yaml @@ -0,0 +1,18 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + emit_async_querier: true + emit_modern_types: true diff --git a/internal/endtoend/testdata/emit_pydantic_models/db/models.py b/internal/endtoend/testdata/emit_pydantic_models/db/models.py index 7676e5c..69c6835 100644 --- a/internal/endtoend/testdata/emit_pydantic_models/db/models.py +++ b/internal/endtoend/testdata/emit_pydantic_models/db/models.py @@ -1,6 +1,6 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 import pydantic from typing import Optional diff --git a/internal/endtoend/testdata/emit_pydantic_models/db/query.py b/internal/endtoend/testdata/emit_pydantic_models/db/query.py index 6f5b76f..d31c850 100644 --- a/internal/endtoend/testdata/emit_pydantic_models/db/query.py +++ b/internal/endtoend/testdata/emit_pydantic_models/db/query.py @@ -1,11 +1,12 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 # source: query.sql -from typing import AsyncIterator, Iterator, Optional +from typing import AsyncIterator, Iterator, Optional, Union, cast import sqlalchemy import sqlalchemy.ext.asyncio +import sqlalchemy.orm from db import models @@ -39,7 +40,9 @@ class Querier: - def __init__(self, conn: sqlalchemy.engine.Connection): + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): self._conn = conn def create_author(self, *, name: str, bio: Optional[str]) -> Optional[models.Author]: @@ -47,9 +50,9 @@ def create_author(self, *, name: str, bio: Optional[str]) -> Optional[models.Aut if row is None: return None return models.Author( - id=row[0], - name=row[1], - bio=row[2], + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), ) def delete_author(self, *, id: int) -> None: @@ -60,23 +63,25 @@ def get_author(self, *, id: int) -> Optional[models.Author]: if row is None: return None return models.Author( - id=row[0], - name=row[1], - bio=row[2], + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), ) def list_authors(self) -> Iterator[models.Author]: result = self._conn.execute(sqlalchemy.text(LIST_AUTHORS)) for row in result: yield models.Author( - id=row[0], - name=row[1], - bio=row[2], + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), ) class AsyncQuerier: - def __init__(self, conn: sqlalchemy.ext.asyncio.AsyncConnection): + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): self._conn = conn async def create_author(self, *, name: str, bio: Optional[str]) -> Optional[models.Author]: @@ -84,9 +89,9 @@ async def create_author(self, *, name: str, bio: Optional[str]) -> Optional[mode if row is None: return None return models.Author( - id=row[0], - name=row[1], - bio=row[2], + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), ) async def delete_author(self, *, id: int) -> None: @@ -97,16 +102,16 @@ async def get_author(self, *, id: int) -> Optional[models.Author]: if row is None: return None return models.Author( - id=row[0], - name=row[1], - bio=row[2], + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), ) async def list_authors(self) -> AsyncIterator[models.Author]: result = await self._conn.stream(sqlalchemy.text(LIST_AUTHORS)) async for row in result: yield models.Author( - id=row[0], - name=row[1], - bio=row[2], + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), ) diff --git a/internal/endtoend/testdata/emit_pydantic_models/sqlc.yaml b/internal/endtoend/testdata/emit_pydantic_models/sqlc.yaml index beae200..2a743ee 100644 --- a/internal/endtoend/testdata/emit_pydantic_models/sqlc.yaml +++ b/internal/endtoend/testdata/emit_pydantic_models/sqlc.yaml @@ -3,7 +3,7 @@ plugins: - name: py wasm: url: file://../../../../bin/sqlc-gen-python.wasm - sha256: "d6846ffad948181e611e883cedd2d2be66e091edc1273a0abc6c9da18399e0ca" + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" sql: - schema: schema.sql queries: query.sql @@ -15,4 +15,4 @@ sql: package: db emit_sync_querier: true emit_async_querier: true - emit_pydantic_models: true \ No newline at end of file + emit_pydantic_models: true diff --git a/internal/endtoend/testdata/emit_querier_protocol/_check.py b/internal/endtoend/testdata/emit_querier_protocol/_check.py new file mode 100644 index 0000000..3dcf901 --- /dev/null +++ b/internal/endtoend/testdata/emit_querier_protocol/_check.py @@ -0,0 +1,47 @@ +from typing import Iterator, Optional, Sequence + +import sqlalchemy.engine +import sqlalchemy.ext.asyncio + +from db import models +from db import query + + +def conforms(conn: sqlalchemy.engine.Connection, aconn: sqlalchemy.ext.asyncio.AsyncConnection) -> None: + q: query.QuerierProtocol = query.Querier(conn) + aq: query.AsyncQuerierProtocol = query.AsyncQuerier(aconn) + + +class FakeQuerier: + def __init__(self) -> None: + self.authors: dict[int, models.Author] = {} + + def create_authors(self, arg: Sequence[query.CreateAuthorsParams]) -> int: + for a in arg: + id = len(self.authors) + 1 + self.authors[id] = models.Author(id=id, name=a.name, bio=a.bio) + return len(arg) + + def delete_author(self, *, id: int) -> None: + self.authors.pop(id, None) + + def get_author(self, *, id: int) -> Optional[models.Author]: + return self.authors.get(id) + + def get_authors_by_id(self, arg: Sequence[query.GetAuthorsByIDParams]) -> Iterator[Optional[models.Author]]: + for a in arg: + yield self.authors.get(a.id) + + def list_authors(self) -> Iterator[models.Author]: + yield from self.authors.values() + + def list_books_by_author(self, arg: Sequence[query.ListBooksByAuthorParams]) -> Iterator[list[models.Book]]: + for _ in arg: + yield [] + + +def use(q: query.QuerierProtocol) -> Optional[models.Author]: + return q.get_author(id=1) + + +use(FakeQuerier()) diff --git a/internal/endtoend/testdata/emit_querier_protocol/db/models.py b/internal/endtoend/testdata/emit_querier_protocol/db/models.py new file mode 100644 index 0000000..2a0c683 --- /dev/null +++ b/internal/endtoend/testdata/emit_querier_protocol/db/models.py @@ -0,0 +1,19 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses +from typing import Optional + + +@dataclasses.dataclass() +class Author: + id: int + name: str + bio: Optional[str] + + +@dataclasses.dataclass() +class Book: + id: int + author_id: int + title: str diff --git a/internal/endtoend/testdata/emit_querier_protocol/db/query.py b/internal/endtoend/testdata/emit_querier_protocol/db/query.py new file mode 100644 index 0000000..9989e2b --- /dev/null +++ b/internal/endtoend/testdata/emit_querier_protocol/db/query.py @@ -0,0 +1,196 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import dataclasses +from typing import Any, AsyncIterator, Iterator, List, Optional, Protocol, Sequence, Union, cast + +import sqlalchemy +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from db import models + + +CREATE_AUTHORS = """-- name: create_authors \\:copyfrom +INSERT INTO authors (name, bio) VALUES (:p1, :p2) +""" + + +@dataclasses.dataclass() +class CreateAuthorsParams: + name: str + bio: Optional[str] + + +DELETE_AUTHOR = """-- name: delete_author \\:exec +DELETE FROM authors WHERE id = :p1 +""" + + +GET_AUTHOR = """-- name: get_author \\:one +SELECT id, name, bio FROM authors WHERE id = :p1 +""" + + +GET_AUTHORS_BY_ID = """-- name: get_authors_by_id \\:batchone +SELECT id, name, bio FROM authors WHERE id = :p1 +""" + + +@dataclasses.dataclass() +class GetAuthorsByIDParams: + id: int + + +LIST_AUTHORS = """-- name: list_authors \\:many +SELECT id, name, bio FROM authors ORDER BY name +""" + + +LIST_BOOKS_BY_AUTHOR = """-- name: list_books_by_author \\:batchmany +SELECT id, author_id, title FROM books WHERE author_id = :p1 +""" + + +@dataclasses.dataclass() +class ListBooksByAuthorParams: + author_id: int + + +class QuerierProtocol(Protocol): + def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: ... + + def delete_author(self, *, id: int) -> None: ... + + def get_author(self, *, id: int) -> Optional[models.Author]: ... + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> Iterator[Optional[models.Author]]: ... + + def list_authors(self) -> Iterator[models.Author]: ... + + def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> Iterator[List[models.Book]]: ... + + +class AsyncQuerierProtocol(Protocol): + async def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: ... + + async def delete_author(self, *, id: int) -> None: ... + + async def get_author(self, *, id: int) -> Optional[models.Author]: ... + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> AsyncIterator[Optional[models.Author]]: ... + + def list_authors(self) -> AsyncIterator[models.Author]: ... + + def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> AsyncIterator[List[models.Book]]: ... + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: + if not arg: + return 0 + result = self._conn.execute(sqlalchemy.text(CREATE_AUTHORS), [{"p1": a.name, "p2": a.bio} for a in arg]) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + def delete_author(self, *, id: int) -> None: + self._conn.execute(sqlalchemy.text(DELETE_AUTHOR), {"p1": id}) + + def get_author(self, *, id: int) -> Optional[models.Author]: + row = self._conn.execute(sqlalchemy.text(GET_AUTHOR), {"p1": id}).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> Iterator[Optional[models.Author]]: + for a in arg: + row = self._conn.execute(sqlalchemy.text(GET_AUTHORS_BY_ID), {"p1": a.id}).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + def list_authors(self) -> Iterator[models.Author]: + result = self._conn.execute(sqlalchemy.text(LIST_AUTHORS)) + for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> Iterator[List[models.Book]]: + for a in arg: + yield [models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + ) for row in self._conn.execute(sqlalchemy.text(LIST_BOOKS_BY_AUTHOR), {"p1": a.author_id})] + + +class AsyncQuerier: + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): + self._conn = conn + + async def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: + if not arg: + return 0 + result = await self._conn.execute(sqlalchemy.text(CREATE_AUTHORS), [{"p1": a.name, "p2": a.bio} for a in arg]) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + async def delete_author(self, *, id: int) -> None: + await self._conn.execute(sqlalchemy.text(DELETE_AUTHOR), {"p1": id}) + + async def get_author(self, *, id: int) -> Optional[models.Author]: + row = (await self._conn.execute(sqlalchemy.text(GET_AUTHOR), {"p1": id})).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + async def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> AsyncIterator[Optional[models.Author]]: + for a in arg: + row = (await self._conn.execute(sqlalchemy.text(GET_AUTHORS_BY_ID), {"p1": a.id})).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + async def list_authors(self) -> AsyncIterator[models.Author]: + result = await self._conn.stream(sqlalchemy.text(LIST_AUTHORS)) + async for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + async def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> AsyncIterator[List[models.Book]]: + for a in arg: + yield [models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + ) for row in await self._conn.execute(sqlalchemy.text(LIST_BOOKS_BY_AUTHOR), {"p1": a.author_id})] diff --git a/internal/endtoend/testdata/emit_querier_protocol/query.sql b/internal/endtoend/testdata/emit_querier_protocol/query.sql new file mode 100644 index 0000000..72087cb --- /dev/null +++ b/internal/endtoend/testdata/emit_querier_protocol/query.sql @@ -0,0 +1,17 @@ +-- name: GetAuthor :one +SELECT * FROM authors WHERE id = $1; + +-- name: ListAuthors :many +SELECT * FROM authors ORDER BY name; + +-- name: CreateAuthors :copyfrom +INSERT INTO authors (name, bio) VALUES ($1, $2); + +-- name: GetAuthorsByID :batchone +SELECT * FROM authors WHERE id = $1; + +-- name: ListBooksByAuthor :batchmany +SELECT * FROM books WHERE author_id = $1; + +-- name: DeleteAuthor :exec +DELETE FROM authors WHERE id = $1; diff --git a/internal/endtoend/testdata/emit_querier_protocol/schema.sql b/internal/endtoend/testdata/emit_querier_protocol/schema.sql new file mode 100644 index 0000000..e54d7c5 --- /dev/null +++ b/internal/endtoend/testdata/emit_querier_protocol/schema.sql @@ -0,0 +1,11 @@ +CREATE TABLE authors ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL, + bio text +); + +CREATE TABLE books ( + id BIGSERIAL PRIMARY KEY, + author_id bigint NOT NULL REFERENCES authors (id), + title text NOT NULL +); diff --git a/internal/endtoend/testdata/emit_querier_protocol/sqlc.yaml b/internal/endtoend/testdata/emit_querier_protocol/sqlc.yaml new file mode 100644 index 0000000..37154a2 --- /dev/null +++ b/internal/endtoend/testdata/emit_querier_protocol/sqlc.yaml @@ -0,0 +1,18 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + emit_async_querier: true + emit_querier_protocol: true diff --git a/internal/endtoend/testdata/emit_query_errors/classic/errors.py b/internal/endtoend/testdata/emit_query_errors/classic/errors.py new file mode 100644 index 0000000..b5998e9 --- /dev/null +++ b/internal/endtoend/testdata/emit_query_errors/classic/errors.py @@ -0,0 +1,108 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import contextlib +from typing import Iterator, Optional + +import sqlalchemy.exc + + +class QueryError(Exception): + """Base class for all query errors.""" + + def __init__(self, message: str, query_name: str, cause: Optional[Exception] = None): + super().__init__(message) + self.query_name = query_name + self.cause = cause + self.constraint_name: Optional[str] = _constraint_name(cause) + + +class UniqueViolationError(QueryError): + """Raised on unique constraint violation (PG 23505).""" + + +class ForeignKeyViolationError(QueryError): + """Raised on foreign key constraint violation (PG 23503).""" + + +class CheckViolationError(QueryError): + """Raised on check constraint violation (PG 23514).""" + + +class NotNullViolationError(QueryError): + """Raised on not-null constraint violation (PG 23502).""" + + +class ExclusionViolationError(QueryError): + """Raised on exclusion constraint violation (PG 23P01).""" + + +class StatementTimeoutError(QueryError): + """Raised on statement timeout (PG 57014).""" + + +class DeadlockError(QueryError): + """Raised on deadlock detected (PG 40P01).""" + + +class SerializationError(QueryError): + """Raised on serialization failure (PG 40001).""" + + +def _sqlstate(e: sqlalchemy.exc.DBAPIError) -> Optional[str]: + # psycopg2 exposes pgcode, psycopg 3 sqlstate; SQLAlchemy's asyncpg + # adapter keeps the asyncpg error, which has sqlstate, as __cause__. + for source in (e.orig, getattr(e.orig, "__cause__", None)): + code = getattr(source, "pgcode", None) or getattr(source, "sqlstate", None) + if isinstance(code, str): + return code + return None + + +def _constraint_name(cause: Optional[Exception]) -> Optional[str]: + orig = getattr(cause, "orig", None) + name = getattr(getattr(orig, "diag", None), "constraint_name", None) + if name is None: + name = getattr(getattr(orig, "__cause__", None), "constraint_name", None) + return name if isinstance(name, str) else None + + +def _wrap_integrity_error(e: sqlalchemy.exc.IntegrityError, query_name: str) -> QueryError: + pgcode = _sqlstate(e) + if pgcode == "23505": + return UniqueViolationError(str(e), query_name, cause=e) + if pgcode == "23503": + return ForeignKeyViolationError(str(e), query_name, cause=e) + if pgcode == "23514": + return CheckViolationError(str(e), query_name, cause=e) + if pgcode == "23502": + return NotNullViolationError(str(e), query_name, cause=e) + if pgcode == "23P01": + return ExclusionViolationError(str(e), query_name, cause=e) + return QueryError(str(e), query_name, cause=e) + + +def _wrap_operational_error(e: sqlalchemy.exc.OperationalError, query_name: str) -> QueryError: + pgcode = _sqlstate(e) + if pgcode == "57014": + return StatementTimeoutError(str(e), query_name, cause=e) + if pgcode == "40P01": + return DeadlockError(str(e), query_name, cause=e) + if pgcode == "40001": + return SerializationError(str(e), query_name, cause=e) + return QueryError(str(e), query_name, cause=e) + + +@contextlib.contextmanager +def _wrap_errors(query_name: str) -> Iterator[None]: + """Re-raise integrity and operational errors as typed QueryErrors. + + Only those two SQLAlchemy exceptions are caught, so GeneratorExit from a + consumer that stops iterating early passes through untouched. + """ + try: + yield + except sqlalchemy.exc.IntegrityError as e: + raise _wrap_integrity_error(e, query_name) from e + except sqlalchemy.exc.OperationalError as e: + raise _wrap_operational_error(e, query_name) from e diff --git a/internal/endtoend/testdata/emit_query_errors/classic/models.py b/internal/endtoend/testdata/emit_query_errors/classic/models.py new file mode 100644 index 0000000..5e32fbe --- /dev/null +++ b/internal/endtoend/testdata/emit_query_errors/classic/models.py @@ -0,0 +1,22 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses +from typing import List, Optional + + +@dataclasses.dataclass() +class Author: + id: int + name: str + bio: Optional[str] + + +@dataclasses.dataclass() +class Job: + id: int + errors: str + cast: Optional[str] + list: Optional[List[str]] + id_2: Optional[int] + dict: Optional[dict[str, int]] diff --git a/internal/endtoend/testdata/emit_query_errors/classic/query.py b/internal/endtoend/testdata/emit_query_errors/classic/query.py new file mode 100644 index 0000000..427b089 --- /dev/null +++ b/internal/endtoend/testdata/emit_query_errors/classic/query.py @@ -0,0 +1,272 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import dataclasses +from typing import AsyncIterator, Iterator, List, Optional, Sequence, Union, cast + +import sqlalchemy +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from classic import errors, models + + +ADD_JOB = """-- name: add_job \\:one +INSERT INTO jobs (errors, "cast") VALUES (:p1, :p2) RETURNING id, errors, "cast", list, id_2, dict +""" + + +CREATE_AUTHOR = """-- name: create_author \\:one +INSERT INTO authors (name, bio) VALUES (:p1, :p2) RETURNING id, name, bio +""" + + +DELETE_AUTHOR = """-- name: delete_author \\:exec +DELETE FROM authors WHERE id = :p1 +""" + + +GET_AUTHORS_BY_ID = """-- name: get_authors_by_id \\:batchone +SELECT id, name, bio FROM authors WHERE id = :p1 +""" + + +@dataclasses.dataclass() +class GetAuthorsByIDParams: + id: int + + +LIST_AUTHORS = """-- name: list_authors \\:many +SELECT id, name, bio FROM authors ORDER BY name +""" + + +LIST_JOBS_BY_DICT = """-- name: list_jobs_by_dict \\:many +SELECT id, errors, "cast", list, id_2, dict FROM jobs WHERE dict = :p1 +""" + + +LIST_JOBS_IN_RANGE = """-- name: list_jobs_in_range \\:many +SELECT id, errors, "cast", list, id_2, dict FROM jobs WHERE id > :p1 AND id < :p2 +""" + + +PROBE_JOBS = """-- name: probe_jobs \\:many +SELECT id, id_2, id, list, dict FROM jobs WHERE list = :p1 AND id > :p2 AND id < :p3 AND id_2 = :p4 +""" + + +@dataclasses.dataclass() +class ProbeJobsRow: + id: int + id_2: Optional[int] + id_3: int + list: Optional[List[str]] + dict: Optional[dict[str, int]] + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def add_job(self, *, errors_: str, cast_: Optional[str]) -> Optional[models.Job]: + with errors._wrap_errors("add_job"): + row = self._conn.execute(sqlalchemy.text(ADD_JOB), {"p1": errors_, "p2": cast_}).first() + if row is None: + return None + return models.Job( + id=cast(int, row[0]), + errors=cast(str, row[1]), + cast=cast(Optional[str], row[2]), + list=cast(Optional[List[str]], row[3]), + id_2=cast(Optional[int], row[4]), + dict=cast(Optional[dict[str, int]], row[5]), + ) + + def create_author(self, *, name: str, bio: Optional[str]) -> Optional[models.Author]: + """Insert an author.""" + with errors._wrap_errors("create_author"): + row = self._conn.execute(sqlalchemy.text(CREATE_AUTHOR), {"p1": name, "p2": bio}).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + def delete_author(self, *, id: int) -> None: + with errors._wrap_errors("delete_author"): + self._conn.execute(sqlalchemy.text(DELETE_AUTHOR), {"p1": id}) + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> Iterator[Optional[models.Author]]: + with errors._wrap_errors("get_authors_by_id"): + for a in arg: + row = self._conn.execute(sqlalchemy.text(GET_AUTHORS_BY_ID), {"p1": a.id}).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + def list_authors(self) -> Iterator[models.Author]: + with errors._wrap_errors("list_authors"): + result = self._conn.execute(sqlalchemy.text(LIST_AUTHORS)) + for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + def list_jobs_by_dict(self, *, dict_: Optional[dict[str, int]]) -> Iterator[models.Job]: + with errors._wrap_errors("list_jobs_by_dict"): + result = self._conn.execute(sqlalchemy.text(LIST_JOBS_BY_DICT), {"p1": dict_}) + for row in result: + yield models.Job( + id=cast(int, row[0]), + errors=cast(str, row[1]), + cast=cast(Optional[str], row[2]), + list=cast(Optional[List[str]], row[3]), + id_2=cast(Optional[int], row[4]), + dict=cast(Optional[dict[str, int]], row[5]), + ) + + def list_jobs_in_range(self, *, id: int, id_2: int) -> Iterator[models.Job]: + with errors._wrap_errors("list_jobs_in_range"): + result = self._conn.execute(sqlalchemy.text(LIST_JOBS_IN_RANGE), {"p1": id, "p2": id_2}) + for row in result: + yield models.Job( + id=cast(int, row[0]), + errors=cast(str, row[1]), + cast=cast(Optional[str], row[2]), + list=cast(Optional[List[str]], row[3]), + id_2=cast(Optional[int], row[4]), + dict=cast(Optional[dict[str, int]], row[5]), + ) + + def probe_jobs(self, *, list_: Optional[List[str]], id: int, id_2: int, id_2_2: Optional[int]) -> Iterator[ProbeJobsRow]: + with errors._wrap_errors("probe_jobs"): + result = self._conn.execute(sqlalchemy.text(PROBE_JOBS), { + "p1": list_, + "p2": id, + "p3": id_2, + "p4": id_2_2, + }) + for row in result: + yield ProbeJobsRow( + id=cast(int, row[0]), + id_2=cast(Optional[int], row[1]), + id_3=cast(int, row[2]), + list=cast(Optional[List[str]], row[3]), + dict=cast(Optional[dict[str, int]], row[4]), + ) + + +class AsyncQuerier: + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): + self._conn = conn + + async def add_job(self, *, errors_: str, cast_: Optional[str]) -> Optional[models.Job]: + with errors._wrap_errors("add_job"): + row = (await self._conn.execute(sqlalchemy.text(ADD_JOB), {"p1": errors_, "p2": cast_})).first() + if row is None: + return None + return models.Job( + id=cast(int, row[0]), + errors=cast(str, row[1]), + cast=cast(Optional[str], row[2]), + list=cast(Optional[List[str]], row[3]), + id_2=cast(Optional[int], row[4]), + dict=cast(Optional[dict[str, int]], row[5]), + ) + + async def create_author(self, *, name: str, bio: Optional[str]) -> Optional[models.Author]: + """Insert an author.""" + with errors._wrap_errors("create_author"): + row = (await self._conn.execute(sqlalchemy.text(CREATE_AUTHOR), {"p1": name, "p2": bio})).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + async def delete_author(self, *, id: int) -> None: + with errors._wrap_errors("delete_author"): + await self._conn.execute(sqlalchemy.text(DELETE_AUTHOR), {"p1": id}) + + async def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> AsyncIterator[Optional[models.Author]]: + with errors._wrap_errors("get_authors_by_id"): + for a in arg: + row = (await self._conn.execute(sqlalchemy.text(GET_AUTHORS_BY_ID), {"p1": a.id})).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + async def list_authors(self) -> AsyncIterator[models.Author]: + with errors._wrap_errors("list_authors"): + result = await self._conn.stream(sqlalchemy.text(LIST_AUTHORS)) + async for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + async def list_jobs_by_dict(self, *, dict_: Optional[dict[str, int]]) -> AsyncIterator[models.Job]: + with errors._wrap_errors("list_jobs_by_dict"): + result = await self._conn.stream(sqlalchemy.text(LIST_JOBS_BY_DICT), {"p1": dict_}) + async for row in result: + yield models.Job( + id=cast(int, row[0]), + errors=cast(str, row[1]), + cast=cast(Optional[str], row[2]), + list=cast(Optional[List[str]], row[3]), + id_2=cast(Optional[int], row[4]), + dict=cast(Optional[dict[str, int]], row[5]), + ) + + async def list_jobs_in_range(self, *, id: int, id_2: int) -> AsyncIterator[models.Job]: + with errors._wrap_errors("list_jobs_in_range"): + result = await self._conn.stream(sqlalchemy.text(LIST_JOBS_IN_RANGE), {"p1": id, "p2": id_2}) + async for row in result: + yield models.Job( + id=cast(int, row[0]), + errors=cast(str, row[1]), + cast=cast(Optional[str], row[2]), + list=cast(Optional[List[str]], row[3]), + id_2=cast(Optional[int], row[4]), + dict=cast(Optional[dict[str, int]], row[5]), + ) + + async def probe_jobs(self, *, list_: Optional[List[str]], id: int, id_2: int, id_2_2: Optional[int]) -> AsyncIterator[ProbeJobsRow]: + with errors._wrap_errors("probe_jobs"): + result = await self._conn.stream(sqlalchemy.text(PROBE_JOBS), { + "p1": list_, + "p2": id, + "p3": id_2, + "p4": id_2_2, + }) + async for row in result: + yield ProbeJobsRow( + id=cast(int, row[0]), + id_2=cast(Optional[int], row[1]), + id_3=cast(int, row[2]), + list=cast(Optional[List[str]], row[3]), + dict=cast(Optional[dict[str, int]], row[4]), + ) diff --git a/internal/endtoend/testdata/emit_query_errors/modern/errors.py b/internal/endtoend/testdata/emit_query_errors/modern/errors.py new file mode 100644 index 0000000..bc58233 --- /dev/null +++ b/internal/endtoend/testdata/emit_query_errors/modern/errors.py @@ -0,0 +1,108 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import contextlib +from collections.abc import Iterator + +import sqlalchemy.exc + + +class QueryError(Exception): + """Base class for all query errors.""" + + def __init__(self, message: str, query_name: str, cause: Exception | None = None): + super().__init__(message) + self.query_name = query_name + self.cause = cause + self.constraint_name: str | None = _constraint_name(cause) + + +class UniqueViolationError(QueryError): + """Raised on unique constraint violation (PG 23505).""" + + +class ForeignKeyViolationError(QueryError): + """Raised on foreign key constraint violation (PG 23503).""" + + +class CheckViolationError(QueryError): + """Raised on check constraint violation (PG 23514).""" + + +class NotNullViolationError(QueryError): + """Raised on not-null constraint violation (PG 23502).""" + + +class ExclusionViolationError(QueryError): + """Raised on exclusion constraint violation (PG 23P01).""" + + +class StatementTimeoutError(QueryError): + """Raised on statement timeout (PG 57014).""" + + +class DeadlockError(QueryError): + """Raised on deadlock detected (PG 40P01).""" + + +class SerializationError(QueryError): + """Raised on serialization failure (PG 40001).""" + + +def _sqlstate(e: sqlalchemy.exc.DBAPIError) -> str | None: + # psycopg2 exposes pgcode, psycopg 3 sqlstate; SQLAlchemy's asyncpg + # adapter keeps the asyncpg error, which has sqlstate, as __cause__. + for source in (e.orig, getattr(e.orig, "__cause__", None)): + code = getattr(source, "pgcode", None) or getattr(source, "sqlstate", None) + if isinstance(code, str): + return code + return None + + +def _constraint_name(cause: Exception | None) -> str | None: + orig = getattr(cause, "orig", None) + name = getattr(getattr(orig, "diag", None), "constraint_name", None) + if name is None: + name = getattr(getattr(orig, "__cause__", None), "constraint_name", None) + return name if isinstance(name, str) else None + + +def _wrap_integrity_error(e: sqlalchemy.exc.IntegrityError, query_name: str) -> QueryError: + pgcode = _sqlstate(e) + if pgcode == "23505": + return UniqueViolationError(str(e), query_name, cause=e) + if pgcode == "23503": + return ForeignKeyViolationError(str(e), query_name, cause=e) + if pgcode == "23514": + return CheckViolationError(str(e), query_name, cause=e) + if pgcode == "23502": + return NotNullViolationError(str(e), query_name, cause=e) + if pgcode == "23P01": + return ExclusionViolationError(str(e), query_name, cause=e) + return QueryError(str(e), query_name, cause=e) + + +def _wrap_operational_error(e: sqlalchemy.exc.OperationalError, query_name: str) -> QueryError: + pgcode = _sqlstate(e) + if pgcode == "57014": + return StatementTimeoutError(str(e), query_name, cause=e) + if pgcode == "40P01": + return DeadlockError(str(e), query_name, cause=e) + if pgcode == "40001": + return SerializationError(str(e), query_name, cause=e) + return QueryError(str(e), query_name, cause=e) + + +@contextlib.contextmanager +def _wrap_errors(query_name: str) -> Iterator[None]: + """Re-raise integrity and operational errors as typed QueryErrors. + + Only those two SQLAlchemy exceptions are caught, so GeneratorExit from a + consumer that stops iterating early passes through untouched. + """ + try: + yield + except sqlalchemy.exc.IntegrityError as e: + raise _wrap_integrity_error(e, query_name) from e + except sqlalchemy.exc.OperationalError as e: + raise _wrap_operational_error(e, query_name) from e diff --git a/internal/endtoend/testdata/emit_query_errors/modern/models.py b/internal/endtoend/testdata/emit_query_errors/modern/models.py new file mode 100644 index 0000000..f2ab455 --- /dev/null +++ b/internal/endtoend/testdata/emit_query_errors/modern/models.py @@ -0,0 +1,21 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses + + +@dataclasses.dataclass() +class Author: + id: int + name: str + bio: str | None + + +@dataclasses.dataclass() +class Job: + id: int + errors: str + cast: str | None + list: list[str] | None + id_2: int | None + dict: dict[str, int] | None diff --git a/internal/endtoend/testdata/emit_query_errors/modern/query.py b/internal/endtoend/testdata/emit_query_errors/modern/query.py new file mode 100644 index 0000000..b4cd9e1 --- /dev/null +++ b/internal/endtoend/testdata/emit_query_errors/modern/query.py @@ -0,0 +1,273 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import dataclasses +from collections.abc import AsyncIterator, Iterator, Sequence +from typing import cast + +import sqlalchemy +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from modern import errors, models + + +ADD_JOB = """-- name: add_job \\:one +INSERT INTO jobs (errors, "cast") VALUES (:p1, :p2) RETURNING id, errors, "cast", list, id_2, dict +""" + + +CREATE_AUTHOR = """-- name: create_author \\:one +INSERT INTO authors (name, bio) VALUES (:p1, :p2) RETURNING id, name, bio +""" + + +DELETE_AUTHOR = """-- name: delete_author \\:exec +DELETE FROM authors WHERE id = :p1 +""" + + +GET_AUTHORS_BY_ID = """-- name: get_authors_by_id \\:batchone +SELECT id, name, bio FROM authors WHERE id = :p1 +""" + + +@dataclasses.dataclass() +class GetAuthorsByIDParams: + id: int + + +LIST_AUTHORS = """-- name: list_authors \\:many +SELECT id, name, bio FROM authors ORDER BY name +""" + + +LIST_JOBS_BY_DICT = """-- name: list_jobs_by_dict \\:many +SELECT id, errors, "cast", list, id_2, dict FROM jobs WHERE dict = :p1 +""" + + +LIST_JOBS_IN_RANGE = """-- name: list_jobs_in_range \\:many +SELECT id, errors, "cast", list, id_2, dict FROM jobs WHERE id > :p1 AND id < :p2 +""" + + +PROBE_JOBS = """-- name: probe_jobs \\:many +SELECT id, id_2, id, list, dict FROM jobs WHERE list = :p1 AND id > :p2 AND id < :p3 AND id_2 = :p4 +""" + + +@dataclasses.dataclass() +class ProbeJobsRow: + id: int + id_2: int | None + id_3: int + list: list[str] | None + dict: dict[str, int] | None + + +class Querier: + _conn: sqlalchemy.engine.Connection | sqlalchemy.orm.Session + + def __init__(self, conn: sqlalchemy.engine.Connection | sqlalchemy.orm.Session): + self._conn = conn + + def add_job(self, *, errors_: str, cast_: str | None) -> models.Job | None: + with errors._wrap_errors("add_job"): + row = self._conn.execute(sqlalchemy.text(ADD_JOB), {"p1": errors_, "p2": cast_}).first() + if row is None: + return None + return models.Job( + id=cast(int, row[0]), + errors=cast(str, row[1]), + cast=cast(str | None, row[2]), + list=cast(list[str] | None, row[3]), + id_2=cast(int | None, row[4]), + dict=cast(dict[str, int] | None, row[5]), + ) + + def create_author(self, *, name: str, bio: str | None) -> models.Author | None: + """Insert an author.""" + with errors._wrap_errors("create_author"): + row = self._conn.execute(sqlalchemy.text(CREATE_AUTHOR), {"p1": name, "p2": bio}).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + ) + + def delete_author(self, *, id: int) -> None: + with errors._wrap_errors("delete_author"): + self._conn.execute(sqlalchemy.text(DELETE_AUTHOR), {"p1": id}) + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> Iterator[models.Author | None]: + with errors._wrap_errors("get_authors_by_id"): + for a in arg: + row = self._conn.execute(sqlalchemy.text(GET_AUTHORS_BY_ID), {"p1": a.id}).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + ) + + def list_authors(self) -> Iterator[models.Author]: + with errors._wrap_errors("list_authors"): + result = self._conn.execute(sqlalchemy.text(LIST_AUTHORS)) + for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + ) + + def list_jobs_by_dict(self, *, dict_: dict[str, int] | None) -> Iterator[models.Job]: + with errors._wrap_errors("list_jobs_by_dict"): + result = self._conn.execute(sqlalchemy.text(LIST_JOBS_BY_DICT), {"p1": dict_}) + for row in result: + yield models.Job( + id=cast(int, row[0]), + errors=cast(str, row[1]), + cast=cast(str | None, row[2]), + list=cast(list[str] | None, row[3]), + id_2=cast(int | None, row[4]), + dict=cast(dict[str, int] | None, row[5]), + ) + + def list_jobs_in_range(self, *, id: int, id_2: int) -> Iterator[models.Job]: + with errors._wrap_errors("list_jobs_in_range"): + result = self._conn.execute(sqlalchemy.text(LIST_JOBS_IN_RANGE), {"p1": id, "p2": id_2}) + for row in result: + yield models.Job( + id=cast(int, row[0]), + errors=cast(str, row[1]), + cast=cast(str | None, row[2]), + list=cast(list[str] | None, row[3]), + id_2=cast(int | None, row[4]), + dict=cast(dict[str, int] | None, row[5]), + ) + + def probe_jobs(self, *, list_: list[str] | None, id: int, id_2: int, id_2_2: int | None) -> Iterator[ProbeJobsRow]: + with errors._wrap_errors("probe_jobs"): + result = self._conn.execute(sqlalchemy.text(PROBE_JOBS), { + "p1": list_, + "p2": id, + "p3": id_2, + "p4": id_2_2, + }) + for row in result: + yield ProbeJobsRow( + id=cast(int, row[0]), + id_2=cast(int | None, row[1]), + id_3=cast(int, row[2]), + list=cast(list[str] | None, row[3]), + dict=cast(dict[str, int] | None, row[4]), + ) + + +class AsyncQuerier: + _conn: sqlalchemy.ext.asyncio.AsyncConnection | sqlalchemy.ext.asyncio.AsyncSession + + def __init__(self, conn: sqlalchemy.ext.asyncio.AsyncConnection | sqlalchemy.ext.asyncio.AsyncSession): + self._conn = conn + + async def add_job(self, *, errors_: str, cast_: str | None) -> models.Job | None: + with errors._wrap_errors("add_job"): + row = (await self._conn.execute(sqlalchemy.text(ADD_JOB), {"p1": errors_, "p2": cast_})).first() + if row is None: + return None + return models.Job( + id=cast(int, row[0]), + errors=cast(str, row[1]), + cast=cast(str | None, row[2]), + list=cast(list[str] | None, row[3]), + id_2=cast(int | None, row[4]), + dict=cast(dict[str, int] | None, row[5]), + ) + + async def create_author(self, *, name: str, bio: str | None) -> models.Author | None: + """Insert an author.""" + with errors._wrap_errors("create_author"): + row = (await self._conn.execute(sqlalchemy.text(CREATE_AUTHOR), {"p1": name, "p2": bio})).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + ) + + async def delete_author(self, *, id: int) -> None: + with errors._wrap_errors("delete_author"): + await self._conn.execute(sqlalchemy.text(DELETE_AUTHOR), {"p1": id}) + + async def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> AsyncIterator[models.Author | None]: + with errors._wrap_errors("get_authors_by_id"): + for a in arg: + row = (await self._conn.execute(sqlalchemy.text(GET_AUTHORS_BY_ID), {"p1": a.id})).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + ) + + async def list_authors(self) -> AsyncIterator[models.Author]: + with errors._wrap_errors("list_authors"): + result = await self._conn.stream(sqlalchemy.text(LIST_AUTHORS)) + async for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + ) + + async def list_jobs_by_dict(self, *, dict_: dict[str, int] | None) -> AsyncIterator[models.Job]: + with errors._wrap_errors("list_jobs_by_dict"): + result = await self._conn.stream(sqlalchemy.text(LIST_JOBS_BY_DICT), {"p1": dict_}) + async for row in result: + yield models.Job( + id=cast(int, row[0]), + errors=cast(str, row[1]), + cast=cast(str | None, row[2]), + list=cast(list[str] | None, row[3]), + id_2=cast(int | None, row[4]), + dict=cast(dict[str, int] | None, row[5]), + ) + + async def list_jobs_in_range(self, *, id: int, id_2: int) -> AsyncIterator[models.Job]: + with errors._wrap_errors("list_jobs_in_range"): + result = await self._conn.stream(sqlalchemy.text(LIST_JOBS_IN_RANGE), {"p1": id, "p2": id_2}) + async for row in result: + yield models.Job( + id=cast(int, row[0]), + errors=cast(str, row[1]), + cast=cast(str | None, row[2]), + list=cast(list[str] | None, row[3]), + id_2=cast(int | None, row[4]), + dict=cast(dict[str, int] | None, row[5]), + ) + + async def probe_jobs(self, *, list_: list[str] | None, id: int, id_2: int, id_2_2: int | None) -> AsyncIterator[ProbeJobsRow]: + with errors._wrap_errors("probe_jobs"): + result = await self._conn.stream(sqlalchemy.text(PROBE_JOBS), { + "p1": list_, + "p2": id, + "p3": id_2, + "p4": id_2_2, + }) + async for row in result: + yield ProbeJobsRow( + id=cast(int, row[0]), + id_2=cast(int | None, row[1]), + id_3=cast(int, row[2]), + list=cast(list[str] | None, row[3]), + dict=cast(dict[str, int] | None, row[4]), + ) diff --git a/internal/endtoend/testdata/emit_query_errors/query.sql b/internal/endtoend/testdata/emit_query_errors/query.sql new file mode 100644 index 0000000..4ad1be2 --- /dev/null +++ b/internal/endtoend/testdata/emit_query_errors/query.sql @@ -0,0 +1,24 @@ +-- Insert an author. +-- name: CreateAuthor :one +INSERT INTO authors (name, bio) VALUES ($1, $2) RETURNING *; + +-- name: ListAuthors :many +SELECT * FROM authors ORDER BY name; + +-- name: GetAuthorsByID :batchone +SELECT * FROM authors WHERE id = $1; + +-- name: DeleteAuthor :exec +DELETE FROM authors WHERE id = $1; + +-- name: AddJob :one +INSERT INTO jobs (errors, "cast") VALUES ($1, $2) RETURNING *; + +-- name: ListJobsInRange :many +SELECT * FROM jobs WHERE id > $1 AND id < $2; + +-- name: ProbeJobs :many +SELECT id, id_2, id, list, dict FROM jobs WHERE list = $1 AND id > $2 AND id < $3 AND id_2 = $4; + +-- name: ListJobsByDict :many +SELECT * FROM jobs WHERE dict = $1; diff --git a/internal/endtoend/testdata/emit_query_errors/schema.sql b/internal/endtoend/testdata/emit_query_errors/schema.sql new file mode 100644 index 0000000..543f6b8 --- /dev/null +++ b/internal/endtoend/testdata/emit_query_errors/schema.sql @@ -0,0 +1,14 @@ +CREATE TABLE authors ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL UNIQUE, + bio text +); + +CREATE TABLE jobs ( + id BIGSERIAL PRIMARY KEY, + errors text NOT NULL, + "cast" text, + list text[], + id_2 bigint, + dict jsonb +); diff --git a/internal/endtoend/testdata/emit_query_errors/sqlc.yaml b/internal/endtoend/testdata/emit_query_errors/sqlc.yaml new file mode 100644 index 0000000..e1d6403 --- /dev/null +++ b/internal/endtoend/testdata/emit_query_errors/sqlc.yaml @@ -0,0 +1,32 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: classic + options: + package: classic + emit_sync_querier: true + emit_async_querier: true + emit_query_errors: true + overrides: + - column: jobs.dict + py_type: dict[str, int] + - plugin: py + out: modern + options: + package: modern + emit_sync_querier: true + emit_async_querier: true + emit_query_errors: true + overrides: + - column: jobs.dict + py_type: dict[str, int] + emit_modern_types: true diff --git a/internal/endtoend/testdata/emit_query_errors_conflict/errors.sql b/internal/endtoend/testdata/emit_query_errors_conflict/errors.sql new file mode 100644 index 0000000..fb46e56 --- /dev/null +++ b/internal/endtoend/testdata/emit_query_errors_conflict/errors.sql @@ -0,0 +1,2 @@ +-- name: CreateAuthor :one +INSERT INTO authors (name, bio) VALUES ($1, $2) RETURNING *; diff --git a/internal/endtoend/testdata/emit_query_errors_conflict/schema.sql b/internal/endtoend/testdata/emit_query_errors_conflict/schema.sql new file mode 100644 index 0000000..575dd39 --- /dev/null +++ b/internal/endtoend/testdata/emit_query_errors_conflict/schema.sql @@ -0,0 +1,5 @@ +CREATE TABLE authors ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL UNIQUE, + bio text +); diff --git a/internal/endtoend/testdata/emit_query_errors_conflict/sqlc.yaml b/internal/endtoend/testdata/emit_query_errors_conflict/sqlc.yaml new file mode 100644 index 0000000..9ccc7e9 --- /dev/null +++ b/internal/endtoend/testdata/emit_query_errors_conflict/sqlc.yaml @@ -0,0 +1,17 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: errors.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + emit_query_errors: true diff --git a/internal/endtoend/testdata/emit_query_errors_conflict/stderr.txt b/internal/endtoend/testdata/emit_query_errors_conflict/stderr.txt new file mode 100644 index 0000000..702b476 --- /dev/null +++ b/internal/endtoend/testdata/emit_query_errors_conflict/stderr.txt @@ -0,0 +1,2 @@ +# package py +error generating code: error generating output: query file errors.sql would overwrite errors.py, which emit_query_errors generates diff --git a/internal/endtoend/testdata/emit_str_enum/db/models.py b/internal/endtoend/testdata/emit_str_enum/db/models.py index 5fdf754..8648be8 100644 --- a/internal/endtoend/testdata/emit_str_enum/db/models.py +++ b/internal/endtoend/testdata/emit_str_enum/db/models.py @@ -1,6 +1,6 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 import dataclasses import enum from typing import Optional @@ -12,8 +12,24 @@ class BookStatus(enum.StrEnum): OVERDUE = "overdue" +class Op(enum.StrEnum): + VALUE_1 = "=" + VALUE_2 = "<>" + IN_PROGRESS = "in-progress" + IN_PROGRESS_2 = "in_progress" + VALUE_1ST = "1st" + SAYHI = "say \"hi\"" + BACKSLASH = "back\\slash" + + @dataclasses.dataclass() class Book: id: int title: str status: Optional[BookStatus] + + +@dataclasses.dataclass() +class Rule: + id: int + op: Op diff --git a/internal/endtoend/testdata/emit_str_enum/db/query.py b/internal/endtoend/testdata/emit_str_enum/db/query.py index 8082889..6f7330e 100644 --- a/internal/endtoend/testdata/emit_str_enum/db/query.py +++ b/internal/endtoend/testdata/emit_str_enum/db/query.py @@ -1,11 +1,12 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 # source: query.sql -from typing import AsyncIterator, Iterator, Optional +from typing import AsyncIterator, Iterator, Optional, Union, cast import sqlalchemy import sqlalchemy.ext.asyncio +import sqlalchemy.orm from db import models @@ -38,7 +39,9 @@ class Querier: - def __init__(self, conn: sqlalchemy.engine.Connection): + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): self._conn = conn def create_book(self, *, title: str, status: Optional[models.BookStatus]) -> Optional[models.Book]: @@ -46,9 +49,9 @@ def create_book(self, *, title: str, status: Optional[models.BookStatus]) -> Opt if row is None: return None return models.Book( - id=row[0], - title=row[1], - status=row[2], + id=cast(int, row[0]), + title=cast(str, row[1]), + status=cast(Optional[models.BookStatus], row[2]), ) def delete_book(self, *, id: int) -> None: @@ -59,23 +62,25 @@ def get_book(self, *, id: int) -> Optional[models.Book]: if row is None: return None return models.Book( - id=row[0], - title=row[1], - status=row[2], + id=cast(int, row[0]), + title=cast(str, row[1]), + status=cast(Optional[models.BookStatus], row[2]), ) def list_books(self) -> Iterator[models.Book]: result = self._conn.execute(sqlalchemy.text(LIST_BOOKS)) for row in result: yield models.Book( - id=row[0], - title=row[1], - status=row[2], + id=cast(int, row[0]), + title=cast(str, row[1]), + status=cast(Optional[models.BookStatus], row[2]), ) class AsyncQuerier: - def __init__(self, conn: sqlalchemy.ext.asyncio.AsyncConnection): + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): self._conn = conn async def create_book(self, *, title: str, status: Optional[models.BookStatus]) -> Optional[models.Book]: @@ -83,9 +88,9 @@ async def create_book(self, *, title: str, status: Optional[models.BookStatus]) if row is None: return None return models.Book( - id=row[0], - title=row[1], - status=row[2], + id=cast(int, row[0]), + title=cast(str, row[1]), + status=cast(Optional[models.BookStatus], row[2]), ) async def delete_book(self, *, id: int) -> None: @@ -96,16 +101,16 @@ async def get_book(self, *, id: int) -> Optional[models.Book]: if row is None: return None return models.Book( - id=row[0], - title=row[1], - status=row[2], + id=cast(int, row[0]), + title=cast(str, row[1]), + status=cast(Optional[models.BookStatus], row[2]), ) async def list_books(self) -> AsyncIterator[models.Book]: result = await self._conn.stream(sqlalchemy.text(LIST_BOOKS)) async for row in result: yield models.Book( - id=row[0], - title=row[1], - status=row[2], + id=cast(int, row[0]), + title=cast(str, row[1]), + status=cast(Optional[models.BookStatus], row[2]), ) diff --git a/internal/endtoend/testdata/emit_str_enum/schema.sql b/internal/endtoend/testdata/emit_str_enum/schema.sql index 6842b27..9cb8ab7 100644 --- a/internal/endtoend/testdata/emit_str_enum/schema.sql +++ b/internal/endtoend/testdata/emit_str_enum/schema.sql @@ -6,3 +6,10 @@ CREATE TABLE books ( title text NOT NULL, status book_status DEFAULT 'available' ); + +CREATE TYPE op AS ENUM ('=', '<>', 'in-progress', 'in_progress', '1st', 'say "hi"', 'back\slash'); + +CREATE TABLE rules ( + id BIGSERIAL PRIMARY KEY, + op op NOT NULL +); diff --git a/internal/endtoend/testdata/emit_str_enum/sqlc.yaml b/internal/endtoend/testdata/emit_str_enum/sqlc.yaml index 04e3feb..2bf6c0f 100644 --- a/internal/endtoend/testdata/emit_str_enum/sqlc.yaml +++ b/internal/endtoend/testdata/emit_str_enum/sqlc.yaml @@ -3,7 +3,7 @@ plugins: - name: py wasm: url: file://../../../../bin/sqlc-gen-python.wasm - sha256: "d6846ffad948181e611e883cedd2d2be66e091edc1273a0abc6c9da18399e0ca" + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" sql: - schema: schema.sql queries: query.sql @@ -16,4 +16,3 @@ sql: emit_sync_querier: true emit_async_querier: true emit_str_enum: true - diff --git a/internal/endtoend/testdata/empty_params_class/db/models.py b/internal/endtoend/testdata/empty_params_class/db/models.py new file mode 100644 index 0000000..534b934 --- /dev/null +++ b/internal/endtoend/testdata/empty_params_class/db/models.py @@ -0,0 +1,10 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses + + +@dataclasses.dataclass() +class Author: + id: int + name: str diff --git a/internal/endtoend/testdata/empty_params_class/db/query.py b/internal/endtoend/testdata/empty_params_class/db/query.py new file mode 100644 index 0000000..2a12edf --- /dev/null +++ b/internal/endtoend/testdata/empty_params_class/db/query.py @@ -0,0 +1,52 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import dataclasses +from typing import Iterator, Optional, Union, cast + +import sqlalchemy +import sqlalchemy.orm + +from db import models + + +COUNT_AUTHORS = """-- name: count_authors \\:one +SELECT count(*) FROM authors +""" + + +@dataclasses.dataclass() +class CountAuthorsParams: + pass + + +LIST_AUTHORS = """-- name: list_authors \\:many +SELECT id, name FROM authors +""" + + +@dataclasses.dataclass() +class ListAuthorsParams: + pass + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def count_authors(self, arg: CountAuthorsParams) -> Optional[int]: + row = self._conn.execute(sqlalchemy.text(COUNT_AUTHORS)).first() + if row is None: + return None + return cast(int, row[0]) + + def list_authors(self, arg: ListAuthorsParams) -> Iterator[models.Author]: + result = self._conn.execute(sqlalchemy.text(LIST_AUTHORS)) + for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + ) diff --git a/internal/endtoend/testdata/empty_params_class/query.sql b/internal/endtoend/testdata/empty_params_class/query.sql new file mode 100644 index 0000000..f669e9a --- /dev/null +++ b/internal/endtoend/testdata/empty_params_class/query.sql @@ -0,0 +1,5 @@ +-- name: ListAuthors :many +SELECT * FROM authors; + +-- name: CountAuthors :one +SELECT count(*) FROM authors; diff --git a/internal/endtoend/testdata/empty_params_class/schema.sql b/internal/endtoend/testdata/empty_params_class/schema.sql new file mode 100644 index 0000000..0b27d3b --- /dev/null +++ b/internal/endtoend/testdata/empty_params_class/schema.sql @@ -0,0 +1,4 @@ +CREATE TABLE authors ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL +); diff --git a/internal/endtoend/testdata/empty_params_class/sqlc.yaml b/internal/endtoend/testdata/empty_params_class/sqlc.yaml new file mode 100644 index 0000000..046960d --- /dev/null +++ b/internal/endtoend/testdata/empty_params_class/sqlc.yaml @@ -0,0 +1,17 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + query_parameter_limit: 0 diff --git a/internal/endtoend/testdata/enum_member_names/db/models.py b/internal/endtoend/testdata/enum_member_names/db/models.py new file mode 100644 index 0000000..e6c7eca --- /dev/null +++ b/internal/endtoend/testdata/enum_member_names/db/models.py @@ -0,0 +1,22 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses +import enum + + +class Op(str, enum.Enum): + """Operators with "quoted" names""" + VALUE_1 = "=" + VALUE_2 = "<>" + IN_PROGRESS = "in-progress" + IN_PROGRESS_2 = "in_progress" + VALUE_1ST = "1st" + SAYHI = "say \"hi\"" + BACKSLASH = "back\\slash" + + +@dataclasses.dataclass() +class Rule: + id: int + op: Op diff --git a/internal/endtoend/testdata/enum_member_names/db/query.py b/internal/endtoend/testdata/enum_member_names/db/query.py new file mode 100644 index 0000000..b600c78 --- /dev/null +++ b/internal/endtoend/testdata/enum_member_names/db/query.py @@ -0,0 +1,31 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +from typing import Iterator, Union, cast + +import sqlalchemy +import sqlalchemy.orm + +from db import models + + +LIST_RULES_BY_OP = """-- name: list_rules_by_op \\:many +SELECT id, op FROM rules +WHERE op = :p1 +""" + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def list_rules_by_op(self, *, op: models.Op) -> Iterator[models.Rule]: + result = self._conn.execute(sqlalchemy.text(LIST_RULES_BY_OP), {"p1": op}) + for row in result: + yield models.Rule( + id=cast(int, row[0]), + op=cast(models.Op, row[1]), + ) diff --git a/internal/endtoend/testdata/enum_member_names/query.sql b/internal/endtoend/testdata/enum_member_names/query.sql new file mode 100644 index 0000000..00811cb --- /dev/null +++ b/internal/endtoend/testdata/enum_member_names/query.sql @@ -0,0 +1,3 @@ +-- name: ListRulesByOp :many +SELECT * FROM rules +WHERE op = $1; diff --git a/internal/endtoend/testdata/enum_member_names/schema.sql b/internal/endtoend/testdata/enum_member_names/schema.sql new file mode 100644 index 0000000..2b15b0f --- /dev/null +++ b/internal/endtoend/testdata/enum_member_names/schema.sql @@ -0,0 +1,8 @@ +CREATE TYPE op AS ENUM ('=', '<>', 'in-progress', 'in_progress', '1st', 'say "hi"', 'back\slash'); + +COMMENT ON TYPE op IS 'Operators with "quoted" names'; + +CREATE TABLE rules ( + id BIGSERIAL PRIMARY KEY, + op op NOT NULL +); diff --git a/internal/endtoend/testdata/enum_member_names/sqlc.yaml b/internal/endtoend/testdata/enum_member_names/sqlc.yaml new file mode 100644 index 0000000..2848a1d --- /dev/null +++ b/internal/endtoend/testdata/enum_member_names/sqlc.yaml @@ -0,0 +1,17 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + emit_async_querier: false diff --git a/internal/endtoend/testdata/exec_result/python/models.py b/internal/endtoend/testdata/exec_result/python/models.py index 034fb2d..f4a55a5 100644 --- a/internal/endtoend/testdata/exec_result/python/models.py +++ b/internal/endtoend/testdata/exec_result/python/models.py @@ -1,6 +1,6 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 import dataclasses diff --git a/internal/endtoend/testdata/exec_result/python/query.py b/internal/endtoend/testdata/exec_result/python/query.py index b68ce39..757a572 100644 --- a/internal/endtoend/testdata/exec_result/python/query.py +++ b/internal/endtoend/testdata/exec_result/python/query.py @@ -1,9 +1,12 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 # source: query.sql +from typing import Any, Union + import sqlalchemy import sqlalchemy.ext.asyncio +import sqlalchemy.orm from querytest import models @@ -14,16 +17,20 @@ class Querier: - def __init__(self, conn: sqlalchemy.engine.Connection): + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): self._conn = conn - def delete_bar_by_id(self, *, id: int) -> sqlalchemy.engine.Result: + def delete_bar_by_id(self, *, id: int) -> "sqlalchemy.engine.Result[Any]": return self._conn.execute(sqlalchemy.text(DELETE_BAR_BY_ID), {"p1": id}) class AsyncQuerier: - def __init__(self, conn: sqlalchemy.ext.asyncio.AsyncConnection): + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): self._conn = conn - async def delete_bar_by_id(self, *, id: int) -> sqlalchemy.engine.Result: + async def delete_bar_by_id(self, *, id: int) -> "sqlalchemy.engine.Result[Any]": return await self._conn.execute(sqlalchemy.text(DELETE_BAR_BY_ID), {"p1": id}) diff --git a/internal/endtoend/testdata/exec_result/schema.sql b/internal/endtoend/testdata/exec_result/schema.sql index 638370a..682614c 100644 --- a/internal/endtoend/testdata/exec_result/schema.sql +++ b/internal/endtoend/testdata/exec_result/schema.sql @@ -1,2 +1 @@ CREATE TABLE bar (id serial not null); - diff --git a/internal/endtoend/testdata/exec_result/sqlc.yaml b/internal/endtoend/testdata/exec_result/sqlc.yaml index ddffc83..2678643 100644 --- a/internal/endtoend/testdata/exec_result/sqlc.yaml +++ b/internal/endtoend/testdata/exec_result/sqlc.yaml @@ -3,7 +3,7 @@ plugins: - name: py wasm: url: file://../../../../bin/sqlc-gen-python.wasm - sha256: "d6846ffad948181e611e883cedd2d2be66e091edc1273a0abc6c9da18399e0ca" + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" sql: - schema: schema.sql queries: query.sql diff --git a/internal/endtoend/testdata/exec_rows/python/models.py b/internal/endtoend/testdata/exec_rows/python/models.py index 034fb2d..f4a55a5 100644 --- a/internal/endtoend/testdata/exec_rows/python/models.py +++ b/internal/endtoend/testdata/exec_rows/python/models.py @@ -1,6 +1,6 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 import dataclasses diff --git a/internal/endtoend/testdata/exec_rows/python/query.py b/internal/endtoend/testdata/exec_rows/python/query.py index 7a9b2a6..686469f 100644 --- a/internal/endtoend/testdata/exec_rows/python/query.py +++ b/internal/endtoend/testdata/exec_rows/python/query.py @@ -1,9 +1,12 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 # source: query.sql +from typing import Any, Union, cast + import sqlalchemy import sqlalchemy.ext.asyncio +import sqlalchemy.orm from querytest import models @@ -14,18 +17,22 @@ class Querier: - def __init__(self, conn: sqlalchemy.engine.Connection): + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): self._conn = conn def delete_bar_by_id(self, *, id: int) -> int: result = self._conn.execute(sqlalchemy.text(DELETE_BAR_BY_ID), {"p1": id}) - return result.rowcount + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount class AsyncQuerier: - def __init__(self, conn: sqlalchemy.ext.asyncio.AsyncConnection): + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): self._conn = conn async def delete_bar_by_id(self, *, id: int) -> int: result = await self._conn.execute(sqlalchemy.text(DELETE_BAR_BY_ID), {"p1": id}) - return result.rowcount + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount diff --git a/internal/endtoend/testdata/exec_rows/schema.sql b/internal/endtoend/testdata/exec_rows/schema.sql index 638370a..682614c 100644 --- a/internal/endtoend/testdata/exec_rows/schema.sql +++ b/internal/endtoend/testdata/exec_rows/schema.sql @@ -1,2 +1 @@ CREATE TABLE bar (id serial not null); - diff --git a/internal/endtoend/testdata/exec_rows/sqlc.yaml b/internal/endtoend/testdata/exec_rows/sqlc.yaml index ddffc83..2678643 100644 --- a/internal/endtoend/testdata/exec_rows/sqlc.yaml +++ b/internal/endtoend/testdata/exec_rows/sqlc.yaml @@ -3,7 +3,7 @@ plugins: - name: py wasm: url: file://../../../../bin/sqlc-gen-python.wasm - sha256: "d6846ffad948181e611e883cedd2d2be66e091edc1273a0abc6c9da18399e0ca" + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" sql: - schema: schema.sql queries: query.sql diff --git a/internal/endtoend/testdata/feature_matrix_classic/_check.py b/internal/endtoend/testdata/feature_matrix_classic/_check.py new file mode 100644 index 0000000..c5a8f6f --- /dev/null +++ b/internal/endtoend/testdata/feature_matrix_classic/_check.py @@ -0,0 +1,37 @@ +from typing import Iterator, List, Optional + +import sqlalchemy.engine +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from db import errors, models, query + + +def sync(conn: sqlalchemy.engine.Connection, session: sqlalchemy.orm.Session) -> None: + q: query.QuerierProtocol = query.Querier(conn) + s: query.QuerierProtocol = query.Querier(session) + try: + created = q.create_author(name="a", bio=None) + except errors.UniqueViolationError as e: + name: Optional[str] = e.constraint_name + return + except errors.QueryError as e: + query_name: str = e.query_name + return + if created is not None: + row = q.get_book_with_author(id=created.id) + if row is not None: + author: models.Author = row.authors + book: models.Book = row.books + found: Iterator[Optional[models.Author]] = q.get_authors_by_id([query.GetAuthorsByIDParams(id=1)]) + books: Iterator[List[models.Book]] = q.list_books_by_author([query.ListBooksByAuthorParams(author_id=1)]) + count: int = q.create_authors([query.CreateAuthorsParams(name="b", bio=None)]) + status: models.BookStatus = models.BookStatus.IN_PRINT + + +async def asynchronous(conn: sqlalchemy.ext.asyncio.AsyncConnection) -> None: + q: query.AsyncQuerierProtocol = query.AsyncQuerier(conn) + async for author in q.get_authors_by_id([query.GetAuthorsByIDParams(id=1)]): + if author is not None: + print(author.name) + await q.create_books([query.CreateBooksParams(author_id=1, title="t", status=models.BookStatus.DRAFT, from_=None)]) diff --git a/internal/endtoend/testdata/feature_matrix_classic/db/errors.py b/internal/endtoend/testdata/feature_matrix_classic/db/errors.py new file mode 100644 index 0000000..b5998e9 --- /dev/null +++ b/internal/endtoend/testdata/feature_matrix_classic/db/errors.py @@ -0,0 +1,108 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import contextlib +from typing import Iterator, Optional + +import sqlalchemy.exc + + +class QueryError(Exception): + """Base class for all query errors.""" + + def __init__(self, message: str, query_name: str, cause: Optional[Exception] = None): + super().__init__(message) + self.query_name = query_name + self.cause = cause + self.constraint_name: Optional[str] = _constraint_name(cause) + + +class UniqueViolationError(QueryError): + """Raised on unique constraint violation (PG 23505).""" + + +class ForeignKeyViolationError(QueryError): + """Raised on foreign key constraint violation (PG 23503).""" + + +class CheckViolationError(QueryError): + """Raised on check constraint violation (PG 23514).""" + + +class NotNullViolationError(QueryError): + """Raised on not-null constraint violation (PG 23502).""" + + +class ExclusionViolationError(QueryError): + """Raised on exclusion constraint violation (PG 23P01).""" + + +class StatementTimeoutError(QueryError): + """Raised on statement timeout (PG 57014).""" + + +class DeadlockError(QueryError): + """Raised on deadlock detected (PG 40P01).""" + + +class SerializationError(QueryError): + """Raised on serialization failure (PG 40001).""" + + +def _sqlstate(e: sqlalchemy.exc.DBAPIError) -> Optional[str]: + # psycopg2 exposes pgcode, psycopg 3 sqlstate; SQLAlchemy's asyncpg + # adapter keeps the asyncpg error, which has sqlstate, as __cause__. + for source in (e.orig, getattr(e.orig, "__cause__", None)): + code = getattr(source, "pgcode", None) or getattr(source, "sqlstate", None) + if isinstance(code, str): + return code + return None + + +def _constraint_name(cause: Optional[Exception]) -> Optional[str]: + orig = getattr(cause, "orig", None) + name = getattr(getattr(orig, "diag", None), "constraint_name", None) + if name is None: + name = getattr(getattr(orig, "__cause__", None), "constraint_name", None) + return name if isinstance(name, str) else None + + +def _wrap_integrity_error(e: sqlalchemy.exc.IntegrityError, query_name: str) -> QueryError: + pgcode = _sqlstate(e) + if pgcode == "23505": + return UniqueViolationError(str(e), query_name, cause=e) + if pgcode == "23503": + return ForeignKeyViolationError(str(e), query_name, cause=e) + if pgcode == "23514": + return CheckViolationError(str(e), query_name, cause=e) + if pgcode == "23502": + return NotNullViolationError(str(e), query_name, cause=e) + if pgcode == "23P01": + return ExclusionViolationError(str(e), query_name, cause=e) + return QueryError(str(e), query_name, cause=e) + + +def _wrap_operational_error(e: sqlalchemy.exc.OperationalError, query_name: str) -> QueryError: + pgcode = _sqlstate(e) + if pgcode == "57014": + return StatementTimeoutError(str(e), query_name, cause=e) + if pgcode == "40P01": + return DeadlockError(str(e), query_name, cause=e) + if pgcode == "40001": + return SerializationError(str(e), query_name, cause=e) + return QueryError(str(e), query_name, cause=e) + + +@contextlib.contextmanager +def _wrap_errors(query_name: str) -> Iterator[None]: + """Re-raise integrity and operational errors as typed QueryErrors. + + Only those two SQLAlchemy exceptions are caught, so GeneratorExit from a + consumer that stops iterating early passes through untouched. + """ + try: + yield + except sqlalchemy.exc.IntegrityError as e: + raise _wrap_integrity_error(e, query_name) from e + except sqlalchemy.exc.OperationalError as e: + raise _wrap_operational_error(e, query_name) from e diff --git a/internal/endtoend/testdata/feature_matrix_classic/db/models.py b/internal/endtoend/testdata/feature_matrix_classic/db/models.py new file mode 100644 index 0000000..c3f68e9 --- /dev/null +++ b/internal/endtoend/testdata/feature_matrix_classic/db/models.py @@ -0,0 +1,37 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses +import datetime +import enum +from typing import List, Optional + + +class BookStatus(str, enum.Enum): + """Lifecycle of a book. +"Out of print" books stay listed.""" + DRAFT = "draft" + IN_PRINT = "in-print" + OUT_OF_PRINT = "out_of_print" + VALUE_4 = "=" + + +@dataclasses.dataclass() +class Author: + """People who write books.""" + id: int + name: str + bio: Optional[str] + created_at: datetime.datetime + + +@dataclasses.dataclass() +class Book: + id: int + author_id: int + title: str + status: BookStatus + # First published. + # NULL until the book is released. + from_: Optional[datetime.date] + tags: Optional[List[str]] diff --git a/internal/endtoend/testdata/feature_matrix_classic/db/query.py b/internal/endtoend/testdata/feature_matrix_classic/db/query.py new file mode 100644 index 0000000..927664e --- /dev/null +++ b/internal/endtoend/testdata/feature_matrix_classic/db/query.py @@ -0,0 +1,424 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import dataclasses +import datetime +from typing import Any, AsyncIterator, Iterator, List, Optional, Protocol, Sequence, Union, cast + +import sqlalchemy +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from db import errors, models + + +COUNT_AUTHORS = """-- name: count_authors \\:one +SELECT count(*) FROM authors +""" + + +CREATE_AUTHOR = """-- name: create_author \\:one +INSERT INTO authors (name, bio) VALUES (:p1, :p2) RETURNING id, name, bio, created_at +""" + + +CREATE_AUTHORS = """-- name: create_authors \\:copyfrom +INSERT INTO authors (name, bio) VALUES (:p1, :p2) +""" + + +@dataclasses.dataclass() +class CreateAuthorsParams: + name: str + bio: Optional[str] + + +CREATE_BOOKS = """-- name: create_books \\:batchexec +INSERT INTO books (author_id, title, status, "from") VALUES (:p1, :p2, :p3, :p4) +""" + + +@dataclasses.dataclass() +class CreateBooksParams: + author_id: int + title: str + status: models.BookStatus + from_: Optional[datetime.date] + + +DELETE_AUTHOR = """-- name: delete_author \\:execrows +DELETE FROM authors WHERE id = :p1 +""" + + +GET_AUTHOR = """-- name: get_author \\:one +SELECT id, name, bio, created_at FROM authors WHERE id = :p1 +""" + + +GET_AUTHORS_BY_ID = """-- name: get_authors_by_id \\:batchone +SELECT id, name, bio, created_at FROM authors WHERE id = :p1 +""" + + +@dataclasses.dataclass() +class GetAuthorsByIDParams: + id: int + + +GET_BOOK_WITH_AUTHOR = """-- name: get_book_with_author \\:one +SELECT books.id, books.author_id, books.title, books.status, books."from", books.tags, authors.id, authors.name, authors.bio, authors.created_at +FROM books JOIN authors ON authors.id = books.author_id +WHERE books.id = :p1 +""" + + +@dataclasses.dataclass() +class GetBookWithAuthorRow: + books: models.Book + authors: models.Author + + +LIST_AUTHORS = """-- name: list_authors \\:many +SELECT id, name, bio, created_at FROM authors ORDER BY name +""" + + +LIST_BOOKS_BY_AUTHOR = """-- name: list_books_by_author \\:batchmany +SELECT id, author_id, title, status, "from", tags FROM books WHERE author_id = :p1 ORDER BY id +""" + + +@dataclasses.dataclass() +class ListBooksByAuthorParams: + author_id: int + + +LIST_BOOKS_BY_STATUS = """-- name: list_books_by_status \\:many +SELECT books.id, books.title, books."from", authors.name AS author_name +FROM books JOIN authors ON authors.id = books.author_id +WHERE books.status = :p1 AND books.author_id = :p2\\:\\:bigint +""" + + +@dataclasses.dataclass() +class ListBooksByStatusRow: + id: int + title: str + from_: Optional[datetime.date] + author_name: str + + +class QuerierProtocol(Protocol): + def count_authors(self) -> Optional[int]: ... + + def create_author(self, *, name: str, bio: Optional[str]) -> Optional[models.Author]: ... + + def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: ... + + def create_books(self, arg: Sequence[CreateBooksParams]) -> None: ... + + def delete_author(self, *, id: int) -> int: ... + + def get_author(self, *, id: int) -> Optional[models.Author]: ... + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> Iterator[Optional[models.Author]]: ... + + def get_book_with_author(self, *, id: int) -> Optional[GetBookWithAuthorRow]: ... + + def list_authors(self) -> Iterator[models.Author]: ... + + def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> Iterator[List[models.Book]]: ... + + def list_books_by_status(self, *, status: models.BookStatus, class_: int) -> Iterator[ListBooksByStatusRow]: ... + + +class AsyncQuerierProtocol(Protocol): + async def count_authors(self) -> Optional[int]: ... + + async def create_author(self, *, name: str, bio: Optional[str]) -> Optional[models.Author]: ... + + async def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: ... + + async def create_books(self, arg: Sequence[CreateBooksParams]) -> None: ... + + async def delete_author(self, *, id: int) -> int: ... + + async def get_author(self, *, id: int) -> Optional[models.Author]: ... + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> AsyncIterator[Optional[models.Author]]: ... + + async def get_book_with_author(self, *, id: int) -> Optional[GetBookWithAuthorRow]: ... + + def list_authors(self) -> AsyncIterator[models.Author]: ... + + def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> AsyncIterator[List[models.Book]]: ... + + def list_books_by_status(self, *, status: models.BookStatus, class_: int) -> AsyncIterator[ListBooksByStatusRow]: ... + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def count_authors(self) -> Optional[int]: + with errors._wrap_errors("count_authors"): + row = self._conn.execute(sqlalchemy.text(COUNT_AUTHORS)).first() + if row is None: + return None + return cast(int, row[0]) + + def create_author(self, *, name: str, bio: Optional[str]) -> Optional[models.Author]: + """Create one author.""" + with errors._wrap_errors("create_author"): + row = self._conn.execute(sqlalchemy.text(CREATE_AUTHOR), {"p1": name, "p2": bio}).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + created_at=cast(datetime.datetime, row[3]), + ) + + def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: + with errors._wrap_errors("create_authors"): + if not arg: + return 0 + result = self._conn.execute(sqlalchemy.text(CREATE_AUTHORS), [{"p1": a.name, "p2": a.bio} for a in arg]) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + def create_books(self, arg: Sequence[CreateBooksParams]) -> None: + with errors._wrap_errors("create_books"): + if not arg: + return None + self._conn.execute(sqlalchemy.text(CREATE_BOOKS), [{ + "p1": a.author_id, + "p2": a.title, + "p3": a.status, + "p4": a.from_, + } for a in arg]) + + def delete_author(self, *, id: int) -> int: + with errors._wrap_errors("delete_author"): + result = self._conn.execute(sqlalchemy.text(DELETE_AUTHOR), {"p1": id}) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + def get_author(self, *, id: int) -> Optional[models.Author]: + with errors._wrap_errors("get_author"): + row = self._conn.execute(sqlalchemy.text(GET_AUTHOR), {"p1": id}).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + created_at=cast(datetime.datetime, row[3]), + ) + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> Iterator[Optional[models.Author]]: + """Look up several authors at once.""" + with errors._wrap_errors("get_authors_by_id"): + for a in arg: + row = self._conn.execute(sqlalchemy.text(GET_AUTHORS_BY_ID), {"p1": a.id}).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + created_at=cast(datetime.datetime, row[3]), + ) + + def get_book_with_author(self, *, id: int) -> Optional[GetBookWithAuthorRow]: + with errors._wrap_errors("get_book_with_author"): + row = self._conn.execute(sqlalchemy.text(GET_BOOK_WITH_AUTHOR), {"p1": id}).first() + if row is None: + return None + return GetBookWithAuthorRow( + books=models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + from_=cast(Optional[datetime.date], row[4]), + tags=cast(Optional[List[str]], row[5]), + ), + authors=models.Author( + id=cast(int, row[6]), + name=cast(str, row[7]), + bio=cast(Optional[str], row[8]), + created_at=cast(datetime.datetime, row[9]), + ), + ) + + def list_authors(self) -> Iterator[models.Author]: + with errors._wrap_errors("list_authors"): + result = self._conn.execute(sqlalchemy.text(LIST_AUTHORS)) + for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + created_at=cast(datetime.datetime, row[3]), + ) + + def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> Iterator[List[models.Book]]: + with errors._wrap_errors("list_books_by_author"): + for a in arg: + yield [models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + from_=cast(Optional[datetime.date], row[4]), + tags=cast(Optional[List[str]], row[5]), + ) for row in self._conn.execute(sqlalchemy.text(LIST_BOOKS_BY_AUTHOR), {"p1": a.author_id})] + + def list_books_by_status(self, *, status: models.BookStatus, class_: int) -> Iterator[ListBooksByStatusRow]: + with errors._wrap_errors("list_books_by_status"): + result = self._conn.execute(sqlalchemy.text(LIST_BOOKS_BY_STATUS), {"p1": status, "p2": class_}) + for row in result: + yield ListBooksByStatusRow( + id=cast(int, row[0]), + title=cast(str, row[1]), + from_=cast(Optional[datetime.date], row[2]), + author_name=cast(str, row[3]), + ) + + +class AsyncQuerier: + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): + self._conn = conn + + async def count_authors(self) -> Optional[int]: + with errors._wrap_errors("count_authors"): + row = (await self._conn.execute(sqlalchemy.text(COUNT_AUTHORS))).first() + if row is None: + return None + return cast(int, row[0]) + + async def create_author(self, *, name: str, bio: Optional[str]) -> Optional[models.Author]: + """Create one author.""" + with errors._wrap_errors("create_author"): + row = (await self._conn.execute(sqlalchemy.text(CREATE_AUTHOR), {"p1": name, "p2": bio})).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + created_at=cast(datetime.datetime, row[3]), + ) + + async def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: + with errors._wrap_errors("create_authors"): + if not arg: + return 0 + result = await self._conn.execute(sqlalchemy.text(CREATE_AUTHORS), [{"p1": a.name, "p2": a.bio} for a in arg]) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + async def create_books(self, arg: Sequence[CreateBooksParams]) -> None: + with errors._wrap_errors("create_books"): + if not arg: + return None + await self._conn.execute(sqlalchemy.text(CREATE_BOOKS), [{ + "p1": a.author_id, + "p2": a.title, + "p3": a.status, + "p4": a.from_, + } for a in arg]) + + async def delete_author(self, *, id: int) -> int: + with errors._wrap_errors("delete_author"): + result = await self._conn.execute(sqlalchemy.text(DELETE_AUTHOR), {"p1": id}) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + async def get_author(self, *, id: int) -> Optional[models.Author]: + with errors._wrap_errors("get_author"): + row = (await self._conn.execute(sqlalchemy.text(GET_AUTHOR), {"p1": id})).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + created_at=cast(datetime.datetime, row[3]), + ) + + async def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> AsyncIterator[Optional[models.Author]]: + """Look up several authors at once.""" + with errors._wrap_errors("get_authors_by_id"): + for a in arg: + row = (await self._conn.execute(sqlalchemy.text(GET_AUTHORS_BY_ID), {"p1": a.id})).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + created_at=cast(datetime.datetime, row[3]), + ) + + async def get_book_with_author(self, *, id: int) -> Optional[GetBookWithAuthorRow]: + with errors._wrap_errors("get_book_with_author"): + row = (await self._conn.execute(sqlalchemy.text(GET_BOOK_WITH_AUTHOR), {"p1": id})).first() + if row is None: + return None + return GetBookWithAuthorRow( + books=models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + from_=cast(Optional[datetime.date], row[4]), + tags=cast(Optional[List[str]], row[5]), + ), + authors=models.Author( + id=cast(int, row[6]), + name=cast(str, row[7]), + bio=cast(Optional[str], row[8]), + created_at=cast(datetime.datetime, row[9]), + ), + ) + + async def list_authors(self) -> AsyncIterator[models.Author]: + with errors._wrap_errors("list_authors"): + result = await self._conn.stream(sqlalchemy.text(LIST_AUTHORS)) + async for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + created_at=cast(datetime.datetime, row[3]), + ) + + async def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> AsyncIterator[List[models.Book]]: + with errors._wrap_errors("list_books_by_author"): + for a in arg: + yield [models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + from_=cast(Optional[datetime.date], row[4]), + tags=cast(Optional[List[str]], row[5]), + ) for row in await self._conn.execute(sqlalchemy.text(LIST_BOOKS_BY_AUTHOR), {"p1": a.author_id})] + + async def list_books_by_status(self, *, status: models.BookStatus, class_: int) -> AsyncIterator[ListBooksByStatusRow]: + with errors._wrap_errors("list_books_by_status"): + result = await self._conn.stream(sqlalchemy.text(LIST_BOOKS_BY_STATUS), {"p1": status, "p2": class_}) + async for row in result: + yield ListBooksByStatusRow( + id=cast(int, row[0]), + title=cast(str, row[1]), + from_=cast(Optional[datetime.date], row[2]), + author_name=cast(str, row[3]), + ) diff --git a/internal/endtoend/testdata/feature_matrix_classic/query.sql b/internal/endtoend/testdata/feature_matrix_classic/query.sql new file mode 100644 index 0000000..4df3a56 --- /dev/null +++ b/internal/endtoend/testdata/feature_matrix_classic/query.sql @@ -0,0 +1,38 @@ +-- Create one author. +-- name: CreateAuthor :one +INSERT INTO authors (name, bio) VALUES ($1, $2) RETURNING *; + +-- name: GetAuthor :one +SELECT * FROM authors WHERE id = $1; + +-- name: ListAuthors :many +SELECT * FROM authors ORDER BY name; + +-- name: DeleteAuthor :execrows +DELETE FROM authors WHERE id = $1; + +-- name: CreateAuthors :copyfrom +INSERT INTO authors (name, bio) VALUES ($1, $2); + +-- name: CreateBooks :batchexec +INSERT INTO books (author_id, title, status, "from") VALUES ($1, $2, $3, $4); + +-- Look up several authors at once. +-- name: GetAuthorsByID :batchone +SELECT * FROM authors WHERE id = $1; + +-- name: ListBooksByAuthor :batchmany +SELECT * FROM books WHERE author_id = $1 ORDER BY id; + +-- name: GetBookWithAuthor :one +SELECT sqlc.embed(books), sqlc.embed(authors) +FROM books JOIN authors ON authors.id = books.author_id +WHERE books.id = $1; + +-- name: ListBooksByStatus :many +SELECT books.id, books.title, books."from", authors.name AS author_name +FROM books JOIN authors ON authors.id = books.author_id +WHERE books.status = $1 AND books.author_id = sqlc.arg(class)::bigint; + +-- name: CountAuthors :one +SELECT count(*) FROM authors; diff --git a/internal/endtoend/testdata/feature_matrix_classic/schema.sql b/internal/endtoend/testdata/feature_matrix_classic/schema.sql new file mode 100644 index 0000000..e08b5fe --- /dev/null +++ b/internal/endtoend/testdata/feature_matrix_classic/schema.sql @@ -0,0 +1,26 @@ +CREATE TYPE book_status AS ENUM ('draft', 'in-print', 'out_of_print', '='); + +COMMENT ON TYPE book_status IS 'Lifecycle of a book. +"Out of print" books stay listed.'; + +CREATE TABLE authors ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL UNIQUE, + bio text, + created_at timestamptz NOT NULL DEFAULT now() +); + +COMMENT ON TABLE authors IS 'People who write books.'; + +CREATE TABLE books ( + id BIGSERIAL PRIMARY KEY, + author_id bigint NOT NULL REFERENCES authors (id), + title text NOT NULL, + status book_status NOT NULL DEFAULT 'draft', + "from" date, + tags text[], + CHECK (title <> '') +); + +COMMENT ON COLUMN books."from" IS 'First published. +NULL until the book is released.'; diff --git a/internal/endtoend/testdata/feature_matrix_classic/sqlc.yaml b/internal/endtoend/testdata/feature_matrix_classic/sqlc.yaml new file mode 100644 index 0000000..246ff1a --- /dev/null +++ b/internal/endtoend/testdata/feature_matrix_classic/sqlc.yaml @@ -0,0 +1,19 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + emit_async_querier: true + emit_querier_protocol: true + emit_query_errors: true diff --git a/internal/endtoend/testdata/feature_matrix_modern/_check.py b/internal/endtoend/testdata/feature_matrix_modern/_check.py new file mode 100644 index 0000000..05d34d0 --- /dev/null +++ b/internal/endtoend/testdata/feature_matrix_modern/_check.py @@ -0,0 +1,35 @@ +from collections.abc import Iterator +from typing import assert_type + +import pydantic +import sqlalchemy.engine +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from db import errors, models, query + + +def sync(conn: sqlalchemy.engine.Connection, session: sqlalchemy.orm.Session) -> None: + q = query.Querier(session) + assert_type(q._conn, sqlalchemy.orm.Session) + p: query.QuerierProtocol = q + c: query.QuerierProtocol = query.Querier(conn) + try: + created = q.create_author(name="a", bio=None) + except errors.UniqueViolationError as e: + name: str | None = e.constraint_name + return + if created is not None: + assert_type(created.created_at, pydantic.AwareDatetime) + found: Iterator[models.Author | None] = q.get_authors_by_id([query.GetAuthorsByIDParams(id=1)]) + books: Iterator[list[models.Book]] = q.list_books_by_author([query.ListBooksByAuthorParams(author_id=1)]) + status: models.BookStatus = models.BookStatus.OUT_OF_PRINT + + +async def asynchronous(session: sqlalchemy.ext.asyncio.AsyncSession) -> None: + q = query.AsyncQuerier(session) + assert_type(q._conn, sqlalchemy.ext.asyncio.AsyncSession) + p: query.AsyncQuerierProtocol = q + async for books in q.list_books_by_author([query.ListBooksByAuthorParams(author_id=1)]): + for b in books: + print(b.title, b.from_) diff --git a/internal/endtoend/testdata/feature_matrix_modern/db/errors.py b/internal/endtoend/testdata/feature_matrix_modern/db/errors.py new file mode 100644 index 0000000..bc58233 --- /dev/null +++ b/internal/endtoend/testdata/feature_matrix_modern/db/errors.py @@ -0,0 +1,108 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import contextlib +from collections.abc import Iterator + +import sqlalchemy.exc + + +class QueryError(Exception): + """Base class for all query errors.""" + + def __init__(self, message: str, query_name: str, cause: Exception | None = None): + super().__init__(message) + self.query_name = query_name + self.cause = cause + self.constraint_name: str | None = _constraint_name(cause) + + +class UniqueViolationError(QueryError): + """Raised on unique constraint violation (PG 23505).""" + + +class ForeignKeyViolationError(QueryError): + """Raised on foreign key constraint violation (PG 23503).""" + + +class CheckViolationError(QueryError): + """Raised on check constraint violation (PG 23514).""" + + +class NotNullViolationError(QueryError): + """Raised on not-null constraint violation (PG 23502).""" + + +class ExclusionViolationError(QueryError): + """Raised on exclusion constraint violation (PG 23P01).""" + + +class StatementTimeoutError(QueryError): + """Raised on statement timeout (PG 57014).""" + + +class DeadlockError(QueryError): + """Raised on deadlock detected (PG 40P01).""" + + +class SerializationError(QueryError): + """Raised on serialization failure (PG 40001).""" + + +def _sqlstate(e: sqlalchemy.exc.DBAPIError) -> str | None: + # psycopg2 exposes pgcode, psycopg 3 sqlstate; SQLAlchemy's asyncpg + # adapter keeps the asyncpg error, which has sqlstate, as __cause__. + for source in (e.orig, getattr(e.orig, "__cause__", None)): + code = getattr(source, "pgcode", None) or getattr(source, "sqlstate", None) + if isinstance(code, str): + return code + return None + + +def _constraint_name(cause: Exception | None) -> str | None: + orig = getattr(cause, "orig", None) + name = getattr(getattr(orig, "diag", None), "constraint_name", None) + if name is None: + name = getattr(getattr(orig, "__cause__", None), "constraint_name", None) + return name if isinstance(name, str) else None + + +def _wrap_integrity_error(e: sqlalchemy.exc.IntegrityError, query_name: str) -> QueryError: + pgcode = _sqlstate(e) + if pgcode == "23505": + return UniqueViolationError(str(e), query_name, cause=e) + if pgcode == "23503": + return ForeignKeyViolationError(str(e), query_name, cause=e) + if pgcode == "23514": + return CheckViolationError(str(e), query_name, cause=e) + if pgcode == "23502": + return NotNullViolationError(str(e), query_name, cause=e) + if pgcode == "23P01": + return ExclusionViolationError(str(e), query_name, cause=e) + return QueryError(str(e), query_name, cause=e) + + +def _wrap_operational_error(e: sqlalchemy.exc.OperationalError, query_name: str) -> QueryError: + pgcode = _sqlstate(e) + if pgcode == "57014": + return StatementTimeoutError(str(e), query_name, cause=e) + if pgcode == "40P01": + return DeadlockError(str(e), query_name, cause=e) + if pgcode == "40001": + return SerializationError(str(e), query_name, cause=e) + return QueryError(str(e), query_name, cause=e) + + +@contextlib.contextmanager +def _wrap_errors(query_name: str) -> Iterator[None]: + """Re-raise integrity and operational errors as typed QueryErrors. + + Only those two SQLAlchemy exceptions are caught, so GeneratorExit from a + consumer that stops iterating early passes through untouched. + """ + try: + yield + except sqlalchemy.exc.IntegrityError as e: + raise _wrap_integrity_error(e, query_name) from e + except sqlalchemy.exc.OperationalError as e: + raise _wrap_operational_error(e, query_name) from e diff --git a/internal/endtoend/testdata/feature_matrix_modern/db/models.py b/internal/endtoend/testdata/feature_matrix_modern/db/models.py new file mode 100644 index 0000000..f8ff668 --- /dev/null +++ b/internal/endtoend/testdata/feature_matrix_modern/db/models.py @@ -0,0 +1,34 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import datetime +import enum +import pydantic + + +class BookStatus(enum.StrEnum): + """Lifecycle of a book. +"Out of print" books stay listed.""" + DRAFT = "draft" + IN_PRINT = "in-print" + OUT_OF_PRINT = "out_of_print" + VALUE_4 = "=" + + +class Author(pydantic.BaseModel): + """People who write books.""" + id: int + name: str + bio: str | None + created_at: pydantic.AwareDatetime + + +class Book(pydantic.BaseModel): + id: int + author_id: int + title: str + status: BookStatus + # First published. + # NULL until the book is released. + from_: datetime.date | None + tags: list[str] | None diff --git a/internal/endtoend/testdata/feature_matrix_modern/db/query.py b/internal/endtoend/testdata/feature_matrix_modern/db/query.py new file mode 100644 index 0000000..3cdeee8 --- /dev/null +++ b/internal/endtoend/testdata/feature_matrix_modern/db/query.py @@ -0,0 +1,546 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import datetime +import pydantic +from collections.abc import AsyncIterator, Iterator, Sequence +from typing import Any, Protocol, cast + +import sqlalchemy +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from db import errors, models + + +COUNT_AUTHORS = """-- name: count_authors \\:one +SELECT count(*) FROM authors +""" + + +CREATE_AUTHOR = """-- name: create_author \\:one +INSERT INTO authors (name, bio) VALUES (:p1, :p2) RETURNING id, name, bio, created_at +""" + + +CREATE_AUTHORS = """-- name: create_authors \\:copyfrom +INSERT INTO authors (name, bio) VALUES (:p1, :p2) +""" + + +class CreateAuthorsParams(pydantic.BaseModel): + name: str + bio: str | None + + +CREATE_BOOKS = """-- name: create_books \\:batchexec +INSERT INTO books (author_id, title, status, "from") VALUES (:p1, :p2, :p3, :p4) +""" + + +class CreateBooksParams(pydantic.BaseModel): + author_id: int + title: str + status: models.BookStatus + from_: datetime.date | None + + +DELETE_AUTHOR = """-- name: delete_author \\:execrows +DELETE FROM authors WHERE id = :p1 +""" + + +GET_AUTHOR = """-- name: get_author \\:one +SELECT id, name, bio, created_at FROM authors WHERE id = :p1 +""" + + +GET_AUTHORS_BY_ID = """-- name: get_authors_by_id \\:batchone +SELECT id, name, bio, created_at FROM authors WHERE id = :p1 +""" + + +class GetAuthorsByIDParams(pydantic.BaseModel): + id: int + + +GET_BOOK_WITH_AUTHOR = """-- name: get_book_with_author \\:one +SELECT books.id, books.author_id, books.title, books.status, books."from", books.tags, authors.id, authors.name, authors.bio, authors.created_at +FROM books JOIN authors ON authors.id = books.author_id +WHERE books.id = :p1 +""" + + +class GetBookWithAuthorRow(pydantic.BaseModel): + books: models.Book + authors: models.Author + + +INT = """-- name: int \\:execrows +DELETE FROM books WHERE id = :p1 +""" + + +LIST = """-- name: list \\:many +SELECT tags FROM books +""" + + +LIST_AUTHORS = """-- name: list_authors \\:many +SELECT id, name, bio, created_at FROM authors ORDER BY name +""" + + +LIST_BOOKS_BY_AUTHOR = """-- name: list_books_by_author \\:batchmany +SELECT id, author_id, title, status, "from", tags FROM books WHERE author_id = :p1 ORDER BY id +""" + + +class ListBooksByAuthorParams(pydantic.BaseModel): + author_id: int + + +LIST_BOOKS_BY_STATUS = """-- name: list_books_by_status \\:many +SELECT books.id, books.title, books."from", authors.name AS author_name +FROM books JOIN authors ON authors.id = books.author_id +WHERE books.status = :p1 AND books.author_id = :p2\\:\\:bigint +""" + + +class ListBooksByStatusRow(pydantic.BaseModel): + id: int + title: str + from_: datetime.date | None + author_name: str + + +LIST_TAGS = """-- name: list_tags \\:many +SELECT tags FROM books WHERE id = :p1 +""" + + +MODELS = """-- name: models \\:one +SELECT id, name, bio, created_at FROM authors WHERE id = :p1 +""" + + +PICK_AUTHOR = """-- name: pick_author \\:one +SELECT id, name, bio, created_at FROM authors WHERE name = :p1 +""" + + +class QuerierProtocol(Protocol): + def count_authors(self) -> int | None: ... + + def create_author(self, *, name: str, bio: str | None) -> models.Author | None: ... + + def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: ... + + def create_books(self, arg: Sequence[CreateBooksParams]) -> None: ... + + def delete_author(self, *, id: int) -> int: ... + + def get_author(self, *, id: int) -> models.Author | None: ... + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> Iterator[models.Author | None]: ... + + def get_book_with_author(self, *, id: int) -> GetBookWithAuthorRow | None: ... + + def int_(self, *, id: int) -> int: ... + + def list_(self) -> Iterator[list[str] | None]: ... + + def list_authors(self) -> Iterator[models.Author]: ... + + def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> Iterator[list[models.Book]]: ... + + def list_books_by_status(self, *, status: models.BookStatus, class_: int) -> Iterator[ListBooksByStatusRow]: ... + + def list_tags(self, *, id: int) -> Iterator[list[str] | None]: ... + + def models_(self, *, id: int) -> models.Author | None: ... + + def pick_author(self, *, name: str) -> models.Author | None: ... + + +class AsyncQuerierProtocol(Protocol): + async def count_authors(self) -> int | None: ... + + async def create_author(self, *, name: str, bio: str | None) -> models.Author | None: ... + + async def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: ... + + async def create_books(self, arg: Sequence[CreateBooksParams]) -> None: ... + + async def delete_author(self, *, id: int) -> int: ... + + async def get_author(self, *, id: int) -> models.Author | None: ... + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> AsyncIterator[models.Author | None]: ... + + async def get_book_with_author(self, *, id: int) -> GetBookWithAuthorRow | None: ... + + async def int_(self, *, id: int) -> int: ... + + def list_(self) -> AsyncIterator[list[str] | None]: ... + + def list_authors(self) -> AsyncIterator[models.Author]: ... + + def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> AsyncIterator[list[models.Book]]: ... + + def list_books_by_status(self, *, status: models.BookStatus, class_: int) -> AsyncIterator[ListBooksByStatusRow]: ... + + def list_tags(self, *, id: int) -> AsyncIterator[list[str] | None]: ... + + async def models_(self, *, id: int) -> models.Author | None: ... + + async def pick_author(self, *, name: str) -> models.Author | None: ... + + +class Querier[_ConnT: sqlalchemy.engine.Connection | sqlalchemy.orm.Session]: + _conn: _ConnT + + def __init__(self, conn: _ConnT): + self._conn = conn + + def count_authors(self) -> int | None: + with errors._wrap_errors("count_authors"): + row = self._conn.execute(sqlalchemy.text(COUNT_AUTHORS)).first() + if row is None: + return None + return cast(int, row[0]) + + def create_author(self, *, name: str, bio: str | None) -> models.Author | None: + """Create one author.""" + with errors._wrap_errors("create_author"): + row = self._conn.execute(sqlalchemy.text(CREATE_AUTHOR), {"p1": name, "p2": bio}).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: + with errors._wrap_errors("create_authors"): + if not arg: + return 0 + result = self._conn.execute(sqlalchemy.text(CREATE_AUTHORS), [{"p1": a.name, "p2": a.bio} for a in arg]) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + def create_books(self, arg: Sequence[CreateBooksParams]) -> None: + with errors._wrap_errors("create_books"): + if not arg: + return None + self._conn.execute(sqlalchemy.text(CREATE_BOOKS), [{ + "p1": a.author_id, + "p2": a.title, + "p3": a.status, + "p4": a.from_, + } for a in arg]) + + def delete_author(self, *, id: int) -> int: + with errors._wrap_errors("delete_author"): + result = self._conn.execute(sqlalchemy.text(DELETE_AUTHOR), {"p1": id}) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + def get_author(self, *, id: int) -> models.Author | None: + with errors._wrap_errors("get_author"): + row = self._conn.execute(sqlalchemy.text(GET_AUTHOR), {"p1": id}).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> Iterator[models.Author | None]: + """Look up several authors at once.""" + with errors._wrap_errors("get_authors_by_id"): + for a in arg: + row = self._conn.execute(sqlalchemy.text(GET_AUTHORS_BY_ID), {"p1": a.id}).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + def get_book_with_author(self, *, id: int) -> GetBookWithAuthorRow | None: + with errors._wrap_errors("get_book_with_author"): + row = self._conn.execute(sqlalchemy.text(GET_BOOK_WITH_AUTHOR), {"p1": id}).first() + if row is None: + return None + return GetBookWithAuthorRow( + books=models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + from_=cast(datetime.date | None, row[4]), + tags=cast(list[str] | None, row[5]), + ), + authors=models.Author( + id=cast(int, row[6]), + name=cast(str, row[7]), + bio=cast(str | None, row[8]), + created_at=cast(pydantic.AwareDatetime, row[9]), + ), + ) + + def int_(self, *, id: int) -> int: + with errors._wrap_errors("int_"): + result = self._conn.execute(sqlalchemy.text(INT), {"p1": id}) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + def list_(self) -> Iterator[list[str] | None]: + with errors._wrap_errors("list_"): + result = self._conn.execute(sqlalchemy.text(LIST)) + for row in result: + yield cast(list[str] | None, row[0]) + + def list_authors(self) -> Iterator[models.Author]: + with errors._wrap_errors("list_authors"): + result = self._conn.execute(sqlalchemy.text(LIST_AUTHORS)) + for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> Iterator[list[models.Book]]: + with errors._wrap_errors("list_books_by_author"): + for a in arg: + yield [models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + from_=cast(datetime.date | None, row[4]), + tags=cast(list[str] | None, row[5]), + ) for row in self._conn.execute(sqlalchemy.text(LIST_BOOKS_BY_AUTHOR), {"p1": a.author_id})] + + def list_books_by_status(self, *, status: models.BookStatus, class_: int) -> Iterator[ListBooksByStatusRow]: + with errors._wrap_errors("list_books_by_status"): + result = self._conn.execute(sqlalchemy.text(LIST_BOOKS_BY_STATUS), {"p1": status, "p2": class_}) + for row in result: + yield ListBooksByStatusRow( + id=cast(int, row[0]), + title=cast(str, row[1]), + from_=cast(datetime.date | None, row[2]), + author_name=cast(str, row[3]), + ) + + def list_tags(self, *, id: int) -> Iterator[list[str] | None]: + with errors._wrap_errors("list_tags"): + result = self._conn.execute(sqlalchemy.text(LIST_TAGS), {"p1": id}) + for row in result: + yield cast(list[str] | None, row[0]) + + def models_(self, *, id: int) -> models.Author | None: + with errors._wrap_errors("models_"): + row = self._conn.execute(sqlalchemy.text(MODELS), {"p1": id}).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + def pick_author(self, *, name: str) -> models.Author | None: + with errors._wrap_errors("pick_author"): + row = self._conn.execute(sqlalchemy.text(PICK_AUTHOR), {"p1": name}).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + +class AsyncQuerier[_ConnT: sqlalchemy.ext.asyncio.AsyncConnection | sqlalchemy.ext.asyncio.AsyncSession]: + _conn: _ConnT + + def __init__(self, conn: _ConnT): + self._conn = conn + + async def count_authors(self) -> int | None: + with errors._wrap_errors("count_authors"): + row = (await self._conn.execute(sqlalchemy.text(COUNT_AUTHORS))).first() + if row is None: + return None + return cast(int, row[0]) + + async def create_author(self, *, name: str, bio: str | None) -> models.Author | None: + """Create one author.""" + with errors._wrap_errors("create_author"): + row = (await self._conn.execute(sqlalchemy.text(CREATE_AUTHOR), {"p1": name, "p2": bio})).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + async def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: + with errors._wrap_errors("create_authors"): + if not arg: + return 0 + result = await self._conn.execute(sqlalchemy.text(CREATE_AUTHORS), [{"p1": a.name, "p2": a.bio} for a in arg]) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + async def create_books(self, arg: Sequence[CreateBooksParams]) -> None: + with errors._wrap_errors("create_books"): + if not arg: + return None + await self._conn.execute(sqlalchemy.text(CREATE_BOOKS), [{ + "p1": a.author_id, + "p2": a.title, + "p3": a.status, + "p4": a.from_, + } for a in arg]) + + async def delete_author(self, *, id: int) -> int: + with errors._wrap_errors("delete_author"): + result = await self._conn.execute(sqlalchemy.text(DELETE_AUTHOR), {"p1": id}) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + async def get_author(self, *, id: int) -> models.Author | None: + with errors._wrap_errors("get_author"): + row = (await self._conn.execute(sqlalchemy.text(GET_AUTHOR), {"p1": id})).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + async def get_authors_by_id(self, arg: Sequence[GetAuthorsByIDParams]) -> AsyncIterator[models.Author | None]: + """Look up several authors at once.""" + with errors._wrap_errors("get_authors_by_id"): + for a in arg: + row = (await self._conn.execute(sqlalchemy.text(GET_AUTHORS_BY_ID), {"p1": a.id})).first() + if row is None: + yield None + else: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + async def get_book_with_author(self, *, id: int) -> GetBookWithAuthorRow | None: + with errors._wrap_errors("get_book_with_author"): + row = (await self._conn.execute(sqlalchemy.text(GET_BOOK_WITH_AUTHOR), {"p1": id})).first() + if row is None: + return None + return GetBookWithAuthorRow( + books=models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + from_=cast(datetime.date | None, row[4]), + tags=cast(list[str] | None, row[5]), + ), + authors=models.Author( + id=cast(int, row[6]), + name=cast(str, row[7]), + bio=cast(str | None, row[8]), + created_at=cast(pydantic.AwareDatetime, row[9]), + ), + ) + + async def int_(self, *, id: int) -> int: + with errors._wrap_errors("int_"): + result = await self._conn.execute(sqlalchemy.text(INT), {"p1": id}) + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount + + async def list_(self) -> AsyncIterator[list[str] | None]: + with errors._wrap_errors("list_"): + result = await self._conn.stream(sqlalchemy.text(LIST)) + async for row in result: + yield cast(list[str] | None, row[0]) + + async def list_authors(self) -> AsyncIterator[models.Author]: + with errors._wrap_errors("list_authors"): + result = await self._conn.stream(sqlalchemy.text(LIST_AUTHORS)) + async for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + async def list_books_by_author(self, arg: Sequence[ListBooksByAuthorParams]) -> AsyncIterator[list[models.Book]]: + with errors._wrap_errors("list_books_by_author"): + for a in arg: + yield [models.Book( + id=cast(int, row[0]), + author_id=cast(int, row[1]), + title=cast(str, row[2]), + status=cast(models.BookStatus, row[3]), + from_=cast(datetime.date | None, row[4]), + tags=cast(list[str] | None, row[5]), + ) for row in await self._conn.execute(sqlalchemy.text(LIST_BOOKS_BY_AUTHOR), {"p1": a.author_id})] + + async def list_books_by_status(self, *, status: models.BookStatus, class_: int) -> AsyncIterator[ListBooksByStatusRow]: + with errors._wrap_errors("list_books_by_status"): + result = await self._conn.stream(sqlalchemy.text(LIST_BOOKS_BY_STATUS), {"p1": status, "p2": class_}) + async for row in result: + yield ListBooksByStatusRow( + id=cast(int, row[0]), + title=cast(str, row[1]), + from_=cast(datetime.date | None, row[2]), + author_name=cast(str, row[3]), + ) + + async def list_tags(self, *, id: int) -> AsyncIterator[list[str] | None]: + with errors._wrap_errors("list_tags"): + result = await self._conn.stream(sqlalchemy.text(LIST_TAGS), {"p1": id}) + async for row in result: + yield cast(list[str] | None, row[0]) + + async def models_(self, *, id: int) -> models.Author | None: + with errors._wrap_errors("models_"): + row = (await self._conn.execute(sqlalchemy.text(MODELS), {"p1": id})).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) + + async def pick_author(self, *, name: str) -> models.Author | None: + with errors._wrap_errors("pick_author"): + row = (await self._conn.execute(sqlalchemy.text(PICK_AUTHOR), {"p1": name})).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(str | None, row[2]), + created_at=cast(pydantic.AwareDatetime, row[3]), + ) diff --git a/internal/endtoend/testdata/feature_matrix_modern/query.sql b/internal/endtoend/testdata/feature_matrix_modern/query.sql new file mode 100644 index 0000000..bbdb704 --- /dev/null +++ b/internal/endtoend/testdata/feature_matrix_modern/query.sql @@ -0,0 +1,53 @@ +-- Create one author. +-- name: CreateAuthor :one +INSERT INTO authors (name, bio) VALUES ($1, $2) RETURNING *; + +-- name: GetAuthor :one +SELECT * FROM authors WHERE id = $1; + +-- name: ListAuthors :many +SELECT * FROM authors ORDER BY name; + +-- name: DeleteAuthor :execrows +DELETE FROM authors WHERE id = $1; + +-- name: CreateAuthors :copyfrom +INSERT INTO authors (name, bio) VALUES ($1, $2); + +-- name: CreateBooks :batchexec +INSERT INTO books (author_id, title, status, "from") VALUES ($1, $2, $3, $4); + +-- Look up several authors at once. +-- name: GetAuthorsByID :batchone +SELECT * FROM authors WHERE id = $1; + +-- name: ListBooksByAuthor :batchmany +SELECT * FROM books WHERE author_id = $1 ORDER BY id; + +-- name: GetBookWithAuthor :one +SELECT sqlc.embed(books), sqlc.embed(authors) +FROM books JOIN authors ON authors.id = books.author_id +WHERE books.id = $1; + +-- name: ListBooksByStatus :many +SELECT books.id, books.title, books."from", authors.name AS author_name +FROM books JOIN authors ON authors.id = books.author_id +WHERE books.status = $1 AND books.author_id = sqlc.arg(class)::bigint; + +-- name: CountAuthors :one +SELECT count(*) FROM authors; + +-- name: List :many +SELECT tags FROM books; + +-- name: Int :execrows +DELETE FROM books WHERE id = $1; + +-- name: Models :one +SELECT * FROM authors WHERE id = $1; + +-- name: ListTags :many +SELECT tags FROM books WHERE id = $1; + +-- name: PickAuthor :one +SELECT * FROM authors WHERE name = $1; diff --git a/internal/endtoend/testdata/feature_matrix_modern/schema.sql b/internal/endtoend/testdata/feature_matrix_modern/schema.sql new file mode 100644 index 0000000..e08b5fe --- /dev/null +++ b/internal/endtoend/testdata/feature_matrix_modern/schema.sql @@ -0,0 +1,26 @@ +CREATE TYPE book_status AS ENUM ('draft', 'in-print', 'out_of_print', '='); + +COMMENT ON TYPE book_status IS 'Lifecycle of a book. +"Out of print" books stay listed.'; + +CREATE TABLE authors ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL UNIQUE, + bio text, + created_at timestamptz NOT NULL DEFAULT now() +); + +COMMENT ON TABLE authors IS 'People who write books.'; + +CREATE TABLE books ( + id BIGSERIAL PRIMARY KEY, + author_id bigint NOT NULL REFERENCES authors (id), + title text NOT NULL, + status book_status NOT NULL DEFAULT 'draft', + "from" date, + tags text[], + CHECK (title <> '') +); + +COMMENT ON COLUMN books."from" IS 'First published. +NULL until the book is released.'; diff --git a/internal/endtoend/testdata/feature_matrix_modern/sqlc.yaml b/internal/endtoend/testdata/feature_matrix_modern/sqlc.yaml new file mode 100644 index 0000000..12a37cd --- /dev/null +++ b/internal/endtoend/testdata/feature_matrix_modern/sqlc.yaml @@ -0,0 +1,24 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + emit_async_querier: true + emit_querier_protocol: true + emit_query_errors: true + emit_pydantic_models: true + emit_str_enum: true + emit_modern_types: true + emit_aware_datetime: true + emit_generic_querier: true diff --git a/internal/endtoend/testdata/inflection_exclude_table_names/python/models.py b/internal/endtoend/testdata/inflection_exclude_table_names/python/models.py index 8ba8803..f238065 100644 --- a/internal/endtoend/testdata/inflection_exclude_table_names/python/models.py +++ b/internal/endtoend/testdata/inflection_exclude_table_names/python/models.py @@ -1,6 +1,6 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 import dataclasses diff --git a/internal/endtoend/testdata/inflection_exclude_table_names/python/query.py b/internal/endtoend/testdata/inflection_exclude_table_names/python/query.py index 1e1e161..188a133 100644 --- a/internal/endtoend/testdata/inflection_exclude_table_names/python/query.py +++ b/internal/endtoend/testdata/inflection_exclude_table_names/python/query.py @@ -1,11 +1,12 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 # source: query.sql -from typing import Optional +from typing import Optional, Union, cast import sqlalchemy import sqlalchemy.ext.asyncio +import sqlalchemy.orm from querytest import models @@ -26,7 +27,9 @@ class Querier: - def __init__(self, conn: sqlalchemy.engine.Connection): + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): self._conn = conn def delete_bar_by_id(self, *, id: int) -> Optional[models.Bar]: @@ -34,8 +37,8 @@ def delete_bar_by_id(self, *, id: int) -> Optional[models.Bar]: if row is None: return None return models.Bar( - id=row[0], - name=row[1], + id=cast(int, row[0]), + name=cast(str, row[1]), ) def delete_exclusion_by_id(self, *, id: int) -> Optional[models.Exclusions]: @@ -43,8 +46,8 @@ def delete_exclusion_by_id(self, *, id: int) -> Optional[models.Exclusions]: if row is None: return None return models.Exclusions( - id=row[0], - name=row[1], + id=cast(int, row[0]), + name=cast(str, row[1]), ) def delete_my_data_by_id(self, *, id: int) -> Optional[models.MyData]: @@ -52,13 +55,15 @@ def delete_my_data_by_id(self, *, id: int) -> Optional[models.MyData]: if row is None: return None return models.MyData( - id=row[0], - name=row[1], + id=cast(int, row[0]), + name=cast(str, row[1]), ) class AsyncQuerier: - def __init__(self, conn: sqlalchemy.ext.asyncio.AsyncConnection): + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): self._conn = conn async def delete_bar_by_id(self, *, id: int) -> Optional[models.Bar]: @@ -66,8 +71,8 @@ async def delete_bar_by_id(self, *, id: int) -> Optional[models.Bar]: if row is None: return None return models.Bar( - id=row[0], - name=row[1], + id=cast(int, row[0]), + name=cast(str, row[1]), ) async def delete_exclusion_by_id(self, *, id: int) -> Optional[models.Exclusions]: @@ -75,8 +80,8 @@ async def delete_exclusion_by_id(self, *, id: int) -> Optional[models.Exclusions if row is None: return None return models.Exclusions( - id=row[0], - name=row[1], + id=cast(int, row[0]), + name=cast(str, row[1]), ) async def delete_my_data_by_id(self, *, id: int) -> Optional[models.MyData]: @@ -84,6 +89,6 @@ async def delete_my_data_by_id(self, *, id: int) -> Optional[models.MyData]: if row is None: return None return models.MyData( - id=row[0], - name=row[1], + id=cast(int, row[0]), + name=cast(str, row[1]), ) diff --git a/internal/endtoend/testdata/inflection_exclude_table_names/schema.sql b/internal/endtoend/testdata/inflection_exclude_table_names/schema.sql index ea5b797..d4a4e22 100644 --- a/internal/endtoend/testdata/inflection_exclude_table_names/schema.sql +++ b/internal/endtoend/testdata/inflection_exclude_table_names/schema.sql @@ -1,4 +1,3 @@ CREATE TABLE bars (id serial not null, name text not null, primary key (id)); CREATE TABLE my_data (id serial not null, name text not null, primary key (id)); CREATE TABLE exclusions (id serial not null, name text not null, primary key (id)); - diff --git a/internal/endtoend/testdata/inflection_exclude_table_names/sqlc.yaml b/internal/endtoend/testdata/inflection_exclude_table_names/sqlc.yaml index efbb150..2dbd9d4 100644 --- a/internal/endtoend/testdata/inflection_exclude_table_names/sqlc.yaml +++ b/internal/endtoend/testdata/inflection_exclude_table_names/sqlc.yaml @@ -3,7 +3,7 @@ plugins: - name: py wasm: url: file://../../../../bin/sqlc-gen-python.wasm - sha256: "d6846ffad948181e611e883cedd2d2be66e091edc1273a0abc6c9da18399e0ca" + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" sql: - schema: schema.sql queries: query.sql diff --git a/internal/endtoend/testdata/method_name_conflict/query.sql b/internal/endtoend/testdata/method_name_conflict/query.sql new file mode 100644 index 0000000..8359180 --- /dev/null +++ b/internal/endtoend/testdata/method_name_conflict/query.sql @@ -0,0 +1,5 @@ +-- name: List :many +SELECT name FROM things; + +-- name: List_ :many +SELECT id FROM things; diff --git a/internal/endtoend/testdata/method_name_conflict/schema.sql b/internal/endtoend/testdata/method_name_conflict/schema.sql new file mode 100644 index 0000000..cf1ff81 --- /dev/null +++ b/internal/endtoend/testdata/method_name_conflict/schema.sql @@ -0,0 +1 @@ +CREATE TABLE things (id integer NOT NULL, name text NOT NULL); diff --git a/internal/endtoend/testdata/method_name_conflict/sqlc.yaml b/internal/endtoend/testdata/method_name_conflict/sqlc.yaml new file mode 100644 index 0000000..b2cb463 --- /dev/null +++ b/internal/endtoend/testdata/method_name_conflict/sqlc.yaml @@ -0,0 +1,16 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true diff --git a/internal/endtoend/testdata/method_name_conflict/stderr.txt b/internal/endtoend/testdata/method_name_conflict/stderr.txt new file mode 100644 index 0000000..7829c37 --- /dev/null +++ b/internal/endtoend/testdata/method_name_conflict/stderr.txt @@ -0,0 +1,2 @@ +# package py +error generating code: error generating output: queries List and List_ both generate method list_ diff --git a/internal/endtoend/testdata/method_name_conflict_nfkc/query.sql b/internal/endtoend/testdata/method_name_conflict_nfkc/query.sql new file mode 100644 index 0000000..e3bc43c --- /dev/null +++ b/internal/endtoend/testdata/method_name_conflict_nfkc/query.sql @@ -0,0 +1,5 @@ +-- name: GetIs :one +SELECT name FROM things WHERE id = $1; + +-- name: GetIſ :one +SELECT id FROM things WHERE id = $1; diff --git a/internal/endtoend/testdata/method_name_conflict_nfkc/schema.sql b/internal/endtoend/testdata/method_name_conflict_nfkc/schema.sql new file mode 100644 index 0000000..cf1ff81 --- /dev/null +++ b/internal/endtoend/testdata/method_name_conflict_nfkc/schema.sql @@ -0,0 +1 @@ +CREATE TABLE things (id integer NOT NULL, name text NOT NULL); diff --git a/internal/endtoend/testdata/method_name_conflict_nfkc/sqlc.yaml b/internal/endtoend/testdata/method_name_conflict_nfkc/sqlc.yaml new file mode 100644 index 0000000..b2cb463 --- /dev/null +++ b/internal/endtoend/testdata/method_name_conflict_nfkc/sqlc.yaml @@ -0,0 +1,16 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true diff --git a/internal/endtoend/testdata/method_name_conflict_nfkc/stderr.txt b/internal/endtoend/testdata/method_name_conflict_nfkc/stderr.txt new file mode 100644 index 0000000..07758ff --- /dev/null +++ b/internal/endtoend/testdata/method_name_conflict_nfkc/stderr.txt @@ -0,0 +1,2 @@ +# package py +error generating code: error generating output: queries GetIs and GetIſ both generate method get_iſ diff --git a/internal/endtoend/testdata/method_name_per_file/db/a.py b/internal/endtoend/testdata/method_name_per_file/db/a.py new file mode 100644 index 0000000..3dee904 --- /dev/null +++ b/internal/endtoend/testdata/method_name_per_file/db/a.py @@ -0,0 +1,38 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: a.sql +from typing import Iterator, Optional, Union, cast + +import sqlalchemy +import sqlalchemy.orm + +from db import models + + +GET_USER = """-- name: get_user \\:one +SELECT name FROM things WHERE id = :p1 +""" + + +LIST = """-- name: list \\:many +SELECT name FROM things +""" + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def get_user(self, *, id: int) -> Optional[str]: + row = self._conn.execute(sqlalchemy.text(GET_USER), {"p1": id}).first() + if row is None: + return None + return cast(str, row[0]) + + def list_(self) -> Iterator[str]: + result = self._conn.execute(sqlalchemy.text(LIST)) + for row in result: + yield cast(str, row[0]) diff --git a/internal/endtoend/testdata/method_name_per_file/db/b.py b/internal/endtoend/testdata/method_name_per_file/db/b.py new file mode 100644 index 0000000..08cb84a --- /dev/null +++ b/internal/endtoend/testdata/method_name_per_file/db/b.py @@ -0,0 +1,38 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: b.sql +from typing import Iterator, Optional, Union, cast + +import sqlalchemy +import sqlalchemy.orm + +from db import models + + +GET_USER = """-- name: get_user \\:one +SELECT id FROM things WHERE id = :p1 +""" + + +LIST_ = """-- name: list_ \\:many +SELECT id FROM things +""" + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def get_user(self, *, id: int) -> Optional[int]: + row = self._conn.execute(sqlalchemy.text(GET_USER), {"p1": id}).first() + if row is None: + return None + return cast(int, row[0]) + + def list_(self) -> Iterator[int]: + result = self._conn.execute(sqlalchemy.text(LIST_)) + for row in result: + yield cast(int, row[0]) diff --git a/internal/endtoend/testdata/method_name_per_file/db/models.py b/internal/endtoend/testdata/method_name_per_file/db/models.py new file mode 100644 index 0000000..47dcaa0 --- /dev/null +++ b/internal/endtoend/testdata/method_name_per_file/db/models.py @@ -0,0 +1,10 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses + + +@dataclasses.dataclass() +class Thing: + id: int + name: str diff --git a/internal/endtoend/testdata/method_name_per_file/query/a.sql b/internal/endtoend/testdata/method_name_per_file/query/a.sql new file mode 100644 index 0000000..5d0cce7 --- /dev/null +++ b/internal/endtoend/testdata/method_name_per_file/query/a.sql @@ -0,0 +1,5 @@ +-- name: GetUser :one +SELECT name FROM things WHERE id = $1; + +-- name: List :many +SELECT name FROM things; diff --git a/internal/endtoend/testdata/method_name_per_file/query/b.sql b/internal/endtoend/testdata/method_name_per_file/query/b.sql new file mode 100644 index 0000000..9e944e7 --- /dev/null +++ b/internal/endtoend/testdata/method_name_per_file/query/b.sql @@ -0,0 +1,5 @@ +-- name: getUser :one +SELECT id FROM things WHERE id = $1; + +-- name: List_ :many +SELECT id FROM things; diff --git a/internal/endtoend/testdata/method_name_per_file/schema.sql b/internal/endtoend/testdata/method_name_per_file/schema.sql new file mode 100644 index 0000000..cf1ff81 --- /dev/null +++ b/internal/endtoend/testdata/method_name_per_file/schema.sql @@ -0,0 +1 @@ +CREATE TABLE things (id integer NOT NULL, name text NOT NULL); diff --git a/internal/endtoend/testdata/method_name_per_file/sqlc.yaml b/internal/endtoend/testdata/method_name_per_file/sqlc.yaml new file mode 100644 index 0000000..55b7851 --- /dev/null +++ b/internal/endtoend/testdata/method_name_per_file/sqlc.yaml @@ -0,0 +1,16 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query/ + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true diff --git a/internal/endtoend/testdata/models_conflict/models.sql b/internal/endtoend/testdata/models_conflict/models.sql new file mode 100644 index 0000000..fb46e56 --- /dev/null +++ b/internal/endtoend/testdata/models_conflict/models.sql @@ -0,0 +1,2 @@ +-- name: CreateAuthor :one +INSERT INTO authors (name, bio) VALUES ($1, $2) RETURNING *; diff --git a/internal/endtoend/testdata/models_conflict/schema.sql b/internal/endtoend/testdata/models_conflict/schema.sql new file mode 100644 index 0000000..575dd39 --- /dev/null +++ b/internal/endtoend/testdata/models_conflict/schema.sql @@ -0,0 +1,5 @@ +CREATE TABLE authors ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL UNIQUE, + bio text +); diff --git a/internal/endtoend/testdata/models_conflict/sqlc.yaml b/internal/endtoend/testdata/models_conflict/sqlc.yaml new file mode 100644 index 0000000..2dc8b85 --- /dev/null +++ b/internal/endtoend/testdata/models_conflict/sqlc.yaml @@ -0,0 +1,16 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: models.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true diff --git a/internal/endtoend/testdata/models_conflict/stderr.txt b/internal/endtoend/testdata/models_conflict/stderr.txt new file mode 100644 index 0000000..d821999 --- /dev/null +++ b/internal/endtoend/testdata/models_conflict/stderr.txt @@ -0,0 +1,2 @@ +# package py +error generating code: error generating output: query file models.sql would overwrite models.py diff --git a/internal/endtoend/testdata/multidim_arrays/db/models.py b/internal/endtoend/testdata/multidim_arrays/db/models.py new file mode 100644 index 0000000..9427af5 --- /dev/null +++ b/internal/endtoend/testdata/multidim_arrays/db/models.py @@ -0,0 +1,13 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses +from typing import List, Optional + + +@dataclasses.dataclass() +class Grid: + id: int + matrix: List[List[int]] + cube: Optional[List[List[List[int]]]] + tags: Optional[List[str]] diff --git a/internal/endtoend/testdata/multidim_arrays/db/query.py b/internal/endtoend/testdata/multidim_arrays/db/query.py new file mode 100644 index 0000000..e628d57 --- /dev/null +++ b/internal/endtoend/testdata/multidim_arrays/db/query.py @@ -0,0 +1,63 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +from typing import List, Optional, Union, cast + +import sqlalchemy +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from db import models + + +CREATE_GRID = """-- name: create_grid \\:exec +INSERT INTO grids (matrix, cube, tags) VALUES (:p1, :p2, :p3) +""" + + +GET_GRID = """-- name: get_grid \\:one +SELECT id, matrix, cube, tags FROM grids WHERE id = :p1 +""" + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def create_grid(self, *, matrix: List[List[int]], cube: Optional[List[List[List[int]]]], tags: Optional[List[str]]) -> None: + self._conn.execute(sqlalchemy.text(CREATE_GRID), {"p1": matrix, "p2": cube, "p3": tags}) + + def get_grid(self, *, id: int) -> Optional[models.Grid]: + row = self._conn.execute(sqlalchemy.text(GET_GRID), {"p1": id}).first() + if row is None: + return None + return models.Grid( + id=cast(int, row[0]), + matrix=cast(List[List[int]], row[1]), + cube=cast(Optional[List[List[List[int]]]], row[2]), + tags=cast(Optional[List[str]], row[3]), + ) + + +class AsyncQuerier: + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): + self._conn = conn + + async def create_grid(self, *, matrix: List[List[int]], cube: Optional[List[List[List[int]]]], tags: Optional[List[str]]) -> None: + await self._conn.execute(sqlalchemy.text(CREATE_GRID), {"p1": matrix, "p2": cube, "p3": tags}) + + async def get_grid(self, *, id: int) -> Optional[models.Grid]: + row = (await self._conn.execute(sqlalchemy.text(GET_GRID), {"p1": id})).first() + if row is None: + return None + return models.Grid( + id=cast(int, row[0]), + matrix=cast(List[List[int]], row[1]), + cube=cast(Optional[List[List[List[int]]]], row[2]), + tags=cast(Optional[List[str]], row[3]), + ) diff --git a/internal/endtoend/testdata/multidim_arrays/query.sql b/internal/endtoend/testdata/multidim_arrays/query.sql new file mode 100644 index 0000000..079a1c6 --- /dev/null +++ b/internal/endtoend/testdata/multidim_arrays/query.sql @@ -0,0 +1,5 @@ +-- name: GetGrid :one +SELECT * FROM grids WHERE id = $1; + +-- name: CreateGrid :exec +INSERT INTO grids (matrix, cube, tags) VALUES ($1, $2, $3); diff --git a/internal/endtoend/testdata/multidim_arrays/schema.sql b/internal/endtoend/testdata/multidim_arrays/schema.sql new file mode 100644 index 0000000..38efdf6 --- /dev/null +++ b/internal/endtoend/testdata/multidim_arrays/schema.sql @@ -0,0 +1,6 @@ +CREATE TABLE grids ( + id BIGSERIAL PRIMARY KEY, + matrix integer[][] NOT NULL, + cube integer[][][], + tags text[] +); diff --git a/internal/endtoend/testdata/multidim_arrays/sqlc.yaml b/internal/endtoend/testdata/multidim_arrays/sqlc.yaml new file mode 100644 index 0000000..da10353 --- /dev/null +++ b/internal/endtoend/testdata/multidim_arrays/sqlc.yaml @@ -0,0 +1,17 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + emit_async_querier: true diff --git a/internal/endtoend/testdata/multiline_comments/db/models.py b/internal/endtoend/testdata/multiline_comments/db/models.py new file mode 100644 index 0000000..31c9789 --- /dev/null +++ b/internal/endtoend/testdata/multiline_comments/db/models.py @@ -0,0 +1,24 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses +import enum +from typing import Optional + + +class Mood(str, enum.Enum): + """How someone feels. +Two values only.""" + HAPPY = "happy" + SAD = "sad" + + +@dataclasses.dataclass() +class Author: + """People who write books. +Second line with "quotes".""" + id: int + # first line + # second line + name: str + mood: Optional[Mood] diff --git a/internal/endtoend/testdata/multiline_comments/db/query.py b/internal/endtoend/testdata/multiline_comments/db/query.py new file mode 100644 index 0000000..e78371d --- /dev/null +++ b/internal/endtoend/testdata/multiline_comments/db/query.py @@ -0,0 +1,33 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +from typing import Optional, Union, cast + +import sqlalchemy +import sqlalchemy.orm + +from db import models + + +GET_AUTHOR = """-- name: get_author \\:one +SELECT id, name, mood FROM authors +WHERE id = :p1 LIMIT 1 +""" + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def get_author(self, *, id: int) -> Optional[models.Author]: + row = self._conn.execute(sqlalchemy.text(GET_AUTHOR), {"p1": id}).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + mood=cast(Optional[models.Mood], row[2]), + ) diff --git a/internal/endtoend/testdata/multiline_comments/query.sql b/internal/endtoend/testdata/multiline_comments/query.sql new file mode 100644 index 0000000..6c0b1c8 --- /dev/null +++ b/internal/endtoend/testdata/multiline_comments/query.sql @@ -0,0 +1,3 @@ +-- name: GetAuthor :one +SELECT * FROM authors +WHERE id = $1 LIMIT 1; diff --git a/internal/endtoend/testdata/multiline_comments/schema.sql b/internal/endtoend/testdata/multiline_comments/schema.sql new file mode 100644 index 0000000..b58400b --- /dev/null +++ b/internal/endtoend/testdata/multiline_comments/schema.sql @@ -0,0 +1,16 @@ +CREATE TYPE mood AS ENUM ('happy', 'sad'); + +COMMENT ON TYPE mood IS 'How someone feels. +Two values only.'; + +CREATE TABLE authors ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL, + mood mood +); + +COMMENT ON TABLE authors IS 'People who write books. +Second line with "quotes".'; + +COMMENT ON COLUMN authors.name IS 'first line +second line'; diff --git a/internal/endtoend/testdata/multiline_comments/sqlc.yaml b/internal/endtoend/testdata/multiline_comments/sqlc.yaml new file mode 100644 index 0000000..2848a1d --- /dev/null +++ b/internal/endtoend/testdata/multiline_comments/sqlc.yaml @@ -0,0 +1,17 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + emit_async_querier: false diff --git a/internal/endtoend/testdata/omit_sqlc_version/db/models.py b/internal/endtoend/testdata/omit_sqlc_version/db/models.py new file mode 100644 index 0000000..49fd77c --- /dev/null +++ b/internal/endtoend/testdata/omit_sqlc_version/db/models.py @@ -0,0 +1,8 @@ +# Code generated by sqlc. DO NOT EDIT. +import dataclasses + + +@dataclasses.dataclass() +class Author: + id: int + name: str diff --git a/internal/endtoend/testdata/omit_sqlc_version/db/query.py b/internal/endtoend/testdata/omit_sqlc_version/db/query.py new file mode 100644 index 0000000..77b1a14 --- /dev/null +++ b/internal/endtoend/testdata/omit_sqlc_version/db/query.py @@ -0,0 +1,28 @@ +# Code generated by sqlc. DO NOT EDIT. +# source: query.sql +from typing import Iterator, Union, cast + +import sqlalchemy +import sqlalchemy.orm + +from db import models + + +LIST_AUTHORS = """-- name: list_authors \\:many +SELECT id, name FROM authors +""" + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def list_authors(self) -> Iterator[models.Author]: + result = self._conn.execute(sqlalchemy.text(LIST_AUTHORS)) + for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + ) diff --git a/internal/endtoend/testdata/omit_sqlc_version/query.sql b/internal/endtoend/testdata/omit_sqlc_version/query.sql new file mode 100644 index 0000000..86636e2 --- /dev/null +++ b/internal/endtoend/testdata/omit_sqlc_version/query.sql @@ -0,0 +1,2 @@ +-- name: ListAuthors :many +SELECT * FROM authors; diff --git a/internal/endtoend/testdata/omit_sqlc_version/schema.sql b/internal/endtoend/testdata/omit_sqlc_version/schema.sql new file mode 100644 index 0000000..0b27d3b --- /dev/null +++ b/internal/endtoend/testdata/omit_sqlc_version/schema.sql @@ -0,0 +1,4 @@ +CREATE TABLE authors ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL +); diff --git a/internal/endtoend/testdata/omit_sqlc_version/sqlc.yaml b/internal/endtoend/testdata/omit_sqlc_version/sqlc.yaml new file mode 100644 index 0000000..363faaa --- /dev/null +++ b/internal/endtoend/testdata/omit_sqlc_version/sqlc.yaml @@ -0,0 +1,17 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + omit_sqlc_version: true diff --git a/internal/endtoend/testdata/omit_unused_structs/db/models.py b/internal/endtoend/testdata/omit_unused_structs/db/models.py new file mode 100644 index 0000000..4af9db2 --- /dev/null +++ b/internal/endtoend/testdata/omit_unused_structs/db/models.py @@ -0,0 +1,22 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses +import enum + + +class Priority(str, enum.Enum): + LOW = "low" + HIGH = "high" + + +class Status(str, enum.Enum): + ACTIVE = "active" + RETIRED = "retired" + + +@dataclasses.dataclass() +class Author: + id: int + name: str + status: Status diff --git a/internal/endtoend/testdata/omit_unused_structs/db/query.py b/internal/endtoend/testdata/omit_unused_structs/db/query.py new file mode 100644 index 0000000..35c194d --- /dev/null +++ b/internal/endtoend/testdata/omit_unused_structs/db/query.py @@ -0,0 +1,52 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import dataclasses +from typing import Iterator, Optional, Union, cast + +import sqlalchemy +import sqlalchemy.orm + +from db import models + + +GET_AUTHOR = """-- name: get_author \\:one +SELECT id, name, status FROM authors WHERE id = :p1 +""" + + +LIST_TASK_TITLES = """-- name: list_task_titles \\:many +SELECT id, title FROM tasks WHERE priority = :p1 +""" + + +@dataclasses.dataclass() +class ListTaskTitlesRow: + id: int + title: str + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def get_author(self, *, id: int) -> Optional[models.Author]: + row = self._conn.execute(sqlalchemy.text(GET_AUTHOR), {"p1": id}).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + status=cast(models.Status, row[2]), + ) + + def list_task_titles(self, *, priority: models.Priority) -> Iterator[ListTaskTitlesRow]: + result = self._conn.execute(sqlalchemy.text(LIST_TASK_TITLES), {"p1": priority}) + for row in result: + yield ListTaskTitlesRow( + id=cast(int, row[0]), + title=cast(str, row[1]), + ) diff --git a/internal/endtoend/testdata/omit_unused_structs/query.sql b/internal/endtoend/testdata/omit_unused_structs/query.sql new file mode 100644 index 0000000..1d77aae --- /dev/null +++ b/internal/endtoend/testdata/omit_unused_structs/query.sql @@ -0,0 +1,5 @@ +-- name: GetAuthor :one +SELECT * FROM authors WHERE id = $1; + +-- name: ListTaskTitles :many +SELECT id, title FROM tasks WHERE priority = $1; diff --git a/internal/endtoend/testdata/omit_unused_structs/schema.sql b/internal/endtoend/testdata/omit_unused_structs/schema.sql new file mode 100644 index 0000000..4ca1530 --- /dev/null +++ b/internal/endtoend/testdata/omit_unused_structs/schema.sql @@ -0,0 +1,20 @@ +CREATE TYPE status AS ENUM ('active', 'retired'); +CREATE TYPE audit_kind AS ENUM ('insert', 'delete'); +CREATE TYPE priority AS ENUM ('low', 'high'); + +CREATE TABLE authors ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL, + status status NOT NULL +); + +CREATE TABLE audit_log ( + id BIGSERIAL PRIMARY KEY, + kind audit_kind NOT NULL +); + +CREATE TABLE tasks ( + id BIGSERIAL PRIMARY KEY, + priority priority NOT NULL, + title text NOT NULL +); diff --git a/internal/endtoend/testdata/omit_unused_structs/sqlc.yaml b/internal/endtoend/testdata/omit_unused_structs/sqlc.yaml new file mode 100644 index 0000000..04bd17b --- /dev/null +++ b/internal/endtoend/testdata/omit_unused_structs/sqlc.yaml @@ -0,0 +1,17 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + omit_unused_structs: true diff --git a/internal/endtoend/testdata/overrides/db/events.py b/internal/endtoend/testdata/overrides/db/events.py new file mode 100644 index 0000000..4161800 --- /dev/null +++ b/internal/endtoend/testdata/overrides/db/events.py @@ -0,0 +1,56 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: events.sql +import decimal +import uuid +from typing import Iterator, List, Optional, Union, cast + +import my_lib +import my_lib.types +import sqlalchemy +import sqlalchemy.orm + +from db import models + + +CREATE_EVENT = """-- name: create_event \\:one +INSERT INTO events (payload, extra) VALUES (:p1, :p2) RETURNING id, payload, extra, history, score, rank +""" + + +LIST_EVENTS_BY_RANK = """-- name: list_events_by_rank \\:many +SELECT id, payload, extra, history, score, rank FROM events WHERE rank > :p1 AND rank < :p2 +""" + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def create_event(self, *, payload: my_lib.types.Payload, extra: Optional[my_lib.types.Payload]) -> Optional[models.Event]: + row = self._conn.execute(sqlalchemy.text(CREATE_EVENT), {"p1": payload, "p2": extra}).first() + if row is None: + return None + return models.Event( + id=cast(int, row[0]), + payload=cast(my_lib.types.Payload, row[1]), + extra=cast(Optional[my_lib.types.Payload], row[2]), + history=cast(Optional[List[my_lib.types.Payload]], row[3]), + score=cast(my_lib.Box[decimal.Decimal], row[4]), + rank=cast(dict[str, uuid.UUID], row[5]), + ) + + def list_events_by_rank(self, *, rank: dict[str, uuid.UUID], rank_2: dict[str, uuid.UUID]) -> Iterator[models.Event]: + result = self._conn.execute(sqlalchemy.text(LIST_EVENTS_BY_RANK), {"p1": rank, "p2": rank_2}) + for row in result: + yield models.Event( + id=cast(int, row[0]), + payload=cast(my_lib.types.Payload, row[1]), + extra=cast(Optional[my_lib.types.Payload], row[2]), + history=cast(Optional[List[my_lib.types.Payload]], row[3]), + score=cast(my_lib.Box[decimal.Decimal], row[4]), + rank=cast(dict[str, uuid.UUID], row[5]), + ) diff --git a/internal/endtoend/testdata/overrides/db/models.py b/internal/endtoend/testdata/overrides/db/models.py new file mode 100644 index 0000000..0f5bcd8 --- /dev/null +++ b/internal/endtoend/testdata/overrides/db/models.py @@ -0,0 +1,36 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses +import decimal +import uuid +from typing import List, Optional +from uuid import UUID + +import my_ids +import my_lib +import my_lib.types + + +@dataclasses.dataclass() +class Author: + id: UUID + name: str + avatar: bytes + + +@dataclasses.dataclass() +class Event: + id: int + payload: my_lib.types.Payload + extra: Optional[my_lib.types.Payload] + history: Optional[List[my_lib.types.Payload]] + score: my_lib.Box[decimal.Decimal] + rank: dict[str, uuid.UUID] + + +@dataclasses.dataclass() +class User: + id: my_ids.UserId + org_id: my_ids.Id + extra: Optional[my_lib.types.Payload] diff --git a/internal/endtoend/testdata/overrides/db/users.py b/internal/endtoend/testdata/overrides/db/users.py new file mode 100644 index 0000000..ecb4519 --- /dev/null +++ b/internal/endtoend/testdata/overrides/db/users.py @@ -0,0 +1,64 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: users.sql +from typing import Iterator, Optional, Union, cast +from uuid import UUID + +import my_ids +import my_lib.types +import sqlalchemy +import sqlalchemy.orm + +from db import models + + +GET_AUTHOR = """-- name: get_author \\:one +SELECT id, name, avatar FROM authors WHERE id = :p1 +""" + + +GET_USER = """-- name: get_user \\:one +SELECT id, org_id, extra FROM users WHERE id = :p1 +""" + + +LIST_USERS_BY_ORG = """-- name: list_users_by_org \\:many +SELECT id, org_id, extra FROM users WHERE org_id = :p1 +""" + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def get_author(self, *, id: UUID) -> Optional[models.Author]: + row = self._conn.execute(sqlalchemy.text(GET_AUTHOR), {"p1": id}).first() + if row is None: + return None + return models.Author( + id=cast(UUID, row[0]), + name=cast(str, row[1]), + avatar=cast(bytes, row[2]), + ) + + def get_user(self, *, id: my_ids.UserId) -> Optional[models.User]: + row = self._conn.execute(sqlalchemy.text(GET_USER), {"p1": id}).first() + if row is None: + return None + return models.User( + id=cast(my_ids.UserId, row[0]), + org_id=cast(my_ids.Id, row[1]), + extra=cast(Optional[my_lib.types.Payload], row[2]), + ) + + def list_users_by_org(self, *, org_id: my_ids.Id) -> Iterator[models.User]: + result = self._conn.execute(sqlalchemy.text(LIST_USERS_BY_ORG), {"p1": org_id}) + for row in result: + yield models.User( + id=cast(my_ids.UserId, row[0]), + org_id=cast(my_ids.Id, row[1]), + extra=cast(Optional[my_lib.types.Payload], row[2]), + ) diff --git a/internal/endtoend/testdata/overrides/queries/events.sql b/internal/endtoend/testdata/overrides/queries/events.sql new file mode 100644 index 0000000..14d7b5a --- /dev/null +++ b/internal/endtoend/testdata/overrides/queries/events.sql @@ -0,0 +1,5 @@ +-- name: CreateEvent :one +INSERT INTO events (payload, extra) VALUES ($1, $2) RETURNING *; + +-- name: ListEventsByRank :many +SELECT * FROM events WHERE rank > $1 AND rank < $2; diff --git a/internal/endtoend/testdata/overrides/queries/users.sql b/internal/endtoend/testdata/overrides/queries/users.sql new file mode 100644 index 0000000..3dabc70 --- /dev/null +++ b/internal/endtoend/testdata/overrides/queries/users.sql @@ -0,0 +1,8 @@ +-- name: GetUser :one +SELECT * FROM users WHERE id = $1; + +-- name: ListUsersByOrg :many +SELECT * FROM users WHERE org_id = $1; + +-- name: GetAuthor :one +SELECT * FROM authors WHERE id = $1; diff --git a/internal/endtoend/testdata/overrides/schema.sql b/internal/endtoend/testdata/overrides/schema.sql new file mode 100644 index 0000000..94187d4 --- /dev/null +++ b/internal/endtoend/testdata/overrides/schema.sql @@ -0,0 +1,20 @@ +CREATE TABLE users ( + id uuid PRIMARY KEY, + org_id uuid NOT NULL, + extra jsonb +); + +CREATE TABLE authors ( + id uuid PRIMARY KEY, + name text NOT NULL, + avatar bytea NOT NULL +); + +CREATE TABLE events ( + id BIGSERIAL PRIMARY KEY, + payload jsonb NOT NULL, + extra jsonb, + history jsonb[], + score double precision NOT NULL, + rank bigint NOT NULL +); diff --git a/internal/endtoend/testdata/overrides/sqlc.yaml b/internal/endtoend/testdata/overrides/sqlc.yaml new file mode 100644 index 0000000..0086123 --- /dev/null +++ b/internal/endtoend/testdata/overrides/sqlc.yaml @@ -0,0 +1,35 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: queries + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + overrides: + - db_type: jsonb + py_type: my_lib.types.Payload + - db_type: jsonb + nullable: true + py_type: my_lib.types.Payload + - db_type: uuid + py_type: my_ids.Id + - column: "users.id" + py_type: my_ids.UserId + - column: "authors.id" + py_type: UUID + py_import: uuid + - db_type: bytea + py_type: bytes + - db_type: double precision + py_type: "my_lib.Box[decimal.Decimal]" + - db_type: bigint + py_type: "dict[str, uuid.UUID]" diff --git a/internal/endtoend/testdata/overrides_global/db/models.py b/internal/endtoend/testdata/overrides_global/db/models.py new file mode 100644 index 0000000..bda5ac6 --- /dev/null +++ b/internal/endtoend/testdata/overrides_global/db/models.py @@ -0,0 +1,13 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses + +import my_lib + + +@dataclasses.dataclass() +class Member: + id: int + email: my_lib.CIText + alias: str diff --git a/internal/endtoend/testdata/overrides_global/db/query.py b/internal/endtoend/testdata/overrides_global/db/query.py new file mode 100644 index 0000000..b312701 --- /dev/null +++ b/internal/endtoend/testdata/overrides_global/db/query.py @@ -0,0 +1,33 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +from typing import Optional, Union, cast + +import my_lib +import sqlalchemy +import sqlalchemy.orm + +from db import models + + +GET_MEMBER_BY_EMAIL = """-- name: get_member_by_email \\:one +SELECT id, email, nickname FROM members WHERE email = :p1 +""" + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def get_member_by_email(self, *, email: my_lib.CIText) -> Optional[models.Member]: + row = self._conn.execute(sqlalchemy.text(GET_MEMBER_BY_EMAIL), {"p1": email}).first() + if row is None: + return None + return models.Member( + id=cast(int, row[0]), + email=cast(my_lib.CIText, row[1]), + alias=cast(str, row[2]), + ) diff --git a/internal/endtoend/testdata/overrides_global/query.sql b/internal/endtoend/testdata/overrides_global/query.sql new file mode 100644 index 0000000..fac258c --- /dev/null +++ b/internal/endtoend/testdata/overrides_global/query.sql @@ -0,0 +1,2 @@ +-- name: GetMemberByEmail :one +SELECT * FROM members WHERE email = $1; diff --git a/internal/endtoend/testdata/overrides_global/schema.sql b/internal/endtoend/testdata/overrides_global/schema.sql new file mode 100644 index 0000000..c7ea07d --- /dev/null +++ b/internal/endtoend/testdata/overrides_global/schema.sql @@ -0,0 +1,7 @@ +CREATE EXTENSION IF NOT EXISTS citext; + +CREATE TABLE members ( + id BIGSERIAL PRIMARY KEY, + email citext NOT NULL, + nickname text NOT NULL +); diff --git a/internal/endtoend/testdata/overrides_global/sqlc.yaml b/internal/endtoend/testdata/overrides_global/sqlc.yaml new file mode 100644 index 0000000..be49c2b --- /dev/null +++ b/internal/endtoend/testdata/overrides_global/sqlc.yaml @@ -0,0 +1,25 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + rename: + nickname: handle +options: + py: + overrides: + - db_type: citext + py_type: my_lib.CIText + rename: + nickname: alias diff --git a/internal/endtoend/testdata/postgresql_type_aliases/db/models.py b/internal/endtoend/testdata/postgresql_type_aliases/db/models.py new file mode 100644 index 0000000..2089698 --- /dev/null +++ b/internal/endtoend/testdata/postgresql_type_aliases/db/models.py @@ -0,0 +1,26 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses +import datetime +from typing import Any + + +@dataclasses.dataclass() +class Everything: + id: int + a_varchar: str + a_charvar: str + a_bpchar: str + a_char: str + a_name: str + a_time: datetime.time + a_timetz: datetime.time + a_time_wo: datetime.time + a_time_w: datetime.time + a_ts: datetime.datetime + a_tstz: datetime.datetime + a_ts_wo: datetime.datetime + a_ts_w: datetime.datetime + a_json: Any + a_jsonb: Any diff --git a/internal/endtoend/testdata/postgresql_type_aliases/db/query.py b/internal/endtoend/testdata/postgresql_type_aliases/db/query.py new file mode 100644 index 0000000..252bae0 --- /dev/null +++ b/internal/endtoend/testdata/postgresql_type_aliases/db/query.py @@ -0,0 +1,118 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import dataclasses +import datetime +from typing import Any, Optional, Union, cast + +import sqlalchemy +import sqlalchemy.orm + +from db import models + + +CAST_ALIASES = """-- name: cast_aliases \\:one +SELECT + :p1\\:\\:varchar AS v, + :p2\\:\\:character varying AS cv, + :p3\\:\\:bpchar AS bp, + :p4\\:\\:name AS n, + :p5\\:\\:time AS t, + :p6\\:\\:timetz AS ttz, + :p7\\:\\:timestamp AS ts, + :p8\\:\\:timestamptz AS tstz, + :p9\\:\\:pg_catalog.json AS j, + :p10\\:\\:pg_catalog.jsonb AS jb +""" + + +@dataclasses.dataclass() +class CastAliasesParams: + column_1: str + column_2: str + column_3: str + column_4: str + column_5: datetime.time + column_6: datetime.time + column_7: datetime.datetime + column_8: datetime.datetime + column_9: Any + column_10: Any + + +@dataclasses.dataclass() +class CastAliasesRow: + v: str + cv: str + bp: str + n: str + t: datetime.time + ttz: datetime.time + ts: datetime.datetime + tstz: datetime.datetime + j: Any + jb: Any + + +GET_EVERYTHING = """-- name: get_everything \\:one +SELECT id, a_varchar, a_charvar, a_bpchar, a_char, a_name, a_time, a_timetz, a_time_wo, a_time_w, a_ts, a_tstz, a_ts_wo, a_ts_w, a_json, a_jsonb FROM everything WHERE id = :p1 +""" + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def cast_aliases(self, arg: CastAliasesParams) -> Optional[CastAliasesRow]: + row = self._conn.execute(sqlalchemy.text(CAST_ALIASES), { + "p1": arg.column_1, + "p2": arg.column_2, + "p3": arg.column_3, + "p4": arg.column_4, + "p5": arg.column_5, + "p6": arg.column_6, + "p7": arg.column_7, + "p8": arg.column_8, + "p9": arg.column_9, + "p10": arg.column_10, + }).first() + if row is None: + return None + return CastAliasesRow( + v=cast(str, row[0]), + cv=cast(str, row[1]), + bp=cast(str, row[2]), + n=cast(str, row[3]), + t=cast(datetime.time, row[4]), + ttz=cast(datetime.time, row[5]), + ts=cast(datetime.datetime, row[6]), + tstz=cast(datetime.datetime, row[7]), + j=cast(Any, row[8]), + jb=cast(Any, row[9]), + ) + + def get_everything(self, *, id: int) -> Optional[models.Everything]: + row = self._conn.execute(sqlalchemy.text(GET_EVERYTHING), {"p1": id}).first() + if row is None: + return None + return models.Everything( + id=cast(int, row[0]), + a_varchar=cast(str, row[1]), + a_charvar=cast(str, row[2]), + a_bpchar=cast(str, row[3]), + a_char=cast(str, row[4]), + a_name=cast(str, row[5]), + a_time=cast(datetime.time, row[6]), + a_timetz=cast(datetime.time, row[7]), + a_time_wo=cast(datetime.time, row[8]), + a_time_w=cast(datetime.time, row[9]), + a_ts=cast(datetime.datetime, row[10]), + a_tstz=cast(datetime.datetime, row[11]), + a_ts_wo=cast(datetime.datetime, row[12]), + a_ts_w=cast(datetime.datetime, row[13]), + a_json=cast(Any, row[14]), + a_jsonb=cast(Any, row[15]), + ) diff --git a/internal/endtoend/testdata/postgresql_type_aliases/query.sql b/internal/endtoend/testdata/postgresql_type_aliases/query.sql new file mode 100644 index 0000000..c39d378 --- /dev/null +++ b/internal/endtoend/testdata/postgresql_type_aliases/query.sql @@ -0,0 +1,15 @@ +-- name: GetEverything :one +SELECT * FROM everything WHERE id = $1; + +-- name: CastAliases :one +SELECT + $1::varchar AS v, + $2::character varying AS cv, + $3::bpchar AS bp, + $4::name AS n, + $5::time AS t, + $6::timetz AS ttz, + $7::timestamp AS ts, + $8::timestamptz AS tstz, + $9::pg_catalog.json AS j, + $10::pg_catalog.jsonb AS jb; diff --git a/internal/endtoend/testdata/postgresql_type_aliases/schema.sql b/internal/endtoend/testdata/postgresql_type_aliases/schema.sql new file mode 100644 index 0000000..18de461 --- /dev/null +++ b/internal/endtoend/testdata/postgresql_type_aliases/schema.sql @@ -0,0 +1,18 @@ +CREATE TABLE everything ( + id BIGSERIAL PRIMARY KEY, + a_varchar varchar(10) NOT NULL, + a_charvar character varying NOT NULL, + a_bpchar bpchar NOT NULL, + a_char character(3) NOT NULL, + a_name name NOT NULL, + a_time time NOT NULL, + a_timetz timetz NOT NULL, + a_time_wo time without time zone NOT NULL, + a_time_w time with time zone NOT NULL, + a_ts timestamp NOT NULL, + a_tstz timestamptz NOT NULL, + a_ts_wo timestamp without time zone NOT NULL, + a_ts_w timestamp with time zone NOT NULL, + a_json json NOT NULL, + a_jsonb jsonb NOT NULL +); diff --git a/internal/endtoend/testdata/postgresql_type_aliases/sqlc.yaml b/internal/endtoend/testdata/postgresql_type_aliases/sqlc.yaml new file mode 100644 index 0000000..b2cb463 --- /dev/null +++ b/internal/endtoend/testdata/postgresql_type_aliases/sqlc.yaml @@ -0,0 +1,16 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true diff --git a/internal/endtoend/testdata/python_keywords/db/models.py b/internal/endtoend/testdata/python_keywords/db/models.py new file mode 100644 index 0000000..132dde9 --- /dev/null +++ b/internal/endtoend/testdata/python_keywords/db/models.py @@ -0,0 +1,12 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses + + +@dataclasses.dataclass() +class Trip: + id: int + from_: str + to: str + class_: str diff --git a/internal/endtoend/testdata/python_keywords/db/query.py b/internal/endtoend/testdata/python_keywords/db/query.py new file mode 100644 index 0000000..9d673a9 --- /dev/null +++ b/internal/endtoend/testdata/python_keywords/db/query.py @@ -0,0 +1,119 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import dataclasses +from typing import AsyncIterator, Iterator, Optional, Union, cast + +import sqlalchemy +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from db import models + + +GET_TRIP = """-- name: get_trip \\:one +SELECT id, "from", "to", class FROM trips +WHERE id = :p1 LIMIT 1 +""" + + +IMPORT = """-- name: import_ \\:exec +INSERT INTO trips ("from", "to", class) VALUES (:p1, :p2, :p3) +""" + + +LIST_DEPARTURES = """-- name: list_departures \\:many +SELECT id, "from" FROM trips +WHERE class = :p1 +""" + + +@dataclasses.dataclass() +class ListDeparturesRow: + id: int + from_: str + + +LIST_TRIPS_BY_CLASS = """-- name: list_trips_by_class \\:many +SELECT id, "from", "to", class FROM trips +WHERE class = :p1 +""" + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def get_trip(self, *, id: int) -> Optional[models.Trip]: + row = self._conn.execute(sqlalchemy.text(GET_TRIP), {"p1": id}).first() + if row is None: + return None + return models.Trip( + id=cast(int, row[0]), + from_=cast(str, row[1]), + to=cast(str, row[2]), + class_=cast(str, row[3]), + ) + + def import_(self, *, from_: str, to: str, class_: str) -> None: + self._conn.execute(sqlalchemy.text(IMPORT), {"p1": from_, "p2": to, "p3": class_}) + + def list_departures(self, *, class_: str) -> Iterator[ListDeparturesRow]: + result = self._conn.execute(sqlalchemy.text(LIST_DEPARTURES), {"p1": class_}) + for row in result: + yield ListDeparturesRow( + id=cast(int, row[0]), + from_=cast(str, row[1]), + ) + + def list_trips_by_class(self, *, class_: str) -> Iterator[models.Trip]: + result = self._conn.execute(sqlalchemy.text(LIST_TRIPS_BY_CLASS), {"p1": class_}) + for row in result: + yield models.Trip( + id=cast(int, row[0]), + from_=cast(str, row[1]), + to=cast(str, row[2]), + class_=cast(str, row[3]), + ) + + +class AsyncQuerier: + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): + self._conn = conn + + async def get_trip(self, *, id: int) -> Optional[models.Trip]: + row = (await self._conn.execute(sqlalchemy.text(GET_TRIP), {"p1": id})).first() + if row is None: + return None + return models.Trip( + id=cast(int, row[0]), + from_=cast(str, row[1]), + to=cast(str, row[2]), + class_=cast(str, row[3]), + ) + + async def import_(self, *, from_: str, to: str, class_: str) -> None: + await self._conn.execute(sqlalchemy.text(IMPORT), {"p1": from_, "p2": to, "p3": class_}) + + async def list_departures(self, *, class_: str) -> AsyncIterator[ListDeparturesRow]: + result = await self._conn.stream(sqlalchemy.text(LIST_DEPARTURES), {"p1": class_}) + async for row in result: + yield ListDeparturesRow( + id=cast(int, row[0]), + from_=cast(str, row[1]), + ) + + async def list_trips_by_class(self, *, class_: str) -> AsyncIterator[models.Trip]: + result = await self._conn.stream(sqlalchemy.text(LIST_TRIPS_BY_CLASS), {"p1": class_}) + async for row in result: + yield models.Trip( + id=cast(int, row[0]), + from_=cast(str, row[1]), + to=cast(str, row[2]), + class_=cast(str, row[3]), + ) diff --git a/internal/endtoend/testdata/python_keywords/query.sql b/internal/endtoend/testdata/python_keywords/query.sql new file mode 100644 index 0000000..d9f318c --- /dev/null +++ b/internal/endtoend/testdata/python_keywords/query.sql @@ -0,0 +1,14 @@ +-- name: GetTrip :one +SELECT * FROM trips +WHERE id = $1 LIMIT 1; + +-- name: ListTripsByClass :many +SELECT * FROM trips +WHERE class = $1; + +-- name: ListDepartures :many +SELECT id, "from" FROM trips +WHERE class = sqlc.arg(class); + +-- name: Import :exec +INSERT INTO trips ("from", "to", class) VALUES ($1, $2, $3); diff --git a/internal/endtoend/testdata/python_keywords/schema.sql b/internal/endtoend/testdata/python_keywords/schema.sql new file mode 100644 index 0000000..c7c6538 --- /dev/null +++ b/internal/endtoend/testdata/python_keywords/schema.sql @@ -0,0 +1,6 @@ +CREATE TABLE trips ( + id BIGSERIAL PRIMARY KEY, + "from" text NOT NULL, + "to" text NOT NULL, + class text NOT NULL +); diff --git a/internal/endtoend/testdata/python_keywords/sqlc.yaml b/internal/endtoend/testdata/python_keywords/sqlc.yaml new file mode 100644 index 0000000..da10353 --- /dev/null +++ b/internal/endtoend/testdata/python_keywords/sqlc.yaml @@ -0,0 +1,17 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + emit_async_querier: true diff --git a/internal/endtoend/testdata/query_comments/db/models.py b/internal/endtoend/testdata/query_comments/db/models.py new file mode 100644 index 0000000..973a3fe --- /dev/null +++ b/internal/endtoend/testdata/query_comments/db/models.py @@ -0,0 +1,12 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses +from typing import Optional + + +@dataclasses.dataclass() +class Author: + id: int + name: str + bio: Optional[str] diff --git a/internal/endtoend/testdata/query_comments/db/query.py b/internal/endtoend/testdata/query_comments/db/query.py new file mode 100644 index 0000000..2b65b5d --- /dev/null +++ b/internal/endtoend/testdata/query_comments/db/query.py @@ -0,0 +1,90 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +from typing import AsyncIterator, Iterator, Optional, Union, cast + +import sqlalchemy +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from db import models + + +DELETE_AUTHOR = """-- name: delete_author \\:exec +DELETE FROM authors WHERE id = :p1 +""" + + +GET_AUTHOR = """-- name: get_author \\:one +SELECT id, name, bio FROM authors WHERE id = :p1 +""" + + +LIST_AUTHORS = """-- name: list_authors \\:many +SELECT id, name, bio FROM authors ORDER BY name +""" + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def delete_author(self, *, id: int) -> None: + self._conn.execute(sqlalchemy.text(DELETE_AUTHOR), {"p1": id}) + + def get_author(self, *, id: int) -> Optional[models.Author]: + """Fetch an author by id.""" + row = self._conn.execute(sqlalchemy.text(GET_AUTHOR), {"p1": id}).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + def list_authors(self) -> Iterator[models.Author]: + """List every author. +Ordered by "name", with a \\ backslash.""" + result = self._conn.execute(sqlalchemy.text(LIST_AUTHORS)) + for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + +class AsyncQuerier: + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): + self._conn = conn + + async def delete_author(self, *, id: int) -> None: + await self._conn.execute(sqlalchemy.text(DELETE_AUTHOR), {"p1": id}) + + async def get_author(self, *, id: int) -> Optional[models.Author]: + """Fetch an author by id.""" + row = (await self._conn.execute(sqlalchemy.text(GET_AUTHOR), {"p1": id})).first() + if row is None: + return None + return models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) + + async def list_authors(self) -> AsyncIterator[models.Author]: + """List every author. +Ordered by "name", with a \\ backslash.""" + result = await self._conn.stream(sqlalchemy.text(LIST_AUTHORS)) + async for row in result: + yield models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + bio=cast(Optional[str], row[2]), + ) diff --git a/internal/endtoend/testdata/query_comments/query.sql b/internal/endtoend/testdata/query_comments/query.sql new file mode 100644 index 0000000..be8ea11 --- /dev/null +++ b/internal/endtoend/testdata/query_comments/query.sql @@ -0,0 +1,11 @@ +-- Fetch an author by id. +-- name: GetAuthor :one +SELECT * FROM authors WHERE id = $1; + +-- List every author. +-- Ordered by "name", with a \ backslash. +-- name: ListAuthors :many +SELECT * FROM authors ORDER BY name; + +-- name: DeleteAuthor :exec +DELETE FROM authors WHERE id = $1; diff --git a/internal/endtoend/testdata/query_comments/schema.sql b/internal/endtoend/testdata/query_comments/schema.sql new file mode 100644 index 0000000..a7b9212 --- /dev/null +++ b/internal/endtoend/testdata/query_comments/schema.sql @@ -0,0 +1,5 @@ +CREATE TABLE authors ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL, + bio text +); diff --git a/internal/endtoend/testdata/query_comments/sqlc.yaml b/internal/endtoend/testdata/query_comments/sqlc.yaml new file mode 100644 index 0000000..da10353 --- /dev/null +++ b/internal/endtoend/testdata/query_comments/sqlc.yaml @@ -0,0 +1,17 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + emit_async_querier: true diff --git a/internal/endtoend/testdata/query_parameter_limit_two/python/models.py b/internal/endtoend/testdata/query_parameter_limit_two/python/models.py index 059675d..c02ab26 100644 --- a/internal/endtoend/testdata/query_parameter_limit_two/python/models.py +++ b/internal/endtoend/testdata/query_parameter_limit_two/python/models.py @@ -1,6 +1,6 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 import dataclasses diff --git a/internal/endtoend/testdata/query_parameter_limit_two/python/query.py b/internal/endtoend/testdata/query_parameter_limit_two/python/query.py index e8b723e..456979a 100644 --- a/internal/endtoend/testdata/query_parameter_limit_two/python/query.py +++ b/internal/endtoend/testdata/query_parameter_limit_two/python/query.py @@ -1,9 +1,12 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 # source: query.sql +from typing import Any, Union, cast + import sqlalchemy import sqlalchemy.ext.asyncio +import sqlalchemy.orm from querytest import models @@ -19,26 +22,30 @@ class Querier: - def __init__(self, conn: sqlalchemy.engine.Connection): + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): self._conn = conn def delete_bar_by_id(self, *, id: int) -> int: result = self._conn.execute(sqlalchemy.text(DELETE_BAR_BY_ID), {"p1": id}) - return result.rowcount + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount def delete_bar_by_id_and_name(self, *, id: int, name: str) -> int: result = self._conn.execute(sqlalchemy.text(DELETE_BAR_BY_ID_AND_NAME), {"p1": id, "p2": name}) - return result.rowcount + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount class AsyncQuerier: - def __init__(self, conn: sqlalchemy.ext.asyncio.AsyncConnection): + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): self._conn = conn async def delete_bar_by_id(self, *, id: int) -> int: result = await self._conn.execute(sqlalchemy.text(DELETE_BAR_BY_ID), {"p1": id}) - return result.rowcount + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount async def delete_bar_by_id_and_name(self, *, id: int, name: str) -> int: result = await self._conn.execute(sqlalchemy.text(DELETE_BAR_BY_ID_AND_NAME), {"p1": id, "p2": name}) - return result.rowcount + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount diff --git a/internal/endtoend/testdata/query_parameter_limit_two/schema.sql b/internal/endtoend/testdata/query_parameter_limit_two/schema.sql index a27c312..42f134f 100644 --- a/internal/endtoend/testdata/query_parameter_limit_two/schema.sql +++ b/internal/endtoend/testdata/query_parameter_limit_two/schema.sql @@ -1,2 +1 @@ CREATE TABLE bar (id serial not null, name text not null, primary key (id)); - diff --git a/internal/endtoend/testdata/query_parameter_limit_two/sqlc.yaml b/internal/endtoend/testdata/query_parameter_limit_two/sqlc.yaml index 336bca7..8766977 100644 --- a/internal/endtoend/testdata/query_parameter_limit_two/sqlc.yaml +++ b/internal/endtoend/testdata/query_parameter_limit_two/sqlc.yaml @@ -3,7 +3,7 @@ plugins: - name: py wasm: url: file://../../../../bin/sqlc-gen-python.wasm - sha256: "d6846ffad948181e611e883cedd2d2be66e091edc1273a0abc6c9da18399e0ca" + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" sql: - schema: schema.sql queries: query.sql @@ -15,4 +15,4 @@ sql: package: querytest emit_sync_querier: true emit_async_querier: true - query_parameter_limit: 2 \ No newline at end of file + query_parameter_limit: 2 diff --git a/internal/endtoend/testdata/query_parameter_limit_undefined/python/models.py b/internal/endtoend/testdata/query_parameter_limit_undefined/python/models.py index 30e80db..f5fbb60 100644 --- a/internal/endtoend/testdata/query_parameter_limit_undefined/python/models.py +++ b/internal/endtoend/testdata/query_parameter_limit_undefined/python/models.py @@ -1,6 +1,6 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 import dataclasses diff --git a/internal/endtoend/testdata/query_parameter_limit_undefined/python/query.py b/internal/endtoend/testdata/query_parameter_limit_undefined/python/query.py index 5a1fbbc..846bc7a 100644 --- a/internal/endtoend/testdata/query_parameter_limit_undefined/python/query.py +++ b/internal/endtoend/testdata/query_parameter_limit_undefined/python/query.py @@ -1,9 +1,12 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 # source: query.sql +from typing import Any, Union, cast + import sqlalchemy import sqlalchemy.ext.asyncio +import sqlalchemy.orm from querytest import models @@ -23,12 +26,14 @@ class Querier: - def __init__(self, conn: sqlalchemy.engine.Connection): + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): self._conn = conn def delete_bar_by_id(self, *, id: int) -> int: result = self._conn.execute(sqlalchemy.text(DELETE_BAR_BY_ID), {"p1": id}) - return result.rowcount + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount def delete_bar_by_id_and_name(self, *, id: int, name1: str, name2: str, name3: str) -> int: result = self._conn.execute(sqlalchemy.text(DELETE_BAR_BY_ID_AND_NAME), { @@ -37,16 +42,18 @@ def delete_bar_by_id_and_name(self, *, id: int, name1: str, name2: str, name3: s "p3": name2, "p4": name3, }) - return result.rowcount + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount class AsyncQuerier: - def __init__(self, conn: sqlalchemy.ext.asyncio.AsyncConnection): + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): self._conn = conn async def delete_bar_by_id(self, *, id: int) -> int: result = await self._conn.execute(sqlalchemy.text(DELETE_BAR_BY_ID), {"p1": id}) - return result.rowcount + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount async def delete_bar_by_id_and_name(self, *, id: int, name1: str, name2: str, name3: str) -> int: result = await self._conn.execute(sqlalchemy.text(DELETE_BAR_BY_ID_AND_NAME), { @@ -55,4 +62,4 @@ async def delete_bar_by_id_and_name(self, *, id: int, name1: str, name2: str, na "p3": name2, "p4": name3, }) - return result.rowcount + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount diff --git a/internal/endtoend/testdata/query_parameter_limit_undefined/schema.sql b/internal/endtoend/testdata/query_parameter_limit_undefined/schema.sql index 1d9131f..1356b7c 100644 --- a/internal/endtoend/testdata/query_parameter_limit_undefined/schema.sql +++ b/internal/endtoend/testdata/query_parameter_limit_undefined/schema.sql @@ -4,4 +4,3 @@ CREATE TABLE bar ( name2 text not null, name3 text not null, primary key (id)); - diff --git a/internal/endtoend/testdata/query_parameter_limit_undefined/sqlc.yaml b/internal/endtoend/testdata/query_parameter_limit_undefined/sqlc.yaml index c20cd57..2678643 100644 --- a/internal/endtoend/testdata/query_parameter_limit_undefined/sqlc.yaml +++ b/internal/endtoend/testdata/query_parameter_limit_undefined/sqlc.yaml @@ -3,7 +3,7 @@ plugins: - name: py wasm: url: file://../../../../bin/sqlc-gen-python.wasm - sha256: "d6846ffad948181e611e883cedd2d2be66e091edc1273a0abc6c9da18399e0ca" + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" sql: - schema: schema.sql queries: query.sql @@ -14,4 +14,4 @@ sql: options: package: querytest emit_sync_querier: true - emit_async_querier: true \ No newline at end of file + emit_async_querier: true diff --git a/internal/endtoend/testdata/query_parameter_limit_zero/python/models.py b/internal/endtoend/testdata/query_parameter_limit_zero/python/models.py index 059675d..c02ab26 100644 --- a/internal/endtoend/testdata/query_parameter_limit_zero/python/models.py +++ b/internal/endtoend/testdata/query_parameter_limit_zero/python/models.py @@ -1,6 +1,6 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 import dataclasses diff --git a/internal/endtoend/testdata/query_parameter_limit_zero/python/query.py b/internal/endtoend/testdata/query_parameter_limit_zero/python/query.py index 47bd6a9..4c0dc35 100644 --- a/internal/endtoend/testdata/query_parameter_limit_zero/python/query.py +++ b/internal/endtoend/testdata/query_parameter_limit_zero/python/query.py @@ -1,11 +1,13 @@ # Code generated by sqlc. DO NOT EDIT. # versions: -# sqlc v1.28.0 +# sqlc v1.31.1 # source: query.sql import dataclasses +from typing import Any, Union, cast import sqlalchemy import sqlalchemy.ext.asyncio +import sqlalchemy.orm from querytest import models @@ -32,26 +34,30 @@ class DeleteBarByIDAndNameParams: class Querier: - def __init__(self, conn: sqlalchemy.engine.Connection): + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): self._conn = conn def delete_bar_by_id(self, arg: DeleteBarByIDParams) -> int: result = self._conn.execute(sqlalchemy.text(DELETE_BAR_BY_ID), {"p1": arg.id}) - return result.rowcount + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount def delete_bar_by_id_and_name(self, arg: DeleteBarByIDAndNameParams) -> int: result = self._conn.execute(sqlalchemy.text(DELETE_BAR_BY_ID_AND_NAME), {"p1": arg.id, "p2": arg.name}) - return result.rowcount + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount class AsyncQuerier: - def __init__(self, conn: sqlalchemy.ext.asyncio.AsyncConnection): + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): self._conn = conn async def delete_bar_by_id(self, arg: DeleteBarByIDParams) -> int: result = await self._conn.execute(sqlalchemy.text(DELETE_BAR_BY_ID), {"p1": arg.id}) - return result.rowcount + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount async def delete_bar_by_id_and_name(self, arg: DeleteBarByIDAndNameParams) -> int: result = await self._conn.execute(sqlalchemy.text(DELETE_BAR_BY_ID_AND_NAME), {"p1": arg.id, "p2": arg.name}) - return result.rowcount + return cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount diff --git a/internal/endtoend/testdata/query_parameter_limit_zero/schema.sql b/internal/endtoend/testdata/query_parameter_limit_zero/schema.sql index a27c312..42f134f 100644 --- a/internal/endtoend/testdata/query_parameter_limit_zero/schema.sql +++ b/internal/endtoend/testdata/query_parameter_limit_zero/schema.sql @@ -1,2 +1 @@ CREATE TABLE bar (id serial not null, name text not null, primary key (id)); - diff --git a/internal/endtoend/testdata/query_parameter_limit_zero/sqlc.yaml b/internal/endtoend/testdata/query_parameter_limit_zero/sqlc.yaml index 6e2cdeb..88e985a 100644 --- a/internal/endtoend/testdata/query_parameter_limit_zero/sqlc.yaml +++ b/internal/endtoend/testdata/query_parameter_limit_zero/sqlc.yaml @@ -3,7 +3,7 @@ plugins: - name: py wasm: url: file://../../../../bin/sqlc-gen-python.wasm - sha256: "d6846ffad948181e611e883cedd2d2be66e091edc1273a0abc6c9da18399e0ca" + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" sql: - schema: schema.sql queries: query.sql diff --git a/internal/endtoend/testdata/query_parameter_no_limit/schema.sql b/internal/endtoend/testdata/query_parameter_no_limit/schema.sql index fd3ca97..42f134f 100644 --- a/internal/endtoend/testdata/query_parameter_no_limit/schema.sql +++ b/internal/endtoend/testdata/query_parameter_no_limit/schema.sql @@ -1 +1 @@ -CREATE TABLE bar (id serial not null, name text not null, primary key (id)); \ No newline at end of file +CREATE TABLE bar (id serial not null, name text not null, primary key (id)); diff --git a/internal/endtoend/testdata/query_parameter_no_limit/sqlc.yaml b/internal/endtoend/testdata/query_parameter_no_limit/sqlc.yaml index c432e4f..d671866 100644 --- a/internal/endtoend/testdata/query_parameter_no_limit/sqlc.yaml +++ b/internal/endtoend/testdata/query_parameter_no_limit/sqlc.yaml @@ -3,7 +3,7 @@ plugins: - name: py wasm: url: file://../../../../bin/sqlc-gen-python.wasm - sha256: "d6846ffad948181e611e883cedd2d2be66e091edc1273a0abc6c9da18399e0ca" + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" sql: - schema: schema.sql queries: query.sql diff --git a/internal/endtoend/testdata/rename/db/models.py b/internal/endtoend/testdata/rename/db/models.py new file mode 100644 index 0000000..a9c2906 --- /dev/null +++ b/internal/endtoend/testdata/rename/db/models.py @@ -0,0 +1,25 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses +import enum +from typing import Optional + + +class MusicGenre(str, enum.Enum): + ROCK = "rock" + HIP_HOP_MUSIC = "hip-hop" + + +@dataclasses.dataclass() +class Artist: + id: int + spotify_link: Optional[str] + kind: MusicGenre + + +@dataclasses.dataclass() +class Human: + id: int + name: str + category: str diff --git a/internal/endtoend/testdata/rename/db/query.py b/internal/endtoend/testdata/rename/db/query.py new file mode 100644 index 0000000..da28f2a --- /dev/null +++ b/internal/endtoend/testdata/rename/db/query.py @@ -0,0 +1,67 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import dataclasses +from typing import Iterator, Optional, Union, cast + +import sqlalchemy +import sqlalchemy.orm + +from db import models + + +GET_ARTIST = """-- name: get_artist \\:one +SELECT id, spotify_url, kind FROM artists WHERE spotify_url = :p1 +""" + + +GET_PERSON = """-- name: get_person \\:one +SELECT id, name, class FROM people WHERE id = :p1 +""" + + +LIST_ARTIST_LINKS = """-- name: list_artist_links \\:many +SELECT id, spotify_url FROM artists WHERE kind = :p1 +""" + + +@dataclasses.dataclass() +class ListArtistLinksRow: + id: int + spotify_link: Optional[str] + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def get_artist(self, *, spotify_link: Optional[str]) -> Optional[models.Artist]: + row = self._conn.execute(sqlalchemy.text(GET_ARTIST), {"p1": spotify_link}).first() + if row is None: + return None + return models.Artist( + id=cast(int, row[0]), + spotify_link=cast(Optional[str], row[1]), + kind=cast(models.MusicGenre, row[2]), + ) + + def get_person(self, *, id: int) -> Optional[models.Human]: + row = self._conn.execute(sqlalchemy.text(GET_PERSON), {"p1": id}).first() + if row is None: + return None + return models.Human( + id=cast(int, row[0]), + name=cast(str, row[1]), + category=cast(str, row[2]), + ) + + def list_artist_links(self, *, kind: models.MusicGenre) -> Iterator[ListArtistLinksRow]: + result = self._conn.execute(sqlalchemy.text(LIST_ARTIST_LINKS), {"p1": kind}) + for row in result: + yield ListArtistLinksRow( + id=cast(int, row[0]), + spotify_link=cast(Optional[str], row[1]), + ) diff --git a/internal/endtoend/testdata/rename/query.sql b/internal/endtoend/testdata/rename/query.sql new file mode 100644 index 0000000..f2498fc --- /dev/null +++ b/internal/endtoend/testdata/rename/query.sql @@ -0,0 +1,8 @@ +-- name: GetPerson :one +SELECT * FROM people WHERE id = $1; + +-- name: GetArtist :one +SELECT * FROM artists WHERE spotify_url = $1; + +-- name: ListArtistLinks :many +SELECT id, spotify_url FROM artists WHERE kind = $1; diff --git a/internal/endtoend/testdata/rename/schema.sql b/internal/endtoend/testdata/rename/schema.sql new file mode 100644 index 0000000..669d451 --- /dev/null +++ b/internal/endtoend/testdata/rename/schema.sql @@ -0,0 +1,13 @@ +CREATE TYPE genre AS ENUM ('rock', 'hip-hop'); + +CREATE TABLE people ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL, + class text NOT NULL +); + +CREATE TABLE artists ( + id BIGSERIAL PRIMARY KEY, + spotify_url text, + kind genre NOT NULL +); diff --git a/internal/endtoend/testdata/rename/sqlc.yaml b/internal/endtoend/testdata/rename/sqlc.yaml new file mode 100644 index 0000000..7ef14bf --- /dev/null +++ b/internal/endtoend/testdata/rename/sqlc.yaml @@ -0,0 +1,22 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: db + options: + package: db + emit_sync_querier: true + rename: + spotify_url: spotify_link + person: Human + genre: MusicGenre + hip-hop: HIP_HOP_MUSIC + class: category diff --git a/internal/endtoend/testdata/sqlc_embed/dc/models.py b/internal/endtoend/testdata/sqlc_embed/dc/models.py new file mode 100644 index 0000000..6208f71 --- /dev/null +++ b/internal/endtoend/testdata/sqlc_embed/dc/models.py @@ -0,0 +1,19 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import dataclasses +from typing import Optional + + +@dataclasses.dataclass() +class Author: + id: int + name: str + + +@dataclasses.dataclass() +class Book: + id: int + author_id: int + title: str + isbn: Optional[str] diff --git a/internal/endtoend/testdata/sqlc_embed/dc/query.py b/internal/endtoend/testdata/sqlc_embed/dc/query.py new file mode 100644 index 0000000..e24b26f --- /dev/null +++ b/internal/endtoend/testdata/sqlc_embed/dc/query.py @@ -0,0 +1,176 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import dataclasses +from typing import AsyncIterator, Iterator, Optional, Union, cast + +import sqlalchemy +import sqlalchemy.ext.asyncio +import sqlalchemy.orm + +from dc import models + + +GET_AUTHOR_ONLY = """-- name: get_author_only \\:one +SELECT authors.id, authors.name FROM authors WHERE id = :p1 +""" + + +@dataclasses.dataclass() +class GetAuthorOnlyRow: + authors: models.Author + + +GET_BOOK_WITH_AUTHOR = """-- name: get_book_with_author \\:one +SELECT authors.id, authors.name, books.title +FROM authors JOIN books ON books.author_id = authors.id +WHERE books.id = :p1 +""" + + +@dataclasses.dataclass() +class GetBookWithAuthorRow: + authors: models.Author + title: str + + +LIST_AUTHORS_AND_BOOKS = """-- name: list_authors_and_books \\:many +SELECT authors.id, authors.name, books.id, books.author_id, books.title, books.isbn +FROM authors JOIN books ON books.author_id = authors.id +""" + + +@dataclasses.dataclass() +class ListAuthorsAndBooksRow: + authors: models.Author + books: models.Book + + +LIST_TITLES_WITH_AUTHOR = """-- name: list_titles_with_author \\:many +SELECT books.id, authors.id, authors.name, books.title +FROM books JOIN authors ON books.author_id = authors.id +""" + + +@dataclasses.dataclass() +class ListTitlesWithAuthorRow: + id: int + authors: models.Author + title: str + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def get_author_only(self, *, id: int) -> Optional[GetAuthorOnlyRow]: + row = self._conn.execute(sqlalchemy.text(GET_AUTHOR_ONLY), {"p1": id}).first() + if row is None: + return None + return GetAuthorOnlyRow( + authors=models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + ), + ) + + def get_book_with_author(self, *, id: int) -> Optional[GetBookWithAuthorRow]: + row = self._conn.execute(sqlalchemy.text(GET_BOOK_WITH_AUTHOR), {"p1": id}).first() + if row is None: + return None + return GetBookWithAuthorRow( + authors=models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + ), + title=cast(str, row[2]), + ) + + def list_authors_and_books(self) -> Iterator[ListAuthorsAndBooksRow]: + result = self._conn.execute(sqlalchemy.text(LIST_AUTHORS_AND_BOOKS)) + for row in result: + yield ListAuthorsAndBooksRow( + authors=models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + ), + books=models.Book( + id=cast(int, row[2]), + author_id=cast(int, row[3]), + title=cast(str, row[4]), + isbn=cast(Optional[str], row[5]), + ), + ) + + def list_titles_with_author(self) -> Iterator[ListTitlesWithAuthorRow]: + result = self._conn.execute(sqlalchemy.text(LIST_TITLES_WITH_AUTHOR)) + for row in result: + yield ListTitlesWithAuthorRow( + id=cast(int, row[0]), + authors=models.Author( + id=cast(int, row[1]), + name=cast(str, row[2]), + ), + title=cast(str, row[3]), + ) + + +class AsyncQuerier: + _conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession] + + def __init__(self, conn: Union[sqlalchemy.ext.asyncio.AsyncConnection, sqlalchemy.ext.asyncio.AsyncSession]): + self._conn = conn + + async def get_author_only(self, *, id: int) -> Optional[GetAuthorOnlyRow]: + row = (await self._conn.execute(sqlalchemy.text(GET_AUTHOR_ONLY), {"p1": id})).first() + if row is None: + return None + return GetAuthorOnlyRow( + authors=models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + ), + ) + + async def get_book_with_author(self, *, id: int) -> Optional[GetBookWithAuthorRow]: + row = (await self._conn.execute(sqlalchemy.text(GET_BOOK_WITH_AUTHOR), {"p1": id})).first() + if row is None: + return None + return GetBookWithAuthorRow( + authors=models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + ), + title=cast(str, row[2]), + ) + + async def list_authors_and_books(self) -> AsyncIterator[ListAuthorsAndBooksRow]: + result = await self._conn.stream(sqlalchemy.text(LIST_AUTHORS_AND_BOOKS)) + async for row in result: + yield ListAuthorsAndBooksRow( + authors=models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + ), + books=models.Book( + id=cast(int, row[2]), + author_id=cast(int, row[3]), + title=cast(str, row[4]), + isbn=cast(Optional[str], row[5]), + ), + ) + + async def list_titles_with_author(self) -> AsyncIterator[ListTitlesWithAuthorRow]: + result = await self._conn.stream(sqlalchemy.text(LIST_TITLES_WITH_AUTHOR)) + async for row in result: + yield ListTitlesWithAuthorRow( + id=cast(int, row[0]), + authors=models.Author( + id=cast(int, row[1]), + name=cast(str, row[2]), + ), + title=cast(str, row[3]), + ) diff --git a/internal/endtoend/testdata/sqlc_embed/pyd/models.py b/internal/endtoend/testdata/sqlc_embed/pyd/models.py new file mode 100644 index 0000000..eb188bc --- /dev/null +++ b/internal/endtoend/testdata/sqlc_embed/pyd/models.py @@ -0,0 +1,17 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +import pydantic +from typing import Optional + + +class Author(pydantic.BaseModel): + id: int + name: str + + +class Book(pydantic.BaseModel): + id: int + author_id: int + title: str + isbn: Optional[str] diff --git a/internal/endtoend/testdata/sqlc_embed/pyd/query.py b/internal/endtoend/testdata/sqlc_embed/pyd/query.py new file mode 100644 index 0000000..436bdc8 --- /dev/null +++ b/internal/endtoend/testdata/sqlc_embed/pyd/query.py @@ -0,0 +1,113 @@ +# Code generated by sqlc. DO NOT EDIT. +# versions: +# sqlc v1.31.1 +# source: query.sql +import pydantic +from typing import Iterator, Optional, Union, cast + +import sqlalchemy +import sqlalchemy.orm + +from pyd import models + + +GET_AUTHOR_ONLY = """-- name: get_author_only \\:one +SELECT authors.id, authors.name FROM authors WHERE id = :p1 +""" + + +class GetAuthorOnlyRow(pydantic.BaseModel): + authors: models.Author + + +GET_BOOK_WITH_AUTHOR = """-- name: get_book_with_author \\:one +SELECT authors.id, authors.name, books.title +FROM authors JOIN books ON books.author_id = authors.id +WHERE books.id = :p1 +""" + + +class GetBookWithAuthorRow(pydantic.BaseModel): + authors: models.Author + title: str + + +LIST_AUTHORS_AND_BOOKS = """-- name: list_authors_and_books \\:many +SELECT authors.id, authors.name, books.id, books.author_id, books.title, books.isbn +FROM authors JOIN books ON books.author_id = authors.id +""" + + +class ListAuthorsAndBooksRow(pydantic.BaseModel): + authors: models.Author + books: models.Book + + +LIST_TITLES_WITH_AUTHOR = """-- name: list_titles_with_author \\:many +SELECT books.id, authors.id, authors.name, books.title +FROM books JOIN authors ON books.author_id = authors.id +""" + + +class ListTitlesWithAuthorRow(pydantic.BaseModel): + id: int + authors: models.Author + title: str + + +class Querier: + _conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session] + + def __init__(self, conn: Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]): + self._conn = conn + + def get_author_only(self, *, id: int) -> Optional[GetAuthorOnlyRow]: + row = self._conn.execute(sqlalchemy.text(GET_AUTHOR_ONLY), {"p1": id}).first() + if row is None: + return None + return GetAuthorOnlyRow( + authors=models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + ), + ) + + def get_book_with_author(self, *, id: int) -> Optional[GetBookWithAuthorRow]: + row = self._conn.execute(sqlalchemy.text(GET_BOOK_WITH_AUTHOR), {"p1": id}).first() + if row is None: + return None + return GetBookWithAuthorRow( + authors=models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + ), + title=cast(str, row[2]), + ) + + def list_authors_and_books(self) -> Iterator[ListAuthorsAndBooksRow]: + result = self._conn.execute(sqlalchemy.text(LIST_AUTHORS_AND_BOOKS)) + for row in result: + yield ListAuthorsAndBooksRow( + authors=models.Author( + id=cast(int, row[0]), + name=cast(str, row[1]), + ), + books=models.Book( + id=cast(int, row[2]), + author_id=cast(int, row[3]), + title=cast(str, row[4]), + isbn=cast(Optional[str], row[5]), + ), + ) + + def list_titles_with_author(self) -> Iterator[ListTitlesWithAuthorRow]: + result = self._conn.execute(sqlalchemy.text(LIST_TITLES_WITH_AUTHOR)) + for row in result: + yield ListTitlesWithAuthorRow( + id=cast(int, row[0]), + authors=models.Author( + id=cast(int, row[1]), + name=cast(str, row[2]), + ), + title=cast(str, row[3]), + ) diff --git a/internal/endtoend/testdata/sqlc_embed/query.sql b/internal/endtoend/testdata/sqlc_embed/query.sql new file mode 100644 index 0000000..5808dfe --- /dev/null +++ b/internal/endtoend/testdata/sqlc_embed/query.sql @@ -0,0 +1,15 @@ +-- name: GetBookWithAuthor :one +SELECT sqlc.embed(authors), books.title +FROM authors JOIN books ON books.author_id = authors.id +WHERE books.id = $1; + +-- name: ListAuthorsAndBooks :many +SELECT sqlc.embed(authors), sqlc.embed(books) +FROM authors JOIN books ON books.author_id = authors.id; + +-- name: GetAuthorOnly :one +SELECT sqlc.embed(authors) FROM authors WHERE id = $1; + +-- name: ListTitlesWithAuthor :many +SELECT books.id, sqlc.embed(authors), books.title +FROM books JOIN authors ON books.author_id = authors.id; diff --git a/internal/endtoend/testdata/sqlc_embed/schema.sql b/internal/endtoend/testdata/sqlc_embed/schema.sql new file mode 100644 index 0000000..e04db15 --- /dev/null +++ b/internal/endtoend/testdata/sqlc_embed/schema.sql @@ -0,0 +1,11 @@ +CREATE TABLE authors ( + id BIGSERIAL PRIMARY KEY, + name text NOT NULL +); + +CREATE TABLE books ( + id BIGSERIAL PRIMARY KEY, + author_id bigint NOT NULL REFERENCES authors (id), + title text NOT NULL, + isbn text +); diff --git a/internal/endtoend/testdata/sqlc_embed/sqlc.yaml b/internal/endtoend/testdata/sqlc_embed/sqlc.yaml new file mode 100644 index 0000000..fa651a9 --- /dev/null +++ b/internal/endtoend/testdata/sqlc_embed/sqlc.yaml @@ -0,0 +1,23 @@ +version: "2" +plugins: + - name: py + wasm: + url: file://../../../../bin/sqlc-gen-python.wasm + sha256: "052fc3c2963971e1945aa5dcf5ebbceae2e84b64f9207128220fb859aa66afaf" +sql: + - schema: schema.sql + queries: query.sql + engine: postgresql + codegen: + - plugin: py + out: dc + options: + package: dc + emit_sync_querier: true + emit_async_querier: true + - plugin: py + out: pyd + options: + package: pyd + emit_sync_querier: true + emit_pydantic_models: true diff --git a/internal/errors_module.go b/internal/errors_module.go new file mode 100644 index 0000000..3e46b5c --- /dev/null +++ b/internal/errors_module.go @@ -0,0 +1,135 @@ +package python + +import ( + "strings" + + pyast "github.com/sqlc-dev/sqlc-gen-python/internal/ast" + "github.com/sqlc-dev/sqlc-gen-python/internal/poet" + pyprint "github.com/sqlc-dev/sqlc-gen-python/internal/printer" +) + +const errorsFileName = "errors.py" + +// The class names, constructor and query_name/cause attributes match +// alt-sqlc-gen-python's emit_query_errors so code written against it keeps +// working. {opt_exc}, {opt_str} and {iter_none} are filled in for the active +// type syntax. +const errorsModuleBody = `class QueryError(Exception): + """Base class for all query errors.""" + + def __init__(self, message: str, query_name: str, cause: {opt_exc} = None): + super().__init__(message) + self.query_name = query_name + self.cause = cause + self.constraint_name: {opt_str} = _constraint_name(cause) + + +class UniqueViolationError(QueryError): + """Raised on unique constraint violation (PG 23505).""" + + +class ForeignKeyViolationError(QueryError): + """Raised on foreign key constraint violation (PG 23503).""" + + +class CheckViolationError(QueryError): + """Raised on check constraint violation (PG 23514).""" + + +class NotNullViolationError(QueryError): + """Raised on not-null constraint violation (PG 23502).""" + + +class ExclusionViolationError(QueryError): + """Raised on exclusion constraint violation (PG 23P01).""" + + +class StatementTimeoutError(QueryError): + """Raised on statement timeout (PG 57014).""" + + +class DeadlockError(QueryError): + """Raised on deadlock detected (PG 40P01).""" + + +class SerializationError(QueryError): + """Raised on serialization failure (PG 40001).""" + + +def _sqlstate(e: sqlalchemy.exc.DBAPIError) -> {opt_str}: + # psycopg2 exposes pgcode, psycopg 3 sqlstate; SQLAlchemy's asyncpg + # adapter keeps the asyncpg error, which has sqlstate, as __cause__. + for source in (e.orig, getattr(e.orig, "__cause__", None)): + code = getattr(source, "pgcode", None) or getattr(source, "sqlstate", None) + if isinstance(code, str): + return code + return None + + +def _constraint_name(cause: {opt_exc}) -> {opt_str}: + orig = getattr(cause, "orig", None) + name = getattr(getattr(orig, "diag", None), "constraint_name", None) + if name is None: + name = getattr(getattr(orig, "__cause__", None), "constraint_name", None) + return name if isinstance(name, str) else None + + +def _wrap_integrity_error(e: sqlalchemy.exc.IntegrityError, query_name: str) -> QueryError: + pgcode = _sqlstate(e) + if pgcode == "23505": + return UniqueViolationError(str(e), query_name, cause=e) + if pgcode == "23503": + return ForeignKeyViolationError(str(e), query_name, cause=e) + if pgcode == "23514": + return CheckViolationError(str(e), query_name, cause=e) + if pgcode == "23502": + return NotNullViolationError(str(e), query_name, cause=e) + if pgcode == "23P01": + return ExclusionViolationError(str(e), query_name, cause=e) + return QueryError(str(e), query_name, cause=e) + + +def _wrap_operational_error(e: sqlalchemy.exc.OperationalError, query_name: str) -> QueryError: + pgcode = _sqlstate(e) + if pgcode == "57014": + return StatementTimeoutError(str(e), query_name, cause=e) + if pgcode == "40P01": + return DeadlockError(str(e), query_name, cause=e) + if pgcode == "40001": + return SerializationError(str(e), query_name, cause=e) + return QueryError(str(e), query_name, cause=e) + + +@contextlib.contextmanager +def _wrap_errors(query_name: str) -> {iter_none}: + """Re-raise integrity and operational errors as typed QueryErrors. + + Only those two SQLAlchemy exceptions are caught, so GeneratorExit from a + consumer that stops iterating early passes through untouched. + """ + try: + yield + except sqlalchemy.exc.IntegrityError as e: + raise _wrap_integrity_error(e, query_name) from e + except sqlalchemy.exc.OperationalError as e: + raise _wrap_operational_error(e, query_name) from e +` + +func buildErrorsModule(ctx *pyTmplCtx) string { + f := newPyFile(ctx.C, false) + f.importModule("contextlib") + f.importModule("sqlalchemy.exc") + render := func(n *pyast.Node) string { + return strings.TrimSpace(string(pyprint.Print(n, pyprint.Options{}).Python)) + } + body := strings.NewReplacer( + "{opt_exc}", render(f.optional(poet.Name("Exception"))), + "{opt_str}", render(f.optional(poet.Name("str"))), + "{iter_none}", render(f.generic("Iterator", poet.Constant(nil))), + ).Replace(errorsModuleBody) + + mod := moduleNode(ctx.SqlcVersion, "", ctx.C.OmitSqlcVersion) + mod.Body = append(mod.Body, importGroup(f.std), importGroup(f.pkg)) + header := string(pyprint.Print(poet.Node(mod), pyprint.Options{}).Python) + return header + "\n" + body +} diff --git a/internal/gen.go b/internal/gen.go index 6e50fae..62831eb 100644 --- a/internal/gen.go +++ b/internal/gen.go @@ -2,7 +2,6 @@ package python import ( "context" - json "encoding/json" "errors" "fmt" "log" @@ -13,6 +12,7 @@ import ( "github.com/sqlc-dev/plugin-sdk-go/metadata" "github.com/sqlc-dev/plugin-sdk-go/plugin" "github.com/sqlc-dev/plugin-sdk-go/sdk" + "golang.org/x/text/unicode/norm" pyast "github.com/sqlc-dev/sqlc-gen-python/internal/ast" "github.com/sqlc-dev/sqlc-gen-python/internal/inflection" @@ -33,30 +33,25 @@ type Enum struct { } type pyType struct { + // InnerType is a dotted name such as "datetime.datetime" or + // "models.Status", or a bare name imported from Import. InnerType string - IsArray bool + Import string + ArrayDims int IsNull bool } -func (t pyType) Annotation() *pyast.Node { - ann := poet.Name(t.InnerType) - if t.IsArray { - ann = subscriptNode("List", ann) - } - if t.IsNull { - ann = subscriptNode("Optional", ann) - } - return ann -} - type Field struct { Name string Type pyType Comment string + // Embed is the model built from this field's run of row columns when + // the query used sqlc.embed(). + Embed *Struct } type Struct struct { - Table plugin.Identifier + Table *plugin.Identifier Name string Fields []Field Comment string @@ -69,20 +64,6 @@ type QueryValue struct { Typ pyType } -func (v QueryValue) Annotation() *pyast.Node { - if v.Typ != (pyType{}) { - return v.Typ.Annotation() - } - if v.Struct != nil { - if v.Emit { - return poet.Name(v.Struct.Name) - } else { - return typeRefNode("models", v.Struct.Name) - } - } - panic("no type for QueryValue: " + v.Name) -} - func (v QueryValue) EmitStruct() bool { return v.Emit } @@ -95,30 +76,17 @@ func (v QueryValue) isEmpty() bool { return v.Typ == (pyType{}) && v.Name == "" && v.Struct == nil } -func (v QueryValue) RowNode(rowVar string) *pyast.Node { - if !v.IsStruct() { - return subscriptNode( - rowVar, - constantInt(0), - ) - } - call := &pyast.Call{ - Func: v.Annotation(), - } - for i, f := range v.Struct.Fields { - call.Keywords = append(call.Keywords, &pyast.Keyword{ - Arg: f.Name, - Value: subscriptNode( - rowVar, - constantInt(i), - ), - }) +func (v QueryValue) annotation(f *pyFile) *pyast.Node { + if v.Typ != (pyType{}) { + return f.annotation(v.Typ) } - return &pyast.Node{ - Node: &pyast.Node_Call{ - Call: call, - }, + if v.Struct != nil { + if v.Emit { + return poet.Name(v.Struct.Name) + } + return typeRefNode("models", v.Struct.Name) } + panic("no type for QueryValue: " + v.Name) } // A struct used to generate methods and fields on the Queries struct @@ -126,7 +94,6 @@ type Query struct { Cmd string Comments []string MethodName string - FieldName string ConstantName string SQL string SourceName string @@ -134,24 +101,41 @@ type Query struct { Args []QueryValue } -func (q Query) AddArgs(args *pyast.Arguments) { +// takesSequence reports whether the command's method takes a sequence of +// params objects instead of a single set of parameters. +func takesSequence(cmd string) bool { + switch cmd { + case metadata.CmdCopyFrom, metadata.CmdBatchExec, metadata.CmdBatchOne, metadata.CmdBatchMany: + return true + } + return false +} + +func (q Query) addArgs(f *pyFile, args *pyast.Arguments) { // A single struct arg does not need to be passed as a keyword argument if len(q.Args) == 1 && q.Args[0].IsStruct() { + ann := q.Args[0].annotation(f) + if takesSequence(q.Cmd) { + ann = f.generic("Sequence", ann) + } args.Args = append(args.Args, &pyast.Arg{ Arg: q.Args[0].Name, - Annotation: q.Args[0].Annotation(), + Annotation: ann, }) return } for _, a := range q.Args { args.KwOnlyArgs = append(args.KwOnlyArgs, &pyast.Arg{ Arg: a.Name, - Annotation: a.Annotation(), + Annotation: a.annotation(f), }) } } -func (q Query) ArgDictNode() *pyast.Node { +// argDictNode builds the SQLAlchemy parameter dict. base, when set, replaces +// the struct argument's name as the attribute base (the loop variable of +// sequence-taking commands). +func (q Query) argDictNode(base string) *pyast.Node { dict := &pyast.Dict{} i := 1 for _, a := range q.Args { @@ -159,9 +143,13 @@ func (q Query) ArgDictNode() *pyast.Node { continue } if a.IsStruct() { + name := a.Name + if base != "" { + name = base + } for _, f := range a.Struct.Fields { dict.Keys = append(dict.Keys, poet.Constant(fmt.Sprintf("p%v", i))) - dict.Values = append(dict.Values, typeRefNode(a.Name, f.Name)) + dict.Values = append(dict.Values, typeRefNode(name, f.Name)) i++ } } else { @@ -180,25 +168,38 @@ func (q Query) ArgDictNode() *pyast.Node { } } -func makePyType(req *plugin.GenerateRequest, col *plugin.Column) pyType { - typ := pyInnerType(req, col) - return pyType{ - InnerType: typ, - IsArray: col.IsArray, +func makePyType(req *plugin.GenerateRequest, conf Config, col *plugin.Column) pyType { + dims := int(col.ArrayDims) + if col.IsArray && dims == 0 { + dims = 1 + } + typ := pyType{ + ArrayDims: dims, IsNull: !col.NotNull, } + if o := conf.findOverride(col, req.Catalog.DefaultSchema); o != nil { + typ.InnerType = o.PyType + typ.Import = o.PyImport + return typ + } + typ.InnerType = pyInnerType(req, conf, col) + return typ } -func pyInnerType(req *plugin.GenerateRequest, col *plugin.Column) string { +func pyInnerType(req *plugin.GenerateRequest, conf Config, col *plugin.Column) string { switch req.Settings.Engine { case "postgresql": - return postgresType(req, col) + return postgresType(req, conf, col) default: log.Println("unsupported engine type") return "Any" } } +func className(name string) string { + return modelName(name, nil) +} + func modelName(name string, settings *plugin.Settings) string { out := "" for _, p := range strings.Split(name, "_") { @@ -226,7 +227,38 @@ func pyEnumValueName(value string) string { return strings.ToUpper(id) } -func buildEnums(req *plugin.GenerateRequest) []Enum { +// enumMemberNames makes every member a unique, valid identifier while the +// values keep their original strings. +func enumMemberNames(conf Config, vals []string) []string { + names := make([]string, len(vals)) + used := map[string]bool{} + for i, v := range vals { + name := pyIdent(v, conf, pyEnumValueName) + if name == "" { + name = fmt.Sprintf("VALUE_%d", i+1) + } else if name[0] >= '0' && name[0] <= '9' { + name = "VALUE_" + name + } + names[i] = uniqueName(used, name) + } + return names +} + +// uniqueName returns name, or name with the lowest free "_N" suffix, and +// marks the result used. +func uniqueName(used map[string]bool, name string) string { + if used[name] { + k := 2 + for used[fmt.Sprintf("%s_%d", name, k)] { + k++ + } + name = fmt.Sprintf("%s_%d", name, k) + } + used[name] = true + return name +} + +func buildEnums(conf Config, req *plugin.GenerateRequest) []Enum { var enums []Enum for _, schema := range req.Catalog.Schemas { if schema.Name == "pg_catalog" || schema.Name == "information_schema" { @@ -240,13 +272,13 @@ func buildEnums(req *plugin.GenerateRequest) []Enum { enumName = schema.Name + "_" + enum.Name } e := Enum{ - Name: modelName(enumName, req.Settings), + Name: pyIdent(enumName, conf, className), Comment: enum.Comment, } - for _, v := range enum.Vals { + for i, name := range enumMemberNames(conf, enum.Vals) { e.Constants = append(e.Constants, Constant{ - Name: pyEnumValueName(v), - Value: v, + Name: name, + Value: enum.Vals[i], Type: e.Name, }) } @@ -280,16 +312,14 @@ func buildModels(conf Config, req *plugin.GenerateRequest) []Struct { }) } s := Struct{ - Table: plugin.Identifier{Schema: schema.Name, Name: table.Rel.Name}, - Name: modelName(structName, req.Settings), + Table: &plugin.Identifier{Schema: schema.Name, Name: table.Rel.Name}, + Name: pyIdent(structName, conf, className), Comment: table.Comment, } for _, column := range table.Columns { - typ := makePyType(req, column) // TODO: This used to call compiler.ConvertColumn? - typ.InnerType = strings.TrimPrefix(typ.InnerType, "models.") s.Fields = append(s.Fields, Field{ - Name: column.Name, - Type: typ, + Name: pyIdent(column.Name, conf, nil), + Type: makePyType(req, conf, column), Comment: column.Comment, }) } @@ -321,32 +351,26 @@ type pyColumn struct { *plugin.Column } -func columnsToStruct(req *plugin.GenerateRequest, name string, columns []pyColumn) *Struct { +func columnsToStruct(req *plugin.GenerateRequest, conf Config, name string, columns []pyColumn, models []Struct) *Struct { gs := Struct{ Name: name, } - seen := map[string]int32{} - suffixes := map[int32]int32{} + used := map[string]bool{} for i, c := range columns { - colName := columnName(c.Column, i) - fieldName := colName - // Track suffixes by the ID of the column, so that columns referring to - // the same numbered parameter can be reused. - var suffix int32 - if o, ok := suffixes[c.id]; ok { - suffix = o - } else if v := seen[colName]; v > 0 { - suffix = v + 1 + field := Field{ + Name: uniqueName(used, pyIdent(columnName(c.Column, i), conf, nil)), + Type: makePyType(req, conf, c.Column), } - suffixes[c.id] = suffix - if suffix > 0 { - fieldName = fmt.Sprintf("%s_%d", fieldName, suffix) + if c.EmbedTable != nil { + for j := range models { + if sdk.SameTableName(c.EmbedTable, models[j].Table, req.Catalog.DefaultSchema) { + field.Embed = &models[j] + field.Type = pyType{InnerType: "models." + models[j].Name} + break + } + } } - gs.Fields = append(gs.Fields, Field{ - Name: fieldName, - Type: makePyType(req, c.Column), - }) - seen[colName]++ + gs.Fields = append(gs.Fields, field) } return &gs } @@ -356,15 +380,26 @@ var postgresPlaceholderRegexp = regexp.MustCompile(`\B\$(\d+)\b`) // Sqlalchemy uses ":name" for placeholders, so "$N" is converted to ":pN" // This also means ":" has special meaning to sqlalchemy, so it must be escaped. func sqlalchemySQL(s, engine string) string { - s = strings.ReplaceAll(s, ":", `\\:`) + s = strings.ReplaceAll(s, ":", `\:`) if engine == "postgresql" { return postgresPlaceholderRegexp.ReplaceAllString(s, ":p$1") } return s } +func queryComments(comments []string) []string { + out := make([]string, len(comments)) + for i, c := range comments { + out[i] = strings.TrimPrefix(c, " ") + } + return out +} + func buildQueries(conf Config, req *plugin.GenerateRequest, structs []Struct) ([]Query, error) { qs := make([]Query, 0, len(req.Queries)) + moduleNames := conf.moduleNames() + methodOwners := map[string]string{} + constantOwners := map[string]string{} for _, query := range req.Queries { if query.Name == "" { continue @@ -372,18 +407,36 @@ func buildQueries(conf Config, req *plugin.GenerateRequest, structs []Struct) ([ if query.Cmd == "" { continue } - if query.Cmd == metadata.CmdCopyFrom { - return nil, errors.New("Support for CopyFrom in Python is not implemented") + switch query.Cmd { + case metadata.CmdOne, metadata.CmdMany, metadata.CmdExec, metadata.CmdExecRows, metadata.CmdExecResult, + metadata.CmdCopyFrom, metadata.CmdBatchExec, metadata.CmdBatchOne, metadata.CmdBatchMany: + default: + return nil, fmt.Errorf("query %s: unsupported command %s", query.Name, query.Cmd) } methodName := methodName(query.Name) + defName := escapeKeyword(methodName) + if _, ok := moduleNames[defName]; ok { + defName += "_" + } + // Python NFKC-normalises identifiers; each query file is its own module. + methodKey := query.Filename + "\x00" + norm.NFKC.String(defName) + if other, ok := methodOwners[methodKey]; ok { + return nil, fmt.Errorf("queries %s and %s both generate method %s", other, query.Name, defName) + } + methodOwners[methodKey] = query.Name + constantName := strings.ToUpper(methodName) + constantKey := query.Filename + "\x00" + norm.NFKC.String(constantName) + if other, ok := constantOwners[constantKey]; ok { + return nil, fmt.Errorf("queries %s and %s both generate constant %s", other, query.Name, constantName) + } + constantOwners[constantKey] = query.Name gq := Query{ Cmd: query.Cmd, - Comments: query.Comments, - MethodName: methodName, - FieldName: sdk.LowerTitle(query.Name) + "Stmt", - ConstantName: strings.ToUpper(methodName), + Comments: queryComments(query.Comments), + MethodName: defName, + ConstantName: constantName, SQL: sqlalchemySQL(query.Text, req.Settings.Engine), SourceName: query.Filename, } @@ -395,7 +448,7 @@ func buildQueries(conf Config, req *plugin.GenerateRequest, structs []Struct) ([ if qpl < 0 { return nil, errors.New("invalid query parameter limit") } - if len(query.Params) > qpl || qpl == 0 { + if len(query.Params) > qpl || qpl == 0 || takesSequence(query.Cmd) { var cols []pyColumn for _, p := range query.Params { cols = append(cols, pyColumn{ @@ -406,48 +459,59 @@ func buildQueries(conf Config, req *plugin.GenerateRequest, structs []Struct) ([ gq.Args = []QueryValue{{ Emit: true, Name: "arg", - Struct: columnsToStruct(req, query.Name+"Params", cols), + Struct: columnsToStruct(req, conf, query.Name+"Params", cols, structs), }} } else { args := make([]QueryValue, 0, len(query.Params)) + used := map[string]bool{} for _, p := range query.Params { + name := pyIdent(paramName(p), conf, nil) + if _, ok := moduleNames[name]; ok { + name += "_" + } args = append(args, QueryValue{ - Name: paramName(p), - Typ: makePyType(req, p.Column), + Name: uniqueName(used, name), + Typ: makePyType(req, conf, p.Column), }) } gq.Args = args } - if len(query.Columns) == 1 { + hasEmbed := false + for _, c := range query.Columns { + if c.EmbedTable != nil { + hasEmbed = true + } + } + + if len(query.Columns) == 1 && !hasEmbed { c := query.Columns[0] gq.Ret = QueryValue{ Name: columnName(c, 0), - Typ: makePyType(req, c), + Typ: makePyType(req, conf, c), } - } else if len(query.Columns) > 1 { + } else if len(query.Columns) > 0 { var gs *Struct var emit bool + // A row with embeds never has a table's shape, so it is never + // a reused model. for _, s := range structs { - if len(s.Fields) != len(query.Columns) { + if hasEmbed || len(s.Fields) != len(query.Columns) { continue } same := true - for i, f := range s.Fields { c := query.Columns[i] - // HACK: models do not have "models." on their types, so trim that so we can find matches - trimmedPyType := makePyType(req, c) - trimmedPyType.InnerType = strings.TrimPrefix(trimmedPyType.InnerType, "models.") - sameName := f.Name == columnName(c, i) - sameType := f.Type == trimmedPyType - sameTable := sdk.SameTableName(c.Table, &s.Table, req.Catalog.DefaultSchema) + sameName := f.Name == pyIdent(columnName(c, i), conf, nil) + sameType := f.Type == makePyType(req, conf, c) + sameTable := sdk.SameTableName(c.Table, s.Table, req.Catalog.DefaultSchema) if !sameName || !sameType || !sameTable { same = false } } if same { + s := s gs = &s break } @@ -461,7 +525,7 @@ func buildQueries(conf Config, req *plugin.GenerateRequest, structs []Struct) ([ Column: c, }) } - gs = columnsToStruct(req, query.Name+"Row", columns) + gs = columnsToStruct(req, conf, query.Name+"Row", columns, structs) emit = true } gq.Ret = QueryValue{ @@ -477,20 +541,67 @@ func buildQueries(conf Config, req *plugin.GenerateRequest, structs []Struct) ([ return qs, nil } -func moduleNode(version, source string) *pyast.Module { +// filterUnusedStructs keeps the enums and models that a query's params, +// result or embedded tables refer to, directly or through a model field. +func filterUnusedStructs(enums []Enum, models []Struct, queries []Query) ([]Enum, []Struct) { + keep := map[string]bool{} + var visitStruct func(s *Struct) + visitStruct = func(s *Struct) { + for _, f := range s.Fields { + keep[strings.TrimPrefix(f.Type.InnerType, "models.")] = true + if f.Embed != nil { + keep[f.Embed.Name] = true + visitStruct(f.Embed) + } + } + } + visitValue := func(v QueryValue) { + if v.Struct == nil { + keep[strings.TrimPrefix(v.Typ.InnerType, "models.")] = true + return + } + if !v.Emit { + keep[v.Struct.Name] = true + } + visitStruct(v.Struct) + } + for _, q := range queries { + visitValue(q.Ret) + for _, a := range q.Args { + visitValue(a) + } + } + + var keptModels []Struct + for i := range models { + if keep[models[i].Name] { + keptModels = append(keptModels, models[i]) + visitStruct(&models[i]) + } + } + var keptEnums []Enum + for _, e := range enums { + if keep[e.Name] { + keptEnums = append(keptEnums, e) + } + } + return keptEnums, keptModels +} + +func moduleNode(version, source string, omitVersion bool) *pyast.Module { mod := &pyast.Module{ Body: []*pyast.Node{ poet.Comment( "Code generated by sqlc. DO NOT EDIT.", ), - poet.Comment( - "versions:", - ), - poet.Comment( - " sqlc " + version, - ), }, } + if !omitVersion { + mod.Body = append(mod.Body, + poet.Comment("versions:"), + poet.Comment(" sqlc "+version), + ) + } if source != "" { mod.Body = append(mod.Body, poet.Comment( @@ -519,17 +630,6 @@ func importNode(name string) *pyast.Node { } } -func classDefNode(name string, bases ...*pyast.Node) *pyast.Node { - return &pyast.Node{ - Node: &pyast.Node_ClassDef{ - ClassDef: &pyast.ClassDef{ - Name: name, - Bases: bases, - }, - }, - } -} - func assignNode(target string, value *pyast.Node) *pyast.Node { return &pyast.Node{ Node: &pyast.Node_Assign{ @@ -585,32 +685,36 @@ func pydanticNode(name string) *pyast.ClassDef { return &pyast.ClassDef{ Name: name, Bases: []*pyast.Node{ - { - Node: &pyast.Node_Attribute{ - Attribute: &pyast.Attribute{ - Value: &pyast.Node{ - Node: &pyast.Node_Name{ - Name: &pyast.Name{Id: "pydantic"}, - }, - }, - Attr: "BaseModel", - }, - }, - }, + poet.Attribute(poet.Name("pydantic"), "BaseModel"), }, } } -func fieldNode(f Field) *pyast.Node { - return &pyast.Node{ - Node: &pyast.Node_AnnAssign{ - AnnAssign: &pyast.AnnAssign{ - Target: &pyast.Name{Id: f.Name}, - Annotation: f.Type.Annotation(), - Comment: f.Comment, - }, - }, +func docstringNode(text string) *pyast.Node { + return poet.Expr(poet.Constant(text)) +} + +// structClassDef builds a model, params or row class. +func structClassDef(f *pyFile, conf Config, s *Struct) *pyast.Node { + var def *pyast.ClassDef + if conf.EmitPydanticModels { + f.importModule("pydantic") + def = pydanticNode(s.Name) + } else { + f.importModule("dataclasses") + def = dataclassNode(s.Name) + } + if s.Comment != "" { + def.Body = append(def.Body, docstringNode(s.Comment)) } + for _, field := range s.Fields { + def.Body = append(def.Body, poet.Node(&pyast.AnnAssign{ + Target: &pyast.Name{Id: field.Name}, + Annotation: f.annotation(field.Type), + Comment: field.Comment, + })) + } + return poet.Node(def) } func typeRefNode(base string, parts ...string) *pyast.Node { @@ -647,45 +751,24 @@ func connMethodNode(method, name string, arg *pyast.Node) *pyast.Node { } } -func buildImportGroup(specs map[string]importSpec) *pyast.Node { - var body []*pyast.Node - for _, spec := range buildImportBlock2(specs) { - if len(spec.Names) > 0 && spec.Names[0] != "" { - imp := &pyast.ImportFrom{ - Module: spec.Module, - } - for _, name := range spec.Names { - imp.Names = append(imp.Names, poet.Alias(name)) - } - body = append(body, &pyast.Node{ - Node: &pyast.Node_ImportFrom{ - ImportFrom: imp, - }, - }) - } else { - body = append(body, importNode(spec.Module)) - } +func buildModelsTree(ctx *pyTmplCtx) *pyast.Node { + f := newPyFile(ctx.C, true) + if ctx.C.EmitPydanticModels { + f.importModule("pydantic") + } else { + f.importModule("dataclasses") } - return &pyast.Node{ - Node: &pyast.Node_ImportGroup{ - ImportGroup: &pyast.ImportGroup{ - Imports: body, - }, - }, + if len(ctx.Enums) > 0 { + f.importModule("enum") } -} - -func buildModelsTree(ctx *pyTmplCtx, i *importer) *pyast.Node { - mod := moduleNode(ctx.SqlcVersion, "") - std, pkg := i.modelImportSpecs() - mod.Body = append(mod.Body, buildImportGroup(std), buildImportGroup(pkg)) + var body []*pyast.Node for _, e := range ctx.Enums { bases := []*pyast.Node{ poet.Name("str"), poet.Attribute(poet.Name("enum"), "Enum"), } - if i.C.EmitStrEnum { + if ctx.C.EmitStrEnum { // override the bases to emit enum.StrEnum (only support in Python >=3.11) bases = []*pyast.Node{ poet.Attribute(poet.Name("enum"), "StrEnum"), @@ -696,372 +779,393 @@ func buildModelsTree(ctx *pyTmplCtx, i *importer) *pyast.Node { Bases: bases, } if e.Comment != "" { - def.Body = append(def.Body, &pyast.Node{ - Node: &pyast.Node_Expr{ - Expr: &pyast.Expr{ - Value: poet.Constant(e.Comment), - }, - }, - }) + def.Body = append(def.Body, docstringNode(e.Comment)) } for _, c := range e.Constants { def.Body = append(def.Body, assignNode(c.Name, poet.Constant(c.Value))) } - mod.Body = append(mod.Body, &pyast.Node{ - Node: &pyast.Node_ClassDef{ - ClassDef: def, - }, + body = append(body, poet.Node(def)) + } + + for i := range ctx.Models { + body = append(body, structClassDef(f, ctx.C, &ctx.Models[i])) + } + + mod := moduleNode(ctx.SqlcVersion, "", ctx.C.OmitSqlcVersion) + mod.Body = append(mod.Body, importGroup(f.std), importGroup(f.pkg)) + mod.Body = append(mod.Body, body...) + return poet.Node(mod) +} + +// querierBuilder builds the methods of Querier (async false) or +// AsyncQuerier (async true), so both stay in step for every command. +type querierBuilder struct { + f *pyFile + conf Config + async bool +} + +type querierMethod struct { + name string + args *pyast.Arguments + body []*pyast.Node + returns *pyast.Node + asyncGen bool +} + +func (m querierMethod) node(async bool) *pyast.Node { + if async { + return poet.Node(&pyast.AsyncFunctionDef{ + Name: m.name, + Args: m.args, + Body: m.body, + Returns: m.returns, }) } + return poet.Node(&pyast.FunctionDef{ + Name: m.name, + Args: m.args, + Body: m.body, + Returns: m.returns, + }) +} - for _, m := range ctx.Models { - var def *pyast.ClassDef - if ctx.C.EmitPydanticModels { - def = pydanticNode(m.Name) +// protocolNode declares m with a `...` body. An async generator is declared +// with a plain def: its call type is AsyncIterator[T], whereas an async def +// stub would mean Coroutine[..., AsyncIterator[T]]. +func (m querierMethod) protocolNode(async bool) *pyast.Node { + stub := querierMethod{ + name: m.name, + args: m.args, + body: []*pyast.Node{poet.Expr(poet.Ellipsis())}, + returns: m.returns, + } + return stub.node(async && !m.asyncGen) +} + +func (b *querierBuilder) await(n *pyast.Node) *pyast.Node { + if b.async { + return poet.Await(n) + } + return n +} + +func (b *querierBuilder) execute(q Query, params *pyast.Node) *pyast.Node { + return b.await(connMethodNode("execute", q.ConstantName, params)) +} + +func (b *querierBuilder) iterator(n *pyast.Node) *pyast.Node { + if b.async { + return b.f.generic("AsyncIterator", n) + } + return b.f.generic("Iterator", n) +} + +func (b *querierBuilder) forNode(target string, iter *pyast.Node, body ...*pyast.Node) *pyast.Node { + if b.async { + return poet.Node(&pyast.AsyncFor{Target: poet.Name(target), Iter: iter, Body: body}) + } + return poet.Node(&pyast.For{Target: poet.Name(target), Iter: iter, Body: body}) +} + +// rowCell reads column i of the row, cast to its annotation so strict type +// checkers accept the untyped Row value. +func (b *querierBuilder) rowCell(t pyType, rowVar string, i int) *pyast.Node { + return poet.Node(&pyast.Call{ + Func: b.f.typing("cast"), + Args: []*pyast.Node{b.f.annotation(t), subscriptNode(rowVar, constantInt(i))}, + }) +} + +func (b *querierBuilder) structCall(s *Struct, callee *pyast.Node, rowVar string, idx *int) *pyast.Node { + call := &pyast.Call{Func: callee} + for _, field := range s.Fields { + var value *pyast.Node + if field.Embed != nil { + value = b.structCall(field.Embed, typeRefNode("models", field.Embed.Name), rowVar, idx) } else { - def = dataclassNode(m.Name) - } - if m.Comment != "" { - def.Body = append(def.Body, &pyast.Node{ - Node: &pyast.Node_Expr{ - Expr: &pyast.Expr{ - Value: poet.Constant(m.Comment), - }, - }, - }) - } - for _, f := range m.Fields { - def.Body = append(def.Body, fieldNode(f)) + value = b.rowCell(field.Type, rowVar, *idx) + *idx++ } - mod.Body = append(mod.Body, &pyast.Node{ - Node: &pyast.Node_ClassDef{ - ClassDef: def, - }, - }) + call.Keywords = append(call.Keywords, &pyast.Keyword{Arg: field.Name, Value: value}) } + return poet.Node(call) +} - return &pyast.Node{Node: &pyast.Node_Module{Module: mod}} +// rowNode builds the query's return value from the row in rowVar. +func (b *querierBuilder) rowNode(v QueryValue, rowVar string) *pyast.Node { + if !v.IsStruct() { + return b.rowCell(v.Typ, rowVar, 0) + } + idx := 0 + return b.structCall(v.Struct, v.annotation(b.f), rowVar, &idx) } -func querierClassDef() *pyast.ClassDef { - return &pyast.ClassDef{ - Name: "Querier", - Body: []*pyast.Node{ - { - Node: &pyast.Node_FunctionDef{ - FunctionDef: &pyast.FunctionDef{ - Name: "__init__", - Args: &pyast.Arguments{ - Args: []*pyast.Arg{ - { - Arg: "self", - }, - { - Arg: "conn", - Annotation: typeRefNode("sqlalchemy", "engine", "Connection"), - }, - }, - }, - Body: []*pyast.Node{ - { - Node: &pyast.Node_Assign{ - Assign: &pyast.Assign{ - Targets: []*pyast.Node{ - poet.Attribute(poet.Name("self"), "_conn"), - }, - Value: poet.Name("conn"), - }, - }, - }, - }, - }, - }, - }, - }, +// rowcount reads result.rowcount. Session.execute is typed as returning +// Result, which lacks rowcount; the cast target is a string so it is never +// evaluated, keeping SQLAlchemy 1.4 importable. +func (b *querierBuilder) rowcount() *pyast.Node { + b.f.typing("Any") + return poet.Attribute(poet.Node(&pyast.Call{ + Func: b.f.typing("cast"), + Args: []*pyast.Node{poet.Constant("sqlalchemy.engine.CursorResult[Any]"), poet.Name("result")}, + }), "rowcount") +} + +func isNone(name string) *pyast.Node { + return poet.Node(&pyast.Compare{ + Left: poet.Name(name), + Ops: []*pyast.Node{poet.Is()}, + Comparators: []*pyast.Node{poet.Constant(nil)}, + }) +} + +// sequenceParams is the executemany parameter list for a sequence-taking +// command: one dict per item of arg. +func sequenceParams(q Query) *pyast.Node { + dict := q.argDictNode("a") + if dict == nil { + dict = poet.Node(&pyast.Dict{}) } + return poet.ListComp(dict, poet.Name("a"), poet.Name("arg")) } -func asyncQuerierClassDef() *pyast.ClassDef { - return &pyast.ClassDef{ - Name: "AsyncQuerier", - Body: []*pyast.Node{ - { - Node: &pyast.Node_FunctionDef{ - FunctionDef: &pyast.FunctionDef{ - Name: "__init__", - Args: &pyast.Arguments{ - Args: []*pyast.Arg{ - { - Arg: "self", - }, - { - Arg: "conn", - Annotation: typeRefNode("sqlalchemy", "ext", "asyncio", "AsyncConnection"), - }, - }, - }, - Body: []*pyast.Node{ - { - Node: &pyast.Node_Assign{ - Assign: &pyast.Assign{ - Targets: []*pyast.Node{ - poet.Attribute(poet.Name("self"), "_conn"), - }, - Value: poet.Name("conn"), - }, - }, - }, - }, - }, - }, +func (b *querierBuilder) method(q Query) (querierMethod, error) { + m := querierMethod{ + name: q.MethodName, + args: &pyast.Arguments{Args: []*pyast.Arg{{Arg: "self"}}}, + } + q.addArgs(b.f, m.args) + + var body []*pyast.Node + switch q.Cmd { + case metadata.CmdOne: + body = append(body, + assignNode("row", poet.Node(&pyast.Call{ + Func: poet.Attribute(b.execute(q, q.argDictNode("")), "first"), + })), + poet.Node(&pyast.If{ + Test: isNone("row"), + Body: []*pyast.Node{poet.Return(poet.Constant(nil))}, + }), + poet.Return(b.rowNode(q.Ret, "row")), + ) + m.returns = b.f.optional(q.Ret.annotation(b.f)) + case metadata.CmdMany: + result := b.execute(q, q.argDictNode("")) + if b.async { + result = poet.Await(connMethodNode("stream", q.ConstantName, q.argDictNode(""))) + } + body = append(body, + assignNode("result", result), + b.forNode("row", poet.Name("result"), poet.Expr(poet.Yield(b.rowNode(q.Ret, "row")))), + ) + m.returns = b.iterator(q.Ret.annotation(b.f)) + m.asyncGen = b.async + case metadata.CmdExec: + body = append(body, b.execute(q, q.argDictNode(""))) + m.returns = poet.Constant(nil) + case metadata.CmdExecRows: + body = append(body, + assignNode("result", b.execute(q, q.argDictNode(""))), + poet.Return(b.rowcount()), + ) + m.returns = poet.Name("int") + case metadata.CmdExecResult: + body = append(body, poet.Return(b.execute(q, q.argDictNode("")))) + b.f.typing("Any") + m.returns = poet.Constant("sqlalchemy.engine.Result[Any]") + case metadata.CmdCopyFrom: + body = append(body, + poet.Node(&pyast.If{ + Test: poet.Not(poet.Name("arg")), + Body: []*pyast.Node{poet.Return(poet.Constant(0))}, + }), + assignNode("result", b.execute(q, sequenceParams(q))), + poet.Return(b.rowcount()), + ) + m.returns = poet.Name("int") + case metadata.CmdBatchExec: + body = append(body, + poet.Node(&pyast.If{ + Test: poet.Not(poet.Name("arg")), + Body: []*pyast.Node{poet.Return(poet.Constant(nil))}, + }), + b.execute(q, sequenceParams(q)), + ) + m.returns = poet.Constant(nil) + case metadata.CmdBatchOne: + body = append(body, poet.Node(&pyast.For{ + Target: poet.Name("a"), + Iter: poet.Name("arg"), + Body: []*pyast.Node{ + assignNode("row", poet.Node(&pyast.Call{ + Func: poet.Attribute(b.execute(q, q.argDictNode("a")), "first"), + })), + poet.Node(&pyast.If{ + Test: isNone("row"), + Body: []*pyast.Node{poet.Expr(poet.Yield(poet.Constant(nil)))}, + OrElse: []*pyast.Node{poet.Expr(poet.Yield(b.rowNode(q.Ret, "row")))}, + }), }, - }, + })) + m.returns = b.iterator(b.f.optional(q.Ret.annotation(b.f))) + m.asyncGen = b.async + case metadata.CmdBatchMany: + body = append(body, poet.Node(&pyast.For{ + Target: poet.Name("a"), + Iter: poet.Name("arg"), + Body: []*pyast.Node{ + poet.Expr(poet.Yield(poet.ListComp( + b.rowNode(q.Ret, "row"), + poet.Name("row"), + b.execute(q, q.argDictNode("a")), + ))), + }, + })) + m.returns = b.iterator(b.f.list(q.Ret.annotation(b.f))) + m.asyncGen = b.async + default: + return m, fmt.Errorf("query %s: unsupported command %s", q.MethodName, q.Cmd) + } + + // The with block encloses loops and yields, so errors raised while the + // caller iterates a generator method are wrapped too. + if b.conf.EmitQueryErrors { + body = []*pyast.Node{poet.With(poet.Node(&pyast.Call{ + Func: typeRefNode("errors", "_wrap_errors"), + Args: []*pyast.Node{poet.Constant(q.MethodName)}, + }), body...)} } + if len(q.Comments) > 0 { + m.body = append(m.body, docstringNode(strings.Join(q.Comments, "\n"))) + } + m.body = append(m.body, body...) + return m, nil } -func buildQueryTree(ctx *pyTmplCtx, i *importer, source string) *pyast.Node { - mod := moduleNode(ctx.SqlcVersion, source) - std, pkg := i.queryImportSpecs(source) - mod.Body = append(mod.Body, buildImportGroup(std), buildImportGroup(pkg)) - mod.Body = append(mod.Body, &pyast.Node{ - Node: &pyast.Node_ImportGroup{ - ImportGroup: &pyast.ImportGroup{ - Imports: []*pyast.Node{ - { - Node: &pyast.Node_ImportFrom{ - ImportFrom: &pyast.ImportFrom{ - Module: ctx.C.Package, - Names: []*pyast.Node{ - poet.Alias("models"), - }, - }, - }, - }, - }, +func initNode(connType *pyast.Node) *pyast.Node { + return poet.Node(&pyast.FunctionDef{ + Name: "__init__", + Args: &pyast.Arguments{ + Args: []*pyast.Arg{ + {Arg: "self"}, + {Arg: "conn", Annotation: connType}, }, }, + Body: []*pyast.Node{ + poet.Node(&pyast.Assign{ + Targets: []*pyast.Node{poet.Attribute(poet.Name("self"), "_conn")}, + Value: poet.Name("conn"), + }), + }, }) +} + +func (b *querierBuilder) connType() *pyast.Node { + if b.async { + b.f.importModule("sqlalchemy.ext.asyncio") + return b.f.union( + typeRefNode("sqlalchemy", "ext", "asyncio", "AsyncConnection"), + typeRefNode("sqlalchemy", "ext", "asyncio", "AsyncSession"), + ) + } + b.f.importModule("sqlalchemy.orm") + return b.f.union( + typeRefNode("sqlalchemy", "engine", "Connection"), + typeRefNode("sqlalchemy", "orm", "Session"), + ) +} + +// classes returns the querier class, preceded by its protocol when enabled. +func (b *querierBuilder) classes(queries []Query) ([]*pyast.Node, error) { + name := "Querier" + if b.async { + name = "AsyncQuerier" + } + cls := &pyast.ClassDef{Name: name} + connType := b.connType() + if b.conf.EmitGenericQuerier { + cls.TypeParams = []*pyast.TypeVar{{Name: "_ConnT", Bound: connType}} + connType = poet.Name("_ConnT") + } + cls.Body = append(cls.Body, + poet.Node(&pyast.AnnAssign{Target: &pyast.Name{Id: "_conn"}, Annotation: connType}), + initNode(connType), + ) + proto := &pyast.ClassDef{Name: name + "Protocol"} + for _, q := range queries { + m, err := b.method(q) + if err != nil { + return nil, err + } + cls.Body = append(cls.Body, m.node(b.async)) + proto.Body = append(proto.Body, m.protocolNode(b.async)) + } + if !b.conf.EmitQuerierProtocol { + return []*pyast.Node{poet.Node(cls)}, nil + } + proto.Bases = []*pyast.Node{b.f.typing("Protocol")} + return []*pyast.Node{poet.Node(proto), poet.Node(cls)}, nil +} + +func buildQueryTree(ctx *pyTmplCtx, source string) (*pyast.Node, error) { + f := newPyFile(ctx.C, false) + f.importModule("sqlalchemy") + var queries []Query for _, q := range ctx.Queries { - if !ctx.OutputQuery(q.SourceName) { - continue + if ctx.OutputQuery(q.SourceName) { + queries = append(queries, q) } - queryText := fmt.Sprintf("-- name: %s \\\\%s\n%s\n", q.MethodName, q.Cmd, q.SQL) - mod.Body = append(mod.Body, assignNode(q.ConstantName, poet.Constant(queryText))) + } + + var body []*pyast.Node + for _, q := range queries { + queryText := fmt.Sprintf("-- name: %s \\%s\n%s\n", escapeKeyword(strings.ToLower(q.ConstantName)), q.Cmd, q.SQL) + body = append(body, assignNode(q.ConstantName, poet.Constant(queryText))) for _, arg := range q.Args { if arg.EmitStruct() { - var def *pyast.ClassDef - if ctx.C.EmitPydanticModels { - def = pydanticNode(arg.Struct.Name) - } else { - def = dataclassNode(arg.Struct.Name) - } - for _, f := range arg.Struct.Fields { - def.Body = append(def.Body, fieldNode(f)) - } - mod.Body = append(mod.Body, poet.Node(def)) + body = append(body, structClassDef(f, ctx.C, arg.Struct)) } } if q.Ret.EmitStruct() { - var def *pyast.ClassDef - if ctx.C.EmitPydanticModels { - def = pydanticNode(q.Ret.Struct.Name) - } else { - def = dataclassNode(q.Ret.Struct.Name) - } - for _, f := range q.Ret.Struct.Fields { - def.Body = append(def.Body, fieldNode(f)) - } - mod.Body = append(mod.Body, poet.Node(def)) + body = append(body, structClassDef(f, ctx.C, q.Ret.Struct)) } } - if ctx.C.EmitSyncQuerier { - cls := querierClassDef() - for _, q := range ctx.Queries { - if !ctx.OutputQuery(q.SourceName) { - continue - } - f := &pyast.FunctionDef{ - Name: q.MethodName, - Args: &pyast.Arguments{ - Args: []*pyast.Arg{ - { - Arg: "self", - }, - }, - }, - } - - q.AddArgs(f.Args) - exec := connMethodNode("execute", q.ConstantName, q.ArgDictNode()) - - switch q.Cmd { - case ":one": - f.Body = append(f.Body, - assignNode("row", poet.Node( - &pyast.Call{ - Func: poet.Attribute(exec, "first"), - }, - )), - poet.Node( - &pyast.If{ - Test: poet.Node( - &pyast.Compare{ - Left: poet.Name("row"), - Ops: []*pyast.Node{ - poet.Is(), - }, - Comparators: []*pyast.Node{ - poet.Constant(nil), - }, - }, - ), - Body: []*pyast.Node{ - poet.Return( - poet.Constant(nil), - ), - }, - }, - ), - poet.Return(q.Ret.RowNode("row")), - ) - f.Returns = subscriptNode("Optional", q.Ret.Annotation()) - case ":many": - f.Body = append(f.Body, - assignNode("result", exec), - poet.Node( - &pyast.For{ - Target: poet.Name("row"), - Iter: poet.Name("result"), - Body: []*pyast.Node{ - poet.Expr( - poet.Yield( - q.Ret.RowNode("row"), - ), - ), - }, - }, - ), - ) - f.Returns = subscriptNode("Iterator", q.Ret.Annotation()) - case ":exec": - f.Body = append(f.Body, exec) - f.Returns = poet.Constant(nil) - case ":execrows": - f.Body = append(f.Body, - assignNode("result", exec), - poet.Return(poet.Attribute(poet.Name("result"), "rowcount")), - ) - f.Returns = poet.Name("int") - case ":execresult": - f.Body = append(f.Body, - poet.Return(exec), - ) - f.Returns = typeRefNode("sqlalchemy", "engine", "Result") - default: - panic("unknown cmd " + q.Cmd) - } - - cls.Body = append(cls.Body, poet.Node(f)) + var protocols, queriers []*pyast.Node + for _, async := range []bool{false, true} { + if (!async && !ctx.C.EmitSyncQuerier) || (async && !ctx.C.EmitAsyncQuerier) { + continue } - mod.Body = append(mod.Body, poet.Node(cls)) - } - - if ctx.C.EmitAsyncQuerier { - cls := asyncQuerierClassDef() - for _, q := range ctx.Queries { - if !ctx.OutputQuery(q.SourceName) { - continue - } - f := &pyast.AsyncFunctionDef{ - Name: q.MethodName, - Args: &pyast.Arguments{ - Args: []*pyast.Arg{ - { - Arg: "self", - }, - }, - }, - } - - q.AddArgs(f.Args) - exec := connMethodNode("execute", q.ConstantName, q.ArgDictNode()) - - switch q.Cmd { - case ":one": - f.Body = append(f.Body, - assignNode("row", poet.Node( - &pyast.Call{ - Func: poet.Attribute(poet.Await(exec), "first"), - }, - )), - poet.Node( - &pyast.If{ - Test: poet.Node( - &pyast.Compare{ - Left: poet.Name("row"), - Ops: []*pyast.Node{ - poet.Is(), - }, - Comparators: []*pyast.Node{ - poet.Constant(nil), - }, - }, - ), - Body: []*pyast.Node{ - poet.Return( - poet.Constant(nil), - ), - }, - }, - ), - poet.Return(q.Ret.RowNode("row")), - ) - f.Returns = subscriptNode("Optional", q.Ret.Annotation()) - case ":many": - stream := connMethodNode("stream", q.ConstantName, q.ArgDictNode()) - f.Body = append(f.Body, - assignNode("result", poet.Await(stream)), - poet.Node( - &pyast.AsyncFor{ - Target: poet.Name("row"), - Iter: poet.Name("result"), - Body: []*pyast.Node{ - poet.Expr( - poet.Yield( - q.Ret.RowNode("row"), - ), - ), - }, - }, - ), - ) - f.Returns = subscriptNode("AsyncIterator", q.Ret.Annotation()) - case ":exec": - f.Body = append(f.Body, poet.Await(exec)) - f.Returns = poet.Constant(nil) - case ":execrows": - f.Body = append(f.Body, - assignNode("result", poet.Await(exec)), - poet.Return(poet.Attribute(poet.Name("result"), "rowcount")), - ) - f.Returns = poet.Name("int") - case ":execresult": - f.Body = append(f.Body, - poet.Return(poet.Await(exec)), - ) - f.Returns = typeRefNode("sqlalchemy", "engine", "Result") - default: - panic("unknown cmd " + q.Cmd) - } - - cls.Body = append(cls.Body, poet.Node(f)) + b := &querierBuilder{f: f, conf: ctx.C, async: async} + classes, err := b.classes(queries) + if err != nil { + return nil, err } - mod.Body = append(mod.Body, poet.Node(cls)) + protocols = append(protocols, classes[:len(classes)-1]...) + queriers = append(queriers, classes[len(classes)-1]) } + body = append(body, protocols...) + body = append(body, queriers...) - return poet.Node(mod) + local := &pyast.ImportFrom{Module: ctx.C.Package} + if ctx.C.EmitQueryErrors { + local.Names = append(local.Names, poet.Alias("errors")) + } + local.Names = append(local.Names, poet.Alias("models")) + + mod := moduleNode(ctx.SqlcVersion, source, ctx.C.OmitSqlcVersion) + mod.Body = append(mod.Body, + importGroup(f.std), + importGroup(f.pkg), + &pyast.Node{Node: &pyast.Node_ImportGroup{ImportGroup: &pyast.ImportGroup{ + Imports: []*pyast.Node{{Node: &pyast.Node_ImportFrom{ImportFrom: local}}}, + }}}, + ) + mod.Body = append(mod.Body, body...) + return poet.Node(mod), nil } type pyTmplCtx struct { @@ -1077,30 +1181,29 @@ func (t *pyTmplCtx) OutputQuery(sourceName string) bool { return t.SourceName == sourceName } -func HashComment(s string) string { - return "# " + strings.ReplaceAll(s, "\n", "\n# ") +func queryFileName(source string) string { + name := source + if !strings.HasSuffix(name, ".py") { + name = strings.TrimSuffix(name, ".sql") + name += ".py" + } + return name } func Generate(_ context.Context, req *plugin.GenerateRequest) (*plugin.GenerateResponse, error) { - var conf Config - if len(req.PluginOptions) > 0 { - if err := json.Unmarshal(req.PluginOptions, &conf); err != nil { - return nil, err - } + conf, err := parseConfig(req) + if err != nil { + return nil, err } - enums := buildEnums(req) + enums := buildEnums(conf, req) models := buildModels(conf, req) queries, err := buildQueries(conf, req, models) if err != nil { return nil, err } - - i := &importer{ - Models: models, - Queries: queries, - Enums: enums, - C: conf, + if conf.OmitUnusedStructs { + enums, models = filterUnusedStructs(enums, models, queries) } tctx := pyTmplCtx{ @@ -1112,7 +1215,7 @@ func Generate(_ context.Context, req *plugin.GenerateRequest) (*plugin.GenerateR } output := map[string]string{} - result := pyprint.Print(buildModelsTree(&tctx, i), pyprint.Options{}) + result := pyprint.Print(buildModelsTree(&tctx), pyprint.Options{}) tctx.SourceName = "models.py" output["models.py"] = string(result.Python) @@ -1121,15 +1224,29 @@ func Generate(_ context.Context, req *plugin.GenerateRequest) (*plugin.GenerateR files[q.SourceName] = struct{}{} } + for source := range files { + if queryFileName(source) == "models.py" { + return nil, fmt.Errorf("query file %s would overwrite models.py", source) + } + } + + if conf.EmitQueryErrors { + for source := range files { + if queryFileName(source) == errorsFileName { + return nil, fmt.Errorf("query file %s would overwrite %s, which emit_query_errors generates", source, errorsFileName) + } + } + output[errorsFileName] = buildErrorsModule(&tctx) + } + for source := range files { tctx.SourceName = source - result := pyprint.Print(buildQueryTree(&tctx, i, source), pyprint.Options{}) - name := source - if !strings.HasSuffix(name, ".py") { - name = strings.TrimSuffix(name, ".sql") - name += ".py" + tree, err := buildQueryTree(&tctx, source) + if err != nil { + return nil, err } - output[name] = string(result.Python) + result := pyprint.Print(tree, pyprint.Options{}) + output[queryFileName(source)] = string(result.Python) } resp := plugin.GenerateResponse{} diff --git a/internal/ident.go b/internal/ident.go new file mode 100644 index 0000000..64ad662 --- /dev/null +++ b/internal/ident.go @@ -0,0 +1,31 @@ +package python + +// Hard keywords from keyword.kwlist in Python 3.12. Soft keywords (match, +// case, type, _) are valid identifiers and are left alone. +var pyKeywords = map[string]struct{}{ + "False": {}, "None": {}, "True": {}, "and": {}, "as": {}, "assert": {}, + "async": {}, "await": {}, "break": {}, "class": {}, "continue": {}, + "def": {}, "del": {}, "elif": {}, "else": {}, "except": {}, "finally": {}, + "for": {}, "from": {}, "global": {}, "if": {}, "import": {}, "in": {}, + "is": {}, "lambda": {}, "nonlocal": {}, "not": {}, "or": {}, "pass": {}, + "raise": {}, "return": {}, "try": {}, "while": {}, "with": {}, "yield": {}, +} + +func escapeKeyword(name string) string { + if _, ok := pyKeywords[name]; ok { + return name + "_" + } + return name +} + +// pyIdent is the single source of generated Python names. A rename entry for +// dbName wins over transform; the result is then keyword-escaped. +func pyIdent(dbName string, conf Config, transform func(string) string) string { + if r, ok := conf.Rename[dbName]; ok && r != "" { + return escapeKeyword(r) + } + if transform != nil { + dbName = transform(dbName) + } + return escapeKeyword(dbName) +} diff --git a/internal/ident_test.go b/internal/ident_test.go new file mode 100644 index 0000000..697dc3c --- /dev/null +++ b/internal/ident_test.go @@ -0,0 +1,56 @@ +package python + +import "testing" + +func TestPyIdent(t *testing.T) { + for _, tc := range []struct { + name string + rename map[string]string + want string + }{ + {name: "from", want: "from_"}, + {name: "class", want: "class_"}, + {name: "match", want: "match"}, + {name: "type", want: "type"}, + {name: "author_id", want: "author_id"}, + {name: "spotify_url", rename: map[string]string{"spotify_url": "spotify_link"}, want: "spotify_link"}, + {name: "source", rename: map[string]string{"source": "from"}, want: "from_"}, + } { + got := pyIdent(tc.name, Config{Rename: tc.rename}, nil) + if got != tc.want { + t.Errorf("pyIdent(%q) = %q, want %q", tc.name, got, tc.want) + } + } +} + +func TestPyIdentTransform(t *testing.T) { + conf := Config{Rename: map[string]string{"person": "Human"}} + if got := pyIdent("person", conf, func(s string) string { return modelName(s, nil) }); got != "Human" { + t.Errorf("rename should bypass transform, got %q", got) + } + if got := pyIdent("book", conf, func(s string) string { return modelName(s, nil) }); got != "Book" { + t.Errorf("got %q", got) + } +} + +func TestEnumMemberNames(t *testing.T) { + for _, tc := range []struct { + vals []string + want []string + }{ + {vals: []string{"=", "<>"}, want: []string{"VALUE_1", "VALUE_2"}}, + {vals: []string{"in-progress", "in_progress", "in progress"}, want: []string{"IN_PROGRESS", "IN_PROGRESS_2", "INPROGRESS"}}, + {vals: []string{"a-b", "a_b", "a_b_2"}, want: []string{"A_B", "A_B_2", "A_B_2_2"}}, + {vals: []string{"1st"}, want: []string{"VALUE_1ST"}}, + {vals: []string{`say "hi"`}, want: []string{"SAYHI"}}, + {vals: []string{"in-progress", "="}, want: []string{"IN_PROGRESS", "VALUE_2"}}, + } { + got := enumMemberNames(Config{}, tc.vals) + for i := range got { + if got[i] != tc.want[i] { + t.Errorf("enumMemberNames(%q) = %q, want %q", tc.vals, got, tc.want) + break + } + } + } +} diff --git a/internal/imports.go b/internal/imports.go index b88c58c..66ab70d 100644 --- a/internal/imports.go +++ b/internal/imports.go @@ -1,276 +1,187 @@ package python import ( - "fmt" + "regexp" "sort" "strings" + + pyast "github.com/sqlc-dev/sqlc-gen-python/internal/ast" + "github.com/sqlc-dev/sqlc-gen-python/internal/poet" ) type importSpec struct { Module string Name string - Alias string } -func (i importSpec) String() string { - if i.Alias != "" { - if i.Name == "" { - return fmt.Sprintf("import %s as %s", i.Module, i.Alias) - } - return fmt.Sprintf("from %s import %s as %s", i.Module, i.Name, i.Alias) - } - if i.Name == "" { - return "import " + i.Module +// pyFile records the imports a generated module needs while its body is +// built, so imports always match what the module references. +type pyFile struct { + modern bool + inModels bool + std map[string]importSpec + pkg map[string]importSpec +} + +func newPyFile(conf Config, inModels bool) *pyFile { + return &pyFile{ + modern: conf.EmitModernTypes, + inModels: inModels, + std: map[string]importSpec{}, + pkg: map[string]importSpec{}, } - return fmt.Sprintf("from %s import %s", i.Module, i.Name) } -type importer struct { - Models []Struct - Queries []Query - Enums []Enum - C Config +var stdlibModules = map[string]struct{}{ + "abc": {}, "collections": {}, "contextlib": {}, "dataclasses": {}, + "datetime": {}, "decimal": {}, "enum": {}, "fractions": {}, + "ipaddress": {}, "json": {}, "pathlib": {}, "typing": {}, "uuid": {}, + // pydantic is third-party, but the first import group has always held it. + "pydantic": {}, } -func structUses(name string, s Struct) bool { - for _, f := range s.Fields { - if name == "typing.List" && f.Type.IsArray { - return true - } - if name == "typing.Optional" && f.Type.IsNull { - return true - } - if f.Type.InnerType == name { - return true - } +func (f *pyFile) group(module string) map[string]importSpec { + top, _, _ := strings.Cut(module, ".") + if _, ok := stdlibModules[top]; ok { + return f.std } - return false + return f.pkg } -func queryValueUses(name string, qv QueryValue) bool { - if !qv.isEmpty() { - if name == "typing.List" && qv.Typ.IsArray { - return true - } - if name == "typing.Optional" && qv.Typ.IsNull { - return true - } - if qv.IsStruct() && qv.EmitStruct() { - if structUses(name, *qv.Struct) { - return true - } - } else { - if qv.Typ.InnerType == name { - return true - } - } - } - return false +func (f *pyFile) importModule(module string) { + f.group(module)[module] = importSpec{Module: module} } -func (i *importer) Imports(fileName string) []string { - if fileName == "models.py" { - return i.modelImports() - } - return i.queryImports(fileName) +func (f *pyFile) importName(module, name string) { + f.group(module)[module+"."+name] = importSpec{Module: module, Name: name} } -func (i *importer) modelImportSpecs() (map[string]importSpec, map[string]importSpec) { - modelUses := func(name string) bool { - for _, model := range i.Models { - if structUses(name, model) { - return true - } - } - return false - } +func (f *pyFile) typing(name string) *pyast.Node { + f.importName("typing", name) + return poet.Name(name) +} - std := stdImports(modelUses) - if i.C.EmitPydanticModels { - std["pydantic"] = importSpec{Module: "pydantic"} - } else { - std["dataclasses"] = importSpec{Module: "dataclasses"} +// abc returns a collections.abc name, spelled from typing unless modern +// syntax is on. +func (f *pyFile) abc(name string) *pyast.Node { + if f.modern { + f.importName("collections.abc", name) + return poet.Name(name) } - if len(i.Enums) > 0 { - std["enum"] = importSpec{Module: "enum"} - } - - pkg := make(map[string]importSpec) + return f.typing(name) +} - return std, pkg +func (f *pyFile) optional(n *pyast.Node) *pyast.Node { + if f.modern { + return poet.BitOr(n, poet.Constant(nil)) + } + f.typing("Optional") + return poet.Subscript("Optional", n) } -func (i *importer) modelImports() []string { - std, pkg := i.modelImportSpecs() - importLines := []string{ - buildImportBlock(std), - "", - buildImportBlock(pkg), +func (f *pyFile) list(n *pyast.Node) *pyast.Node { + if f.modern { + return poet.Subscript("list", n) } - return importLines + f.typing("List") + return poet.Subscript("List", n) } -func (i *importer) queryImportSpecs(fileName string) (map[string]importSpec, map[string]importSpec) { - queryUses := func(name string) bool { - for _, q := range i.Queries { - if q.SourceName != fileName { - continue - } - if queryValueUses(name, q.Ret) { - return true - } - for _, arg := range q.Args { - if queryValueUses(name, arg) { - return true - } - } +func (f *pyFile) union(ns ...*pyast.Node) *pyast.Node { + if f.modern { + out := ns[0] + for _, n := range ns[1:] { + out = poet.BitOr(out, n) } - return false + return out } + f.typing("Union") + return poet.Subscript("Union", ns...) +} - std := stdImports(queryUses) - - pkg := make(map[string]importSpec) - pkg["sqlalchemy"] = importSpec{Module: "sqlalchemy"} - if i.C.EmitAsyncQuerier { - pkg["sqlalchemy.ext.asyncio"] = importSpec{Module: "sqlalchemy.ext.asyncio"} - } +// generic wraps n in a collections.abc generic: Iterator, AsyncIterator or +// Sequence. +func (f *pyFile) generic(name string, n *pyast.Node) *pyast.Node { + f.abc(name) + return poet.Subscript(name, n) +} - queryValueModelImports := func(qv QueryValue) { - if qv.IsStruct() && qv.EmitStruct() { - if i.C.EmitPydanticModels { - std["pydantic"] = importSpec{Module: "pydantic"} - } else { - std["dataclasses"] = importSpec{Module: "dataclasses"} - } +func (f *pyFile) typeName(t pyType) *pyast.Node { + switch { + case t.Import != "": + f.importName(t.Import, t.InnerType) + return poet.Name(t.InnerType) + case t.InnerType == "Any": + return f.typing("Any") + case strings.HasPrefix(t.InnerType, "models."): + if f.inModels { + return poet.Name(strings.TrimPrefix(t.InnerType, "models.")) } + return poet.Name(t.InnerType) } - - for _, q := range i.Queries { - if q.SourceName != fileName { - continue - } - if q.Cmd == ":one" { - std["typing.Optional"] = importSpec{Module: "typing", Name: "Optional"} - } - if q.Cmd == ":many" { - if i.C.EmitSyncQuerier { - std["typing.Iterator"] = importSpec{Module: "typing", Name: "Iterator"} - } - if i.C.EmitAsyncQuerier { - std["typing.AsyncIterator"] = importSpec{Module: "typing", Name: "AsyncIterator"} - } - } - queryValueModelImports(q.Ret) - for _, qv := range q.Args { - queryValueModelImports(qv) - } + for _, module := range typeModules(t.InnerType) { + f.importModule(module) } - - return std, pkg + return poet.Name(t.InnerType) } -func (i *importer) queryImports(fileName string) []string { - std, pkg := i.queryImportSpecs(fileName) +var dottedNamePattern = regexp.MustCompile(`[A-Za-z_]\w*(?:\.[A-Za-z_]\w*)+`) - modelImportStr := fmt.Sprintf("from %s import models", i.C.Package) - - importLines := []string{ - buildImportBlock(std), - "", - buildImportBlock(pkg), - "", - modelImportStr, +// typeModules returns the module of every dotted name in a type expression, +// so "dict[str, decimal.Decimal]" needs decimal. +func typeModules(typ string) []string { + var modules []string + for _, name := range dottedNamePattern.FindAllString(typ, -1) { + modules = append(modules, name[:strings.LastIndex(name, ".")]) } - return importLines -} - -type importFromSpec struct { - Module string - Names []string - Alias string + return modules } -func buildImportBlock2(pkgs map[string]importSpec) []importFromSpec { - pkgImports := make([]importFromSpec, 0) - fromImports := make(map[string][]string) - for _, is := range pkgs { - if is.Name == "" || is.Alias != "" { - pkgImports = append(pkgImports, importFromSpec{ - Module: is.Module, - Names: []string{is.Name}, - Alias: is.Alias, - }) - } else { - names, ok := fromImports[is.Module] - if !ok { - names = make([]string, 0, 1) - } - names = append(names, is.Name) - fromImports[is.Module] = names - } +func (f *pyFile) annotation(t pyType) *pyast.Node { + ann := f.typeName(t) + for i := 0; i < t.ArrayDims; i++ { + ann = f.list(ann) } - for modName, names := range fromImports { - sort.Strings(names) - pkgImports = append(pkgImports, importFromSpec{ - Module: modName, - Names: names, - }) + if t.IsNull { + ann = f.optional(ann) } - sort.Slice(pkgImports, func(i, j int) bool { - return pkgImports[i].Module < pkgImports[j].Module || pkgImports[i].Names[0] < pkgImports[j].Names[0] - }) - return pkgImports + return ann } -func buildImportBlock(pkgs map[string]importSpec) string { - pkgImports := make([]importSpec, 0) - fromImports := make(map[string][]string) - for _, is := range pkgs { - if is.Name == "" || is.Alias != "" { - pkgImports = append(pkgImports, is) +// importGroup prints straight imports before from-imports, each sorted by +// module, as isort does. +func importGroup(specs map[string]importSpec) *pyast.Node { + var modules []string + names := map[string][]string{} + for _, spec := range specs { + if spec.Name == "" { + modules = append(modules, spec.Module) } else { - names, ok := fromImports[is.Module] - if !ok { - names = make([]string, 0, 1) - } - names = append(names, is.Name) - fromImports[is.Module] = names + names[spec.Module] = append(names[spec.Module], spec.Name) } } - - importStrings := make([]string, 0, len(pkgImports)+len(fromImports)) - for _, is := range pkgImports { - importStrings = append(importStrings, is.String()) - } - for modName, names := range fromImports { - sort.Strings(names) - nameString := strings.Join(names, ", ") - importStrings = append(importStrings, fmt.Sprintf("from %s import %s", modName, nameString)) + sort.Strings(modules) + fromModules := make([]string, 0, len(names)) + for m := range names { + fromModules = append(fromModules, m) } - sort.Strings(importStrings) - return strings.Join(importStrings, "\n") -} + sort.Strings(fromModules) -func stdImports(uses func(name string) bool) map[string]importSpec { - std := make(map[string]importSpec) - if uses("decimal.Decimal") { - std["decimal"] = importSpec{Module: "decimal"} - } - if uses("datetime.date") || uses("datetime.time") || uses("datetime.datetime") || uses("datetime.timedelta") { - std["datetime"] = importSpec{Module: "datetime"} - } - if uses("uuid.UUID") { - std["uuid"] = importSpec{Module: "uuid"} + var body []*pyast.Node + for _, m := range modules { + body = append(body, importNode(m)) } - if uses("typing.List") { - std["typing.List"] = importSpec{Module: "typing", Name: "List"} - } - if uses("typing.Optional") { - std["typing.Optional"] = importSpec{Module: "typing", Name: "Optional"} + for _, m := range fromModules { + sort.Strings(names[m]) + imp := &pyast.ImportFrom{Module: m} + for _, name := range names[m] { + imp.Names = append(imp.Names, poet.Alias(name)) + } + body = append(body, &pyast.Node{Node: &pyast.Node_ImportFrom{ImportFrom: imp}}) } - if uses("Any") { - std["typing.Any"] = importSpec{Module: "typing", Name: "Any"} + return &pyast.Node{ + Node: &pyast.Node_ImportGroup{ + ImportGroup: &pyast.ImportGroup{Imports: body}, + }, } - return std } diff --git a/internal/poet/builders.go b/internal/poet/builders.go index d38ed7c..fb50228 100644 --- a/internal/poet/builders.go +++ b/internal/poet/builders.go @@ -88,3 +88,83 @@ func Yield(value *ast.Node) *ast.Node { }, } } + +func BitOr(left, right *ast.Node) *ast.Node { + return &ast.Node{ + Node: &ast.Node_BinOp{ + BinOp: &ast.BinOp{ + Left: left, + Op: &ast.Node{Node: &ast.Node_BitOr{BitOr: &ast.BitOr{}}}, + Right: right, + }, + }, + } +} + +func Ellipsis() *ast.Node { + return &ast.Node{ + Node: &ast.Node_Constant{ + Constant: &ast.Constant{ + Value: &ast.Constant_Ellipsis{Ellipsis: true}, + }, + }, + } +} + +func ListComp(elt, target, iter *ast.Node) *ast.Node { + return &ast.Node{ + Node: &ast.Node_ListComp{ + ListComp: &ast.ListComp{ + Elt: elt, + Generators: []*ast.Comprehension{ + {Target: target, Iter: iter}, + }, + }, + }, + } +} + +func Subscript(value string, slice ...*ast.Node) *ast.Node { + s := slice[0] + if len(slice) > 1 { + s = Tuple(slice...) + } + return &ast.Node{ + Node: &ast.Node_Subscript{ + Subscript: &ast.Subscript{ + Value: &ast.Name{Id: value}, + Slice: s, + }, + }, + } +} + +func Tuple(elts ...*ast.Node) *ast.Node { + return &ast.Node{ + Node: &ast.Node_Tuple{ + Tuple: &ast.Tuple{Elts: elts}, + }, + } +} + +func With(contextExpr *ast.Node, body ...*ast.Node) *ast.Node { + return &ast.Node{ + Node: &ast.Node_With{ + With: &ast.With{ + Items: []*ast.WithItem{{ContextExpr: contextExpr}}, + Body: body, + }, + }, + } +} + +func Not(operand *ast.Node) *ast.Node { + return &ast.Node{ + Node: &ast.Node_UnaryOp{ + UnaryOp: &ast.UnaryOp{ + Op: &ast.Node{Node: &ast.Node_Not{Not: &ast.Not{}}}, + Operand: operand, + }, + }, + } +} diff --git a/internal/poet/poet.go b/internal/poet/poet.go index 22b488c..e0b9c8a 100644 --- a/internal/poet/poet.go +++ b/internal/poet/poet.go @@ -140,6 +140,20 @@ func Node(node proto) *ast.Node { // case *ast.Node_Keyword: // w.printKeyword(n.Keyword, indent) + case *ast.ListComp: + return &ast.Node{ + Node: &ast.Node_ListComp{ + ListComp: n, + }, + } + + case *ast.With: + return &ast.Node{ + Node: &ast.Node_With{ + With: n, + }, + } + case *ast.Module: return &ast.Node{ Node: &ast.Node_Module{ diff --git a/internal/postgresql_type.go b/internal/postgresql_type.go index 3d0891b..734cc35 100644 --- a/internal/postgresql_type.go +++ b/internal/postgresql_type.go @@ -7,7 +7,7 @@ import ( "github.com/sqlc-dev/plugin-sdk-go/sdk" ) -func postgresType(req *plugin.GenerateRequest, col *plugin.Column) string { +func postgresType(req *plugin.GenerateRequest, conf Config, col *plugin.Column) string { columnType := sdk.DataType(col.Type) switch columnType { @@ -19,19 +19,24 @@ func postgresType(req *plugin.GenerateRequest, col *plugin.Column) string { return "decimal.Decimal" case "boolean", "bool", "pg_catalog.bool": return "bool" - case "json", "jsonb": + case "json", "jsonb", "pg_catalog.json", "pg_catalog.jsonb": return "Any" case "bytea", "blob", "pg_catalog.bytea": return "memoryview" - case "date": + case "date", "pg_catalog.date": return "datetime.date" - case "pg_catalog.time", "pg_catalog.timetz": + case "time", "pg_catalog.time", "timetz", "pg_catalog.timetz", "time without time zone", "time with time zone": return "datetime.time" - case "pg_catalog.timestamp", "pg_catalog.timestamptz", "timestamptz": + case "timestamp", "pg_catalog.timestamp", "timestamp without time zone": + return "datetime.datetime" + case "timestamptz", "pg_catalog.timestamptz", "timestamp with time zone": + if conf.EmitAwareDatetime { + return "pydantic.AwareDatetime" + } return "datetime.datetime" case "interval", "pg_catalog.interval": return "datetime.timedelta" - case "text", "pg_catalog.varchar", "pg_catalog.bpchar", "string", "citext": + case "text", "pg_catalog.text", "varchar", "pg_catalog.varchar", "character varying", "bpchar", "pg_catalog.bpchar", "character", "name", "pg_catalog.name", "string", "citext": return "str" case "uuid": return "uuid.UUID" @@ -50,9 +55,9 @@ func postgresType(req *plugin.GenerateRequest, col *plugin.Column) string { for _, enum := range schema.Enums { if columnType == enum.Name { if schema.Name == req.Catalog.DefaultSchema { - return "models." + modelName(enum.Name, req.Settings) + return "models." + pyIdent(enum.Name, conf, className) } - return "models." + modelName(schema.Name+"_"+enum.Name, req.Settings) + return "models." + pyIdent(schema.Name+"_"+enum.Name, conf, className) } } } diff --git a/internal/printer/printer.go b/internal/printer/printer.go index 0660c6a..e69eab1 100644 --- a/internal/printer/printer.go +++ b/internal/printer/printer.go @@ -61,6 +61,12 @@ func (w *writer) printNode(node *ast.Node, indent int32) { case *ast.Node_Await: w.printAwait(n.Await, indent) + case *ast.Node_BinOp: + w.printBinOp(n.BinOp, indent) + + case *ast.Node_BitOr: + w.print("|") + case *ast.Node_Call: w.printCall(n.Call, indent) @@ -106,6 +112,9 @@ func (w *writer) printNode(node *ast.Node, indent int32) { case *ast.Node_Keyword: w.printKeyword(n.Keyword, indent) + case *ast.Node_ListComp: + w.printListComp(n.ListComp, indent) + case *ast.Node_Module: w.printModule(n.Module, indent) @@ -121,6 +130,20 @@ func (w *writer) printNode(node *ast.Node, indent int32) { case *ast.Node_Subscript: w.printSubscript(n.Subscript, indent) + case *ast.Node_Tuple: + w.printTuple(n.Tuple, indent) + + case *ast.Node_UnaryOp: + w.printNode(n.UnaryOp.Op, indent) + w.print(" ") + w.printNode(n.UnaryOp.Operand, indent) + + case *ast.Node_Not: + w.print("not") + + case *ast.Node_With: + w.printWith(n.With, indent) + case *ast.Node_Yield: w.printYield(n.Yield, indent) @@ -132,10 +155,12 @@ func (w *writer) printNode(node *ast.Node, indent int32) { func (w *writer) printAnnAssign(aa *ast.AnnAssign, indent int32) { if aa.Comment != "" { - w.print("# ") - w.print(aa.Comment) - w.print("\n") - w.printIndent(indent) + for _, line := range strings.Split(aa.Comment, "\n") { + w.print("# ") + w.print(line) + w.print("\n") + w.printIndent(indent) + } } w.printName(aa.Target, indent) w.print(": ") @@ -180,6 +205,14 @@ func (w *writer) printAsyncFunctionDef(afd *ast.AsyncFunctionDef, indent int32) }, indent) } +func (w *writer) printBinOp(b *ast.BinOp, indent int32) { + w.printNode(b.Left, indent) + w.print(" ") + w.printNode(b.Op, indent) + w.print(" ") + w.printNode(b.Right, indent) +} + func (w *writer) printAttribute(a *ast.Attribute, indent int32) { if _, ok := a.Value.Node.(*ast.Node_Await); ok { w.print("(") @@ -227,6 +260,20 @@ func (w *writer) printClassDef(cd *ast.ClassDef, indent int32) { } w.print("class ") w.print(cd.Name) + if len(cd.TypeParams) > 0 { + w.print("[") + for i, tp := range cd.TypeParams { + w.print(tp.Name) + if tp.Bound != nil { + w.print(": ") + w.printNode(tp.Bound, indent) + } + if i != len(cd.TypeParams)-1 { + w.print(", ") + } + } + w.print("]") + } if len(cd.Bases) > 0 { w.print("(") for i, node := range cd.Bases { @@ -238,6 +285,10 @@ func (w *writer) printClassDef(cd *ast.ClassDef, indent int32) { w.print(")") } w.print(":\n") + if len(cd.Body) == 0 { + w.printIndent(indent + 1) + w.print("pass\n") + } for i, node := range cd.Body { if i != 0 { if _, ok := node.Node.(*ast.Node_FunctionDef); ok { @@ -253,14 +304,10 @@ func (w *writer) printClassDef(cd *ast.ClassDef, indent int32) { // definition. Such a docstring becomes the __doc__ special // attribute of that object. if i == 0 { - if e, ok := node.Node.(*ast.Node_Expr); ok { - if c, ok := e.Expr.Value.Node.(*ast.Node_Constant); ok { - w.print(`""`) - w.printConstant(c.Constant, indent) - w.print(`""`) - w.print("\n") - continue - } + if doc, ok := docstring(node); ok { + w.printDocstring(doc) + w.print("\n") + continue } } w.printNode(node, indent+1) @@ -276,14 +323,19 @@ func (w *writer) printConstant(c *ast.Constant, indent int32) { case *ast.Constant_None: w.print("None") + case *ast.Constant_Ellipsis: + w.print("...") + case *ast.Constant_Str: - str := `"` if strings.Contains(n.Str, "\n") { - str = `"""` + w.print(`"""`) + w.print(escapeTripleQuoted(n.Str)) + w.print(`"""`) + } else { + w.print(`"`) + w.print(escapeQuoted(n.Str)) + w.print(`"`) } - w.print(str) - w.print(n.Str) - w.print(str) default: panic(n) @@ -347,25 +399,19 @@ func (w *writer) printFor(n *ast.For, indent int32) { w.print(" in ") w.printNode(n.Iter, indent) w.print(":\n") - for i, node := range n.Body { - w.printIndent(indent + 1) - w.printNode(node, indent+1) - if i != len(n.Body)-1 { - w.print("\n") - } - } + w.printBlock(n.Body, indent, false) } func (w *writer) printIf(i *ast.If, indent int32) { w.print("if ") w.printNode(i.Test, indent) w.print(":\n") - for j, node := range i.Body { - w.printIndent(indent + 1) - w.printNode(node, indent+1) - if j != len(i.Body)-1 { - w.print("\n") - } + w.printBlock(i.Body, indent, false) + if len(i.OrElse) > 0 { + w.print("\n") + w.printIndent(indent) + w.print("else:\n") + w.printBlock(i.OrElse, indent, false) } } @@ -395,16 +441,106 @@ func (w *writer) printFunctionDef(fd *ast.FunctionDef, indent int32) { w.print(" -> ") w.printNode(fd.Returns, indent) } + if len(fd.Body) == 1 && isEllipsis(fd.Body[0]) { + w.print(": ...") + return + } w.print(":\n") - for i, node := range fd.Body { + w.printBlock(fd.Body, indent, true) +} + +// printBlock prints body one level deeper than indent, without a trailing +// newline after the last statement. +func (w *writer) printBlock(body []*ast.Node, indent int32, allowDocstring bool) { + if len(body) == 0 { w.printIndent(indent + 1) - w.printNode(node, indent+1) - if i != len(fd.Body)-1 { + w.print("pass") + return + } + for i, node := range body { + w.printIndent(indent + 1) + if doc, ok := docstring(node); ok && i == 0 && allowDocstring { + w.printDocstring(doc) + } else { + w.printNode(node, indent+1) + } + if i != len(body)-1 { w.print("\n") } } } +func docstring(node *ast.Node) (string, bool) { + e, ok := node.Node.(*ast.Node_Expr) + if !ok { + return "", false + } + c, ok := e.Expr.Value.Node.(*ast.Node_Constant) + if !ok { + return "", false + } + str, ok := c.Constant.Value.(*ast.Constant_Str) + if !ok { + return "", false + } + return str.Str, true +} + +func isEllipsis(node *ast.Node) bool { + e, ok := node.Node.(*ast.Node_Expr) + if !ok { + return false + } + c, ok := e.Expr.Value.Node.(*ast.Node_Constant) + if !ok { + return false + } + _, ok = c.Constant.Value.(*ast.Constant_Ellipsis) + return ok +} + +func (w *writer) printDocstring(s string) { + w.print(`"""`) + w.print(escapeTripleQuoted(s)) + w.print(`"""`) +} + +func escapeQuoted(s string) string { + var b strings.Builder + for _, r := range s { + switch r { + case '\\': + b.WriteString(`\\`) + case '"': + b.WriteString(`\"`) + case '\r': + b.WriteString(`\r`) + default: + b.WriteRune(r) + } + } + return b.String() +} + +// escapeTripleQuoted escapes a quote only where it would end the literal: when +// it starts a run of three, or is the last character before the closing quotes. +func escapeTripleQuoted(s string) string { + var b strings.Builder + for i := 0; i < len(s); i++ { + switch c := s[i]; { + case c == '\\': + b.WriteString(`\\`) + case c == '\r': + b.WriteString(`\r`) + case c == '"' && (i == len(s)-1 || strings.HasPrefix(s[i+1:], `""`)): + b.WriteString(`\"`) + default: + b.WriteByte(c) + } + } + return b.String() +} + func (w *writer) printImport(imp *ast.Import, indent int32) { w.print("import ") for i, node := range imp.Names { @@ -442,6 +578,22 @@ func (w *writer) printImportGroup(n *ast.ImportGroup, indent int32) { func (w *writer) printIs(i *ast.Is, indent int32) { w.print("is") } +func (w *writer) printListComp(lc *ast.ListComp, indent int32) { + w.print("[") + w.printNode(lc.Elt, indent) + for _, gen := range lc.Generators { + w.print(" for ") + w.printNode(gen.Target, indent) + w.print(" in ") + w.printNode(gen.Iter, indent) + for _, cond := range gen.Ifs { + w.print(" if ") + w.printNode(cond, indent) + } + } + w.print("]") +} + func (w *writer) printKeyword(k *ast.Keyword, indent int32) { w.print(k.Arg) w.print("=") @@ -488,6 +640,31 @@ func (w *writer) printSubscript(ss *ast.Subscript, indent int32) { } +func (w *writer) printTuple(t *ast.Tuple, indent int32) { + for i, elt := range t.Elts { + w.printNode(elt, indent) + if i != len(t.Elts)-1 { + w.print(", ") + } + } +} + +func (w *writer) printWith(n *ast.With, indent int32) { + w.print("with ") + for i, item := range n.Items { + w.printNode(item.ContextExpr, indent) + if item.OptionalVars != nil { + w.print(" as ") + w.printNode(item.OptionalVars, indent) + } + if i != len(n.Items)-1 { + w.print(", ") + } + } + w.print(":\n") + w.printBlock(n.Body, indent, false) +} + func (w *writer) printYield(n *ast.Yield, indent int32) { w.print("yield ") w.printNode(n.Value, indent) diff --git a/internal/printer/printer_test.go b/internal/printer/printer_test.go index 21823f9..5f55690 100644 --- a/internal/printer/printer_test.go +++ b/internal/printer/printer_test.go @@ -7,6 +7,7 @@ import ( "github.com/google/go-cmp/cmp" "github.com/sqlc-dev/sqlc-gen-python/internal/ast" + "github.com/sqlc-dev/sqlc-gen-python/internal/poet" ) type testcase struct { @@ -68,7 +69,10 @@ func TestPrinter(t *testing.T) { }, }, }, - Expected: `class Foo(str, enum.Enum):`, + Expected: ` +class Foo(str, enum.Enum): + pass +`, }, "dataclass": { Node: &ast.Node{ @@ -188,6 +192,141 @@ class Foo: }, Expected: `from pkg import foo, bar`, }, + "string-escape": { + Node: assign("X", poet.Constant(`say "hi" a\b`)), + Expected: `X = "say \"hi\" a\\b"`, + }, + "string-escape-multiline": { + Node: assign("X", poet.Constant("a\\b\n\"\"\"q\"")), + Expected: `X = """a\\b` + "\n" + `\"""q\""""`, + }, + "class-docstring-multiline": { + Node: poet.Node(&ast.ClassDef{ + Name: "Foo", + Body: []*ast.Node{ + poet.Expr(poet.Constant("first line\nsecond \"line\"")), + annAssign("bar", poet.Name("int"), ""), + }, + }), + Expected: ` +class Foo: + """first line +second "line\"""" + bar: int +`, + }, + "function-docstring": { + Node: poet.Node(&ast.FunctionDef{ + Name: "f", + Args: &ast.Arguments{Args: []*ast.Arg{{Arg: "self"}}}, + Body: []*ast.Node{ + poet.Expr(poet.Constant("Fetch an author.")), + poet.Return(poet.Constant(1)), + }, + }), + Expected: ` +def f(self): + """Fetch an author.""" + return 1 +`, + }, + "class-empty": { + Node: poet.Node(&ast.ClassDef{Name: "Params"}), + Expected: ` +class Params: + pass +`, + }, + "ann-assign-multiline-comment": { + Node: poet.Node(&ast.ClassDef{ + Name: "Foo", + Body: []*ast.Node{ + annAssign("bar", poet.Name("int"), "first line\nsecond line"), + }, + }), + Expected: ` +class Foo: + # first line + # second line + bar: int +`, + }, + "list-comp": { + Node: poet.ListComp( + dict(poet.Constant("p1"), poet.Attribute(poet.Name("a"), "x")), + poet.Name("a"), + poet.Name("arg"), + ), + Expected: `[{"p1": a.x} for a in arg]`, + }, + "with": { + Node: poet.With( + poet.Node(&ast.Call{ + Func: poet.Attribute(poet.Name("errors"), "_wrap_errors"), + Args: []*ast.Node{poet.Constant("q")}, + }), + poet.Return(poet.Constant(nil)), + ), + Expected: ` +with errors._wrap_errors("q"): + return None +`, + }, + "bin-op": { + Node: poet.BitOr(poet.Subscript("list", poet.Name("int")), poet.Constant(nil)), + Expected: `list[int] | None`, + }, + "subscript-tuple": { + Node: poet.Subscript("Union", poet.Name("A"), poet.Name("B")), + Expected: `Union[A, B]`, + }, + "function-ellipsis": { + Node: poet.Node(&ast.FunctionDef{ + Name: "f", + Args: &ast.Arguments{Args: []*ast.Arg{{Arg: "self"}}}, + Body: []*ast.Node{poet.Expr(poet.Ellipsis())}, + Returns: poet.Name("int"), + }), + Expected: `def f(self) -> int: ...`, + }, + "class-type-params": { + Node: poet.Node(&ast.ClassDef{ + Name: "Q", + TypeParams: []*ast.TypeVar{ + {Name: "T", Bound: poet.BitOr(poet.Name("A"), poet.Name("B"))}, + }, + Body: []*ast.Node{annAssign("_conn", poet.Name("T"), "")}, + }), + Expected: ` +class Q[T: A | B]: + _conn: T +`, + }, + "if-not": { + Node: poet.Node(&ast.If{ + Test: poet.Not(poet.Name("arg")), + Body: []*ast.Node{poet.Return(poet.Constant(0))}, + }), + Expected: ` +if not arg: + return 0 +`, + }, + "if-else": { + Node: poet.Node(&ast.If{ + Test: poet.Name("row"), + Body: []*ast.Node{poet.Expr(poet.Yield(poet.Name("row")))}, + OrElse: []*ast.Node{ + poet.Expr(poet.Yield(poet.Constant(nil))), + }, + }), + Expected: ` +if row: + yield row +else: + yield None +`, + }, } { tc := tc t.Run(name, func(t *testing.T) { @@ -198,3 +337,33 @@ class Foo: }) } } + +func assign(target string, value *ast.Node) *ast.Node { + return &ast.Node{ + Node: &ast.Node_Assign{ + Assign: &ast.Assign{ + Targets: []*ast.Node{poet.Name(target)}, + Value: value, + }, + }, + } +} + +func annAssign(target string, annotation *ast.Node, comment string) *ast.Node { + return poet.Node(&ast.AnnAssign{ + Target: &ast.Name{Id: target}, + Annotation: annotation, + Comment: comment, + }) +} + +func dict(key, value *ast.Node) *ast.Node { + return &ast.Node{ + Node: &ast.Node_Dict{ + Dict: &ast.Dict{ + Keys: []*ast.Node{key}, + Values: []*ast.Node{value}, + }, + }, + } +} diff --git a/mise.toml b/mise.toml index 183e8c2..379e1bc 100644 --- a/mise.toml +++ b/mise.toml @@ -1,2 +1,5 @@ [tools] -go = "1.19" +go = "1.27" +prek = "latest" +rumdl = "latest" +zizmor = "latest" diff --git a/openspec/changes/sqlc-feature-parity/.openspec.yaml b/openspec/changes/sqlc-feature-parity/.openspec.yaml new file mode 100644 index 0000000..75289e4 --- /dev/null +++ b/openspec/changes/sqlc-feature-parity/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-09-24 diff --git a/openspec/changes/sqlc-feature-parity/design.md b/openspec/changes/sqlc-feature-parity/design.md new file mode 100644 index 0000000..bc7567b --- /dev/null +++ b/openspec/changes/sqlc-feature-parity/design.md @@ -0,0 +1,460 @@ +# Design + +## Context + +Generation is a single pass in `internal/gen.go`. The steps run in order: +`buildEnums`, then `buildModels`, then `buildQueries` (which also works out row +and params structs and reuses a model when its columns match), then +`buildModelsTree` and `buildQueryTree`. The last two produce a Python AST +(`internal/ast`, protobuf), which `internal/printer` renders. + +Types are carried as `pyType{InnerType string, IsArray, IsNull bool}`. +`InnerType` is a dotted name such as `datetime.datetime` or `models.Status`, and +`importer` (`internal/imports.go`) scans for these names to decide which imports +each file needs. + +A few facts about the current code shape the plan: + +- `Query.Comments` is filled in but never emitted. +- `:copyfrom` returns an error. +- `Column.EmbedTable` and `Column.ArrayDims` are ignored. +- `printClassDef` prints an empty body when there are no fields. +- `AnnAssign.Comment` is printed after `#` as-is, so a newline in a comment + breaks out into code. + +sqlc forwards the top-level `options.` block of `sqlc.yaml` as +`GenerateRequest.global_options`. The plugin currently ignores it. + +The plugin SDK is already at the latest tag (`plugin-sdk-go` v1.23.0), so no +protocol upgrade is needed. Every field used below (`embed_table`, `array_dims`, +`global_options`, `comments`) is already in the vendored protobuf. + +## Goals / Non-Goals + +**Goals:** + +- Keep the one-pass pipeline. Add small, focused helpers rather than + restructuring `gen.go`. +- Resolve names and types in exactly one place each. This way rename, keyword + escaping, and overrides cannot drift between models, row construction, and + param dicts. + +**Non-Goals:** + +- Rewriting the importer or the printer beyond the fixes and the new nodes + listed below. +- Matching Go's full override schema (`go_struct_tag`, `unsigned`, the object + form of `go_type`). Only `db_type`, `column`, `nullable`, and `py_type` are in + scope. + +## Decisions + +### 1. `pyType` gets `ArrayDims int` in place of `IsArray bool` + +`Annotation()` wraps the element type in `List[...]` `ArrayDims` times. +`ArrayDims = max(col.ArrayDims, 1 if col.IsArray else 0)`, because older sqlc +versions set only `IsArray`. The struct stays comparable with `==`, so model +matching in `buildQueries` keeps working. + +The importer check for `typing.List` becomes `ArrayDims > 0`. + +### 2. Overrides are resolved inside `makePyType` + +Config parsing compiles overrides once: column patterns via `path.Match`-style +globbing on each dotted segment, and `db_type` matched with and without the +`pg_catalog.` prefix. `makePyType(req, conf, col)` checks overrides in this +order: + +1. `column` entries +2. `db_type` entries whose `nullable` matches `!col.NotNull` +3. the engine type map + +The override's `py_type` string becomes `InnerType`. + +`makePyType` is also used for parameters, because sqlc sets `Column.Table` and +`Column.Name` on parameters it can attribute to a column. That makes column +overrides flow to parameters for free. + +For imports, any `InnerType` with a dot that is not `models.*` adds +`import `, replacing the fixed list of stdlib checks. Stdlib imports and +package (third-party) imports keep separate groups, as today: known stdlib +modules go in the std group, and override modules go in the pkg group. + +As implemented, the importer no longer predicts imports from `Query`/`Struct` +data. Each generated file gets a recorder (`pyFile`) that every annotation, +`cast`, typing name and module reference goes through while the body is built; +the import groups are assembled from what was recorded. This keeps imports +correct for protocols, errors, `Sequence`, embeds and both syntaxes without a +second list to keep in sync. Existing fixtures are byte-identical. + +Two forms are accepted: + +- A dotted `py_type` with no `py_import` (`my_lib.types.Payload` → + `import my_lib.types`). This mirrors Go's `go_type: "pkg/path.Type"`. +- A bare `py_type` plus `py_import` (`UUID` + `uuid` → `from uuid import UUID`). + This is what alt-sqlc-gen-python and upstream PR #83 already use, so their + configs work unchanged. + +The importer already models `from X import Y` (`importSpec.Name`), so the second +form becomes a `from` import in the pkg group, or the std group when the module +is stdlib. + +### 3. Global options merge + +`Generate` parses `req.GlobalOptions` into the same `Config` type, keeping only +`overrides` and `rename`, and merges as sqlc's Go `opts.Parse` does: + +- overrides: `append(local, global...)`. Local entries come first, so they are + checked first and win. +- rename: global values are copied over local ones. + +The spec pins down both orders. Note that sqlc's Go codegen actually does +`append(global, local...)` with first match winning, so there global overrides +win; this plugin deliberately checks the codegen block first, as the spec says. + +### 4. One identifier function + +`pyIdent(dbName string, conf Config) string` applies `rename` first, then +escapes keywords with a trailing `_`, using a hard-coded copy of +`keyword.kwlist` for Python 3.12. It is the only source of names for: + +- `Field.Name` +- param arg names (`paramName`) +- method names +- model and enum class names (after `modelName`) + +`RowNode` and `ArgDictNode` already read `Field.Name` and `QueryValue.Name`, so +they pick up the result automatically. + +The model-reuse check in `buildQueries` compares `f.Name == columnName(c, i)`. +It changes to compare against `pyIdent(columnName(c, i))`, so renamed and +escaped models still match. + +Soft keywords (`match`, `case`, `type`, `_`) are valid identifiers and are not +escaped. + +### 5. Embeds + +In `buildQueries`, a result column with `EmbedTable != nil` resolves to the +model whose `Table` matches the embedded table (by `sdk.SameTableName`). A new +`Field.Embed *Struct` records this. + +When `Field.Embed` is set: + +- `RowNode` emits a nested `models.X(...)` call that uses `len(Embed.Fields)` + consecutive row indexes. +- A running index replaces the current `i`. +- The field's annotation is `models.X`. + +Model reuse is skipped for any query that has an embed, because the row shape +never matches a table. The scalar-return shortcut (`len(query.Columns) == 1`) is +also skipped when that single column is an embed, as in Go. + +Risk: this assumes sqlc expands each embed into the table's columns, in table +order, in the result row. The e2e fixture checks this against real `sqlc` +output. + +### 6. Copyfrom and batchexec + +`:copyfrom` and `:batchexec` share one code path. They always build a params +struct (the `qpl == 0` path) and emit the same `executemany` call. The only +differences: `:copyfrom` returns `rowcount`, and `:batchexec` returns `None`, +which matches alt-sqlc-gen-python's `:batchexec`. The `:copyfrom` method: + +```python +def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int: + if not arg: + return 0 + result = self._conn.execute(sqlalchemy.text(CREATE_AUTHORS), [{"p1": a.name, "p2": a.bio} for a in arg]) + return result.rowcount +``` + +The async version uses `await`. SQLAlchemy runs a list of parameter dicts as +`executemany`. `internal/ast` has no list-comprehension nodes (checked: +`protos/ast/ast.proto` has no `ListComp`), so they are added along with the +other new nodes (decision 14). `ast.pb.go` is regenerated with buf +(`buf.gen.yaml` pins `protocolbuffers/go:v1.30.0`). The `ArgDictNode` code is +reused with the loop variable `a` as the attribute base. + +Alternative considered: an explicit `for` loop that appends to a list. Rejected, +because it still needs a new `List` node, and it reads worse in generated code. + +Alternative considered: pgcopy or psycopg `copy()`. Rejected, because it is tied +to one driver, and the plugin is built on SQLAlchemy's `text()` and is +driver-agnostic. + +Guard: `Sequence` is imported only when a copyfrom or batch query exists, from +`typing`, or from `collections.abc` under modern syntax. + +Signature divergence from the fork: alt-sqlc-gen-python's `:batchexec` takes a +keyword-only `args: list[Params]`. Here every sequence-taking command takes one +positional `arg: Sequence[Params]`, to stay consistent with `:copyfrom` and the +single-struct convention in `Query.AddArgs`. The README calls this out for users +migrating from the fork. + +### 7. Query docstrings + +`Query.Comments` becomes the first body statement of each querier method: an +`Expr(Constant(strings.Join(comments, "\n")))`. `printFunctionDef` and +`printAsyncFunctionDef` need the same docstring handling `printClassDef` already +has, printed as `"""..."""`. Leading single spaces left over from `--` are +trimmed, and no other text changes. + +### 8. Printer fixes + +- `printClassDef`: when `len(cd.Body) == 0`, print an indented `pass`. This + covers params and row classes and a zero-column table. +- Comments on `AnnAssign`: when `Comment` contains `\n`, print the first line + inline as today. Each later line prints on its own line at the same indent, + prefixed with `# `. +- `printConstant` currently writes string contents unescaped, so a `"` in an + enum value, comment, or docstring produces invalid Python. Backslashes and the + active quote character are escaped. Docstrings are printed through one helper + that always uses `"""` and escapes `\` and `"""`. This replaces the current + `""` + constant + `""` trick, which yields five quotes for multi-line + comments. `sqlalchemySQL` and the `-- name:` header in `buildQueryTree` + currently write the backslash already doubled for Python (they emit `\\:`). + Once the printer escapes backslashes, they switch to emitting the runtime text + that SQLAlchemy needs (`\:`), and the printer doubles it. This keeps the + generated SQL constants byte-identical, and the existing fixtures verify that. + +### 9. Enum member names + +`pyEnumValueName` stays the sanitizer. `buildEnums` then post-processes the +members of each enum: + +1. empty → `VALUE_` +2. leading digit → `VALUE_` + name +3. keyword collision (for example `NONE` is fine, but `None` cannot happen after + upper-casing) → no-op, kept only as a check +4. duplicates → `_2`, `_3`, and so on, in value order + +### 10. `omit_unused_structs` + +This is a port of Go's `filterUnusedStructs`, run after `buildQueries`. + +1. Collect a keep-set of `InnerType` names (with `models.` removed) from every + query's args, return value, embed fields, and emitted struct fields. +2. Add the enums used by kept models' fields. +3. Filter `models` and `enums` by the keep-set. + +The filtered lists feed both `importer` and `tctx`. + +### 11. `omit_sqlc_version` + +`moduleNode` takes a flag and skips the two version lines. + +### 12. Type aliases + +This adds the missing Go spellings to the existing `switch` in +`postgresql_type.go`. There is no structural change. `json`/`jsonb` keep `Any`, +with only the `pg_catalog.` aliases added. + +### 13. Tooling + +- `mise.toml` pins Go to `1.23`, and adds `buf` and a Python 3.12 with `mypy` + for fixture type checks. +- CI (`.github/workflows/*.yml`) moves to sqlc `1.31.1`. +- All fixtures and `examples/` are regenerated with `sqlc generate` so that + `sqlc diff` stays clean. +- `examples/sqlc.yaml` points at the locally built + `file://../bin/sqlc-gen-python.wasm` (no sha256, which sqlc then computes) + instead of the released 1.2.0 WASM, so CI's `sqlc diff` checks this tree. + +`go.mod` keeps `go 1.19` as its language version. Raising it is not needed, and +leaving it avoids forcing downstream builds to change. + +### 14. New AST nodes + +One proto change, regenerated once, covers every feature that needs new syntax: + +- `ListComp` and `Comprehension`, for copyfrom, batchexec, and batchmany +- `With` and `WithItem`, for error wrapping +- `BinOp` with a `BitOr` operator, for `X | None` +- a `Constant.ellipsis` oneof case, for protocol stubs +- `ClassDef.type_params` (a repeated `TypeVar{name, bound}`), printed as + `[_ConnT: ]` after the class name, for generic queriers +- `Tuple`, for `Union[A, B]` subscripts +- `UnaryOp` with a `Not` operator, for the `if not arg:` guard of + `:copyfrom`/`:batchexec` + +The printer gets a case for each. `If.or_else` already exists in the proto, but +`printer.go` never prints it (it would drop an `else:` silently), so the printer +gets `else` support, which `:batchone` needs. + +Alternative considered: encoding `X | None` or `...` as raw `Name` text. +Rejected, because nested annotations (`list[X | None] | None`) then need string +building in Go, and that is where precedence bugs hide. + +### 15. Type syntax is a render-time switch + +`pyType` and `QueryValue` keep a structural form: inner type, `ArrayDims`, +`IsNull`. `Annotation()` becomes `Annotation(syntax typeSyntax)`, rendering +either `Optional[List[X]]` or `list[X] | None`. It is the only place annotations +are built. The generic wrappers used in method returns and params (`Iterator`, +`AsyncIterator`, `Sequence`, `Optional`) go through one helper that picks the +module (`typing` or `collections.abc`) and spelling. The importer asks the same +helper which names it used, instead of keeping its own list of `typing.*` names. + +`cast()` targets reuse `Annotation(syntax)`, so they always match the field +annotation. With modern syntax, the `cast(str | None, ...)` expressions evaluate +`|` at runtime, which is fine on 3.10+, the documented minimum for that option. + +### 16. Aware datetimes + +The switch lives in `postgresType`: it receives `conf`, and returns +`pydantic.AwareDatetime` for `timestamptz` spellings when `emit_aware_datetime` +is set. Overrides run before the type map (decision 2), so they still win. +`Config` validation runs right after parsing and rejects `emit_aware_datetime` +without `emit_pydantic_models`. `pydantic` stays in the first import group, +where `import pydantic` for `BaseModel` has always been printed, so existing +pydantic fixtures do not change. The importer (decision 2) picks it up from the +dotted name. + +### 17. Batch one/many + +Both commands loop over `arg` in the method body, so the method is a (sync or +async) generator, which gives the lazy semantics the spec requires: + +- `:batchone`: `row = .first()`, then `if row is None: yield None` / + `else: yield `. +- `:batchmany`: `yield [ for row in ]`. Async uses + `await self._conn.execute(...)` rather than `stream`, because each item's rows + are collected into a list anyway. + +The per-item param dict reuses `ArgDictNode` with base `a`, as in decision 6. +Both queriers share one body builder that takes an `async bool`, instead of +duplicating the switch arms. This also shrinks the existing sync/async +duplication in `buildQueryTree` for the arms it touches. + +### 18. Queriers accept sessions + +The `__init__` annotation and a class-level `_conn: ` annotation use: + +- sync: `Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]`, or `|` + under modern syntax +- async: + `sqlalchemy.ext.asyncio.AsyncConnection | sqlalchemy.ext.asyncio.AsyncSession` + +`Session.execute` and `AsyncSession.execute`/`stream` accept the same `text()` +and parameter arguments, so method bodies do not change. One typing consequence: +`Session.execute` is typed as returning `Result`, which has no `rowcount`, so +`:execrows` and `:copyfrom` read +`cast("sqlalchemy.engine.CursorResult[Any]", result).rowcount`, and +`:execresult` returns `"sqlalchemy.engine.Result[Any]"`. Both are strings so no +subscripted SQLAlchemy type is evaluated at runtime. + +With `emit_generic_querier`, the same union becomes the bound of a PEP 695 type +parameter: `class Querier[_ConnT: ]`, with `_conn: _ConnT` and `conn: _ConnT`. The union +is spelled by `Annotation(syntax)` (decision 15). The class body is otherwise +identical, so the method builder (task 6.1) takes no notice of the flag, and +only `querierClassDef`/`asyncQuerierClassDef` branch on it. It is opt-in, not +the default, because PEP 695 syntax is a `SyntaxError` before Python 3.12. + +Alternative considered: a pre-3.12 `T = TypeVar("T", bound=...)` plus +`Generic[T]`. Rejected, because it would add a module-level TypeVar per file and +a second spelling to maintain, for a feature whose users are already on 3.12. + +### 19. Protocols + +`buildQueryTree` builds each querier's `FunctionDef`s first. When +`emit_querier_protocol` is set, it derives the protocol from them: same name, +args, and returns, and a body of `Expr(Constant(ellipsis))`, with no docstring. +Protocol methods for async generators (`:many`, `:batchone`, `:batchmany`) are +emitted as a plain `FunctionDef`, not `AsyncFunctionDef`. An async-generator +function's call type is `AsyncIterator[T]`, whereas +`async def f() -> AsyncIterator[T]` in a stub means +`Coroutine[..., AsyncIterator[T]]`, so mypy would reject the real querier. +Protocols come before the concrete classes in the file. The concrete classes do +not subclass them; conformance is structural. A fixture-level mypy check (a +`_check.py` that assigns each querier to its protocol) proves the match. + +### 20. Query errors + +`errors.py` shares the header, version and import logic with the other modules +(printed through the AST), followed by a fixed text template whose +`Optional`/`Iterator` spellings follow the active syntax. It matches the fork's +classes and its `_wrap_integrity_error` and `_wrap_operational_error` helpers, +and adds: + +- `constraint_name`, read best-effort from `orig.diag.constraint_name` (psycopg2 + and 3) or `orig.__cause__.constraint_name` (asyncpg through SQLAlchemy's + adapter), falling back to `None` +- a `_wrap_errors(query_name)` context manager (`contextlib.contextmanager`) + that catches `IntegrityError` and `OperationalError` and re-raises through the + two helpers `from e` + +SQLSTATE is read as +`getattr(orig, "pgcode", None) or getattr(orig, "sqlstate", None)`, which covers +psycopg2 (`pgcode`) and psycopg 3 and asyncpg (`sqlstate`). + +Each querier method body is wrapped in `with errors._wrap_errors(""):`. +A sync context manager works inside both `def` and `async def`. Because the +`with` encloses the whole body, including `for`/`async for` and `yield`, errors +raised while the caller iterates are wrapped too. The fork wraps only the +`execute` call. When the consumer stops iterating early, `GeneratorExit` passes +through the context manager untouched, because only the two SQLAlchemy types are +caught. + +Query files add `from import errors` next to `models`. `Generate` +fails if any query source maps to `errors.py`. + +Alternative considered: try/except in each method, as the fork does. Rejected: +it needs `Try` and `ExceptHandler` AST nodes, and it repeats the handler in +every method, where one `With` line does the job. + +### 21. Existing options are covered by a matrix, not by code changes + +`emit_sync_querier`, `emit_async_querier`, `emit_pydantic_models`, and +`emit_str_enum` need no new logic. The risk is that new features forget one of +them, for example emitting a dataclass for a batch params class in pydantic +mode. Two "kitchen-sink" fixtures use one schema and query set that exercises +every feature (embed, copyfrom, all batch commands, comments, enums, keywords, +and timestamptz): + +- `feature_matrix_classic`: both queriers, dataclasses, `(str, enum.Enum)`, + typing syntax, protocols and errors on. +- `feature_matrix_modern`: both queriers, pydantic, `StrEnum`, modern syntax, + aware datetimes, generic queriers, protocols and errors on. + +Each is checked with `py_compile` and `mypy --strict`, plus pytest against +Postgres in `examples/`. + +## Risks / Trade-offs + +- [Output churn from keyword escaping, type aliases, and the sqlc version header + across every fixture] → Regenerate fixtures in a separate commit before any + feature commits, so reviewers can tell header churn from real changes. +- [The `rowcount` from `executemany` depends on the driver. Some DBAPIs report + `-1`] → Document this in the README. The generated code returns what + SQLAlchemy reports and does not guess. +- [Existing users with a column named like a keyword get a renamed attribute + (`from` → `from_`)] → That code could not have imported before, so nothing + that worked breaks. Call it out in the release notes anyway. +- [`cast()` and the widened `_conn` types change every fixture and every user's + generated output] → They are annotation-only. Land them in their own commit, + after the version-bump regeneration, so their diff is reviewable alone. +- [`constraint_name` extraction depends on driver internals (`diag`, + `__cause__`)] → Best-effort with a `None` fallback. The examples' pytest suite + exercises psycopg and asyncpg, so a driver change shows up as a failing test, + not a crash. +- [`emit_generic_querier` output fails to import on Python < 3.12, and older + mypy rejects PEP 695 syntax] → Documented as a requirement of the option. CI + runs the fixture checks on Python 3.12 with a current mypy. +- [Protocol and implementation drift] → Both come from the same `FunctionDef` + list (decision 19), and mypy checks the fixtures. +- [`:batchone`/`:batchmany` run N round-trips] → Documented. Users who need bulk + reads should use `= ANY($1::int[])` with `:many`. +- [Positional mapping for embeds breaks if sqlc changes the expansion order] → + The e2e fixture catches it on the next sqlc bump. +- [New AST nodes need buf codegen, and `buf` is not installed locally] → Add + `buf` to `mise.toml` tools, and regenerate `ast.pb.go` in its own task and + commit. + +## Migration Plan + +Release this as the next minor version (1.4.0). Every new option defaults to +off, so the only upgrade impact is the correctness fixes, type aliases, `cast()` +wrappers, and widened `_conn` annotations in generated output, and users see +those as a diff when they run `sqlc generate`. To roll back, pin the previous +WASM URL and sha256. diff --git a/openspec/changes/sqlc-feature-parity/proposal.md b/openspec/changes/sqlc-feature-parity/proposal.md new file mode 100644 index 0000000..db63c4f --- /dev/null +++ b/openspec/changes/sqlc-feature-parity/proposal.md @@ -0,0 +1,155 @@ +# Proposal + +## Why + +sqlc-gen-python has not tracked sqlc since v1.28.0. sqlc's own Go codegen (at +v1.31.1) and the plugin protocol now carry features the Python plugin ignores: +type overrides, renames, `sqlc.embed()`, `:copyfrom`, query comments, and more. +Some inputs also produce Python that will not import, such as a column named +`from` or an empty params class. Users hit these walls and work around them, or +wait on unmerged upstream PRs (#83, #89, #92, #95, #100, #103, #105). + +## What Changes + +**Config parity with sqlc's Go codegen** + +- New `overrides` option: map a `db_type` or a `column` to a `py_type`, given + either as a dotted Python path or as `py_type` plus `py_import` (the form + alt-sqlc-gen-python uses). The plugin adds the import. It also reads overrides + from sqlc's global `options`. +- New `rename` option: map a database identifier to the Python name used for a + class, field, or enum member. +- New `omit_unused_structs` option: leave out enums and models that no query + references. +- New `omit_sqlc_version` option: leave the sqlc version out of the file header. + +**New opt-in output styles** + +- New `emit_modern_types` option: emit Python 3.10+ annotations (`X | None`, + `list[X]`, and `collections.abc` iterators and sequences) instead of + `Optional`, `List`, and the `typing` aliases. +- New `emit_aware_datetime` option: annotate `timestamptz` columns and + parameters as `pydantic.AwareDatetime`. It requires `emit_pydantic_models`. +- New `emit_generic_querier` option: emit `Querier` and `AsyncQuerier` as PEP + 695 generic classes (`class Querier[_ConnT: Connection | Session]`) with + `_conn: _ConnT`, so `Querier(session)._conn` is typed as `Session`. It requires + Python 3.12+. +- New `emit_querier_protocol` option: emit `QuerierProtocol` and + `AsyncQuerierProtocol` (`typing.Protocol`) classes. They mirror the generated + queriers, so a fake can stand in for one in tests. +- New `emit_query_errors` option: generate an `errors.py` module of typed + exceptions, all subclasses of `QueryError`. Querier methods re-raise + constraint violations (`UniqueViolationError`, `ForeignKeyViolationError`, + `CheckViolationError`, `NotNullViolationError`, `ExclusionViolationError`) and + operational failures (`StatementTimeoutError`, `DeadlockError`, + `SerializationError`) as the matching type, carrying `query_name`, `cause`, + and `constraint_name`. The names match alt-sqlc-gen-python's option of the + same name, so code written against that fork keeps working. + +**Query features** + +- `sqlc.embed(table)`: the row class gets one field typed as the table's model, + filled from that table's columns. +- Comments on a query (the `--` lines above `-- name:`) become the docstring of + each querier method. +- `:copyfrom`: emits a method that takes a sequence of params objects and runs + one `executemany`. It returns the row count. Today this command fails + generation. +- `:batchexec`: emits a method that takes a sequence of params objects and + writes them all with one `executemany`. +- `:batchone`, `:batchmany`: emit a method that takes a sequence of params + objects and runs the statement once per item. Results come back lazily, one + per input, in input order. +- Today all three batch commands fail generation with a panic. +- Queriers also accept `sqlalchemy.orm.Session` and `AsyncSession`, not only + connections, and declare a typed `_conn`. +- Values read from result rows are wrapped in `typing.cast(, row[i])`, so + strict type checkers accept the generated code. +- Multi-dimensional array columns (`int[][]`) emit nested `List[...]` types. + +**Existing options keep working with every new feature** + +- `emit_sync_querier`, `emit_async_querier`, `emit_pydantic_models`, and + `emit_str_enum` get specs and fixtures. Every new query feature (embed, + copyfrom, batch, docstrings, typed errors, protocols) must generate valid, + type-consistent code under each of them. + +**Correctness fixes (generated output may change)** + +- Identifiers that are Python keywords (`from`, `class`, …) get a trailing `_`. +- A class with no fields gets a `pass` body instead of an empty body, which is a + syntax error. +- Enum values that sanitize to an empty, duplicate, or digit-leading name get a + valid unique member name. +- Multi-line table and column comments print as valid multi-line Python + comments. +- String literals (enum values, docstrings) escape backslashes and quotes, so a + `"` in an enum value or comment no longer breaks the module. +- PostgreSQL type aliases that sqlc's Go codegen maps but this plugin types as + `Any` get real types: `varchar`, `character varying`, `bpchar`, `character`, + `name`, `timestamp`, `time`, `timetz`, `pg_catalog.json`/`jsonb`, and the + `… with/without time zone` spellings. + +**Tooling** + +- CI, examples, and test fixtures move to sqlc v1.31.1. Local Go moves to 1.23 + because `GOOS=wasip1` does not build on Go 1.19. + +Behavior that is not a bug fix sits behind an option and is off by default. Two +exceptions, both annotation-only with no runtime effect: the widened querier +connection types and the `cast()` wrappers. Bug fixes and type-map gains change +generated output for affected schemas. Code that relied on those outputs (for +example, `Any` for a `varchar` column) will see different annotations. + +## Non-goals + +- MySQL and SQLite engine support, and `sqlc.slice()`. +- Real driver pipelining for `:batchone`/`:batchmany`. Their items run one + statement at a time over the given connection. +- Always-on modern syntax, aware datetimes, and PEP 695 generic queriers, as + alt-sqlc-gen-python does them. Here each is opt-in, so Python 3.9 users see no + change. +- PostgreSQL range and multirange types. They stay `Any`, and overrides can type + them. +- Changing `bytea` from `memoryview` to `bytes`, or changing async `:many` from + `stream` to `execute`. + +## Capabilities + +### New Capabilities + +- `codegen-config`: the plugin options `overrides`, `rename`, + `omit_unused_structs`, `omit_sqlc_version`, `emit_pydantic_models`, + `emit_str_enum`, `emit_modern_types`, and `emit_aware_datetime`, and how + global options merge with them. +- `query-codegen`: how queries become Python: sync and async queriers, querier + protocols, typed error wrapping, embedded tables, query docstrings, + `:copyfrom`, batch commands, and multi-dimensional arrays. +- `generated-code-validity`: guarantees that generated modules are valid Python + (identifiers, empty classes, enum member names, comments). +- `postgresql-type-mapping`: the PostgreSQL-to-Python type map, including the + added aliases and the aware-datetime mapping. + +### Modified Capabilities + + +## Impact + +- Code: `internal/config.go`, `internal/gen.go`, `internal/imports.go`, + `internal/postgresql_type.go`, `internal/printer/printer.go`, and + `protos/ast/ast.proto` (new `ListComp`, `With`, `BinOp`, and ellipsis nodes). + New helper files as needed. +- Generated package: can now include an `errors.py` module (with + `emit_query_errors`). A query file named `errors.sql` then conflicts with it + and fails generation. +- Runtime requirements: `emit_modern_types` needs Python 3.10+, + `emit_generic_querier` needs Python 3.12+ (and mypy 1.12+ or a recent pyright + to type-check), and `emit_aware_datetime` needs pydantic 2+. +- Tests: new `internal/endtoend/testdata/*` fixtures for each feature. Existing + fixture headers get regenerated for sqlc v1.31.1. +- Docs: new README sections for each option. +- Tooling: `.github/workflows/*.yml` (sqlc 1.31.1), `mise.toml` (Go 1.23), and + regenerated `examples/`. +- Compatibility: generated output changes as listed under the correctness fixes. + The config stays backward compatible, because every new option defaults to off + or empty. diff --git a/openspec/changes/sqlc-feature-parity/specs/codegen-config/spec.md b/openspec/changes/sqlc-feature-parity/specs/codegen-config/spec.md new file mode 100644 index 0000000..4801d21 --- /dev/null +++ b/openspec/changes/sqlc-feature-parity/specs/codegen-config/spec.md @@ -0,0 +1,261 @@ +# Spec Delta + +## Purpose + +Defines the plugin's generation options: the ones that match sqlc's Go codegen +(type overrides, renames, omitting unused models, omitting the sqlc version) and +the Python output styles (pydantic models, `StrEnum`, modern type syntax, aware +datetimes). Also defines how per-codegen options combine with sqlc's global +options. + +## ADDED Requirements + +### Requirement: Type overrides by database type + +The plugin SHALL accept an `overrides` list. An entry with `db_type` and +`py_type` SHALL replace the Python type of every column and parameter whose +database type matches `db_type`. This SHALL apply in models, params classes, row +classes, and method signatures. Matching SHALL follow sqlc's rules: a `db_type` +given without a schema also matches the `pg_catalog`-qualified name. By default +an entry SHALL apply only to non-null columns. With `nullable: true` it SHALL +apply only to nullable columns. A nullable column that an entry matches SHALL +keep its `Optional[...]` wrapper, and an array column SHALL keep its `List[...]` +wrapper around the override type. + +#### Scenario: Override a non-null jsonb column + +- **WHEN** `overrides: [{db_type: jsonb, py_type: my_lib.types.Payload}]` is set + and table `events` has `payload jsonb NOT NULL` +- **THEN** the `Event` model declares `payload: my_lib.types.Payload` + +#### Scenario: Override does not apply to nullable columns by default + +- **WHEN** the same override is set and a column is `extra jsonb` (nullable) +- **THEN** that column keeps its default type `Optional[Any]` + +#### Scenario: Nullable override + +- **WHEN** an entry sets `db_type: jsonb`, `py_type: my_lib.types.Payload`, + `nullable: true` and a column is `extra jsonb` +- **THEN** the column is declared `extra: Optional[my_lib.types.Payload]` + +### Requirement: Type overrides by column + +An `overrides` entry with `column` and `py_type` SHALL replace the type of that +column. The `column` value SHALL have the form `table.column` or +`schema.table.column`. `*` SHALL match any run of characters within one segment. +When a column matches both a `column` entry and a `db_type` entry, the `column` +entry SHALL win. Parameters that sqlc attributes to a column SHALL get the same +type as the column. + +#### Scenario: Column override wins over db_type override + +- **WHEN** overrides contain `{db_type: uuid, py_type: my_ids.Id}` and + `{column: "users.id", py_type: my_ids.UserId}` +- **THEN** `User.id` is `my_ids.UserId`, and every other `uuid` column is + `my_ids.Id` + +#### Scenario: Parameter follows column override + +- **WHEN** `users.id` is overridden to `my_ids.UserId` and a query has + `WHERE id = $1` +- **THEN** the generated method's `id` parameter is annotated `my_ids.UserId` + +### Requirement: Override imports + +When `py_type` is a dotted path, the plugin SHALL add `import ` to every +generated file that uses the type, where `` is the path without its last +segment. The annotation SHALL be the full dotted path. When `py_type` has no dot +(for example `str` or `bytes`) and no `py_import` is given, the plugin SHALL NOT +add an import. + +An entry MAY also set `py_import`, as alt-sqlc-gen-python and upstream PR #83 +do. Then the plugin SHALL add `from import ` to every file +that uses the type, and the annotation SHALL be the bare `py_type`. When +`py_import` is set, `py_type` SHALL be a single identifier, with no dot. + +#### Scenario: Import added only where used + +- **WHEN** `my_lib.types.Payload` appears in `models.py` and no query file uses + it +- **THEN** `models.py` contains `import my_lib.types` and the query files do not + +#### Scenario: py_import form + +- **WHEN** `{column: "authors.id", py_type: UUID, py_import: uuid}` is set +- **THEN** `Author.id` is annotated `UUID`, and `models.py` contains + `from uuid import UUID` + +#### Scenario: Builtin override needs no import + +- **WHEN** `{db_type: bytea, py_type: bytes}` is set +- **THEN** `bytea` columns are annotated `bytes` and no import is added for them + +### Requirement: Global overrides + +The plugin SHALL read `overrides` and `rename` from sqlc's global options for +this plugin as well as from the codegen `options`. For overrides, entries from +the codegen `options` SHALL be checked before global entries. For renames, a key +present in both SHALL use the global value, which matches sqlc's Go codegen. + +#### Scenario: Global override applies + +- **WHEN** a global `overrides` entry maps `db_type: citext` to `my_lib.CIText` + and the codegen options have no overrides +- **THEN** `citext` columns are annotated `my_lib.CIText` + +### Requirement: Invalid override entries fail generation + +An override entry SHALL fail generation with an error that names the entry when +it sets neither `db_type` nor `column`, sets both, has an empty `py_type`, or +sets `py_import` with a dotted `py_type`. + +#### Scenario: Missing target + +- **WHEN** an entry has only `py_type: str` +- **THEN** `sqlc generate` fails with an error that identifies the invalid + override + +### Requirement: Rename identifiers + +The plugin SHALL accept a `rename` map from database identifier to Python +identifier. Keys SHALL match: + +- the singularized table name (or the exact name when `emit_exact_table_names` + is set), for model class names +- the enum type name, for enum class names +- the column name, for field and parameter names +- the enum value, for enum member names + +The renamed identifier SHALL appear in every place that refers to it, including +row-to-object construction and parameter dictionaries. + +#### Scenario: Rename a field + +- **WHEN** `rename: {spotify_url: spotify_link}` is set and table `artists` has + column `spotify_url` +- **THEN** the `Artist` model declares `spotify_link`, and queries that return + it build `Artist(..., spotify_link=cast(Optional[str], row[n]))` + +#### Scenario: Rename a model + +- **WHEN** `rename: {person: Human}` is set for table `people` +- **THEN** the model class is named `Human`, and query return types refer to + `models.Human` + +### Requirement: Omit unused structs + +When `omit_unused_structs` is true, `models.py` SHALL contain only the enums and +models that some query's parameters, result columns, or embedded tables refer +to, directly or through a field type. When it is false or unset, output SHALL be +unchanged. + +#### Scenario: Unused table dropped + +- **WHEN** the schema has tables `authors` and `audit_log`, only `authors` is + queried, and `omit_unused_structs: true` +- **THEN** `models.py` defines `Author` and does not define `AuditLog` + +#### Scenario: Enum kept through a model field + +- **WHEN** an enum `status` is used only as a column of a queried table +- **THEN** the `Status` enum is still emitted + +### Requirement: Omit sqlc version + +When `omit_sqlc_version` is true, generated files SHALL NOT contain the +`# versions:` and `# sqlc ` header lines. The other header lines +SHALL stay. + +#### Scenario: Header without version + +- **WHEN** `omit_sqlc_version: true` +- **THEN** each generated file starts with + `# Code generated by sqlc. DO NOT EDIT.` and contains no line that mentions + the sqlc version + +### Requirement: Pydantic models + +With `emit_pydantic_models: true`, every model, params class, and row class +SHALL subclass `pydantic.BaseModel`, and the file SHALL `import pydantic`. +Otherwise these classes SHALL be `@dataclasses.dataclass()` classes. The choice +SHALL apply the same way to classes produced by every feature in this change: +embed row classes, `:copyfrom` and batch params, and renamed or keyword-escaped +fields. + +#### Scenario: Pydantic embed row + +- **WHEN** `emit_pydantic_models: true` and a query uses `sqlc.embed(authors)` +- **THEN** the row class is a `pydantic.BaseModel` whose `authors` field is + typed `models.Author`, and constructing it from a query result passes pydantic + validation + +#### Scenario: Default dataclasses + +- **WHEN** `emit_pydantic_models` is unset +- **THEN** `models.py` imports `dataclasses` and decorates every model with + `@dataclasses.dataclass()` + +### Requirement: StrEnum enums + +With `emit_str_enum: true`, every enum class SHALL subclass `enum.StrEnum`. +Otherwise every enum class SHALL subclass `(str, enum.Enum)`. Enum member +naming, renames, and value escaping SHALL behave the same either way. + +#### Scenario: StrEnum with sanitized members + +- **WHEN** `emit_str_enum: true` and an enum has values `in-progress` and `=` +- **THEN** the class is `class Status(enum.StrEnum)` with members + `IN_PROGRESS = "in-progress"` and `VALUE_2 = "="` + +### Requirement: Modern type syntax + +With `emit_modern_types: true`, generated annotations SHALL use Python 3.10+ +syntax: + +- `X | None` instead of `Optional[X]` +- `list[X]` instead of `List[X]` +- `Iterator`, `AsyncIterator`, and `Sequence` imported from `collections.abc` + instead of `typing` +- `Union` replaced with `|` everywhere, including the querier `_conn` + annotations + +`Optional` and `List` SHALL then not be imported. `typing` SHALL still supply +`Any`, `cast`, and `Protocol` when they are used. With the option off, +annotations SHALL use the current `typing` forms. The option SHALL apply to +models, params and row classes, querier signatures, protocols, `cast` targets, +and `errors.py`. + +#### Scenario: Nullable array field + +- **WHEN** `emit_modern_types: true` and a column is `tags text[]` (nullable) +- **THEN** it is annotated `list[str] | None`, and the file does not import + `Optional` or `List` + +#### Scenario: Iterator import + +- **WHEN** `emit_modern_types: true` and a sync `:many` query exists +- **THEN** the query file contains `from collections.abc import Iterator` + +### Requirement: Aware datetimes + +With `emit_aware_datetime: true`, `timestamptz` columns and parameters (all +spellings listed in the type-mapping capability) SHALL be annotated +`pydantic.AwareDatetime`. Other types SHALL be unchanged; `timestamp` without +time zone stays `datetime.datetime`. Nullability and array wrappers SHALL apply +as usual. Setting `emit_aware_datetime: true` without +`emit_pydantic_models: true` SHALL fail generation, with an error saying that +`emit_aware_datetime` requires `emit_pydantic_models`. A matching override SHALL +take precedence over this option. + +#### Scenario: Aware timestamp field + +- **WHEN** `emit_aware_datetime` and `emit_pydantic_models` are true and a + column is `created_at timestamptz NOT NULL` +- **THEN** it is annotated `pydantic.AwareDatetime`, and constructing the model + with a naive `datetime` raises a pydantic `ValidationError` + +#### Scenario: Requires pydantic + +- **WHEN** `emit_aware_datetime: true` is set without `emit_pydantic_models` +- **THEN** `sqlc generate` fails with an error naming both options diff --git a/openspec/changes/sqlc-feature-parity/specs/generated-code-validity/spec.md b/openspec/changes/sqlc-feature-parity/specs/generated-code-validity/spec.md new file mode 100644 index 0000000..ebac109 --- /dev/null +++ b/openspec/changes/sqlc-feature-parity/specs/generated-code-validity/spec.md @@ -0,0 +1,93 @@ +# Spec Delta + +## Purpose + +Guarantees that every generated module is syntactically valid Python that +imports cleanly, whatever the identifiers, comments, and enum values in the +schema are. + +## ADDED Requirements + +### Requirement: Python keywords are escaped + +Any generated field, parameter, or method name that is a Python keyword SHALL +get a trailing underscore (for example `from` becomes `from_`, and `class` +becomes `class_`). A trailing underscore SHALL also apply when the name was set +through `rename`. SQL text and parameter placeholders SHALL be unchanged. + +#### Scenario: Keyword column + +- **WHEN** a table has a column named `from` +- **THEN** the model declares `from_`, and queries that return the column build + the model with `from_=cast(..., row[n])` + +#### Scenario: Keyword parameter + +- **WHEN** a query parameter is named `class` +- **THEN** the method has the keyword-only parameter `class_`, and the parameter + dict maps the placeholder to `class_` + +### Requirement: Empty class bodies + +A generated class with no fields and no docstring SHALL have the body `pass`. + +#### Scenario: Params class for a query without parameters + +- **WHEN** `query_parameter_limit: 0` is set and a query has no parameters +- **THEN** the generated params class body is `pass`, and the module imports + without a `SyntaxError` + +### Requirement: Valid enum member names + +Every enum member name SHALL be a valid, non-keyword Python identifier, and +member names within one enum SHALL be unique. A value that sanitizes to an empty +string SHALL be named `VALUE_`, where `` is the value's 1-based position. +A name that starts with a digit SHALL get the prefix `VALUE_`. A name that +duplicates an earlier member SHALL get the suffix `_`, starting at `_2`. +Member values SHALL keep the original strings. + +#### Scenario: Symbol-only value + +- **WHEN** an enum has values `=` and `<>`, in that order +- **THEN** the members are `VALUE_1 = "="` and `VALUE_2 = "<>"` + +#### Scenario: Colliding values + +- **WHEN** an enum has values `in-progress` and `in_progress` +- **THEN** the members are `IN_PROGRESS = "in-progress"` and + `IN_PROGRESS_2 = "in_progress"` + +#### Scenario: Digit-leading value + +- **WHEN** an enum has the value `1st` +- **THEN** the member is `VALUE_1ST = "1st"` + +### Requirement: Multi-line comments + +A table, column, or enum comment that spans more than one line SHALL be emitted +so that each line stays inside a Python comment or docstring. No comment text +SHALL appear as bare code. + +#### Scenario: Multi-line column comment + +- **WHEN** a column comment is `"first line\nsecond line"` +- **THEN** the field is preceded by `# first line` and a second line + `# second line`, and the module parses + +### Requirement: String literals are escaped + +Every string literal the plugin emits SHALL be a valid Python literal that +evaluates to exactly the source text. This covers enum values, docstrings from +table, enum, and query comments, and SQL text. Backslashes and quote characters +in the source text SHALL be escaped. + +#### Scenario: Enum value with a double quote + +- **WHEN** an enum has the value `say "hi"` +- **THEN** the member's value evaluates in Python to the string `say "hi"` + +#### Scenario: Multi-line table comment + +- **WHEN** a table comment spans two lines +- **THEN** the model's docstring is one triple-quoted string that contains both + lines diff --git a/openspec/changes/sqlc-feature-parity/specs/postgresql-type-mapping/spec.md b/openspec/changes/sqlc-feature-parity/specs/postgresql-type-mapping/spec.md new file mode 100644 index 0000000..f2fce55 --- /dev/null +++ b/openspec/changes/sqlc-feature-parity/specs/postgresql-type-mapping/spec.md @@ -0,0 +1,53 @@ +# Spec Delta + +## Purpose + +Defines which Python type annotation the plugin emits for each PostgreSQL type, +aligned with the spellings that sqlc's Go codegen recognizes. + +## ADDED Requirements + +### Requirement: PostgreSQL type aliases + +The plugin SHALL map these PostgreSQL type spellings to these Python types. It +SHALL do so whether or not the spelling is qualified with `pg_catalog.`: + +| PostgreSQL spellings | Python type | +|---|---| +| `text`, `varchar`, `character varying`, `bpchar`, `character`, `name`, `citext`, `string` | `str` | +| `time`, `timetz`, `time without time zone`, `time with time zone` | `datetime.time` | +| `timestamp`, `timestamptz`, `timestamp without time zone`, `timestamp with time zone` | `datetime.datetime` | +| `json`, `jsonb` | `Any` | + +Existing mappings for the other types SHALL stay the same. With +`emit_aware_datetime: true`, the `timestamptz` spellings (`timestamptz`, +`timestamp with time zone`) SHALL map to `pydantic.AwareDatetime` instead. The +codegen-config capability defines that option. + +#### Scenario: Unqualified varchar + +- **WHEN** a column's type reaches the plugin as `varchar` +- **THEN** it is annotated `str`, not `Any` + +#### Scenario: Timestamp without time zone + +- **WHEN** a column's type reaches the plugin as `timestamp without time zone` +- **THEN** it is annotated `datetime.datetime` + +#### Scenario: Aware mapping only for timestamptz + +- **WHEN** `emit_aware_datetime` and `emit_pydantic_models` are true, and a + table has `a timestamptz NOT NULL` and `b timestamp NOT NULL` +- **THEN** `a` is annotated `pydantic.AwareDatetime` and `b` is annotated + `datetime.datetime` + +### Requirement: Unknown types fall back to Any + +Any type that has no mapping, is not a known enum, and has no matching override +SHALL be annotated `Any`, and the plugin SHALL log it as an unknown PostgreSQL +type. + +#### Scenario: Range type + +- **WHEN** a column is `during tstzrange` +- **THEN** it is annotated `Any` diff --git a/openspec/changes/sqlc-feature-parity/specs/query-codegen/spec.md b/openspec/changes/sqlc-feature-parity/specs/query-codegen/spec.md new file mode 100644 index 0000000..efe48e7 --- /dev/null +++ b/openspec/changes/sqlc-feature-parity/specs/query-codegen/spec.md @@ -0,0 +1,334 @@ +# Spec Delta + +## Purpose + +Defines how sqlc queries become Python querier code: sync and async queriers and +their protocols, typed error wrapping, typed row values, embedded tables, query +comments, bulk inserts with `:copyfrom`, batch commands, and multi-dimensional +array types. + +## ADDED Requirements + +### Requirement: Embedded tables + +When a query's result uses `sqlc.embed()`, the generated row class SHALL +have one field for the embed. The field SHALL be named after the embedded table +(or its alias) and typed as that table's model (`models.`). The querier +SHALL build the embedded model from that table's columns, in order, at their +positions in the result row. The row index of every field after the embed SHALL +be shifted to match. A query whose only result column is an embed SHALL still +return a row class, not a bare scalar. + +#### Scenario: Embed alongside a plain column + +- **WHEN** a `:one` query is + `SELECT sqlc.embed(authors), books.title FROM authors JOIN books ...` and + `authors` has columns `id` and `name` +- **THEN** the row class has fields `authors: models.Author` and `title: str`, + and it is built as + `Row(authors=models.Author(id=cast(int, row[0]), name=cast(str, row[1])), title=cast(str, row[2]))` + +#### Scenario: Two embeds + +- **WHEN** a query embeds both `authors` and `books` +- **THEN** the row class has a field for each, each built from its own + consecutive column range + +### Requirement: Query comments as docstrings + +Comment lines written above a query's `-- name:` line SHALL become the docstring +of the generated sync and async querier methods, one comment line per docstring +line, in order. Queries without comments SHALL produce no docstring. + +#### Scenario: Commented query + +- **WHEN** a query is preceded by `-- Fetch an author by id.` +- **THEN** the `get_author` method body starts with the docstring + `"""Fetch an author by id."""` + +### Requirement: Copyfrom + +A `:copyfrom` query SHALL generate a params class holding every query parameter, +even when there are fewer parameters than `query_parameter_limit`. The sync and +async querier methods SHALL take one positional argument, a `Sequence` of that +params class. Each method SHALL run the statement once over all the items as a +single `executemany` call and return the driver-reported row count as `int`. +Generation SHALL no longer fail for `:copyfrom`. + +#### Scenario: Bulk insert + +- **WHEN** a query is `-- name: CreateAuthors :copyfrom` with + `INSERT INTO authors (name, bio) VALUES ($1, $2)` +- **THEN** the querier has + `def create_authors(self, arg: Sequence[CreateAuthorsParams]) -> int` (and an + `async def` form), which passes a list of parameter dicts, one per item, to a + single `execute` call + +#### Scenario: Empty input + +- **WHEN** `create_authors` is called with an empty sequence +- **THEN** it returns `0` without running the statement + +### Requirement: Multi-dimensional arrays + +A column or parameter with N array dimensions SHALL be annotated with N nested +`List[...]` wrappers around its element type. `Optional[...]`, when the column +is nullable, SHALL wrap the outermost list. + +#### Scenario: Two-dimensional array + +- **WHEN** a column is `matrix integer[][] NOT NULL` +- **THEN** it is annotated `List[List[int]]` + +#### Scenario: One-dimensional array unchanged + +- **WHEN** a column is `tags text[]` +- **THEN** it is annotated `Optional[List[str]]`, as before + +### Requirement: Sync and async queriers + +With `emit_sync_querier: true`, each query file SHALL define a `Querier` class +whose constructor accepts a `sqlalchemy.engine.Connection` or a +`sqlalchemy.orm.Session`. With `emit_async_querier: true`, each query file SHALL +define an `AsyncQuerier` class whose constructor accepts a +`sqlalchemy.ext.asyncio.AsyncConnection` or a +`sqlalchemy.ext.asyncio.AsyncSession`. Each querier SHALL declare its `_conn` +attribute with that union type. Both flags MAY be set together. Every query +command the plugin supports (`:one`, `:many`, `:exec`, `:execrows`, +`:execresult`, `:copyfrom`, `:batchone`, `:batchmany`, `:batchexec`) SHALL +generate a method on each enabled querier. The method SHALL have the same name, +parameters, and result type on both, except that async methods SHALL be +awaitable or async-iterable. With neither flag set, query files SHALL contain +only the SQL constants and the params and row classes. + +#### Scenario: Session accepted + +- **WHEN** a `Querier` is constructed with a `sqlalchemy.orm.Session`, or an + `AsyncQuerier` with an `AsyncSession` +- **THEN** every generated method works and type-checks exactly as it does with + a connection + +#### Scenario: Non-generic by default + +- **WHEN** `emit_generic_querier` is unset +- **THEN** `Querier` has no type parameters, and `_conn` is annotated with the + connection-or-session union + +#### Scenario: Both queriers + +- **WHEN** both flags are true and the file has a `:many` query `ListAuthors` +- **THEN** `Querier.list_authors` returns an `Iterator` of rows, and + `AsyncQuerier.list_authors` is an async generator returning an `AsyncIterator` + of the same row type + +### Requirement: Generic queriers + +With `emit_generic_querier: true`, each emitted querier SHALL be a PEP 695 +generic class whose single type parameter `_ConnT` is bounded by that querier's +connection-or-session union: + +- `class Querier[_ConnT: ]` +- `class AsyncQuerier[_ConnT: ]` + +`_conn` SHALL be annotated `_ConnT`, and `__init__` SHALL take `conn: _ConnT`. The bound +SHALL be spelled in the active type syntax (`Union[...]` or `|`). Method +signatures and bodies SHALL be the same as for non-generic queriers. Querier +protocols SHALL stay non-generic, and a generic querier of any valid `_ConnT` SHALL +still satisfy its protocol. + +#### Scenario: Session type is preserved + +- **WHEN** `emit_generic_querier` and `emit_sync_querier` are true and code runs + `q = Querier(session)` with a `sqlalchemy.orm.Session` +- **THEN** a type checker infers `q` as `Querier[Session]` and `q._conn` as + `Session` + +#### Scenario: Bound is enforced + +- **WHEN** code calls `Querier(object())` with the option on +- **THEN** a type checker reports that the argument violates the bound of `_ConnT` + +#### Scenario: Works with both syntax styles + +- **WHEN** `emit_generic_querier` is true, once with `emit_modern_types` on and + once with it off +- **THEN** the bounds read + `sqlalchemy.engine.Connection | sqlalchemy.orm.Session` and + `Union[sqlalchemy.engine.Connection, sqlalchemy.orm.Session]` respectively, + and both files compile on Python 3.12 + +### Requirement: Typed row values + +Every value that a querier method reads from a result row SHALL be wrapped in +`typing.cast(, row[i])` (`cast` imported from `typing`). +`` SHALL be the field's or scalar's annotation, spelled in the +active type syntax. This applies to scalar returns, row-class fields, reused +models, and embedded models. It SHALL have no runtime effect. + +#### Scenario: Row field cast + +- **WHEN** a `:one` query returns `models.Author` with fields `id: int` and + `bio: Optional[str]` +- **THEN** the method builds + `models.Author(id=cast(int, row[0]), bio=cast(Optional[str], row[1]))` + +#### Scenario: New commands on both queriers + +- **WHEN** both flags are true and the file has `:copyfrom` and `:batchone` + queries +- **THEN** both commands appear as methods on `Querier` and `AsyncQuerier` + +### Requirement: Batch commands + +A `:batchone`, `:batchmany`, or `:batchexec` query SHALL generate a params class +holding every query parameter, as `:copyfrom` does, whatever +`query_parameter_limit` is. Each method SHALL take one positional argument, a +`Sequence` of that params class. An empty sequence SHALL run no statements. + +`:batchexec` SHALL run the statement once as a single `executemany` over all +items, and SHALL return `None` (awaitable on `AsyncQuerier`). + +`:batchone` and `:batchmany` SHALL run the statement once per item, in order, on +the querier's connection. They SHALL produce results lazily, as the caller +consumes them: + +| Command | Sync return | Async return | Value per input item | +|---|---|---|---| +| `:batchone` | `Iterator[Row \| None]` | `AsyncIterator[Row \| None]` | the first row, or `None` when there is no row | +| `:batchmany` | `Iterator[list[Row]]` | `AsyncIterator[list[Row]]` | all rows, possibly empty | + +`Row` here means the query's row class, the reused model, or the scalar type, +chosen by the same rules as `:one` and `:many`. The annotations above use modern +spelling. Generated code SHALL spell them in the active type syntax +(`Optional[Row]` and `List[Row]` when `emit_modern_types` is off). + +#### Scenario: Batchone yields per input + +- **WHEN** `GetAuthorsByID :batchone` is called with three params objects and + the second id does not exist +- **THEN** iterating the result yields a row, then `None`, then a row + +#### Scenario: Batchmany yields a list per input + +- **WHEN** `ListBooksByAuthor :batchmany` is called with two params objects +- **THEN** iterating the result yields exactly two lists, one per params object, + in input order + +#### Scenario: Lazy execution + +- **WHEN** a `:batchone` method is called and the caller stops after the first + yielded result +- **THEN** only the first item's statement has run + +#### Scenario: Batchexec uses one executemany + +- **WHEN** `CreateAuthors :batchexec` + (`INSERT INTO authors (name, bio) VALUES ($1, $2)`) is called with two params + objects +- **THEN** the method makes one `execute` call with a list of two parameter + dicts, and returns `None` once both rows are written + +### Requirement: Querier protocols + +With `emit_querier_protocol: true`, each query file SHALL also define +`QuerierProtocol` (when the sync querier is emitted) and `AsyncQuerierProtocol` +(when the async querier is emitted). Both SHALL subclass `typing.Protocol`. Each +protocol SHALL declare every method of the matching querier, with the same +parameters and return annotation and a body of `...`. The generated `Querier` +and `AsyncQuerier` SHALL satisfy their protocols structurally under mypy and +pyright. For async methods that are async generators (`:many`, `:batchone`, +`:batchmany`), the protocol SHALL declare a plain `def` that returns +`AsyncIterator[...]`, so that the async-generator implementation matches it. +With the option off, no protocol classes SHALL be emitted. + +#### Scenario: Protocol mirrors querier + +- **WHEN** `emit_querier_protocol` and `emit_sync_querier` are true +- **THEN** `QuerierProtocol` declares + `get_author(self, *, id: int) -> Optional[models.Author]: ...`, and + `q: QuerierProtocol = Querier(conn)` type-checks + +#### Scenario: Async generator method in protocol + +- **WHEN** `emit_querier_protocol` and `emit_async_querier` are true and the + file has a `:many` query +- **THEN** `AsyncQuerierProtocol` declares it as + `def list_authors(self) -> AsyncIterator[models.Author]: ...`, and + `AsyncQuerier` type-checks against the protocol + +#### Scenario: Hand-written fake + +- **WHEN** a test defines a class that implements the `QuerierProtocol` methods + with in-memory data +- **THEN** it can be passed where a `QuerierProtocol` is expected and + type-checks + +### Requirement: Typed query errors + +With `emit_query_errors: true`, the plugin SHALL generate `errors.py` in the +output package. It SHALL define `QueryError(Exception)`, with a constructor +`(message, query_name, cause=None)` and attributes `query_name`, `cause` (the +original SQLAlchemy exception), and `constraint_name` (a string when the driver +exposes it, otherwise `None`). It SHALL also define these direct subclasses of +`QueryError`: + +| Class | Raised for | SQLSTATE | +|---|---|---| +| `UniqueViolationError` | `sqlalchemy.exc.IntegrityError` | `23505` | +| `ForeignKeyViolationError` | `sqlalchemy.exc.IntegrityError` | `23503` | +| `CheckViolationError` | `sqlalchemy.exc.IntegrityError` | `23514` | +| `NotNullViolationError` | `sqlalchemy.exc.IntegrityError` | `23502` | +| `ExclusionViolationError` | `sqlalchemy.exc.IntegrityError` | `23P01` | +| `StatementTimeoutError` | `sqlalchemy.exc.OperationalError` | `57014` | +| `DeadlockError` | `sqlalchemy.exc.OperationalError` | `40P01` | +| `SerializationError` | `sqlalchemy.exc.OperationalError` | `40001` | + +When a querier method raises `sqlalchemy.exc.IntegrityError` or +`sqlalchemy.exc.OperationalError`, it SHALL raise the class matching the +SQLSTATE, `from` the original. A SQLSTATE that is not in the table, or missing, +SHALL raise `QueryError` itself. This SHALL hold for sync and async methods, for +every command, and while a `:many`, `:batchone`, or `:batchmany` result is being +iterated. The SQLSTATE SHALL be read in a way that works with psycopg2, psycopg +3, and asyncpg. Every other exception SHALL propagate unchanged. + +These class names, the constructor, and `query_name` and `cause` SHALL match +alt-sqlc-gen-python's `emit_query_errors`, so code written against that fork +keeps working. With the option off, no `errors.py` SHALL be generated and +methods SHALL not catch exceptions. When a query file would also produce +`errors.py`, generation SHALL fail with an error that names the conflict. + +#### Scenario: Unique violation + +- **WHEN** `create_author` inserts a duplicate value into a `UNIQUE` column + named by constraint `authors_name_key` +- **THEN** it raises `errors.UniqueViolationError`, with + `query_name == "create_author"`, `constraint_name == "authors_name_key"`, and + `cause` and `__cause__` both set to the original + `sqlalchemy.exc.IntegrityError` + +#### Scenario: Serialization failure + +- **WHEN** a query in a `SERIALIZABLE` transaction fails with SQLSTATE `40001` +- **THEN** it raises `errors.SerializationError` + +#### Scenario: Catch by base class + +- **WHEN** any wrapped error occurs inside a querier method +- **THEN** `except errors.QueryError` catches it + +#### Scenario: Other errors pass through + +- **WHEN** a querier method fails with a `sqlalchemy.exc.ProgrammingError` +- **THEN** that same exception propagates, unwrapped + +#### Scenario: Error while iterating + +- **WHEN** a `:many` method's result stream fails with a deadlock partway + through iteration +- **THEN** the consumer's loop raises `errors.DeadlockError` + +#### Scenario: Works with psycopg and asyncpg + +- **WHEN** the same unique violation happens through a sync psycopg connection + and through an async asyncpg connection +- **THEN** both raise `errors.UniqueViolationError` diff --git a/openspec/changes/sqlc-feature-parity/tasks.md b/openspec/changes/sqlc-feature-parity/tasks.md new file mode 100644 index 0000000..8d35a65 --- /dev/null +++ b/openspec/changes/sqlc-feature-parity/tasks.md @@ -0,0 +1,201 @@ +# Tasks + +## 1. Tooling and baseline + +- [x] 1.1 In `mise.toml`, pin Go to 1.23 and add `buf` and Python 3.12. Add + `mypy>=1.12`, `pydantic>=2`, `sqlalchemy>=2`, `psycopg`, and `asyncpg` to + `examples/requirements.txt`. Verify `mise install && make all` produces + `bin/sqlc-gen-python.wasm` +- [x] 1.2 Bump the sqlc version to `1.31.1` in `.github/workflows/*.yml`, and + verify with `grep -rn sqlc-version .github/` +- [x] 1.3 With no code changes, regenerate every `internal/endtoend/testdata/*` + fixture and `examples/` using sqlc v1.31.1 (`sqlc generate` in each dir). + Commit it on its own and verify `make test` passes and + `cd examples && sqlc diff` is clean + +## 2. Printer and AST foundations + +- [x] 2.1 Escape `\` and the active quote character in `printConstant`. Switch + `sqlalchemySQL` and the `-- name:` header in `buildQueryTree` to emit + runtime text (single `\:`). Verify `make test` shows no fixture diffs and + a new `printer_test.go` case round-trips `say "hi"` and `a\b` +- [x] 2.2 Add one triple-quoted docstring helper and use it in `printClassDef`, + `printFunctionDef`, and `printAsyncFunctionDef`. Verify with a printer + test for a multi-line class docstring and a function docstring +- [x] 2.3 Print `pass` for a `ClassDef` with an empty body, and verify with a + printer test +- [x] 2.4 Print multi-line `AnnAssign.Comment` values as one `#` line per source + line at the field's indent, and verify with a printer test +- [x] 2.5 Add `ListComp`, `Comprehension`, `With`, `WithItem`, `BinOp` (with + `BitOr`), a `Constant.ellipsis` case, and `ClassDef.type_params` + (`TypeVar{name, bound}`) to `protos/ast/ast.proto`, and regenerate + `internal/ast/ast.pb.go` with `buf generate`. Verify `go build ./...` + succeeds +- [x] 2.6 Add printer support for each node from 2.5 and for `If.or_else` + (`else:`). Verify with printer tests that render + `[{"p1": a.x} for a in arg]`, `with errors._wrap_errors("q"):` plus a + body, `list[int] | None`, `def f(self) -> int: ...`, `class Q[T: A | B]:`, + and an `if`/`else` + +## 3. Identifiers and enums + +- [x] 3.1 Add the `pyIdent` helper (rename lookup, then keyword escaping using + the Python 3.12 hard keywords) and unit tests for `from`→`from_`, `match` + unchanged, and rename applied before escaping +- [x] 3.2 Route field names, param names, method names, and model and enum class + names through `pyIdent`, and compare against the escaped name in the + model-reuse check. Verify with a new fixture `testdata/python_keywords` (a + column `from`, a param `class`) whose generated modules pass + `python -m py_compile` +- [x] 3.3 Post-process enum member names (`VALUE_` for empty names, a + `VALUE_` prefix for a leading digit, `_2`… for duplicates). Verify with a + new fixture `testdata/enum_member_names` that covers `=`, `<>`, + `in-progress`/`in_progress`, `1st`, and `say "hi"`, and extend the + existing `emit_str_enum` fixture with the same values +- [x] 3.4 Add a fixture `testdata/empty_params_class` + (`query_parameter_limit: 0` with a no-param query) and a fixture + `testdata/multiline_comments` (multi-line table and column comments). + Verify `make test` passes and the outputs `py_compile` + +## 4. Types and annotation syntax + +- [x] 4.1 Change `pyType.IsArray` to `ArrayDims int`, and emit nested list + wrappers. Verify with a new fixture `testdata/multidim_arrays` (`int[][]` + gives `List[List[int]]`, and `text[]` is unchanged) +- [x] 4.2 Add the missing PostgreSQL aliases (`varchar`, `character varying`, + `bpchar`, `character`, `name`, `time`/`timetz`/`timestamp` and the + `with/without time zone` spellings, `pg_catalog.json`/`jsonb`) to + `postgresql_type.go`. Verify with a new fixture + `testdata/postgresql_type_aliases` that shows no `Any` for them +- [x] 4.3 Make `Annotation(syntax)` the only annotation builder, add the + generic-wrapper helper + (`Optional`/`List`/`Iterator`/`AsyncIterator`/`Sequence` → typing or + modern forms, `BinOp` for `|`), and have the importer ask that helper + which names were used. Verify existing fixtures show no diff with + `emit_modern_types` off +- [x] 4.4 Add `emit_modern_types`, and verify with a new fixture + `testdata/emit_modern_types` (`list[str] | None`, + `from collections.abc import Iterator, AsyncIterator`, and no + `Optional`/`List` imports) +- [x] 4.5 Add `emit_aware_datetime` in `postgresType` (only the `timestamptz` + spellings), with the config check that requires `emit_pydantic_models`. + Verify with a new fixture `testdata/emit_aware_datetime` (`timestamptz` → + `pydantic.AwareDatetime`, `timestamp` unchanged) and a fixture + `testdata/emit_aware_datetime_no_pydantic` whose `stderr.txt` holds the + expected error + +## 5. Config options + +- [x] 5.1 Add `Overrides` (with `db_type`, `column`, `nullable`, `py_type`, and + `py_import`), `Rename`, `OmitUnusedStructs`, `OmitSqlcVersion`, + `EmitModernTypes`, `EmitGenericQuerier`, `EmitAwareDatetime`, + `EmitQuerierProtocol`, and `EmitQueryErrors` to `Config`. Parse + `req.GlobalOptions` and merge it (local overrides first; global rename + values win). Validate override entries (exactly one of `db_type`/`column`, + a non-empty `py_type`, and no dotted `py_type` with `py_import`). Verify + with Go unit tests for merge order and each validation error +- [x] 5.2 Resolve overrides in `makePyType` (`column` entries with `*` globbing + first, then `db_type` with `nullable`, then the type map) for both columns + and params. Verify with a new fixture `testdata/overrides` that covers the + db_type, nullable, column-wins, and param-follows-column scenarios +- [x] 5.3 Generalize the importer so every dotted non-`models.` `InnerType` adds + `import `, and every `py_import` override adds + `from import `, both only in the files that use them + and in the right std/pkg group. Verify the `overrides` fixture covers both + forms (`import my_lib.types` and `from uuid import UUID`), and that + existing fixtures show no import diffs +- [x] 5.4 Add a fixture `testdata/overrides_global` that uses the top-level + `options: {py: {overrides: ...}}` in `sqlc.yaml`, and verify the global + override applies +- [x] 5.5 Apply `rename` to model names, enum names, fields, and enum members. + Verify with a new fixture `testdata/rename` that covers a renamed field + and a renamed model referenced from a query's return type +- [x] 5.6 Port `filterUnusedStructs` behind `omit_unused_structs`, run after + `buildQueries` and before building the importer and context. Verify with a + new fixture `testdata/omit_unused_structs` (an unused table dropped, and + an enum kept through a model field) +- [x] 5.7 Honor `omit_sqlc_version` in `moduleNode`. Verify with a new fixture + `testdata/omit_sqlc_version` whose files have no `sqlc v` line + +## 6. Querier code generation + +- [x] 6.1 Refactor the sync and async querier bodies in `buildQueryTree` into + one builder with an `async` flag, for all existing commands. Verify every + existing fixture regenerates with no diff +- [x] 6.2 Widen the querier connection types to `Connection | Session` and + `AsyncConnection | AsyncSession`, add the class-level `_conn` annotation, + and wrap every row value in `cast(, row[i])`. Land it as its + own commit that regenerates all fixtures and `examples/`, and verify + `make test` and `cd examples && sqlc diff` pass +- [x] 6.3 Add `emit_generic_querier`: emit `class Querier[_ConnT: ]` and + `class AsyncQuerier[_ConnT: ]` with `_conn: _ConnT` and `conn: _ConnT`, with the + bound in the active syntax. Verify with a new fixture + `testdata/emit_generic_querier` (in typing and modern syntax) plus a + `_check.py` where `reveal_type(Querier(session)._conn)` is `Session` and a + protocol assignment passes under `mypy --strict` +- [x] 6.4 Emit `Query.Comments` as the docstring of sync and async querier + methods. Verify with a new fixture `testdata/query_comments`, and check + that uncommented queries have no docstring +- [x] 6.5 Support `sqlc.embed()`: add `Field.Embed`, build nested + `models.X(...)` from consecutive row indexes using a running index (with + `cast` on each value), and skip model reuse and the scalar shortcut for + embeds. Verify with a new fixture `testdata/sqlc_embed` (an embed plus a + plain column, and two embeds) against real sqlc v1.31.1 output, in both + dataclass and pydantic mode +- [x] 6.6 Support `:copyfrom` and `:batchexec` on the shared `executemany` path: + force a params struct, take `arg: Sequence[Params]`, return early on empty + input, and return `rowcount` or `None` respectively. Verify with new + fixtures `testdata/copyfrom` and `testdata/batch_exec`, and check that the + old "not implemented" error and the `unknown cmd` panic are gone +- [x] 6.7 Support `:batchone` (`if`/`else` yield per item) and `:batchmany` + (yield a list comprehension per item) as sync and async generators. Verify + with a new fixture `testdata/batch_one_many` in both typing and modern + syntax +- [x] 6.8 Emit `QuerierProtocol` and `AsyncQuerierProtocol` behind + `emit_querier_protocol`, derived from the built `FunctionDef`s, with + async-generator methods as plain `def`. Verify with a new fixture + `testdata/emit_querier_protocol` plus a `_check.py` that assigns each + querier to its protocol, and that `mypy --strict` passes on it +- [x] 6.9 Generate `errors.py` behind `emit_query_errors` (the fork's classes, + `_wrap_integrity_error`/`_wrap_operational_error`, `constraint_name`, and + the `_wrap_errors` context manager), wrap every querier method body in + `with errors._wrap_errors(""):`, import `errors` in query files, + and fail when a query file maps to `errors.py`. Verify with new fixtures + `testdata/emit_query_errors` (in typing and modern syntax) and + `testdata/emit_query_errors_conflict` (with `stderr.txt`) + +## 7. Integration, docs, release + +- [x] 7.1 Add the fixtures `testdata/feature_matrix_classic` and + `testdata/feature_matrix_modern` (design decision 21), each with a + `_check.py`. Verify `make test` passes, and `mypy --strict` passes on both +- [x] 7.2 Move the `db` CI job from Python 3.9 to 3.12 (needed to parse the PEP + 695 fixtures), and add a step that runs `python -m py_compile` over every + `internal/endtoend/testdata/**/*.py` and `mypy --strict` over the two + matrix fixtures, and verify it passes locally +- [x] 7.3 Extend `examples/` with `:copyfrom`, `:batchexec`, `:batchone`, + `:batchmany`, and `sqlc.embed()` queries, `emit_query_errors`, and + `emit_querier_protocol`. Add pytest cases against Postgres for: + - lazy batch results + - unique, foreign-key, and not-null violations raising the typed errors, with + `constraint_name`, through psycopg (sync) and asyncpg (async) + - a `Session`-backed `Querier` + - a fake that satisfies `QuerierProtocol` + + Verify `pytest` passes against a local `postgres` container +- [x] 7.4 Add README sections for: + - `overrides` (both forms, global `options`, and `nullable`), `rename`, + `omit_unused_structs`, and `omit_sqlc_version` + - `emit_modern_types` (Python 3.10+) and `emit_aware_datetime` (requires + pydantic 2) + - `emit_generic_querier` (Python 3.12+), `emit_querier_protocol`, and + `emit_query_errors` + - `:copyfrom` (driver-dependent `rowcount`), the batch commands (N round-trips + for `:batchone`/`:batchmany`), and `sqlc.embed()` + - Session support + - a "Migrating from alt-sqlc-gen-python" note (the `:batchexec` signature, and + the opt-in modern syntax, aware datetimes, and generic queriers) + + Verify every new option name appears in the README +- [x] 7.5 Run `make test`, `go vet ./...`, and `cd examples && sqlc diff` for a + final check, and confirm all pass diff --git a/protos/ast/ast.proto b/protos/ast/ast.proto index a8daa62..fa34312 100644 --- a/protos/ast/ast.proto +++ b/protos/ast/ast.proto @@ -36,6 +36,13 @@ message Node { Await await = 28 [json_name="Await"]; AsyncFor async_for = 29 [json_name="AsyncFor"]; ImportGroup import_group = 30 [json_name="ImportGroup"]; + ListComp list_comp = 31 [json_name="ListComp"]; + With with = 32 [json_name="With"]; + BinOp bin_op = 33 [json_name="BinOp"]; + BitOr bit_or = 34 [json_name="BitOr"]; + Tuple tuple = 35 [json_name="Tuple"]; + UnaryOp unary_op = 36 [json_name="UnaryOp"]; + Not not = 37 [json_name="Not"]; } } @@ -111,6 +118,7 @@ message ClassDef repeated Node keywords = 3 [json_name="keywords"]; repeated Node body = 4 [json_name="body"]; repeated Node decorator_list = 5 [json_name="decorator_list"]; + repeated TypeVar type_params = 6 [json_name="type_params"]; } // The Python ast module does not parse comments. It's not clear if this is the @@ -133,6 +141,7 @@ message Constant string str = 1 [json_name="string"]; int32 int = 2 [json_name="int"]; bool none = 3 [json_name="none"]; + bool ellipsis = 4 [json_name="ellipsis"]; } } @@ -183,15 +192,15 @@ message ImportFrom // Imports are always put at the top of the file, just after any module // comments and docstrings, and before module globals and constants. -// +// // Imports should be grouped in the following order: -// +// // Standard library imports. // Related third party imports. // Local application/library specific imports. -// +// // You should put a blank line between each group of imports. -// +// // https://www.python.org/dev/peps/pep-0008/#imports message ImportGroup { @@ -234,9 +243,65 @@ message Subscript Node slice = 2 [json_name="slice"]; } -message Yield +message ListComp { - Node value = 1 [json_name="value"]; + Node elt = 1 [json_name="elt"]; + repeated Comprehension generators = 2 [json_name="generators"]; +} + +message Comprehension +{ + Node target = 1 [json_name="target"]; + Node iter = 2 [json_name="iter"]; + repeated Node ifs = 3 [json_name="ifs"]; } +message With +{ + repeated WithItem items = 1 [json_name="items"]; + repeated Node body = 2 [json_name="body"]; +} +message WithItem +{ + Node context_expr = 1 [json_name="context_expr"]; + Node optional_vars = 2 [json_name="optional_vars"]; +} + +message BinOp +{ + Node left = 1 [json_name="left"]; + Node op = 2 [json_name="op"]; + Node right = 3 [json_name="right"]; +} + +message BitOr +{ +} + +message UnaryOp +{ + Node op = 1 [json_name="op"]; + Node operand = 2 [json_name="operand"]; +} + +message Not +{ +} + +message Tuple +{ + repeated Node elts = 1 [json_name="elts"]; +} + +// PEP 695 type parameter, printed as `T: ` inside `class C[...]`. +message TypeVar +{ + string name = 1 [json_name="name"]; + Node bound = 2 [json_name="bound"]; +} + +message Yield +{ + Node value = 1 [json_name="value"]; +} diff --git a/protos/buf.yaml b/protos/buf.yaml index 4c02d5a..2112523 100644 --- a/protos/buf.yaml +++ b/protos/buf.yaml @@ -1,5 +1,5 @@ version: v1 -name: github.com/sqlc-dev/sqlc-gen-python +name: github.com/kvdomingo/sqlc-gen-python breaking: use: - FILE