Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 6 additions & 15 deletions .github/actions/setup/action.yml
Original file line number Diff line number Diff line change
@@ -1,44 +1,35 @@
name: Set up the pinned toolchain
description: Install the Node, npm, Bun and sqlc versions pinned by verification/compatibility.json, then install project dependencies.
description: Install the pinned Node, npm and Bun, optionally a requested sqlc, then install project dependencies.

inputs:
bun:
description: Install the pinned Bun when "true".
default: "false"
sqlc:
description: Install sqlc — "ceiling" for the tested ceiling, or an explicit version such as 1.24.0. Empty installs none.
description: Install this sqlc version, such as 1.31.1. Empty installs none.
default: ""

outputs:
sqlc-matrix:
description: The sqlc sample matrix as JSON.
value: ${{ steps.pins.outputs.sqlc-matrix }}

runs:
using: composite
steps:
- id: pins
shell: bash
run: node scripts/workflows/pins.mjs

- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: ${{ steps.pins.outputs.node }}
node-version: 24.12.0
cache: npm

- if: inputs.bun == 'true'
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: ${{ steps.pins.outputs.bun }}
bun-version: 1.3.10

- if: inputs.sqlc != ''
uses: sqlc-dev/setup-sqlc@6bd2de0e87f5adfd968b55f140ce55461f71db69 # v4
with:
sqlc-version: ${{ inputs.sqlc == 'ceiling' && steps.pins.outputs.sqlc-ceiling || inputs.sqlc }}
sqlc-version: ${{ inputs.sqlc }}

- shell: bash
run: |
npm install --global "npm@${{ steps.pins.outputs.npm }}"
npm install --global "npm@11.6.2"
npm ci
echo "==> Toolchain on this runner:"
echo " node $(node --version)"
Expand Down
17 changes: 7 additions & 10 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,12 @@ jobs:
verify:
name: Verify
runs-on: ubuntu-latest
outputs:
sqlc-matrix: ${{ steps.setup.outputs.sqlc-matrix }}
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5.1.0
- id: setup
uses: ./.github/actions/setup
- uses: ./.github/actions/setup
with:
bun: "true"
sqlc: ceiling
sqlc: 1.31.1

- name: Check formatting (run make fmt to fix)
run: make fmt-check
Expand All @@ -35,26 +32,26 @@ jobs:
if-no-files-found: error

sqlc-compatibility:
name: sqlc ${{ matrix.sqlc.version }}
name: sqlc ${{ matrix.sqlc }}
needs: verify
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
sqlc: ${{ fromJSON(needs.verify.outputs.sqlc-matrix) }}
sqlc: ["1.25.0", "1.31.1"]
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5.1.0
- uses: ./.github/actions/setup
with:
sqlc: ${{ matrix.sqlc.install }}
sqlc: ${{ matrix.sqlc }}
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: plugin-wasm
path: build

- name: Generate and type-check the fixtures with sqlc ${{ matrix.sqlc.version }}
- name: Generate and type-check the fixtures with sqlc ${{ matrix.sqlc }}
run: |
node scripts/verify-sqlc-compatibility.ts \
--candidate "$PWD/build/plugin.wasm" \
--sqlc-version "${{ matrix.sqlc.version }}" \
--sqlc-version "${{ matrix.sqlc }}" \
--sqlc "$(command -v sqlc)"
16 changes: 7 additions & 9 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ jobs:
name: Verify
runs-on: ubuntu-latest
outputs:
sqlc-matrix: ${{ steps.setup.outputs.sqlc-matrix }}
version: ${{ steps.intent.outputs.version }}
tag: ${{ steps.intent.outputs.tag }}
source-commit: ${{ steps.intent.outputs.source-commit }}
Expand All @@ -31,11 +30,10 @@ jobs:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5.1.0
with:
fetch-depth: 0
- id: setup
uses: ./.github/actions/setup
- uses: ./.github/actions/setup
with:
bun: "true"
sqlc: ceiling
sqlc: 1.31.1

- name: Validate the release identity before building anything
id: intent
Expand All @@ -62,28 +60,28 @@ jobs:
if-no-files-found: error

sqlc-compatibility:
name: sqlc ${{ matrix.sqlc.version }}
name: sqlc ${{ matrix.sqlc }}
needs: verify
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
sqlc: ${{ fromJSON(needs.verify.outputs.sqlc-matrix) }}
sqlc: ["1.25.0", "1.31.1"]
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5.1.0
- uses: ./.github/actions/setup
with:
sqlc: ${{ matrix.sqlc.install }}
sqlc: ${{ matrix.sqlc }}
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: plugin-wasm
path: build

- name: Generate and type-check the fixtures with sqlc ${{ matrix.sqlc.version }}
- name: Generate and type-check the fixtures with sqlc ${{ matrix.sqlc }}
run: |
node scripts/verify-sqlc-compatibility.ts \
--candidate "$PWD/build/plugin.wasm" \
--sqlc-version "${{ matrix.sqlc.version }}" \
--sqlc-version "${{ matrix.sqlc }}" \
--sqlc "$(command -v sqlc)"

# The only job that writes anything outside this run. R2 first, because the release
Expand Down
26 changes: 15 additions & 11 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ A sqlc code-generation plugin that emits TypeScript executed through a Cloudflar

Prettier owns formatting. Run `make fmt` before committing; `make fmt-check` gates CI.

Everything in `scripts/` is TypeScript run directly by Node (`node scripts/foo.ts`) — Node strips the types, so those files must stay within erasable syntax (no `enum`, no `namespace`, no constructor parameter properties) and must use explicit `.ts` extensions on relative imports. The same applies to `src/` and `test/`, because the tests run those sources directly. `scripts/workflows/pins.mjs` is the one exception: it chooses the Node version, so it runs before the toolchain is pinned and must stay plain JavaScript.
Everything in `scripts/` is TypeScript run directly by Node (`node scripts/foo.ts`) — Node strips the types, so those files must stay within erasable syntax (no `enum`, no `namespace`, no constructor parameter properties) and must use explicit `.ts` extensions on relative imports. The same applies to `src/` and `test/`, because the tests run those sources directly.

## The generator and the shipped runtime (`src/`)

Expand All @@ -13,6 +13,8 @@ Everything in `scripts/` is TypeScript run directly by Node (`node scripts/foo.t
| `src/app.ts` | Javy entry point: read stdin, write stdout and stderr, throw on failure. |
| `src/plugin.ts` | Decode the generate request, run validation and generation, render diagnostics. The one place a failure becomes stderr text. |
| `src/validation.ts` | Protocol, options, and query boundary validation. `SUPPORTED_COMMANDS` is the command surface. |
| `src/compatibility.ts` | The supported sqlc floor and the tested ceiling, as two literals and nothing else. |
| `src/semver.ts` | SemVer parsing and precedence, shared with `scripts/release.ts`. Carries build metadata; never compares it. |
| `src/diagnostics.ts` | `[CATEGORY/REASON]` identifiers, severity, and redaction of SQL and values. |
| `src/emission-plan.ts` | Naming, collision avoidance, argument and row field plans, per-command result shapes. |
| `src/embeds.ts` | `sqlc.embed` reconstruction and private alias rewriting. |
Expand Down Expand Up @@ -43,13 +45,13 @@ Native D1 errors pass through unwrapped, so a consumer can recognize them. A `Qu

`make verify-local` builds the plugin once and runs everything that needs no credentials. The layers each answer a different question about the same build:

| Target | Question |
| --------------------- | ---------------------------------------------------------------------------------------------------------- |
| `make test-unit` | Pure request-to-file generator behaviour, plus the script contracts. |
| `make test-candidate` | The same scenarios through the real wasm, and the public type surface on the floor and current TypeScript. |
| `make test-drift` | Does the committed generated output still match what the plugin emits? |
| `make test-miniflare` | Real workerd and D1 storage, fresh per test. |
| `make test-example` | The canonical Worker still builds and passes. |
| Target | Question |
| --------------------- | --------------------------------------------------------------------------------------- |
| `make test-unit` | Pure request-to-file generator behaviour, plus the script contracts. |
| `make test-candidate` | The same scenarios through the real wasm, and the public type surface under TypeScript. |
| `make test-drift` | Does the committed generated output still match what the plugin emits? |
| `make test-miniflare` | Real workerd and D1 storage, fresh per test. |
| `make test-example` | The canonical Worker still builds and passes. |

Tests run straight from TypeScript (`node --test`), and the unit target globs `test/*.test.ts test/generator/*.test.ts` — adding a test file needs no list edited anywhere.

Expand All @@ -68,17 +70,19 @@ done

**Sub-project formatting.** `examples/d1-worker/` and `test/miniflare/` are bun sub-projects that keep their own Prettier configuration; Prettier resolves configuration per file, so their hand-written sources stay tab-indented. Use bun inside those directories.

**Compatibility configuration.** `verification/compatibility.json` pins the sqlc samples and their rationale, the known exceptions, the TypeScript floor and current versions, and the Node/npm/Bun versions CI installs. It holds only what is not already recorded elsewhere — the Cloudflare versions live in the fixtures' `bun.lock` and `wrangler.jsonc`, and the buf and javy pins live in their install scripts. `scripts/workflows/pins.mjs` feeds the workflows from it and `docs/compatibility.md` presents it to consumers.
**The supported sqlc range is two literals.** `src/compatibility.ts` holds the supported floor and the tested ceiling; validation enforces the floor, `docs/compatibility.md` presents both to consumers, and the CI matrix runs exactly those two versions. Every other pinned version lives with the thing it pins — Node, npm and Bun in `.github/actions/setup/action.yml`, the Cloudflare versions in the fixtures' `bun.lock` and `wrangler.jsonc`, the buf and javy pins in their install scripts.

**Decided, against the obvious default:** sqlc is sampled strategically — the floor, the tested ceiling, and the intervening releases tied to a material protocol or metadata change. A matrix over every sqlc minor was considered and rejected ([#13](https://github.com/mkuznets/sqlc-d1-typescript/issues/13), [#38](https://github.com/mkuznets/sqlc-d1-typescript/issues/38)).
**The floor cell does not run on macOS.** `make test-sqlc-compatibility SQLC_VERSION=1.25.0` dies with `SIGKILL` on macOS arm64: that sqlc's wasm runtime cannot execute the plugin there, though the binary itself runs. It passes on the Linux x64 CI runner, which is where that cell is meant to run. Reproduce it locally with Docker rather than concluding the plugin is broken.

**Decided, against the obvious default:** the matrix samples only the floor and the ceiling. A matrix over every sqlc minor, and a matrix with intervening samples, were both considered and rejected ([#13](https://github.com/mkuznets/sqlc-d1-typescript/issues/13), [#38](https://github.com/mkuznets/sqlc-d1-typescript/issues/38)).

## Release

`.github/workflows/release.yml` validates the release identity, builds the plugin once, runs the same gates CI runs, then publishes.

**A valid tag is the approval.** A strict SemVer `v*` tag on default-branch lineage is the maintainer's release approval, and it is the only one. There is no dry-run mode: to rehearse a release, cut the next patch version.

**Publication order is the safety property.** R2 first, then the public origin is re-downloaded and compared, then the GitHub Release is cut — the release notes may only advertise a URL that already serves the right bytes. The R2 write uses `--if-none-match '*'`, so a version key can never be replaced once published. `contents: write` appears on the `publish` job and nowhere else, and `test/verification-contracts.test.ts` holds the workflow to that. Operations live in `docs/release-publication.md`.
**Publication order is the safety property.** R2 first, then the public origin is re-downloaded and compared, then the GitHub Release is cut — the release notes may only advertise a URL that already serves the right bytes. The R2 write uses `--if-none-match '*'`, so a version key can never be replaced once published. `contents: write` appears on the `publish` job and nowhere else. Operations live in `docs/release-publication.md`.

**Canonical names.** `sqlc-gen-d1-typescript_<version>.wasm` and `sqlc-gen-d1-typescript_<version>.manifest.json`, with no aliases. The URL shape and the manifest contract live in `scripts/release.ts`.

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ sql:
interface: workers
```

The SHA-256 is part of the selected-release configuration, not a placeholder to omit. The release manifest binds the version, tag, source commit, artifact size, URL, and digest to the same published artifact, alongside the tool versions it was tested against.
The SHA-256 is part of the selected-release configuration, not a placeholder to omit. The release manifest binds the version, tag, source commit, artifact size, URL, and digest to the same published artifact, alongside the sqlc versions it was tested against.

## Generate your first query

Expand Down
23 changes: 4 additions & 19 deletions docs/compatibility.md
Original file line number Diff line number Diff line change
@@ -1,39 +1,24 @@
# Compatibility

The source of truth is [`verification/compatibility.json`](../verification/compatibility.json). Everything on this page is derived from it.

## sqlc support policy

- Supported floor: **v1.18.0**.
- Tested ceiling: **v1.31.1**.
- Supported floor: **v1.25.0**. Older versions are rejected with `[COMPATIBILITY/UNSUPPORTED_SQLC_VERSION]`.
- Tested ceiling: **v1.31.1**. Every CI run generates and type-checks the repository's fixtures with both the floor and the ceiling.
- Versions newer than the ceiling are not rejected. They produce `[COMPATIBILITY/UNTESTED_SQLC_VERSION]` and continue generation unless another incompatibility is found.
- sqlc is sampled strategically: the floor, the tested ceiling, and the intervening releases tied to a material protocol or metadata change. Every sample below is tested on every CI run; this is not a claim that each intervening release has its own cell.

| sqlc | Role | Why this sample |
| ------- | ----------- | -------------------------------------------------------- |
| v1.18.0 | floor | Oldest supported Plugin protocol baseline. |
| v1.20.0 | intervening | Includes the material `sqlc.slice` generation fix. |
| v1.24.0 | intervening | Refactors the Plugin interface around `GenerateRequest`. |
| v1.31.1 | ceiling | Newest release verified by the compatibility suite. |

Known exceptions:

1. sqlc v1.18.0 cannot parse the current `sqlc.arg`, `sqlc.narg`, `sqlc.slice`, or `sqlc.embed` fixture syntax; its matrix cell uses `test/sqlc-v1-18` to cover all six ordinary commands and positional binds.
2. sqlc v1.20.0 and v1.24.0 parse the current fixture syntax, but their legacy Plugin WASM runtimes cannot execute the plugin from the official release binaries on macOS arm64; their Ubuntu x64 CI cells run both current fixture corpora.

See [sqlc-to-D1 translation](sqlc-to-d1.md) for the commands, macros, metadata, and value representations in the compatibility surface.

## TypeScript

Generated code compiles under both the supported floor compiler (**5.2.2**) and the current one (**5.9.3**). Consumers need neither exact version; the floor is the oldest compiler the emitted types are known to satisfy.
Generated code is strict-mode TypeScript, compiled and verified against a current TypeScript on every CI run. No particular compiler version is required of consumers.

## Cloudflare

Local verification runs the plugin's output under Miniflare and workerd, at the versions pinned in `test/miniflare/bun.lock` and the compatibility date in `test/miniflare/wrangler.jsonc`. Those are the repository's own test environment, **not** minimum consumer dependencies — nothing here asks you to copy the repository lockfiles.

## What a release binds

Each GitHub release ships the WASM plus a `sqlc-gen-d1-typescript_<version>.manifest.json` recording the version, tag, source commit, the artifact's permanent URL, its lowercase SHA-256 and size, and the tool versions it was tested against.
Each GitHub release ships the WASM plus a `sqlc-gen-d1-typescript_<version>.manifest.json` recording the version, tag, source commit, the artifact's permanent URL, its lowercase SHA-256 and size, and the sqlc floor and ceiling it was tested against.

Before the release is cut, the artifact is written to its permanent R2 key with a create-only conditional write, then re-downloaded from the public URL unauthenticated and compared to the bytes just published. A release you can see is one whose artifact was already proven to serve at that digest.

Expand Down
4 changes: 2 additions & 2 deletions docs/release-publication.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Everything below is executed by the `publish` job in [`.github/workflows/release

The `verify` job validates the tag, builds the plugin once, and runs every uncredentialed check against it. The `sqlc-compatibility` matrix tests those same bytes across the sampled sqlc versions. Only then does `publish` run:

1. **Manifest** — the wasm is renamed to its canonical filename and `scripts/release.ts manifest` records its digest, size, permanent URL, and the tested tool versions.
1. **Manifest** — the wasm is renamed to its canonical filename and `scripts/release.ts manifest` records its digest, size, permanent URL, and the tested sqlc range.
2. **Version key** — `aws s3api put-object` with `--if-none-match '*'` and `--content-md5`. The conditional write is what makes the key immutable: if it already holds bytes, R2 answers `412` and the step fails rather than replacing what is already advertised.
3. **Public verification** — the public URL is fetched unauthenticated, exactly as a consumer does, and its SHA-256 compared to the bytes just published. The release notes may only advertise a URL that already serves the right bytes.
4. **Publish** — `gh release create` attaches the wasm and its manifest. This is the last write of the run.
Expand All @@ -31,7 +31,7 @@ Configure these identifiers (never commit their values):

The `v*` rule must be created as a **tag** rule; created as a branch rule it matches no tag, and every tag push is then refused the Environment.

GitHub credentials are the job-scoped `GITHUB_TOKEN` only. `contents: write` appears on the publish job and nowhere else; `test/verification-contracts.test.ts` holds the workflow to that.
GitHub credentials are the job-scoped `GITHUB_TOKEN` only. `contents: write` appears on the publish job and nowhere else.

## How R2 is reached

Expand Down
Loading
Loading