Move ClickHouse testgen into internal/testcheck and verify end-to-end cases

testcheck replaces testgen. It generates nothing: each engine package
finds the analyze cases under internal/endtoend/testdata, loads a case's
schema, fixture and queries into a real database, and compares what the
database reports with the output.json the case committed, byte for byte.
The ClickHouse package is the testgen code moved over; other engines get
their own package alongside it.

sqlc analyze now prints each column's type as a call expression instead
of data_type, not_null and is_array, so its output and the database's
answer share one format. The compiler's flat column description maps
onto it as the data type wrapped in one array node per dimension with
the column's nullability on the outermost node. Every analyze case's
expected output is renamed from stdout.txt to output.json, which the
end-to-end harness now reads first, and regenerated.

The cases from testgen's testdata become analyze_types,
analyze_expressions, analyze_subqueries and analyze_exec under the
ClickHouse dialect, with fixture.sql next to the schema, and the existing
analyze_basic and analyze_params ClickHouse cases gain fixtures. Two
queries from the subqueries case, a CTE and a SELECT * over a join, are
left out because sqlc cannot analyze them yet.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6XkyWnx7iJFEb8q3AnYps

Author: claude

CLAUDE.md

@@ -120,8 +120,8 @@ A case is a directory holding the inputs and the expected output. `exec.json`
 names the command and its arguments — omit it and the case runs `generate`,
 comparing the generated files against the ones committed alongside; give it
 `{"command": "analyze", "args": [...]}` and the case compares the command's
-stdout against `stdout.txt`. A case that is expected to fail commits its
-`stderr.txt`. Regenerate a golden by running the command in its directory and
+stdout against `output.json` (or `stdout.txt` for a command that does not
+print JSON). A case that is expected to fail commits its `stderr.txt`. Regenerate a golden by running the command in its directory and
 writing the output back over the committed file.
 
 `TestReplay` runs the whole corpus once per *context*. `base` runs each case as
@@ -215,9 +215,6 @@ MYSQL_SERVER_URI="root:mysecretpassword@tcp(127.0.0.1:3306)/mysql?multiStatement
   - `/postgresql/` - PostgreSQL parser and converter
   - `/dolphin/` - MySQL parser (uses TiDB parser)
   - `/sqlite/` - SQLite parser
-  - `/clickhouse/testgen/` - Nested module that records what a real
-    ClickHouse reports about a schema, fixture and queries, in the shape of
-    `sqlc analyze` output with types as call expressions; see its README
   - `/duckdb/` - DuckDB 2.0 parser (uses darkwing, the pure Go port of
     DuckDB's PEG parser); its dialect seeds are generated by
     `/internal/tools/sqlc-duckdb-gen` from a live DuckDB CLI
@@ -228,6 +225,9 @@ MYSQL_SERVER_URI="root:mysecretpassword@tcp(127.0.0.1:3306)/mysql?multiStatement
 - `/internal/codegen/` - Code generation for different languages
 - `/internal/config/` - Configuration file parsing
 - `/internal/endtoend/` - End-to-end tests
+- `/internal/testcheck/` - Nested module that verifies the analyze cases under
+  `/internal/endtoend/testdata/` against a real database, one package per
+  engine; see its README
 - `/internal/sqltest/` - Test database setup (Docker, native, local detection)
 - `/examples/` - Example projects for testing
 

docs/howto/analyze.md

@@ -70,23 +70,24 @@ reports the result columns and parameters:
     "columns": [
       {
         "name": "id",
-        "data_type": "bigserial",
-        "not_null": true,
-        "is_array": false,
+        "type": {
+          "name": "bigserial"
+        },
         "table": "authors"
       },
       {
         "name": "name",
-        "data_type": "text",
-        "not_null": true,
-        "is_array": false,
+        "type": {
+          "name": "text"
+        },
         "table": "authors"
       },
       {
         "name": "bio",
-        "data_type": "text",
-        "not_null": false,
-        "is_array": false,
+        "type": {
+          "name": "text",
+          "nullable": true
+        },
         "table": "authors"
       }
     ],
@@ -95,9 +96,9 @@ reports the result columns and parameters:
         "number": 1,
         "column": {
           "name": "id",
-          "data_type": "bigserial",
-          "not_null": true,
-          "is_array": false,
+          "type": {
+            "name": "bigserial"
+          },
           "table": "authors"
         }
       }
@@ -106,6 +107,13 @@ reports the result columns and parameters:
 ]
 ```
 
+A column's `type` is written as a call expression: a `name` applied to
+`args`, each of which carries an optional `label` and exactly one of `type`,
+`int`, `bool` or `string`, with `nullable` set at whatever depth it applies.
+An array of text is `array` applied to `text`; a `Map(String, Nullable(UInt8))`
+in ClickHouse is `map` applied to `string` and a nullable `uint8`. Names are
+recorded as the engine reports them.
+
 Pass `--ast` to also include each statement's parsed AST under an `ast` key. It
 has the same shape as the output of [`parse`](parse.md), with every node tagged
 by type.

internal/cmd/analyze.go

@@ -204,18 +204,36 @@ type analyzedQuery struct {
 }
 
 type analyzedColumn struct {
-	Name     string `json:"name"`
-	DataType string `json:"data_type"`
-	NotNull  bool   `json:"not_null"`
-	IsArray  bool   `json:"is_array"`
-	Table    string `json:"table,omitempty"`
+	Name  string        `json:"name"`
+	Type  *analyzedType `json:"type,omitempty"`
+	Table string        `json:"table,omitempty"`
 }
 
 type analyzedParam struct {
 	Number int            `json:"number"`
 	Column analyzedColumn `json:"column"`
 }
 
+// analyzedType writes a type as a call expression: a name applied to
+// arguments that are other types, integers, booleans or strings, each with
+// an optional label, and a nullable flag at whatever depth it applies. An
+// array of text is array(text); a nullable column of it has nullable set on
+// the array node. Names are recorded as the engine reports them and resolve
+// against the catalog afterwards.
+type analyzedType struct {
+	Name     string        `json:"name"`
+	Nullable bool          `json:"nullable,omitempty"`
+	Args     []analyzedArg `json:"args,omitempty"`
+}
+
+type analyzedArg struct {
+	Label  string        `json:"label,omitempty"`
+	Type   *analyzedType `json:"type,omitempty"`
+	Int    *int64        `json:"int,omitempty"`
+	Bool   *bool         `json:"bool,omitempty"`
+	String *string       `json:"string,omitempty"`
+}
+
 func newAnalyzedQuery(q *compiler.Query, includeAST bool) analyzedQuery {
 	aq := analyzedQuery{
 		Name:    q.Metadata.Name,
@@ -243,13 +261,30 @@ func newAnalyzedColumn(col *compiler.Column) analyzedColumn {
 		return analyzedColumn{}
 	}
 	ac := analyzedColumn{
-		Name:     col.Name,
-		DataType: col.DataType,
-		NotNull:  col.NotNull,
-		IsArray:  col.IsArray,
+		Name: col.Name,
+		Type: newAnalyzedType(col),
 	}
 	if col.Table != nil {
 		ac.Table = col.Table.Name
 	}
 	return ac
 }
+
+// newAnalyzedType builds the type expression the compiler's flat column
+// description amounts to: the data type wrapped in one array node per
+// dimension, with the column's nullability on the outermost node.
+func newAnalyzedType(col *compiler.Column) *analyzedType {
+	if col.DataType == "" {
+		return nil
+	}
+	t := &analyzedType{Name: col.DataType}
+	dims := col.ArrayDims
+	if col.IsArray && dims == 0 {
+		dims = 1
+	}
+	for i := 0; i < dims; i++ {
+		t = &analyzedType{Name: "array", Args: []analyzedArg{{Type: t}}}
+	}
+	t.Nullable = !col.NotNull
+	return t
+}

internal/endtoend/case_test.go

@@ -52,17 +52,22 @@ func parseStderr(t *testing.T, dir, testctx string) []byte {
 	return nil
 }
 
+// parseStdout reads the command's expected output: output.json for a
+// command that prints JSON, so editors highlight it, otherwise stdout.txt.
 func parseStdout(t *testing.T, dir string) []byte {
 	t.Helper()
-	path := filepath.Join(dir, "stdout.txt")
-	if _, err := os.Stat(path); os.IsNotExist(err) {
-		return nil
-	}
-	blob, err := os.ReadFile(path)
-	if err != nil {
-		t.Fatal(err)
+	for _, name := range []string{"output.json", "stdout.txt"} {
+		path := filepath.Join(dir, name)
+		if _, err := os.Stat(path); os.IsNotExist(err) {
+			continue
+		}
+		blob, err := os.ReadFile(path)
+		if err != nil {
+			t.Fatal(err)
+		}
+		return blob
 	}
-	return blob
+	return nil
 }
 
 // hasSQLCConfig reports whether dir contains an sqlc configuration file.

…stdata/analyze_ast/postgresql/stdout.txt …tdata/analyze_ast/postgresql/output.jsoninternal/endtoend/testdata/analyze_ast/postgresql/stdout.txt renamed to internal/endtoend/testdata/analyze_ast/postgresql/output.json internal/endtoend/testdata/analyze_ast/postgresql/stdout.txt renamed to internal/endtoend/testdata/analyze_ast/postgresql/output.json

@@ -5,9 +5,9 @@
     "columns": [
       {
         "name": "name",
-        "data_type": "text",
-        "not_null": true,
-        "is_array": false,
+        "type": {
+          "name": "text"
+        },
         "table": "authors"
       }
     ],
@@ -16,9 +16,9 @@
         "number": 1,
         "column": {
           "name": "id",
-          "data_type": "bigserial",
-          "not_null": true,
-          "is_array": false,
+          "type": {
+            "name": "bigserial"
+          },
           "table": "authors"
         }
       }

internal/endtoend/testdata/analyze_basic/clickhouse/fixture.sql

@@ -0,0 +1 @@
+INSERT INTO events (id, name, tag, amount, created) VALUES (1, 'signup', NULL, 9.5, '2024-01-01 00:00:00');

…data/analyze_basic/clickhouse/stdout.txt …ata/analyze_basic/clickhouse/output.jsoninternal/endtoend/testdata/analyze_basic/clickhouse/stdout.txt renamed to internal/endtoend/testdata/analyze_basic/clickhouse/output.json internal/endtoend/testdata/analyze_basic/clickhouse/stdout.txt renamed to internal/endtoend/testdata/analyze_basic/clickhouse/output.json

@@ -5,37 +5,38 @@
     "columns": [
       {
         "name": "id",
-        "data_type": "uint64",
-        "not_null": true,
-        "is_array": false,
+        "type": {
+          "name": "uint64"
+        },
         "table": "events"
       },
       {
         "name": "name",
-        "data_type": "string",
-        "not_null": true,
-        "is_array": false,
+        "type": {
+          "name": "string"
+        },
         "table": "events"
       },
       {
         "name": "tag",
-        "data_type": "string",
-        "not_null": false,
-        "is_array": false,
+        "type": {
+          "name": "string",
+          "nullable": true
+        },
         "table": "events"
       },
       {
         "name": "amount",
-        "data_type": "float64",
-        "not_null": true,
-        "is_array": false,
+        "type": {
+          "name": "float64"
+        },
         "table": "events"
       },
       {
         "name": "created",
-        "data_type": "datetime",
-        "not_null": true,
-        "is_array": false,
+        "type": {
+          "name": "datetime"
+        },
         "table": "events"
       }
     ],

…testdata/analyze_basic/duckdb/stdout.txt …estdata/analyze_basic/duckdb/output.jsoninternal/endtoend/testdata/analyze_basic/duckdb/stdout.txt renamed to internal/endtoend/testdata/analyze_basic/duckdb/output.json internal/endtoend/testdata/analyze_basic/duckdb/stdout.txt renamed to internal/endtoend/testdata/analyze_basic/duckdb/output.json

@@ -5,37 +5,38 @@
     "columns": [
       {
         "name": "id",
-        "data_type": "bigint",
-        "not_null": true,
-        "is_array": false,
+        "type": {
+          "name": "bigint"
+        },
         "table": "authors"
       },
       {
         "name": "name",
-        "data_type": "text",
-        "not_null": true,
-        "is_array": false,
+        "type": {
+          "name": "text"
+        },
         "table": "authors"
       },
       {
         "name": "bio",
-        "data_type": "text",
-        "not_null": false,
-        "is_array": false,
+        "type": {
+          "name": "text",
+          "nullable": true
+        },
         "table": "authors"
       },
       {
         "name": "royalties",
-        "data_type": "decimal",
-        "not_null": true,
-        "is_array": false,
+        "type": {
+          "name": "decimal"
+        },
         "table": "authors"
       },
       {
         "name": "created",
-        "data_type": "timestamp",
-        "not_null": true,
-        "is_array": false,
+        "type": {
+          "name": "timestamp"
+        },
         "table": "authors"
       }
     ],

…tdata/analyze_basic/googlesql/stdout.txt …data/analyze_basic/googlesql/output.jsoninternal/endtoend/testdata/analyze_basic/googlesql/stdout.txt renamed to internal/endtoend/testdata/analyze_basic/googlesql/output.json internal/endtoend/testdata/analyze_basic/googlesql/stdout.txt renamed to internal/endtoend/testdata/analyze_basic/googlesql/output.json

@@ -5,16 +5,16 @@
     "columns": [
       {
         "name": "id",
-        "data_type": "int64",
-        "not_null": true,
-        "is_array": false,
+        "type": {
+          "name": "int64"
+        },
         "table": "users"
       },
       {
         "name": "name",
-        "data_type": "string",
-        "not_null": true,
-        "is_array": false,
+        "type": {
+          "name": "string"
+        },
         "table": "users"
       }
     ],
@@ -23,9 +23,9 @@
         "number": 1,
         "column": {
           "name": "id",
-          "data_type": "int64",
-          "not_null": true,
-          "is_array": false,
+          "type": {
+            "name": "int64"
+          },
           "table": "users"
         }
       }