Skip to content

GH-53872: [Format][FlightSQL][C++] Add Substrait dialect SqlInfo codes - #54360

Open
tokoko wants to merge 1 commit into
apache:mainfrom
tokoko:flight-sql-substrait-dialect
Open

tokoko wants to merge 1 commit into
apache:mainfrom
tokoko:flight-sql-substrait-dialect

Conversation

@tokoko

@tokoko tokoko commented Oct 18, 2026

Copy link
Copy Markdown

Rationale for this change

Flight SQL can already report whether a server accepts Substrait plans and which versions it accepts (FLIGHT_SQL_SERVER_SUBSTRAIT, _MIN_VERSION, _MAX_VERSION). It can't report which part of Substrait the server accepts. In practice no consumer implements the whole spec, so a producer only learns that a plan is unsupported when execution fails.

Substrait now has a standard format for this: dialect files. A dialect is a YAML document that lists the types, relations, expressions and function signatures a system supports. substrait-java already publishes one for Spark (spark_dialect.yaml).

What changes are included in this PR?

Two new SqlInfo codes in FlightSql.proto, mirrored in the C++ SqlInfoOptions::SqlInfo enum:

  • SQL_SUBSTRAIT_DIALECT = 577: a UTF-8 string holding a Substrait dialect YAML document. Should only be returned if FLIGHT_SQL_SERVER_SUBSTRAIT is true. If it isn't returned, the server doesn't describe which parts of Substrait it accepts; that doesn't mean it accepts all of Substrait.
  • SQL_SUBSTRAIT_DIALECT_HASH = 578: an opaque string identifying the dialect's content, for client-side caching. It must change whenever the dialect changes (e.g. a hex-encoded SHA-256 digest of the dialect). Clients compare it for equality only.

The hash code exists because of size. A dialect covering every function in the standard Substrait extensions plus all types, relations and expressions is about 34 KiB of YAML (about 3.6 KiB gzipped). Real dialects are subsets of that. This is fine to fetch once per connection, but chatty clients can fetch the hash first and only refetch the dialect when the hash changes.

Both codes are in the [500, 1000) range with the other SQL syntax codes, since they describe the syntax the server accepts. Because ADBC reuses Flight SQL's info codes, ADBC gets the same codes without a separate change.

Are these changes tested?

No. The change only adds enum values and documentation; no behavior changes.

Are there any user-facing changes?

Yes. Two new SqlInfo codes are available to Flight SQL servers and clients. The change is backwards-compatible: no existing codes or types change.

@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #53872 has been automatically assigned in GitHub to PR creator.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant