Add exact-match uri filter to GET /assets endpoint by seanmuth · Pull Request #69489 · apache/airflow

seanmuth · 2026-07-06T18:54:16Z

What

Adds a uri query parameter to GET /api/v2/assets that matches an asset by its exact URI, backed by a new single-column index on asset.uri.

Why

Today, resolving a single asset when you already know its full URI (for example, discovering an asset_id before posting to /assets/{asset_id}/events for cross-deployment dependencies) requires uri_pattern. That parameter compiles to ILIKE '%...%', which cannot use a B-tree index and degrades to a full table scan on large asset tables — the API docs themselves warn about this.

The Airflow 2 REST API offered a fast exact lookup via GET /datasets/{uri} (later GET /assets/{uri}), but it was not carried forward when the endpoint migrated to FastAPI under AIP-84 — the single-resource route became GET /assets/{asset_id} (integer PK only). This PR restores the exact-URI lookup as a query parameter.

Passing the URI as a query parameter rather than a path segment also avoids the URI-encoding problems path parameters have with the /, :, and %2F characters that asset URIs contain.

What changed

New uri exact-match filter on GET /assets (equality comparison via the existing filter_param_factory).
New non-unique index idx_asset_uri on asset.uri (the existing unique index leads with name and cannot serve uri-only lookups), with an Alembic migration.
Regenerated OpenAPI spec + UI client, updated REVISION_HEADS_MAP and migration ref doc.
Tests: exact-match hits plus negative cases confirming a substring does not match (unlike uri_pattern).

Was generative AI tooling used to co-author this PR?

Yes — Claude Code (Opus 4.8, 1M context)

Generated-by: Claude Code (Opus 4.8, 1M context) following the guidelines

Drafted-by: Claude Code (Opus 4.8, 1M context) (no human review before posting)

Resolving a single asset by its full URI currently requires uri_pattern, which compiles to ILIKE '%...%' and cannot use a database index, so it degrades to a full table scan on large asset tables. This is the fast, index-backed exact-URI lookup that the Airflow 2 REST API exposed via GET /datasets/{uri} but that was not carried forward when the endpoint moved to FastAPI under AIP-84. The uri query parameter matches an asset by its exact URI using an equality comparison, backed by a new single-column index on asset.uri (the existing unique index leads with name and cannot serve uri-only lookups). Passing it as a query parameter rather than a path segment avoids the URI-encoding problems that path parameters have with the '/' and ':' characters in asset URIs.

pierrejeambrun

Thanks for the PR, makes sense!

A few suggestions bellow :)

pierrejeambrun · 2026-07-07T09:03:49Z

 | Revision ID             | Revises ID       | Airflow Version   | Description                                                  |
 +=========================+==================+===================+==============================================================+
-| ``d2f4e1b3c5a7`` (head) | ``9ff64e1c35d3`` | ``3.3.0``         | Add partition_date to asset_partition_dag_run.               |
+| ``c4e7a1f9b2d0`` (head) | ``d2f4e1b3c5a7`` | ``3.3.0``         | Add index on asset.uri.                                      |


Target version needs to be updated, 3.3.0 was just released, that would be for 3.4.0

pierrejeambrun · 2026-07-07T09:04:01Z

@@ -0,0 +1 @@
+Add an indexed exact-match ``uri`` query parameter to ``GET /api/v2/assets`` for fast single-asset lookup by URI (much faster than ``uri_pattern``, which uses an unindexed ``ILIKE '%...%'``).


This needs to be removed. (I know doc mentions that user facing change should have a fragment, but that's only true for significant / behavior change that needs a user warning basically).

pierrejeambrun · 2026-07-07T09:08:10Z

+QueryUriExactMatch = Annotated[
+    FilterParam[str | None],
+    Depends(
+        filter_param_factory(
+            AssetModel.uri,
+            str | None,
+            filter_name="uri",
+            description=(
+                "Exact-match filter on the full asset URI. Compiles to an indexed equality "
+                "comparison (``uri = ...``), so it is far faster than ``uri_pattern`` (which uses "
+                "``ILIKE '%...%'`` and cannot use an index) for resolving a single asset by its "
+                "known URI."
+            ),
+        )
+    ),
+]


I would remove the piece that compares to search params. They have their own doc explaining the performance pitfalls, this can stay about 'exact match'.

Suggested change

QueryUriExactMatch = Annotated[

FilterParam[str | None],

Depends(

filter_param_factory(

AssetModel.uri,

str | None,

filter_name="uri",

description=(

"Exact-match filter on the full asset URI. Compiles to an indexed equality "

"comparison (``uri = ...``), so it is far faster than ``uri_pattern`` (which uses "

"``ILIKE '%...%'`` and cannot use an index) for resolving a single asset by its "

"known URI."

),

)

),

]

QueryUriExactMatch = Annotated[

FilterParam[str | None],

Depends(

filter_param_factory(

AssetModel.uri,

str | None,

filter_name="uri",

description=(

"Exact-match filter on the full asset URI. Compiles to an indexed equality "

"comparison (``uri = ...``).

),

)

),

]

pierrejeambrun · 2026-07-07T09:10:59Z

+QueryUriExactMatch = Annotated[
+    FilterParam[str | None],
+    Depends(
+        filter_param_factory(
+            AssetModel.uri,
+            str | None,
+            filter_name="uri",
+            description=(
+                "Exact-match filter on the full asset URI. Compiles to an indexed equality "


We could make this filter accept multiple values.

To allow ?uri=some_uri1&uri=some_uri2 to get both assets with those URI. (And it makes more sense to me for a list endpoint to accept multiple values, otherwise it's really close to a single GET endpoint and maybe we should instead have GET assets/{uri}, but lets not go there)

It's really a simple change, as an exemple for implementation:

QueryTIStateFilter = Annotated[ FilterParam[list[str]], Depends( filter_param_factory( TaskInstance.state, list[str], FilterOptionEnum.ANY_EQUAL, default_factory=list, transform_callable=_transform_ti_states, ) ), ]

pierrejeambrun · 2026-07-07T09:16:29Z

+
+def upgrade():
+    """Apply Add index on asset.uri."""
+    with op.batch_alter_table("asset", schema=None) as batch_op:
+        batch_op.create_index("idx_asset_uri", ["uri"], unique=False)
+
+
+def downgrade():
+    """Unapply Add index on asset.uri."""
+    with op.batch_alter_table("asset", schema=None) as batch_op:
+        batch_op.drop_index("idx_asset_uri")


It's fine to have a 'filter/order_by' in the API that isn't backed by an index. Indeed it will be slow if the table grows out of control but does the table grows out of control in most common cases? I would avoid adding an index (costly) just for a single filter of a single endpoint.
I'm not sure here but I would say the table isn't huge most of the time. (millions of assets) and no index is probably fine for most? (the index probably wasn't there in 2.x as well)

~~If someone happen to have a huge asset table they can add their own index to improve performance following https://github.com/apache/airflow/blob/main/airflow-core/docs/howto/performance.rst~~

I just checked, in 2.x there was an index indeed, so the table was worth indexing. Disregard my previous comment, this makes sense.

pierrejeambrun · 2026-07-07T09:17:15Z

+        # Single-column index so exact-match lookups by URI (GET /assets?uri=...) can use an
+        # index; the composite index above leads with ``name`` and cannot serve uri-only queries.


I would remove this, to not mention the API layer in ORM models.

Address review feedback: let the exact-match uri filter accept repeated values (?uri=a&uri=b) so the list endpoint can resolve several assets in one call, target the migration at 3.4.0 (3.3.0 is already released), and drop the newsfragment since this is not a behaviour change that needs a user warning.

seanmuth requested review from XD-DENG, ashb, bbovenzi, bugraoz93, choo121600, ephraimbuddy, guan404ming, henry3260, jason810496, pierrejeambrun, rawwar, ryanahamilton, shubhamraj-git and vatsrahul1001 as code owners July 6, 2026 18:54

boring-cyborg Bot added area:API Airflow's REST/HTTP API area:db-migrations PRs with DB migration area:UI Related to UI/UX. For Frontend Developers. kind:documentation labels Jul 6, 2026

seanmuth force-pushed the restore-asset-uri-exact-lookup branch from d549e9b to 5416302 Compare July 6, 2026 18:54

pierrejeambrun reviewed Jul 7, 2026

View reviewed changes

pierrejeambrun added this to the Airflow 3.4.0 milestone Jul 7, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add exact-match uri filter to GET /assets endpoint#69489

Add exact-match uri filter to GET /assets endpoint#69489
seanmuth wants to merge 2 commits into
apache:mainfrom
seanmuth:restore-asset-uri-exact-lookup

seanmuth commented Jul 6, 2026

Uh oh!

pierrejeambrun left a comment •

edited

Loading

Uh oh!

pierrejeambrun Jul 7, 2026 •

edited

Loading

Uh oh!

pierrejeambrun Jul 7, 2026

Uh oh!

pierrejeambrun Jul 7, 2026

Uh oh!

pierrejeambrun Jul 7, 2026

Uh oh!

pierrejeambrun Jul 7, 2026 •

edited

Loading

Uh oh!

pierrejeambrun Jul 7, 2026

Uh oh!

pierrejeambrun Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		@@ -0,0 +1 @@
		Add an indexed exact-match ``uri`` query parameter to ``GET /api/v2/assets`` for fast single-asset lookup by URI (much faster than ``uri_pattern``, which uses an unindexed ``ILIKE '%...%'``).

		# Single-column index so exact-match lookups by URI (GET /assets?uri=...) can use an
		# index; the composite index above leads with ``name`` and cannot serve uri-only queries.

Uh oh!

Conversation

seanmuth commented Jul 6, 2026

What

Why

What changed

Was generative AI tooling used to co-author this PR?

Uh oh!

pierrejeambrun left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pierrejeambrun Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pierrejeambrun Jul 7, 2026

Choose a reason for hiding this comment

Uh oh!

pierrejeambrun Jul 7, 2026

Choose a reason for hiding this comment

Uh oh!

pierrejeambrun Jul 7, 2026

Choose a reason for hiding this comment

Uh oh!

pierrejeambrun Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pierrejeambrun Jul 7, 2026

Choose a reason for hiding this comment

Uh oh!

pierrejeambrun Jul 7, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pierrejeambrun left a comment •

edited

Loading

pierrejeambrun Jul 7, 2026 •

edited

Loading

pierrejeambrun Jul 7, 2026 •

edited

Loading