Skip to content

fix: apply sortby when building vector tiles - #275

Merged
vincentsarago merged 4 commits into
developmentseed:mainfrom
slesaad:fix/tile-sortby
Sep 3, 2026
Merged

fix: apply sortby when building vector tiles#275
vincentsarago merged 4 commits into
developmentseed:mainfrom
slesaad:fix/tile-sortby

Conversation

@slesaad

@slesaad slesaad commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Collection.get_tile() accepts a sortby argument and never uses it, so ?sortby= has no effect on vector tiles. /items sorts correctly; /tiles silently drops the parameter. Feature order inside an MVT is the only draw-order control renderers such as deck.gl have, so overlapping polygons stack arbitrarily. Present in every released version I checked (1.0.0 through 1.6.0).

Decisions

ORDER BY goes before LIMIT. Beyond fixing feature order, this makes truncation deterministic — LIMIT with no ORDER BY currently keeps an arbitrary subset once a tile exceeds tipg_max_features_per_tile.

Applied unconditionally, not only when sortby is passed. With no sortby, _sortby() already falls back to the primary key, so this matches /items. It adds a sort to the default path in exchange for deterministic truncation. Happy to gate it on sortby if you would rather not pay that.

Ordering is repeated inside the ST_AsMVT aggregate. PostgreSQL only guarantees the order rows reach an aggregate when the ORDER BY is inside the aggregate call. The CTE-level ORDER BY does work today, because the LIMIT forces the sort to be kept — but that protection is incidental, and the failure mode is a silent 200 with wrongly stacked features. To be clear about how strong this is: I could not construct a plan that actually reorders, so this is insurance, not an observed fix. No measurable cost (timings overlap fully).

Only an explicit sortby adds the aggregate ORDER BY. The primary-key default keeps the CTE ordering alone, so existing properties=-restricted requests are unaffected.

A sort column excluded by properties= is rejected rather than silently unsorted. The aggregate can only order by columns present in its input row. The alternative — quietly widening the projection — would return a property the caller explicitly excluded. Raises InvalidPropertyName (404), consistent with how /items treats an unknown column.

_sortby() split into _sort_expressions(), which returns (column, SQL fragment) pairs so get_tile() can reuse one parse for the CTE, the aggregate, and the projection check. _sortby() is now a thin wrapper; its existing callers are unchanged.

Behaviour changes

  • Invalid sort column on /tiles: 200 (ignored) → 404, matching /items
  • ?properties=x&sortby=y where y is not in x: 200 unsorted → 404
  • Tile feature order now follows sortby; LIMIT truncation is deterministic

Testing

Four tests, each watched fail against main first. Full suite passes (106 → 108).

Also verified against real data (NOAA HMS smoke polygons, 932 features, three overlapping density classes). Counting features drawn after something heavier was already down, on one tile of 257 features:

query buried
no sortby 167 / 257
sortby=density_rank 0

Note

This supports the AIR4US work for visualizing HMS smoke data. The vector tiles have density property - denser features should be drawn on top of overlapping less dense features. Even though sortby is advertised as a query param for tiles, it isn't honored.

`Collection.get_tile()` accepts a `sortby` argument and never uses it, so
`?sortby=` has no effect on the order of features inside an MVT. Feature
order is the only draw-order control a renderer like deck.gl has, so
overlapping polygons render in whatever order the query plan produced.

Adds two tests, both failing:

- `test_tile_sortby` — `sortby=-ogc_fid` returns features in physical
  order (601, 602, 603, ...) rather than descending. The ascending case
  passes only because physical order happens to match it, which is why
  the descending assertion is the one that matters.
- `test_tile_sortby_invalid_column` — an unknown sort column returns 200
  on /tiles because the value is never parsed; /items returns 404.
Adds the ORDER BY clause `get_tile()` was already parsing everywhere
else, positioned before LIMIT so `?sortby=` controls both the order
features appear in the tile and, when a tile exceeds
`tipg_max_features_per_tile`, which features survive truncation. The
latter was previously arbitrary: LIMIT with no ORDER BY picks whatever
subset the plan happens to produce.

Applied unconditionally. With no `sortby`, `_sortby()` falls back to the
collection's primary key, matching /items and making the default path
deterministic too.

An unknown sort column now raises InvalidPropertyName (404) on /tiles,
as it already did on /items, instead of being silently ignored.
An ORDER BY in the CTE makes LIMIT truncation deterministic but does not
guarantee the order rows reach the aggregate: a CTE has not been an
optimisation fence since PG12, so it can be inlined and a parallel plan
may reorder freely. PostgreSQL only guarantees aggregate input order when
the ORDER BY is inside the aggregate call, so the previous commit alone
can pass today and silently regress under a different plan or a larger
table.

Splits the ORDER BY builder into `_sort_expressions()`, which returns
(column, SQL fragment) pairs, so `get_tile()` can reuse the parsed sort
for both the CTE and the aggregate. `_sortby()` is now a thin wrapper and
its callers are unchanged.

The aggregate can only order by columns of `t`, so an explicit `sortby`
naming a column that `properties` filters out is now rejected with 404
rather than silently unsorted. Only an explicit `sortby` adds the
aggregate ORDER BY; the primary-key default keeps the CTE ordering alone,
so existing `properties`-restricted requests are unaffected.
The parameter was already listed for /tiles but described as if it only
sorted a feature list. Says what it actually controls — draw order within
the tile and which features survive `limit` truncation — and documents
that the sort column has to be in `properties` when that is set.
@slesaad slesaad changed the title fix(collections): apply sortby when building vector tiles fix: apply sortby when building vector tiles Sep 2, 2026
@slesaad
slesaad marked this pull request as ready for review September 2, 2026 18:31
@vincentsarago
vincentsarago merged commit 12e5fc8 into developmentseed:main Sep 3, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants