Skip to content

fix(arcgis): return incidents the service has not geocoded - #7

Merged
NateShoffner merged 3 commits into
masterfrom
fix/arcgis-missing-incidents
Aug 11, 2026
Merged

fix(arcgis): return incidents the service has not geocoded#7
NateShoffner merged 3 commits into
masterfrom
fix/arcgis-missing-incidents

Conversation

@NateShoffner

Copy link
Copy Markdown
Owner

Closes #3

Problem

The ArcGIS service routinely holds rows whose geometry is null, because the address has not been geocoded yet or never will be. Any query that asks for geometry drops those rows without comment, and a spatial filter implies one, so those incidents were invisible to the client.

Same layer, same instant, only the output flags varying:

geom=false=4   geom=true=2   countOnly=4

The dropped rows carry no geometry at all: returnExtentOnly on one returns NaN. They are unreachable through every output variant tried, including f=geojson, returnCentroid, and alternate outSR values.

The spatial extent was never the cause. returnGeometry=true with no spatial filter at all still returns the reduced set, so widening the envelope or splitting the county into quadrants could not have helped.

Fix

Each layer is queried twice: once with returnGeometry=false for the authoritative row set, once with returnGeometry=true purely as a coordinate lookup keyed on IncidentNumber. Incidents come from the first query so nothing is lost, and a failed lookup only costs the coordinates. The envelope is removed, since where=1=1 alone already returns every row the service holds.

Also in here:

  • ArcGISIncident.coordinates is now Optional, since an incident can legitimately have none. The JSON decoder and the example are updated to match.
  • Dedupe on IncidentNumber. The service sometimes carries one incident as two rows, typically one geocoded and one not.
  • Fixes a NameError on the non-200 path, which raised ArcGISException(error) before error was assigned. That is the F821 currently failing the lint gate on master.

Measurements

Over a 90 minute window, 180 samples at 30 second intervals, 18 distinct incidents:

  • 26 of 180 samples held rows a geometry query would have dropped, up to half the rows in the worst sample
  • The client returned coordinates for 741 of 775 incidents (96%)
  • Feed and web clients returned identical incident sets in 180 of 180 samples

Known gaps, not addressed here

The service never publishes ROUTINE TRANSFER-CLASS 3 or EMS ACTIVITY incidents at all. Four such incidents were absent for their entire lifetimes. This is an upstream filter on the public map, and no client change can recover them. Callers needing those should use FeedClient or WebClient.

docs/arcgis-service-behavior.md records this along with the ingest lag, clear-down lag, and transient single-poll dropouts.

@NateShoffner
NateShoffner merged commit 3a8005d into master Aug 11, 2026
2 checks passed
@NateShoffner
NateShoffner deleted the fix/arcgis-missing-incidents branch August 11, 2026 06:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incidents missing throughout county (ArcGIS)

1 participant