fix(arcgis): return incidents the service has not geocoded - #7
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3
Problem
The ArcGIS service routinely holds rows whose geometry is null, because the address has not been geocoded yet or never will be. Any query that asks for geometry drops those rows without comment, and a spatial filter implies one, so those incidents were invisible to the client.
Same layer, same instant, only the output flags varying:
The dropped rows carry no geometry at all:
returnExtentOnlyon one returnsNaN. They are unreachable through every output variant tried, includingf=geojson,returnCentroid, and alternateoutSRvalues.The spatial extent was never the cause.
returnGeometry=truewith no spatial filter at all still returns the reduced set, so widening the envelope or splitting the county into quadrants could not have helped.Fix
Each layer is queried twice: once with
returnGeometry=falsefor the authoritative row set, once withreturnGeometry=truepurely as a coordinate lookup keyed onIncidentNumber. Incidents come from the first query so nothing is lost, and a failed lookup only costs the coordinates. The envelope is removed, sincewhere=1=1alone already returns every row the service holds.Also in here:
ArcGISIncident.coordinatesis nowOptional, since an incident can legitimately have none. The JSON decoder and the example are updated to match.IncidentNumber. The service sometimes carries one incident as two rows, typically one geocoded and one not.NameErroron the non-200 path, which raisedArcGISException(error)beforeerrorwas assigned. That is the F821 currently failing the lint gate on master.Measurements
Over a 90 minute window, 180 samples at 30 second intervals, 18 distinct incidents:
Known gaps, not addressed here
The service never publishes
ROUTINE TRANSFER-CLASS 3orEMS ACTIVITYincidents at all. Four such incidents were absent for their entire lifetimes. This is an upstream filter on the public map, and no client change can recover them. Callers needing those should useFeedClientorWebClient.docs/arcgis-service-behavior.mdrecords this along with the ingest lag, clear-down lag, and transient single-poll dropouts.