Add ORDS v0.3 repair data export endpoint - #58
Conversation
Groups store country_code as alpha-2 but the Open Repair Data Standard requires alpha-3, and Fixometer::getAllCountries only maps alpha-2 to a localised display name. Checked in as a static map rather than pulling in league/iso3166 so the exporter stays dependency-free for other instances. The key set is verified to match lang/instances/base/en/countries.php.
devices.problem is unsanitised free text with no Purify mutator, so it can carry raw HTML, email addresses, phone numbers and URLs with tracking parameters. Redaction is a standalone pass rather than logic inside the mapper so it can be toggled, tested against fixtures and audited on its own, and it reports per-run counts by type so an export can be checked before handover. Every pass fails closed: a regex error yields an empty string rather than leaking the unredacted original. The phone heuristic uses a nine-digit floor to keep hyphenated substance such as rpm ranges, part numbers and firmware versions intact.
Emits the fourteen standard columns in spec order. Most map straight across because the standard was derived from this schema, but three need work: year_of_manufacture is not stored so it is derived from the event year minus the item age, country needs alpha-3, and product_category_id needs a name lookup because our idcategories do not match the published ones. Vocabulary handling follows the Open Repair Alliance's published releases rather than the table schema, which is stale in two places: the standard collapses our screen-size and laptop-size splits into single categories, repair_status carries Unknown as a real value, and the barrier wording drops the "the" we seed.
Holds the instance identity and the vocabulary maps in config rather than a database table so a change shows up in a diff and is versioned alongside the standard it tracks. id_prefix and data_provider are deliberately undefaulted. The identifier is a stable key the consumer upserts on across releases, so serving under an unassigned or borrowed namespace would overwrite another provider's records; the endpoint refuses to serve while either is blank.
Adds GET /api/public/v2/repairs behind a new repairs:read scope, serving JSON or CSV with filters for update time, event window, powered category and pagination. The per-page ceiling is 1000 rather than the events endpoint's 100 because this is a bulk export rather than a display API. Visibility mirrors PublicEventController: approved events on approved groups, soft-deleted rows excluded, and allowed_network_ids honoured. The network restriction is a subquery rather than a join so a group belonging to several permitted networks does not multiply its devices. The feature flag moves from the shared public/v2 group onto each scope so the export can ship dark without taking the live events API down with it. CSV output escapes cells opening with a formula character, which spreadsheet software would otherwise execute on open, and carries the pagination and sync metadata in headers since the format has no envelope for it. Documented in docs/public-repairs-api.md alongside the events API.
repairs:read is the first scope beyond events:read, so the option help now lists the valid values instead of only showing the default.
Covers the auth surface including an events:read token being refused on repairs and the reverse, the feature flag in both directions, the config guards, all fourteen columns against a fixture, CSV column order and formula escaping, visibility exclusions, network restriction, filters, pagination and the redaction passes.
Ships every value blank or off. The identity settings are per deployment rather than defaulted in the chart, and the endpoint refuses to serve until an instance sets them.
8179e34 to
3618e63
Compare
|
QA 👍
|
|
1. The new OPTIONS guard never runs, and preflight stopped 404ing
Before this PR the group ran Nothing covers it. config([
'restarters.features.public_events_api' => false,
'restarters.features.public_repairs_api' => false,
]);
$this->options('/api/public/v2/repairs')->assertStatus(404);Route-level middleware won't fix it, since group middleware merges in front. The check has to go in the group list ahead of ->middleware(['publicApiEnabled', 'publicApiCors'])with 2.
$raw = $request->input('powered');
if ($raw === null || $raw === '') {
return;
}
$normalised = filter_var($raw, FILTER_VALIDATE_BOOL, FILTER_NULL_ON_FAILURE);Four non-blocking notes: CORS headers, the chart's problem-text default, the error body, the categories joinA browser can't read the CSV pagination headers
A question: the chart ships
|
Description
Adds
GET /api/public/v2/repairs, which exports repair records in Open Repair Data Standard v0.3 for the Open Repair Alliance. They asked for this directly since they currently hand-export from the database. No exporter exists in this fork or upstream, so this is the first codified one. Ships behindFEATURE__PUBLIC_REPAIRS_API, off by default.Changes
GET /api/public/v2/repairsunder a newrepairs:readscope, JSON or CSV, withupdated_since,event_start,event_end,powered,pageandper_pagefilters. Per-page ceiling is 1000 rather than the events endpoint's 100 since this is a bulk export. Visibility mirrorsPublicEventController: approved events on approved groups, soft-deletes excluded,allowed_network_idshonoured. The network filter is a subquery rather than a join, so a group in several permitted networks doesn't multiply its devices. Files:API/PublicRepairController.php,EnsurePublicRepairsApiEnabled.php.public/v2group onto each scope, so repairs can ship dark without taking the live events API down with it. Files:routes/api.php,bootstrap/app.php,config/restarters.php.year_of_manufactureisn't stored so it's derived from the event year minusdevices.age,countryconverts alpha-2 to alpha-3, andproduct_category_idis a name lookup because ouridcategoriesdon't match theirs. Files:Services/Ords/OrdsRecordMapper.php,Helpers/Iso3166.php.config/ords.phpfollows ORA's published data rather than theirtableschema.json, which is stale. The standard collapses our screen-size and laptop-size splits, carriesUnknownas a realrepair_status, and drops the "the" from the barrier we seed.devices.problemis free text with no Purify mutator.ProblemTextScrubberstrips HTML and redacts emails, phone numbers, digit runs of 8 or more, and URL query strings, logging counts by type per request. Names aren't pattern-detectable and aren't removed.ORDS_ID_PREFIXandORDS_DATA_PROVIDERhave no defaults and the endpoint 503s while either is blank. The id is a stable key ORA upserts on across releases, so a borrowed namespace would overwrite another provider's rows.docs/public-repairs-api.md, alongside the existing events one.Output was validated against ORA's published
tableschema.json. Column names and order match exactly and every declared constraint passes, except theiridregex, which uses a hyphen while all 305,649 of their own published rows use an underscore.Deferred:
updated_sincedoesn't catch approval flips, so approving an old event leaves it invisible to an incremental consumer. Thecategoriesinner join silently drops devices with an orphanedcategory. Pagination is offset-based so a full crawl is quadratic. Throttling runs after auth, so invalid tokens hit the database unrated.QA Notes
We'll need to enable the flag on the test cluster and validate a full export against the spec.