ci: weekly data-ingestors HEAD-drift canary + mirrored-source alert (cli#289)

[LukasWodka]

Fixes #289. Part of epic tracebloc/backend#1106 (WS-C.4).

What

New dedicated workflow .github/workflows/head-drift-canary.yml — weekly (Mon 06:00 UTC) + workflow_dispatch, advisory only: no PR trigger, and the run stays green even when drift is found. The signal channel is a single tracking issue, not a red build.

It watches data-ingestors develop HEAD (deliberately not the pin) two ways:

1. Goldens check vs HEAD — scripts/sync-validator-goldens.sh --check against a checkout of HEAD: catches verdict/value changes the next pin bump would have to adopt.
2. Mirrored-source drift — git content comparison (pin vs HEAD, equivalent to hashing each file at both refs) over the upstream surface the internal/push + internal/schema mirror tracks: validators/, modalities/, utils/{coercion,columns,constants,validators_mapping}.py, ingestors/{preflight,csv_ingestor,record_processor,base}.py, file_transfer.py, schema_inference.py, database.py, schema/. This catches the di#365 class: a new or extended upstream validator with no corpus case yet — invisible to the goldens check by definition.

On either signal (or an unreachable pin SHA — rewritten upstream history) it opens or comments on ONE tracking issue, keyed by the head-drift-canary marker label (created idempotently with gh label create --force; open-issue lookup by label, comment if found, create if not). The issue body carries pin/HEAD SHAs, both verdicts, the changed-file list, and the pin-bump playbook (cli#286 shape).

Design notes

• Never blocking: cron/dispatch only; the goldens step captures its exit code instead of failing; the reporting step raises the issue. A checkout failure would red the cron run — acceptable for a canary, it blocks nothing.
• Issue creation uses github.token (permissions: issues: write). Since GITHUB_TOKEN events don't trigger downstream workflows, the tracking issue will NOT be auto-added to the kanban by add-to-kanban.yml — it's an operational pointer, the actual work happens via a pin-bump ticket.
• All shell interpolation goes through env: (no ${{ }} inside run: script bodies beyond env mapping); the issue body is built with printf + --body-file.
• Cross-repo checkout: same pattern as the goldens-drift job (ci: goldens-drift job — validator goldens vs data-ingestors @ pin (cli#287) #308) — data-ingestors is currently public, default token works; CROSS_REPO_READ_TOKEN preferred if it ever exists.

Verification (what actually ran locally)

• Detection logic against real history: the exact git diff --name-only <pin> HEAD -- <paths> at the OLD pin 7b4ecac flags precisely the di#358+di#365 surface (9 files, incl. mask_id_validator.py, image_validator.py, coercion.py); at the current pin 8f89aec it returns empty. Both directions correct.
• Goldens check vs HEAD ran green locally (checkout at 8f89aec, deps from its requirements.txt).
• Issue-body construction exercised with sample env — renders as intended (table + fenced file list + marker comment).
• actionlint clean; YAML parses. Not run in real Actions (schedule-only; fires once merged — can be smoke-tested with workflow_dispatch after merge). No test issue was created against the repo.

🤖 Generated with Claude Code

---

Note

Low Risk
CI-only advisory automation with read/issue permissions; no application runtime or auth paths change, and drift never fails the workflow.

Overview
Adds .github/workflows/head-drift-canary.yml, a weekly (Mon 06:00 UTC) and manually dispatchable advisory job that compares scripts/.data-ingestors-ref to tracebloc/data-ingestors develop HEAD—not the pinned SHA used in PR CI.

It flags stale pins in two ways: running sync-validator-goldens.sh --check against the HEAD checkout (goldens/verdict drift), and a git diff --name-only over the mirrored validator/schema surface between pin and HEAD (upstream changes with no golden flip yet). Unreachable pin SHAs are treated as drift too.

When any signal fires, the workflow does not fail; it opens or comments on a single head-drift-canary-labeled issue with pin/HEAD SHAs, drift details, changed paths, and the pin-bump playbook. issues: write and cross-repo checkout follow the existing goldens-drift pattern (CROSS_REPO_READ_TOKEN optional).

^{Reviewed by Cursor Bugbot for commit 103a541. Bugbot is set up for automated code reviews on this repo. Configure here.}

[LukasWodka]

👋 Heads-up — Code review queue is at 55 / 30

Above the WIP limit. The team convention is to review existing PRs before opening new work.

Open PRs currently in Code review (oldest first):

• averaging-service#181 — feat(weights): normalize-on-read — mixed cycles average instead of rejecting (SEC-03 §8 step 2) · author: @shujaatTracebloc · no reviewer assigned
• averaging-service#182 — feat(weights): averaging writes SafeTensors (TF + PyTorch) — SEC-03 §8 step 4 · author: @shujaatTracebloc · no reviewer assigned
• backend#1079 — feat(global_meta): edge dataset_meta exposure + attributes contract at ingest (#924 G4a) · author: @divyasinghds · no reviewer assigned
• backend#1086 — docs(rfc): SafeTensors weight-format migration — SEC-03 Phase 1 (RFC 0004) · author: @shujaatTracebloc · no reviewer assigned
• backend#1093 — chore(deps): bump django from 5.2.14 to 5.2.15 · author: @dependabot · no reviewer assigned
• backend#1095 — feat(experiment): configurable preprocessing knobs incl. tabular scaler — RFC 0003 L1 + L1b (#1094) · author: @LukasWodka · no reviewer assigned
• backend#1100 — fix(boot): pin SDK install to tracebloc==0.11.2, drop 404 dev line (#1098) · author: @LukasWodka · reviewer: @saqlainsyed007
• backend#1105 — perf(api): query micro-fixes — notifications N+1, cached data_scientist, composite index, sampling (#975) · author: @aptracebloc · no reviewer assigned
• backend#1107 — perf(experiments): prefetch ExperimentListSerializer relations + cache carbon intensity (#977) · author: @aptracebloc · reviewer: @saadqbal
• backend#1108 — test(api): CLI response-shape contract fixtures for the 9 CLI endpoints (WS-D.2) · author: @LukasWodka · reviewer: @saadqbal

Pull from review before opening new work. (This is a nudge from the kanban WIP check, not a block.)

[LukasWodka]

@BugBot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 103a541. Configure here.}

[saadqbal]

👍 Clean advisory canary — cron/dispatch only, succeeds on drift (gates nothing), and the single-issue reuse (comment vs duplicate) keeps it quiet. issues: write is the minimal grant for the alert and it only runs on trusted triggers, so no injection surface. Good offset scheduling vs the other weekly jobs.