Run the canonical import daily and add lead importers for the desk by JesperDramsch · Pull Request #501 · JesperDramsch/python-deadlines

JesperDramsch · 2026-10-14T03:10:26Z

Fifth and last PR in the desk stack, on top of #500. Two things: the canonical import runs every day and can do so without supervision, and three free lead importers feed the desk queue. Nothing an importer reads reaches _data/ without the desk checking it on the primary source. It also carries the fixes from a review of the whole stack (below).

Why PyCon Ireland and PyCon SK never arrived

The python.org calendar listed PyCon Ireland's move to 21 November from 9 September, and PyCon SK 2027 from 23 September. Neither reached the data until #494 and #495. canonical-source-update.yml only ran when an outside webhook fired, and the last run with PAYLOAD_SOURCE: ics was on 19 July. The workflow now also runs daily at 05:37 UTC. A moved date, like Ireland's, becomes a merge question in the desk queue (#500) rather than being silently kept.

The desk pushes to auto/conference-updates too. So this workflow and enrich-tba-conferences.yml now merge main into it instead of rebasing, and push without force. A daily rebase would rewrite the desk's commits and break the commit links in the PR table.

Safeguards for an unattended import

Run against the live sources on today's data, the import added two duplicates and one fake event, and it stopped on two rows without a website. Each has a deterministic fix, with tests in tests/test_import_safeguards.py:

Seen	Cause	Fix
`XtremePython 2026` next to Xtreme Python 2026	the merge ignored `alt_name`	`apply_alt_names`: a remote name that is a YAML entry's `alt_name` takes that entry's name. A shared alt_name is left alone. A name that is also a series of its own (`PyHEP`) is renamed only when it starts on the same day as the alt_name entry.
`Python Nordeste (PyNE )` next to Python Nordeste 2026	year removal left `(PyNE )`	`normalize_conference_name` tidies the brackets
`Python E-Commerce Forum 2026`	the import never read the denylist	both importers drop rows on denylisted domains (`tidy_conf/denylist.py`, the list the critic uses); a row on a `review` domain goes to the queue as a candidate
validation error, nothing written	PyDay Chile Santiago and Rancagua have no website in the sheet	the organizers import leaves out merged rows that still have no link, and queues them for the desk to find a website

Lead importers (`utils/desk_leads.py`, `desk-leads.yml`)

pretalx (daily). Two checks:
- For upcoming entries hosted on pretalx.com, it reads the CfP page and compares the deadline pretalx enforces with ours, in our timezone or AoE. Today it finds two:
  - PyCascades 2027 closes at 05:00 PDT on 26 October, while we have 23:59 AoE.
  - PyCon Namibia 2027 closed on 13 October, while we have 21 September, which is probably an extension.
- For series hosted on pretalx, it probes next year's slug.
pretalx's robots.txt disallows /api/, so the job only reads /{slug}/cfp pages, one second apart.
developers.events (weekly). Leads for Python-named events we don't track, including names like EuroPython and DjangoCon. The data is CC BY-NC, so a record carries only the name, year and link, and the desk takes every value from the primary source. A python tag alone doesn't count: in a live run, 22 of 23 tag matches were general events (DevFests, SQLSaturday).
Registries (weekly). Series on the PSF PyCon trademark list or on pycon.org (PyCon/pycon.org) that the data doesn't track. Today: PyCon Nepal, Romania, Venezuela, Arewa, Bolivia, PyHausa, PyKatsina, PythonCamp, and the PyData events page.

Calling All Papers is left out. Its API host's robots.txt is Disallow: /, and it states no data licence.

Matching follows AGENTS.md: series names and alt_names through titles.yml, or the link with years taken out. A shared domain on its own only makes a MAYBE. Each finding has a stable id, so a weekly job queues it once, and again only if it changes.

Fixes from a review of the whole stack

With all five PRs in place, I reviewed the stack as one change, then fixed what it found in several rounds, each checked by an independent reviewer. The fixes land here because they cut across #498 to #500 (desk.md, the skills, the guard hook, the queue tools) as well as this PR's importers.

Canonical import (this PR):

Country names were replaced inside every field the merge compared, so the alt_name PyCon Korea became PyCon South Korea, then PyCon South South Korea the next day. Only place is normalised now.
merge_duplicates keyed on the raw place, so an archived "London, United Kingdom" next to the importer's "London, UK" kept two entries and stopped the next import. It keys on the normalised place, and a later duplicate's empty value no longer raises or replaces a value (a blank timezone used to replace UTC).
An upcoming entry with two or more extra_places or location items stopped both importers.
In January both importers dropped last year's editions the sort had not archived yet, and check_data_loss failed the job.
check_links never checked coc.
Name normalisation is idempotent (0+) gave 0 ) on one run and 0) on the next).
Tenants of shared hosts (github.io, gitlab.io, netlify.app, pages.dev, vercel.app, readthedocs.io, org.pl) count as different sites for the redirect check.

Rolling branch:

utils/merge_data.py: main and auto/conference-updates sort the data on their own, so once both had sorted an entry the branch edited, every merge of main conflicted in _data/. It merges entry by entry on (conference, year), stops on a real conflict, and exits 2 for a conflict outside the data files. Both writers and the desk use it.
Both writers share one concurrency group and retry a rejected push without force. When the rolling PR was merged mid-run, they stop instead of pushing old history.
utils/desk_pr_body.py reads the PR body, splices the desk's part in front of  and writes it in one command, so rows the writers append in between are kept, CRLF included.
link-check.yml keys its concurrency group by ref, so pushes to the rolling branch no longer cancel the weekly run on main.

Queue and worklist:

Leads can be snoozed (next_check), and snoozed items survive the 50-item cap. The desk snoozes leads for seven days, so it stops researching the same lead every day.
A repeated manual "check this conference" run gets its own id, so it is not taken as already handled.
Candidates route by the worklist's own match: an edition the data holds goes to the enricher, a missing one to the new-year-checker, an unmatched name to the scouter.
The TBA rotation no longer picks cfp: None or Cancelled entries, deadline checks count from the deadline's UTC date, and enricher items without an upcoming entry bind their archived edition (from Add the desk worklist and fix cfp_candidates series matching #499, merged in).
Each missing edition is its own new-year-checker item, past editions included, so two missing years, or a missing year next to the series' stale-series check, no longer fold into one item that looks for only one of them.

Desk procedure and skills:

Maintainer requests go to the front of the worklist, and the "Feedback read to" cut-off moves only once each one is settled, so an interrupted run loses none.
Identity: step 5 acts only on comments by user 2620316 posted through no GitHub App, skips the desk's and the review Routine's own posts, and every reply is posted from a file.
Step 2 builds from main's side when the rolling branch lacks main, so .claude/ and the guard stay in the working tree.
The desk handles a rolling PR that was merged, or a branch opened again from main, mid-run.
Subagents research in parallel only across conferences: a conference's next item starts once its earlier one is pushed, so two items never add the same edition.
The enricher covers past editions and pretalx drift (TBA becomes cfp, a later date goes in cfp_ext, an earlier one only with a source). The new-year-checker looks for missing past editions as well as the next one.

Guard hook:

Every MCP tool and the Cron, RemoteTrigger and ScheduleWakeup tools reach the hook, and desk runs block them.
A missing deny list blocks every desk call, and in desk runs the settings command turns any failure of the hook itself (a crash on import, a missing interpreter) into a block. Its tests pass on Python 3.9, apart from one that needs 3.10 to list the standard library.
In desk runs, a push must go to origin and write only auto/conference-updates or desk-queue, with no force, delete, mirror or prune in any spelling git accepts, abbreviations included. gh runs only as gh api, with four write endpoints.
Subagents get read-only git and gh api reads, and cannot edit the repository or run the scripts that write the data or the queue.
Desk runs refuse every git config change: -c and --config-env settings, git config writes and GIT_CONFIG* variables. That closes the alias, include and push-config routes in one rule.
env -S strings are split and inspected (and refused in desk runs), and wrapper options such as env -iC dir move the directory the guard resolves against.
git send-pack and git http-push follow the push rules, so git send-pack origin HEAD:main no longer moves main. An inline variable in a push refspec is resolved, and desk runs block a refspec the guard cannot resolve.
Desk subagents run as a new agent type, desk-research (.claude/agents/desk-research.md), limited to Read, Grep, Glob, WebFetch and WebSearch. The skills need nothing else. In desk runs the guard blocks an Agent call for any other type and any call from another subagent type, so the subagent shell rules (including the new block on redirections and file-writing programs into a repository) are a second layer.
desk.md runs a tamper check after each subagent: the git directory's config, hooks and info files, HEAD, and the guard, procedure and tool files, ignored files included. Anything changed stops the run before it pushes. Before each data commit, a stray-write check allows only this conference's accepted edits and the sort's own work.

Known limits:

REST reviews and review comments carry no app attribution, so for them the desk's identity check rests on the user id and the body checks.
The writers push with GITHUB_TOKEN, and GitHub refuses a GITHUB_TOKEN push that changes files under .github/workflows/. So when main changes a workflow, the writers cannot push their merge of main: merge it into auto/conference-updates once by hand, or merge the rolling PR.
Self-hosted pretalx instances and pretalx events with no English page are not covered by the drift check.
The guard does not inspect programs other than git and gh, or code an interpreter runs inline. Desk subagents have no shell, and the tamper check covers the rest.
The guard runs inside the session it guards. A GitHub ruleset on main (require a pull request, block force pushes and deletion) and one blocking force pushes on auto/conference-updates and desk-queue hold whatever happens in a session.

Needs your call

pretalx's API: asking pretalx whether a daily read of /api/events/ is acceptable would add discovery of new pretalx events. As built, the job doesn't touch the API.

Checks

The first review of this PR found nothing blocking; its five should-fix issues and three nits were fixed in 6b4b87f. Each round of the stack review above then had its own reviewer, and the guard rounds also had testers per mode running real git against local bare remotes. The tester for subagent shell commands did not finish; that mode now has no shell at all (desk-research).
Tests: the pinned uv run gives 2448 passed, 0 failed. The lead tests block sockets.
uvx ruff@0.4.4 check utils tests .claude/hooks: clean.
pre-commit run on every file this PR changes: passes.
actionlint on all workflows: clean.
python utils/sort_yaml.py --skip_links leaves _data/ unchanged.
utils/main.py against the live sources on the final head: exit 0, no entry added or removed. Its only data change is one tutorial deadline from the sheet (PyCon Colombia 2026), plus reordering by the sort. Without the importer's local cache, as in CI, it queues 79 merge questions and 2 candidates, with stable ids.
Every command in desk.md, with its placeholders filled in, passes the guard as the desk's main agent in a desk run.
Live smoke run of each lead source: 2 pretalx drifts and 9 registry leads. developers.events gives 1 lead today, PyConf, whose domain is on the denylist's review list. All records pass the queue checks and route to the expected skill.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

The desk queue lives on an orphan desk-queue branch: one JSON file per series under queue/, keyed by the slug of the series name, plus a README that init writes from the module so the format is documented in one place. GitHub Actions jobs add records with "add", from JSON lines or from options; the desk removes handled items with "remove". Each item gets a stable id from its source, kind, URL and payload, so the same finding queued twice is one item. A file keeps its 50 newest items and long payload strings are cut at 20,000 characters, flagged with "truncated". The module uses only the standard library, so any job can run it without the data tooling. clean_watch_title and the name lookup move here from desk_worklist.py, which now imports them, so the queue file key and the worklist's series matching read names the same way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

check-conference-update.yml ran a paid-API triage on every changed page, then two agent jobs that edited the data. The triage passed 26% of events at 38% precision and 58% recall, and never saw the data, so it flagged dates already recorded and dropped extensions and moved dates. The workflow now validates the payload and writes it to the desk-queue branch unchanged; the desk judges it on its next run with the data and the live site in hand. The payload is read from the event file rather than interpolated or passed through the environment, so page text cannot reach the shell and a long diff cannot exceed the environment size limit. The new composite action desk-queue-write is the one place that writes to the queue: it creates the orphan branch when missing, adds records with utils/desk_queue.py, and retries a rejected push from a fresh copy of the branch, never with force. Ids are stable, so a retry adds nothing twice. scheduled-extension-checker.yml goes: the desk's worklist checks every deadline 1, 3 and 7 days after it passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

A series counted as tracked only when an upcoming entry had exactly the same conference string, so a respelt upcoming edition left the archived series listed as a candidate: archive "Xtreme Python" against an upcoming "XtremePython", and archive "PyHEP.dev" (alt_name PyHEP) against an upcoming "PyHEP". Names are now compared through normalize_conference_name with the titles.yml mappings, ignoring case, accents and whitespace, and a series is tracked when its name or any alt_name of its archived editions matches the name or alt_name of an entry in conferences.yml. Editions are grouped by the same key, so "PyCon Mini Tokai" and "PyCon mini Tokai" are one series. The logic moves into find_candidates() so other scripts can call it; the CLI and its JSON output are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

The TBA enrichment keeps its deterministic link extraction: social accounts, sponsor pages and financial aid pages found on each conference's homepage. Deadlines, dates and time zones are now researched by the desk, so the full mode, its prompt and its API calls are removed, together with the --level option and the confidence handling only the API results needed. The workflow always runs the deterministic enrichment and no longer needs the API key. The canonical source workflow triggers it without a level input. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

build_worklist(data, queue_items, today) turns the conference data and the desk-queue items into one JSON list of work, each item routed to a skill: - enricher: the series has an entry in conferences.yml, or the queue item is a question, redirect or drift. - new-year-checker: the series is only in archive.yml or legacy.yml, or an importer candidate is MISSING_EDITION. - scouter: no series matches, or the item is a NEW_SERIES or MAYBE candidate or a registry change. Queue items are matched on payload names, then the watch title with its "[Conference] " prefix and " YYYY - Main" or " YYYY - CFP" suffix removed, then the queue file's series, using the same keys as cfp_candidates.py. When a name matches several series the item keeps its route and lists the matches in ambiguous_matches. Deadline checks fire 1, 3 and 7 days after the effective deadline (cfp_ext, else cfp), counted from its calendar date in the entry's timezone, AoE when unset. TBA entries that have not started and stale series from cfp_candidates.find_candidates() each come up on one weekday in seven, picked by a SHA-1 of the name. One scouting item closes the list unless --no-scout is given. Items for the same skill, conference and year are merged, and the order is fixed: deadline checks, queue items, rotations, scout. The CLI takes --queue-dir, --base, --today and --no-scout, prints the JSON on stdout and never touches the network. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

The import asks a person to settle conflicts it cannot resolve: which of two values to keep, whether a later deadline is an extension, whether two similar names are one series. In CI no one is at a terminal, so query_yes_no took the default and the question was lost. query_yes_no now takes an optional context. Without a terminal it still returns the default, so the import stays deterministic, and it also records the question with the conference, year, field, both values and the source. With DESK_QUEUE_OUT set, the record is written as a JSON line, and canonical-source-update.yml adds the file to the desk queue, where the enricher answers it from the live site. The JSON-lines writer moves into tidy_conf/queue_out.py, shared with the link checker's redirect records. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

A changedetection watch is titled after the edition it was created for, and later editions reuse the watch, so its signals keep carrying that old year. A signal for "PyCon US 2026" was bound to the PyCon US 2027 entry but kept year 2026: the item contradicted its entry and did not merge with the deadline check or TBA rotation for 2027, so the desk got the same entry twice. A queue item bound to an upcoming entry now takes that entry's year. When no entry matches the queue year, or the item has none, it is bound to the next edition that has not ended rather than the newest one, so a signal for a conference starting next week no longer goes to the TBA entry for the year after. A differing queue year is kept in context["queue_year"]. For the new-year-checker, a queue year at or before the latest known edition cannot be the edition it is looking for. It moves to context["queue_year"] and the item merges with the stale-series item for the series. The module docstring now says how names are matched and that the script needs the same packages as sort_yaml.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

The link check now records a link that ends on a different site instead of counting a 200 there as healthy, but nothing ran it on a schedule. redirect-scan.yml runs it every Sunday with DESK_QUEUE_OUT set and adds the records to the desk queue. The desk decides each one: a series that moved gets its new link, a lapsed or hijacked domain an archived copy. The scan is report-only. The link check's other edits stay with the writers that commit data. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

Since it normalises names through tidy_conf, cfp_candidates.py needs the same packages as sort_yaml.py, not PyYAML alone. The docstring now says so. A comment and a test name the known cost of counting the alt_names of every archived edition: an edition whose alt_name is its host conference hides the whole series while the host has an entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

The canonical import asks the same standing questions on every run (an end date one day apart, "Brasil" and "Brazil"), and the redirect scan finds the same redirects every week. Ids only deduplicated while an item was queued, so once the desk had handled one, the next run queued it again. desk_queue.py remove now records each id in its file's "handled" list, the last 500 per file, and add reports a handled id instead of queueing it. A changed finding has a new id and is queued as usual. For a redirect the id leaves out the status and the final URL's query and fragment, which can change between checks. A queue file that is not valid JSON stopped every writer bound for it, and under set -e the canonical merge was never committed. add now moves such a file to <key>.json.bad and starts a new one; remove and list skip it. The composite action turns these messages and rejected records into workflow warnings, and the canonical workflow's queue step can no longer hold back the merge. Also from review: - the follow-up "Is this an extension?" question is no longer queued: the cfp question before it carries both deadlines, while its record gave the YAML cfp as the cfp_ext value; - the redirect scan's link check has a step timeout below the job's, so a slow scan still queues what it found; - canonical-source-update gets actions: write, which gh workflow run needs to start the TBA enrichment; - the action matches refs/heads/desk-queue exactly. desk.md step 8 now removes items only through desk_queue.py remove. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

Run against the live sources on today's data, the import added: - "XtremePython 2026", a second copy of Xtreme Python 2026, whose alt_name is XtremePython; - "Python Nordeste (PyNE )", a second copy of Python Nordeste 2026: the calendar's "(PyNE 2026)" lost its year and kept the space; - "Python E-Commerce Forum 2026", on a domain the denylist rejects; and it stopped with a validation error on two sheet rows without a website (PyDay Chile Santiago and Rancagua), so nothing was written. Each now has a deterministic fix: - fuzzy_match gives a remote row named by a YAML entry's alt_name that entry's name, unless the alt_name is shared or is itself a YAML name; - normalize_conference_name tidies the brackets year removal leaves; - both importers drop rows on denylisted domains (tidy_conf/denylist.py, the list the critic uses); a row on a review domain is queued for the desk as a candidate instead; - the organizers import leaves out merged rows that still have no link. With these, the full pipeline on today's data exits 0 and adds nothing but the sheet's tutorial deadline for one archived edition. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

The import only ran when an outside webhook fired, and the calendar's webhook has been quiet since 19 July: PyCon Ireland's move to 21 November (in the calendar since 9 September) and PyCon SK 2027 (since 23 September) never reached the data until they were added by hand. The workflow now also runs every day at 05:37 UTC for all sources. Moved dates come back as merge questions in the desk queue. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

…tries utils/desk_leads.py writes queue records for the desk from three sources: - pretalx, daily: a drift record when an upcoming entry's deadline is not the instant its pretalx CfP page states (AoE when the entry has no timezone), and a MISSING_EDITION candidate when the next edition's CfP page, guessed from the latest slug, exists. - developers-events, weekly: upcoming Python events the data does not track, as MISSING_EDITION, MAYBE or NEW_SERIES candidates. - registry, weekly: names on the PSF PyCon trademark list and events in the pycon.org registry that no data file tracks. Only the public /<slug>/cfp pages of pretalx are read, a second apart, because its robots.txt disallows /api/. developers.events data is CC BY-NC 4.0, so its records carry only the event's name, year and link and are flagged lead-only. A find is tracked by the rules in AGENTS.md "Names", across all three data files; a shared host such as pydata.org or pretalx.com proves nothing on its own. The desk-leads workflow adds the records to the desk-queue branch. Nothing reaches the data without the desk checking it on the primary source. The queue drops the callingallpapers and psf-ics-drift sources, which are no longer planned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

A live run gave 23 leads, 22 of them from the "python" tag alone: DevFests, EnvoyCon, SQLSaturday, an Odoo hackathon. These are general events with a Python talk, which AGENTS.md leaves out, and each would cost the desk a scouter visit. The name match stays. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

- Candidate ids leave out "matched": it says which file holds the matched entry, so a lead got a new id, and was queued again, when the sort moved that entry to the archive. - A pretalx drift stores the deadline and zone instead of the sentence, which turns from "close" to "closed" once the deadline passes, and compares to the minute, as the page shows: 23:59:59 is its 23:59. - Names match Python words inside longer words: the live feed has EuroPython, EuroSciPy and DjangoCon US, which \bpython\b missed. - apply_alt_names learns the series names of every year. A remote name that is a series of its own is renamed only when it starts on the same day as the entry with that alt_name: the sheet's "PyHEP" row of 7 September 2026 is PyHEP.dev 2026, a PyHEP workshop on other dates is not. - Sheet rows left out for want of a website go to the desk queue as candidates, so they are looked at instead of dropped unseen. - The denylist reads links without a scheme, and a review candidate names the link that matched. - A pretalx page answering 401, 403 or 451 counts as a failure, so a blocked job fails as a whole; desk-leads turns red when a source fails, after queueing what the others found. - canonical-source-update and enrich-tba merge main into the rolling branch instead of rebasing it, and push without force: the desk pushes there too, and a daily rebase would rewrite its commits. The full pipeline on today's data still exits 0 and adds or removes no entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

- Country names were replaced inside every field the merge compared, on every row, matched or not. A Google Form id with "GB" in it became "UK", and the alt_name "PyCon Korea" became "PyCon South Korea", then "PyCon South South Korea" the next day. merge_conferences now normalises country names in the place field only, and the unused COUNTRY_NORMALIZATION import is gone. - merge_duplicates keyed on the raw place. The importers write a copy of each current-year edition with its place normalised, so an archived edition stored with "London, United Kingdom" kept a second entry, and the duplicate stopped the next importer. The key now uses normalize_place, and the first entry keeps its stored place, so no stored data is rewritten. - An upcoming entry with two or more extra_places or location items stopped both importers: merge_conferences passed a list to pd.isnull and tested the result for truth. Lists and mappings are now merged before the null checks. The YAML value is kept unless it is empty, and lists are compared by equality. write_conference_yaml had the same fault with pd.notnull and failed next, so it is fixed as well. - In January both importers rebuilt conferences.yml from this year's editions only, so last year's editions that the sort had not yet archived vanished and check_data_loss failed the job. Both importers now carry over the earlier editions of conferences.yml unchanged, and the sort archives them on schedule. - check_links never checked coc. It does now, coc is in REDUNDANT_LINK_FIELDS, since a homepage is not a Code of Conduct page, and the docstrings that list the link fields name it. tests/test_import_daily.py reproduces each fault. The link checking test in test_integration_comprehensive.py and the URL format test in test_yaml_integrity.py now include coc. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

Fix three defects in the desk queue and worklist. Leads stayed queued and were researched again every day. desk_queue.py gains a snooze command and snooze_items(), which set next_check on the named items and report unknown ids on stderr. next_check is not part of the id, so adding the same finding again keeps the item and its date. build_worklist leaves out queue items whose next_check is after today; a missing or invalid date means the item is due. The queue README documents next_check. A repeated manual "check this conference" run was taken as already handled, because its payload was the same each time. The workflow now adds the run id as "requested" to the payload of a workflow_dispatch run only, and the README table lists it. Candidate routing overrode the worklist's own match: MISSING_EDITION always went to the new-year-checker and MAYBE always to the scouter, even for a tracked series. A candidate whose names match one series now goes to the enricher with the edition for its year when any data file holds it, past editions included, else to the new-year-checker, which keeps the year when it is later than the latest edition. A candidate that matches nothing goes to the scouter. Only an ambiguous match still follows the classification. The module docstring and the tests, including the leads round trip, follow the new routing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

Fixes these defects in the rolling-branch tooling: 1. Main and auto/conference-updates sort the data independently, so once both sides have sorted an entry the rolling branch edited, every `git merge origin/main` conflicts in _data. New utils/merge_data.py runs while the merge is stopped. It exits 2 when a conflicted path is outside the three data files, keys every entry on (conference, year) across the files at the merge base, HEAD and MERGE_HEAD, and merges them field by field. It stops with each conflict on stderr when both sides changed a field differently, and otherwise writes the result, runs sort_data(skip_links=True), checks that no key was lost and stages the files. The old pairs the base already repeats (PyCon China held in several cities in one year, RuPy 2012) are told apart by place, since the real data has them; any other repeated key is an error. 2. Both writers fall back to merge_data.py when `git merge origin/main` fails, and still abort and report the conflicted files when it cannot resolve them. Setup Pixi now runs before the branch setup. 3. Both writers share the concurrency group auto-conference-updates and retry a rejected push up to five times: fetch, merge the remote branch (with the same fallback) and push again, never forcing. When the branch existed at setup and is gone, or no longer holds the commit the run started from, the rolling PR was merged and the job stops instead of pushing old history. The PR row keeps linking the data commit. 4. enrich-tba-conferences.yml fails the step when the enrich command fails. 5. link-check.yml keys its concurrency group by ref, so pushes to the rolling branch no longer cancel the weekly run on main. 6. The desk's PR body update lost rows the writers appended between its read and its PATCH. New utils/desk_pr_body.py reads the body, splices the desk's part in front of everything after the first DESK_END line, byte for byte, and PATCHes it in one command. It refuses a desk file that holds the marker, and --print shows the result without writing. Tests cover the merge on in-memory lists, the command line in temporary git repositories (including the reported sort conflict end to end), the splice and the PR body command line with subprocess mocked. ruff still reports S404 and S603 for the one subprocess call in each new module. They need the noqa comments git_parser.py uses, left for review. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

…list The cross-stack review found five gaps in .claude/hooks/desk_guard.py and the matchers that send calls to it. This fixes each one and reuses the existing command parser for the new Bash rules. 1. Tools that never reached the hook. settings.json sent only Bash, the file-edit tools, the claude-code-remote tools and WebFetch, so GitHub MCP tools and CronCreate, CronDelete, RemoteTrigger and ScheduleWakeup ran unchecked. The remote-session matcher is now the regular expression "mcp__.*|Cron.*|Remote.*|Schedule.*". In a desk run the hook blocks every mcp__ tool and CronCreate, CronDelete, CronList, RemoteTrigger and ScheduleWakeup. Outside desk runs they pass as before. 2. The deny list could be switched off. A missing ~/.desk-guard.json (deleted, moved or a dangling link) now blocks every desk call with a clear reason, as a malformed or unreadable one does. deny_paths are matched against the path as given and against the absolute path it names from cwd, with ~ expanded and ., .. and repeated slashes removed, so .claude//hooks/x and .claude/./hooks/x no longer slip past. 3. Subagents had the desk's write powers. When agent_id is in the hook input of a desk run, git runs only its read-only subcommands (and config --get), gh only as a gh api read, and commands that run desk_queue.py add, remove, snooze or init, desk_pr_body.py, merge_data.py or sort_yaml.py (also through pixi run sort or links) are blocked. Edit, Write, MultiEdit and NotebookEdit are blocked inside the repository that holds cwd or the path, symlinks resolved; paths such as /tmp stay open. 4. The main agent's pushes and gh calls were wider than desk.md allows. In a desk run git push must name origin and refspecs that write auto/conference-updates or desk-queue, also through git -C. Force, force-with-lease, -f in any bundle, delete, mirror, all, branches, tags, follow-tags and prune are blocked, as are refspecs that start with + or :. gh runs only as gh api, and its writes are limited to POST pulls, PATCH pulls/<n>, POST issues/<n>/comments and POST issues/<n>/labels on repos/JesperDramsch/python-deadlines, with or without a leading slash or the https://api.github.com/ prefix. Outside desk runs only the old rules apply: no pushes to main or master, and no .github/ edits on the rolling branch. 5. The module docstring now lists exactly what is enforced, for every session, for desk runs and for subagents in desk runs. The tests cover each new block, every command desk.md gives the desk, the desk scripts in the main agent and in a subagent, the missing deny list, path normalisation, the subagent rules, and the settings matchers read the way Claude Code reads them. HOME in the tests now holds an empty deny list, since a desk run without one blocks every call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

desk.md: - Feedback cut-off: this run's section carries the previous run's "Feedback read to" until step 7, which records this run's cut-off only when every maintainer-request item is actioned, dismissed or deferred on a needs-maintainer result or verdict. Maintainer requests go to the front of the worklist, so an interrupted run or a failed subagent no longer loses them. - Identity: step 5 acts only on entries with user id 2620316 posted through no GitHub App. Issue comments carry performed_via_github_app, which must be null; review comments and reviews carry no such field (checked with gh api), so the body check decides there. Bodies that start with "Desk:" or "", or carry the Claude Code attribution footer, are passed over. - Step 2 no longer checks out an old rolling branch, which removed .claude/ and the guard from the working tree. When the branch lacks main, it builds from main's side and merges the branch in, still a fast-forward, and checks that the guard exists afterwards. - Merges in step 2 and "Pushing" resolve data conflicts with utils/merge_data.py, and abort and report on any other exit. - Every comment and PR body, decline replies included, is posted from a scratch file with -F body=@<file>, never inline. - Without a desk-queue branch, steps 7 and 8 leave the run's section in the final report; the writers create the branch on their next run. - Step 8 snoozes the queue items behind leads for seven days with desk_queue.py snooze. Failed and unreached items stay without one. - "Pushing" checks before each push that the rolling branch still holds what this run last pushed. When it is gone or was opened again from main, it starts again from step 2 and cherry-picks only this run's unpushed conference commits. - PR body updates go through utils/desk_pr_body.py, so the read and the write happen in one command. - The ground rules and the AGENTS.md hook line list the guard's wider coverage in desk runs. Skills: - new-year-checker returns no-change only when the data holds the edition for the item's year, or an edition that has not ended when the year is null, and searches from the queue items first. - enricher: a question on "conference" returns needs-maintainer naming the entry the import wrote and the titles.yml variation to add, or no-change. - enricher: drift comes from pretalx, the platform that enforces the deadline; a later date goes in cfp_ext, and the conference's own prose stating another deadline is flagged sources-disagree. - The enricher and new-year-checker cite only the maintainer-request comment the desk passed through step 5. - The scouter and new-year-checker explain the lead-only, no-website and denylist-review candidate flags, and the critic returns needs-maintainer for a denylist-review flag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

Both run git or gh from an absolute path found with shutil.which and a fixed argument list, the same pattern utils/git_parser.py marks with noqa: S404 and S603. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

…ites - normalize_conference_name tidied brackets before "+" became a space and before whitespace was collapsed, so "0+)" gave "0 )" and then "0)". The bracket tidy now runs after the "+" replacement, the Conf expansion and the whitespace collapse, just before the final strip, and collapses whitespace again. It removes emptied brackets until none are left, so "X (( ))" gives "X" in one pass. The bracket cases in test_import_safeguards.py keep their results, and new tests check that "0+)", "X ( 2026 )", "A+(B)" and "X (( ))" are stable on a second pass. - The merge_duplicates docstring said the first entry keeps its stored place and no stored data is rewritten. The first entry is usually the importer's copy from conferences.yml, so its normalised spelling replaces the archive's. The docstring now says so. Behaviour is the same. - test_interactive_merge_uses_country_normalization claimed that interactive_merge uses COUNTRY_NORMALIZATION, which it no longer does. It is now TestCountryNormalizationMapping and test_long_country_names_map_to_short_forms, with a docstring that says what it checks. - The URL format smoke test now checks coc, as test_yaml_integrity.py does. - github.io, gitlab.io, netlify.app, pages.dev, vercel.app, readthedocs.io and org.pl are in TWO_LEVEL_PUBLIC_SUFFIXES, so tenants of those hosts are different sites: pyconx.github.io and squatter.github.io, or pycon.org.pl and casino.org.pl. test_link_redirects.py covers each suffix and a redirect from one github.io tenant to another. No current data link changes behaviour: only three legacy entries use github.io, and the sort leaves _data unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

- A re-found finding keeps its first "received", so a lead snoozed for long is among the oldest items in its file. Once the file went over 50 items, trimming dropped it, the next add queued it again with no next_check, and it was due again. add_items now keeps items whose next_check is after today ahead of the rest, then the newest by received. It takes a today argument, defaulting to the current UTC date. Tests cover a kept lead, a lapsed snooze treated as due, and the UTC default. - desk_queue.py list now prints next_check, null when unset. The list test checks a full row. The README in desk_queue.py says that snoozed items survive the cap. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

find_tba_conferences picked every entry whose cfp is in TBA_WORDS, which also holds "cancelled", so the job visited the homepage of a cancelled edition and could fill its sponsor or finaid link. It now leaves cancelled editions out. Entries with cfp None stay in: since the API mode is gone, the job only fills Bluesky, YouTube, Mastodon, sponsor and finaid links, and an edition with no public CfP still happens. On the current data the only change is PyCon Estonia 2026. The new tests fail on the previous code for the cancelled cases. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

…g-tesla-g7fn0r-5

The desk queue lives on an orphan desk-queue branch: one JSON file per series under queue/, keyed by the slug of the series name, plus a README that init writes from the module so the format is documented in one place. GitHub Actions jobs add records with "add", from JSON lines or from options; the desk removes handled items with "remove". Each item gets a stable id from its source, kind, URL and payload, so the same finding queued twice is one item. A file keeps its 50 newest items and long payload strings are cut at 20,000 characters, flagged with "truncated". The module uses only the standard library, so any job can run it without the data tooling. clean_watch_title and the name lookup move here from desk_worklist.py, which now imports them, so the queue file key and the worklist's series matching read names the same way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

check-conference-update.yml ran a paid-API triage on every changed page, then two agent jobs that edited the data. The triage passed 26% of events at 38% precision and 58% recall, and never saw the data, so it flagged dates already recorded and dropped extensions and moved dates. The workflow now validates the payload and writes it to the desk-queue branch unchanged; the desk judges it on its next run with the data and the live site in hand. The payload is read from the event file rather than interpolated or passed through the environment, so page text cannot reach the shell and a long diff cannot exceed the environment size limit. The new composite action desk-queue-write is the one place that writes to the queue: it creates the orphan branch when missing, adds records with utils/desk_queue.py, and retries a rejected push from a fresh copy of the branch, never with force. Ids are stable, so a retry adds nothing twice. scheduled-extension-checker.yml goes: the desk's worklist checks every deadline 1, 3 and 7 days after it passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

The TBA enrichment keeps its deterministic link extraction: social accounts, sponsor pages and financial aid pages found on each conference's homepage. Deadlines, dates and time zones are now researched by the desk, so the full mode, its prompt and its API calls are removed, together with the --level option and the confidence handling only the API results needed. The workflow always runs the deterministic enrichment and no longer needs the API key. The canonical source workflow triggers it without a level input. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

The import asks a person to settle conflicts it cannot resolve: which of two values to keep, whether a later deadline is an extension, whether two similar names are one series. In CI no one is at a terminal, so query_yes_no took the default and the question was lost. query_yes_no now takes an optional context. Without a terminal it still returns the default, so the import stays deterministic, and it also records the question with the conference, year, field, both values and the source. With DESK_QUEUE_OUT set, the record is written as a JSON line, and canonical-source-update.yml adds the file to the desk queue, where the enricher answers it from the live site. The JSON-lines writer moves into tidy_conf/queue_out.py, shared with the link checker's redirect records. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

The link check now records a link that ends on a different site instead of counting a 200 there as healthy, but nothing ran it on a schedule. redirect-scan.yml runs it every Sunday with DESK_QUEUE_OUT set and adds the records to the desk queue. The desk decides each one: a series that moved gets its new link, a lapsed or hijacked domain an archived copy. The scan is report-only. The link check's other edits stay with the writers that commit data. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

The canonical import asks the same standing questions on every run (an end date one day apart, "Brasil" and "Brazil"), and the redirect scan finds the same redirects every week. Ids only deduplicated while an item was queued, so once the desk had handled one, the next run queued it again. desk_queue.py remove now records each id in its file's "handled" list, the last 500 per file, and add reports a handled id instead of queueing it. A changed finding has a new id and is queued as usual. For a redirect the id leaves out the status and the final URL's query and fragment, which can change between checks. A queue file that is not valid JSON stopped every writer bound for it, and under set -e the canonical merge was never committed. add now moves such a file to <key>.json.bad and starts a new one; remove and list skip it. The composite action turns these messages and rejected records into workflow warnings, and the canonical workflow's queue step can no longer hold back the merge. Also from review: - the follow-up "Is this an extension?" question is no longer queued: the cfp question before it carries both deadlines, while its record gave the YAML cfp as the cfp_ext value; - the redirect scan's link check has a step timeout below the job's, so a slow scan still queues what it found; - canonical-source-update gets actions: write, which gh workflow run needs to start the TBA enrichment; - the action matches refs/heads/desk-queue exactly. desk.md step 8 now removes items only through desk_queue.py remove. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

find_tba_conferences picked every entry whose cfp is in TBA_WORDS, which also holds "cancelled", so the job visited the homepage of a cancelled edition and could fill its sponsor or finaid link. It now leaves cancelled editions out. Entries with cfp None stay in: since the API mode is gone, the job only fills Bluesky, YouTube, Mastodon, sponsor and finaid links, and an edition with no public CfP still happens. On the current data the only change is PyCon Estonia 2026. The new tests fail on the previous code for the cancelled cases. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

…ting-tesla-g7fn0r-5 GitHub rebased the base branch onto main when #499 was merged. Its content is unchanged and this branch already holds it, so the merge keeps this branch's tree and only records the new base as a parent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

The desk queue lives on an orphan desk-queue branch: one JSON file per series under queue/, keyed by the slug of the series name, plus a README that init writes from the module so the format is documented in one place. GitHub Actions jobs add records with "add", from JSON lines or from options; the desk removes handled items with "remove". Each item gets a stable id from its source, kind, URL and payload, so the same finding queued twice is one item. A file keeps its 50 newest items and long payload strings are cut at 20,000 characters, flagged with "truncated". The module uses only the standard library, so any job can run it without the data tooling. clean_watch_title and the name lookup move here from desk_worklist.py, which now imports them, so the queue file key and the worklist's series matching read names the same way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

check-conference-update.yml ran a paid-API triage on every changed page, then two agent jobs that edited the data. The triage passed 26% of events at 38% precision and 58% recall, and never saw the data, so it flagged dates already recorded and dropped extensions and moved dates. The workflow now validates the payload and writes it to the desk-queue branch unchanged; the desk judges it on its next run with the data and the live site in hand. The payload is read from the event file rather than interpolated or passed through the environment, so page text cannot reach the shell and a long diff cannot exceed the environment size limit. The new composite action desk-queue-write is the one place that writes to the queue: it creates the orphan branch when missing, adds records with utils/desk_queue.py, and retries a rejected push from a fresh copy of the branch, never with force. Ids are stable, so a retry adds nothing twice. scheduled-extension-checker.yml goes: the desk's worklist checks every deadline 1, 3 and 7 days after it passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

The TBA enrichment keeps its deterministic link extraction: social accounts, sponsor pages and financial aid pages found on each conference's homepage. Deadlines, dates and time zones are now researched by the desk, so the full mode, its prompt and its API calls are removed, together with the --level option and the confidence handling only the API results needed. The workflow always runs the deterministic enrichment and no longer needs the API key. The canonical source workflow triggers it without a level input. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

The import asks a person to settle conflicts it cannot resolve: which of two values to keep, whether a later deadline is an extension, whether two similar names are one series. In CI no one is at a terminal, so query_yes_no took the default and the question was lost. query_yes_no now takes an optional context. Without a terminal it still returns the default, so the import stays deterministic, and it also records the question with the conference, year, field, both values and the source. With DESK_QUEUE_OUT set, the record is written as a JSON line, and canonical-source-update.yml adds the file to the desk queue, where the enricher answers it from the live site. The JSON-lines writer moves into tidy_conf/queue_out.py, shared with the link checker's redirect records. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

The link check now records a link that ends on a different site instead of counting a 200 there as healthy, but nothing ran it on a schedule. redirect-scan.yml runs it every Sunday with DESK_QUEUE_OUT set and adds the records to the desk queue. The desk decides each one: a series that moved gets its new link, a lapsed or hijacked domain an archived copy. The scan is report-only. The link check's other edits stay with the writers that commit data. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

The canonical import asks the same standing questions on every run (an end date one day apart, "Brasil" and "Brazil"), and the redirect scan finds the same redirects every week. Ids only deduplicated while an item was queued, so once the desk had handled one, the next run queued it again. desk_queue.py remove now records each id in its file's "handled" list, the last 500 per file, and add reports a handled id instead of queueing it. A changed finding has a new id and is queued as usual. For a redirect the id leaves out the status and the final URL's query and fragment, which can change between checks. A queue file that is not valid JSON stopped every writer bound for it, and under set -e the canonical merge was never committed. add now moves such a file to <key>.json.bad and starts a new one; remove and list skip it. The composite action turns these messages and rejected records into workflow warnings, and the canonical workflow's queue step can no longer hold back the merge. Also from review: - the follow-up "Is this an extension?" question is no longer queued: the cfp question before it carries both deadlines, while its record gave the YAML cfp as the cfp_ext value; - the redirect scan's link check has a step timeout below the job's, so a slow scan still queues what it found; - canonical-source-update gets actions: write, which gh workflow run needs to start the TBA enrichment; - the action matches refs/heads/desk-queue exactly. desk.md step 8 now removes items only through desk_queue.py remove. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

find_tba_conferences picked every entry whose cfp is in TBA_WORDS, which also holds "cancelled", so the job visited the homepage of a cancelled edition and could fill its sponsor or finaid link. It now leaves cancelled editions out. Entries with cfp None stay in: since the API mode is gone, the job only fills Bluesky, YouTube, Mastodon, sponsor and finaid links, and an edition with no public CfP still happens. On the current data the only change is PyCon Estonia 2026. The new tests fail on the previous code for the cancelled cases. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

…ting-tesla-g7fn0r-5 GitHub rebased the base branch onto main after #488 (SciPy India's CfP extension) merged. The rebase brings only that change to _data/conferences.yml. The eight files that conflicted are ones this branch changes on top of the base branch, so this branch's side is kept in each; the result is this branch's tree plus #488's change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

…shes From the routine review of #500: - The manual check-conference-update run asks for a name such as "PyCon US 2027", but clean_watch_title took a year only from a changedetection suffix (" 2027 - Main"), so that input was queued with year null under queue/pycon-us-2027.json. A trailing year from 1900 to 2099 with no suffix now counts too, so it goes to queue/pycon-us.json with year 2027. Watch titles keep their suffix and are read as before; "Python 3000" and "PyCon 2027 Online" keep their names. - A comment next to the merge prompts in interactive_merge.py says that each prompt's text is part of its queued question's id, so rewording one queues every standing question again, once. The new tests fail on the previous code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzyF2ciB5jhvcTVpj4ErMr

…g-tesla-g7fn0r-5 # Conflicts: # tests/test_desk_queue.py

JesperDramsch · 2026-10-15T11:12:06Z

Routine review

1. Request

No linked issue, so the request is the PR description. It asks for three things:

Daily canonical import: a schedule trigger on canonical-source-update.yml, writers that merge instead of rebasing, and the four import safeguards (apply_alt_names, bracket tidying in normalize_conference_name, drop_denied in both importers, drop_rows_without_link).
Three lead importers: pretalx, developers.events and the registries, in utils/desk_leads.py and desk-leads.yml.
Stack-wide fixes: everything under "Fixes from a review of the whole stack", covering the canonical import, the rolling branch (merge_data.py, desk_pr_body.py, push retries, link-check.yml concurrency), the queue and worklist (snooze, routing), desk.md and the skills, the guard hook, and the new desk-research agent.

2. Checked against

Today's date: Thu Oct 15 2026 (from date).
developers.events data licence: https://raw.githubusercontent.com/scraly/developers-conferences-agenda/master/LICENSE-CONTENT reads "Creative Commons Attribution-NonCommercial 4.0 International". This matches the PR's CC BY-NC claim and the lead-only handling in desk_leads.py.
unverified: the PR says pretalx.com's robots.txt disallows /api/. I fetched https://pretalx.com/robots.txt and the sandbox could not resolve the host (getaddrinfo ENOTFOUND pretalx.com).
unverified: the PR says Calling All Papers' API host serves Disallow: / and states no data licence. I did not fetch it.
unverified: the live pretalx drifts the PR reports (PyCascades 2027 closing at 05:00 PDT on 26 October, PyCon Namibia 2027 closed on 13 October).

3. Needs your decision

Scope. About half of this ~10k-line diff is fixes to work from Add AGENTS.md, the desk run procedure, four skills and a guard hook #498–Queue work for the desk and retire the paid API jobs #500: the guard hook (+1725 lines), desk.md, the skills and the queue and worklist tools. The PR body explains why they land here. Keep them in this PR, or split the guard hardening into its own PR? My recommendation is to keep them. The stack merges as one, and the guard and desk.md changes refer to each other.
pretalx API (raised in the PR). Should you ask pretalx whether a daily read of /api/events/ is acceptable? My recommendation: not before this merges. The CfP-page-only design works without it.
enrich-tba-conferences.yml now fails the job when enrich_tba.py fails. It adds exit 1 in the else branch, where before it only set success=false and carried on. The PR body doesn't list this change. I recommend keeping it, since a partly written conferences.yml should not be committed, but you should know that the TBA job will now go red where it used to stay green.

4. Findings

Important: None.

Nits:

utils/import_python_organizers.py:326: drop_denied filters df_csv_raw. That frame is also the base of df_csv_output, which write_csv writes at line 481, so denied rows also drop out of the regenerated CSV under utils/conferences. This is harmless if nothing consumes that CSV upstream. If something does, filter a copy for the merge instead.
PR body, "Checks": it says "2448 passed", and the pinned run below gives 2452 passed. The numbers are only stale; nothing fails.

5. Tests

Invariant tests: the new tests follow the request's invariants rather than pinning current values.
- tests/test_import_safeguards.py covers each of the four rows in the safeguards table.
- tests/test_import_daily.py covers both importers for the January carry-over and the multi-item extra_places/location case.
- tests/test_merge_data.py drives merge_entries and resolve through real git merges.
- tests/test_desk_leads.py covers each lead source.
Base-revision run: not done. The sandbox's permission classifier refused it, so I cannot confirm that these tests fail on the base (see Bounds).

Checks on the change (head dd5a329):

$ uv run --python 3.11 --with "pandas<2.3",pytest,... pytest tests/ -q -p no:cacheprovider
2452 passed, 5 skipped, 3 xfailed, 2 xpassed, 251 warnings in 159.10s (0:02:39)

$ uvx ruff@0.4.4 check utils tests .claude/hooks
All checks passed!

$ uv run --python 3.11 --with "pandas<2.3",... python utils/sort_yaml.py --skip_links
exit 0 — "Conference data sorting and cleaning completed successfully"
$ git diff --stat -- _data
(empty)

$ npm ci && npm test
Test Suites: 15 passed, 15 total
Tests:       423 passed, 423 total

pre-commit.ci - pr status: not quoted. I did not read the PR's check runs.

6. Bounds

Route: PR read through gh api REST. Head checked out by git fetch origin refs/pull/501/head; rev-parse matched dd5a3296d73d0ea6e40683d5ace810718c6b324d. Diff taken against ai-agent/exciting-tesla-g7fn0r-4.
Read in full:
- utils/merge_data.py and utils/tidy_conf/denylist.py
- the diffs of sort_yaml.py, interactive_merge.py, titles.py, links.py, yaml.py, both importers, desk_queue.py and desk_worklist.py
- all three writer workflows plus desk-leads.yml and link-check.yml
- settings.json, desk-research.md and AGENTS.md
Read in part: the pretalx section of utils/desk_leads.py (lines 600–880) and the docstring of .claude/hooks/desk_guard.py.
Not read: the rest of desk_leads.py; the guard's implementation (~1700 lines) beyond its docstring; desk_pr_body.py; the desk.md and skill diffs; and most test files, which I reviewed only by name and by running them.
Could not run:
- Direct probes of the guard hook with push and refspec shapes: refused by the sandbox's permission classifier. The guard's correctness rests on tests/test_desk_guard.py, which passes.
- The new fix tests against the base revision: refused the same way.
- actionlint, pre-commit, and Python 3.9 for the guard.
- pretalx.com: DNS fails in the sandbox.
Pre-existing: the official importer keeps only YAML years year..year+9, while the organizers importer keeps up to now+10. Not introduced here.

Generated by Claude Code

#500 landed on main as the squash commit ce5df9f, whose tree is the old #500 head 87bc32e. Conflicts keep this branch's side, so the branch's diff against main is #501's own change (87bc32e..dd5a329). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CztuQqAuxRRJLkKiQYasnS

JesperDramsch · 2026-10-15T11:47:53Z

Main is merged in as 6e8c2e8, a merge commit with no history rewritten. #500 landed as ce5df9f, whose tree is identical to the old #500 head 87bc32e, so the conflicts keep this branch's side. git diff origin/main 6e8c2e8 is byte-identical to git diff 87bc32e dd5a329, and the new head's tree is dd5a329's.

On the merged head:

pinned uv pytest run: 2452 passed, 5 skipped, 3 xfailed, 2 xpassed, 0 failed
uvx ruff@0.4.4 check utils tests .claude/hooks: all checks passed
pre-commit run --from-ref origin/main --to-ref HEAD: every hook passes

Both nits from the Routine review stay as they are:

drop_denied on df_csv_raw (utils/import_python_organizers.py:326): utils/conferences is in .gitignore, and canonical-source-update.yml stages only the three _data/ files. The regenerated CSV is a local by-product that nothing reads, so dropping denied rows from it costs nothing.
Test count in the body: 2452 above is the current number. The body's 2448 is from before the last round of fixes.

Generated by Claude Code

Brings in #500 and #501. The writers push with GITHUB_TOKEN, which cannot push this merge because main changed files under .github/workflows/, so it is merged by hand once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CztuQqAuxRRJLkKiQYasnS

github-actions Bot added CI/CD Tests labels Oct 14, 2026

JesperDramsch mentioned this pull request Oct 14, 2026

Link rules, an optional coc field, and fail-loud branch sync #497

Merged

github-actions Bot added the Documentation label Oct 14, 2026

JesperDramsch added this pull request to stack #502 October 14, 2026 14:19

github-advanced-security Bot found potential problems Oct 14, 2026

View reviewed changes

Comment thread tests/test_import_safeguards.py Fixed

Comment thread tests/test_import_safeguards.py Fixed

JesperDramsch mentioned this pull request Oct 14, 2026

Add AGENTS.md, the desk run procedure, four skills and a guard hook #498

Merged

claude added 23 commits October 15, 2026 09:12

claude added 9 commits October 15, 2026 08:02

Merge branch 'ai-agent/exciting-tesla-g7fn0r-4' into ai-agent/excitin…

2f870be

…g-tesla-g7fn0r-5

claude added 8 commits October 15, 2026 10:30

claude added 3 commits October 15, 2026 10:44

Merge branch 'ai-agent/exciting-tesla-g7fn0r-4' into ai-agent/excitin…

dd5a329

…g-tesla-g7fn0r-5 # Conflicts: # tests/test_desk_queue.py

JesperDramsch marked this pull request as ready for review October 15, 2026 11:06

Base automatically changed from ai-agent/exciting-tesla-g7fn0r-4 to main October 15, 2026 11:30

JesperDramsch merged commit db3eab2 into main Oct 15, 2026
11 checks passed

JesperDramsch deleted the ai-agent/exciting-tesla-g7fn0r-5 branch October 15, 2026 11:52

Navigation Menu

Repository navigation

Run the canonical import daily and add lead importers for the desk - #501

Run the canonical import daily and add lead importers for the desk#501
JesperDramsch merged 65 commits into
mainfrom
ai-agent/exciting-tesla-g7fn0r-5

JesperDramsch commented Oct 14, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

JesperDramsch commented Oct 15, 2026

Uh oh!

JesperDramsch commented Oct 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

JesperDramsch commented Oct 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why PyCon Ireland and PyCon SK never arrived

Safeguards for an unattended import

Lead importers (utils/desk_leads.py, desk-leads.yml)

Fixes from a review of the whole stack

Needs your call

Checks

Uh oh!

Uh oh!

Uh oh!

JesperDramsch commented Oct 15, 2026

1. Request

2. Checked against

3. Needs your decision

4. Findings

5. Tests

6. Bounds

Uh oh!

JesperDramsch commented Oct 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

JesperDramsch commented Oct 14, 2026 •

edited

Loading

Lead importers (`utils/desk_leads.py`, `desk-leads.yml`)