Skip to content

Three recorded MATLAB/Python divergences that need a decision, not a port: epochdir ANY-vs-ALL, non-functional cedspike2, t0_t1 wire orientation #100

Closed
Closed
Three recorded MATLAB/Python divergences that need a decision, not a port: epochdir ANY-vs-ALL, non-functional cedspike2, t0_t1 wire orientation#100

Description

@stevevanhooser
stevevanhooser
opened on Sep 1, 2026

Part of #90 (PR #61 catch-up). PR #61 deliberately did not change behaviour for any of these — it recorded them. They are filed here so the decisions do not get lost with the branch.

1. epochdir accepts an epoch on ANY FileParameter match; MATLAB requires ALL

src/ndi/file/navigator/epochdir.py:118-124:

for subdir in subdirs:
    matched = self._match_files_in_dir(subdir, patterns)
    if matched:
        groups.append(sorted(matched))

A directory matching one of several declared file parameters is accepted as an epoch. MATLAB requires every declared parameter to match. This is the phantom-epoch defect MATLAB fixed for vhlab — and in Python it exists generically, in the base navigator, affecting every lab rather than one.

Why it needs a decision: this is a behaviour change in a core navigator. Directories that currently register as epochs would stop doing so. That is the correct behaviour, but it will change what existing sessions enumerate, so it wants its own TDD task and a deliberate call about migration — not a quiet fix folded into something else.

2. cedspike2 is non-functional for both son32 and son64

src/ndi/daq/reader/mfdaq/cedspike2.py advertises .smr and .smrx in FILE_EXTENSIONS, but the SpikeInterface adapter has no CED branch and spikeinterface is not installed. Failures are swallowed to [] — so the reader reports "no data" rather than "not supported", which is the worst of both.

son64 (S4) is blocked, verified: NDR-python has zero son64/sonpipe support and hard-rejects .smrx; Python's cedspike2 never routed through NDR at all. Porting it means NDR feature work plus a reader re-architecture, not a port.

The decision: whether to (a) fix the honesty problem now — stop advertising extensions that cannot be read, and raise instead of swallowing — while leaving the capability gap, or (b) schedule the NDR work. (a) is cheap and strictly an improvement; I would do it regardless of the answer on (b).

3. t0_t1 wire-format orientation is unresolved across languages

The element_epoch and epochclocktimes schemas declare [2, NaN]; Python emits N×2. Flat JSON cannot distinguish the orientation, so neither language can validate it and a validator that enforced declared dimensions would reject valid documents.

This blocks declared-dimension enforcement in ndi.validate (deliberately excluded from the S6 port — see the ndi_common issue). It needs a cross-language decision on the canonical orientation before any validator work proceeds.

Also recorded: S8 vhlab navigator fixes have no Python surface

MATLAB PR #863's vhlab navigator fixes cannot be ported because the targets do not exist: there is no vhlab_np_epochdir, and VHAudreyBPod is a stub. Porting means building those subsystems, which is new work rather than a sync. Noting it so it is not mistaken for a missed port. (vhPrairie2p is the exception — main has it, at src/ndi/setup/file/navigator/vhprairie2p.py.)

Activity

  1. stevevanhooser commented on Sep 4, 2026

    @stevevanhooser
    stevevanhooser
    on Sep 4, 2026 – with Claude
    ContributorAuthor

    Part 3 (t0_t1 wire-format orientation) is resolved by #193 (merged).

    Decision: adopt MATLAB's canonical 2×N orientation on the wire (row 0 = t0's, row 1 = t1's, one column per clocktype), as the schemas already declare (parameters: [2, NaN]). Python was the drift — ndi.fun.epoch.t0_t1cell2array returned N×2 while claiming "exact match" with MATLAB's ndi.fun.doc.t0_t1cell2array, and the reader treated any non-flat value as an N×2 list of [t0, t1] pairs. A document with ≥2 clocktypes would have been silently misread.

    What #193 changed:

    • ndi.fun.epoch.t0_t1cell2array now returns 2×N, matching MATLAB.
    • New ndi.element._parse_t0_t1 reads both wire forms MATLAB emits (flat [t0, t1] from the single-clock 2×1 collapse; nested [[t0_1,…],[t1_1,…]] for multi-clock).
    • Writer unchanged — the single-clock addepoch already emitted flat [t0, t1], matching MATLAB's 2×1 jsonencode collapse.
    • Bridge YAML decision log corrected.
    • Tests updated for the new shape plus 5 new tests exercising the wire-form parser.

    Parts 1 and 2 are still open, so this issue stays open:

    • Part 1: epochdir accepts an epoch on ANY FileParameter match; MATLAB requires ALL. Behavior change in a core navigator — still needs its own TDD task and migration call.
    • Part 2: cedspike2 is non-functional for .smr/.smrx. Still needs the (a) honesty fix now vs. (b) schedule NDR work decision.

    Generated by Claude Code

  2. stevevanhooser
    added a commit that references this issue on Sep 4, 2026

    Merge pull request #195 from Waltham-Data-Science/claude/issue-100-sc…

    fbdcdbb

  3. stevevanhooser commented on Sep 4, 2026

    @stevevanhooser
    stevevanhooser
    on Sep 4, 2026 – with Claude
    ContributorAuthor

    Item 1 (epochdir ANY→ALL): done.

    Landed in #195 (merged 2026-09-04). Python's base ndi.file.navigator.epochdir.selectfilegroups_disk now requires every declared filematch pattern to match at least one file in a subdirectory before that directory is accepted as an epoch, matching MATLAB's findfilegroups semantics. Regression coverage in tests/test_batch_a.py::TestEpochDirNavigatorAllMatch. Bridge YAML entry updated with the DISCREPANCY FIXED note and MATLAB sync hash e782ead.

    Items 2 (cedspike2 honesty/capability) and 3 (t0_t1 wire-format orientation) remain open — item 2 is being worked in a separate agent session.

    Generated by Claude Code

  4. stevevanhooser commented on Sep 4, 2026

    @stevevanhooser
    stevevanhooser
    on Sep 4, 2026 – with Claude
    ContributorAuthor

    Closing: all three parts landed.

    • Part 1 — epochdir ANY→ALL: #195 merged. Base ndi.file.navigator.epochdir.selectfilegroups_disk now requires every declared filematch pattern to match at least one file before accepting a subdirectory as an epoch, matching MATLAB findfilegroups. Regression coverage in tests/test_batch_a.py::TestEpochDirNavigatorAllMatch; bridge YAML carries the DISCREPANCY FIXED note.

    • Part 2 — cedspike2 non-functional: #192 merged, coordinated with VH-Lab/NDR-python#14. Went beyond option (a): the reader is now a thin wrapper over NDR-python's sonpipe-backed CED helpers (ndr.format.ced.read_SOMSMR_header / read_SOMSMR_datafile), covering both .smr and .smrx through CED's own sonpy binding out of process. NDR-python#14 exposes the install via a [ced] extra plus python -m ndr.setup.sonpipe (port of +ndr/+setup/sonpipe.m); NDI-python#192 adds a ced-integration CI job on Python 3.14 that installs sonpipe via that entry point and exercises the wrapper against NDR-python's shipped example.smr fixture with NDI_REQUIRE_SONPIPE=1 (skip → hard failure) so a missing binary can't produce a vacuous pass. Failures also propagate now instead of collapsing to [] — bad file counts, unknown channels, and missing sonpipe each raise clearly.

    • Part 3 — t0_t1 wire-format orientation: #193 merged. Canonical 2×N on the wire, matching MATLAB and the declared schemas; reader accepts both wire forms MATLAB emits; bridge YAML corrected.

    The "Also recorded" S8 vhlab navigator note remains recorded-for-awareness only (no vhlab_np_epochdir, VHAudreyBPod is a stub — new subsystems, not sync work).

    Closing as completed.

    Generated by Claude Code

  5. stevevanhooser
    closed this as completedon Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions