File series (DID-matlab #173, PR #65) shipped untested: every function above the manifest codec is at 1/N coverage #68

Summary

The file-series API landed in f292b16 ("Port DID-matlab #173 file series + per-uid fetch lock; refresh bridge", PR #65) and shipped with no tests above the manifest codec. The suite is green — 522 passed, 8 skipped, 1 xfailed — because nothing in it exercises the feature.

The one-sentence version: no test in DID-python ever constructs a document with a populated series. grep -rn "add_file_series\|series_count\|series_has\|series_members\|file_series" tests/ returns nothing.

This is not a bug report. The implementation reads carefully and is not known to be wrong. It is unverified, and NDI-python is about to build on it — NDI-python#215 is exactly the cloud round trip these functions are for.

Reproducing

python3 -m pip install -e ".[dev]" && python3 -m pip install coverage
python3 -m coverage run --source=did.document,did.implementations.sqlitedb,did.database,did.file -m pytest tests/ -q

Per-function statement coverage (covered / total, def line included), measured on 7e92695:

did.document
  series_names                    7/10     is_file_series                 1/3
  is_series_member                3/3      series_member_of               6/18
  series_count                    1/5      series_source_root             1/5
  series_ingest_locations         1/10     add_file_series                1/54
  remove_file_series              1/14     strip_series_ingest_locations  9/17
  file_uids                       1/14     _series_info                   1/8
  _series_info_index              1/14     _longest_common_directory      1/19
  _relative_names                 1/14     _series_ingest_records         1/15

did.implementations.sqlitedb
  _reject_series_without_ingest_locations  2/5
  _ingest_series_members          3/32
  _open_series_member             1/53
  _manifest_locations_for_handler 1/21

did.database
  cached_path_for_file            1/11     series_has                     1/10
  series_members                  1/16     _series_manifest_path          1/9
  _series_member_path             1/18

did.file
  read_series_manifest           37/48     read_series_manifest_uid      21/30
  write_series_manifest          37/41

Read that as two different situations:

• did.file's manifest codec is genuinely covered, by tests/symmetry/make_artifacts/file/test_series_manifest.py, which writes five manifests (dense, sparse, named, allEmpty, wideUid) and reads them back. The shared binary format is proven.
• Everything above the codec is at "the def line ran, nothing else did". 1/54 for add_file_series and 1/53 for _open_series_member mean those bodies never execute at all.

So the manifest bytes are proven and every behaviour built on them is not.

Two further notes on what looks like coverage but isn't:

• tests/symmetry/read_artifacts/file/test_series_manifest.py skips when the MATLAB artifact is absent, by design — so the cross-language half is not load-bearing in an ordinary run either.
• did.database.cached_path_for_file and did.document.file_uids are also from f292b16 and also entirely untested, even though neither is series-specific. cached_path_for_file is the documented no-SQL, no-network file lookup; its series-member fallback branch is the entry point into all of the above.

What is untested, specifically

1. did.document series API. add_file_series is the largest untested body in the feature: manifest writing, uid minting, one-based member index → zero-based manifest slot, n_present vs count for a sparse series, source-root derivation and relative-name recording, and the two refusals ("already added", "not declared in files.file_series") plus the index validations (length mismatch, non-positive, duplicate). Also remove_file_series, series_count, series_source_root, series_ingest_locations, and series_member_of past its early returns.

2. The MembersNotLocatable guard — sqlitedb._reject_series_without_ingest_locations (DID-matlab#185). It refuses a document whose series records n_present > 0 with no ingest_locations. Nothing proves it fires, and nothing proves it doesn't fire on a legitimate add. It is a hard ValueError out of add_docs, so a false positive bricks a caller and a false negative silently stores half a series.

3. sqlitedb._ingest_series_members — the member copy loop: the _is_safe_uid refusal (DID-python#58), the remote-without-handler warn-and-skip, the already-present short circuit, the per-member failure warning, and delete_original.

4. sqlitedb._open_series_member — the whole DID-matlab#188 indirection: manifest located, member uid read from the slot, local cache hit, then the remote path that offers the manifest's locations to custom_file_handler with {"documentId", "filename", "seriesName", "uid", "mode"} in the context. Plus the DID-matlab#191 guard against a handler that returns the manifest's own bytes. None of its FileAccessError identifiers is asserted anywhere: ManifestNotLocal, ManifestUnreadable, NoSuchMember, MemberNotHere, HandlerReturnedManifest, MemberFetchFailed. Nor is _manifest_locations_for_handler's remote-before-local ordering, or its rule that a local location is offered only when the file actually exists.

5. did.database.series_has / series_members, and the _series_manifest_path / _series_member_path helpers they and cached_path_for_file share. series_members is documented as the iterator for walking a pyramid level; its sparse-skipping behaviour is unasserted.

6. The round trip — strip_series_ingest_locations on store, read back, then the guard on re-add. This is the shape a cloud round trip produces and it is exactly what NDI-python#215 is about.

Two related gaps found while measuring

The demo document class was not ported. DID-matlab has src/did/example_schema/demo_schema1/database_documents/demoSeries.json and its schema — a demoSeries class declaring files.file_series: ["chunkdata.bin"], added in MATLAB commit c80ba33 alongside the feature. DID-python has neither, so today there is no shipped document class in this repo that declares a file series at all. MATLAB's own series tests are written against it. (demoDouble.json is missing from the same directory too, though that is unrelated to series.) The bridge's file check only walks .m files, so a missing .json schema is invisible to it.

MATLAB has the test suite Python skipped. Roughly 2,350 lines with no Python counterpart:

DID-matlab is the source of truth, so these are the specification for what the Python behaviour is supposed to be.

Correction to two details

While reading the implementation to write this up:

• The DID-matlab#191 guard is not only a size comparison — _open_series_member compares sizes first and then reads both files and compares the bytes, so an equal-sized-but-different member is accepted. Worth pinning either way.
• There are six FileAccessError identifiers on the series read path, not five; MemberFetchFailed is chosen when the last failure was not the #191 guard.

Plan

Tests, in reviewable pieces, one PR each:

• did.document series API unit tests (plus the missing demoSeries class, ported from MATLAB)
• sqlitedb add side — the MembersNotLocatable guard and _ingest_series_members
• sqlitedb read side — _open_series_member, its error identifiers, and the handler-context contract
• store / strip / re-add round trip, and did.database.series_has / series_members

Where a test would pin behaviour that looks wrong, it will be raised separately rather than encoded — a test that locks in a bug is worse than no test.

References: f292b16, PR #65, DID-matlab issues #173, #183, #185, #186, #188, #191.