Skip to content

Bind the files these fixtures declare - #294

Merged
stevevanhooser merged 1 commit into
mainfrom
claude/issue-293-test-fixtures-bind-files
Sep 9, 2026
Merged

stevevanhooser merged 1 commit into
mainfrom
claude/issue-293-test-fixtures-bind-files

Conversation

@stevevanhooser

Copy link
Copy Markdown
Contributor

Closes #293.

VH-Lab/DID-python#86 closed the fail-open in check_files step 3, so a document that declares a required file in its file_list and binds nothing in files.file_info is refused where it is created rather than accepted and failed on at read. #292 fixed the product half (element_timeseries.addepoch); this is the test half — the ~20 construction sites that built such a document.

No schema is relaxed. A generic_file genuinely requires generic_file.ext, the same constraint NDI-matlab#977 respected when makeSourceFile moved to fileReference rather than emitting a fileless generic_file. Each site now attaches a file the way tests/matlab_tests/conftest.py::_add_doc_with_file already did, or uses a document class that declares none.

Measured

Both runs on this container with did and vhlab-toolbox-python installed from main, back to back, same command:

failed passed skipped
before 21 5155 552
after 0 5176 552

comm -13 baseline.txt after.txt is empty — nothing fails that was not already failing. 5155 + 21 = 5176 accounts for every test. ruff check and black --check --target-version py310 pass on all eight touched files.

The two sites that were not "attach a file"

test_extract_doc_files_series.py::test_a_document_with_no_files_is_left_alone wants a document with no files, so it now builds a bare base document instead of a demoNDI. Attaching a file would have inverted the test.

test_did_integration.py::TestBundledDefinitions::test_definitions_validate_or_are_listed is the one the issue flagged as different in kind, and it is. The gate validates a blank template, and a blank template legitimately binds nothing — the document that binds the files is the one that gets stored. The files check now runs ahead of the field checks, so seven definitions began reporting DID:Database:ValidationFiles instead of whatever is actually wrong with them.

That is the right answer about a document and the wrong question about a definition. This is the same situation _BLANK_TEMPLATE_IDENTIFIERS already handles for an unfilled dependency ("a blank template legitimately leaves required dependencies empty"), so the gate now binds the declared names before validating rather than discounting the identifier.

Discounting it would have been the lossy choice, because _structural_failure returns on the first error: five of the seven would have come back clean and tripped the test's own newly_fixed assertion, and the field errors they are listed for would have stopped being checked. Binding instead leaves the allowlist completely unchanged and still earning its keep:

definition listed as reports now
data/generic_file ValidationFieldDouble ValidationFieldDouble
demoNDI ValidationFieldInteger ValidationFieldInteger
demoNDISeries ValidationFieldInteger ValidationFieldInteger
demoNDISeriesMixed ValidationFieldInteger ValidationFieldInteger
mock/demoNDIMock ValidationFieldInteger ValidationFieldInteger
data/geneList (not listed) validates clean
sorting/SpikeInterfaceSortingOutputs (not listed) validates clean

The last two are the corroboration: they were never on the allowlist, they were tripping only the new instance-level files check, and once it is asked the right question they pass. Nothing was structurally wrong with them, and adding all seven to the allowlist under the new code would have recorded that there was.

A file_list name ending in # is a file series, so it is bound under its first member's name (is_filename_match reads the stem plus digits). Placeholder locations are https:// URLs: can_find_one_file does no network I/O and does not pre-check a non-local location, so the gate still touches no filesystem.

Note, not fixed here

tests/test_session_database_rm.py asserts session.database_search(ndi_query("base.id") == doc.id) == [] in four places. ndi_document.id is a property, so these are real comparisons and the assertions do hold — I checked, because did.document.Document.id is a method and the same line against a raw DID document would be vacuously true. Flagging it only because the two types differ here and it is an easy trap for the next person.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WCfsstd8ma954CDDXg4Hm2


Generated by Claude Code

DID-python#86 closed the fail-open in check_files step 3, so a document
that declares a required file in its file_list and binds nothing in
files.file_info is refused where it is created rather than accepted and
failed on at read. #292 fixed the product half; these twenty-odd
construction sites are the test half.

Each site now attaches a file, as tests/matlab_tests/conftest.py's
_add_doc_with_file already did, or uses a document class that declares
none. No schema is relaxed: a generic_file genuinely requires
generic_file.ext, and NDI-matlab#977 respected that same constraint when
makeSourceFile moved to fileReference rather than emitting a fileless
generic_file. The schemas describe reality; the fixtures were wrong.

Two sites are not simply "attach a file":

test_extract_doc_files_series.py's test_a_document_with_no_files_is_left_alone
wants a document with NO files, so it builds a bare base document instead
of a demoNDI. Attaching a file there would have inverted the test.

test_did_integration.py's definitions gate validates a BLANK template, and
a blank template legitimately binds nothing -- the document that binds the
files is the one that gets stored. The files check now runs ahead of the
field checks, so seven definitions reported DID:Database:ValidationFiles
instead of whatever is actually wrong with them. The gate asks about a
definition's structure, so it now binds the declared names first, exactly
as _BLANK_TEMPLATE_IDENTIFIERS already discounts an unfilled dependency.
The allowlist is unchanged and still earns its keep: the five listed
definitions report the same ValidationFieldInteger/ValidationFieldDouble
they were listed for, and data/geneList and
sorting/SpikeInterfaceSortingOutputs -- which were never listed -- validate
clean, confirming nothing was structurally wrong with them. Adding all
seven to the allowlist under the new code would have erased that record.

Measured on this container with did and vhlab-toolbox-python installed
from main: 21 failed / 5155 passed / 552 skipped before, 5176 passed / 552
skipped after, with no test failing that was not failing already.

Closes #293

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WCfsstd8ma954CDDXg4Hm2
@stevevanhooser
stevevanhooser merged commit 380fc09 into main Sep 9, 2026
9 checks passed
@stevevanhooser
stevevanhooser deleted the claude/issue-293-test-fixtures-bind-files branch September 9, 2026 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test fixtures build documents that declare a required file and bind none

2 participants