Repository navigation
Bind the files these fixtures declare - #294
Merged
Merged
Conversation
DID-python#86 closed the fail-open in check_files step 3, so a document that declares a required file in its file_list and binds nothing in files.file_info is refused where it is created rather than accepted and failed on at read. #292 fixed the product half; these twenty-odd construction sites are the test half. Each site now attaches a file, as tests/matlab_tests/conftest.py's _add_doc_with_file already did, or uses a document class that declares none. No schema is relaxed: a generic_file genuinely requires generic_file.ext, and NDI-matlab#977 respected that same constraint when makeSourceFile moved to fileReference rather than emitting a fileless generic_file. The schemas describe reality; the fixtures were wrong. Two sites are not simply "attach a file": test_extract_doc_files_series.py's test_a_document_with_no_files_is_left_alone wants a document with NO files, so it builds a bare base document instead of a demoNDI. Attaching a file there would have inverted the test. test_did_integration.py's definitions gate validates a BLANK template, and a blank template legitimately binds nothing -- the document that binds the files is the one that gets stored. The files check now runs ahead of the field checks, so seven definitions reported DID:Database:ValidationFiles instead of whatever is actually wrong with them. The gate asks about a definition's structure, so it now binds the declared names first, exactly as _BLANK_TEMPLATE_IDENTIFIERS already discounts an unfilled dependency. The allowlist is unchanged and still earns its keep: the five listed definitions report the same ValidationFieldInteger/ValidationFieldDouble they were listed for, and data/geneList and sorting/SpikeInterfaceSortingOutputs -- which were never listed -- validate clean, confirming nothing was structurally wrong with them. Adding all seven to the allowlist under the new code would have erased that record. Measured on this container with did and vhlab-toolbox-python installed from main: 21 failed / 5155 passed / 552 skipped before, 5176 passed / 552 skipped after, with no test failing that was not failing already. Closes #293 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WCfsstd8ma954CDDXg4Hm2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #293.
VH-Lab/DID-python#86 closed the fail-open in
check_filesstep 3, so a document that declares a required file in itsfile_listand binds nothing infiles.file_infois refused where it is created rather than accepted and failed on at read. #292 fixed the product half (element_timeseries.addepoch); this is the test half — the ~20 construction sites that built such a document.No schema is relaxed. A
generic_filegenuinely requiresgeneric_file.ext, the same constraint NDI-matlab#977 respected whenmakeSourceFilemoved tofileReferencerather than emitting a filelessgeneric_file. Each site now attaches a file the waytests/matlab_tests/conftest.py::_add_doc_with_filealready did, or uses a document class that declares none.Measured
Both runs on this container with
didandvhlab-toolbox-pythoninstalled frommain, back to back, same command:comm -13 baseline.txt after.txtis empty — nothing fails that was not already failing. 5155 + 21 = 5176 accounts for every test.ruff checkandblack --check --target-version py310pass on all eight touched files.The two sites that were not "attach a file"
test_extract_doc_files_series.py::test_a_document_with_no_files_is_left_alonewants a document with no files, so it now builds a barebasedocument instead of ademoNDI. Attaching a file would have inverted the test.test_did_integration.py::TestBundledDefinitions::test_definitions_validate_or_are_listedis the one the issue flagged as different in kind, and it is. The gate validates a blank template, and a blank template legitimately binds nothing — the document that binds the files is the one that gets stored. The files check now runs ahead of the field checks, so seven definitions began reportingDID:Database:ValidationFilesinstead of whatever is actually wrong with them.That is the right answer about a document and the wrong question about a definition. This is the same situation
_BLANK_TEMPLATE_IDENTIFIERSalready handles for an unfilled dependency ("a blank template legitimately leaves required dependencies empty"), so the gate now binds the declared names before validating rather than discounting the identifier.Discounting it would have been the lossy choice, because
_structural_failurereturns on the first error: five of the seven would have come back clean and tripped the test's ownnewly_fixedassertion, and the field errors they are listed for would have stopped being checked. Binding instead leaves the allowlist completely unchanged and still earning its keep:data/generic_fileValidationFieldDoubleValidationFieldDoubledemoNDIValidationFieldIntegerValidationFieldIntegerdemoNDISeriesValidationFieldIntegerValidationFieldIntegerdemoNDISeriesMixedValidationFieldIntegerValidationFieldIntegermock/demoNDIMockValidationFieldIntegerValidationFieldIntegerdata/geneListsorting/SpikeInterfaceSortingOutputsThe last two are the corroboration: they were never on the allowlist, they were tripping only the new instance-level files check, and once it is asked the right question they pass. Nothing was structurally wrong with them, and adding all seven to the allowlist under the new code would have recorded that there was.
A
file_listname ending in#is a file series, so it is bound under its first member's name (is_filename_matchreads the stem plus digits). Placeholder locations arehttps://URLs:can_find_one_filedoes no network I/O and does not pre-check a non-local location, so the gate still touches no filesystem.Note, not fixed here
tests/test_session_database_rm.pyassertssession.database_search(ndi_query("base.id") == doc.id) == []in four places.ndi_document.idis a property, so these are real comparisons and the assertions do hold — I checked, becausedid.document.Document.idis a method and the same line against a raw DID document would be vacuously true. Flagging it only because the two types differ here and it is an easy trap for the next person.🤖 Generated with Claude Code
https://claude.ai/code/session_01WCfsstd8ma954CDDXg4Hm2
Generated by Claude Code