Live-cloud file-series round-trip test (#304)

[stevevanhooser]

Closes #304. Ports NDI-matlab's FileSeriesRoundTripTest.testMembersSurviveADownloadFromTheCloud[WithSyncFilesFalse] -- the acceptance test the closing note on #300 flagged as missing.

Why this test

The retracted retirement PRs (VH-Lab/NDI-matlab#989 / #301) got all the way to green on the mocked path while breaking the actual read: without the manifest's second ndicloud location or an equivalent mechanism, NDI's download_file_from_cloud refuses any sourcePath that does not start with ndic://, DID reads a handler that produces nothing as a plain miss, and every member of a downloaded series reads as absent -- silently. NDI-matlab caught this on FileSeriesRoundTripTest. NDI-python had no equivalent until now.

Both offline suites (tests/test_cloud_series_reconstruction.py, tests/test_cloud_series_reconstruction_remote.py) mock fetch_cloud_file or install a mock handler. They pin the reconstruction shape but cannot detect a real regression in the resolve path. This file closes that gap.

What it does

Module-scoped setup (once per test run):

1. Authenticate.
2. Build a small local dataset with one demoNDISeries document, chunkdata.bin naming four members of distinct bytes (via did.document.Document.add_file_series, which ndi_document does not wrap).
3. uploadDataset(sync_files=True) into a fresh cloud dataset named NDI_PYTEST_FILE_SERIES_<uuid>.
4. waitForAllBulkUploads so server-side zip extraction is finished before anything reads.
5. Best-effort teardown deletes the cloud dataset with when="now".

Two acceptance tests:

• test_members_survive_a_download_from_the_cloud -- SyncFiles=true. Downloads into a fresh temp dir, asserts the downloaded manifest names the same uids in the same slots (so a reshuffle would resolve every member to the wrong file), then opens each member end-to-end and compares bytes byte-for-byte against the uploaded content. Batch counters: uid_misses == 0 and signer_calls == 1.

• test_members_survive_a_download_from_the_cloud_with_sync_files_false -- SyncFiles=false. Precondition: no member is on this machine yet (an install would mask the fetch this test exists to exercise). Opens each member end-to-end through the fetch-manifest-through-handler path Fetch a remote-only series manifest through the custom_file_handler (#87) VH-Lab/DID-python#88 + PR Rebuild series ingest_locations on SyncFiles=false download (#302) #303 laid; compares bytes byte-for-byte. Batch counters: uid_misses == 0. signer_calls is left loose because the server may answer both manifest and members from one whole-document scope or use a series-scoped one on the first member, and pinning one over the other would guess at server behaviour this suite does not otherwise cover (matches NDI-matlab#988's comment).

Notes

• Both tests clear ndi.cloud.batch_signed_url.get_default() before the open loop, so the uid_misses assertion measures this test's opens and nothing else. Without that, a run of per-uid getFileDetails could quietly do the work (NDI-matlab#968).
• Bytes compared per-member, not by length. All members have the same 64-byte length by construction; the differences are in content. Two members swapped by a uid mixup would come back the right size and the wrong file.
• Cleanly skips when NDI_CLOUD_USERNAME / NDI_CLOUD_PASSWORD are not set. The scheduled test-cloud-api.yml workflow picks the file up by the test_cloud_*.py glob; the linux test lane in ci.yml also runs it with the same credentials injected on every push.

Verification

Locally: pytest tests/test_cloud_file_series_round_trip.py -v collects 2 tests and skips both cleanly without credentials. The doc-building half of the fixture was smoke-tested against a real local ndi_dataset_dir (four members added via did.Document.add_file_series; manifest reads back with the four uids; a local database_openbinarydoc("chunkdata.bin_1") returns the right bytes).

ruff check and black --check clean on the new file.

References

• Retire the manifest cloud-reference workaround and add an offline SyncFiles=false round-trip test once DID-matlab#201 lands VH-Lab/NDI-matlab#986 (parent tracking issue)
• No test reads a file series MEMBER back out of the cloud: the round trip is only proven for the manifest VH-Lab/NDI-matlab#966 ("a suite of five green tests managed to prove nothing about a member's bytes")
• Offline test for a cloud-only series manifest (SyncFiles=false shape) (#986) VH-Lab/NDI-matlab#987, #988 (offline coverage + SyncFiles=false rebuild)
• Retire the manifest cloud-reference workaround (#986) VH-Lab/NDI-matlab#989 (the retracted retirement)
• File series: accessors, remote ingest, an unlocatable-series guard, and one fetch per uid VH-Lab/DID-matlab#185 (the MembersNotLocatable guard the reconstruction satisfies)
• localSeriesManifestPath cannot fetch a remote manifest — series reads fail on cloud-only downloads VH-Lab/DID-matlab#201, Fetch a remote-only series manifest through the custom_file_handler (#87) VH-Lab/DID-python#88 (the customFileHandler contract for a manifest fetch)
• Retire the manifest cloud-reference workaround (companion to VH-Lab/NDI-matlab#986) #300 (parent, closed; the closing comment flagged this gap)
• Rebuild series ingest_locations on SyncFiles=false download (companion to VH-Lab/NDI-matlab#988) #302 / PR Rebuild series ingest_locations on SyncFiles=false download (#302) #303 (the offline SyncFiles=false rebuild this test exercises live)

🤖 Generated with Claude Code

https://claude.ai/code/session_01AdGZ8fRPeY9JSpUHgXm9ao

---

Generated by Claude Code