Add ndi.lightsheet: napariViewLightsheet console script + PyramidReader (scaffold)

scripts/napariViewLightsheet

@@ -0,0 +1,24 @@
+#!/usr/bin/env bash
+# napariViewLightsheet - shell wrapper around the NDI-python console script.
+#
+# The MATLAB LightsheetZarrManager shells out to this file at
+# /usr/local/bin/napariViewLightsheet. It exists so MATLAB's library
+# paths (LD_LIBRARY_PATH, DYLD_LIBRARY_PATH, DYLD_FALLBACK_LIBRARY_PATH)
+# do not leak into the Python process it starts -- MATLAB's copies of
+# libtiff, libcurl, etc. crash or subtly misbehave when loaded into a
+# non-MATLAB Python.
+#
+# Install (once):
+#   sudo install -m 0755 scripts/napariViewLightsheet \
+#       /usr/local/bin/napariViewLightsheet
+#
+# Adjust NAPARI_VIEW_LIGHTSHEET_PYTHON to point at the Python (or venv
+# python) that has ``ndi`` and ``napari[all]`` installed.
+
+set -euo pipefail
+
+unset LD_LIBRARY_PATH DYLD_LIBRARY_PATH DYLD_FALLBACK_LIBRARY_PATH
+unset LD_PRELOAD DYLD_INSERT_LIBRARIES
+
+PYTHON_BIN="${NAPARI_VIEW_LIGHTSHEET_PYTHON:-python3}"
+exec "$PYTHON_BIN" -m ndi.lightsheet.napari_view "$@"

src/ndi/lightsheet/README-lightsheet-zarr.md

@@ -0,0 +1,74 @@
+# ndi.lightsheet (Python)
+
+Python-side companion to `+ndi/+fun/+doc/+lightsheet/` in NDI-matlab.
+Provides the `napariViewLightsheet` console script that the MATLAB
+`LightsheetZarrManager` GUI shells out to, and the `PyramidReader`
+that pulls `lightsheetZarrPyramid` + `lightsheetZarrLevel` documents
+through the NDI cloud API and presents them to napari.
+
+## Layout
+
+```
+src/ndi/lightsheet/
+    __init__.py
+    napari_view.py            # console-script entry
+    pyramid_reader.py         # dask-backed reader (scaffold)
+scripts/
+    napariViewLightsheet      # shell wrapper installed to /usr/local/bin
+```
+
+## pyproject.toml additions
+
+Add to the `[project.optional-dependencies]` block (or `[project.dependencies]`
+if you want the viewer to be part of the base install):
+
+```toml
+[project.optional-dependencies]
+viewer = [
+    "napari[all]>=0.5",
+    "magicgui>=0.8",
+    "qtpy>=2.4",
+    "dask>=2024.3",
+    "zarr>=2.13",
+    "numcodecs>=0.10",
+]
+```
+
+Add the console script:
+
+```toml
+[project.scripts]
+napariViewLightsheet = "ndi.lightsheet.napari_view:main"
+```
+
+## Status
+
+Scaffold. What is wired:
+
+- `napari_view.py` parses the argument set that
+  `ndi.fun.doc.lightsheet.viewCommand` in NDI-matlab emits.
+- `PyramidReader` names the exact hooks the console script uses
+  (`open`, `list_levels`, `multiscale`, `sibling`, `attach_controls`).
+
+What still needs building (each `raise NotImplementedError` in
+`pyramid_reader.py` marks a hook):
+
+1. Session open + parent-document resolution through the NDI cloud
+   API (via `ndi.session.dir` or the cloud-cache equivalent).
+2. `list_levels` - the depends_on query for `lightsheetZarrLevel`
+   with a matching `lightsheetZarrPyramid_id`.
+3. `multiscale` - the actual chunk fetcher. Each level is a
+   `dask.array.Array` built from `dask.array.from_delayed` blocks;
+   each delayed block reads one chunk from the level's `chunk.bin_#`
+   file series (through the cloud API), decodes it per the level's
+   `codec` field, and reshapes to the level's `chunks` shape in
+   `dtype` / `chunk_order`. Missing chunks (not in
+   `n_chunks_stored`) are filled with `fill_value`.
+4. `sibling` - session-scope query by `(subject_id, source_file_id,
+   reduction)` to find the sibling parent for a reduction swap.
+5. `attach_controls` - magicgui dock widgets:
+     * reduction (radio: mean / max) - switches the layer's data
+       source via `sibling`
+     * channel (dropdown) - swaps which channel(s) render
+     * level lock (spin box; -1 = auto) - forces a level regardless
+       of napari's zoom

src/ndi/lightsheet/__init__.py

@@ -0,0 +1,15 @@
+"""ndi.lightsheet - lightsheet OME-Zarr pyramid tools for NDI sessions.
+
+Mirrors +ndi/+fun/+doc/+lightsheet in NDI-matlab. The primary consumer is
+the ``napariViewLightsheet`` console script (see :mod:`ndi.lightsheet.napari_view`),
+which the MATLAB GUI shell-execs via
+``/usr/local/bin/napariViewLightsheet``. The console script reads
+``lightsheetZarrPyramid`` + ``lightsheetZarrLevel`` documents from an
+NDI session through the (HIPAA-compliant) NDI cloud API and hands the
+lazy multiscale arrays to napari.
+"""
+
+from ndi.lightsheet.napari_view import main as napari_view_main
+from ndi.lightsheet.pyramid_reader import PyramidReader
+
+__all__ = ["napari_view_main", "PyramidReader"]

src/ndi/lightsheet/napari_view.py

@@ -0,0 +1,129 @@
+"""napariViewLightsheet - open a lightsheetZarrPyramid in napari.
+
+Console-script entry point declared in NDI-python's ``pyproject.toml``.
+
+Usage
+-----
+
+    napariViewLightsheet <session-path> --pyramid <id> [--reduction mean|max]
+                        [--channel N] [--level L] [--no-controls]
+                        [--name NAME]
+
+The command line is built by ``ndi.fun.doc.lightsheet.viewCommand`` in
+NDI-matlab. Keep the flag set in sync there.
+
+Why a console script and not a napari plugin
+--------------------------------------------
+
+The MATLAB GUI shells out to
+``/usr/local/bin/napariViewLightsheet``, a small wrapper that scrubs
+``LD_LIBRARY_PATH`` / ``DYLD_*`` (MATLAB pollutes them) and then execs
+this console script. If the Python side were a napari plugin the wrapper
+would still need to exist -- MATLAB's library paths kill any Python
+started from MATLAB -- but the plugin would then have to be installed
+into whatever napari environment the user has, and there is no way for
+the MATLAB manager to check whether it is. A console script installed
+alongside NDI-python is what the wrapper points at and what the manager
+knows how to depend on.
+
+This file is currently a scaffold: it parses arguments, opens the
+session, resolves the pyramid document, and exits with a message that
+says which piece is still to land. The reader lives in
+:mod:`ndi.lightsheet.pyramid_reader`; that is where dask + napari get
+wired up.
+"""
+
+from __future__ import annotations
+
+import argparse
+import sys
+from typing import Sequence
+
+
+def build_parser() -> argparse.ArgumentParser:
+    p = argparse.ArgumentParser(
+        prog="napariViewLightsheet",
+        description="Open a lightsheetZarrPyramid from an NDI session in napari.",
+    )
+    p.add_argument(
+        "session_path",
+        help="Path to the ndi.session.dir on disk (its top-level directory).",
+    )
+    p.add_argument(
+        "--pyramid",
+        required=True,
+        help="Document id of the lightsheetZarrPyramid to open.",
+    )
+    p.add_argument(
+        "--reduction",
+        choices=("mean", "max"),
+        default=None,
+        help="Open the sibling pyramid with this reduction instead of "
+        "the one --pyramid names.",
+    )
+    p.add_argument(
+        "--channel",
+        type=int,
+        default=None,
+        help="1-based channel index to display. Omit to add every "
+        "channel as its own napari layer.",
+    )
+    p.add_argument(
+        "--level",
+        type=int,
+        default=None,
+        help="0-based level to display at startup. Omit to let the "
+        "viewer pick from the window size.",
+    )
+    p.add_argument(
+        "--no-controls",
+        dest="controls",
+        action="store_false",
+        default=True,
+        help="Do not dock the reduction / channel / level panels.",
+    )
+    p.add_argument(
+        "--name",
+        default=None,
+        help="Layer name. Omit to use the pyramid document's label.",
+    )
+    return p
+
+
+def main(argv: Sequence[str] | None = None) -> int:
+    args = build_parser().parse_args(argv)
+
+    # Lazy imports so ``--help`` does not pay the napari / ndi cost.
+    try:
+        import ndi  # noqa: F401
+    except ImportError as exc:  # pragma: no cover
+        print(f"ndi package not importable: {exc}", file=sys.stderr)
+        return 2
+
+    from ndi.lightsheet.pyramid_reader import PyramidReader
+
+    try:
+        reader = PyramidReader.open(args.session_path, pyramid_id=args.pyramid)
+    except NotImplementedError as exc:
+        print(f"napariViewLightsheet: {exc}", file=sys.stderr)
+        print(
+            "This is the scaffold PR. See README-lightsheet-zarr.md at "
+            "the top of ndi/lightsheet/ for the next steps.",
+            file=sys.stderr,
+        )
+        return 3
+
+    # The rest of the wiring lands with the PyramidReader implementation:
+    #   import napari
+    #   viewer = napari.Viewer()
+    #   arrays = reader.multiscale(reduction=args.reduction, channel=args.channel)
+    #   viewer.add_image(arrays, name=args.name or reader.default_name,
+    #                    multiscale=True)
+    #   if args.controls: reader.attach_controls(viewer)
+    #   napari.run()
+
+    return 0
+
+
+if __name__ == "__main__":  # pragma: no cover
+    raise SystemExit(main())

src/ndi/lightsheet/pyramid_reader.py

@@ -0,0 +1,123 @@
+"""PyramidReader - read lightsheetZarrPyramid + lightsheetZarrLevel documents
+through the (HIPAA-compliant) NDI cloud API and present each level as a lazy
+dask array.
+
+Where this fits: :mod:`ndi.lightsheet.napari_view` opens a
+``PyramidReader``, asks for its multiscale array list, and hands that
+list to :func:`napari.Viewer.add_image`.
+
+Status: scaffold. The parts that need writing are called out inline.
+
+Design
+------
+
+- ``PyramidReader.open(session_path, pyramid_id)`` opens the NDI session
+  (local dir or via ``ndi.session.dir``) and resolves the parent
+  ``lightsheetZarrPyramid`` document by id.
+- ``PyramidReader.list_levels()`` queries every ``lightsheetZarrLevel``
+  whose ``lightsheetZarrPyramid_id`` depends_on the parent id and
+  returns them in finest-first order.
+- ``PyramidReader.multiscale(reduction=None, channel=None)`` returns a
+  list of dask arrays, one per level, that each fetch chunks lazily
+  from the level document's ``chunk.bin_#`` file series through the
+  NDI cloud cache. napari expects such a list under ``multiscale=True``.
+- ``PyramidReader.attach_controls(viewer)`` docks small magicgui panels
+  for reduction switching (mean/max) and channel selection.
+
+Reduction switching under this design is *pyramid switching*: mean and
+max are separate ``lightsheetZarrPyramid`` documents in the session
+(NDI documents are immutable, so a new reduction is a new document, not
+an edit). ``PyramidReader.sibling(reduction)`` locates the sibling
+parent that shares this pyramid's ``subject_id`` +
+``source_file_id`` and returns another ``PyramidReader`` for it.
+
+Chunk file series
+-----------------
+
+Each ``lightsheetZarrLevel`` owns a ``chunk.bin_#`` file series. To
+find chunk ``(i0, i1, ..., in)`` in the level's ``chunk_grid``:
+
+    # ``#`` is 1-based; C-order flatten of the chunk grid coordinates.
+    n = 1 + ravel_multi_index(coords, chunk_grid, order="C")
+
+Every file is one chunk's raw byte payload after the level's ``codec``
+compression is applied (`raw` = uncompressed little-endian bytes in
+``dtype`` / ``chunk_order``). The reader in this module does the
+inverse.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Any, List, Optional
+
+
+@dataclass
+class PyramidReader:
+    """Reader for a single lightsheetZarrPyramid + its levels.
+
+    Attributes
+    ----------
+    session_path
+        Path to the ndi.session.dir on disk (or the cloud-cache local
+        mirror of one).
+    pyramid_id
+        Document id of the parent lightsheetZarrPyramid.
+    """
+
+    session_path: str
+    pyramid_id: str
+    _parent: Optional[Any] = None
+    _levels: Optional[List[Any]] = None
+
+    @classmethod
+    def open(cls, session_path: str, pyramid_id: str) -> "PyramidReader":
+        """Open the session and resolve the parent document."""
+        raise NotImplementedError(
+            "PyramidReader.open needs the NDI-python session + cloud cache "
+            "loader hooked up. See README-lightsheet-zarr.md at the top of "
+            "ndi/lightsheet/ for the design and status."
+        )
+
+    @property
+    def default_name(self) -> str:
+        if self._parent is None:
+            return "lightsheet zarr"
+        props = self._parent.document_properties["lightsheetZarrPyramid"]
+        label = props.get("label") or props.get("pyramid_name") or "lightsheet zarr"
+        return label
+
+    def list_levels(self) -> List[Any]:
+        """Return the level documents, finest-first."""
+        raise NotImplementedError(
+            "list_levels needs the depends_on query wired up through the "
+            "session's database. See README-lightsheet-zarr.md."
+        )
+
+    def multiscale(
+        self,
+        reduction: Optional[str] = None,
+        channel: Optional[int] = None,
+    ) -> List[Any]:
+        """Return one lazy dask array per level, aligned with the level order.
+
+        napari consumes this list under ``add_image(..., multiscale=True)``.
+        """
+        raise NotImplementedError(
+            "multiscale needs the chunk.bin_# fetcher + a small dask "
+            "wrapper. See README-lightsheet-zarr.md."
+        )
+
+    def sibling(self, reduction: str) -> "PyramidReader":
+        """Return a PyramidReader for the sibling pyramid with a different reduction."""
+        raise NotImplementedError(
+            "sibling() needs a session-scope query by (subject_id, "
+            "source_file_id, reduction). See README-lightsheet-zarr.md."
+        )
+
+    def attach_controls(self, viewer: Any) -> None:
+        """Dock reduction / channel / level panels on the napari viewer."""
+        raise NotImplementedError(
+            "attach_controls needs the magicgui panels. See "
+            "README-lightsheet-zarr.md."
+        )