Issue 2656 windowed array#2671
Open
fluidnumericsJoe wants to merge 5 commits into
Open
Conversation
for more information, see https://pre-commit.ci
Comment on lines
+130
to
+157
| def to_windowed_arrays(self, *, max_levels: int | None = None): | ||
| """Wrap dask-backed field data in a rolling time-window cache. | ||
|
|
||
| Opt-in optimization for forward-marching simulations where all particles | ||
| share a single clock. For each dask-backed field ``isel`` then samples | ||
| a resident NumPy window instead of re-reading chunks and paying the | ||
| dask scheduling overhead on every kernel step. | ||
|
|
||
| NumPy-backed fields are left unchanged, so this is safe to call | ||
| more than once. | ||
|
|
||
| Parameters | ||
| ---------- | ||
| max_levels : int, optional | ||
| Cap on the number of time levels kept resident per field. ``None`` | ||
| (default) retains every level that the advancing clock still brackets. | ||
|
|
||
| Returns | ||
| ------- | ||
| FieldSet | ||
| ``self``, to allow chaining. | ||
| """ | ||
| for field in self.fields.values(): | ||
| components = (field.U, field.V, field.W) if isinstance(field, VectorField) else (field,) | ||
| for component in components: | ||
| if component is not None: | ||
| component.data = maybe_windowed(component.data, max_levels=max_levels) | ||
| return self |
Contributor
willirath
reviewed
Jun 16, 2026
| """Bulk, sequential read of one time level into NumPy (the dask->NumPy step).""" | ||
| return np.asarray(self._data.isel({self._tdim: int(lvl)}).values) | ||
|
|
||
| def _ensure(self, levels: np.ndarray) -> None: |
Contributor
There was a problem hiding this comment.
Suggested change
| def _ensure(self, levels: np.ndarray) -> None: | |
| def _ensure(self, levels: np.ndarray) -> None: | |
| if self._max < len(levels): | |
| raise ValueError("Trying to ensure more levels than we can hold.") |
willirath
reviewed
Jun 16, 2026
| ds["U"].data[:] = 1.0 # steady zonal flow -> in-bounds, deterministic | ||
|
|
||
| def run(windowed): | ||
| d = ds.chunk({"time": 1}) if chunked else ds |
Contributor
There was a problem hiding this comment.
Suggested change
| d = ds.chunk({"time": 1}) if chunked else ds | |
| d = ds.chunk({"time": 1}) if windowed else ds |
| self._cache[lvl] = self._read_level(lvl) | ||
| self.loads += 1 | ||
| self.bytes_read += self._slab_bytes | ||
| # retire stale levels (the clock only moves forward across the window) |
Member
There was a problem hiding this comment.
Have you checked whether this also works in time-backward (dt<0) mode?
Contributor
Author
There was a problem hiding this comment.
I have not :)
Member
There was a problem hiding this comment.
This class would also work on xgrids, right? Test it there too?
Contributor
Author
There was a problem hiding this comment.
The uxinterpolators changes here actually are from the #2666 base - in order for this PR to be possible, we need the unstructured bits to use vectorized isel
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Checklist
mainfor normal development,v3-supportfor v3 support)AI Disclosure