import numpy as np
import dascore as dc
from dascore.core.source import ArraySource
from dascore.lazy.execute import chunk_boxes, compute, read_schedule
from dascore.lazy.recipe import Recipe
recipe = Recipe.from_source(ArraySource.full((100, 8), 1.0))
coords = {"time": np.arange(100), "distance": np.arange(8)}
patch = dc.Patch(data=recipe, coords=coords, dims=("time", "distance"))
lazy = patch.select(time=(10, 90), samples=True)
lazy = lazy.median_filter(time=3, samples=True).data
# Four boxes of 20 samples; the second is read with a sample either side.
boxes = chunk_boxes(lazy, block_size=22 * 8 * 8)
assert [x[0] for x in boxes[:2]] == [slice(0, 20), slice(20, 40)]
assert read_schedule(lazy, boxes)[1][0] == slice(29, 51)
assert compute(lazy, boxes[1]).shape == (20, 8)execute
Running a lazy array: which chunks, what each one reads, and computing it.
A chunk is a box of the final array, and the work is three separate steps: chunk_boxes cuts the array into boxes, read_schedule says which box of the stored array each one needs, and compute reads that box and runs each step on it. load cuts and computes, and puts the boxes together in memory.
Examples
Functions
| Name | Description |
|---|---|
| chunk_boxes | Cut a lazy array into the boxes it is computed in. |
| compute | Read what one box of a lazy array needs, and compute it. |
| load | Compute a lazy array a box at a time, and return the whole of it. |
| read_schedule | Return the box of the stored array each box of the result reads. |