lazy_array

A recipe for an array which is read a member at a time.

A LazyArray holds no data. It holds members, and each member says “this window of this source goes in this box of the output”. Everything is positional: axis numbers and sample indices, never dimension names, coordinates or units.

Many arrays share one LazyTable, which owns the storage; an array is a view of one of its rows. Members of an array are stored together, in canonical placement order, so slicing, joining and rechunking are vectorized over members and never open a file.

Examples

import numpy as np
from dascore.core.source import ArraySource
from dascore.core.lazy_array import LazyArray, concat

left = LazyArray.from_source(ArraySource.full((4, 3), 1.0))
right = LazyArray.from_source(ArraySource.full((4, 3), 2.0))
array = concat([left, right], axis=0)
assert array.shape == (8, 3) and len(array) == 2
assert np.array_equal(array[0:5].load()[:4], np.ones((4, 3)))

Functions

Name	Description
concat	Join arrays end to end along one axis, in one pass over their members.
stack	Join arrays along a new axis, in one pass over their members.

Classes

Name	Description
LazyArray	An array which says where each of its members is read from.
LazyTable	The storage many lazy arrays share.