Patch Conversions
DASCore converts patches to common library objects and back.
Pandas
import dascore as dc
patch = dc.get_example_patch()
df = patch.io.to_dataframe()
patch_from_df = dc.io.dataframe_to_patch(df)Xarray
import dascore as dc
patch = dc.get_example_patch()
dar = patch.io.to_xarray()
patch_from_dar = dc.io.xarray_to_patch(dar)Coordinate labels are materialized, as in any xarray index. With lazy_coords=True (or a collection of names) the DataArray’s index instead holds the patch’s own coordinate and computes labels only for the samples a selection or .values asks about. Selection answers as it does on a materialized index (partial datetime strings, method="nearest", tolerance), and converting back returns the coordinate exactly, fractional sampling rates and gaps included.
import dascore as dc
patch = dc.get_example_patch()
lazy = patch.io.to_xarray(lazy_coords=True)xarray aligns a lazy index only with others of its kind, so combining a lazy array with an array whose index holds its labels (built from numpy, loaded from netCDF, or converted with the default), or reindexing it to new labels, raises an AlignmentError. To combine them, give the lazy array ordinary indexes: drop_indexes removes the lazy index but keeps the coordinate’s labels, and set_xindex builds an ordinary index from them, reading every label into memory (8 bytes a sample; a day at 1 kHz is about 0.7 GB). The data stays as it is. Select first where you can, so only the selection’s labels are read:
eager = lazy.drop_indexes(["time", "distance"]).set_xindex("time").set_xindex("distance")The other way round often costs less: give the ordinary array the lazy index type instead. Its labels are already in memory, and where they equal the lazy array’s the two align without reading the lazy labels at all (labels that differ are joined as ordinary indexes would be, which reads both).
from dascore.xarray.index import CoordIndex
aligned = eager.drop_indexes("time").set_xindex("time", CoordIndex)A whole spool converts to a lazy, dask-backed DataTree with one node per group of related patches (partitioned exactly as chunk partitions them) and one child node per merged output segment. Building the tree reads no data, and the coordinate of the dimension segments are merged along is lazy (as with lazy_coords=True above), so a segment spanning years of samples costs nothing to label. Other dimensions keep ordinary indexes, so per-channel arrays such as gains combine with a segment directly; along the merged dimension, use the drop_indexes/set_xindex swap above. Computing a selection loads only the source patches it touches.
import dascore as dc
spool = dc.get_example_spool("diverse_das")
# Convert the spool to a DataTree of lazy arrays.
tree = spool.io.to_xarray()
# Nothing has been loaded yet; this reads only the patches it needs.
first = next(node for node in tree.subtree if "data" in node.dataset)
small = first.dataset["data"].isel(time=slice(0, 100)).compute()A selection reads only the samples it names, however large the files behind it are. block_size bounds a bulk read instead: computing a whole segment asks for one block at a time, so a source patch larger than it is read in several windows rather than whole. Unset, it takes the configured xarray_block_size (256 MiB by default). A format which cannot hand back a window reads whole whatever the setting says, since splitting it would read the file once per block.
Patch methods on a DataArray
A DataArray DASCore produced carries a dc accessor holding every public Patch method. A call converts to a patch, runs the method, and converts a patch it returns back; a result which is not a patch, such as an array or a spool, comes back as it is. A method taking another patch takes another DataArray.
import dascore as dc
dar = dc.get_example_patch().io.to_xarray()
filtered = dar.dc.pass_filter(time=(1, 10))
times = dar.dc.get_array("time")
patch = dar.dc.to_patch()A DataArray from somewhere else needs import dascore.xarray.accessor first, since registering the accessor means importing xarray, which DASCore does not do on its own.
A dask-backed DataArray is passed through as it stands, so a method which works on the array’s own backend leaves it lazy. One which cannot converts, as it does on a patch. Where such a method announces that conversion, and the array is larger than free memory, the accessor raises rather than attempting it; a method which converts silently is not caught, and fails on memory as it would anywhere else.
A coordinate a DataArray states rather than stores (a lazy one, whose labels are computed on demand) is stated again in what a method hands back, so labelling a long acquisition costs no more after a call than before it. Every DataArray in the call says how its own coordinates arrived, an argument as much as the one called on. A coordinate which arrived spelled out anywhere is spelled out in the result, since a spelled-out index is what xarray aligns on. Laziness follows the name, so a method which renames such a coordinate spells its labels out under the new name.
ObsPy
import dascore as dc
patch = dc.get_example_patch()
stream = patch.io.to_obspy()
patch_from_stream = dc.io.obspy_to_patch(stream)obspy_to_patch requires every trace to have the same data length and the requested dimension in stats (distance by default).