Annotations

Open in JupyterLite

An AnnotationSet records where things are in a patch’s dimensions: phase picks, boxes around noise, vehicle tracks, outlines of a region. It is a dataframe first. A picker’s output is one table, and the same set can grow into grouped events, paths and polygons without changing what it already holds.

Annotations describe data but never hold it. They are stated in coordinates, not sample indices, so they stay valid when the data is decimated, trimmed or re-chunked.

The model

A set has three parts:

• annotations: one row per located thing. For each dimension a row states a single value (time), a half-open range (time_min, time_max), or nothing, in which case it spans that dimension. A row stating only time is an instant across the whole fiber.
• features: what rows compose. Every row belongs to one feature, named by its feature_id; a row with none is its own feature. A feature is a group (unordered, the default), a path (vertices ordered by seq), or a polygon (rings ordered by seq, ring 0 outside, holes after). part numbers the pieces of a multipart shape.
• bases: named curves, such as a line or a moveout, that a path is drawn from. A path may state only its basis and is sampled when needed.

Any other column, such as phase, confidence or note, is yours and is kept as written. Features hold no coordinates; their bounds come from their rows.

time	distance_min	distance_max	phase	feature_id	reads as
0.01			P		an instant across the fiber
0.01	650	750	P	event_1	a pick on a stretch, part of event_1
	400	450			a stretch of fiber at all times

Picks

A picker’s output is a single table. AnnotationSet.from_patch takes the dimensions from the patch and stamps its acquisition_key and data_id, so the set records which data it was made on.

import tempfile
from pathlib import Path

import pandas as pd

import dascore as dc

patch = dc.get_example_patch("example_event_2")
picks = pd.DataFrame(
    {
        "time": [0.018, 0.010, 0.017, 0.052, 0.026, 0.045],
        "distance": [600.0, 700.0, 800.0, 600.0, 700.0, 800.0],
        "phase": ["P", "P", "P", "S", "S", "S"],
        "confidence": [0.9, 0.95, 0.8, 0.7, 0.9, 0.4],
    }
)
ann = dc.AnnotationSet.from_patch(patch, picks)
ann.annotations

	time	distance	phase	confidence	feature_id
0	0.018	600.0	P	0.90	None
1	0.010	700.0	P	0.95	None
2	0.017	800.0	P	0.80	None
3	0.052	600.0	S	0.70	None
4	0.026	700.0	S	0.90	None
5	0.045	800.0	S	0.40	None

A set of only annotations writes as one bare file. The file holds the annotations table alone. A CSV does not record the dimensions, so they are stated again on reading; io.to_parquet writes a Parquet file that keeps them, and needs pyarrow. The set’s attributes, such as data_id, need the directory form described under Saving and loading.

folder = Path(tempfile.mkdtemp())
path = folder / "picks.csv"
ann.io.to_csv(path)
assert dc.annotations(path, dims=patch.dims).annotations.equals(ann.annotations)

Building features

Picks become an event by sharing a feature_id. A feature_id naming no feature creates a group, so the picker never writes a features table:

event = dc.AnnotationSet.from_patch(patch, picks.assign(feature_id="event_1"))
event.features

	geometry	id
0	None	event_1

Builders return new sets. In add_path an array keyword is an annotations column, one value per vertex, and a scalar keyword is a features column:

ann = event.add_path(
    "s_wave",
    time=[0.026, 0.06, 0.09],
    distance=[700.0, 580.0, 470.0],
    amplitude=[20.0, 12.0, 6.0],
    wave="S",
)
ann["s_wave"].kind, ann["s_wave"].extra

('path', <FrozenDict {'wave': 'S'}>)

A path can also be drawn from a basis, given as a model or its document, and no vertices:

line = {
    "object_type": "Line",
    "start": {"time": 0.01, "distance": 700.0},
    "end": {"time": 0.04, "distance": 900.0},
}
ann = ann.add_path("p_wave", basis=line, wave="P")

add_polygon takes rings, the outer ring first and then any holes, each a table of vertices in order. Closure is implied:

outer = {"time": [0.06, 0.1, 0.1, 0.06], "distance": [400.0, 400.0, 450.0, 450.0]}
ann = ann.add_polygon("noise", rings=[outer], note="traffic")
ann

DASCore AnnotationSet 🏷

Dimensions (distance, time)

    *distance: min: 400.000 max: 800.000 <400.000> (value)
    *time: min: 0.010 max: 0.100 <0.09> (value)

Contents

    annotations: 13
    features: 4 (groups: 1, paths: 2, polygons: 1)
    bases: 1
    annotation columns: time, distance, phase, confidence, feature_id, amplitude, seq, part, ring
    feature columns: geometry, id, wave, basis, note

Attributes

    data_id: a261c8f083616ba0dcbd0a6c14f8f0c7

Querying

select filters on columns of either table. A value matches, a list is membership, and a tuple is an inclusive range:

confident_p = ann.select(phase="P", confidence=(0.85, None))
confident_p.annotations

	time	distance	phase	confidence	feature_id	amplitude	seq	part	ring
0	0.018	600.0	P	0.90	event_1	None	<NA>	<NA>	<NA>
1	0.010	700.0	P	0.95	event_1	None	<NA>	<NA>	<NA>

overlapping selects by location and keeps or drops each feature whole. Given a patch, it queries that patch’s extent:

early = ann.overlapping(time=(0.0, 0.02))
in_patch = ann.overlapping(patch.select(distance=(600, 800)))
[x.id for x in early], [x.id for x in in_patch]

(['event_1', 'p_wave'], ['event_1', 's_wave', 'p_wave'])

Three interval rules meet here: a row’s _min/_max range is half-open, a select tuple includes both ends, and overlapping given a patch includes the patch’s last sample.

bounds gives each feature’s extent, one row per feature:

ann.bounds()

	feature_id	annotation	kind	distance_min	distance_max	time_min	time_max
0	event_1	<NA>	group	600.0	800.0	0.010	0.052
1	s_wave	<NA>	path	470.0	700.0	0.026	0.090
2	p_wave	<NA>	path	700.0	900.0	0.010	0.040
3	noise	<NA>	polygon	400.0	450.0	0.060	0.100

Plotting

ann.viz.plot draws every feature on an axis, putting dimensions where waterfall puts them. Draw the patch first and the annotations over it. color may name a column, giving one color per value and a legend; features with no value, here the paths, the polygon and a trigger time added across the fiber, are grey:

ann = ann.add(pd.DataFrame({"time": [0.08], "note": ["trigger"]}))
ax = patch.viz.waterfall()
ax = ann.viz.plot(ax=ax, color="phase")

A pick is a marker, a path a line, a polygon a filled shape, and a row spanning one axis a line or band across it.

Editing

update changes the columns of one feature or one annotation, and remove drops one. merge combines sets stated in the same dimensions, such as hand and automatic picks.

ann = ann.update(feature="event_1", magnitude=1.2).remove(feature="noise")

Saving and loading

A set with features or bases is saved as a directory holding attrs.json, the annotations and features tables, and bases.json. dc.annotations reads it back, and also reads bare tables and collections of sets:

directory = ann.io.save(folder / "event_1")
assert dc.annotations(directory) == ann