protobuf_utils

module of dascore.io.sintela source

Utilities for reading Sintela protobuf MTLV recordings.

File format

A recording is a flat sequence of MTLV (magic-tag-length-value) envelope records. Each record is laid out as:

magic    4 bytes   little-endian uint32, always ``PBUF_MAGIC``
tag      4 bytes   ASCII, null-padded (e.g. ``META``, ``TS05``, ``FFT``)
size     4 bytes   little-endian uint32, payload length in bytes
payload  size      a serialized protobuf message

The tag identifies the payload’s packet family (see TS_TAGS etc.); the payload is decoded with the matching message class built below.

Protobuf schema strategy

Rather than vendoring Sintela’s generated *_pb2.py modules, we build the small subset of their schema that DASCore needs at runtime via descriptor_pb2 (see _build_proto_messages). This keeps protobuf an optional dependency, avoids committing generated code, and lets us skip the sample payloads entirely when scanning. The field numbers below must match Sintela’s real wire schema.

Why not just ship a .proto file? Neither obvious file-based option works well here:

  • A checked-in .proto cannot be loaded at runtime by the protobuf package alone – the pure-Python runtime has no .proto text parser. Compiling one requires the protoc compiler (or grpcio-tools), a heavier, non-pure-Python build/runtime dependency we don’t want to add for an optional format.
  • A committed generated *_pb2.py is tightly coupled to the installed protobuf runtime version (generated code has broken across protobuf major releases). Since protobuf is optional and unpinned, a user could have any version installed.

Building descriptors at runtime through the lower-level, more stable descriptor_pb2 reflection API sidesteps both problems.

Functions

Name Description
get_supported_family_tag Return the first supported data tag in a file without using protobuf.
read_payload Decode a Sintela protobuf file into data, coords, and attrs.
scan_payload Decode a Sintela protobuf file and return FiberIO scan payloads.

Classes

Name Description
BandMetadata Validated band packets laid out as (time, distance, band).
EnvelopeRecord The envelope information for one MTLV record.
FFTMetadata Validated FFT packets laid out as (time, distance, frequency).
ParsedMeta Selected metadata fields promoted from META packets.
SintelaProtobufAttrs Patch attributes for Sintela protobuf recordings.
TimeseriesMetadata Validated timeseries packets laid out as (time, distance).