catalog

module of dascore.io.index source

PatchCatalog: one metadata engine for every spool type.

The catalog owns the index tables (through a backend) and the composed selection state; a resolver turns flat-relation rows into patches (from files via dc.read, or from a live registry for in-memory spools); a syncer (the directory indexer) keeps directory-backed catalogs in step with the filesystem. See the spool index design doc and discussion #648.

Laziness contract: creating a catalog from patches does no metadata work until the first metadata operation (select/len/iteration), because backend bootstrap costs ~10s of ms while holding a patch list is free. Selection composes Query predicates without running SQL; realization (len, to_df, iteration) runs exactly one query per view.

Functions

Name Description
apply_exact_residuals Apply a view’s exact residual selections to a loaded patch.

Classes

Name Description
CompositeResolver Route rows to a live registry, a plan, or the filesystem by scheme.
FileResolver Load patches through dc.read; remoteness is the path layer’s job.
LiveResolver Serve patches from an in-memory registry.
PatchCatalog Query-composable metadata catalog over the spool index tables.
PatchResolver Turn one flat-relation row into a Patch.