pipe

module of dascore.workflow

source

A pipe: several tasks arranged into the shape of one operation.

A Pipe is a directed acyclic graph of Task objects. It is itself immutable, fingerprintable and serializable, so a whole processing chain can be compared, written to a file, handed to another process, and run later against different data.

Running one is deliberately boring: each task is given its inputs, in order, once everything feeding it has run. Nothing here schedules, streams or fuses; those belong to whatever runs the pipe.

Examples

from dascore.workflow import Task

class PipeAddExample(Task):
    '''Add a number.'''
    value: int = 1
    def run(self, number):
        return number + self.value

class PipeTimesExample(Task):
    '''Multiply by a number.'''
    value: int = 2
    def run(self, number):
        return number * self.value

pipe = PipeAddExample(value=2) | PipeTimesExample(value=3)
assert pipe(1) == 9

Functions

Name	Description
default_key	Return the name a task takes in a pipe which does not name it.
join	Return the pipe which runs one side and then the other.
unique_key	Return a node name nothing in a pipe has claimed yet.

Classes

Name	Description
Pipe	A directed acyclic graph of tasks.