provide more documentation for MultipipesStepper

This commit is contained in:
Scott Sadler 2017-02-21 13:04:43 +01:00
parent fc04cd7bcd
commit 6110693ae8

View File

@ -2,6 +2,33 @@
Pipeline stepping is a way to advance the asynchronous data pipeline Pipeline stepping is a way to advance the asynchronous data pipeline
deterministically by exposing each step separately and advancing the states deterministically by exposing each step separately and advancing the states
manually. manually.
The multipipes.Pipeline class implements a pipeline that advanced
asynchronously and concurrently. This module provides an interface to the
BigchainDB pipelines that is static, ie, does not advance without prompting.
Rather than having a pipeline that is in an all or nothing running / not running
state, one can do the following:
steps = create_stepper()
with steps.start():
tx = my_create_and_write_tx()
steps.block_changefeed(timeout=1)
steps.block_filter_tx()
steps.block_validate_tx()
steps.block_create(timeout=True)
assert steps.counts == {'block_write': 1}
Pending items are held in the `.queues` attribute, and every task has it's own
queue (as in multipipes.Pipeline). Queues are just lists though so they can
be easily inspected.
As a shortcut, the `.counts` attribute is provided which returns the number of
pending items for each task. This is useful to assert the expected status of
the queues after performing some sequence.
""" """