provide more documentation for MultipipesStepper

This commit is contained in:
Scott Sadler 2017-02-21 13:04:43 +01:00
parent fc04cd7bcd
commit 6110693ae8

View File

@ -2,6 +2,33 @@
Pipeline stepping is a way to advance the asynchronous data pipeline
deterministically by exposing each step separately and advancing the states
manually.
The multipipes.Pipeline class implements a pipeline that advanced
asynchronously and concurrently. This module provides an interface to the
BigchainDB pipelines that is static, ie, does not advance without prompting.
Rather than having a pipeline that is in an all or nothing running / not running
state, one can do the following:
steps = create_stepper()
with steps.start():
tx = my_create_and_write_tx()
steps.block_changefeed(timeout=1)
steps.block_filter_tx()
steps.block_validate_tx()
steps.block_create(timeout=True)
assert steps.counts == {'block_write': 1}
Pending items are held in the `.queues` attribute, and every task has it's own
queue (as in multipipes.Pipeline). Queues are just lists though so they can
be easily inspected.
As a shortcut, the `.counts` attribute is provided which returns the number of
pending items for each task. This is useful to assert the expected status of
the queues after performing some sequence.
"""