From 00785e5f02d21d8e2eb226d21f08d108c4f94392 Mon Sep 17 00:00:00 2001 From: Brett Sun Date: Tue, 24 Jan 2017 19:02:37 +0100 Subject: [PATCH 1/2] Add WebSocket Event Stream API spec --- .../http-client-server-api.rst | 3 + docs/server/source/drivers-clients/index.rst | 1 + .../websocket-event-stream-api.rst | 114 ++++++++++++++++++ 3 files changed, 118 insertions(+) create mode 100644 docs/server/source/drivers-clients/websocket-event-stream-api.rst diff --git a/docs/server/source/drivers-clients/http-client-server-api.rst b/docs/server/source/drivers-clients/http-client-server-api.rst index 5444be8f..26ccd2f5 100644 --- a/docs/server/source/drivers-clients/http-client-server-api.rst +++ b/docs/server/source/drivers-clients/http-client-server-api.rst @@ -10,6 +10,7 @@ If you set up a BigchainDB node or reverse proxy yourself, and you're not sure what the API Root URL is, then see the last section of this page for help. +.. _bigchaindb-root-url: BigchainDB Root URL ------------------- @@ -393,6 +394,8 @@ Votes :statuscode 400: The request wasn't understood by the server, e.g. just requesting ``/votes``, without defining ``block_id``. +.. _determining-the-api-root-url: + Determining the API Root URL ---------------------------- diff --git a/docs/server/source/drivers-clients/index.rst b/docs/server/source/drivers-clients/index.rst index 18894f60..704832c0 100644 --- a/docs/server/source/drivers-clients/index.rst +++ b/docs/server/source/drivers-clients/index.rst @@ -15,6 +15,7 @@ community projects listed below. :maxdepth: 1 http-client-server-api + websocket-event-stream-api The Python Driver Transaction CLI diff --git a/docs/server/source/drivers-clients/websocket-event-stream-api.rst b/docs/server/source/drivers-clients/websocket-event-stream-api.rst new file mode 100644 index 00000000..f10575a9 --- /dev/null +++ b/docs/server/source/drivers-clients/websocket-event-stream-api.rst @@ -0,0 +1,114 @@ +The WebSocket Event Stream API +============================== + +.. important:: + This is currently scheduled to be implemented in BigchainDB Server 0.10. + +BigchainDB provides real-time event streams over the WebSocket protocol with +the Event Stream API. + +Connecting to an event stream from your application enables a BigchainDB node +to notify you as events are processed, such as new `validated transactions <#valid-transactions>`_. + + +Demoing the API +--------------- + +You may be interested in demoing the Event Stream API with the `WebSocket echo test `_ +to familiarize yourself before attempting an integration. + + +Determining Support for the Event Stream API +-------------------------------------------- + +In practice, it's a good idea to make sure that the node you're connecting with +has advertised support for the Event Stream API. To do so, send a HTTP GET +request to the node's :ref:`Root URL ` and check that the +response contains a ``streams_`` property in ``_links``:: + + { + "_links": { + "streams_v1": "ws://example.com:9984/api/v1/streams/" + } + } + + +Connection Keep Alive +~~~~~~~~~~~~~~~~~~~~~ + +The Event Stream API requires clients to signal that they'd like their +connection to stay open by sending "pings" across the open connection. +BigchainDB nodes will automatically close any connections that haven't sent a +ping in the last three minutes. + +.. note:: + + While three minutes is the limit before a BigchainDB server will terminate + a connection, we suggest sending a ping every 30 seconds for better + reliability. + +A "ping" consists of a message containing only the string ``"ping"``, for example +in JavaScript: + +.. code-block:: javascript + + new WebSocket("...").send("ping") + +If the BigchainDB node received the ping, it'll respond back with a message +containing only the string ``"pong"``. + + +Streams +------- + +Each stream is meant as a unidirectional communication channel, where the +BigchainDB node is the only party sending messages (except for `keep-alive +pings <#connection-keep-alive>`_). Any messages sent to the BigchainDB node +(except the keep-alive pings) will be ignored. + +Streams will always be under the WebSocket protocol (so ``ws://`` or +``wss://``) and accessible as extensions to the ``/api/v/streams/`` +API root URL (for example, `validated transactions <#valid-transactions>`_ +would be accessible under ``/api/v1/streams/valid_tx``). If you're running your +own BigchainDB instance and need help determining its root URL, you can find +more :ref:`here `. + +All messages sent in a stream are in the JSON format. + +.. note:: + + For simplicity, BigchainDB initially only provides a stream for all + validated transactions. In the future, we may provide streams for other + information, such as new blocks, new votes, or invalid transactions. We may + also provide the ability to filter the stream for specific qualities, such + as a specific ``output``'s ``public_key``. + + If you have specific use cases that you think would fit as part of this + API, feel free to reach out via `gitter `_ + or `email `_. + +Valid Transactions +~~~~~~~~~~~~~~~~~~ + +``/valid_tx`` + +Streams an event for any newly validated transactions. Message bodies contain +the transaction's ID, associated asset ID, and containing block's ID. + +Example message:: + + { + "txid": "", + "assetid": "", + "blockid": "" + } + + +.. note:: + + Transactions in BigchainDB are validated in batches ("blocks") and will, + therefore, be streamed in batches. Each block can contain up to a 1000 + transactions, ordered by the time at which they were included in the block. + The ``/valid_tx`` stream will send these transactions in the same order + that the block stored them in, but this does **NOT** guarantee that you + will recieve the events in that same order. From 9307c21baa1677ee0a06a2ba3d283fde83bcd3c3 Mon Sep 17 00:00:00 2001 From: Brett Sun Date: Thu, 16 Feb 2017 15:26:17 +0100 Subject: [PATCH 2/2] Remove connect keep alive protocol from Event Stream API --- .../websocket-event-stream-api.rst | 30 ++++--------------- 1 file changed, 6 insertions(+), 24 deletions(-) diff --git a/docs/server/source/drivers-clients/websocket-event-stream-api.rst b/docs/server/source/drivers-clients/websocket-event-stream-api.rst index f10575a9..22effbc1 100644 --- a/docs/server/source/drivers-clients/websocket-event-stream-api.rst +++ b/docs/server/source/drivers-clients/websocket-event-stream-api.rst @@ -36,35 +36,17 @@ response contains a ``streams_`` property in ``_links``:: Connection Keep Alive ~~~~~~~~~~~~~~~~~~~~~ -The Event Stream API requires clients to signal that they'd like their -connection to stay open by sending "pings" across the open connection. -BigchainDB nodes will automatically close any connections that haven't sent a -ping in the last three minutes. - -.. note:: - - While three minutes is the limit before a BigchainDB server will terminate - a connection, we suggest sending a ping every 30 seconds for better - reliability. - -A "ping" consists of a message containing only the string ``"ping"``, for example -in JavaScript: - -.. code-block:: javascript - - new WebSocket("...").send("ping") - -If the BigchainDB node received the ping, it'll respond back with a message -containing only the string ``"pong"``. - +The Event Stream API initially does not provide any mechanisms for connection +keep alive other than enabling TCP keepalive on each open WebSocket connection. +In the future, we may add additional functionality to handle ping/pong frames +or payloads designed for keep alive. Streams ------- Each stream is meant as a unidirectional communication channel, where the -BigchainDB node is the only party sending messages (except for `keep-alive -pings <#connection-keep-alive>`_). Any messages sent to the BigchainDB node -(except the keep-alive pings) will be ignored. +BigchainDB node is the only party sending messages. Any messages sent to the +BigchainDB node will be ignored. Streams will always be under the WebSocket protocol (so ``ws://`` or ``wss://``) and accessible as extensions to the ``/api/v/streams/``