diff --git a/docs/root/source/bft.md b/docs/root/source/bft.md index 0255c7b5..cde6834a 100644 --- a/docs/root/source/bft.md +++ b/docs/root/source/bft.md @@ -4,7 +4,7 @@ We have Byzantine fault tolerance (BFT) in our roadmap, as a switch that people Among the big, industry-used distributed databases in production today (e.g. DynamoDB, Bigtable, MongoDB, Cassandra, Elasticsearch), none of them are BFT. Indeed, almost all wide-area distributed systems in production are not BFT, including military, banking, healthcare, and other security-sensitive systems. -The are many more practical things that nodes can do to increase security (e.g. firewalls, key management, access controls). +There are many more practical things that nodes can do to increase security (e.g. firewalls, key management, access controls). From a [recent essay by Ken Birman](http://sigops.org/sosp/sosp15/history/05-birman.pdf) (of Cornell): diff --git a/docs/server/source/websocket-event-stream-api.rst b/docs/server/source/websocket-event-stream-api.rst index 3ce86553..0310ef63 100644 --- a/docs/server/source/websocket-event-stream-api.rst +++ b/docs/server/source/websocket-event-stream-api.rst @@ -8,9 +8,8 @@ The WebSocket Event Stream API BigchainDB provides real-time event streams over the WebSocket protocol with the Event Stream API. - Connecting to an event stream from your application enables a BigchainDB node -to notify you as events are processed, such as new `validated transactions <#valid-transactions>`_. +to notify you as events occur, such as new `validated transactions <#valid-transactions>`_. Demoing the API @@ -23,25 +22,31 @@ to familiarize yourself before attempting an integration. Determining Support for the Event Stream API -------------------------------------------- -In practice, it's a good idea to make sure that the node you're connecting with +It's a good idea to make sure that the node you're connecting with has advertised support for the Event Stream API. To do so, send a HTTP GET -request to the node's :ref:`Root URL ` and check that the -response contains a ``streams_`` property in ``_links``:: +request to the node's :ref:`API Root Endpoint` +(e.g. ``http://localhost:9984/api/v1/``) and check that the +response contains a ``streams_`` property in ``_links``: + +.. code:: JSON { "_links": { - "streams_v1": "ws://example.com:9985/api/v1/streams/" + ..., + "streams_v1": "ws://example.com:9985/api/v1/streams/valid_tx", + ... } } -Connection Keep Alive -~~~~~~~~~~~~~~~~~~~~~ +Connection Keep-Alive +--------------------- The Event Stream API initially does not provide any mechanisms for connection -keep alive other than enabling TCP keepalive on each open WebSocket connection. +keep-alive other than enabling TCP keepalive on each open WebSocket connection. In the future, we may add additional functionality to handle ping/pong frames -or payloads designed for keep alive. +or payloads designed for keep-alive. + Streams ------- @@ -54,8 +59,8 @@ Streams will always be under the WebSocket protocol (so ``ws://`` or ``wss://``) and accessible as extensions to the ``/api/v/streams/`` API root URL (for example, `validated transactions <#valid-transactions>`_ would be accessible under ``/api/v1/streams/valid_tx``). If you're running your -own BigchainDB instance and need help determining its root URL, you can find -more :ref:`here `. +own BigchainDB instance and need help determining its root URL, +then see the page titled :ref:`Determining the API Root URL`. All messages sent in a stream are in the JSON format. @@ -68,7 +73,7 @@ All messages sent in a stream are in the JSON format. as a specific ``output``'s ``public_key``. If you have specific use cases that you think would fit as part of this - API, feel free to reach out via `gitter `_ + API, feel free to reach out via `Gitter `_ or `email `_. Valid Transactions @@ -79,7 +84,9 @@ Valid Transactions Streams an event for any newly validated transactions. Message bodies contain the transaction's ID, associated asset ID, and containing block's ID. -Example message:: +Example message: + +.. code:: JSON { "tx_id": "", diff --git a/k8s/nginx/container/Dockerfile b/k8s/nginx/container/Dockerfile index c6c4dd3f..04c69e84 100644 --- a/k8s/nginx/container/Dockerfile +++ b/k8s/nginx/container/Dockerfile @@ -7,5 +7,5 @@ RUN apt-get update \ && apt-get clean COPY nginx.conf.template /etc/nginx/nginx.conf COPY nginx_entrypoint.bash / -EXPOSE 80 443 27017 +EXPOSE 80 81 443 444 27017 ENTRYPOINT ["/nginx_entrypoint.bash"] diff --git a/k8s/nginx/container/README.md b/k8s/nginx/container/README.md index 30f42bfe..5a1e1273 100644 --- a/k8s/nginx/container/README.md +++ b/k8s/nginx/container/README.md @@ -40,6 +40,8 @@ docker run \ --env "BIGCHAINDB_FRONTEND_PORT=" \ --env "BIGCHAINDB_BACKEND_HOST=" \ --env "BIGCHAINDB_BACKEND_PORT=" \ +--env "BIGCHAINDB_WS_BACKEND_PORT=" \ +--env "BIGCHAINDB_WS_FRONTEND_PORT=" \ --env "MONGODB_WHITELIST=" \ --env "DNS_SERVER=" \ --name=ngx \ @@ -59,6 +61,8 @@ docker run \ --env "BIGCHAINDB_FRONTEND_PORT=80" \ --env "BIGCHAINDB_BACKEND_HOST=localhost" \ --env "BIGCHAINDB_BACKEND_PORT=9984" \ +--env="BIGCHAINDB_WS_FRONTEND_PORT=81" \ +--env="BIGCHAINDB_WS_BACKEND_PORT=9985" \ --env "MONGODB_WHITELIST=192.168.0.0/16:10.0.2.0/24" \ --name=ngx \ --publish=80:80 \ @@ -67,3 +71,8 @@ docker run \ bigchaindb/nginx ``` +### Note: +You can test the WebSocket server by using +[wsc](https://slack-redir.net/link?url=https%3A%2F%2Fwww.npmjs.com%2Fpackage%2Fwsc) tool with a command like: +`wsc -er ws://localhost:9985/api/v1/streams/valid_tx`. + diff --git a/k8s/nginx/container/nginx.conf.template b/k8s/nginx/container/nginx.conf.template index 6167dceb..bae805a5 100644 --- a/k8s/nginx/container/nginx.conf.template +++ b/k8s/nginx/container/nginx.conf.template @@ -63,6 +63,26 @@ http { proxy_pass http://$bdb_backend:BIGCHAINDB_BACKEND_PORT; } } + + server { + listen BIGCHAINDB_WS_FRONTEND_PORT; + underscores_in_headers on; + + # keepalive connection settings + keepalive_timeout 20s; + + # `slowloris` attack mitigation settings + client_body_timeout 10s; + client_header_timeout 10s; + + location /api/v1/streams/ { + proxy_read_timeout 300s; + proxy_pass http://$bdb_backend:BIGCHAINDB_WS_BACKEND_PORT; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + } + } } # NGINX stream block for TCP and UDP proxies diff --git a/k8s/nginx/container/nginx_entrypoint.bash b/k8s/nginx/container/nginx_entrypoint.bash index e40d89f4..49578433 100755 --- a/k8s/nginx/container/nginx_entrypoint.bash +++ b/k8s/nginx/container/nginx_entrypoint.bash @@ -7,6 +7,8 @@ mongo_backend_port=`printenv MONGODB_BACKEND_PORT` bdb_frontend_port=`printenv BIGCHAINDB_FRONTEND_PORT` bdb_backend_host=`printenv BIGCHAINDB_BACKEND_HOST` bdb_backend_port=`printenv BIGCHAINDB_BACKEND_PORT` +bdb_ws_frontend_port=`printenv BIGCHAINDB_WS_FRONTEND_PORT` +bdb_ws_backend_port=`printenv BIGCHAINDB_WS_BACKEND_PORT` mongo_whitelist=`printenv MONGODB_WHITELIST` dns_server=`printenv DNS_SERVER` @@ -17,6 +19,8 @@ if [[ -z "${mongo_frontend_port}" || \ -z "${bdb_frontend_port}" || \ -z "${bdb_backend_host}" || \ -z "${bdb_backend_port}" || \ + -z "${bdb_ws_backend_port}" || \ + -z "${bdb_ws_frontend_port}" || \ -z "${dns_server}" ]] ; then echo "Invalid environment settings detected. Exiting!" exit 1 @@ -31,6 +35,8 @@ sed -i "s|MONGODB_BACKEND_PORT|${mongo_backend_port}|g" $NGINX_CONF_FILE sed -i "s|BIGCHAINDB_FRONTEND_PORT|${bdb_frontend_port}|g" $NGINX_CONF_FILE sed -i "s|BIGCHAINDB_BACKEND_HOST|${bdb_backend_host}|g" $NGINX_CONF_FILE sed -i "s|BIGCHAINDB_BACKEND_PORT|${bdb_backend_port}|g" $NGINX_CONF_FILE +sed -i "s|BIGCHAINDB_WS_FRONTEND_PORT|${bdb_ws_frontend_port}|g" $NGINX_CONF_FILE +sed -i "s|BIGCHAINDB_WS_BACKEND_PORT|${bdb_ws_backend_port}|g" $NGINX_CONF_FILE sed -i "s|DNS_SERVER|${dns_server}|g" $NGINX_CONF_FILE # populate the whitelist in the conf file as per MONGODB_WHITELIST env var