Merge remote-tracking branch 'origin/master' into kyber-master-feat-cors

docs/root/source/bft.md

@@ -4,7 +4,7 @@ We have Byzantine fault tolerance (BFT) in our roadmap, as a switch that people
 
 Among the big, industry-used distributed databases in production today (e.g. DynamoDB, Bigtable, MongoDB, Cassandra, Elasticsearch), none of them are BFT. Indeed, almost all wide-area distributed systems in production are not BFT, including military, banking, healthcare, and other security-sensitive systems.
 
-The are many more practical things that nodes can do to increase security (e.g. firewalls, key management, access controls).
+There are many more practical things that nodes can do to increase security (e.g. firewalls, key management, access controls).
 
 From a [recent essay by Ken Birman](http://sigops.org/sosp/sosp15/history/05-birman.pdf) (of Cornell):
 

docs/server/source/websocket-event-stream-api.rst

@@ -8,9 +8,8 @@ The WebSocket Event Stream API
 
 BigchainDB provides real-time event streams over the WebSocket protocol with
 the Event Stream API.
-
 Connecting to an event stream from your application enables a BigchainDB node
-to notify you as events are processed, such as new `validated transactions <#valid-transactions>`_.
+to notify you as events occur, such as new `validated transactions <#valid-transactions>`_.
 
 
 Demoing the API
@@ -23,25 +22,31 @@ to familiarize yourself before attempting an integration.
 Determining Support for the Event Stream API
 --------------------------------------------
 
-In practice, it's a good idea to make sure that the node you're connecting with
+It's a good idea to make sure that the node you're connecting with
 has advertised support for the Event Stream API. To do so, send a HTTP GET
-request to the node's :ref:`Root URL <bigchaindb-root-url>` and check that the
-response contains a ``streams_<version>`` property in ``_links``::
+request to the node's :ref:`API Root Endpoint` 
+(e.g. ``http://localhost:9984/api/v1/``) and check that the
+response contains a ``streams_<version>`` property in ``_links``:
+
+.. code:: JSON
 
     {
       "_links": {
-         "streams_v1": "ws://example.com:9985/api/v1/streams/"
+         ...,
+         "streams_v1": "ws://example.com:9985/api/v1/streams/valid_tx",
+         ...
       }
     }
 
 
-Connection Keep Alive
-~~~~~~~~~~~~~~~~~~~~~
+Connection Keep-Alive
+---------------------
 
 The Event Stream API initially does not provide any mechanisms for connection
-keep alive other than enabling TCP keepalive on each open WebSocket connection.
+keep-alive other than enabling TCP keepalive on each open WebSocket connection.
 In the future, we may add additional functionality to handle ping/pong frames
-or payloads designed for keep alive.
+or payloads designed for keep-alive.
+
 
 Streams
 -------
@@ -54,8 +59,8 @@ Streams will always be under the WebSocket protocol (so ``ws://`` or
 ``wss://``) and accessible as extensions to the ``/api/v<version>/streams/``
 API root URL (for example, `validated transactions <#valid-transactions>`_
 would be accessible under ``/api/v1/streams/valid_tx``). If you're running your
-own BigchainDB instance and need help determining its root URL, you can find
-more :ref:`here <determining-the-api-root-url>`.
+own BigchainDB instance and need help determining its root URL,
+then see the page titled :ref:`Determining the API Root URL`.
 
 All messages sent in a stream are in the JSON format.
 
@@ -68,7 +73,7 @@ All messages sent in a stream are in the JSON format.
     as a specific ``output``'s ``public_key``.
 
     If you have specific use cases that you think would fit as part of this
-    API, feel free to reach out via `gitter <https://gitter.im/bigchaindb/bigchaindb>`_
+    API, feel free to reach out via `Gitter <https://gitter.im/bigchaindb/bigchaindb>`_
     or `email <mailto:product@bigchaindb.com>`_.
 
 Valid Transactions
@@ -79,7 +84,9 @@ Valid Transactions
 Streams an event for any newly validated transactions. Message bodies contain
 the transaction's ID, associated asset ID, and containing block's ID.
 
-Example message::
+Example message:
+
+.. code:: JSON
 
     {
         "tx_id": "<sha3-256 hash>",

k8s/nginx/container/Dockerfile

@@ -7,5 +7,5 @@ RUN apt-get update \
     && apt-get clean
 COPY nginx.conf.template /etc/nginx/nginx.conf
 COPY nginx_entrypoint.bash /
-EXPOSE 80 443 27017
+EXPOSE 80 81 443 444 27017
 ENTRYPOINT ["/nginx_entrypoint.bash"]

k8s/nginx/container/README.md

@@ -40,6 +40,8 @@ docker run \
 --env "BIGCHAINDB_FRONTEND_PORT=<port where nginx listens for BigchainDB connections>" \
 --env "BIGCHAINDB_BACKEND_HOST=<ip/hostname of instance where BigchainDB is running>" \
 --env "BIGCHAINDB_BACKEND_PORT=<port where BigchainDB is listening for connections>" \
+--env "BIGCHAINDB_WS_BACKEND_PORT=<port where BigchainDB is listening for websocket connections>" \
+--env "BIGCHAINDB_WS_FRONTEND_PORT=<port where nginx listens for BigchainDB WebSocket connections>" \
 --env "MONGODB_WHITELIST=<a ':' separated list of IPs that can connect to MongoDB>" \
 --env "DNS_SERVER=<ip of the dns server>" \
 --name=ngx \
@@ -59,6 +61,8 @@ docker run \
 --env "BIGCHAINDB_FRONTEND_PORT=80" \
 --env "BIGCHAINDB_BACKEND_HOST=localhost" \
 --env "BIGCHAINDB_BACKEND_PORT=9984" \
+--env="BIGCHAINDB_WS_FRONTEND_PORT=81" \
+--env="BIGCHAINDB_WS_BACKEND_PORT=9985" \
 --env "MONGODB_WHITELIST=192.168.0.0/16:10.0.2.0/24" \
 --name=ngx \
 --publish=80:80 \
@@ -67,3 +71,8 @@ docker run \
 bigchaindb/nginx
 ```
 
+### Note:
+You can test the WebSocket server by using 
+[wsc](https://slack-redir.net/link?url=https%3A%2F%2Fwww.npmjs.com%2Fpackage%2Fwsc) tool with a command like:
+`wsc -er ws://localhost:9985/api/v1/streams/valid_tx`.
+

k8s/nginx/container/nginx.conf.template

@@ -63,6 +63,26 @@ http {
       proxy_pass http://$bdb_backend:BIGCHAINDB_BACKEND_PORT;
     }
   }
+
+  server {
+    listen BIGCHAINDB_WS_FRONTEND_PORT;
+    underscores_in_headers on;
+
+    # keepalive connection settings
+    keepalive_timeout 20s;
+
+    # `slowloris` attack mitigation settings
+    client_body_timeout 10s;
+    client_header_timeout 10s;
+
+    location /api/v1/streams/ {
+      proxy_read_timeout 300s;
+      proxy_pass http://$bdb_backend:BIGCHAINDB_WS_BACKEND_PORT;
+      proxy_http_version 1.1;
+      proxy_set_header Upgrade $http_upgrade;
+      proxy_set_header Connection "upgrade";
+    }
+  }
 }
 
 # NGINX stream block for TCP and UDP proxies

k8s/nginx/container/nginx_entrypoint.bash

@@ -7,6 +7,8 @@ mongo_backend_port=`printenv MONGODB_BACKEND_PORT`
 bdb_frontend_port=`printenv BIGCHAINDB_FRONTEND_PORT`
 bdb_backend_host=`printenv BIGCHAINDB_BACKEND_HOST`
 bdb_backend_port=`printenv BIGCHAINDB_BACKEND_PORT`
+bdb_ws_frontend_port=`printenv BIGCHAINDB_WS_FRONTEND_PORT`
+bdb_ws_backend_port=`printenv BIGCHAINDB_WS_BACKEND_PORT`
 mongo_whitelist=`printenv MONGODB_WHITELIST`
 dns_server=`printenv DNS_SERVER`
 
@@ -17,6 +19,8 @@ if [[ -z "${mongo_frontend_port}" || \
     -z "${bdb_frontend_port}" || \
     -z "${bdb_backend_host}" || \
     -z "${bdb_backend_port}" || \
+    -z "${bdb_ws_backend_port}" || \
+    -z "${bdb_ws_frontend_port}" || \
     -z "${dns_server}" ]] ; then
   echo "Invalid environment settings detected. Exiting!"
   exit 1
@@ -31,6 +35,8 @@ sed -i "s|MONGODB_BACKEND_PORT|${mongo_backend_port}|g" $NGINX_CONF_FILE
 sed -i "s|BIGCHAINDB_FRONTEND_PORT|${bdb_frontend_port}|g" $NGINX_CONF_FILE
 sed -i "s|BIGCHAINDB_BACKEND_HOST|${bdb_backend_host}|g" $NGINX_CONF_FILE
 sed -i "s|BIGCHAINDB_BACKEND_PORT|${bdb_backend_port}|g" $NGINX_CONF_FILE
+sed -i "s|BIGCHAINDB_WS_FRONTEND_PORT|${bdb_ws_frontend_port}|g" $NGINX_CONF_FILE
+sed -i "s|BIGCHAINDB_WS_BACKEND_PORT|${bdb_ws_backend_port}|g" $NGINX_CONF_FILE
 sed -i "s|DNS_SERVER|${dns_server}|g" $NGINX_CONF_FILE
 
 # populate the whitelist in the conf file as per MONGODB_WHITELIST env var