From 81b8cd9fd6988b5b14b1f1d2b6ac13202a286181 Mon Sep 17 00:00:00 2001 From: Troy McConaghy Date: Tue, 9 Oct 2018 12:14:13 +0200 Subject: [PATCH] Revised Simple Deployment Template: added NGINX instructions --- .../deploy-a-machine.md | 69 ++++++ .../simple-deployment-template/index.rst | 24 +- .../network-setup.md | 227 +++--------------- .../set-up-nginx.md | 42 ++++ .../set-up-node-software.md | 104 ++++++++ .../source/simple-deployment-template/tips.md | 59 +++++ nginx/nginx.conf | 125 ++++++++++ 7 files changed, 454 insertions(+), 196 deletions(-) create mode 100644 docs/server/source/simple-deployment-template/deploy-a-machine.md create mode 100644 docs/server/source/simple-deployment-template/set-up-nginx.md create mode 100644 docs/server/source/simple-deployment-template/set-up-node-software.md create mode 100644 docs/server/source/simple-deployment-template/tips.md create mode 100644 nginx/nginx.conf diff --git a/docs/server/source/simple-deployment-template/deploy-a-machine.md b/docs/server/source/simple-deployment-template/deploy-a-machine.md new file mode 100644 index 00000000..764229d0 --- /dev/null +++ b/docs/server/source/simple-deployment-template/deploy-a-machine.md @@ -0,0 +1,69 @@ + + +# Deploy a Machine for Your BigchainDB Node + +The first step is to deploy a machine for your BigchainDB node. +It might be a virtual machine (VM) or a real machine. +If you follow this simple deployment template, all your node's +software will run on that one machine. + +We don't make any assumptions about _where_ you run the machine. +It might be in Azure, AWS, your data center or a Raspberry Pi. + +## IP Addresses + +The following instructions assume all the nodes +in the network (including yours) have public IP addresses. +(A BigchainDB network _can_ be run inside a private network, +using private IP addresses, but we don't cover that here.) + +## Operating System + +**Use Ubuntu 18.04 or Ubuntu Server 18.04 as the operating system.** + +Similar instructions will work on other versions of Ubuntu, +and other recent Debian-like Linux distros, +but you may have to change the names of the packages, +or install more packages. + +## Network Security Group + +If your machine is in AWS or Azure, for example, _and_ +you want users to connect to BigchainDB via HTTPS, +then you should configure its network security group +to allow all incoming and outgoing traffic for: + +* TCP on port 22 (SSH) +* TCP on port 80 (HTTP) +* TCP on port 443 (HTTPS) +* Any protocol on port 26656 (Tendermint P2P) + +If you don't care about HTTPS, then forget about port 443, +and replace port 80 with port 9984 (the default BigchainDB HTTP port). + +## Update Your System + +SSH into your machine and update all its OS-level packages: + +``` +sudo apt update +sudo apt full-upgrade +``` + +## Node Security + +If you're going to use your node in production, +then you should take additional steps to secure it. +We don't cover that here; there are many books and websites +about securing Linux machines. + +## DNS Setup + +* Register a domain name for your BigchainDB node, such as `example.com` +* Pick a subdomain of that domain for your BigchainDB node, such as `bnode.example.com` +* Create a DNS "A Record" pointing your chosen subdomain (such as `bnode.example.com`) + at your machine's IP address. diff --git a/docs/server/source/simple-deployment-template/index.rst b/docs/server/source/simple-deployment-template/index.rst index 9ee713c2..55f277da 100644 --- a/docs/server/source/simple-deployment-template/index.rst +++ b/docs/server/source/simple-deployment-template/index.rst @@ -8,14 +8,34 @@ Simple Deployment Template ========================== -This section describes *one* way to deploy a BigchainDB network. -You can modify it as you see fit. +This section describes *one* way to deploy a BigchainDB network +(i.e. a set of connected BigchainDB nodes). +You can modify this simple deployment template as you see fit. It's "simple" in the sense that each BigchainDB node is installed and run on a single virtual machine (or real machine). We also have a :ref:`kubernetes-deployment-template` (not simple). +**Note 1:** These instructions will also work for a "network" with only one node. +If you want your network to be able to handle the failure or misbehavior +of one node, then your network must have at least four nodes. +Nodes can be added or removed from a network after is it up and running. + +**Note 2:** You might not need to set up your own network yet. +You should start by creating a proof-of-concept app that writes to +[the BigchainDB Testnet](https://testnet.bigchaindb.com/), +and if that goes well, then you can look into setting up your own network. + +**Note 3:** If you want to set up a node or network +so that you can contribute to developing and testing the BigchainDB code, +then see +[the docs about contributing to BigchainDB](https://docs.bigchaindb.com/projects/contributing/en/latest/index.html). + .. toctree:: :maxdepth: 1 + deploy-a-machine + set-up-nginx + set-up-node-software network-setup + tips troubleshooting diff --git a/docs/server/source/simple-deployment-template/network-setup.md b/docs/server/source/simple-deployment-template/network-setup.md index c3057895..79bb4385 100644 --- a/docs/server/source/simple-deployment-template/network-setup.md +++ b/docs/server/source/simple-deployment-template/network-setup.md @@ -6,132 +6,23 @@ Code is Apache-2.0 and docs are CC-BY-4.0 # How to Set Up a BigchainDB Network -Note 1: These instructions will also work for a "network" with only one node. +Until now, everything could be done by a node operator, by themselves. +Now the node operators, also called **Members**, must share some information +with each other, so they can form a network. -Note 2: You might not need to set up your own network yet. You should start by creating a proof-of-concept app that writes to [the BigchainDB Testnet](https://testnet.bigchaindb.com/), and if that goes well, then you can look into setting up your own network. +There is one special Member who helps coordinate everyone: the **Coordinator**. -Note 3: If you want to set up a node or network so that you can contribute to developing and testing the BigchainDB code, then see [the docs about contributing to BigchainDB](https://docs.bigchaindb.com/projects/contributing/en/latest/index.html). +## Member: Share hostname, pub_key.value and node_id -
+Each BigchainDB node is identified by its: -The process to create a network is both *social* and *technical*: *social* because someone (that we will call **Coordinator**) needs to find at least three other **Members** willing to join the network, and coordinate the effort; *technical* because each member of the network needs to set up a machine running BigchainDB. (Note: a Coordinator is a Member as well.) +* `hostname`, i.e. the node's DNS subdomain, such as `bnode.example.com`, or its IP address, such as `46.145.17.32` +* Tendermint `pub_key.value` +* Tendermint `node_id` -A **BigchainDB Network** (or just *Network*) is a set of **4 or more BigchainDB Nodes** (or *Nodes*). Every Node is independently managed by a Member, and runs an instance of the [BigchainDB Server software][bdb:software]. At the **Genesis** of a Network, there **MUST** be at least **4** Nodes ready to connect. After the Genesis, a Network can dynamically add new Nodes or remove old Nodes. - -A Network will stop working if more than one third of the Nodes are down or faulty _in any way_. The bigger a Network, the more failures it can handle. A Network of size 4 can tolerate only 1 failure, so if 3 out of 4 Nodes are online, everything will work as expected. Eventually, the Node that was offline will automatically sync with the others. - -## Before We Start - -This tutorial assumes you have basic knowledge on how to manage a GNU/Linux machine. - -**Please note: The commands on this page work on Ubuntu 18.04. Similar commands will work on other versions of Ubuntu, and other recent Debian-like Linux distros, but you may have to change the names of the packages, or install more packages.** - -We don't make any assumptions about **where** you run the Node. -You can run BigchainDB Server on a Virtual Machine on the cloud, on a machine in your data center, or even on a Raspberry Pi. Just make sure that your Node is reachable by the other Nodes. Here's a **non-exhaustive list of examples**: - -- **good**: all Nodes running in the cloud using public IPs. -- **bad**: some Nodes running in the cloud using public IPs, some Nodes in a private network. -- **good**: all Nodes running in a private network. - -The rule of thumb is: if Nodes can ping each other, then you are good to go. - -The next sections are labelled with **Member** or **Coordinator**, depending on who should follow the instructions. Remember, a Coordinator is also a Member. - -## Member: Set Up a Node - -Every Member in the Network **must** set up its own Node. The process consists of installing three components, BigchainDB Server, Tendermint Core, and MongoDB, and configuring the firewall. - -**Important note on security: it's up to the Member to harden their system.** - -### Install the Required Software - -Make sure your system is up to date. - -``` -sudo apt update -sudo apt full-upgrade -``` - -#### Install BigchainDB Server - -BigchainDB Server requires **Python 3.6+**, so make sure your system has it. Install the required packages: - -``` -# For Ubuntu 18.04: -sudo apt install -y python3-pip libssl-dev -# Ubuntu 16.04, and other Linux distros, may require other packages or more packages -``` - -Now install the latest version of BigchainDB. You can find the latest version by going to the [BigchainDB project release history page on PyPI][bdb:pypi]. For example, to install version 2.0.0b7, you would do: - -``` -# Change 2.0.0b7 to the latest version as explained above: -sudo pip3 install bigchaindb==2.0.0b7 -``` - -Check that you installed the correct version of BigchainDB Server using `bigchaindb --version`. - -#### Install (and Start) MongoDB - -Install a recent version of MongoDB. BigchainDB Server requires version 3.4 or newer. - -``` -sudo apt install mongodb -``` - -If you install MongoDB using the above command (which installs the `mongodb` package), it also configures MongoDB, starts MongoDB (in the background), and installs a MongoDB startup script (so that MongoDB will be started automatically when the machine is restarted). - -Note: The `mongodb` package is _not_ the official MongoDB package from MongoDB the company. If you want to install the official MongoDB package, please see [the MongoDB documentation](https://docs.mongodb.com/manual/installation/). Note that installing the official package _doesn't_ also start MongoDB. - -#### Install Tendermint - -The version of BigchainDB Server described in these docs only works well with Tendermint 0.22.8 (not a higher version number). Install that: - -``` -sudo apt install -y unzip -wget https://github.com/tendermint/tendermint/releases/download/v0.22.8/tendermint_0.22.8_linux_amd64.zip -unzip tendermint_0.22.8_linux_amd64.zip -rm tendermint_0.22.8_linux_amd64.zip -sudo mv tendermint /usr/local/bin -``` - -### Set Up the Firewall - -Make sure to accept inbound connections on ports `9984`, `9985`, and `26656`. You might also want to add port `22` so that you can continue to access the machine via SSH. - -``` -sudo ufw allow 22/tcp -sudo ufw allow 9984/tcp -sudo ufw allow 9985/tcp -sudo ufw allow 26656/tcp -sudo ufw enable -``` - -Some cloud providers, like Microsoft Azure, require you to change "security groups" (virtual firewalls) using their portal or other APIs (such as their CLI). - -## Member: Configure BigchainDB Server - -To configure BigchainDB Server, run: - -``` -bigchaindb configure -``` - -The first question is ``API Server bind? (default `localhost:9984`)``. To expose the API to the public, bind the API Server to `0.0.0.0:9984`. Unless you have specific needs, you can keep the default value for all other questions. - -## Member: Generate the Private Key and Node id - -A Node is identified by the triplet ``. - -As a Member, it's your duty to create and store securely your private key, and share your `hostname`, `node_id`, and `public_key` with the other members of the network. - -To generate all of that, run: - -``` -tendermint init -``` - -The `public_key` is stored in the file `.tendermint/config/priv_validator.json`, and it should look like: +The Tendermint `pub_key.value` is stored +in the file `$HOME/.tendermint/config/priv_validator.json`. +That file should look like: ```json { @@ -150,7 +41,7 @@ The `public_key` is stored in the file `.tendermint/config/priv_validator.json`, } ``` -To extract your `node_id`, run the command: +To get your Tendermint `node_id`, run the command: ``` tendermint show_node_id @@ -158,15 +49,13 @@ tendermint show_node_id An example `node_id` is `9b989cd5ac65fec52652a457aed6f5fd200edc22`. -An example hostname is `charlie5.cloudservers.company.com`. You can also use a public IP addres, like `46.145.17.32`, instead of a hostname, but make sure that IP address won't change. +**Share your `hostname`, `pub_key.value` and `node_id` with all other Members.** -Share the `node_id`, `pub_key.value` and hostname of your Node with all other Members. +## Coordinator: Create & Share the genesis.json File -**Important note on security: each Member should take extra steps to verify the public keys they receive from the other Members have not been tampered with, e.g. a key signing party would be one way.** - -## Coordinator: Initialize the Network - -At this point the Coordinator should have received the data from all the Members, and should combine them in the `.tendermint/config/genesis.json` file: +At this point the Coordinator should have received the data +from all the Members, and should combine them in the file +`$HOME/.tendermint/config/genesis.json`: ```json { @@ -225,9 +114,12 @@ At this point the Coordinator should have received the data from all the Members } ``` -**Note:** `consensus_params` in the `genesis.json` are default values for Tendermint consensus. +**Note:** The above `consensus_params` in the `genesis.json` +are default values. -The new `genesis.json` file contains the data that describes the Network. The key `name` is the Member's moniker; it can be any valid string, but put something human-readable like `"Alice's Node Shop"`. +The new `genesis.json` file contains the data that describes the Network. +The key `name` is the Member's moniker; it can be any valid string, +but put something human-readable like `"Alice's Node Shop"`. At this point, the Coordinator must share the new `genesis.json` file with all Members. @@ -235,11 +127,13 @@ At this point, the Coordinator must share the new `genesis.json` file with all M At this point the Member should have received the `genesis.json` file. -**Important note on security: each Member should verify that the `genesis.json` file contains the correct public keys.** +The Member must copy the `genesis.json` file +into their local `$HOME/.tendermint/config` directory. +Every Member now shares the same `chain_id` and `genesis_time` (used to identify the Network), +and the same list of `validators`. -The Member must copy the `genesis.json` file in the local `.tendermint/config` directory. Every Member now shares the same `chain_id`, `genesis_time`, used to identify the Network, and the same list of `validators`. - -The Member must edit the `.tendermint/config/config.toml` file and make the following changes: +Each Member must edit their `$HOME/.tendermint/config/config.toml` file +and make the following changes: ``` moniker = "Name of our node" @@ -256,6 +150,9 @@ recv_rate = 102400000 recheck = false ``` +Note: The list of `persistent_peers` doesn't have to include all nodes +in the network. + ## Member: Start MongoDB If you installed MongoDB using `sudo apt install mongodb`, then MongoDB should already be running in the background. You can check using `systemctl status mongodb`. @@ -300,65 +197,7 @@ If you want to start and manage the BigchainDB and Tendermint processes yourself ## How Others Can Access Your Node -If you followed the above instructions, then your node should be publicly-accessible with BigchainDB Root URL `http://hostname:9984` (where hostname is something like `bdb7.canada.vmsareus.net` or `17.122.200.76`). That is, anyone can interact with your node using the [BigchainDB HTTP API](http-client-server-api.html) exposed at that address. The most common way to do that is to use one of the [BigchainDB Drivers](./drivers-clients/index.html). - -## Refreshing Your Node - -If you want to refresh your node back to a fresh empty state, then your best bet is to terminate it and deploy a new virtual machine, but if that's not an option, then you can: - -- drop the `bigchain` database in MongoDB using `bigchaindb drop` (but that only works if MongoDB is running) -- reset Tendermint using `tendermint unsafe_reset_all` -- delete the directory `$HOME/.tendermint` - -## Shutting down BigchainDB - -If you want to stop/kill BigchainDB, you can do so by sending `SIGINT`, `SIGQUIT` or `SIGTERM` to the running BigchainDB -process(es). Depending on how you started BigchainDB i.e. foreground or background. e.g. you started BigchainDB in the background as mentioned above in the guide: - -```bash -$ nohup bigchaindb start 2>&1 > bigchaindb.log & - -$ # Check the PID of the main BigchainDB process -$ ps -ef | grep bigchaindb - *