mirror of
https://github.com/bigchaindb/bigchaindb.git
synced 2024-10-13 13:34:05 +00:00
Docs on NTP daemon setup, RethinkDB config for cluster nodes
This commit is contained in:
troymc
parent
1670362500
commit
9a3fd04e40
@ -11,5 +11,6 @@ Appendices
|
||||
cryptography
|
||||
the-Bigchain-class
|
||||
consensus
|
||||
ntp-notes
|
||||
local-rethinkdb-cluster
|
||||
licenses
|
||||
41
docs/source/appendices/ntp-notes.md
Normal file
41
docs/source/appendices/ntp-notes.md
Normal file
@ -0,0 +1,41 @@
|
||||
# Notes on NTP Daemon Setup
|
||||
|
||||
As mentioned [elsewhere in these docs](../nodes/setup-run-node.html), there are several NTP daemons available, including:
|
||||
|
||||
* The reference NTP daemon from ntp.org; see [their support website](http://support.ntp.org/bin/view/Support/WebHome)
|
||||
* [OpenNTPD](http://www.openntpd.org/)
|
||||
* [chrony](https://chrony.tuxfamily.org/index.html)
|
||||
* Maybe [NTPsec](https://www.ntpsec.org/), once it's production-ready
|
||||
* Maybe [Ntimed](http://nwtime.org/projects/ntimed/), once it's production-ready
|
||||
* [More](https://en.wikipedia.org/wiki/Ntpd#Implementations)
|
||||
|
||||
It's tricky to make an NTP daemon setup secure. Always install the latest version and read the documentation about how to configure and run it securely.
|
||||
|
||||
|
||||
## Ubuntu Packages
|
||||
|
||||
The [Ubuntu 14.04 (Trusty Tahr) package `ntp`](https://launchpad.net/ubuntu/trusty/+source/ntp) is based on the reference implementation of an NTP daemon.
|
||||
|
||||
The following commands will uninstall the `ntp` and `ntpdate` packages, install the latest `ntp` package (which _might not be based on the latest ntpd code_), and start the NTP daemon (a local NTP server). (`ntpdate` is not reinstalled because it's [deprecated](https://askubuntu.com/questions/297560/ntpd-vs-ntpdate-pros-and-cons) and you shouldn't use it.)
|
||||
```text
|
||||
sudo apt-get --purge remove ntp ntpdate
|
||||
sudo apt-get autoremove
|
||||
sudo apt-get update
|
||||
sudo apt-get install ntp
|
||||
# That should start the NTP daemon too, but just to be sure:
|
||||
sudo service ntp restart
|
||||
```
|
||||
|
||||
You can check if NTP is running using `sudo ntpq -p`.
|
||||
|
||||
You may want to use different NTP time servers. You can change them by editing the NTP config file `/etc/ntp.conf`.
|
||||
|
||||
Note: A server running the NTP daemon can be used by others for DRDoS amplification attacks. The above installation procedure should install a default NTP configuration file `/etc/ntp.conf` with the lines:
|
||||
```text
|
||||
restrict -4 default kod notrap nomodify nopeer noquery
|
||||
restrict -6 default kod notrap nomodify nopeer noquery
|
||||
```
|
||||
|
||||
Those lines should prevent the NTP daemon from being used in an attack. (The first line is for IPv4, the second for IPv6.)
|
||||
|
||||
There are additional things you can do to make NTP more secure. See the [NTP Support Website](http://support.ntp.org/bin/view/Support/WebHome) for more details.
|
||||
@ -11,13 +11,33 @@ This section goes through the steps to set up a BigchainDB node (running Rethink
|
||||
The first step is to make sure you have a server (or equivalent) which meets [the requirements for a BigchainDB node](node-requirements.html).
|
||||
|
||||
|
||||
## Set Up the File System
|
||||
## System Clock Sync
|
||||
|
||||
This step isn't always necessary.
|
||||
If you're just setting up a stand-alone node, then you can skip this step.
|
||||
|
||||
See `def install_rethinkdb()` in `deploy-cluster-aws/fabfile.py` for an example.
|
||||
A BigchainDB node uses its system clock to generate timestamps for blocks and votes, so that clock should be kept in sync with some standard clock(s). The standard way to do that is to run an NTP daemon (Network Time Protocol daemon) on the node. (You could also use tlsdate, which uses TLS timestamps rather than NTP, but don't: it's not very accurate and it will break with TLS 1.3, which removes the timestamp.)
|
||||
|
||||
TODO: This step needs more explanation.
|
||||
NTP is a standard protocol. There are many NTP daemons implementing it. We don't recommend a particular one. On the contrary, we recommend that different nodes in a federation run different NTP daemons, so that a problem with one daemon won't affect all nodes. Some options are:
|
||||
|
||||
* The reference NTP daemon from ntp.org; see [their support website](http://support.ntp.org/bin/view/Support/WebHome)
|
||||
* [OpenNTPD](http://www.openntpd.org/)
|
||||
* [chrony](https://chrony.tuxfamily.org/index.html)
|
||||
* Maybe [NTPsec](https://www.ntpsec.org/), once it's production-ready
|
||||
* Maybe [Ntimed](http://nwtime.org/projects/ntimed/), once it's production-ready
|
||||
* [More](https://en.wikipedia.org/wiki/Ntpd#Implementations)
|
||||
|
||||
It's tricky to make an NTP daemon setup secure. Always install the latest version and read the documentation about how to configure and run it securely.
|
||||
|
||||
The appendix has [some notes on NTP daemon setup](../appendices/ntp-notes.html).
|
||||
|
||||
|
||||
## Set Up the File System for RethinkDB
|
||||
|
||||
If you're just setting up a stand-alone node, then you can probably skip this step.
|
||||
|
||||
See `def install_rethinkdb()` in `deploy-cluster-aws/fabfile.py` for an example of configuring a file system on an AWS instance running Ubuntu.
|
||||
|
||||
TODO: This section needs more elaboration
|
||||
|
||||
|
||||
## Install RethinkDB Server
|
||||
@ -33,19 +53,28 @@ If you're setting up a stand-alone node (i.e. not intending for it to join a clu
|
||||
|
||||
### Cluster Node
|
||||
|
||||
If you're setting up a node that will be part of a RethinkDB cluster, then you need to...
|
||||
If you're setting up a node that will be part of a RethinkDB cluster, then it needs to find out the hostnames of all the other nodes somehow. You _could_ designate one node to be the one that every other node asks, and put that node's hostname in the RethinkDB config file, but that wouldn't be very decentralized. Instead, we will list _every_ node in the list of nodes-to-ask:
|
||||
|
||||
1. Download the [sample RethinkDB config file from RethinkDB](https://github.com/rethinkdb/rethinkdb/blob/next/packaging/assets/config/default.conf.sample).
|
||||
2. Edit that file to add one line for each node in the federation, like so:
|
||||
```text
|
||||
join=node0_hostname:29015
|
||||
join=node1_hostname:29015
|
||||
join=node2_hostname:29015
|
||||
# continue until all federation node hostnames are included
|
||||
```
|
||||
|
||||
where the hostnames must be replaced by the hostnames of the RethinkDB servers in the cluster, e.g. `jackfish.servers.organization45.net`.
|
||||
|
||||
You may want to change other things in the RethinkDB config file as well. RethinkDB has [a documentation page with more info](https://www.rethinkdb.com/docs/config-file/).
|
||||
|
||||
TODO: (edit the default .conf file: add list of all hostnames in the federation)
|
||||
|
||||
TODO: Steps to make the RethinkDB cluster more secure.
|
||||
|
||||
|
||||
## Run RethinkDB Server
|
||||
|
||||
You can run RethinkDB by opening a Terminal and entering:
|
||||
```text
|
||||
$ rethinkdb
|
||||
```
|
||||
You can run RethinkDB by opening a Terminal and entering `rethinkdb`. You could do that now or wait until just before you start BigchainDB.
|
||||
|
||||
|
||||
## Install Python 3.4+
|
||||
@ -59,14 +88,14 @@ BigchainDB Server has some OS-level dependencies.
|
||||
|
||||
On Ubuntu 14.04, we found that the following was enough:
|
||||
```text
|
||||
$ sudo apt-get update
|
||||
$ sudo apt-get install g++ python3-dev
|
||||
sudo apt-get update
|
||||
sudo apt-get install g++ python3-dev
|
||||
```
|
||||
|
||||
On Fedora 23, we found that the following was enough (tested in February 2015):
|
||||
```text
|
||||
$ sudo dnf update
|
||||
$ sudo dnf install gcc-c++ redhat-rpm-config python3-devel
|
||||
sudo dnf update
|
||||
sudo dnf install gcc-c++ redhat-rpm-config python3-devel
|
||||
```
|
||||
|
||||
(If you're using a version of Fedora before version 22, you may have to use `yum` instead of `dnf`.)
|
||||
@ -78,14 +107,14 @@ With OS-level dependencies installed, you can install BigchainDB Server with `pi
|
||||
|
||||
BigchainDB (i.e. both the Server and the officially-supported drivers) is distributed as a Python package on PyPI so you can install it using `pip`. First, make sure you have a version of `pip` installed for Python 3.4+:
|
||||
```text
|
||||
$ pip -V
|
||||
pip -V
|
||||
```
|
||||
|
||||
If it says that `pip` isn't installed, or it says `pip` is associated with a Python version less than 3.4, then you must install a `pip` version associated with Python 3.4+. See [the `pip` installation instructions](https://pip.pypa.io/en/stable/installing/). On Ubuntu 14.04, we found that this works:
|
||||
```text
|
||||
$ sudo apt-get install python3-setuptools
|
||||
$ sudo easy_install3 pip
|
||||
$ pip3 install --upgrade pip wheel setuptools
|
||||
sudo apt-get install python3-setuptools
|
||||
sudo easy_install3 pip
|
||||
pip3 install --upgrade pip wheel setuptools
|
||||
```
|
||||
|
||||
(Note: Using `sudo apt-get python3-pip` also installs a Python 3 version of `pip` (named `pip3`) but we found it installed a very old version and there were issues with updating it.)
|
||||
@ -104,12 +133,12 @@ If you want to install BitchainDB from source because you want to contribute cod
|
||||
|
||||
Otherwise, clone the public repository:
|
||||
```text
|
||||
$ git clone git@github.com:bigchaindb/bigchaindb.git
|
||||
git clone git@github.com:bigchaindb/bigchaindb.git
|
||||
```
|
||||
|
||||
and then install from source:
|
||||
```text
|
||||
$ python setup.py install
|
||||
python setup.py install
|
||||
```
|
||||
|
||||
|
||||
@ -117,7 +146,7 @@ $ python setup.py install
|
||||
|
||||
Start by creating a default BigchainDB configuration file (named `.bigchaindb`) in your `$HOME` directory using:
|
||||
```text
|
||||
$ bigchaindb -y configure
|
||||
bigchaindb -y configure
|
||||
```
|
||||
|
||||
There's documentation for the `bigchaindb` command is in the section on [the BigchainDB Command Line Interface (CLI)](bigchaindb-cli.html).
|
||||
@ -140,12 +169,14 @@ For more information about the BigchainDB config file, see [Configuring a Bigcha
|
||||
|
||||
Once you've installed BigchainDB Server, you can run it. First make sure you have RethinkDB running:
|
||||
```text
|
||||
$ rethinkdb
|
||||
rethinkdb
|
||||
```
|
||||
|
||||
You can verify that RethinkDB is running by opening the RethinkDB web interface in your web browser. It should be at `http://rethinkdb-hostname:8080/`. If you're running RethinkDB on localhost, that would be [http://localhost:8080/](http://localhost:8080/).
|
||||
|
||||
You can start BigchainDB Server using:
|
||||
```text
|
||||
$ bigchaindb start
|
||||
bigchaindb start
|
||||
```
|
||||
|
||||
If it's the first time you've run `bigchaindb start`, then it creates the database (a RethinkDB database), the tables, the indexes, and the genesis block. It then starts BigchainDB. If you're run `bigchaindb start` or `bigchaindb init` before (and you haven't dropped the database), then `bigchaindb start` just starts BigchainDB.
|
||||
@ -173,7 +204,7 @@ the `-y` option to accept all the default values. The configuration file will
|
||||
be stored in a file on your host machine at `~/bigchaindb_docker/.bigchaindb`:
|
||||
|
||||
```text
|
||||
$ docker run --rm -v "$HOME/bigchaindb_docker:/data" -ti \
|
||||
docker run --rm -v "$HOME/bigchaindb_docker:/data" -ti \
|
||||
bigchaindb/bigchaindb -y configure
|
||||
Generating keypair
|
||||
Configuration written to /data/.bigchaindb
|
||||
@ -198,7 +229,7 @@ Let's analyze that command:
|
||||
After configuring the system, you can run BigchainDB with the following command:
|
||||
|
||||
```text
|
||||
$ docker run -v "$HOME/bigchaindb_docker:/data" -d \
|
||||
docker run -v "$HOME/bigchaindb_docker:/data" -d \
|
||||
--name bigchaindb \
|
||||
-p "58080:8080" -p "59984:9984" \
|
||||
bigchaindb/bigchaindb start
|
||||
@ -242,7 +273,7 @@ You should see a container named `bigchaindb` in the list.
|
||||
You can load test the BigchainDB running in that container by running the `bigchaindb load` command in a second container:
|
||||
|
||||
```text
|
||||
$ docker run --rm -v "$HOME/bigchaindb_docker:/data" -ti \
|
||||
docker run --rm -v "$HOME/bigchaindb_docker:/data" -ti \
|
||||
--link bigchaindb \
|
||||
bigchaindb/bigchaindb load
|
||||
```
|
||||
@ -253,7 +284,7 @@ Aside: The `bigchaindb load` command has several options (e.g. `-m`). You can re
|
||||
|
||||
If you look at the RethinkDB dashboard (in your web browser), you should see the effects of the load test. You can also see some effects in the Docker logs using:
|
||||
```text
|
||||
$ docker logs -f bigchaindb
|
||||
docker logs -f bigchaindb
|
||||
```
|
||||
|
||||
### Building Your Own Image
|
||||
@ -262,12 +293,12 @@ Assuming you have Docker installed, you would proceed as follows.
|
||||
|
||||
In a terminal shell:
|
||||
```text
|
||||
$ git clone git@github.com:bigchaindb/bigchaindb.git
|
||||
git clone git@github.com:bigchaindb/bigchaindb.git
|
||||
```
|
||||
|
||||
Build the Docker image:
|
||||
```text
|
||||
$ docker build --tag local-bigchaindb .
|
||||
docker build --tag local-bigchaindb .
|
||||
```
|
||||
|
||||
Now you can use your own image to run BigchainDB containers.
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user