Mirroristas/bigchaindb
2
0
Fork 0
mirror of https://github.com/bigchaindb/bigchaindb.git synced 2024-10-13 13:34:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

Problem: No automated way to set up a dev/test network (#2300)

Browse Source 
* Problem: No automated way to set up a dev/test network

* Problem: docs not updated for stack, ansible and docker based dev environments

* Problem: Using apt triggers java runtime installation on MacOS

- Update pre_tasks in ansible
- Update bigchaindb-stop playbook to handle pre_tasks

* Update Tendermint version to 0.19.3 from 0.19.2

* Update tendermint version to 0.19.3 everywhere else

* Problem: Sphinx warns about duplicate section labels

Solution: Don't use sphinx.ext.autosectionlabel

* Problem: Sphinx complains that run-node-with-docker not in any TOC

Solution: Add run-node-with-docker to a TOC (i.e. to an index.rst file)

* Tendermint has not tagged `0.19.3` container

* Problem: Internal hyperlinks to new pages not working

Solution: Add .html to the ends of the filename strings

* Problem: Invalid script imports

* Problem: Invalid comparison for supported OS version

* Addressing comments I

* Problem: No way to configure dev/forked repo for developers

* Problem: Docs not updated with STACK_REPO parameter

* Addressing comments II
This commit is contained in:
Muawia Khan
2018-05-31 16:56:45 +02:00
committed by GitHub
parent 0f56e4372d
commit 10e55aa4e7
86 changed files with 2235 additions and 1508 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
docs/server/source/appendices/index.rst
View File

@@ -5,7 +5,6 @@ Appendices
:maxdepth: 1
install-os-level-deps
run-with-docker
json-serialization
cryptography
the-Bigchain-class
@@ -17,5 +16,3 @@ Appendices
firewall-notes
ntp-notes
licenses
run-with-vagrant
run-with-ansible

167
docs/server/source/appendices/run-with-ansible.md
View File

@@ -1,167 +0,0 @@
# Run BigchainDB with Ansible
**NOT for Production Use**
You can use the following instructions to deploy a single or multi node
BigchainDB setup for dev/test using Ansible. Ansible will setup BigchainDB node(s) along with
[Docker](https://www.docker.com/), [Docker Compose](https://docs.docker.com/compose/),
[MongoDB](https://www.mongodb.com/), [BigchainDB Python driver](https://docs.bigchaindb.com/projects/py-driver/en/latest/).
Currently, this workflow is only supported for the following distributions:
- Ubuntu >= 16.04
- CentOS >= 7
- Fedora >= 24
## Minimum Requirements | Ansible
Minimum resource requirements for a single node BigchainDB dev setup. **The more the better**:
- Memory >= 512MB
- VCPUs >= 1
## Clone the BigchainDB repository | Ansible
```text
$ git clone https://github.com/bigchaindb/bigchaindb.git
```
## Install dependencies | Ansible
- [Ansible](http://docs.ansible.com/ansible/latest/intro_installation.html)
You can also install `ansible` and other dependencies, if any, using the `boostrap.sh` script
inside the BigchainDB repository.
Navigate to `bigchaindb/pkg/scripts` and run the `bootstrap.sh` script to install the dependecies
for your OS. The script also checks if the OS you are running is compatible with the
supported versions.
**Note**: `bootstrap.sh` only supports Ubuntu >= 16.04, CentOS >= 7 and Fedora >=24.
```text
$ cd bigchaindb/pkg/scripts/
$ sudo ./bootstrap.sh
```
### BigchainDB Setup Configuration(s) | Ansible
#### Local Setup | Ansible
You can run the Ansible playbook `bdb-deploy.yml` on your local dev machine and set up the BigchainDB node where
BigchainDB can be run as a process or inside a Docker container(s) depending on your configuration.
Before, running the playbook locally, you need to update the `hosts` and `bdb-config.yml` configuration, which will notify Ansible that we need to run the play locally.
##### Update Hosts | Local
Navigate to `bigchaindb/pkg/configuration/hosts` inside the BigchainDB repository.
```text
$ cd bigchaindb/pkg/configuration/hosts
```
Edit `all` configuration file:
```text
# Delete any existing configuration in this file and insert
# Hostname of dev machine
<HOSTNAME> ansible_connection=local
```
##### Update Configuration | Local
Navigate to `bigchaindb/pkg/configuration/vars` inside the BigchainDB repository.
```text
$ cd bigchaindb/pkg/configuration/vars/bdb-config.yml
```
Edit `bdb-config.yml` configuration file as per your requirements, sample configuration file(s):
```text
---
deploy_docker: false #[true, false]
docker_cluster_size: 1 # Only needed if `deploy_docker` is true
bdb_hosts:
- name: "<HOSTNAME>" # Hostname of dev machine
```
**Note**: You can also orchestrate a multi-node BigchainDB cluster on a local dev host using Docker containers.
Here is a sample `bdb-config.yml`
```text
---
deploy_docker: true #[true, false]
docker_cluster_size: 3
bdb_hosts:
- name: "<LOCAL_DEV_HOST_HOSTNAME>"
```
### BigchainDB Setup | Ansible
Now, You can safely run the `bdb-deploy.yml` playbook and everything will be taken care of by `Ansible`. To run the playbook please navigate to the `bigchaindb/pkg/configuration` directory inside the BigchainDB repository and run the `bdb-deploy.yml` playbook.
```text
$ cd bigchaindb/pkg/configuration/
$ sudo ansible-playbook bdb-deploy.yml -i hosts/all
```
After successful execution of the playbook, you can verify that BigchainDB docker(s)/process(es) is(are) running.
Verify BigchainDB process(es):
```text
$ ps -ef | grep bigchaindb
```
OR
Verify BigchainDB Docker(s):
```text
$ docker ps | grep bigchaindb
```
The playbook also installs the BigchainDB Python Driver,
so you can use it to make transactions
and verify the functionality of your BigchainDB node.
See the [BigchainDB Python Driver documentation](https://docs.bigchaindb.com/projects/py-driver/en/latest/index.html)
for details on how to use it.
**Note**: The `bdb_root_url` can be be one of the following:
```text
# BigchainDB is running as a process
bdb_root_url = http://<HOST-IP>:9984
OR
# BigchainDB is running inside a docker container
bdb_root_url = http://<HOST-IP>:<DOCKER-PUBLISHED-PORT>
```
**Note**: BigchainDB has [other drivers as well](../drivers-clients/index.html).
### Experimental: Running Ansible a Remote Dev/Host
#### Remote Setup | Ansible
You can also run the Ansible playbook `bdb-deploy.yml` on remote machine(s) and set up the BigchainDB node where
BigchainDB can run as a process or inside a Docker container(s) depending on your configuration.
Before, running the playbook on a remote host, you need to update the `hosts` and `bdb-config.yml` configuration, which will notify Ansible that we need to
run the play on a remote host.
##### Update Hosts | Remote
Navigate to `bigchaindb/pkg/configuration/hosts` inside the BigchainDB repository.
```text
$ cd bigchaindb/pkg/configuration/hosts
```
Edit `all` configuration file:
```text
# Delete any existing configuration in this file and insert
<Remote_Host_IP/Hostname> ansible_ssh_user=<USERNAME> ansible_sudo_pass=<ROOT_PASSWORD>
```
**Note**: You can add multiple hosts to the `all` configuration file. Root password is needed because ansible
will run some tasks that require root permissions.
**Note**: You can also use other methods to get inside the remote machines instead of password based SSH. For other methods
please consult [Ansible Documentation](http://docs.ansible.com/ansible/latest/intro_getting_started.html).
##### Update Configuration | Remote
Navigate to `bigchaindb/pkg/configuration/vars` inside the BigchainDB repository.
```text
$ cd bigchaindb/pkg/configuration/vars/bdb-config.yml
```
Edit `bdb-config.yml` configuration file as per your requirements, sample configuration file(s):
```text
---
deploy_docker: false #[true, false]
docker_cluster_size: 1 # Only needed if `deploy_docker` is true
bdb_hosts:
- name: "<REMOTE_MACHINE_HOSTNAME>"
```
After, the configuration of remote hosts, [run the Ansible playbook and verify your deployment](#bigchaindb-setup-ansible).

157
docs/server/source/appendices/run-with-docker.md
View File

@@ -1,157 +0,0 @@
# Run BigchainDB with Docker
**NOT for Production Use**
For those who like using Docker and wish to experiment with BigchainDB in
non-production environments, we currently maintain a Docker image and a
`Dockerfile` that can be used to build an image for `bigchaindb`.
## Prerequisite(s)
- [Docker](https://docs.docker.com/engine/installation/)
## Pull and Run the Image from Docker Hub
With Docker installed, you can proceed as follows.
In a terminal shell, pull the latest version of the BigchainDB Docker image using:
```text
docker pull bigchaindb/bigchaindb
```
### Configuration
A one-time configuration step is required to create the config file; we will use
the `-y` option to accept all the default values. The configuration file will
be stored in a file on your host machine at `~/bigchaindb_docker/.bigchaindb`:
```text
docker run \
--interactive \
--rm \
--tty \
--volume $HOME/bigchaindb_docker:/data \
--env BIGCHAINDB_DATABASE_HOST=172.17.0.1 \
bigchaindb/bigchaindb \
-y configure \
[mongodb|rethinkdb]
Generating keypair
Configuration written to /data/.bigchaindb
Ready to go!
```
Let's analyze that command:
* `docker run` tells Docker to run some image
* `--interactive` keep STDIN open even if not attached
* `--rm` remove the container once we are done
* `--tty` allocate a pseudo-TTY
* `--volume "$HOME/bigchaindb_docker:/data"` map the host directory
`$HOME/bigchaindb_docker` to the container directory `/data`;
this allows us to have the data persisted on the host machine,
you can read more in the [official Docker
documentation](https://docs.docker.com/engine/tutorials/dockervolumes)
* `--env BIGCHAINDB_DATABASE_HOST=172.17.0.1`, `172.17.0.1` is the default `docker0` bridge
IP address, for fresh Docker installations. It is used for the communication between BigchainDB and database
containers.
* `bigchaindb/bigchaindb` the image to use. All the options after the container name are passed on to the entrypoint inside the container.
* `-y configure` execute the `configure` sub-command (of the `bigchaindb`
command) inside the container, with the `-y` option to automatically use all the default config values
* `mongodb` or `rethinkdb` specifies the database backend to use with bigchaindb
### Run the backend database
From v0.9 onwards, you can run either RethinkDB or MongoDB.
You can also use docker host networking or bind to your primary (eth)
interface, if needed.
#### For RethinkDB
```text
docker run \
--detach \
--name=rethinkdb \
--publish=28015:28015 \
--publish=58080:8080 \
--restart=always \
--volume $HOME/bigchaindb_docker:/data \
rethinkdb:2.3
```
<!-- Don't hyperlink http://172.17.0.1:58080/ because Sphinx will fail when you do "make linkcheck" -->
You can also access the RethinkDB dashboard at http://172.17.0.1:58080/
#### For MongoDB
Note: MongoDB runs as user `mongodb` which had the UID `999` and GID `999`
inside the container. For the volume to be mounted properly, as user `mongodb`
in your host, you should have a `mongodb` user with UID and GID `999`.
If you have another user on the host with UID `999`, the mapped files will
be owned by this user in the host.
If there is no owner with UID 999, you can create the corresponding user and
group.
`useradd -r --uid 999 mongodb` OR `groupadd -r --gid 999 mongodb && useradd -r --uid 999 -g mongodb mongodb` should work.
```text
docker run \
--detach \
--name=mongodb \
--publish=27017:27017 \
--restart=always \
--volume=$HOME/mongodb_docker/db:/data/db \
--volume=$HOME/mongodb_docker/configdb:/data/configdb \
mongo:3.4.9 --replSet=bigchain-rs
```
### Run BigchainDB
```text
docker run \
--detach \
--name=bigchaindb \
--publish=59984:9984 \
--restart=always \
--volume=$HOME/bigchaindb_docker:/data \
bigchaindb/bigchaindb \
start
```
The command is slightly different from the previous one, the differences are:
* `--detach` run the container in the background
* `--name bigchaindb` give a nice name to the container so it's easier to
refer to it later
* `--publish "59984:9984"` map the host port `59984` to the container port `9984`
(the BigchainDB API server)
* `start` start the BigchainDB service
Another way to publish the ports exposed by the container is to use the `-P` (or
`--publish-all`) option. This will publish all exposed ports to random ports. You can
always run `docker ps` to check the random mapping.
If that doesn't work, then replace `localhost` with the IP or hostname of the
machine running the Docker engine. If you are running docker-machine (e.g. on
Mac OS X) this will be the IP of the Docker machine (`docker-machine ip
machine_name`).
## Building Your Own Image
Assuming you have Docker installed, you would proceed as follows.
In a terminal shell:
```text
git clone git@github.com:bigchaindb/bigchaindb.git
```
Build the Docker image:
```text
docker build --tag local-bigchaindb .
```
Now you can use your own image to run BigchainDB containers.

170
docs/server/source/appendices/run-with-vagrant.md
View File

@@ -1,170 +0,0 @@
# Run BigchainDB with Vagrant
**NOT for Production Use**
You can use the following instructions to deploy a single or multi node
BigchainDB setup for dev/test using Vagrant. Vagrant will set up the BigchainDB node(s)
with all the dependencies along with MongoDB and BigchainDB Python driver. You
can also tweak the following configurations for the BigchainDB node(s).
- Vagrant Box
- Currently, we support the following boxes:
- `ubuntu/xenial64 # >=16.04`
- `centos/7 # >=7`
- `fedora/24 # >=24`
- **NOTE** : You can choose any other vagrant box of your choice but these are
the minimum versioning requirements.
- Resources and specs for your box.
- RAM
- VCPUs
- Network Type
- Currently, only `private_network` is supported.
- IP Address
- Deploy node with Docker
- Deploy all the services in Docker containers or as processes.
- Number of BigchainDB nodes
- If you want to deploy the services inside Docker containers, you
can specify number of member(s) in the BigchainDB cluster.
- Upstart Script
- Vagrant Provider
- Virtualbox
- VMware
## Minimum Requirements | Vagrant
Minimum resource requirements for a single node BigchainDB dev setup. **The more the better**:
- Memory >= 512MB
- VCPUs >= 1
## Install dependencies | Vagrant
1. [VirtualBox](https://www.virtualbox.org/wiki/Downloads) >= 5.0.0
2. [Vagrant](https://www.vagrantup.com/downloads.html) >= 1.16.0
## Clone the BigchainDB repository | Vagrant
```text
$ git clone https://github.com/bigchaindb/bigchaindb.git
```
## Configuration | Vagrant
Navigate to `bigchaindb/pkg/configuration/vars/` inside the BigchainDB repository.
```text
$ cd bigchaindb/pkg/configuration/vars/
```
Edit `bdb-config.yml` as per your requirements. Sample `bdb-config.yml`:
```text
---
deploy_docker: false #[true, false]
docker_cluster_size: 1
upstart: "/bigchaindb/scripts/bootstrap.sh"
bdb_hosts:
- name: "bdb-node-01"
box:
name: "ubuntu/xenial64"
ram: "2048"
vcpus: "2"
network:
ip: "10.20.30.40"
type: "private_network"
```
**Note**: You can spawn multiple instances to orchestrate a multi-node BigchainDB cluster.
Here is a sample `bdb-config.yml`:
```text
---
deploy_docker: false #[true, false]
docker_cluster_size: 1
upstart: "/bigchaindb/scripts/bootstrap.sh"
bdb_hosts:
- name: "bdb-node-01"
box:
name: "ubuntu/xenial64"
ram: "2048"
vcpus: "2"
network:
ip: "10.20.30.40"
type: "private_network"
- name: "bdb-node-02"
box:
name: "ubuntu/xenial64"
ram: "2048"
vcpus: "2"
network:
ip: "10.20.30.50"
type: "private_network"
```
**Note**: You can also orchestrate a multi-node BigchainDB cluster on a single dev host using Docker containers.
Here is a sample `bdb-config.yml`
```text
---
deploy_docker: true #[true, false]
docker_cluster_size: 3
upstart: "/bigchaindb/scripts/bootstrap.sh"
bdb_hosts:
- name: "bdb-node-01"
box:
name: "ubuntu/xenial64"
ram: "8192"
vcpus: "4"
network:
ip: "10.20.30.40"
type: "private_network"
```
The above mentioned configuration will deploy a 3 node BigchainDB cluster with Docker containers
on your specified host.
## BigchainDB Setup | Vagrant
**Note**: There are some vagrant plugins required for the installation,
user will be prompted to install them if they are not present. To install
the required plugins, run the following command:
```text
$ vagrant plugin install vagrant-cachier vagrant-vbguest vagrant-hosts
```
To bring up the BigchainDB node(s), run the following command:
```text
$ vagrant up
```
After successful execution of Vagrant, you can log in to your fresh BigchainDB node.
```text
$ vagrant ssh <instance-name>
```
## Make your first transaction
Once you are inside the BigchainDB node, you can verify that BigchainDB
docker(s)/process(es) is(are) running.
Verify BigchainDB process(es):
```text
$ ps -ef | grep bigchaindb
```
OR
Verify BigchainDB Docker(s):
```text
$ docker ps | grep bigchaindb
```
The BigchainDB Python Driver is pre-installed in the instance,
so you can use it to make transactions
and verify the functionality of your BigchainDB node.
See the [BigchainDB Python Driver documentation](https://docs.bigchaindb.com/projects/py-driver/en/latest/index.html)
for details on how to use it.
Note 1: The `bdb_root_url` can be one of the following:
```text
# BigchainDB is running as a process
bdb_root_url = http://<HOST-IP>:9984
OR
# BigchainDB is running inside a docker container
bdb_root_url = http://<HOST-IP>:<DOCKER-PUBLISHED-HOST-PORT>
```
Note 2: BigchainDB has [other drivers as well](../drivers-clients/index.html).