Problem: standardize docker-compose workflows (#2130)

- Standardize docker-compose workflow
- Change docker-compose version to 2.1
  - why one might ask? because compose version
    3.0 does not support depends on and inherits
    like we want to and is more aimed towards migration to using
    `docker stack`, for our current strategy `2.1` is a better choice.
- change `bdb` service `bigchaindb` service
  - why? Introduced a new proxy service `bdb` which is just a dummy
    `busybox` image.
    - why? because this ensure via healthcheck of bigchaindb that BigchainDB
      has started properly and makes a `curl` to ensure HTTP API server is up
      and running.
      - why? Because we have had scenarios where BigchainDB is not started
        via docker compose and user has to check out the logs to find out what
        the problem might be. This ensure that bigchaindb is up and running.
- Does this change deployment workflow? No.
  - The only thing change is that if you want to run commands inside a bigchaindb
    container e.g. `pytest` now you have to run the following command:
    `docker-compose run --rm --no-deps bigchaindb pytest -v --cov=bigchaindb`
     as opposed to `docker-compose run --rm --no-deps bdb pytest -v --cov=bigchaindb`
- Remove env variable `BIGCHAINDB_START_TENDERMINT`
- Remove TENDERMINT_INTEGRATION.rst and move to the new docs
- Change mdb -> mongodb because the other services were named with
  full name.
- Add example to run specific tests or from a file
- Update config.toml for tendermint to use `bigchaindb` as proxy app
  instead of `bdb`
- Remove `network` directory because it is deprecated
- Add comment about why PYTHONBUFFERED is used
This commit is contained in:
Ahmed Muawia Khan 2018-03-21 12:42:43 +01:00 committed by vrde
parent ef26220c7f
commit 9be68c972d
41 changed files with 196 additions and 921 deletions

View File

@ -3,5 +3,5 @@
set -e -x
if [[ -z ${TOXENV} ]]; then
docker-compose -f docker-compose.travis.yml up -d bdb
docker-compose up -d bdb
fi

View File

@ -7,6 +7,6 @@ pip install --upgrade pip
if [[ -n ${TOXENV} ]]; then
pip install --upgrade tox
else
docker-compose -f docker-compose.travis.yml build --no-cache
docker-compose build --no-cache bigchaindb
pip install --upgrade codecov
fi

View File

@ -5,5 +5,5 @@ set -e -x
if [[ -n ${TOXENV} ]]; then
tox -e ${TOXENV}
else
docker-compose -f docker-compose.travis.yml run --rm --no-deps bdb pytest -v --cov=bigchaindb
docker-compose run --rm --no-deps bigchaindb pytest -v --cov=bigchaindb
fi

View File

@ -19,6 +19,5 @@ ENV BIGCHAINDB_WSSERVER_SCHEME ws
ENV BIGCHAINDB_WSSERVER_ADVERTISED_HOST 0.0.0.0
ENV BIGCHAINDB_WSSERVER_ADVERTISED_SCHEME ws
ENV BIGCHAINDB_WSSERVER_ADVERTISED_PORT 9985
ENV BIGCHAINDB_START_TENDERMINT 0
ENTRYPOINT ["bigchaindb"]
CMD ["start"]

View File

@ -8,6 +8,14 @@ RUN apt-get update \
&& apt-get autoremove \
&& apt-get clean
ARG backend
# When developing with Python in a docker container, we are using PYTHONBUFFERED
# to force stdin, stdout and stderr to be totally unbuffered and to capture logs/outputs
ENV PYTHONUNBUFFERED 0
ENV BIGCHAINDB_DATABASE_PORT 27017
ENV BIGCHAINDB_DATABASE_BACKEND $backend
ENV BIGCHAINDB_SERVER_BIND 0.0.0.0:9984
ENV BIGCHAINDB_WSSERVER_HOST 0.0.0.0
ENV BIGCHAINDB_WSSERVER_SCHEME ws
@ -15,10 +23,11 @@ ENV BIGCHAINDB_WSSERVER_SCHEME ws
ENV BIGCHAINDB_WSSERVER_ADVERTISED_HOST 0.0.0.0
ENV BIGCHAINDB_WSSERVER_ADVERTISED_SCHEME ws
ARG backend
ENV BIGCHAINDB_TENDERMINT_PORT 46657
RUN mkdir -p /usr/src/app
COPY . /usr/src/app/
WORKDIR /usr/src/app
RUN pip install --no-cache-dir -e .[dev]
RUN bigchaindb -y configure "$backend"

View File

@ -1,55 +0,0 @@
**********************
Tendermint Integration
**********************
Quick reference for developers working on the Tendermint integration in
BigchainDB.
Running a single node with ``docker-compose``
=============================================
.. code-block:: bash
$ docker-compose -f docker-compose.tendermint.yml up bdb
The above command will launch all 3 main required services/processes:
* ``mongodb``
* ``tendermint``
* ``bigchaindb``
To follow the logs of the ``tendermint`` service:
.. code-block:: bash
$ docker-compose -f docker-compose.tendermint.yml logs -f tendermint
Simple health check:
.. code-block:: bash
$ docker-compose -f docker-compose.tendermint.yml up curl-client
Post and retrieve a transaction -- copy/paste a driver basic example of a
``CREATE`` transaction:
.. code-block:: bash
$ docker-compose -f docker-compose.tendermint.yml run --rm driver ipython
.. todo:: small python script to post and retrieve a transaction.
Running a 4-node cluster with ``docker-compose``
================================================
.. code-block:: bash
$ docker-compose -f docker-compose.network.yml up -d bdb-one bdb-two bdb-three bdb-four
Simple health check:
.. code-block:: bash
$ docker-compose -f docker-compose.network.yml up curl-client

View File

@ -1,32 +0,0 @@
FROM python:3.6
LABEL maintainer "dev@bigchaindb.com"
RUN apt-get update \
&& apt-get install -y vim \
&& pip install -U pip \
&& pip install pynacl \
&& apt-get autoremove \
&& apt-get clean
ARG backend
ENV PYTHONUNBUFFERED 0
ENV BIGCHAINDB_DATABASE_PORT 27017
ENV BIGCHAINDB_DATABASE_BACKEND $backend
ENV BIGCHAINDB_SERVER_BIND 0.0.0.0:9984
ENV BIGCHAINDB_WSSERVER_HOST 0.0.0.0
ENV BIGCHAINDB_WSSERVER_SCHEME ws
ENV BIGCHAINDB_WSSERVER_ADVERTISED_HOST 0.0.0.0
ENV BIGCHAINDB_WSSERVER_ADVERTISED_SCHEME ws
ENV BIGCHAINDB_START_TENDERMINT 0
ENV BIGCHAINDB_TENDERMINT_PORT 46657
RUN mkdir -p /usr/src/app
COPY . /usr/src/app/
WORKDIR /usr/src/app
RUN pip install --no-cache-dir -e .[dev]
RUN bigchaindb -y configure "$backend"

View File

@ -1,30 +0,0 @@
FROM python:3.6
LABEL maintainer "dev@bigchaindb.com"
RUN apt-get update \
&& pip install -U pip \
&& apt-get autoremove \
&& apt-get clean
ARG backend
ENV PYTHONUNBUFFERED 0
ENV BIGCHAINDB_DATABASE_PORT 27017
ENV BIGCHAINDB_DATABASE_BACKEND $backend
ENV BIGCHAINDB_SERVER_BIND 0.0.0.0:9984
ENV BIGCHAINDB_WSSERVER_HOST 0.0.0.0
ENV BIGCHAINDB_WSSERVER_SCHEME ws
ENV BIGCHAINDB_WSSERVER_ADVERTISED_HOST 0.0.0.0
ENV BIGCHAINDB_WSSERVER_ADVERTISED_SCHEME ws
ENV BIGCHAINDB_START_TENDERMINT 0
ENV BIGCHAINDB_TENDERMINT_PORT 46657
RUN mkdir -p /usr/src/app
COPY . /usr/src/app/
WORKDIR /usr/src/app
RUN find . | grep -E "(__pycache__|\.pyc|\.pyo$)" | xargs rm -rf
RUN pip install --no-cache-dir .[test]
RUN bigchaindb -y configure "$backend"

View File

@ -1,27 +0,0 @@
version: '2'
services:
bdb:
build:
context: .
dockerfile: Dockerfile-dev
args:
backend: mongodb
volumes:
- ./bigchaindb:/usr/src/app/bigchaindb
- ./tests:/usr/src/app/tests
- ./docs:/usr/src/app/docs
- ./k8s:/usr/src/app/k8s
- ./setup.py:/usr/src/app/setup.py
- ./setup.cfg:/usr/src/app/setup.cfg
- ./pytest.ini:/usr/src/app/pytest.ini
- ./tox.ini:/usr/src/app/tox.ini
- ./scripts:/usr/src/app/scripts
environment:
BIGCHAINDB_DATABASE_BACKEND: mongodb
BIGCHAINDB_DATABASE_HOST: mdb
BIGCHAINDB_DATABASE_PORT: 27017
BIGCHAINDB_SERVER_BIND: 0.0.0.0:9984
ports:
- "9984"
command: bigchaindb start

View File

@ -1,18 +0,0 @@
version: '3'
services:
bdocs:
build:
context: .
dockerfile: ./compose/bigchaindb-server/Dockerfile
args:
backend: localmongodb
volumes:
- .:/usr/src/app/
command: make -C docs/server html
vdocs:
image: nginx
ports:
- '33333:80'
volumes:
- ./docs/server/build/html:/usr/share/nginx/html

View File

@ -1,170 +0,0 @@
version: '3'
services:
#############################################################################
# #
# NODE 1 #
# #
#############################################################################
mdb-one:
image: mongo:3.4.3
ports:
- "27017"
command: mongod
bdb-one:
depends_on:
- mdb-one
- tendermint-one
build:
context: .
dockerfile: ./compose/bigchaindb-server/Dockerfile
args:
backend: localmongodb
volumes:
- ./bigchaindb:/usr/src/app/bigchaindb
- ./tests:/usr/src/app/tests
environment:
BIGCHAINDB_DATABASE_BACKEND: localmongodb
BIGCHAINDB_DATABASE_HOST: mdb-one
BIGCHAINDB_TENDERMINT_HOST: tendermint-one
ports:
- "9984"
command: bigchaindb -l DEBUG start
tendermint-one:
image: tendermint/tendermint
volumes:
- ./network/node1:/tendermint
entrypoint: ''
command: bash -c "tendermint unsafe_reset_all && tendermint --log_level debug node"
#############################################################################
# #
# NODE 2 #
# #
#############################################################################
mdb-two:
image: mongo:3.4.3
ports:
- "27017"
command: mongod
bdb-two:
depends_on:
- mdb-two
- tendermint-two
build:
context: .
dockerfile: ./compose/bigchaindb-server/Dockerfile
args:
backend: localmongodb
volumes:
- ./bigchaindb:/usr/src/app/bigchaindb
- ./tests:/usr/src/app/tests
environment:
BIGCHAINDB_DATABASE_BACKEND: localmongodb
BIGCHAINDB_DATABASE_HOST: mdb-two
BIGCHAINDB_TENDERMINT_HOST: tendermint-two
ports:
- "9984"
command: bigchaindb -l DEBUG start
tendermint-two:
image: tendermint/tendermint
volumes:
- ./network/node2:/tendermint
entrypoint: ''
command: bash -c "tendermint unsafe_reset_all && tendermint --log_level debug node"
#############################################################################
# #
# NODE 3 #
# #
#############################################################################
mdb-three:
image: mongo:3.4.3
ports:
- "27017"
command: mongod
bdb-three:
depends_on:
- mdb-three
- tendermint-three
build:
context: .
dockerfile: ./compose/bigchaindb-server/Dockerfile
args:
backend: localmongodb
volumes:
- ./bigchaindb:/usr/src/app/bigchaindb
- ./tests:/usr/src/app/tests
environment:
BIGCHAINDB_DATABASE_BACKEND: localmongodb
BIGCHAINDB_DATABASE_HOST: mdb-three
BIGCHAINDB_TENDERMINT_HOST: tendermint-three
ports:
- "9984"
command: bigchaindb -l DEBUG start
tendermint-three:
image: tendermint/tendermint
volumes:
- ./network/node3:/tendermint
entrypoint: ''
command: bash -c "tendermint unsafe_reset_all && tendermint --log_level debug node"
#############################################################################
# #
# NODE 4 #
# #
#############################################################################
mdb-four:
image: mongo:3.4.3
ports:
- "27017"
command: mongod
bdb-four:
depends_on:
- mdb-four
- tendermint-four
build:
context: .
dockerfile: ./compose/bigchaindb-server/Dockerfile
args:
backend: localmongodb
volumes:
- ./bigchaindb:/usr/src/app/bigchaindb
- ./tests:/usr/src/app/tests
environment:
BIGCHAINDB_DATABASE_BACKEND: localmongodb
BIGCHAINDB_DATABASE_HOST: mdb-four
BIGCHAINDB_TENDERMINT_HOST: tendermint-four
ports:
- "9984"
command: bigchaindb -l DEBUG start
tendermint-four:
image: tendermint/tendermint
volumes:
- ./network/node4:/tendermint
entrypoint: ''
command: bash -c "tendermint unsafe_reset_all && tendermint --log_level debug node"
#############################################################################
#############################################################################
#############################################################################
#
# clients
#
#############################################################################
#############################################################################
#############################################################################
curl-client:
image: appropriate/curl
volumes:
- ./network/health-check.sh:/health-check.sh
command: /bin/sh health-check.sh
driver:
build:
context: .
dockerfile: ./compose/bigchaindb-driver/Dockerfile

View File

@ -1,48 +0,0 @@
version: '2'
services:
rdb:
image: rethinkdb
ports:
- "58080:8080"
- "28015"
volumes_from:
- rdb-data
rdb-2:
image: rethinkdb
ports:
- "8080"
- "29015"
command: rethinkdb --join rdb:29015 --bind all
rdb-data:
image: rethinkdb:2.3.5
volumes:
- /data
command: "true"
bdb-rdb:
build:
context: .
dockerfile: Dockerfile-dev
args:
backend: rethinkdb
container_name: docker-bigchaindb
volumes:
- ./bigchaindb:/usr/src/app/bigchaindb
- ./tests:/usr/src/app/tests
- ./docs:/usr/src/app/docs
- ./k8s:/usr/src/app/k8s
- ./setup.py:/usr/src/app/setup.py
- ./setup.cfg:/usr/src/app/setup.cfg
- ./pytest.ini:/usr/src/app/pytest.ini
- ./tox.ini:/usr/src/app/tox.ini
- ./Makefile:/usr/src/app/Makefile
environment:
BIGCHAINDB_DATABASE_BACKEND: rethinkdb
BIGCHAINDB_DATABASE_HOST: rdb
BIGCHAINDB_SERVER_BIND: 0.0.0.0:9984
ports:
- "9984"
command: bigchaindb start

View File

@ -1,26 +0,0 @@
version: '2'
services:
mdb:
image: mongo:3.4.3
volumes:
- ./tests/backend/mongodb-ssl/certs/mongodb.pem:/etc/ssl/private/mongodb.pem
- ./tests/backend/mongodb-ssl/certs/ca-chain.cert.pem:/etc/ssl/ca-chain.pem
- ./tests/backend/mongodb-ssl/certs/crl.pem:/etc/ssl/crl.pem
command: mongod --replSet=bigchain-rs --sslMode=requireSSL --sslCAFile=/etc/ssl/ca-chain.pem --sslCRLFile=/etc/ssl/crl.pem --sslPEMKeyFile=/etc/ssl/private/mongodb.pem
bdb:
environment:
BIGCHAINDB_DATABASE_SSL: 1
BIGCHAINDB_DATABASE_CA_CERT: /etc/ssl/ca-chain.pem
BIGCHAINDB_DATABASE_CRLFILE: /etc/ssl/crl.pem
BIGCHAINDB_DATABASE_CERTFILE: /etc/ssl/certs/bigchaindb.cert.pem
BIGCHAINDB_DATABASE_KEYFILE: /etc/ssl/private/bigchaindb.key.pem
BIGCHAINDB_MDB_PEM_KEY_TEST: /etc/ssl/private/
volumes:
- ./tests/backend/mongodb-ssl/ca:/root/ca
- ./tests/backend/mongodb-ssl/certs/ca-chain.cert.pem:/etc/ssl/ca-chain.pem
- ./tests/backend/mongodb-ssl/certs/bigchaindb.cert.pem:/etc/ssl/certs/bigchaindb.cert.pem
- ./tests/backend/mongodb-ssl/certs/bigchaindb.key.pem:/etc/ssl/private/bigchaindb.key.pem
- ./tests/backend/mongodb-ssl/certs/crl.pem:/etc/ssl/crl.pem
- ./tests/backend/mongodb-ssl/certs/mongodb.pem:/etc/ssl/private/mongodb.pem

View File

@ -1,54 +0,0 @@
version: '3'
services:
mdb:
image: mongo:3.4.3
ports:
- "27017"
command: mongod
bdb:
depends_on:
- mdb
- tendermint
build:
context: .
dockerfile: ./compose/bigchaindb-server/Dockerfile
args:
backend: localmongodb
volumes:
- ./bigchaindb:/usr/src/app/bigchaindb
- ./tests:/usr/src/app/tests
- ./docs:/usr/src/app/docs
- ./setup.py:/usr/src/app/setup.py
- ./setup.cfg:/usr/src/app/setup.cfg
- ./pytest.ini:/usr/src/app/pytest.ini
- ./tox.ini:/usr/src/app/tox.ini
environment:
BIGCHAINDB_DATABASE_BACKEND: localmongodb
BIGCHAINDB_DATABASE_HOST: mdb
BIGCHAINDB_DATABASE_PORT: 27017
BIGCHAINDB_SERVER_BIND: 0.0.0.0:9984
BIGCHAINDB_WSSERVER_HOST: 0.0.0.0
BIGCHAINDB_START_TENDERMINT: 0
BIGCHAINDB_TENDERMINT_HOST: tendermint
BIGCHAINDB_TENDERMINT_PORT: 46657
ports:
- "9984"
- "46658"
command: bigchaindb -l DEBUG start
tendermint:
image: tendermint/tendermint:0.13
volumes:
- ./tmdata/config.toml:/tendermint/config.toml
entrypoint: ''
ports:
- "46656"
- "46657"
command: bash -c "tendermint init && tendermint node"
curl-client:
image: appropriate/curl
command: /bin/sh -c "curl http://tendermint:46657/abci_query && curl http://bdb:9984/"
driver:
build:
context: .
dockerfile: ./compose/bigchaindb-driver/Dockerfile

View File

@ -1,34 +0,0 @@
version: '3'
services:
mdb:
image: mongo:3.4.3
command: mongod
bdb:
depends_on:
- mdb
- tendermint
build:
context: .
dockerfile: ./compose/travis/Dockerfile
args:
backend: localmongodb
volumes:
- .:/usr/src/app/
environment:
BIGCHAINDB_DATABASE_BACKEND: localmongodb
BIGCHAINDB_DATABASE_HOST: mdb
BIGCHAINDB_DATABASE_PORT: 27017
BIGCHAINDB_SERVER_BIND: 0.0.0.0:9984
BIGCHAINDB_WSSERVER_HOST: 0.0.0.0
BIGCHAINDB_START_TENDERMINT: 0
BIGCHAINDB_TENDERMINT_HOST: tendermint
BIGCHAINDB_TENDERMINT_PORT: 46657
command: bigchaindb start
tendermint:
image: tendermint/tendermint:0.13
volumes:
- ./tmdata/config.toml:/tendermint/config.toml
- ./tmdata/genesis.json:/tendermint/genesis.json
- ./tmdata/priv_validator.json:/tendermint/priv_validator.json
entrypoint: ["/bin/tendermint", "node", "--proxy_app=dummy"]

View File

@ -1,33 +1,93 @@
version: '2'
version: '2.1'
services:
mdb:
image: mongo:3.4.3
# Build: docker-compose build -d bigchaindb
# Run: docker-compose run -d bdb
mongodb:
image: mongo:3.4.13
ports:
- "27017"
command: mongod
bdb:
bigchaindb:
depends_on:
- mongodb
- tendermint
build:
context: .
dockerfile: Dockerfile-dev
args:
backend: mongodb
backend: localmongodb
volumes:
- ./bigchaindb:/usr/src/app/bigchaindb
- ./tests:/usr/src/app/tests
- ./docs:/usr/src/app/docs
- ./k8s:/usr/src/app/k8s
- ./setup.py:/usr/src/app/setup.py
- ./setup.cfg:/usr/src/app/setup.cfg
- ./pytest.ini:/usr/src/app/pytest.ini
- ./tox.ini:/usr/src/app/tox.ini
environment:
BIGCHAINDB_DATABASE_BACKEND: localmongodb
BIGCHAINDB_DATABASE_HOST: mdb
BIGCHAINDB_DATABASE_HOST: mongodb
BIGCHAINDB_DATABASE_PORT: 27017
BIGCHAINDB_SERVER_BIND: 0.0.0.0:9984
BIGCHAINDB_WSSERVER_HOST: 0.0.0.0
BIGCHAINDB_START_TENDERMINT: 0
BIGCHAINDB_TENDERMINT_HOST: tendermint
BIGCHAINDB_TENDERMINT_PORT: 46657
ports:
- "9984"
command: bigchaindb start
- "9985"
- "46658"
healthcheck:
test: ["CMD", "bash", "-c", "curl http://bigchaindb:9984 && curl http://tendermint:46657/abci_query"]
interval: 3s
timeout: 5s
retries: 3
command: bigchaindb -l DEBUG start
tendermint:
image: tendermint/tendermint:0.12
volumes:
- ./tmdata/config.toml:/tendermint/config.toml
entrypoint: ''
ports:
- "46656"
- "46657"
command: bash -c "tendermint init && tendermint node"
bdb:
image: busybox
depends_on:
bigchaindb:
condition: service_healthy
# curl client to check the health of development env
curl-client:
image: appropriate/curl
command: /bin/sh -c "curl http://tendermint:46657/abci_query && curl http://bigchaindb:9984/"
# Python BigchainDB driver to interact with the BigchainDB server
bdb-driver:
build:
context: .
dockerfile: ./tools/py-bigchaindb-driver/Dockerfile
# Build docs only
# docker-compose build bdocs
# docker-compose up -d bdocs
bdocs:
depends_on:
- vdocs
build:
context: .
dockerfile: Dockerfile-dev
args:
backend: localmongodb
volumes:
- .:/usr/src/app/
command: make -C docs/server html
vdocs:
image: nginx
ports:
- '33333:80'
volumes:
- ./docs/server/build/html:/usr/share/nginx/html

View File

@ -30,3 +30,15 @@ make html
```
It should tell you where the generated documentation (HTML files) can be found. You can view it in your web browser.
# Building Docs with Docker Compose
You can also use [Docker Compose](https://docs.docker.com/compose/) to build and host docs.
```text
$ docker-compose up -d bdocs
```
The docs will be hosted on port **33333**, and can be accessed over [localhost](http:/localhost:33333), [127.0.0.1](http:/127.0.0.1:33333)
OR http:/HOST_IP:33333.

View File

@ -8,6 +8,7 @@ Ways to Contribute
issue-rules
write-a-bep
setup-dev-environment
write-code
vanshdeep-notes
write-docs

View File

@ -0,0 +1,80 @@
# Setup development environment
## Setting up a single node development environment with ``docker-compose``
You can also use [Docker Compose](https://docs.docker.com/compose/) to run all the tests.
```bash
$ docker-compose build bigchaindb
$ docker-compose up -d bdb
```
The above command will launch all 3 main required services/processes:
* ``mongodb``
* ``tendermint``
* ``bigchaindb``
To follow the logs of the ``tendermint`` service:
```bash
$ docker-compose logs -f tendermint
```
To follow the logs of the ``bigchaindb`` service:
```bash
$ docker-compose logs -f bigchaindb
```
To follow the logs of the ``mongodb`` service:
```bash
$ docker-compose logs -f mdb
```
Simple health check:
```bash
$ docker-compose up curl-client
```
Post and retrieve a transaction -- copy/paste a driver basic example of a
``CREATE`` transaction:
```bash
$ docker-compose -f docker-compose.yml run --rm bdb-driver ipython
```
**TODO**: A python script to post and retrieve a transaction(s).
### Running Tests
Run all the tests using:
```bash
$ docker-compose run --rm --no-deps bigchaindb pytest -v
```
Run tests from a file:
```bash
$ docker-compose run --rm --no-deps bigchaindb pytest /path/to/file -v
```
Run specific tests:
```bash
$ docker-compose run --rm --no-deps bigchaindb pytest /path/to/file -k "<test_name>" -v
```
### Building Docs
You can also develop and build the BigchainDB docs using ``docker-compose``:
```bash
$ docker-compose build bdocs
$ docker-compose up -d bdocs
```
The docs will be hosted on port **33333**, and can be accessed over [localhost](http:/localhost:33333), [127.0.0.1](http:/127.0.0.1:33333)
OR http:/HOST_IP:33333.

View File

@ -143,7 +143,7 @@ One could mark a specific test and execute the same by appending `-m my_mark` to
Although the above should prove sufficient in most cases but in case tests are failing on Travis CI then the following command can be used to possibly replicate the failure locally,
```bash
$ docker-compose -f docker-compose.travis.yml run --rm --no-deps bdb pytest -v --cov=bigchaindb
$ docker-compose run --rm --no-deps bdb pytest -v --cov=bigchaindb
```
NOTE: before executing the above command the user must ensure that they reset the Tendermint container by executing `tendermint usafe_reset_all` command in the Tendermint container.

View File

@ -8,14 +8,10 @@ The value of each BigchainDB Server configuration setting is determined accordin
For convenience, here's a list of all the relevant environment variables (documented below):
`BIGCHAINDB_KEYPAIR_PUBLIC`<br>
`BIGCHAINDB_KEYPAIR_PRIVATE`<br>
`BIGCHAINDB_KEYRING`<br>
`BIGCHAINDB_DATABASE_BACKEND`<br>
`BIGCHAINDB_DATABASE_HOST`<br>
`BIGCHAINDB_DATABASE_PORT`<br>
`BIGCHAINDB_DATABASE_NAME`<br>
`BIGCHAINDB_DATABASE_REPLICASET`<br>
`BIGCHAINDB_DATABASE_CONNECTION_TIMEOUT`<br>
`BIGCHAINDB_DATABASE_MAX_TRIES`<br>
`BIGCHAINDB_SERVER_BIND`<br>
@ -40,14 +36,7 @@ For convenience, here's a list of all the relevant environment variables (docume
`BIGCHAINDB_LOG_FMT_LOGFILE`<br>
`BIGCHAINDB_LOG_GRANULAR_LEVELS`<br>
`BIGCHAINDB_LOG_PORT`<br>
`BIGCHAINDB_DATABASE_SSL`<br>
`BIGCHAINDB_DATABASE_LOGIN`<br>
`BIGCHAINDB_DATABASE_PASSWORD`<br>
`BIGCHAINDB_DATABASE_CA_CERT`<br>
`BIGCHAINDB_DATABASE_CERTFILE`<br>
`BIGCHAINDB_DATABASE_KEYFILE`<br>
`BIGCHAINDB_DATABASE_KEYFILE_PASSPHRASE`<br>
`BIGCHAINDB_DATABASE_CRLFILE`<br>
The local config file is `$HOME/.bigchaindb` by default (a file which might not even exist), but you can tell BigchainDB to use a different file by using the `-c` command-line option, e.g. `bigchaindb -c path/to/config_file.json start`
or using the `BIGCHAINDB_CONFIG_PATH` environment variable, e.g. `BIGHAINDB_CONFIG_PATH=.my_bigchaindb_config bigchaindb start`.
@ -55,112 +44,41 @@ Note that the `-c` command line option will always take precedence if both the `
You can read the current default values in the file [bigchaindb/\_\_init\_\_.py](https://github.com/bigchaindb/bigchaindb/blob/master/bigchaindb/__init__.py). (The link is to the latest version.)
Running `bigchaindb -y configure mongodb` will generate a local config file in `$HOME/.bigchaindb` with all the default values (for using MongoDB as the database backend), with two exceptions: it will generate a valid private/public keypair, rather than using the default keypair (`None` and `None`).
Running `bigchaindb -y configure localmongodb` will generate a local config file in `$HOME/.bigchaindb` with all the default values.
## keypair.public & keypair.private
The [cryptographic keypair](../appendices/cryptography.html) used by the node. The public key is how the node idenifies itself to the world. The private key is used to generate cryptographic signatures. Anyone with the public key can verify that the signature was generated by whoever had the corresponding private key.
**Example using environment variables**
```text
export BIGCHAINDB_KEYPAIR_PUBLIC=8wHUvvraRo5yEoJAt66UTZaFq9YZ9tFFwcauKPDtjkGw
export BIGCHAINDB_KEYPAIR_PRIVATE=5C5Cknco7YxBRP9AgB1cbUVTL4FAcooxErLygw1DeG2D
```
**Example config file snippet**
```js
"keypair": {
"public": "8wHUvvraRo5yEoJAt66UTZaFq9YZ9tFFwcauKPDtjkGw",
"private": "5C5Cknco7YxBRP9AgB1cbUVTL4FAcooxErLygw1DeG2D"
}
```
Internally (i.e. in the Python code), both keys have a default value of `None`, but that's not a valid key. Therefore you can't rely on the defaults for the keypair. If you want to run BigchainDB, you must provide a valid keypair, either in the environment variables or in the local config file. You can generate a local config file with a valid keypair (and default everything else) using `bigchaindb -y configure mongodb`.
## keyring
A list of the public keys of all the nodes in the cluster, excluding the public key of this node.
**Example using an environment variable**
```text
export BIGCHAINDB_KEYRING=BnCsre9MPBeQK8QZBFznU2dJJ2GwtvnSMdemCmod2XPB:4cYQHoQrvPiut3Sjs8fVR1BMZZpJjMTC4bsMTt9V71aQ
```
Note how the keys in the list are separated by colons.
**Example config file snippet**
```js
"keyring": ["BnCsre9MPBeQK8QZBFznU2dJJ2GwtvnSMdemCmod2XPB",
"4cYQHoQrvPiut3Sjs8fVR1BMZZpJjMTC4bsMTt9V71aQ"]
```
**Default value (from a config file)**
```js
"keyring": []
```
## database.*
The settings with names of the form `database.*` are for the database backend
(currently either MongoDB or RethinkDB). They are:
(currently only MongoDB). They are:
* `database.backend` is either `mongodb` or `rethinkdb`.
* `database.backend` is only `localmongodb`, currently.
* `database.host` is the hostname (FQDN) of the backend database.
* `database.port` is self-explanatory.
* `database.name` is a user-chosen name for the database inside MongoDB or RethinkDB, e.g. `bigchain`.
* `database.replicaset` is only relevant if using MongoDB; it's the name of the MongoDB replica set, e.g. `bigchain-rs`.
* `database.name` is a user-chosen name for the database inside MongoDB, e.g. `bigchain`.
* `database.connection_timeout` is the maximum number of milliseconds that BigchainDB will wait before giving up on one attempt to connect to the database backend.
* `database.max_tries` is the maximum number of times that BigchainDB will try to establish a connection with the database backend. If 0, then it will try forever.
* `database.ssl` is a flag that determines if BigchainDB connects to the
backend database over TLS/SSL or not. This can be set to either `true` or
`false` (the default).
Note: This parameter is only supported for the MongoDB backend currently.
* `database.login` and `database.password` are the login and password used to
authenticate to the database before performing any operations, specified in
plaintext. The default values for both are currently `null`, which means that
BigchainDB will not authenticate with the backend database.
Note: These parameters are only supported for the MongoDB backend currently.
* `database.ca_cert`, `database.certfile`, `database.keyfile` and `database.crlfile` are the paths to the CA, signed certificate, private key and certificate revocation list files respectively.
Note: These parameters are only supported for the MongoDB backend currently.
* `database.keyfile_passphrase` is the private key decryption passphrase, specified in plaintext.
Note: This parameter is only supported for the MongoDB backend currently.
**Example using environment variables**
```text
export BIGCHAINDB_DATABASE_BACKEND=mongodb
export BIGCHAINDB_DATABASE_BACKEND=localmongodb
export BIGCHAINDB_DATABASE_HOST=localhost
export BIGCHAINDB_DATABASE_PORT=27017
export BIGCHAINDB_DATABASE_NAME=bigchain
export BIGCHAINDB_DATABASE_REPLICASET=bigchain-rs
export BIGCHAINDB_DATABASE_CONNECTION_TIMEOUT=5000
export BIGCHAINDB_DATABASE_MAX_TRIES=3
```
**Default values**
If (no environment variables were set and there's no local config file), or you used `bigchaindb -y configure rethinkdb` to create a default local config file for a RethinkDB backend, then the defaults will be:
If (no environment variables were set and there's no local config file), or you used `bigchaindb -y configure localmongodb` to create a default local config file for a `localmongodb` backend, then the defaults will be:
```js
"database": {
"backend": "rethinkdb",
"host": "localhost",
"port": 28015,
"name": "bigchain",
"connection_timeout": 5000,
"max_tries": 3
}
```
If you used `bigchaindb -y configure mongodb` to create a default local config file for a MongoDB backend, then the defaults will be:
```js
"database": {
"backend": "mongodb",
"backend": "localmongodb",
"host": "localhost",
"port": 27017,
"name": "bigchain",
"replicaset": "bigchain-rs",
"replicaset": null,
"connection_timeout": 5000,
"max_tries": 3,
"login": null,

View File

@ -99,8 +99,6 @@ spec:
configMapKeyRef:
name: bdb-config
key: bdb-user
- name: BIGCHAINDB_START_TENDERMINT
value: "0"
- name: BIGCHAINDB_TENDERMINT_HOST
valueFrom:
configMapKeyRef:

View File

@ -1,4 +1,4 @@
FROM tendermint/tendermint:0.13
FROM tendermint/tendermint:0.12
LABEL maintainer "dev@bigchaindb.com"
WORKDIR /
COPY genesis.json.template /etc/tendermint/genesis.json

View File

@ -1,27 +0,0 @@
echo "########################################################################"
echo "# #"
echo "# NODE ONE #"
echo "# #"
echo "########################################################################"
curl http://tendermint-one:46657/abci_query && curl http://bdb-one:9984/
echo "########################################################################"
echo "# #"
echo "# NODE TWO #"
echo "# #"
echo "########################################################################"
curl http://tendermint-two:46657/abci_query && curl http://bdb-two:9984/
echo "########################################################################"
echo "# #"
echo "# NODE THREE #"
echo "# #"
echo "########################################################################"
curl http://tendermint-three:46657/abci_query && curl http://bdb-three:9984/
echo "########################################################################"
echo "# #"
echo "# NODE FOUR #"
echo "# #"
echo "########################################################################"
curl http://tendermint-four:46657/abci_query && curl http://bdb-four:9984/

View File

@ -1,18 +0,0 @@
# This is a TOML config file.
# For more information, see https://github.com/toml-lang/toml
proxy_app = "tcp://bdb-one:46658"
moniker = "anonymous"
fast_sync = true
db_backend = "leveldb"
log_level = "state:info,*:error"
[consensus]
create_empty_blocks = false
[rpc]
laddr = "tcp://0.0.0.0:46657"
[p2p]
laddr = "tcp://0.0.0.0:46656"
seeds = ""

View File

@ -1,39 +0,0 @@
{
"genesis_time": "0001-01-01T00:00:00Z",
"chain_id": "test-chain-KPI1Ud",
"validators": [
{
"pub_key": {
"type":"ed25519",
"data":"60C55D531F87D9AA0DFA3AE2DC1842FBFB531DA2161BE3B8D6CC03948CFC39C8"
},
"power": 10,
"name": "node1"
},
{
"pub_key": {
"type": "ed25519",
"data": "981B3879F24CAC4833AFC7EBB71AAEBE79C61E717D7791295B253E7A8E454518"
},
"power": 10,
"name": "node2"
},
{
"pub_key": {
"type": "ed25519",
"data": "E8C4FEE10BA60982CBD9C41614CF42E31E3DF9FE50124BA4C0A5E3CEA897DA9F"
},
"power": 10,
"name": "node3"
},
{
"pub_key": {
"type": "ed25519",
"data": "531248E1D7E35EDB26F0C19F3211FA5CFDA818B226F1F9206D0D2047B48B20B9"
},
"power": 10,
"name": "node4"
}
],
"app_hash":""
}

View File

@ -1 +0,0 @@
{"address":"F81B79DE3F8D8455F76BA1D2FCB62726D69B1253","pub_key":{"type":"ed25519","data":"60C55D531F87D9AA0DFA3AE2DC1842FBFB531DA2161BE3B8D6CC03948CFC39C8"},"last_height":0,"last_round":0,"last_step":0,"last_signature":null,"priv_key":{"type":"ed25519","data":"CD6BAD1433AA62AD9B384C49FD83507829FE39306FC0475903CCF0BA114FCCCB60C55D531F87D9AA0DFA3AE2DC1842FBFB531DA2161BE3B8D6CC03948CFC39C8"}}

View File

@ -1,19 +0,0 @@
# This is a TOML config file.
# For more information, see https://github.com/toml-lang/toml
proxy_app = "tcp://bdb-two:46658"
moniker = "anonymous"
fast_sync = true
db_backend = "leveldb"
log_level = "state:info,*:error"
[consensus]
create_empty_blocks = false
[rpc]
laddr = "tcp://0.0.0.0:46657"
# TODO peers
[p2p]
laddr = "tcp://0.0.0.0:46656"
seeds = "tendermint-one:46656"

View File

@ -1,39 +0,0 @@
{
"genesis_time": "0001-01-01T00:00:00Z",
"chain_id": "test-chain-KPI1Ud",
"validators": [
{
"pub_key": {
"type":"ed25519",
"data":"60C55D531F87D9AA0DFA3AE2DC1842FBFB531DA2161BE3B8D6CC03948CFC39C8"
},
"power": 10,
"name": "node1"
},
{
"pub_key": {
"type": "ed25519",
"data": "981B3879F24CAC4833AFC7EBB71AAEBE79C61E717D7791295B253E7A8E454518"
},
"power": 10,
"name": "node2"
},
{
"pub_key": {
"type": "ed25519",
"data": "E8C4FEE10BA60982CBD9C41614CF42E31E3DF9FE50124BA4C0A5E3CEA897DA9F"
},
"power": 10,
"name": "node3"
},
{
"pub_key": {
"type": "ed25519",
"data": "531248E1D7E35EDB26F0C19F3211FA5CFDA818B226F1F9206D0D2047B48B20B9"
},
"power": 10,
"name": "node4"
}
],
"app_hash":""
}

View File

@ -1 +0,0 @@
{"address":"0DDAB4527921A5E36C099A3260900E8B49F55096","pub_key":{"type":"ed25519","data":"981B3879F24CAC4833AFC7EBB71AAEBE79C61E717D7791295B253E7A8E454518"},"last_height":0,"last_round":0,"last_step":0,"last_signature":null,"priv_key":{"type":"ed25519","data":"BE14C56DA02B1FEBD101EF6EF986F3F2984C0ABE96FEE9FEC9616384CC4F71FF981B3879F24CAC4833AFC7EBB71AAEBE79C61E717D7791295B253E7A8E454518"}}

View File

@ -1,18 +0,0 @@
# This is a TOML config file.
# For more information, see https://github.com/toml-lang/toml
proxy_app = "tcp://bdb-three:46658"
moniker = "anonymous"
fast_sync = true
db_backend = "leveldb"
log_level = "state:info,*:error"
[consensus]
create_empty_blocks = false
[rpc]
laddr = "tcp://0.0.0.0:46657"
[p2p]
laddr = "tcp://0.0.0.0:46656"
seeds = "tendermint-two:46656"

View File

@ -1,39 +0,0 @@
{
"genesis_time": "0001-01-01T00:00:00Z",
"chain_id": "test-chain-KPI1Ud",
"validators": [
{
"pub_key": {
"type":"ed25519",
"data":"60C55D531F87D9AA0DFA3AE2DC1842FBFB531DA2161BE3B8D6CC03948CFC39C8"
},
"power": 10,
"name": "node1"
},
{
"pub_key": {
"type": "ed25519",
"data": "981B3879F24CAC4833AFC7EBB71AAEBE79C61E717D7791295B253E7A8E454518"
},
"power": 10,
"name": "node2"
},
{
"pub_key": {
"type": "ed25519",
"data": "E8C4FEE10BA60982CBD9C41614CF42E31E3DF9FE50124BA4C0A5E3CEA897DA9F"
},
"power": 10,
"name": "node3"
},
{
"pub_key": {
"type": "ed25519",
"data": "531248E1D7E35EDB26F0C19F3211FA5CFDA818B226F1F9206D0D2047B48B20B9"
},
"power": 10,
"name": "node4"
}
],
"app_hash":""
}

View File

@ -1 +0,0 @@
{"address":"F51FD23581F3FA49C68BBDC229461F24EBE202BE","pub_key":{"type":"ed25519","data":"E8C4FEE10BA60982CBD9C41614CF42E31E3DF9FE50124BA4C0A5E3CEA897DA9F"},"last_height":0,"last_round":0,"last_step":0,"last_signature":null,"priv_key":{"type":"ed25519","data":"E9849DEE8A524F00530EF1A486AD8E93A3AD2FCE4BD59F2F08282AD2BE3AE183E8C4FEE10BA60982CBD9C41614CF42E31E3DF9FE50124BA4C0A5E3CEA897DA9F"}}

View File

@ -1,20 +0,0 @@
# This is a TOML config file.
# This is a TOML config file.
# This is a TOML config file.
# For more information, see https://github.com/toml-lang/toml
proxy_app = "tcp://bdb-four:46658"
moniker = "anonymous"
fast_sync = true
db_backend = "leveldb"
log_level = "state:info,*:error"
[consensus]
create_empty_blocks = false
[rpc]
laddr = "tcp://0.0.0.0:46657"
[p2p]
laddr = "tcp://0.0.0.0:46656"
seeds = "tendermint-one:46656,tendermint-three:46656"

View File

@ -1,39 +0,0 @@
{
"genesis_time": "0001-01-01T00:00:00Z",
"chain_id": "test-chain-KPI1Ud",
"validators": [
{
"pub_key": {
"type":"ed25519",
"data":"60C55D531F87D9AA0DFA3AE2DC1842FBFB531DA2161BE3B8D6CC03948CFC39C8"
},
"power": 10,
"name": "node1"
},
{
"pub_key": {
"type": "ed25519",
"data": "981B3879F24CAC4833AFC7EBB71AAEBE79C61E717D7791295B253E7A8E454518"
},
"power": 10,
"name": "node2"
},
{
"pub_key": {
"type": "ed25519",
"data": "E8C4FEE10BA60982CBD9C41614CF42E31E3DF9FE50124BA4C0A5E3CEA897DA9F"
},
"power": 10,
"name": "node3"
},
{
"pub_key": {
"type": "ed25519",
"data": "531248E1D7E35EDB26F0C19F3211FA5CFDA818B226F1F9206D0D2047B48B20B9"
},
"power": 10,
"name": "node4"
}
],
"app_hash":""
}

View File

@ -1 +0,0 @@
{"address":"EB1E972D69E212E928CD98B581E50233E9C43F51","pub_key":{"type":"ed25519","data":"531248E1D7E35EDB26F0C19F3211FA5CFDA818B226F1F9206D0D2047B48B20B9"},"last_height":0,"last_round":0,"last_step":0,"last_signature":null,"priv_key":{"type":"ed25519","data":"864CA8AB5B2F3CBD42994381648D6F75E827E0D53C3B28872E03D7AEB2E10F14531248E1D7E35EDB26F0C19F3211FA5CFDA818B226F1F9206D0D2047B48B20B9"}}

View File

@ -9,7 +9,7 @@ A few notes:
- [`tests/common/`](./common/) contains self-contained tests only testing
[`bigchaindb/common/`](../bigchaindb/common/)
- [`tests/backend/`](./backend/) contains tests requiring
the database backend (RethinkDB or MongoDB)
the database backend (MongoDB)
## Writing Tests
@ -26,15 +26,8 @@ didn't install the tests. Before you can run all the tests, you must install
BigchainDB from source. The [`CONTRIBUTING.md` file](../CONTRIBUTING.md) has
instructions for how to do that.
Next, make sure you have RethinkDB or MongoDB running in the background. You
can run RethinkDB using `rethinkdb --daemon` or MongoDB using `mongod --replSet=bigchain-rs`.
If you wish to test with a TLS/SSL enabled MongoDB, use the command
```text
mongod --replSet=bigchain-rs --sslAllowInvalidHostnames --sslMode=requireSSL \
-sslCAFile=bigchaindb/tests/backend/mongodb-ssl/certs/ca.crt \
--sslCRLFile=bigchaindb/tests/backend/mongodb-ssl/certs/crl.pem \
--sslPEMKeyFile=bigchaindb/tests/backend/mongodb-ssl/certs/test_mdb_ssl_cert_and_key.pem
```
Next, make sure you have Local MongoDB running in the background. You
can run MongoDB using `mongod`.
The `pytest` command has many options. If you want to learn about all the
things you can do with pytest, see [the pytest
@ -42,13 +35,11 @@ documentation](http://pytest.org/latest/). We've also added a customization to
pytest:
`--database-backend`: Defines the backend to use for the tests. It defaults to
`rethinkdb`
It must be one of the backends available in the [server
configuration](https://docs.bigchaindb.com/projects/server/en/latest/server-reference/configuration.html).
`localmongodb`.
Now you can run all tests using:
```text
py.test -v
pytest -v
```
or, if that doesn't work, try:
@ -61,15 +52,6 @@ or:
python setup.py test
```
**Note**: the above pytest commands default to use RethinkDB as the backend. If
you wish to run the tests against MongoDB add the `--database-backend=mongodb`
to the `pytest` command. If you wish to run tests against a TLS/SSL enabled
MongoDB instance (as mentioned above), use the command
```text
pytest -v --database-backend=mongodb-ssl -m bdb_ssl
```
How does `python setup.py test` work? The documentation for [pytest-runner](https://pypi.python.org/pypi/pytest-runner) explains.
The `pytest` command has many options. If you want to learn about all the things you can do with pytest, see [the pytest documentation](http://pytest.org/latest/). We've also added a customization to pytest:
@ -79,47 +61,21 @@ The `pytest` command has many options. If you want to learn about all the things
You can also use [Docker Compose](https://docs.docker.com/compose/) to run all the tests.
#### With MongoDB as the backend
First, start `MongoDB` in the background:
First, bring up all the services BigchainDB, MongoDB, Tendermint in the background:
```text
$ docker-compose up -d mdb
$ docker-compose up -d bdb
```
then run the tests using:
```text
$ docker-compose run --rm bdb py.test -v
$ docker-compose run --rm --no-deps bigchaindb pytest -v
```
If you've upgraded to a newer version of BigchainDB, you might have to rebuild
the images before being able to run the tests. Run:
```text
$ docker-compose build
```
#### With RethinkDB as the backend
First, start `RethinkDB` in the background:
```text
$ docker-compose -f docker-compose.rdb.yml up -d rdb
```
then run the tests using:
```text
$ docker-compose -f docker-compose.rdb.yml run --rm bdb-rdb py.test -v
```
to rebuild all the images (usually you only need to rebuild the `bdb` and
`bdb-rdb` images). If that fails, then do `make clean-pyc` and try again.
## Automated Testing of All Pull Requests
We use [Travis CI](https://travis-ci.com/), so that whenever someone creates a new BigchainDB pull request on GitHub, Travis CI gets the new code and does _a bunch of stuff_. You can find out what we tell Travis CI to do in [the `.travis.yml` file](.travis.yml): it tells Travis CI how to install BigchainDB, how to run all the tests, and what to do "after success" (e.g. run `codecov`). (We use [Codecov](https://codecov.io/) to get a rough estimate of our test coverage.)
We use [Travis CI](https://travis-ci.com/), so that whenever someone creates a new BigchainDB pull request on GitHub, Travis CI gets the new code and does _a bunch of stuff_. We use the same `docker-compose.yml` for tests. It tells Travis CI how to install BigchainDB, how to run all the tests, and what to do "after success" (e.g. run `codecov`). (We use [Codecov](https://codecov.io/) to get a rough estimate of our test coverage.)
### Tox

View File

@ -1,7 +1,7 @@
# This is a TOML config file.
# For more information, see https://github.com/toml-lang/toml
proxy_app = "tcp://bdb:46658"
proxy_app = "tcp://bigchaindb:46658"
moniker = "anonymous"
fast_sync = true
db_backend = "leveldb"

View File

@ -1 +0,0 @@
{"genesis_time":"0001-01-01T00:00:00Z","chain_id":"test-chain-JCYeEN","validators":[{"pub_key":{"type":"ed25519","data":"0C988282C02CFF72E5E296DB78CE26D922178549B327C375D992548C9AFCCE6D"},"power":10,"name":""}],"app_hash":""}

View File

@ -1 +0,0 @@
{"address":"E6CB05DA326F70BB4CC0A4AF83FC3BBF70B9A4D5","pub_key":{"type":"ed25519","data":"0C988282C02CFF72E5E296DB78CE26D922178549B327C375D992548C9AFCCE6D"},"last_height":0,"last_round":0,"last_step":0,"last_signature":null,"priv_key":{"type":"ed25519","data":"D4488996BDF92CE1D80670C66923D4996AE1B772FE0F76DAE33EDC410DC1D58F0C988282C02CFF72E5E296DB78CE26D922178549B327C375D992548C9AFCCE6D"}}