4 Commits

Author SHA1 Message Date
Krish
cc66d5aaa5 Single node setup (#1418)
* Add more tools to the toolbox container

* Add mongodb monitoring agent
* Add a bigchaindb/mongodb-monitoring-agent container that includes the
monitoring agent.
* It makes use of an api key provided by MongoDB Cloud Manager. This is
included in the configuration/config-map.yaml file.

* Changes to mongodb StatefulSet configuration
Changes to bump up mongodb version to v3.4.3.
Add configuration settings for mongodb instance name in ConfigMap.
Split the mongodb service to a new configuration file.

* Modify bigchaindb deployment config
* Bugfix to remove keyring field for the first node.
* Split the mongodb service to a new configuration file.

* Add mongodb backup agent
* Add a bigchaindb/mongodb-backup-agent container that includes the
backup agent.
* It makes use of an api key provided by MongoDB Cloud Manager. This is
included in the configuration/config-map.yaml file.

* Changes to nginx deployment config
* Allow 'all' by default for now. This is included in the
configuration/config-map.yaml file.
* Dynamically resolve DNS addresses of our backend services; cache DNS
resolution for 20s.
* Configure DNS based on user provided resolver. This helps in user
deciding to provide 8.8.8.8 or a custom DNS for name resolution. For k8s
deployments, we use the hardcoded k8s DNS IP of 10.0.0.10.

* Changes to nginx-3scale deployment config
* Use the common ConfigMap in configuration/config-map.yaml file.

* Removing prefix `v` from the docker tag for mongodb-monitoring-agent and mongodb containers

* Bumping up version for nginx-3scale container

* Add small helper scripts for docker build and push of mongodb monitoring
and backup agents

* Documentation for setting up the first node with monitoring and backup
agents
2017-04-21 14:41:12 +02:00
Krish
425397f644 NGINX frontend for MongoDB and BigchainDB (#1304)
- Added NGINX deployment to frontend both BDB and MDB.
- Nginx is configured with a whitelist (which is read from a ConfigMap)
to allow only other MDB nodes in the closter to communicate with it.
- Azure LB apparently does not support proxy protocol and hence
whitelisting fails as nginx always observer the LB IP instead of the
real IP in the TCP stream.
- Whitelisting source IPs for MongoDB
- Removing deprecated folder
- Better log format
- Intuitive port number usage
- README and examples
- Addressed a typo in PYTHON_STYLE_GUIDE.md
- Azure LB apparently does not support proxy protocol and hence
whitelisting fails as nginx always observer the LB IP instead of the
real IP in the TCP stream.
- Whitelisting source IPs for MongoDB
- Removing deprecated folder
- Multiple changes:
- Better log format
- Intuitive port number usage
- README and examples
- Addressed a typo in PYTHON_STYLE_GUIDE.md
- Documentation
- add the k8s directory to the ignore list in codecov.yml
2017-03-22 14:25:25 +01:00
Krish
ea6ce5c1a1 Single node/cluster bootstrap and node addition workflow in k8s (#1278)
* Combining configs
*  Combining the persistent volume claims into a single file.
*  Combining the storage classes into a single file.

* Updating documentation

* Multiple changes
*  Support for ConfigMap
*  Custom MongoDB container for BigchainDB
*  Update documentation to run a single node on k8s

* Additional documentation

* Documentation to add a node to an existing BigchainDB cluster

* Commit on rolling upgrades

* Fixing minor documentation mistakes

* Documentation updates as per @ttmc's comments

* Block formatting error

* Change in ConfigMap yaml config
2017-03-15 16:22:49 +01:00
Krish
7dbd374838 Running a single node on k8s (#1269)
* Single node as a StatefulSet in k8s
- uses bigchaindb/bigchaindb:0.9.1

* Updating README

* rdb, mdb as stateful services

* [WIP] bdb as a statefulset

* [WIP] bdb w/ rdb and bdb w/ mdb backends
- does not work as of now

* Split mdb & bdb into separate pods + enhancements
*  discovery of the mongodb service by the bdb pod by using dns name.
*  using separate storage classes to map 2 different volumes exposed by the
mongo docker container; one for /data/db (dbPath) and the other for
 /data/configdb (configDB).
*  using the `persistentVolumeReclaimPolicy: Retain` in k8s pvc. However,
this seems to be unsupported in Azure and the disks still show a reclaim
policy of `delete`.
*  mongodb container runs the `mongod` process as user `mongodb` and group
`mongodb. The corresponding `uid` and `gid` for the `mongod` process is 999
and 999 respectively. When the constinaer runs on a host with a mounted disk,
the writes fail, when there is no user with uid 999. To avoid this, I use the
docker provided feature of --cap-add=FOWNER in k8s. This bypasses the uid and
gid permission checks during writes and allows writes.
Ref: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities

* Delete redundant k8s files, add cluster deletion steps.

* Single node as a StatefulSet in k8s
- uses bigchaindb/bigchaindb:0.9.1

* Updating README

* rdb, mdb as stateful services

* [WIP] bdb as a statefulset

* [WIP] bdb w/ rdb and bdb w/ mdb backends
- does not work as of now

* Split mdb & bdb into separate pods + enhancements
*  discovery of the mongodb service by the bdb pod by using dns name.
*  using separate storage classes to map 2 different volumes exposed by the
mongo docker container; one for /data/db (dbPath) and the other for
 /data/configdb (configDB).
*  using the `persistentVolumeReclaimPolicy: Retain` in k8s pvc. However,
this seems to be unsupported in Azure and the disks still show a reclaim
policy of `delete`.
*  mongodb container runs the `mongod` process as user `mongodb` and group
`mongodb. The corresponding `uid` and `gid` for the `mongod` process is 999
and 999 respectively. When the constinaer runs on a host with a mounted disk,
the writes fail, when there is no user with uid 999. To avoid this, I use the
docker provided feature of --cap-add=FOWNER in k8s. This bypasses the uid and
gid permission checks during writes and allows writes.
Ref: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities

* Delete redundant k8s files, add cluster deletion steps.

* Documentation: running a single node with distinct mongodb and bigchaindb
pods on k8s

* Updates as per @ttmc's comments
2017-03-09 16:53:00 +01:00