53 Commits

Author SHA1 Message Date
Troy McConaghy
abdd23f5a6 Problem: Source files contain no license info (#2455)
* Problem: Source files contain no license info

Solution: Add comments with SPDX license info to source files

* Python 3 files don't need # -*- coding: utf-8 -*-
2018-08-16 12:31:32 +02:00
vrde
f86704bae9 Problem: BigchainDB does not support newer MongoDB (#2298)
* Problem: BigchainDB does not support newer MongoDB

Solution: Update driver to the last version, that is compatible with all
versions of MongoDB

* Update to MongoDB 3.6
2018-05-17 15:19:56 +02:00
Troy McConaghy
aeaf9c934d
Last PR before the release of version 2.0 Alpha 5 (#2283)
* Updated CHANGELOG.md for version 2.0 Alpha 5

* Updated image tags in k8s files for 2.0 Alpha 5

* Updated version.py for 2.0 Alpha 5

* Problem: Missing alpha5 image update for test network containers

* Problem: CHANGELOG not updated with 2284

* Problem: Liveness probe frequency change not the important
2018-05-11 15:23:15 +02:00
Ahmed Muawia Khan
dbabe94887 Problem: BigchainDB and Tendermint inconsistencies because disjoint services (#2242)
Solution
Since BigchainDB and Tendermint are so tightly coupled we need to introduce a process supervisor to make them act like a single microservice, so that if BigchainDB crashes, Tendermint does as well and both are restarted and Tendermint requests a connection towards the proxy app.

In Kubernetes, they can be exposed as part of a one POD.
For BigchainDB as a system service/process, we need to introduce a process supervisor such as systemd.
This PR only solves the former.

Changes
Upgrade deployment from Tendermint v0.12.0 to v0.19.0
Update some documentation
Fix nginx-http entrypoint issues.
Update generate-configs.sh script to handle config generation without https-certificates.
Update Dockerfile to process dependency links introduced by abci
Integrate BigchainDB and Tendermint as a single microservice.
This required making BigchainDB to be exposed as a statefulset.
Introduce new liveness probe checks.
Issues Resolved
Partially fixes #2232
2018-04-27 15:54:47 +02:00
Ahmed Muawia Khan
7683ea00fc Problem: Outdated docker image tags for 2.0.0-alpha release (#2174) 2018-04-03 14:52:30 +02:00
Ahmed Muawia Khan
98c939bd29 use mdb-instance-0 as default name for configuring users 2018-02-27 13:35:20 +01:00
Ahmed Muawia Khan
4c6ac6bd19 Update production deployment template after v1 automation 2018-02-27 13:26:51 +01:00
Shahbaz Nazir
d69c75ee93 fixes for k8s deployment automation
Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>
2018-02-27 02:40:59 +01:00
Ahmed Muawia Khan
8835fd5fd7 Add support for optional users in MongoDB user creation 2018-02-26 17:54:07 +01:00
Ahmed Muawia Khan
3c09de97b0 remove redundant variable assignment 2018-02-26 14:54:34 +01:00
Ahmed Muawia Khan
7cea7298f0 Update mongodb user configuration
- Update docs for azure cluster deployment template
- Update entrypoint for MongoDB to handle user configuration
  more efficiently
- remove `use admin` in js file, that is an invalid format
2018-02-26 14:45:21 +01:00
Ahmed Muawia Khan
d977753831 Automate MongoDB user creation for prod/test deployments
- Currently, we had to manually log into the MongoDB container
  and create users, this change will configure the relevant users
  from a single script `configure_mdb.sh`
- Improvements can be done but keeping it minimal for the workshop
2018-02-26 03:37:43 +01:00
Ahmed Muawia Khan
96db5b9f12
Merge pull request #2084 from bigchaindb/update-bdb-docker-image-k8s
Update docker images and Dockerfiles
2018-02-22 12:23:31 +01:00
Ahmed Muawia Khan
93e2b3bb4c Update imagePullPolicy for unstable images
- Update imagePullPolicy from IfNotPresent to Always
  for all unstable images we are using in k8s template,
  because those images are updated regularly, we will
  keep it unstable until the official release.
- Use Tendermint v0.13 as base docker image, since our current
  integration is based out of v0.13
2018-02-22 11:56:58 +01:00
Shahbaz Nazir
a782fa4635
Hardcode static ports for k8s deployment (#2080)
Description
Currently we manually configure ports in deployment and service files for k8s deployment e.g here. Instead these ports should be constant since these are always the same for each deployment

Issues This PR Fixes
Fixes #2077
2018-02-22 11:52:41 +01:00
Ahmed Muawia Khan
a8a10edf0e Update Dockerfiles and remove tmt references 2018-02-21 20:31:53 +01:00
Ahmed Muawia Khan
8e6b4332b0 Update mongodb base image 2018-02-21 20:22:38 +01:00
Ahmed Muawia Khan
2978caaee6 Update images used by yaml files
- Using tag `unstable` currently, until BDB+Tendermint is released
2018-02-21 20:17:24 +01:00
Ahmed Muawia Khan
de05e5bc3e Merge branch 'tendermint' into tendermint-k8s-dep 2018-02-21 11:32:45 +01:00
muawiakh
1a0c165487 Address comments
- Keep only tendermint specific files, remove reference to older
  deployment strategy
- Update Bigchaindb Dockerfile
2018-02-01 13:02:39 +01:00
muawiakh
61f36435fc Separate tendermint integration deployment files
- Separate deployments, services, statefulsets etc
  `yaml` files for tendermint based deployment.
- Separate Dockerfiles for mongodb, nginx for
  tendermint integrated BigchainDB.
2018-01-11 12:13:36 +01:00
muawiakh
9d45f735fd Improving code readability 2018-01-04 14:53:23 +01:00
muawiakh
e2316a3de8 Fix error checking of optional parameter storage-engine-cache-size
- Keep the parameter optional
- Remove redundant checking of the same parameter
- Do not exit if the parameter is not specified
2018-01-04 13:58:28 +01:00
muawiakh
a348c72696 Deployment of Tendermint + BigchainDB cluster
- Update existing docker-containers to support tendermint integration, nginx,
  mongodb, bigchaindb
- Add tendermint configuration files for statefulset, pvc, pv and service.
- Update some READMEs.
2018-01-02 14:51:23 +01:00
Roman Sokolkov
779d31ef3e
Fix issue with STORAGE_ENGINE_CACHE_SIZE parameter
In mongodb docker image STORAGE_ENGINE_CACHE_SIZE
will always be set to '' (two quotation marks)
and verification will fail. So it's impossible to
run image without --storage-engine-cache-size set.

This change removes double quotes while assigning
default value to STORAGE_ENGINE_CACHE_SIZE.
2017-12-30 17:31:55 +01:00
muawiakh
977a6426fe Make premium_lrs default 2017-11-16 12:44:21 +01:00
muawiakh
1eb8764e4a Update deployment with tectonic documentation
- Add details about CA configuration
- Update storage class template
2017-11-16 12:44:21 +01:00
Shahbaz Nazir
0a8bf89bc6 Add workflow to retain Azure storage disks on PVC or PV delete (#1782) 2017-10-13 23:26:13 +02:00
muawiakh
3d2daa3119 Update nginx-http(s) and mongodb deployments files to use latest versions 2017-09-28 10:42:28 +02:00
muawiakh
1e53f13be3 Use config map for storage engine cache size 2017-09-27 14:40:12 +02:00
muawiakh
8b1fd605e2 Address some bugs
- Pushed the wrong codebase in the previous commit
2017-09-27 14:19:31 +02:00
muawiakh
7abdca205a Adjust resource for MongoDB Stateful Set
- Currently, MongoDB container crashed because of resource constaints i.e.
out of memory exception. This change updates the resources and provides
data on how the configure/calculate them(if not following the guide).
- Also, add the ability to specify the storage engine(WiredTiger) cache
size for MongoDB, this configuration also helps with keeping the resources
constrained for MongoDB containers.
- Minor changes in some other documents as well.
2017-09-27 14:19:31 +02:00
krish7919 (Krish)
9445647be5 MongoDB data dir config change 2017-09-07 11:00:03 +02:00
Ahmed Muawia Khan
e41a378df4 Merge pull request #1720 from bigchaindb/remove-reclaim-policy-k8s-pvc
Remove persistentVolumeReclaimPolicy from the pvc request
2017-08-17 15:12:40 +02:00
Ahmed Muawia Khan
a3de849820 Merge pull request #1724 from bigchaindb/imporve-error-message-entrypoint
Improve error message for all bdb docker entrypoint scripts
2017-08-17 11:27:20 +02:00
Ahmed Muawia Khan
ea32694435 Merge pull request #1721 from bigchaindb/common-secret-for-ca
Common secret for CA in secret.yaml
2017-08-17 11:07:35 +02:00
Muawia Khan
7658097221 Use env variables names for error messages
- Address comments
2017-08-17 10:46:19 +02:00
Muawia Khan
d11c100ab5 Improve error message for all bdb docker entrypoint scripts
- If a mandatory variable is not specified, it will exit with the relevant
code and error message.
- For more verbosity, we will also echo the values for all the mandatory
variables.
2017-08-16 13:15:24 +02:00
Muawia Khan
0cf46b331f Verify and fix BDB multi node deployment guide
- Documentation support to add a new BDB node to an existing
  replica set, using x.509 certificates and SSL/TSL connections, across
  geographically dispersed clusters.
- Fix some documentation issues and add more references i.e.
  specifically about signing of MongoDB member certificates.
- Minor fixes for nginx-https-dep.yaml(invalid configMap var)
- Reconfigure nginx keep_alive between MongoDB front and backend ports.
- Editor removed whitespaces
2017-08-16 10:27:17 +02:00
Muawia Khan
fbee5db40f Common secret for CA in secret.yaml
- Creating a common secret for CA, since all the members of the replica set
and the clients need to have a common CA, moving all the relevant configuration
to a common secret.
- Modifying Dockerfiles for some components, once changes are approved
we will publish the new images.
- No documentation changes required.
2017-08-15 14:26:47 +02:00
Muawia Khan
04b84f6ab4 Remove persistentVolumeReclaimPolicy from the pvc request
- Reclaim policy can only be specified when we explicitly create
a persistent Volume. Removing this from a persistent volume claim
request yaml.
2017-08-15 10:07:50 +02:00
Krish
8e6a94e158 Change imagePullPolicy in k8s configs (#1680) 2017-07-19 20:14:56 +02:00
Krish
e0efc2286a Simpler MongoDB Instance Configuration in K8s (#1678)
* Simpler configuration of MongoDB instance in a node

* Update docs for MDB configuration
2017-07-17 17:57:13 +02:00
Krish
9116836e4c Simpler BigchainDB Instance Configuration in K8s (#1677)
* Simpler configuration of BigchainDB instance in a node

* Update docs for BDB configuration

* Changes as per @ttmc's comments
2017-07-17 17:29:51 +02:00
muawiakh
b0106c0203 Fix for MongoDB hitting memory limit on k8s
- MongoDB StatefulSet hitting memory limit, so k8s restarts it.
We have had multiple instances of restarts lately.
-  Changing it to 3.5 GB, data and reasoning to back it up
is mentioned in the ticket #1655
2017-07-12 16:17:50 +02:00
Krish
ece195ff8d Deployment changes: (#1623)
Update MongoDB container tag to `3.0`.

Doc change to reflect bdb-config.bdb-user parameter usage.

Fix typo in configuration.md.

Add BIGCHAINDB_DATABASE_SSL parameter to bigchaindb-dep.yaml for
Kubernetes deployments.

Refer the the `bdb-user` parameter from ConfigMap in
bigchaindb-dep.yaml.

Consolidate all BigchainDB parameter values under the
`bdb-config` ConfigMap.

Remove `bdb-user` from secrets.yaml.
2017-07-03 15:55:43 +02:00
Krish
9ceea89537 Enable Auth over TLS connections (#1552)
* Changes to support auth on the infrastructure
* Auth over TLS/SSL support in BigchainDB, MongoDB, Monitoring Agent, Backup Agent
* Update certificates: Different OUs specified now
* Code formatting
- Make flake happy!
* Raise proper authentication failed error
* Documentation changes for auth
* Support auth in k8s deployment
* Commit certs for monitoring and backup agents
* Configuration to allow Cloud Manager Backup Agent to backup data
* Update docs and remove authentication error
2017-06-22 16:32:04 +02:00
krish7919 (Krish)
cf10d3199e Support for TLS connections in MongoDB StatefulSet 2017-06-06 10:01:53 +02:00
Krish
b6ec3e5f5c Mongodb TLS (#1456)
* Support for secure TLS communication in MongoDB, MongoDB Monitoring
Agent and MongoDB Backup Agent
- Move from Golang to Bash for entrypoint program
- Update image tag to 2.0 for Backup and Monitoring Agents and to
3.4.4 for MongoDB
- Add documentation

* changed title & rewrote Step 1 of workflow.rst

* copy-edited ca-installation.rst

* copy-edited & modified structure of workflow.rst

* moved repeated Easy-RSA install & config docs to new page

* edited the sentences describing the Easy-RSA dirs

* copy-edited the page about generating server certificate

* copy-edited the page about generating client certificate

* renamed page to 'How to Set Up a Self-Signed Certificate Authority'

* copy-edited page about how to revoke a certificate

* Comments on how to uniquely name all instances in the cluster

* Added comments about the other questions when setting up a CA

* Added note about one Agent Api Key per Cloud Manager backup

* docs: clarified instructions for generating server CSR

* docs: added back 'from your PKI infrastructure'

* docs: fixed step & added step re/ FQDNs & certs in workflow.rst

* docs: added note re/ the Distinguished Name

* Update docs for env vars setup

* docs: added tip: how to get help with the easyrsa command
2017-05-19 11:18:44 +02:00
Krish
cc66d5aaa5 Single node setup (#1418)
* Add more tools to the toolbox container

* Add mongodb monitoring agent
* Add a bigchaindb/mongodb-monitoring-agent container that includes the
monitoring agent.
* It makes use of an api key provided by MongoDB Cloud Manager. This is
included in the configuration/config-map.yaml file.

* Changes to mongodb StatefulSet configuration
Changes to bump up mongodb version to v3.4.3.
Add configuration settings for mongodb instance name in ConfigMap.
Split the mongodb service to a new configuration file.

* Modify bigchaindb deployment config
* Bugfix to remove keyring field for the first node.
* Split the mongodb service to a new configuration file.

* Add mongodb backup agent
* Add a bigchaindb/mongodb-backup-agent container that includes the
backup agent.
* It makes use of an api key provided by MongoDB Cloud Manager. This is
included in the configuration/config-map.yaml file.

* Changes to nginx deployment config
* Allow 'all' by default for now. This is included in the
configuration/config-map.yaml file.
* Dynamically resolve DNS addresses of our backend services; cache DNS
resolution for 20s.
* Configure DNS based on user provided resolver. This helps in user
deciding to provide 8.8.8.8 or a custom DNS for name resolution. For k8s
deployments, we use the hardcoded k8s DNS IP of 10.0.0.10.

* Changes to nginx-3scale deployment config
* Use the common ConfigMap in configuration/config-map.yaml file.

* Removing prefix `v` from the docker tag for mongodb-monitoring-agent and mongodb containers

* Bumping up version for nginx-3scale container

* Add small helper scripts for docker build and push of mongodb monitoring
and backup agents

* Documentation for setting up the first node with monitoring and backup
agents
2017-04-21 14:41:12 +02:00