diff --git a/docs/server/source/production-deployment-template/add-node-on-kubernetes.rst b/docs/server/source/production-deployment-template/add-node-on-kubernetes.rst index 3fe7f9b6..7ab94ceb 100644 --- a/docs/server/source/production-deployment-template/add-node-on-kubernetes.rst +++ b/docs/server/source/production-deployment-template/add-node-on-kubernetes.rst @@ -73,7 +73,7 @@ Claims, and to run MongoDB in the new cluster: 1. :ref:`Add Storage Classes `. 2. :ref:`Add Persistent Volume Claims `. -3. :ref:`Create the Config Map `. +3. :ref:`Create the Config Map `. 4. :ref:`Run MongoDB instance `. diff --git a/docs/server/source/production-deployment-template/cloud-manager.rst b/docs/server/source/production-deployment-template/cloud-manager.rst index 8c6277b8..cfd44fac 100644 --- a/docs/server/source/production-deployment-template/cloud-manager.rst +++ b/docs/server/source/production-deployment-template/cloud-manager.rst @@ -2,7 +2,7 @@ Configure MongoDB Cloud Manager for Monitoring and Backup ========================================================= This document details the steps required to configure MongoDB Cloud Manager to -enable monitoring and back up of data in a MongoDB Replica Set. +enable monitoring and backup of data in a MongoDB Replica Set. Configure MongoDB Cloud Manager for Monitoring diff --git a/docs/server/source/production-deployment-template/node-config-map-and-secrets.rst b/docs/server/source/production-deployment-template/node-config-map-and-secrets.rst index 88bfbe31..6e25c77b 100644 --- a/docs/server/source/production-deployment-template/node-config-map-and-secrets.rst +++ b/docs/server/source/production-deployment-template/node-config-map-and-secrets.rst @@ -1,72 +1,102 @@ -Configure the Node -================== +How to Configure a BigchainDB Node +================================== -Use the ConfigMap template in ``configuration/config-map.yaml`` file to -configure the node. Update all the values for the keys in the -ConfigMaps ``vars``, ``bdb-public-key``, ``bdb-keyring`` and -``mongodb-whitelist``. +This page outlines the steps to set a bunch of configuration settings +in your BigchainDB node. +They are pushed to the Kubernetes cluster in two files, +named ``config-map.yaml`` (a set of ConfigMaps) +and ``secret.yaml`` (a set of Secrets). +They are stored in the Kubernetes cluster's key-value store (etcd). + +Make sure you did all the things listed in the section titled +:ref:`Things Each Node Operator Must Do` +(including generation of all the SSL certificates needed +for MongoDB auth). -Use the Secret template in ``configuration/secret.yaml`` file to configure -the secrets for this node. Update all the values for the keys in the Secrets -``mdb-agent-api-key``, ``https-certs``, ``bdb-private-key``, -``threescale-credentials`` and ``mdb-certs``. +Edit config-map.yaml +-------------------- -You might not need all the keys during the deployment. -For example, if you plan to access the BigchainDB API over HTTP, you might -not need the ``https-certs`` Secret. - +Make a copy of the file ``k8s/configuration/config-map.yaml`` +and edit the data values in the various ConfigMaps. +That file already contains many comments to help you +understand each data value, but we make some additional +remarks on some of the values below. -Ensure that all the secrets are base64 encoded values and the unused ones -are set to an empty string. -For example, assuming that the public key chain is named ``cert.pem`` and -private key is ``cert.key``, run the following commands to encode the -certificates into single continuous string that can be embedded in yaml, -and then copy the contents of ``cert.pem.b64`` in the ``cert.pem`` field, -and the contents of ``cert.key.b64`` in the ``cert.key`` field. - +Note: None of the data values in ``config-map.yaml`` need +to be base64-encoded. (This is unlike ``secret.yaml``, +where all data values must be base64-encoded. +This is true of all Kubernetes ConfigMaps and Secrets.) + + +vars +~~~~ + +Your BigchainDB cluster organization should have a standard way +of naming instances, so the instances in your BigchainDB node +should conform to that standard (i.e. you can't just make up some names). +There are some things worth noting about the ``mdb-instance-name``: + +* MongoDB reads the local ``/etc/hosts`` file while bootstrapping a replica + set to resolve the hostname provided to the ``rs.initiate()`` command. + It needs to ensure that the replica set is being initialized in the same + instance where the MongoDB instance is running. +* We use the value in the ``mdb-instance-name`` field to achieve this. +* This field will be the DNS name of your MongoDB instance, and Kubernetes + maps this name to its internal DNS. +* This field will also be used by other MongoDB instances when forming a + MongoDB replica set. +* We use ``mdb-instance-0``, ``mdb-instance-1`` and so on in our + documentation. Your BigchainDB cluster may use a different naming convention. + +bdb-keyring.bdb-keyring +~~~~~~~~~~~~~~~~~~~~~~~ + +This lists the BigchainDB public keys +of all *other* nodes in your BigchainDB cluster +(not including the public key of your BigchainDB node). Cases: + +* If you're deploying the first node in the cluster, + the value should be ``""`` (an empty string). +* If you're deploying the second node in the cluster, + the value should be one public key inside double quotes. + For example, + ``"EPQk5i5yYpoUwGVM8VKZRjM8CYxB6j8Lu8i8SG7kGGce"`` +* If there are two or more other nodes already in the cluster, + the value should be a colon-separated list of public keys + inside double quotes. + For example, + ``"DPjpKbmbPYPKVAuf6VSkqGCf5jzrEh69Ldef6TrLwsEQ:EPQk5i5yYpoUwGVM8VKZRjM8CYxB6j8Lu8i8SG7kGGce"`` + + +Edit secret.yaml +---------------- + +Make a copy of the file ``k8s/configuration/secret.yaml`` +and edit the data values in the various Secrets. +That file includes many comments to explain the required values. +**In particular, note that all values must be base64-encoded.** +There are tips at the top of the file +explaining how to convert values into base64-encoded values. + +Your BigchainDB node might not need all the Secrets. +For example, if you plan to access the BigchainDB API over HTTP, you +don't need the ``https-certs`` Secret. +You can delete the Secrets you don't need, +or set their data values to ``""``. + +Note that ``ca.pem`` is just another name for ``ca.crt`` +(the certificate of your BigchainDB cluster's self-signed CA). + + +Deploy Your config-map.yaml and secret.yaml +------------------------------------------- + +You can deploy your edited ``config-map.yaml`` and ``secret.yaml`` +files to your Kubernetes cluster using the commands: .. code:: bash - cat cert.pem | base64 -w 0 > cert.pem.b64 - - cat cert.key | base64 -w 0 > cert.key.b64 - - -Create the ConfigMap and Secret using the commands: - -.. code:: bash - - kubectl --context k8s-bdb-test-cluster-0 apply -f configuration/config-map.yaml - - kubectl --context k8s-bdb-test-cluster-0 apply -f configuration/secret.yaml - - -Some of the Node Configuration Options --------------------------------------- - -1. ConfigMap vars.mdb-instance-name - - * MongoDB reads the local ``/etc/hosts`` file while bootstrapping a replica - set to resolve the hostname provided to the ``rs.initiate()`` command. - It needs to ensure that the replica set is being initialized in the same - instance where the MongoDB instance is running. - * We use the value in the ``mdb-instance-name`` field to achieve this. - * This field will be the DNS name of your MongoDB instance, and Kubernetes - maps this name to its internal DNS. - * This field will also be used by other MongoDB instances when forming a - MongoDB replica set. - * We use ``mdb-instance-0``, ``mdb-instance-1`` and so on in our - documentation. - -2. ConfigMap bdb-keyring.bdb-keyring - - * This value specifies the public keys of all the nodes in a BigchainDB - cluster. - * It is a ':' separated list, similar to the PATH variables in Unix systems. - - -3. ConfigMap bdb-public-key.bdb-public-key - - * This value specifies the public key of the current BigchainDB node. + $ kubectl apply -f config-map.yaml + $ kubectl apply -f secret.yaml diff --git a/docs/server/source/production-deployment-template/node-on-kubernetes.rst b/docs/server/source/production-deployment-template/node-on-kubernetes.rst index 9d955bf9..44c91fec 100644 --- a/docs/server/source/production-deployment-template/node-on-kubernetes.rst +++ b/docs/server/source/production-deployment-template/node-on-kubernetes.rst @@ -9,8 +9,8 @@ It assumes you already have a running Kubernetes cluster. If you want to add a new BigchainDB node to an existing BigchainDB cluster, refer to :doc:`the page about that `. -We refer to many files by their directory and filename in the documentation -below, such as ``configuration/config-map.yaml``. Those files are files in the +Below, we refer to many files by their directory and filename, +such as ``configuration/config-map.yaml``. Those files are files in the `bigchaindb/bigchaindb repository on GitHub `_ in the ``k8s/`` directory. Make sure you're getting those files from the appropriate Git branch on @@ -26,23 +26,9 @@ If you don't already have it installed, then see the `Kubernetes docs to install it `_. - The default location of the kubectl configuration file is ``~/.kube/config``. If you don't have that file, then you need to get it. -Find out the ``kubectl context`` of your Kubernetes cluster using the command: - -.. code:: bash - - $ kubectl config view - -The context will be one of the entries in ``context.cluster`` under the -``contexts`` list in the output. - -Assuming that the current context for your cluster is -``k8s-bdb-test-cluster-0``, you will always specify the context in the -following commands as ``kubectl --context k8s-bdb-test-cluster-0``. - **Azure.** If you deployed your Kubernetes cluster on Azure using the Azure CLI 2.0 (as per :doc:`our template `), then you can get the ``~/.kube/config`` file using: @@ -59,29 +45,63 @@ but you get an error message, then try adding ``--ssh-key-file ~/.ssh/`` to the above command (i.e. the path to the private key). +.. note:: -Step 2: Connect to the Cluster UI - (optional) ----------------------------------------------- + **About kubectl contexts.** You might manage several + Kubernetes clusters. To make it easy to switch from one to another, + kubectl has a notion of "contexts," e.g. the context for cluster 1 or + the context for cluster 2. To find out the current context, do: -* Get the kubectl context for this cluster using ``kubectl config view``. - -* For the above commands, the context would be ``k8s-bdb-test-cluster-0``. - - .. code:: bash + .. code:: bash + + $ kubectl config view - $ kubectl --context k8s-bdb-test-cluster-0 proxy -p 8001 + and then look for the ``current-context`` in the output. + The output also lists all clusters, contexts and users. + (You might have only one of each.) + You can switch to a different context using: + + .. code:: bash + + $ kubectl config use-context + + You can also switch to a different context for just one command + by inserting ``--context `` into any kubectl command. + For example: + + .. code:: bash + + $ kubectl --context k8s-bdb-test-cluster-0 get pods + + will get a list of the pods in the Kubernetes cluster associated + with the context named ``k8s-bdb-test-cluster-0``. + +Step 2: Connect to Your Cluster's Web UI (Optional) +--------------------------------------------------- + +You can connect to your cluster's +`Kubernetes Dashboard `_ +(also called the Web UI) using: + +.. code:: bash + + $ kubectl proxy -p 8001 + +or, if you prefer to be explicit about the context (explained above): + +.. code:: bash + + $ kubectl --context k8s-bdb-test-cluster-0 proxy -p 8001 + +The output should be something like ``Starting to serve on 127.0.0.1:8001``. +That means you can visit the dashboard in your web browser at +`http://127.0.0.1:8001/ui `_. -Step 3: Configure the Node --------------------------- - - * You need to have all the information :ref:`listed here `. +Step 3: Configure Your BigchainDB Node +-------------------------------------- - * The information needs to be populated in ``configuration/config-map.yaml`` - and ``configuration/secret.yaml``. - - * For more details, refer to the document on how to - :ref:`configure a node `. +See the page titled :ref:`How to Configure a BigchainDB Node`. Step 4: Start the NGINX Service @@ -164,13 +184,11 @@ Step 5: Assign DNS Name to the NGINX Public IP **Set up DNS mapping in Azure.** - Select the current Azure resource group and look for the ``Public IP`` resource. You should see at least 2 entries there - one for the Kubernetes master and the other for the MongoDB instance. You may have to ``Refresh`` the Azure web page listing the resources in a resource group for the latest changes to be reflected. - Select the ``Public IP`` resource that is attached to your service (it should have the Azure DNS prefix name along with a long random string, without the ``master-ip`` string), select ``Configuration``, add the DNS assigned above @@ -335,7 +353,6 @@ see `the Kubernetes docs about persistent volumes The first thing to do is create the Kubernetes storage classes. **Set up Storage Classes in Azure.** - First, you need an Azure storage account. If you deployed your Kubernetes cluster on Azure using the Azure CLI 2.0 @@ -467,7 +484,7 @@ Step 11: Start a Kubernetes StatefulSet for MongoDB .. code:: bash - $ kubectl --context k8s-bdb-test-cluster-0 get po -w + $ kubectl --context k8s-bdb-test-cluster-0 get pods -w Step 12: Start a Kubernetes Deployment for MongoDB Monitoring Agent diff --git a/docs/server/source/production-deployment-template/workflow.rst b/docs/server/source/production-deployment-template/workflow.rst index c0b61907..4fa00d45 100644 --- a/docs/server/source/production-deployment-template/workflow.rst +++ b/docs/server/source/production-deployment-template/workflow.rst @@ -103,10 +103,9 @@ you must ask the managing organization for all relevant 3scale credentials. ☐ If the cluster uses MongoDB Cloud Manager for monitoring and backup, -you must ask the managing organization for the ``Agent Api Key``. -(Each Cloud Manager backup will have its own ``Agent Api Key``. -If there's one Cloud Manager backup, -there will be one ``Agent Api Key`` for the whole cluster.) +you must ask the managing organization for the ``Agent API Key``. +(Each Cloud Manager "group" has its own ``Agent API Key``. +It can be found under **Settings - Group Settings**.) ☐ Generate four keys and corresponding certificate signing requests (CSRs): diff --git a/k8s/configuration/config-map.yaml b/k8s/configuration/config-map.yaml index 631c8e86..231f5b33 100644 --- a/k8s/configuration/config-map.yaml +++ b/k8s/configuration/config-map.yaml @@ -1,18 +1,21 @@ -####################################################### -# This YAML file desribes a ConfigMap for the cluster # -####################################################### +## Note: data values do NOT have to be base64-encoded in this file. -## Common Env Variables For This Node +## vars is common environment variables for this BigchaindB node apiVersion: v1 kind: ConfigMap metadata: name: vars namespace: default data: + # MongoDB mdb-instance-name: "" + # BigchainDB bdb-instance-name: "" + # NGINX ngx-instance-name: "" + # MongoDB Monitoring Agent mdb-mon-instance-name: "" + # MongoDB Backup Agent mdb-bak-instance-name: "" --- apiVersion: v1 @@ -21,7 +24,7 @@ metadata: name: mongodb-whitelist namespace: default data: - # We support only 'all' currently + # We only support "all"" currently allowed-hosts: "all" --- apiVersion: v1 @@ -30,7 +33,8 @@ metadata: name: bdb-keyring namespace: default data: - bdb-keyring: "':' separated list of public keys>" + # Colon-separated list of all *other* nodes' BigchainDB public keys. + bdb-keyring: "<':' separated list of public keys>" --- apiVersion: v1 kind: ConfigMap @@ -38,4 +42,6 @@ metadata: name: bdb-public-key namespace: default data: + # BigchainDB public key of *this* node. + # Example: "EPQk5i5yYpoUwGVM8VKZRjM8CYxB6j8Lu8i8SG7kGGce" bdb-public-key: "" diff --git a/k8s/configuration/secret.yaml b/k8s/configuration/secret.yaml index d8802ea2..7ea83cb1 100644 --- a/k8s/configuration/secret.yaml +++ b/k8s/configuration/secret.yaml @@ -1,7 +1,11 @@ -# All secret data should be base64 encoded before embedding them here by -# using `echo "secret string" | base64 -w 0 > secret.string.b64` and then -# copy the resulting value here. +# All secret data should be base64 encoded before embedding them here. +# Short strings can be encoded using, e.g. +# echo "secret string" | base64 -w 0 > secret.string.b64 +# Files (e.g. certificates) can be encoded using, e.g. +# cat cert.pem | base64 -w 0 > cert.pem.b64 +# then copy the contents of cert.pem.b64 (for example) below. # Ref: https://kubernetes.io/docs/concepts/configuration/secret/ +# Unused values can be set to "" apiVersion: v1 kind: Secret @@ -10,8 +14,8 @@ metadata: namespace: default type: Opaque data: - # This is the API Key obtained from MongoDB Cloud Manager - api-key: "" + # Base64-encoded Agent API Key (obtained from MongoDB Cloud Manager) + api-key: "" --- apiVersion: v1 kind: Secret @@ -20,7 +24,8 @@ metadata: namespace: default type: Opaque data: - private.key: "" + # Base64-encoded BigchainDB private key of *this* node + private.key: "" --- apiVersion: v1 kind: Secret @@ -29,9 +34,12 @@ metadata: namespace: default type: Opaque data: - mdb-instance.pem: "" - ca.pem: "" - mdb-crl.pem: "" + # Base64-encoded, concatenated certificate and private key + mdb-instance.pem: "" + # Base64-encoded CA certificate (ca.crt) + ca.pem: "" + # Base64-encoded MongoDB CRL + mdb-crl.pem: "" --- apiVersion: v1 kind: Secret @@ -40,8 +48,10 @@ metadata: namespace: default type: Opaque data: - mdb-mon-instance.pem: "" - ca.pem: "" + # Base64-encoded, concatenated certificate and private key + mdb-mon-instance.pem: "" + # Base64-encoded CA certificate (ca.crt) + ca.pem: "" --- apiVersion: v1 kind: Secret @@ -50,8 +60,10 @@ metadata: namespace: default type: Opaque data: - mdb-bak-instance.pem: "" - ca.pem: "" + # Base64-encoded, concatenated certificate and private key + mdb-bak-instance.pem: "" + # Base64-encoded CA certificate (ca.crt) + ca.pem: "" --- apiVersion: v1 kind: Secret @@ -60,8 +72,10 @@ metadata: namespace: default type: Opaque data: - bdb-instance.pem: "" - ca.pem: "" + # Base64-encoded, concatenated certificate and private key + bdb-instance.pem: "" + # Base64-encoded CA certificate (ca.crt) + ca.pem: "" --- apiVersion: v1 kind: Secret @@ -70,7 +84,9 @@ metadata: namespace: default type: Opaque data: + # Base64-encoded HTTPS private key cert.key: "" + # Base64-encoded HTTPS Signed Certificate or Certificate Chain cert.pem: "" --- apiVersion: v1