From 6ac09e95c3cd03bfb7ae18e2501f9c8d9265b319 Mon Sep 17 00:00:00 2001 From: troymc Date: Wed, 15 Jun 2016 15:32:41 +0200 Subject: [PATCH] New top-level section of docs: Clusters & Federations --- .../{nodes => clusters-feds}/deploy-on-aws.md | 2 +- .../source/clusters-feds/federation-set-up.md | 38 ++++++++++++ docs/source/clusters-feds/index.rst | 13 +++++ .../{nodes => clusters-feds}/monitoring.md | 2 +- docs/source/index.rst | 1 + docs/source/nodes/configuration.md | 2 +- docs/source/nodes/federation-set-up.md | 58 ------------------- docs/source/nodes/index.rst | 19 +----- docs/source/topic-guides/index.rst | 1 + docs/source/topic-guides/node-cluster-fed.md | 13 +++++ 10 files changed, 71 insertions(+), 78 deletions(-) rename docs/source/{nodes => clusters-feds}/deploy-on-aws.md (99%) create mode 100644 docs/source/clusters-feds/federation-set-up.md create mode 100644 docs/source/clusters-feds/index.rst rename docs/source/{nodes => clusters-feds}/monitoring.md (95%) delete mode 100644 docs/source/nodes/federation-set-up.md create mode 100644 docs/source/topic-guides/node-cluster-fed.md diff --git a/docs/source/nodes/deploy-on-aws.md b/docs/source/clusters-feds/deploy-on-aws.md similarity index 99% rename from docs/source/nodes/deploy-on-aws.md rename to docs/source/clusters-feds/deploy-on-aws.md index 4e3596cb..f725f59a 100644 --- a/docs/source/nodes/deploy-on-aws.md +++ b/docs/source/clusters-feds/deploy-on-aws.md @@ -124,7 +124,7 @@ fab --fabfile=fabfile-monitor.py --hosts= run_monitor For more information about monitoring (e.g. how to view the Grafana dashboard in your web browser), see the [Monitoring](monitoring.html) section of this documentation. -To configure a BigchainDB node to send monitoring data to the monitoring server, change the statsd host in the configuration of the BigchainDB node. The section on [Configuring a BigchainDB Node](configuration.html) explains how you can do that. (For example, you can change the statsd host in `$HOME/.bigchaindb`.) +To configure a BigchainDB node to send monitoring data to the monitoring server, change the statsd host in the configuration of the BigchainDB node. The section on [Configuring a BigchainDB Node](../nodes/configuration.html) explains how you can do that. (For example, you can change the statsd host in `$HOME/.bigchaindb`.) ## Deploy a BigchainDB Cluster diff --git a/docs/source/clusters-feds/federation-set-up.md b/docs/source/clusters-feds/federation-set-up.md new file mode 100644 index 00000000..018bb515 --- /dev/null +++ b/docs/source/clusters-feds/federation-set-up.md @@ -0,0 +1,38 @@ +# Set Up and Run a Federation + +This section is about how to set up and run a BigchainDB _federation_, where each node is operated by a different operator. If you want to set up and run a BigchainDB cluster on AWS (where all nodes are operated by you), then see [the section about that](deploy-on-aws.html). + + +## Initial Checklist + +* Do you have a governance process for making federation-level decisions, such as how to admit new members? +* What will you store in creation transactions (data payload)? Is there a data schema? +* Will you use transfer transactions? Will they include a non-empty data payload? +* Who will be allowed to submit transactions? Who will be allowed to read or query transactions? How will you enforce the access rules? + + +## Set Up the Initial Cluster + +When you first start a federation cluster, the initial nodes will all start at roughly the same time. + +The steps to set up a cluster node are outlined in the section titled [Set Up and Run a Node](../nodes/setup-run-node.html). You'll need two pieces of information from all other nodes in the federation: + +1. Their RethinkDB hostname, e.g. `rdb.farm2.organization.org` +2. Their BigchainDB public key, e.g. `Eky3nkbxDTMgkmiJC8i5hKyVFiAQNmPP4a2G4JdDxJCK` + + +## Documentation to Come + +* Backing Up & Restoring data +* Adding a node (including resharding etc.) +* Removing a node +* Logging +* Node monitoring & crash recovery +* Node Security + * Securing your OS + * Firewalls and security groups. Remember to open port 123 for NTP. + * (Private) key management + * RethinkDB security +* Cluster monitoring +* Internal watchdogs +* External watchdogs \ No newline at end of file diff --git a/docs/source/clusters-feds/index.rst b/docs/source/clusters-feds/index.rst new file mode 100644 index 00000000..dcda2e85 --- /dev/null +++ b/docs/source/clusters-feds/index.rst @@ -0,0 +1,13 @@ +.. You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +BigchainDB Clusters & Federations +================================= + +.. toctree:: + :maxdepth: 1 + + federation-set-up + deploy-on-aws + monitoring + \ No newline at end of file diff --git a/docs/source/nodes/monitoring.md b/docs/source/clusters-feds/monitoring.md similarity index 95% rename from docs/source/nodes/monitoring.md rename to docs/source/clusters-feds/monitoring.md index 626230fe..629ccbd5 100644 --- a/docs/source/nodes/monitoring.md +++ b/docs/source/clusters-feds/monitoring.md @@ -28,7 +28,7 @@ You can view the Grafana dashboard in your web browser at: (You may want to replace `localhost` with another hostname in that URL, e.g. the hostname of a remote monitoring server.) -The login and password are `admin` by default. If BigchainDB is running and processing transactions, you should see analytics—if not, [start BigchainDB](setup-run-node.html#run-bigchaindb) and load some test transactions: +The login and password are `admin` by default. If BigchainDB is running and processing transactions, you should see analytics—if not, [start BigchainDB](../nodes/setup-run-node.html#run-bigchaindb) and load some test transactions: ```text $ bigchaindb load ``` diff --git a/docs/source/index.rst b/docs/source/index.rst index 2ef1a8fc..9173d125 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -13,6 +13,7 @@ Table of Contents introduction topic-guides/index nodes/index + clusters-feds/index drivers-clients/index release-notes appendices/index diff --git a/docs/source/nodes/configuration.md b/docs/source/nodes/configuration.md index fad2b416..9fe13925 100644 --- a/docs/source/nodes/configuration.md +++ b/docs/source/nodes/configuration.md @@ -60,7 +60,7 @@ environment variables available are: - `BIGCHAINDB_KEYPAIR_PUBLIC` defines the public key of the BigchainDB node. - `BIGCHAINDB_KEYPAIR_PRIVATE` defines the private key of the BigchainDB node. - `BIGCHAINDB_KEYRING` is a colon-separated list of the public keys of all _other_ nodes in the cluster. -- `BIGCHAINDB_STATSD_HOST` defines the hostname of the statsd server for [monitoring](monitoring.html). +- `BIGCHAINDB_STATSD_HOST` defines the hostname of the statsd server for [monitoring](../clusters-feds/monitoring.html). - `BIGCHAINDB_STATSD_PORT` defines the port of the statsd server for monitoring. - `BIGCHAINDB_STATSD_RATE` is a float between `0` and `1` that defines the fraction of transaction operations sampled. - `BIGCHAINDB_API_ENDPOINT` defines the API endpoint to use (e.g. `http://localhost:9984/api/v1`). diff --git a/docs/source/nodes/federation-set-up.md b/docs/source/nodes/federation-set-up.md deleted file mode 100644 index 68b47f3f..00000000 --- a/docs/source/nodes/federation-set-up.md +++ /dev/null @@ -1,58 +0,0 @@ -# Set Up and Run a Federation - -This section is about how to set up and run a BigchainDB _federation_, where each node is operated by a different operator. If you want to set up and run a BigchainDB cluster on AWS (where all nodes are operated by you), then see [the section about that](deploy-on-aws.html). - - -## Answer Basic Questions - -* Do you have a governance process for making federation-level decisions, such as how to admit new members? -* What will you store in creation transactions (data payload)? Is there a data schema? -* Will you use transfer transactions? Will they include a non-empty data payload? -* Who will be allowed to submit transactions? Who will be allowed to read or query transactions? How will you enforce the access rules? - - -## Set Up the Initial Cluster - -When you first start a federation cluster, the initial nodes will all start at roughly the same time. Here are the steps to be taken by each node operator: - -**Create a RethinkDB Cluster** - -* Go through the steps of [Set Up and Run a Node](setup-run-node.html), up to "Configure RethinkDB Server" -* Determine the hostname of the server running RethinkDB (e.g. `rethinkdb2.farm3.organization5.com`) -* Share your RethinkDB hostname with all other members of the federation -* Once you have all the RethinkDB hostnames, add them to your local RethinkDB configuration file as explained in [Configure RethinkDB Server](setup-run-node.html#configure-rethinkdb-server) -* (TODO: Section with steps to make RethinkDB more secure) - -**Install, Configure and Run BigchainDB** - -* Continue with the steps of [Set Up and Run a Node](setup-run-node.html) -* You will generate a default `.bigchaindb` config file at some point. It contains your public/private keypair. -* Send your **public** key to all other federation members. If you accidentally send your private key, then delete your `.bigchaindb` config file, generate a new one, and send your new public key to all other federation members. -* Once you have the public keys of all other federation members, put them in the `keyring` (list) in your `.bigchaindb` file. (Your keyring should _not_ include your public key.) - -* TODO: Are there other things they should change in their `.bigchaindb` file? - -* Make sure RethinkDB is Running (TODO: say how) -* Start BigchainDB - -```text -$ bigchaindb start -``` - - - -## Documentation to Come - -* Backing Up & Restoring data -* Adding a node (including resharding etc.) -* Removing a node -* Logging -* Node monitoring & crash recovery -* Node Security - * Securing your OS - * Firewalls and security groups. Remember to open port 123 for NTP. - * (Private) key management - * RethinkDB security -* Cluster monitoring -* Internal watchdogs -* External watchdogs \ No newline at end of file diff --git a/docs/source/nodes/index.rst b/docs/source/nodes/index.rst index f9a03180..ecf92b8e 100644 --- a/docs/source/nodes/index.rst +++ b/docs/source/nodes/index.rst @@ -1,31 +1,16 @@ .. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. -BigchainDB Nodes, Clusters & Federations -======================================== - -A **BigchainDB node** is a server or set of closely-linked servers running RethinkDB Server, BigchainDB Server, and other BigchainDB-related software. Each node is controlled by one person or organization. - -A set of BigchainDB nodes can connect to each other to form a **cluster**. Each node in the cluster runs the same software. A cluster contains one logical RethinkDB datastore. A cluster may have additional servers to do things such as cluster monitoring. - -The people and organizations that run the nodes in a cluster belong to a **federation** (i.e. another organization). A federation must have some sort of governance structure to make decisions. If a cluster is run by a single company, then the federation is just that company. - -**What's the Difference Between a Cluster and a Federation?** - -A cluster is just a bunch of connected nodes (computers). A cluster might be operated by just one person. A federation is an organization which has a cluster, and where each node in the cluster has a different operator. - -Confusingly, we sometimes call a federation's cluster its "federation." You can probably tell what we mean from context. +BigchainDB Nodes +================ .. toctree:: :maxdepth: 1 node-requirements - federation-set-up setup-run-node run-with-docker running-unit-tests configuration bigchaindb-cli python-server-api-examples - deploy-on-aws - monitoring diff --git a/docs/source/topic-guides/index.rst b/docs/source/topic-guides/index.rst index 309620e2..4785fcc9 100644 --- a/docs/source/topic-guides/index.rst +++ b/docs/source/topic-guides/index.rst @@ -14,3 +14,4 @@ Topic guides give background and explain concepts at a high level. immutable assets models + node-cluster-fed diff --git a/docs/source/topic-guides/node-cluster-fed.md b/docs/source/topic-guides/node-cluster-fed.md new file mode 100644 index 00000000..2987d477 --- /dev/null +++ b/docs/source/topic-guides/node-cluster-fed.md @@ -0,0 +1,13 @@ +# Nodes, Clusters & Federations + +A **BigchainDB node** is a server or set of closely-linked servers running RethinkDB Server, BigchainDB Server, and other BigchainDB-related software. Each node is controlled by one person or organization. + +A set of BigchainDB nodes can connect to each other to form a **cluster**. Each node in the cluster runs the same software. A cluster contains one logical RethinkDB datastore. A cluster may have additional servers to do things such as cluster monitoring. + +The people and organizations that run the nodes in a cluster belong to a **federation** (i.e. another organization). A federation must have some sort of governance structure to make decisions. If a cluster is run by a single company, then the federation is just that company. + +**What's the Difference Between a Cluster and a Federation?** + +A cluster is just a bunch of connected nodes (computers). A cluster might be operated by just one person. A federation is an organization which has a cluster, and where each node in the cluster has a different operator. + +Confusingly, we sometimes call a federation's cluster its "federation." You can probably tell what we mean from context. \ No newline at end of file