Mirroristas/bigchaindb
2
0
Fork 0
mirror of https://github.com/bigchaindb/bigchaindb.git synced 2024-10-13 13:34:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

Docs: Added 2 concepts to backup strategies

Browse Source
This commit is contained in:
troymc 2016-07-05 17:40:53 +02:00
parent 471f032c1a
commit 28b8158255
1 changed files with 23 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

31
docs/source/clusters-feds/backup.md
View File

@ -3,20 +3,28 @@
There are several ways to backup and restore the data in a BigchainDB cluster.
## RethinkDB's Replication is a form of Backup
## RethinkDB's Replication as a form of Backup
RethinkDB already has internal replication: every document is stored on _R_ different nodes, where _R_ is the replication factor (set using `bigchaindb set-replicas R`). Those replicas can be thought of as "live backups" because if one node goes down, the cluster will continue to work and no data will be lost.
At this point, there should be someone saying, "But replication isn't backup!"
It's true. Replication alone isn't enough, because something bad might happen _inside_ the database, and that could affect the replicas. For example, what if someone logged in as a RethinkDB admin and did a "drop table"? We currently plan for each node to be protected by a next-generation firewall (or something similar) to prevent such things from getting very far. For example, see [issue #240](https://github.com/bigchaindb/bigchaindb/issues/240).
Nevertheless, you should still consider having normal, "cold" backups, because bad things can still happen.
## Live Replication of RethinkDB Data Files
All RethinkDB data is stored in one directory. You could set up the node's file system so that directory lives on its own hard drive. Furthermore, you could make that hard drive part of a [RAID](https://en.wikipedia.org/wiki/RAID) array, so that a second hard drive would always have a copy of the original. If the original hard drive fails, then the second hard drive could take its place and the node would continue to function. Meanwhile, the original hard drive could be replaced.
Each BigchainDB node stores its subset of the RethinkDB data in one directory. You could set up the node's file system so that directory lives on its own hard drive. Furthermore, you could make that hard drive part of a [RAID](https://en.wikipedia.org/wiki/RAID) array, so that a second hard drive would always have a copy of the original. If the original hard drive fails, then the second hard drive could take its place and the node would continue to function. Meanwhile, the original hard drive could be replaced.
That's just one possible way of setting up the file system so as to provide extra reliability. It's debatable whether it's a "backup strategy," but one could argue that the second hard drive is like a backup of the original.
That's just one possible way of setting up the file system so as to provide extra reliability.
Another way to get similar reliability would be to mount the RethinkDB data directory on an [Amazon EBS](https://aws.amazon.com/ebs/) volume. Each Amazon EBS volume is, "automatically replicated within its Availability Zone to protect you from component failure, offering high availability and durability.""
Another way to get similar reliability would be to mount the RethinkDB data directory on an [Amazon EBS](https://aws.amazon.com/ebs/) volume. Each Amazon EBS volume is, "automatically replicated within its Availability Zone to protect you from component failure, offering high availability and durability."
See [the section on file system setup](../nodes/setup-run-node.html#set-up-the-file-system-for-rethinkdb) for more details.
See [the section on setting up storage for RethinkDB](../nodes/setup-run-node.html#set-up-storage-for-rethinkdb-data) for more details.
As with shard replication, live file-system replication protects against many failure modes, but it doesn't protect against them all. You should still consider having normal, "cold" backups.
## rethinkdb dump (to a File)
@ -39,7 +47,7 @@ There's [more information about the `rethinkdb dump` command in the RethinkDB do
* If the `rethinkdb dump` subcommand fails and the last line of the Traceback says "NameError: name 'file' is not defined", then you need to update your RethinkDB Python driver; do a `pip install --upgrade rethinkdb`
* It can take a very long time to backup data this way. The more data, the longer it will take.
* It might take a long time to backup data this way. The more data, the longer it will take.
* You need enough free disk space to store the backup file.
@ -65,7 +73,7 @@ Yes, in principle, but it would be difficult to know if you've recovered every b
## Backup by Copying RethinkDB Data Files
It's _possible_ to back up a BigchainDB database by creating a point-in-tim copy of the RethinkDB data files (on all nodes, at roughly the same time). It's not a very practical approach to backup: the resulting set of files will be much larger (collectively) than what one would get using `rethinkdb dump`, and there are no guarantees on how consistent that data will be, especially for recently-written data.
It's _possible_ to back up a BigchainDB database by creating a point-in-time copy (or "snapshot") of the RethinkDB data files (on all nodes, at roughly the same time). It's not a very practical approach to backup: the resulting set of files will be much larger (collectively) than what one would get using `rethinkdb dump`, and there are no guarantees on how consistent that data will be, especially for recently-written data.
If you're curious about what's involved, see the [MongoDB documentation about "Backup by Copying Underlying Data Files"](https://docs.mongodb.com/manual/core/backups/#backup-with-file-copies). (Yes, that's documentation for MongoDB, but the principles are the same.)
@ -93,7 +101,8 @@ Considerations for BigchainDB:
* We'd like the backup to be decentralized, with no single point of control or single point of failure. (Note: some file systems have a single point of failure. For example, HDFS has one Namenode.)
* We only care to back up blocks and votes, and once written, those never change. There are no updates or deletes, just new blocks and votes.
**RethinkDB Replication as Continuous Backup**
## Combining RethinkDB Replication with File System Snapshots
Although it's not advertised as such, RethinkDB's built-in replication feature is similar to continous backup, except the "backup" (i.e. the set of replica shards) is spread across all the nodes. One could take that idea a bit farther by creating a set of backup-only servers with one full backup:
@ -102,3 +111,9 @@ Although it's not advertised as such, RethinkDB's built-in replication feature i
* Send a RethinkDB reconfigure command to the RethinkDB cluster to make it so that the `original` set has the same number of replicas as before (or maybe one less), and the `backup` set has one replica. Also, make sure the `primary_replica_tag='original'` so that all primary shards live on the `original` nodes.
The [RethinkDB documentation on sharding and replication](https://www.rethinkdb.com/docs/sharding-and-replication/) has the details of how to set server tags and do RethinkDB reconfiguration.
Once you've set up a set of backup-only RethinkDB servers, you could make a point-in-time snapshot of their file systems, as a form of backup.
You might want to disconnect the `backup` set from the `original` set first, and then wait for reads and writes in the `backup` set to stop. That way, all the data in the `backup` set will be consistent before you take the snapshot.
You will want to re-connect the `backup` set to the `original` set as soon as possible, so it's able to catch up.