From 7a274bd84d2967e5c3569c8612367a3cd09d2519 Mon Sep 17 00:00:00 2001 From: haad Date: Tue, 4 Oct 2016 18:58:04 +0200 Subject: [PATCH] Update documentation --- README.md | 9 ++- docs/architecture.md | 107 -------------------------------- docs/ordering.md | 141 ------------------------------------------- 3 files changed, 7 insertions(+), 250 deletions(-) delete mode 100644 docs/architecture.md delete mode 100644 docs/ordering.md diff --git a/README.md b/README.md index a2f21bc..486a169 100644 --- a/README.md +++ b/README.md @@ -16,8 +16,13 @@ This is the Javascript implementation and it works both in **Node.js** and **Bro - Aggregation happens on client side and data is eventually consistent - Designed to work offline first -![Screenshot1](https://raw.githubusercontent.com/haadcode/orbit-db/feat/ipfs-pubsub/screenshots/orbit-db-demo1.gif) -![Screenshot2](https://raw.githubusercontent.com/haadcode/orbit-db/feat/ipfs-pubsub/screenshots/orbit-db-demo3.gif) +**Node.js** + + + +**Browser** + + ### Data stores diff --git a/docs/architecture.md b/docs/architecture.md deleted file mode 100644 index 9b62fef..0000000 --- a/docs/architecture.md +++ /dev/null @@ -1,107 +0,0 @@ -***WIP*** - -### Structure -The database has one log (OrbitList) for each *channel*. *Channel* is comparable to a *"topic"* or *"table"*. - -``` -DB -|-- channel 1 -| |-- list 1 -| | |-- node 1 -| | |-- operation -| | |-- value -| |-- list 2 -|-- channel 2 - |-- list 3 -... -``` - -### Operation Logs -- Each *channel* is saved as a log of operations: add, put, del -- Operations are stored in an append-only log linked list -- Each node in the linked list points to the previous node -- Event log: take latest add or del operation for -- JV-store: take last put or del operation for - -### CRDTs -- orbit-db is a CmRDT and implements an LWW-element-set -- Operation-based CRDT -- Uses Merkle Trees for partial ordering - -### add/put IO: -==> Not expensive - -1. Create POST for the content -`ipfs object get QmZzic86oN7B5GMMYnTFvZMyDMjkKEBXe5SYsryXPeoGdb?` -```json -{ - "content": "hello 1", - "ts": 1457703735759, - "meta": { - "type": "text", - "size": 7, - "ts": 1457703735759 - } -} -``` - -2. Create POST for the DB operation -`ipfs object get QmZzBNrUiYATJ4aicPTbupnEUWH3AHDmfBbDTQK3fhhYDE` -```json -{ - "op": "ADD", - "key": "QmZzic86oN7B5GMMYnTFvZMyDMjkKEBXe5SYsryXPeoGdb", - "value": "QmZzic86oN7B5GMMYnTFvZMyDMjkKEBXe5SYsryXPeoGdb", - "meta": { - "type": "orbit-db-op", - "size": 15, - "ts": 1457703735779 - }, - "by": "userA" -} -``` - -3. Create ipfs object for the nodet -`ipfs object get QmX2Jq1JHmgjgM3YVuHyGSRpUWMDbv4PuRhqBhsZqDmagD` -```json -{ - "id": "userA", - "seq": 0, - "ver": 1, - "data": "QmZzBNrUiYATJ4aicPTbupnEUWH3AHDmfBbDTQK3fhhYDE", - "next": { - "userA.0.0": "QmXUTgYPG4cSaHW8dKggJRfLvPWjaDktWyRAgn5NPwTqtz" - } -} -``` - -4. Create ipfs object for the current list -`ipfs object get Qmb2rpex9TdmoXvwLKLL24atWa2HfPbArobN7XiBAFvmZ9` -```json -{ - "id": "userA", - "seq": 1, - "ver": 0, - "items": { - "userA.0.0": "QmXUTgYPG4cSaHW8dKggJRfLvPWjaDktWyRAgn5NPwTqtz", - "userA.0.1": "QmX2Jq1JHmgjgM3YVuHyGSRpUWMDbv4PuRhqBhsZqDmagD" - } -} -``` - -5. Pubsub.publish (send to socket, network IO) -```json -{ - channel: '', - hash: 'Qmb2rpex9TdmoXvwLKLL24atWa2HfPbArobN7XiBAFvmZ9' -} -``` - -### get IO: -==> A little expensive - -1. Payload (get from ipfs-hash, network IO) - -### sync/merge IO: -==> Expensive! -TODO diff --git a/docs/ordering.md b/docs/ordering.md deleted file mode 100644 index 419ae27..0000000 --- a/docs/ordering.md +++ /dev/null @@ -1,141 +0,0 @@ -# Odering in OrbitList - -Ordering in OrbitList is done with Interval Tree Clocks (https://github.com/ricardobcl/Interval-Tree-Clocks). Each node tracks a global sequence *seq* and a local version *ver*. When an item is added to a list, local version *ver* is increased by 1. Whenever a list is *joined* with another list (eg. upon sync), the global sequence *seq* is set to *Max(global.seq, local.seq)*. When adding an item to the list, the item has a reference to all *heads* where *head* is an item that is not referenced by any other item in the list. - -### Graph -``` - A B C - - 0.0 - | -0.0 0.1 - | | -0.1 0.2 - \ / <--- Sync B with A and C - 1.0 - | - 1.1 - / <--- Sync A with B -2.0 - \ - \ <--- Sync C with A - \ - 3.0 - <--- Sync ALL -``` - -### Evolution of the state -Initial state: -``` -A = [] -B = [] -C = [] -``` - -***Two nodes add events concurrently*** - -Add items to A: -``` -listA.add("mango") -listA.add("banana") - -// "A": [ -// { "id": "A", "seq": 0, "ver": 0, "prev": null} -// { "id": "A", "seq": 0, "ver": 1, "prev": ["A.0.0"]} -// ] -``` - -Add items to C: -``` -listC.add("apple") -listC.add("strawberry") -listC.add("orange") - -// "C": [ -// { "id": "C", "seq": 0, "ver": 0, "prev": null} -// { "id": "C", "seq": 0, "ver": 1, "prev": ["C.0.0"]} -// { "id": "C", "seq": 0, "ver": 2, "prev": ["C.0.1"]} -// ] -``` - -B receives a 'sync' from A and C: -``` -B.join(A) -B.join(C) -``` - -Add items to B: -``` -listB.add("pineapple") -listB.add("papaya") - -// "B": [ -// { "id": "A", "seq": 0, "ver": 0, "prev": null} -// { "id": "A", "seq": 0, "ver": 1, "prev": ["A.0.0"]} -// { "id": "C", "seq": 0, "ver": 0, "prev": null} -// { "id": "C", "seq": 0, "ver": 1, "prev": ["C.0.0"]} -// { "id": "C", "seq": 0, "ver": 2, "prev": ["C.0.1"]} -// { "id": "B", "seq": 1, "ver": 0, "prev": ["A.0.1", "C.0.2"]} -// { "id": "B", "seq": 1, "ver": 1, "prev": ["B.1.0"]} -// ] -``` - -A receives a 'sync' from B: -``` -A.join(B) -``` - -``` -listA.add("kiwi") - -// "A": [ -// { "id": "A", "seq": 0, "ver": 0, "prev": null} -// { "id": "A", "seq": 0, "ver": 1, "prev": ["A.0.0"]} -// { "id": "C", "seq": 0, "ver": 0, "prev": null} -// { "id": "C", "seq": 0, "ver": 1, "prev": ["C.0.0"]} -// { "id": "C", "seq": 0, "ver": 2, "prev": ["C.0.1"]} -// { "id": "B", "seq": 1, "ver": 0, "prev": ["A.0.1", "C.0.2"]} -// { "id": "B", "seq": 1, "ver": 1, "prev": ["B.1.0"]} -// { "id": "A", "seq": 2, "ver": 0, "prev": ["B.1.1"]} -// ] -``` - -C receives a 'sync' from A: -``` -C.join(A) -``` - -``` -listC.add("blueberry") - -// "C": [ -// { "id": "A", "seq": 0, "ver": 0, "prev": null} -// { "id": "A", "seq": 0, "ver": 1, "prev": ["A.0.0"]} -// { "id": "C", "seq": 0, "ver": 0, "prev": null} -// { "id": "C", "seq": 0, "ver": 1, "prev": ["C.0.0"]} -// { "id": "C", "seq": 0, "ver": 2, "prev": ["C.0.1"]} -// { "id": "B", "seq": 1, "ver": 0, "prev": ["A.0.1", "C.0.2"]} -// { "id": "B", "seq": 1, "ver": 1, "prev": ["B.1.0"]} -// { "id": "A", "seq": 2, "ver": 0, "prev": ["B.1.1"]} -// { "id": "C", "seq": 3, "ver": 0, "prev": ["A.2.0"]} -// ] -``` - -A receives a 'sync' from C, B receives a 'sync' from C: -``` -A.join(C) -B.join(C) -``` - -Converged data set: -```json -{ "id": "A", "seq": 0, "ver": 0, "prev": null}, -{ "id": "A", "seq": 0, "ver": 1, "prev": ["A.0.0"]}, -{ "id": "C", "seq": 0, "ver": 0, "prev": null}, -{ "id": "C", "seq": 0, "ver": 1, "prev": ["C.0.0"]}, -{ "id": "C", "seq": 0, "ver": 2, "prev": ["C.0.1"]}, -{ "id": "B", "seq": 1, "ver": 0, "prev": ["A.0.1", "C.0.2"]}, -{ "id": "B", "seq": 1, "ver": 1, "prev": ["B.1.0"]} -{ "id": "A", "seq": 2, "ver": 0, "prev": ["B.1.1"]} -{ "id": "C", "seq": 3, "ver": 0, "prev": ["A.2.0]"]} -```