Mirroristas/orbitdb
2
0
Fork 0
mirror of https://github.com/orbitdb/orbitdb.git synced 2025-05-21 14:26:38 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #103 from haadcode/feat/ipfs-pubsub

Browse Source 
Feature: IPFS Pubsub
This commit is contained in:
Haad 2016-10-04 19:10:11 +02:00 committed by GitHub
commit ffb40dd50e
28 changed files with 947 additions and 44213 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
.gitignore vendored
View File

@ -6,4 +6,7 @@ node_modules/
dump.rdb dump.rdb
Vagrantfile Vagrantfile
orbit-db-cache.json orbit-db-cache.json
examples/browser/bundle.js examples/browser/bundle.js
examples/browser/*.map
dist/*.map
dist/orbitdb.js

4
CHANGELOG.md Normal file
View File

@ -0,0 +1,4 @@
# Changelog
## v0.12.0
- IPFS pubsub

263
README.md
View File

@ -16,22 +16,15 @@ This is the Javascript implementation and it works both in **Node.js** and **Bro
- Aggregation happens on client side and data is eventually consistent - Aggregation happens on client side and data is eventually consistent
- Designed to work offline first - Designed to work offline first
Check out a visualization of the data flow at https://github.com/haadcode/proto2 **Node.js**
Live demo: http://celebdil.benet.ai:8080/ipfs/Qmezm7g8mBpWyuPk6D84CNcfLKJwU6mpXuEN5GJZNkX3XK/ <img src="https://raw.githubusercontent.com/haadcode/orbit-db/feat/ipfs-pubsub/screenshots/orbit-db-demo3.gif" width="60%">
![Screenshot](https://raw.githubusercontent.com/haadcode/proto2/master/screenshot.png) **Browser**
**NOTE: the README can be out of date, I'm working to get up to date. If you find a problem, please open an issue or a PR.** <img src="https://raw.githubusercontent.com/haadcode/orbit-db/feat/ipfs-pubsub/screenshots/orbit-db-demo1.gif" height="60%">
_Currently requires [orbit-server](https://github.com/haadcode/orbit-server) for pubsub communication. This will change in the future as soon as IPFS provides pubsub._ ### Data stores
## Install
```
npm install orbit-db
```
## Data stores
Currently available data stores: Currently available data stores:
@ -40,177 +33,145 @@ Currently available data stores:
- [orbit-db-feedstore](https://github.com/haadcode/orbit-db-feedstore) - [orbit-db-feedstore](https://github.com/haadcode/orbit-db-feedstore)
- [orbit-db-counterstore](https://github.com/haadcode/orbit-db-counterstore) - [orbit-db-counterstore](https://github.com/haadcode/orbit-db-counterstore)
## Install
From npm:
```
npm install orbit-db
```
From git:
```
git clone https://github.com/haadcode/orbit-db.git
cd orbit-db
npm install
```
## Usage ## Usage
```javascript ```javascript
'use strict'
const IpfsApi = require('ipfs-api')
const OrbitDB = require('orbit-db') const OrbitDB = require('orbit-db')
const IPFS = require('ipfs')
OrbitDB.connect('178.62.241.75:3333', 'tester', null, new IPFS(), { allowOffline: true }) const ipfs = IpfsApi('localhost', '5001')
.then((orbitdb) => { const orbitdb = new OrbitDB(ipfs)
const kvstore = orbitdb.kvstore('db name')
const events = orbitdb.eventlog('db name')
const feed = orbitdb.feed('db name')
const counters = orbitdb.counter('db name')
kvstore.put('key1', 'hello1') const db = orbitdb.eventlog("feed name")
.then(() => kvstore.get('key1'))
.then((value) => console.log(value)) // 'hello!' db.add("hello world")
.then(() => {
const latest = db.iterator({ limit: 5 }).collect()
console.log(latest.join("\n"))
}) })
``` ```
Documentation for individual stores are WIP, please see each store's source code for available public methods.
## Examples ## Examples
*To run the examples below, make sure to run a local [orbit-server](https://github.com/haadcode/orbit-server)* ### Browser example
### Browser examples
Build the examples:
```bash ```bash
npm install npm install
npm run build:examples npm run build:examples
npm run examples:browser
``` ```
Then open `examples/browser.html` or `examples/index.html`. See the full example [here](https://github.com/haadcode/orbit-db/blob/master/examples/browser/browser.html). ### Node.js example
```html
<html>
<head>
<meta charset="utf-8">
</head>
<body>
<script type="text/javascript" src="../dist/orbitdb.min.js" charset="utf-8"></script>
<script type="text/javascript" src="../node_modules/logplease/dist/logplease.min.js" charset="utf-8"></script>
<script type="text/javascript" src="../node_modules/ipfs/dist/index.min.js" charset="utf-8"></script>
<script type="text/javascript">
const ipfs = window.Ipfs();
OrbitDB.connect('localhost:3333', 'user1', '', ipfs)
.then((orbit) => orbit.kvstore('test'))
.then((db) => db.put('hello', 'world'))
.then((res) => {
const result = db.get(key)
console.log(result)
})
</script>
</body>
</html>
```
### Node.js examples
Before running the examples, install dependencies with:
```
npm install
```
Key-Value store [example](https://github.com/haadcode/orbit-db/blob/master/examples/keyvalue.js):
```
node examples/keyvalue.js <host:port> <username> <channel> <key> <value>
```
Event log [example](https://github.com/haadcode/orbit-db/blob/master/examples/eventlog.js) (run several in separate shells):
```
node examples/eventlog.js <host:port> <username> <channel> <data> <interval in ms>
```
Benchmark writes:
```bash ```bash
node examples/benchmark.js <host:port> <username> <channel>; npm install
npm run examples:node
```
See detailed [example](https://github.com/haadcode/orbit-db/blob/master/examples/eventlog.js) and run it with:
```bash
node examples/eventlog.js
```
```javascript
'use strict'
const IpfsDaemon = require('ipfs-daemon')
const OrbitDB = require('orbit-db')
IpfsDaemon()
.then((res) => {
const orbitdb = new OrbitDB(res.ipfs)
const db = orbitdb.eventlog("|orbit-db|examples|eventlog-example")
const creatures = ['🐙', '🐷', '🐬', '🐞', '🐈', '🙉', '🐸', '🐓']
const query = () => {
const index = Math.floor(Math.random() * creatures.length)
db.add(creatures[index])
.then(() => {
const latest = db.iterator({ limit: 5 }).collect()
let output = ``
output += `---------------------------------------------------\n`
output += `Latest Visitors\n`
output += `---------------------------------------------------\n`
output += latest.reverse().map((e) => e.payload.value).join('\n') + `\n`
console.log(output)
})
.catch((e) => {
console.error(e.stack)
})
}
setInterval(query, 1000)
})
.catch((err) => console.error(err))
``` ```
## API ## API
**NOTE: the API documentation is currently out of date. It will be updated soon!**
_See usage example below_ **TODO**
_OrbitDB calls its namespaces channels. A channel is similar to "table", "keyspace", "topic", "feed" or "collection" in other db systems._ - [orbit-db-kvstore](https://github.com/haadcode/orbit-db-kvstore)
- put(key, value)
- set(key, value)
- get(key)
- [orbit-db-eventstore](https://github.com/haadcode/orbit-db-eventstore)
- add(value)
- get(hash)
- iterator(options)
- [orbit-db-feedstore](https://github.com/haadcode/orbit-db-feedstore)
- add(value)
- del(hash)
- get(hash)
- iterator(options)
- [orbit-db-counterstore](https://github.com/haadcode/orbit-db-counterstore)
- inc([value])
connect(<host:port>, username, password) ## Development
channel(name, password)
.add(data: String) // Insert an event to a channel, returns <ipfs-hash> of the event
.iterator([options]) // Returns an iterator of events
// options : {
// gt: <ipfs-hash>, // Return events newer than <ipfs-hash>
// gte: <ipfs-hash>, // Return events newer then <ipfs-hash> (inclusive)
// lt: <ipfs-hash>, // Return events older than <ipfs-hash>
// lte: <ipfs-hash>, // Return events older than <ipfs-hash> (inclusive)
// limit: -1, // Number of events to return, -1 returns all, default 1
// reverse: true // Return items oldest first, default latest first
// }
.put(key, data: String) // Insert (key,value) to a channel
.get(key) // Retrieve value
.del({ key: <key or hash> }) // Remove entry
.delete() // Deletes the channel, all data will be "removed" (unassociated with the channel, actual data is not deleted from ipfs)
## Usage
```javascript
const async = require('asyncawait/async');
const ipfsAPI = require('ipfs-api');
const OrbitDB = require('orbit-db');
// local ipfs daemon
const ipfs = ipfsAPI();
async(() => {
// Connect
const orbit = await(OrbitClient.connect('localhost:3333', 'usernamne', '', ipfs));
/* Event Log */
const eventlog = orbit.eventlog('eventlog test');
const hash = await(eventlog.add('hello')); // <ipfs-hash>
// Remove event
await(eventlog.remove(hash));
// Iterator options
const options = { limit: -1 }; // fetch all messages
// Get events
const iter = eventlog.iterator(options); // Symbol.iterator
const next = iter.next(); // { value: <item>, done: false|true}
// OR:
// var all = iter.collect(); // returns all elements as an array
// OR:
// for(let i of iter)
// console.log(i.hash, i.item);
/* Delete database locally */
eventlog.delete();
/* KV Store */
const kvstore = orbit.kvstore('kv test');
await(kvstore.put('key1', 'hello world'));
kvstore.get('key1'); // returns "hello world"
await(kvstore.del('key1'));
})();
```
### Development
#### Run Tests #### Run Tests
```bash ```bash
npm test npm test
``` ```
Keep tests running while development:
```bash
mocha -w
```
#### Build distributables #### Build distributables
```bash ```bash
npm install
npm run build npm run build
``` ```
## Background
**TODO**
Check out a visualization of the data flow at https://github.com/haadcode/proto2
Live demo: http://celebdil.benet.ai:8080/ipfs/Qmezm7g8mBpWyuPk6D84CNcfLKJwU6mpXuEN5GJZNkX3XK/
![Screenshot](https://raw.githubusercontent.com/haadcode/proto2/master/screenshot.png)
**TODO: list of modules used, orbit-db-pubsub, etc.**
## Contributing
**TODO**
## License
MIT ©️ 2016, Haadcode

7
circle.yml
View File

@ -1,8 +1,3 @@
machine: machine:
node: node:
version: 4.3.1 version: 6.1.0
services:
- redis
dependencies:
pre:
- npm install -g npm@3.x.x

42646
dist/orbitdb.js vendored
View File

File diff suppressed because it is too large Load Diff

68
dist/orbitdb.min.js vendored
View File

File diff suppressed because one or more lines are too long

107
docs/architecture.md
View File

@ -1,107 +0,0 @@
***WIP***
### Structure
The database has one log (OrbitList) for each *channel*. *Channel* is comparable to a *"topic"* or *"table"*.
```
DB
|-- channel 1
| |-- list 1
| | |-- node 1
| | |-- operation
| | |-- value
| |-- list 2
|-- channel 2
|-- list 3
...
```
### Operation Logs
- Each *channel* is saved as a log of operations: add, put, del
- Operations are stored in an append-only log linked list
- Each node in the linked list points to the previous node
- Event log: take latest add or del operation for <hash>
- JV-store: take last put or del operation for <key>
### CRDTs
- orbit-db is a CmRDT and implements an LWW-element-set
- Operation-based CRDT
- Uses Merkle Trees for partial ordering
### add/put IO:
==> Not expensive
1. Create POST for the content
`ipfs object get QmZzic86oN7B5GMMYnTFvZMyDMjkKEBXe5SYsryXPeoGdb?`
```json
{
"content": "hello 1",
"ts": 1457703735759,
"meta": {
"type": "text",
"size": 7,
"ts": 1457703735759
}
}
```
2. Create POST for the DB operation
`ipfs object get QmZzBNrUiYATJ4aicPTbupnEUWH3AHDmfBbDTQK3fhhYDE`
```json
{
"op": "ADD",
"key": "QmZzic86oN7B5GMMYnTFvZMyDMjkKEBXe5SYsryXPeoGdb",
"value": "QmZzic86oN7B5GMMYnTFvZMyDMjkKEBXe5SYsryXPeoGdb",
"meta": {
"type": "orbit-db-op",
"size": 15,
"ts": 1457703735779
},
"by": "userA"
}
```
3. Create ipfs object for the nodet
`ipfs object get QmX2Jq1JHmgjgM3YVuHyGSRpUWMDbv4PuRhqBhsZqDmagD`
```json
{
"id": "userA",
"seq": 0,
"ver": 1,
"data": "QmZzBNrUiYATJ4aicPTbupnEUWH3AHDmfBbDTQK3fhhYDE",
"next": {
"userA.0.0": "QmXUTgYPG4cSaHW8dKggJRfLvPWjaDktWyRAgn5NPwTqtz"
}
}
```
4. Create ipfs object for the current list
`ipfs object get Qmb2rpex9TdmoXvwLKLL24atWa2HfPbArobN7XiBAFvmZ9`
```json
{
"id": "userA",
"seq": 1,
"ver": 0,
"items": {
"userA.0.0": "QmXUTgYPG4cSaHW8dKggJRfLvPWjaDktWyRAgn5NPwTqtz",
"userA.0.1": "QmX2Jq1JHmgjgM3YVuHyGSRpUWMDbv4PuRhqBhsZqDmagD"
}
}
```
5. Pubsub.publish (send to socket, network IO)
```json
{
channel: '<channel name>',
hash: 'Qmb2rpex9TdmoXvwLKLL24atWa2HfPbArobN7XiBAFvmZ9'
}
```
### get IO:
==> A little expensive
1. Payload (get from ipfs-hash, network IO)
### sync/merge IO:
==> Expensive!
TODO

141
docs/ordering.md
View File

@ -1,141 +0,0 @@
# Odering in OrbitList
Ordering in OrbitList is done with Interval Tree Clocks (https://github.com/ricardobcl/Interval-Tree-Clocks). Each node tracks a global sequence *seq* and a local version *ver*. When an item is added to a list, local version *ver* is increased by 1. Whenever a list is *joined* with another list (eg. upon sync), the global sequence *seq* is set to *Max(global.seq, local.seq)*. When adding an item to the list, the item has a reference to all *heads* where *head* is an item that is not referenced by any other item in the list.
### Graph
```
A B C
0.0
|
0.0 0.1
| |
0.1 0.2
\ / <--- Sync B with A and C
1.0
|
1.1
/ <--- Sync A with B
2.0
\
\ <--- Sync C with A
\
3.0
<--- Sync ALL
```
### Evolution of the state
Initial state:
```
A = []
B = []
C = []
```
***Two nodes add events concurrently***
Add items to A:
```
listA.add("mango")
listA.add("banana")
// "A": [
// { "id": "A", "seq": 0, "ver": 0, "prev": null}
// { "id": "A", "seq": 0, "ver": 1, "prev": ["A.0.0"]}
// ]
```
Add items to C:
```
listC.add("apple")
listC.add("strawberry")
listC.add("orange")
// "C": [
// { "id": "C", "seq": 0, "ver": 0, "prev": null}
// { "id": "C", "seq": 0, "ver": 1, "prev": ["C.0.0"]}
// { "id": "C", "seq": 0, "ver": 2, "prev": ["C.0.1"]}
// ]
```
B receives a 'sync' from A and C:
```
B.join(A)
B.join(C)
```
Add items to B:
```
listB.add("pineapple")
listB.add("papaya")
// "B": [
// { "id": "A", "seq": 0, "ver": 0, "prev": null}
// { "id": "A", "seq": 0, "ver": 1, "prev": ["A.0.0"]}
// { "id": "C", "seq": 0, "ver": 0, "prev": null}
// { "id": "C", "seq": 0, "ver": 1, "prev": ["C.0.0"]}
// { "id": "C", "seq": 0, "ver": 2, "prev": ["C.0.1"]}
// { "id": "B", "seq": 1, "ver": 0, "prev": ["A.0.1", "C.0.2"]}
// { "id": "B", "seq": 1, "ver": 1, "prev": ["B.1.0"]}
// ]
```
A receives a 'sync' from B:
```
A.join(B)
```
```
listA.add("kiwi")
// "A": [
// { "id": "A", "seq": 0, "ver": 0, "prev": null}
// { "id": "A", "seq": 0, "ver": 1, "prev": ["A.0.0"]}
// { "id": "C", "seq": 0, "ver": 0, "prev": null}
// { "id": "C", "seq": 0, "ver": 1, "prev": ["C.0.0"]}
// { "id": "C", "seq": 0, "ver": 2, "prev": ["C.0.1"]}
// { "id": "B", "seq": 1, "ver": 0, "prev": ["A.0.1", "C.0.2"]}
// { "id": "B", "seq": 1, "ver": 1, "prev": ["B.1.0"]}
// { "id": "A", "seq": 2, "ver": 0, "prev": ["B.1.1"]}
// ]
```
C receives a 'sync' from A:
```
C.join(A)
```
```
listC.add("blueberry")
// "C": [
// { "id": "A", "seq": 0, "ver": 0, "prev": null}
// { "id": "A", "seq": 0, "ver": 1, "prev": ["A.0.0"]}
// { "id": "C", "seq": 0, "ver": 0, "prev": null}
// { "id": "C", "seq": 0, "ver": 1, "prev": ["C.0.0"]}
// { "id": "C", "seq": 0, "ver": 2, "prev": ["C.0.1"]}
// { "id": "B", "seq": 1, "ver": 0, "prev": ["A.0.1", "C.0.2"]}
// { "id": "B", "seq": 1, "ver": 1, "prev": ["B.1.0"]}
// { "id": "A", "seq": 2, "ver": 0, "prev": ["B.1.1"]}
// { "id": "C", "seq": 3, "ver": 0, "prev": ["A.2.0"]}
// ]
```
A receives a 'sync' from C, B receives a 'sync' from C:
```
A.join(C)
B.join(C)
```
Converged data set:
```json
{ "id": "A", "seq": 0, "ver": 0, "prev": null},
{ "id": "A", "seq": 0, "ver": 1, "prev": ["A.0.0"]},
{ "id": "C", "seq": 0, "ver": 0, "prev": null},
{ "id": "C", "seq": 0, "ver": 1, "prev": ["C.0.0"]},
{ "id": "C", "seq": 0, "ver": 2, "prev": ["C.0.1"]},
{ "id": "B", "seq": 1, "ver": 0, "prev": ["A.0.1", "C.0.2"]},
{ "id": "B", "seq": 1, "ver": 1, "prev": ["B.1.0"]}
{ "id": "A", "seq": 2, "ver": 0, "prev": ["B.1.1"]}
{ "id": "C", "seq": 3, "ver": 0, "prev": ["A.2.0]"]}
```

19
examples/Timer.js
View File

@ -1,19 +0,0 @@
'use strict';
class Timer {
constructor(startImmediately) {
this.startTime = startImmediately ? new Date().getTime() : null;
this.endTime = null;
}
start() {
this.startTime = new Date().getTime();
}
stop() {
this.endTime = new Date().getTime();
return (this.endTime - this.startTime);
}
}
module.exports = Timer;

110
examples/benchmark.js
View File

@ -1,89 +1,49 @@
'use strict'; 'use strict'
// const ipfsd = require('ipfsd-ctl'); const IpfsApi = require('ipfs-api')
const IPFS = require('ipfs') const OrbitDB = require('../src/OrbitDB')
const ipfsd = require('ipfsd-ctl');
const OrbitDB = require('../src/OrbitDB');
const Timer = require('./Timer');
// usage: benchmark.js <network hash> <username> <channel>; const username = process.argv[2] ? process.argv[2] : 'testrunner'
const channelName = process.argv[3] ? process.argv[3] : 'c1'
// orbit-server
const network = 'localhost:3333';
const username = process.argv[2] ? process.argv[2] : 'testrunner';
const password = '';
const channelName = process.argv[3] ? process.argv[3] : 'c1';
const startIpfs = () => {
return new Promise((resolve, reject) => {
ipfsd.disposableApi((err, ipfs) => {
if(err) console.error(err);
resolve(ipfs);
});
// ipfsd.local((err, node) => {
// if(err) reject(err);
// node.startDaemon((err, ipfs) => {
// if(err) reject(err);
// resolve(ipfs);
// });
// });
// const ipfs = new IPFS('/tmp/benchmark')
// ipfs.goOnline(() => {
// resolve(ipfs)
// })
});
};
const util = require('util');
// Metrics // Metrics
let totalQueries = 0; let totalQueries = 0
let seconds = 0; let seconds = 0
let queriesPerSecond = 0; let queriesPerSecond = 0
let lastTenSeconds = 0; let lastTenSeconds = 0
let store;
// Main loop
const queryLoop = (db) => { const queryLoop = (db) => {
// let timer = new Timer();
// timer.start();
db.add(username + totalQueries).then(() => { db.add(username + totalQueries).then(() => {
// console.log(`${timer.stop(true)} ms - ${process._getActiveRequests().length} ${process._getActiveHandles().length}`); totalQueries ++
// console.log(util.inspect(process.memoryUsage())); lastTenSeconds ++
totalQueries ++; queriesPerSecond ++
lastTenSeconds ++; process.nextTick(() => queryLoop(db))
queriesPerSecond ++; })
process.nextTick(() => queryLoop(db)); }
});
};
let run = (() => { let run = (() => {
// Connect // Connect
console.log(`Connecting...`) console.log(`Connecting...`)
startIpfs() const ipfs = IpfsApi('localhost', '5001')
.then((ipfs) => OrbitDB.connect(network, username, password, ipfs)) const orbit = new OrbitDB(ipfs, 'benchmark')
.then((orbit) => orbit.eventlog(channelName)) const db = orbit.eventlog(channelName)
.then((db) => {
queryLoop(db); // Metrics output
setInterval(() => {
seconds ++
if(seconds % 10 === 0) {
console.log(`--> Average of ${lastTenSeconds/10} q/s in the last 10 seconds`)
if(lastTenSeconds === 0)
throw new Error("Problems!")
lastTenSeconds = 0
}
console.log(`${queriesPerSecond} queries per second, ${totalQueries} queries in ${seconds} seconds`)
queriesPerSecond = 0
}, 1000)
// Metrics output // Start
setInterval(() => { queryLoop(db)
seconds ++; })()
if(seconds % 10 === 0) {
console.log(`--> Average of ${lastTenSeconds/10} q/s in the last 10 seconds`);
if(lastTenSeconds === 0)
throw new Error("Problems!");
lastTenSeconds = 0;
}
console.log(`${queriesPerSecond} queries per second, ${totalQueries} queries in ${seconds} seconds`);
queriesPerSecond = 0;
}, 1000);
})
.catch((e) => {
console.error("error:", e);
console.error(e.stack);
process.exit(1);
})
})();
module.exports = run; module.exports = run

12
examples/browser/browser.html
View File

@ -6,10 +6,8 @@
<div id="result">Loading...</div> <div id="result">Loading...</div>
<script type="text/javascript" src="../../dist/orbitdb.min.js" charset="utf-8"></script> <script type="text/javascript" src="../../dist/orbitdb.min.js" charset="utf-8"></script>
<script type="text/javascript" src="../../node_modules/logplease/dist/logplease.min.js" charset="utf-8"></script>
<script type="text/javascript" src="../../node_modules/ipfs/dist/index.min.js" charset="utf-8"></script> <script type="text/javascript" src="../../node_modules/ipfs/dist/index.min.js" charset="utf-8"></script>
<script type="text/javascript"> <script type="text/javascript">
const logger = Logger.create("orbit-db example", { color: Logger.Colors.Green, showTimestamp: false, showLevel: false })
const network = '178.62.241.75:3333' const network = '178.62.241.75:3333'
const username = 'user1' const username = 'user1'
const password = '' const password = ''
@ -26,7 +24,7 @@
const startTime = new Date().getTime() const startTime = new Date().getTime()
db.put(key, value + " " + count).then((res) => { db.put(key, value + " " + count).then((res) => {
const endTime = new Date().getTime() const endTime = new Date().getTime()
logger.debug(`db.put (#${count}) took ${(endTime - startTime)} ms\n`) console.log(`db.put (#${count}) took ${(endTime - startTime)} ms\n`)
count ++ count ++
const result = db.get(key) const result = db.get(key)
@ -38,12 +36,12 @@
---------------------------------------------------` ---------------------------------------------------`
elm.innerHTML = output.split("\n").join("<br>") elm.innerHTML = output.split("\n").join("<br>")
logger.debug(output) console.log(output)
}).catch((e) => logger.error(e)) }).catch((e) => console.error(e))
}; };
setInterval(query, 1000) setInterval(query, 1000)
}).catch((e) => logger.error(e)) }).catch((e) => console.error(e))
}).catch((e) => logger.error(e)) }).catch((e) => console.error(e))
</script> </script>
</body> </body>
</html> </html>

45
examples/browser/browser.js
View File

@ -1,45 +0,0 @@
'use strict';
const IPFS = require('exports?Ipfs!ipfs/dist/index.js')
const Logger = require('logplease')
const logger = Logger.create("orbit-db example", { color: Logger.Colors.Green, showTimestamp: false, showLevel: false })
const OrbitDB = require('../../src/OrbitDB')
const network = '178.62.241.75:3333'
const username = 'user1'
const password = ''
const channel = 'browsertest1'
const key = 'greeting'
const value = 'Hello world'
try {
const elm = document.getElementById("result")
const ipfs = new IPFS()
OrbitDB.connect(network, username, password, ipfs).then((orbit) => {
orbit.kvstore(channel).then((db) => {
let count = 1
const query = () => {
const startTime = new Date().getTime()
db.put(key, value + " " + count).then((res) => {
const endTime = new Date().getTime()
logger.debug(`db.put (#${count}) took ${(endTime - startTime)} ms\n`)
count ++
const result = db.get(key)
const output = `
---------------------------------------------------
Key | Value
---------------------------------------------------
${key} | ${result}
---------------------------------------------------`
elm.innerHTML = output.split("\n").join("<br>")
logger.debug(output)
}).catch((e) => logger.error(e.stack))
}
setInterval(query, 1000)
})
})
} catch(e) {
logger.error(e.stack)
}

71
examples/browser/index.js Normal file
View File

@ -0,0 +1,71 @@
'use strict'
const IpfsApi = require('exports?IpfsApi!ipfs-api/dist/index.js')
const OrbitDB = require('../../src/OrbitDB')
const username = new Date().getTime()
const channel = 'browser-example'
const key = 'greeting'
try {
const elm = document.getElementById("result")
const ipfs = IpfsApi('localhost', '5001')
const orbit = new OrbitDB(ipfs, username)
const db = orbit.kvstore(channel)
const log = orbit.eventlog(channel + ".log")
const counter = orbit.counter(channel + ".count")
const creatures = ['👻', '🐙', '🐷', '🐬', '🐞', '🐈', '🙉', '🐸', '🐓']
let count = 1
const query = () => {
const startTime = new Date().getTime()
const idx = Math.floor(Math.random() * creatures.length)
// Set a key-value pair
db.put(key, "db.put #" + count + " - GrEEtinGs to " + creatures[idx])
.then((res) => {
const endTime = new Date().getTime()
console.log(`db.put (#${count}) took ${(endTime - startTime)} ms\n`)
count ++
})
.then(() => counter.inc()) // Increase the counter by one
.then(() => log.add(creatures[idx])) // Add an event to 'latest visitors' log
.then(() => {
const result = db.get(key)
const latest = log.iterator({ limit: 5 }).collect()
const count = counter.value()
const output =
`<b>Key-Value Store</b>
---------------------------------------------------
Key | Value
---------------------------------------------------
${key} | ${result}
---------------------------------------------------
<b>Eventlog</b>
---------------------------------------------------
Latest Visitors
---------------------------------------------------
${latest.reverse().map((e) => e.payload.value + " (" + e.payload.from + ") at" + new Date(e.payload.meta.ts).toISOString()).join('\n')}
<b>Counter</b>
---------------------------------------------------
Visitor Count: ${count}
---------------------------------------------------
`
elm.innerHTML = output.split("\n").join("<br>")
})
.catch((e) => {
elm.innerHTML = "<i>" + e.message + "</i><br><br>" + "Make sure you have an IPFS daemon running at localhost:5001"
console.error(e.stack)
})
}
setInterval(query, 1000)
} catch(e) {
console.error(e.stack)
elm.innerHTML = e.message
}

6
examples/eventlog-reader.js
View File

@ -9,7 +9,7 @@ const Timer = require('./Timer');
// usage: reader.js <network hash> <username> <channel> <interval in ms> // usage: reader.js <network hash> <username> <channel> <interval in ms>
// orbit-server // orbit-server
const network = 'QmYPobvobKsyoCKTw476yTui611XABf927KxUPCf4gRLRr'; // 'localhost:3333' const network = '178.62.241.75:3333'; // 'localhost:3333'
const username = process.argv[2] ? process.argv[2] : 'testrunner'; const username = process.argv[2] ? process.argv[2] : 'testrunner';
const password = ''; const password = '';
const channelName = process.argv[3] ? process.argv[3] : 'c2'; const channelName = process.argv[3] ? process.argv[3] : 'c2';
@ -28,7 +28,7 @@ let run = (async(() => {
try { try {
const ipfs = await(startIpfs()); const ipfs = await(startIpfs());
const orbit = await(OrbitDB.connect(network, username, password, ipfs)); const orbit = await(OrbitDB.connect(network, username, password, ipfs));
const db = await(orbit.eventlog(channelName)); const db = orbit.eventlog(channelName);
let count = 1; let count = 1;
let running = false; let running = false;
@ -42,7 +42,7 @@ let run = (async(() => {
console.log("---------------------------------------------------") console.log("---------------------------------------------------")
console.log("Timestamp | Value") console.log("Timestamp | Value")
console.log("---------------------------------------------------") console.log("---------------------------------------------------")
console.log(items.map((e) => `${e.meta.ts} | ${e.value}`).join("\n")); console.log(items.map((e) => `${e.payload.meta.ts} | ${e.payload.value}`).join("\n"));
console.log("---------------------------------------------------") console.log("---------------------------------------------------")
console.log(`Query #${count} took ${timer2.stop(true)} ms\n`); console.log(`Query #${count} took ${timer2.stop(true)} ms\n`);

95
examples/eventlog.js
View File

@ -1,65 +1,42 @@
'use strict'; 'use strict'
const async = require('asyncawait/async'); const IpfsDaemon = require('ipfs-daemon')
const await = require('asyncawait/await'); const OrbitDB = require('../src/OrbitDB')
const ipfsd = require('ipfsd-ctl');
const OrbitDB = require('../src/OrbitDB');
const Timer = require('./Timer');
// usage: reader.js <network hash> <username> <channel> <data> <interval in ms> const userId = Math.floor(Math.random() * 100)
const conf = {
IpfsDataDir: '/tmp/' + userId,
Addresses: {
API: '/ip4/127.0.0.1/tcp/0',
Swarm: ['/ip4/0.0.0.0/tcp/0'],
Gateway: '/ip4/0.0.0.0/tcp/0'
},
}
// orbit-server IpfsDaemon(conf)
const network = 'QmYPobvobKsyoCKTw476yTui611XABf927KxUPCf4gRLRr'; // 'localhost:3333' .then((res) => {
const username = process.argv[2] ? process.argv[2] : 'testrunner'; const orbitdb = new OrbitDB(res.ipfs)
const password = ''; const db = orbitdb.eventlog("|orbit-db|examples|eventlog-example")
const channelName = process.argv[3] ? process.argv[3] : 'c2';
const prefix = process.argv[4] ? process.argv[4] : 'Hello';
const startIpfs = () => { const creatures = ['🐙', '🐷', '🐬', '🐞', '🐈', '🙉', '🐸', '🐓']
return new Promise((resolve, reject) => {
ipfsd.disposableApi((err, ipfs) => {
if(err) console.error(err);
resolve(ipfs);
});
});
};
let run = (async(() => { const query = () => {
try { const index = Math.floor(Math.random() * creatures.length)
const ipfs = await(startIpfs()); db.add({ avatar: creatures[index], userId: userId })
const orbit = await(OrbitDB.connect(network, username, password, ipfs)); .then(() => {
const db = await(orbit.eventlog(channelName)); const latest = db.iterator({ limit: 5 }).collect()
let output = ``
output += `--------------------\n`
output += `Latest Visitors\n`
output += `--------------------\n`
output += latest.reverse().map((e) => e.payload.value.avatar + " (userId: " + e.payload.value.userId + ")").join('\n') + `\n`
console.log(output)
})
.catch((e) => {
console.error(e.stack)
})
}
let count = 1; setInterval(query, 1000)
let running = false; })
.catch((err) => console.error(err))
setInterval(async(() => {
if(!running) {
running = true;
let timer = new Timer(true);
await(db.add(prefix + count));
console.log(`Query #${count} took ${timer.stop(true)} ms\n`);
let timer2 = new Timer(true);
let items = db.iterator({ limit: -1 }).collect();
console.log("----------------------------------------------------------------------------------------")
console.log("Hash | Timestamp | Value")
console.log("----------------------------------------------------------------------------------------")
console.log(items.map((e) => `${e.hash} | ${e.meta.ts} | ${e.value}`).join("\n"));
console.log("----------------------------------------------------------------------------------------")
console.log(`Query 2 #${count} took ${timer2.stop(true)} ms\n`);
running = false;
count ++;
}
}), process.argv[6] ? process.argv[6] : 1000);
} catch(e) {
console.error(e.stack);
console.log("Exiting...")
process.exit(1);
}
}))();
module.exports = run;

13
examples/start-daemon.js Normal file
View File

@ -0,0 +1,13 @@
const IpfsDaemon = require('ipfs-daemon')
module.exports = IpfsDaemon({
IpfsDataDir: './tmp',
API: {
HTTPHeaders: {
"Access-Control-Allow-Origin": ['*'],
"Access-Control-Allow-Methods": ["PUT", "GET", "POST"],
"Access-Control-Allow-Credentials": ["true"]
}
}
})
.then((res) => console.log("started"))
.catch((err) => console.error(err))

28
package.json
View File

@ -1,6 +1,6 @@
{ {
"name": "orbit-db", "name": "orbit-db",
"version": "0.11.1", "version": "0.11.2",
"description": "Distributed p2p database on IPFS", "description": "Distributed p2p database on IPFS",
"author": "Haad", "author": "Haad",
"license": "MIT", "license": "MIT",
@ -9,35 +9,41 @@
"url": "https://github.com/haadcode/orbit-db" "url": "https://github.com/haadcode/orbit-db"
}, },
"engines": { "engines": {
"node": "^4.x.x" "node": "^6.x.x"
}, },
"main": "src/OrbitDB.js", "main": "src/OrbitDB.js",
"dependencies": { "dependencies": {
"logplease": "^1.2.7", "logplease": "^1.2.7",
"orbit-db-counterstore": "0.1.1", "orbit-db-counterstore": "0.1.2",
"orbit-db-eventstore": "0.1.1", "orbit-db-eventstore": "0.1.4",
"orbit-db-feedstore": "0.1.1", "orbit-db-feedstore": "0.1.3",
"orbit-db-kvstore": "0.1.1", "orbit-db-kvstore": "0.1.2",
"socket.io-client": "^1.4.5" "orbit-db-pubsub": "0.0.4"
}, },
"devDependencies": { "devDependencies": {
"asyncawait": "^1.0.6", "asyncawait": "^1.0.6",
"babel-core": "^6.11.4", "babel-core": "^6.11.4",
"babel-loader": "^6.2.4", "babel-loader": "^6.2.4",
"babel-plugin-transform-runtime": "^6.8.0", "babel-plugin-transform-runtime": "^6.8.0",
"babel-polyfill": "^6.16.0",
"babel-preset-es2015": "^6.6.0", "babel-preset-es2015": "^6.6.0",
"bluebird": "^3.4.6",
"exports-loader": "^0.6.3", "exports-loader": "^0.6.3",
"ipfs": "^0.13.0", "fs-pull-blob-store": "^0.3.0",
"ipfs-api": "^6.0.3", "html5-fs": "https://github.com/haadcode/html5-fs.git",
"ipfsd-ctl": "^0.14.0", "ipfs-daemon": "0.0.3",
"ipfs-test-apis": "0.0.1",
"json-loader": "^0.5.4", "json-loader": "^0.5.4",
"lodash": "^4.3.0", "lodash": "^4.3.0",
"mocha": "^2.4.5", "mocha": "^2.4.5",
"orbit-server": "^0.2.3",
"stream-http": "^2.2.1", "stream-http": "^2.2.1",
"webpack": "^2.1.0-beta.7" "webpack": "^2.1.0-beta.7"
}, },
"scripts": { "scripts": {
"examples": "npm run example:node",
"examples:node": "LOG=debug node examples/eventlog.js",
"examples:browser": "open examples/browser/index.html && LOG=debug node examples/start-daemon.js",
"postinstall": "./scripts/post_install.sh",
"test": "mocha", "test": "mocha",
"build": "npm run build:dist && npm run build:minified && npm run build:examples", "build": "npm run build:dist && npm run build:minified && npm run build:examples",
"build:dist": "./node_modules/.bin/webpack --config webpack.config.js", "build:dist": "./node_modules/.bin/webpack --config webpack.config.js",

BIN
screenshots/orbit-db-demo1.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 136 KiB

BIN
screenshots/orbit-db-demo3.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 131 KiB

0
screenshot.png → screenshots/screenshot.png
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 28 KiB

After

Width:  |  Height:  |  Size: 28 KiB

4
scripts/post_install.sh Executable file
View File

@ -0,0 +1,4 @@
# !/bin/sh
rm -rf ./node_modules/ipfs/node_modules/ipfs-api
rm -rf ./node_modules/ipfsd-ctl/node_modules/go-ipfs-dep
rm -rf ./node_modules/ipfsd-ctl/node_modules/ipfs-api

152
src/OrbitDB.js
View File

@ -1,66 +1,63 @@
'use strict'; 'use strict'
const EventEmitter = require('events').EventEmitter; const EventEmitter = require('events').EventEmitter
const Logger = require('logplease'); const EventStore = require('orbit-db-eventstore')
const logger = Logger.create("orbit-db", { color: Logger.Colors.Magenta }); const FeedStore = require('orbit-db-feedstore')
const EventStore = require('orbit-db-eventstore'); const KeyValueStore = require('orbit-db-kvstore')
const FeedStore = require('orbit-db-feedstore'); const CounterStore = require('orbit-db-counterstore')
const KeyValueStore = require('orbit-db-kvstore'); const Pubsub = require('orbit-db-pubsub')
const CounterStore = require('orbit-db-counterstore'); const Cache = require('./Cache')
const PubSub = require('./PubSub');
const Cache = require('./Cache'); const defaultNetworkName = 'Orbit DEV Network'
class OrbitDB { class OrbitDB {
constructor(ipfs) { constructor(ipfs, id = 'default', options = {}) {
this._ipfs = ipfs; this._ipfs = ipfs
this._pubsub = null; this._pubsub = options && options.broker ? new options.broker(ipfs) : new Pubsub(ipfs)
this.user = null; this.user = { id: id }
this.network = null; this.network = { name: defaultNetworkName }
this.events = new EventEmitter(); this.events = new EventEmitter()
this.stores = {}; this.stores = {}
} }
/* Databases */ /* Databases */
feed(dbname, options) { feed(dbname, options) {
return this._createStore(FeedStore, dbname, options); return this._createStore(FeedStore, dbname, options)
} }
eventlog(dbname, options) { eventlog(dbname, options) {
return this._createStore(EventStore, dbname, options); return this._createStore(EventStore, dbname, options)
} }
kvstore(dbname, options) { kvstore(dbname, options) {
return this._createStore(KeyValueStore, dbname, options); return this._createStore(KeyValueStore, dbname, options)
} }
counter(dbname, options) { counter(dbname, options) {
return this._createStore(CounterStore, dbname, options); return this._createStore(CounterStore, dbname, options)
} }
disconnect() { disconnect() {
if(this._pubsub) this._pubsub.disconnect(); if (this._pubsub) this._pubsub.disconnect()
this.events.removeAllListeners('data'); this.events.removeAllListeners('data')
Object.keys(this.stores).map((e) => this.stores[e]).forEach((store) => { Object.keys(this.stores).map((e) => this.stores[e]).forEach((store) => {
store.events.removeAllListeners('data'); store.events.removeAllListeners('data')
store.events.removeAllListeners('write'); store.events.removeAllListeners('write')
store.events.removeAllListeners('close'); store.events.removeAllListeners('close')
}); })
this.stores = {}; this.stores = {}
this.user = null; this.user = null
this.network = null; this.network = null
} }
_createStore(Store, dbname, options) { /* Private methods */
if(!options) options = {}; _createStore(Store, dbname, options = { subscribe: true }) {
const replicate = options.subscribe !== undefined ? options.subscribe : true; const store = new Store(this._ipfs, this.user.id, dbname, options)
const store = new Store(this._ipfs, this.user.username, dbname, options); this.stores[dbname] = store
this.stores[dbname] = store; return this._subscribe(store, dbname, options.subscribe, options)
return this._subscribe(store, dbname, replicate, options);
} }
_subscribe(store, dbname, subscribe, options) { _subscribe(store, dbname, subscribe = true, options) {
if(subscribe === undefined) subscribe = true
store.events.on('data', this._onData.bind(this)) store.events.on('data', this._onData.bind(this))
store.events.on('write', this._onWrite.bind(this)) store.events.on('write', this._onWrite.bind(this))
store.events.on('close', this._onClose.bind(this)) store.events.on('close', this._onClose.bind(this))
@ -68,37 +65,33 @@ class OrbitDB {
if(subscribe && this._pubsub) if(subscribe && this._pubsub)
this._pubsub.subscribe(dbname, this._onMessage.bind(this), this._onConnected.bind(this), store.options.maxHistory > 0) this._pubsub.subscribe(dbname, this._onMessage.bind(this), this._onConnected.bind(this), store.options.maxHistory > 0)
else else
store.loadHistory().catch((e) => logger.error(e.stack)); store.loadHistory().catch((e) => console.error(e.stack))
Cache.loadCache(options.cacheFile).then(() => { Cache.loadCache(options.cacheFile).then(() => {
const hash = Cache.get(dbname) const hash = Cache.get(dbname)
store.loadHistory(hash).catch((e) => logger.error(e.stack)) store.loadHistory(hash).catch((e) => console.error(e.stack))
}) })
return store return store
} }
/* Connected to the message broker */ /* Connected to the message broker */
_onConnected(dbname, hash) { _onConnected(dbname, hash) {
// console.log(".CONNECTED", dbname, hash, this.user.username); // console.log(".CONNECTED", dbname, hash)
const store = this.stores[dbname] const store = this.stores[dbname]
store.loadHistory(hash).catch((e) => logger.error(e.stack)) store.loadHistory(hash).catch((e) => console.error(e.stack))
} }
/* Replication request from the message broker */ /* Replication request from the message broker */
_onMessage(dbname, hash) { _onMessage(dbname, hash) {
// console.log(".MESSAGE", dbname, hash, this.user.username) // console.log(".MESSAGE", dbname, hash)
const store = this.stores[dbname] const store = this.stores[dbname]
store.sync(hash) store.sync(hash)
.then((res) => Cache.set(dbname, hash)) .then((res) => Cache.set(dbname, hash))
.catch((e) => logger.error(e.stack)) .catch((e) => console.error(e.stack))
} }
/* Data events */ /* Data events */
_onWrite(dbname, hash) { _onWrite(dbname, hash) {
// 'New entry written to database...', after adding a new db entry locally // 'New entry written to database...', after adding a new db entry locally
// console.log(".WRITE", dbname, hash, this.user.username) // console.log(".WRITE", dbname, hash, this.user.username)
@ -109,7 +102,7 @@ class OrbitDB {
_onData(dbname, item) { _onData(dbname, item) {
// 'New database entry...', after a new entry was added to the database // 'New database entry...', after a new entry was added to the database
// console.log(".SYNCED", dbname, items.length); // console.log(".SYNCED", dbname, items.length)
this.events.emit('data', dbname, item) this.events.emit('data', dbname, item)
} }
@ -117,63 +110,6 @@ class OrbitDB {
if(this._pubsub) this._pubsub.unsubscribe(dbname) if(this._pubsub) this._pubsub.unsubscribe(dbname)
delete this.stores[dbname] delete this.stores[dbname]
} }
_connect(hash, username, password, allowOffline) {
if(allowOffline === undefined) allowOffline = false
const readNetworkInfo = (hash) => {
return new Promise((resolve, reject) => {
resolve(JSON.stringify({
name: 'Orbit DEV Network',
publishers: [hash]
}));
});
};
let host, port, name;
return readNetworkInfo(hash)
.then((object) => {
this.network = JSON.parse(object);
name = this.network.name;
host = this.network.publishers[0].split(":")[0];
port = this.network.publishers[0].split(":")[1];
})
.then(() => {
this._pubsub = new PubSub();
logger.debug(`Connecting to network ${hash} (${host}:${port})`);
return this._pubsub.connect(host, port, username, password)
})
.then(() => {
logger.debug(`Connected to network ${hash} (${host}:${port})`);
this.user = { username: username, id: username } // TODO: user id from ipfs hash
return;
})
.catch((e) => {
logger.warn(`Couldn't connect to ${hash} network: ${e.message}`);
if(!allowOffline) {
logger.debug(`'allowOffline' set to false, terminating`);
if(this._pubsub) this._pubsub.disconnect();
throw e;
}
this.user = { username: username, id: username } // TODO: user id from ipfs hash
return;
});
}
} }
class OrbitClientFactory { module.exports = OrbitDB
static connect(network, username, password, ipfs, options) {
if(!options) options = { allowOffline: false };
if(!ipfs) {
logger.error("IPFS instance not provided");
throw new Error("IPFS instance not provided");
}
const client = new OrbitDB(ipfs);
return client._connect(network, username, password, options.allowOffline)
.then(() => client)
}
}
module.exports = OrbitClientFactory;

63
src/PubSub.js
View File

@ -1,63 +0,0 @@
'use strict';
const io = require('socket.io-client');
const logger = require('logplease').create("orbit-db.Pubsub");
class Pubsub {
constructor() {
this._socket = null;
this._subscriptions = {};
}
connect(host, port, username, password) {
return new Promise((resolve, reject) => {
if(!this._socket)
this._socket = io.connect(`http://${host}:${port}`, { 'forceNew': true });
this._socket.on('connect', resolve);
this._socket.on('connect_error', (err) => reject(new Error(`Connection refused to Pubsub at '${host}:${port}'`)));
this._socket.on('disconnect', (socket) => logger.warn(`Disconnected from Pubsub at 'http://${host}:${port}'`));
this._socket.on('error', (e) => logger.error('Pubsub socket error:', e));
this._socket.on('message', this._handleMessage.bind(this));
this._socket.on('subscribed', this._handleSubscribed.bind(this));
});
}
disconnect() {
if(this._socket)
this._socket.disconnect();
}
subscribe(hash, callback, onSubscribed, fetchHistory) {
if(!this._subscriptions[hash]) {
this._subscriptions[hash] = { callback: callback, history: fetchHistory, onSubscribed: onSubscribed };
this._socket.emit('subscribe', { channel: hash }); // calls back with 'subscribed' event
}
}
unsubscribe(hash) {
if(this._subscriptions[hash]) {
this._socket.emit('unsubscribe', { channel: hash });
delete this._subscriptions[hash];
}
}
publish(hash, message) {
if(this._subscriptions[hash])
this._socket.send(JSON.stringify({ channel: hash, message: message }));
}
_handleMessage(hash, message) {
const subscription = this._subscriptions[hash];
if(subscription && subscription.callback)
subscription.callback(hash, message);
}
_handleSubscribed(hash, message) {
const subscription = this._subscriptions[hash];
if(subscription && subscription.history && subscription.onSubscribed)
subscription.onSubscribed(hash, message)
}
}
module.exports = Pubsub;

872
test/client.test.js
View File

File diff suppressed because it is too large Load Diff

243
test/counterdb.test.js
View File

@ -1,165 +1,104 @@
'use strict'; // 'use strict'
const assert = require('assert'); // const assert = require('assert')
const path = require('path'); // const path = require('path')
const fs = require('fs'); // const fs = require('fs')
const Promise = require('bluebird'); // const Promise = require('bluebird')
const rimraf = require('rimraf') // const rimraf = require('rimraf')
const OrbitDB = require('../src/OrbitDB'); // const IpfsApis = require('ipfs-test-apis')
const OrbitServer = require('orbit-server/src/server'); // const IpfsDaemon = require('ipfs-daemon')
const ipfsd = require('ipfsd-ctl'); // const OrbitDB = require('../src/OrbitDB')
const IPFS = require('ipfs')
// Mute logging // const username = 'testrunner'
require('logplease').setLogLevel('ERROR'); // const username2 = 'rennurtset'
// const cacheFile = path.join(process.cwd(), '/tmp/orbit-db-tests/cache.json')
const network = 'localhost:3333'; // IpfsApis.forEach(function(ipfsApi) {
const username = 'testrunner'; // let ipfs, ipfsDaemon
const username2 = 'rennurtset';
const ipfsPath = '/tmp/orbittests';
const cacheFile = path.join(process.cwd(), '/test/orbit-db-cache.json')
let ipfs, ipfsDaemon; // describe('CounterStore with ' + ipfsApi.name, function() {
const IpfsApis = [ // this.timeout(40000)
{ // let client1, client2
// js-ipfs // let daemon1, daemon2
name: 'js-ipfs',
start: () => {
return new Promise((resolve, reject) => {
const IPFS = require('ipfs')
const ipfs = new IPFS('/tmp/orbit-db-test');
const init = () => {
return new Promise((resolve, reject) => {
ipfs.init({}, (err) => {
if (err) {
if (err.message === 'repo already exists') {
return resolve();
}
return reject(err);
}
resolve();
});
});
};
// resolve(ipfs); // before((done) => {
return init().then(() => { // // rimraf.sync('./orbit-db-cache.json')
// resolve(ipfs); // daemon1
ipfs.goOnline((err) => { // Promise.all([
if(err) reject(err) // IpfsDaemon({ IpfsDataDir: '/tmp/daemon1' }),
return resolve(ipfs) // IpfsDaemon({ IpfsDataDir: '/tmp/daemon2' })
}); // ])
}); // .then((res) => {
}); // ipfs = [res[0].ipfs, res[1].ipfs]
}, // daemon1 = res[0].daemon
// stop: () => Promise.resolve() // daemon2 = res[1].daemon
stop: () => new Promise((resolve, reject) => ipfs.goOffline(resolve)) // done()
}, // })
{ // })
// js-ipfs-api via local daemon
name: 'js-ipfs-api',
start: () => {
return new Promise((resolve, reject) => {
ipfsd.disposableApi((err, ipfs) => {
if(err) reject(err);
resolve(ipfs);
});
// ipfsd.local((err, node) => {
// if(err) reject(err);
// ipfsDaemon = node;
// ipfsDaemon.startDaemon((err, ipfs) => {
// if(err) reject(err);
// resolve(ipfs);
// });
// });
});
},
stop: () => Promise.resolve()
// stop: () => new Promise((resolve, reject) => ipfsDaemon.stopDaemon(resolve))
}
];
// OrbitServer.start(); // uncomment if running this test suite stand-alone // after((done) => {
IpfsApis.forEach(function(ipfsApi) { // daemon1.stopDaemon()
// daemon2.stopDaemon()
// done()
// })
describe('CounterStore with ' + ipfsApi.name, function() { // beforeEach(() => {
this.timeout(40000); // client1 = new OrbitDB(ipfs[0], username, { cacheFile: cacheFile })
let client1, client2; // client2 = new OrbitDB(ipfs[1], username2, { cacheFile: cacheFile })
// })
before((done) => { // afterEach(() => {
rimraf.sync('./orbit-db-cache.json') // if(client1) client1.disconnect()
ipfsApi.start() // if(client2) client2.disconnect()
// .then((ipfs) => { // })
// return ipfs.add(path.resolve(process.cwd(), './test/network.json')).then(() => ipfs)
// })
.then((res) => {
ipfs = res;
return Promise.map([username, username2], (login) => {
return OrbitDB.connect(network, login, '', ipfs, { allowOffline: false, cacheFile: cacheFile });
}).then((clients) => {
client1 = clients[0];
client2 = clients[1];
return;
}).catch((e) => {
console.log(e.stack);
assert.equal(e, null);
});
})
.then(done)
});
after((done) => { // describe('counters', function() {
if(client1) client1.disconnect(); // it('increases a counter value', function(done) {
if(client2) client2.disconnect(); // const timeout = setTimeout(() => done(new Error('event was not fired')), 2000)
ipfsApi.stop().then(() => { // const counter = client1.counter('counter test', { subscribe: false, cacheFile: cacheFile })
rimraf(cacheFile, done) // counter.events.on('ready', () => {
}); // Promise.map([13, 1], (f) => counter.inc(f), { concurrency: 1 })
}); // .then(() => {
// assert.equal(counter.value(), 14)
// clearTimeout(timeout)
// client1.disconnect()
// done()
// })
// .catch(done)
// })
// })
describe('counters', function() { // it('creates a new counter from cached data', function(done) {
it('increases a counter value', function(done) { // const timeout = setTimeout(() => done(new Error('event was not fired')), 2000)
const timeout = setTimeout(() => done(new Error('event was not fired')), 2000) // const counter = client1.counter('counter test', { subscribe: false, cacheFile: cacheFile })
const counter = client1.counter('counter test', { subscribe: false, cacheFile: cacheFile }) // counter.events.on('ready', () => {
counter.events.on('ready', () => { // assert.equal(counter.value(), 14)
Promise.map([13, 1], (f) => counter.inc(f), { concurrency: 1 }) // clearTimeout(timeout)
.then(() => { // client1.disconnect()
assert.equal(counter.value(), 14) // done()
clearTimeout(timeout) // })
done() // })
})
.catch(done)
})
});
it('creates a new counter from cached data', function(done) { // it.only('syncs counters', (done) => {
const timeout = setTimeout(() => done(new Error('event was not fired')), 2000) // const name = new Date().getTime()
const counter = client1.counter('counter test', { subscribe: false, cacheFile: cacheFile }) // const counter1 = client1.counter(name)
counter.events.on('ready', () => { // const counter2 = client2.counter(name)
assert.equal(counter.value(), 14) // const numbers = [[13, 10], [2, 5]]
clearTimeout(timeout) // // const res1 = ([13, 10]).map((f) => counter1.inc(f))//, { concurrency: 1 })
done() // // const res2 = ([2, 5]).map((f) => counter2.inc(f))//, { concurrency: 1 })
}) // Promise.map([counter1, counter2], (counter, i) => numbers[i].map((e) => counter.inc(e)) , { concurrency: 1 })
}) // .then((res) => {
// // wait for a while to make sure db's have been synced
// setTimeout(() => {
// assert.equal(counter2.value(), 30)
// assert.equal(counter1.value(), 30)
// done()
// }, 10000)
// })
// .catch(done)
// })
it('syncs counters', (done) => { // })
const name = new Date().getTime(); // })
const counter1 = client1.counter(name)
const counter2 = client2.counter(name)
const res1 = Promise.map([13, 10], (f) => counter1.inc(f), { concurrency: 1 })
const res2 = Promise.map([2, 5], (f) => counter2.inc(f), { concurrency: 1 })
Promise.all([res1, res2])
.then((res) => {
// wait for a while to make sure db's have been synced
setTimeout(() => {
assert.equal(counter1.value(), 30)
assert.equal(counter2.value(), 30)
done()
}, 4000)
})
.catch(done)
})
}); // })
});
});

60
webpack.config.js
View File

@ -1,5 +1,5 @@
const webpack = require('webpack'); const webpack = require('webpack')
const path = require('path'); const path = require('path')
module.exports = { module.exports = {
entry: './src/OrbitDB.js', entry: './src/OrbitDB.js',
@ -8,21 +8,18 @@ module.exports = {
library: 'OrbitDB', library: 'OrbitDB',
filename: './dist/orbitdb.js' filename: './dist/orbitdb.js'
}, },
devtool: 'sourcemap',
node: { node: {
console: false, console: false,
process: 'mock', process: 'mock',
Buffer: 'buffer' Buffer: true
},
resolveLoader: {
root: path.join(__dirname, 'node_modules')
}, },
resolve: { resolve: {
modulesDirectories: [ modules: [
path.join(__dirname, 'node_modules') path.join(__dirname, 'node_modules')
], ],
alias: { alias: {
'orbit-db-stre': require.resolve('./node_modules/orbit-db-store'), 'fs': path.join(__dirname + '/node_modules', 'html5-fs'),
fs: require.resolve('./node_modules/logplease/src/fs-mock'),
http: 'stream-http', http: 'stream-http',
https: 'https-browserify', https: 'https-browserify',
Buffer: 'buffer' Buffer: 'buffer'
@ -30,32 +27,33 @@ module.exports = {
}, },
module: { module: {
loaders: [ loaders: [
{ {
test: /\.js$/, test: /\.js$/,
exclude: /node_modules/, exclude: /node_modules/,
loader: 'babel', loader: 'babel',
query: { query: {
presets: require.resolve('babel-preset-es2015'), presets: require.resolve('babel-preset-es2015'),
plugins: require.resolve('babel-plugin-transform-runtime') plugins: require.resolve('babel-plugin-transform-runtime')
}
},
{
test: /\.js$/,
include: /node_modules\/(hoek|qs|wreck|boom|ipfs-.+|orbit-db.+|logplease|crdts|promisify-es6)/,
loader: 'babel',
query: {
presets: require.resolve('babel-preset-es2015'),
plugins: require.resolve('babel-plugin-transform-runtime')
}
},
{
test: /\.json$/,
loader: 'json'
} }
}, ]
{
test: /\.js$/,
include: /node_modules\/(hoek|qs|wreck|boom)/,
loader: 'babel',
query: {
presets: require.resolve('babel-preset-es2015'),
plugins: require.resolve('babel-plugin-transform-runtime')
}
},
{
test: /\.json$/,
loader: 'json'
}]
}, },
externals: { externals: {
net: '{}', net: '{}',
tls: '{}', tls: '{}',
'require-dir': '{}' 'require-dir': '{}'
} }
}; }

61
webpack.config.minified.js
View File

@ -1,5 +1,5 @@
const webpack = require('webpack'); const webpack = require('webpack')
const path = require('path'); const path = require('path')
module.exports = { module.exports = {
entry: './src/OrbitDB.js', entry: './src/OrbitDB.js',
@ -8,10 +8,11 @@ module.exports = {
library: 'OrbitDB', library: 'OrbitDB',
filename: './dist/orbitdb.min.js' filename: './dist/orbitdb.min.js'
}, },
devtool: 'sourcemap',
node: { node: {
console: false, console: false,
process: 'mock', process: 'mock',
Buffer: 'buffer' Buffer: true
}, },
plugins: [ plugins: [
new webpack.optimize.UglifyJsPlugin({ new webpack.optimize.UglifyJsPlugin({
@ -20,43 +21,41 @@ module.exports = {
}) })
], ],
resolveLoader: { resolveLoader: {
root: path.join(__dirname, 'node_modules') modules: [path.join(__dirname, 'node_modules')]
}, },
resolve: { resolve: {
modulesDirectories: [ modules: [
path.join(__dirname, 'node_modules') path.join(__dirname, 'node_modules')
], ],
alias: { alias: {
fs: require.resolve('./node_modules/logplease/src/fs-mock'), 'node_modules': path.join(__dirname + '/node_modules'),
http: 'stream-http', 'fs': path.join(__dirname + '/node_modules', 'html5-fs'),
https: 'https-browserify',
Buffer: 'buffer'
} }
}, },
module: { module: {
loaders: [ loaders: [
{ {
test: /\.js$/, test: /\.js$/,
exclude: /node_modules/, exclude: /node_modules/,
loader: 'babel', loader: 'babel',
query: { query: {
presets: require.resolve('babel-preset-es2015'), presets: require.resolve('babel-preset-es2015'),
plugins: require.resolve('babel-plugin-transform-runtime') plugins: require.resolve('babel-plugin-transform-runtime')
}
},
{
test: /\.js$/,
include: /node_modules\/(hoek|qs|wreck|boom|ipfs-.+|orbit-db.+|logplease|crdts|promisify-es6)/,
loader: 'babel',
query: {
presets: require.resolve('babel-preset-es2015'),
plugins: require.resolve('babel-plugin-transform-runtime')
}
},
{
test: /\.json$/,
loader: 'json'
} }
},
{
test: /\.js$/,
include: /node_modules\/(hoek|qs|wreck|boom|ipfs-.+|orbit-db-.+|logplease|crdts)/,
loader: 'babel',
query: {
presets: require.resolve('babel-preset-es2015'),
plugins: require.resolve('babel-plugin-transform-runtime')
}
},
{
test: /\.json$/,
loader: 'json'
}
] ]
}, },
externals: { externals: {
@ -64,4 +63,4 @@ module.exports = {
tls: '{}', tls: '{}',
'require-dir': '{}' 'require-dir': '{}'
} }
}; }

65
webpack.example.config.js
View File

@ -1,15 +1,18 @@
const webpack = require('webpack'); const webpack = require('webpack')
const path = require('path'); const path = require('path')
module.exports = { module.exports = {
entry: './examples/browser/browser.js', entry: [
'./examples/browser/index.js',
],
output: { output: {
filename: './examples/browser/bundle.js' filename: './examples/browser/bundle.js'
}, },
devtool: 'sourcemap',
node: { node: {
console: false, console: false,
process: 'mock', process: 'mock',
Buffer: 'buffer' Buffer: true
}, },
plugins: [ plugins: [
new webpack.optimize.UglifyJsPlugin({ new webpack.optimize.UglifyJsPlugin({
@ -17,44 +20,38 @@ module.exports = {
compress: { warnings: false } compress: { warnings: false }
}) })
], ],
resolveLoader: {
root: path.join(__dirname, 'node_modules')
},
resolve: { resolve: {
modulesDirectories: [ modules: [
path.join(__dirname, 'node_modules') path.join(__dirname, 'node_modules')
], ],
alias: { alias: {
fs: require.resolve('./node_modules/logplease/src/fs-mock'), 'fs': path.join(__dirname + '/node_modules', 'html5-fs'),
http: 'stream-http',
https: 'https-browserify',
Buffer: 'buffer'
} }
}, },
module: { module: {
loaders: [ loaders: [
{ {
test: /\.js$/, test: /\.js$/,
exclude: /node_modules/, exclude: /node_modules/,
loader: 'babel', loader: 'babel',
query: { query: {
presets: require.resolve('babel-preset-es2015'), presets: require.resolve('babel-preset-es2015'),
plugins: require.resolve('babel-plugin-transform-runtime') plugins: require.resolve('babel-plugin-transform-runtime')
}
},
{
test: /\.js$/,
include: /node_modules\/(hoek|qs|wreck|boom|ipfs.+|orbit.+|logplease|crdts|promisify-es|whatwg-fetch|node-fetch|isomorphic-fetch|db\.js)/,
loader: 'babel',
query: {
presets: require.resolve('babel-preset-es2015'),
plugins: require.resolve('babel-plugin-transform-runtime')
}
},
{
test: /\.json$/,
loader: 'json'
} }
},
{
test: /\.js$/,
include: /node_modules\/(hoek|qs|wreck|boom|ipfs-.+|orbit-db-.+|logplease|crdts)/,
loader: 'babel',
query: {
presets: require.resolve('babel-preset-es2015'),
plugins: require.resolve('babel-plugin-transform-runtime')
}
},
{
test: /\.json$/,
loader: 'json'
}
] ]
}, },
externals: { externals: {
@ -62,4 +59,4 @@ module.exports = {
tls: '{}', tls: '{}',
'require-dir': '{}' 'require-dir': '{}'
} }
}; }