Mirroristas/orbitdb
2
0
Fork 0
mirror of https://github.com/orbitdb/orbitdb.git synced 2025-05-29 10:16:39 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #240 from orbitdb/feat/write-access

Browse Source 
Write-permissioned databases
This commit is contained in:
Haad 2017-11-28 09:17:21 +01:00 committed by GitHub
commit 6328d5d713
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
58 changed files with 15159 additions and 16782 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
.gitignore vendored
View File

@ -1,13 +1,8 @@
*sublime*
node_modules/
*.log
.vagrant/
.idea/
dump.rdb
Vagrantfile
examples/browser/bundle.js
examples/browser/*.map
examples/browser/browser-webpack-example/bundle.js
examples/browser/browser-webpack-example/*.map
examples/browser/lib
dist/*.map
orbit-db/
ipfs/
dist/orbitdb.js
orbitdb/

434
API.md
View File

@ -1,17 +1,21 @@
# orbit-db API documentation
OrbitDB provides various types of databases for different data models:
- [kvstore](#kvstorename) is a key-value database just like your favourite key-value database.
- [eventlog](#eventlogname) is an append-only log with traversable history. Useful for *"latest N"* use cases or as a message queue.
- [feed](#feedname) is a log with traversable history. Entries can be added and removed. Useful for *"shopping cart" type of use cases, or for example as a feed of blog posts or "tweets".
- [counter](#countername) for counting. Useful for example counting events separate from log/feed data.
- [docstore](##docstorename-options) is a document database to which documents can be stored and indexed by a specified key. Useful for example building search indices or version controlling documents and data.
- [log](#lognameaddress) is an append-only log with traversable history. Useful for *"latest N"* use cases or as a message queue.
- [feed](#feednameaddress) is a log with traversable history. Entries can be added and removed. Useful for *"shopping cart" type of use cases, or for example as a feed of blog posts or "tweets".
- [keyvalue](#keyvaluenameaddress) is a key-value database just like your favourite key-value database.
- [docs](#docsnameaddress-options) is a document database to which documents can be stored and indexed by a specified key. Useful for example building search indices or version controlling documents and data.
- [counter](#counternameaddress) for counting. Useful for example counting events separate from log/feed data.
Which database to use depends on your use case and data model.
## Getting Started
## Usage
Install `orbit-db` and [ipfs](https://www.npmjs.com/package/ipfs) from npm:
Read the **[GETTING STARTED](https://github.com/orbitdb/orbit-db/blob/master/GUIDE.md)** guide for a more in-depth tutorial and to understand how OrbitDB works.
### Using as a module
Install [orbit-db](https://www.npmjs.com/package/orbit-db) and [ipfs](https://www.npmjs.com/package/ipfs) from npm:
```
npm install orbit-db ipfs
@ -24,13 +28,33 @@ const IPFS = require('ipfs')
const OrbitDB = require('orbit-db')
const ipfs = new IPFS()
ipfs.on('ready', () => {
const orbitdb = new OrbitDB(ipfs)
// Create / Open a database
const db = await orbitdb.log('hello')
await db.load()
// Listen for updates from peers
db.events.on('replicated', (address) => {
console.log(db.iterator({ limit: -1 }).collect())
})
// Add an entry
const hash = await db.add('world')
console.log(hash)
// Query
const result = db.iterator({ limit: -1 }).collect()
console.log(result)
})
```
`orbitdb` is now the [OrbitDB](#orbitdb) instance we can use to interact with the databases.
This will tell `orbit-db` to use the [Javascript implementation](https://github.com/ipfs/js-ipfs) of IPFS. Choose this options if you're using `orbitd-db` to develop **Browser** applications.
This will tell `orbit-db` to use the [Javascript implementation](https://github.com/ipfs/js-ipfs) of IPFS. Choose this options if you're using `orbitd-db` to develop **browser** applications.
### Using with a running IPFS daemon
Alternatively, you can use [ipfs-api](https://npmjs.org/package/ipfs-api) to use `orbit-db` with a locally running IPFS daemon:
```
@ -43,118 +67,122 @@ const OrbitDB = require('orbit-db')
const ipfs = IpfsApi('localhost', '5001')
const orbitdb = new OrbitDB(ipfs)
const db = await orbitdb.log('hello')
...
```
`orbitdb` is now the [OrbitDB](#orbitdb) instance we can use to interact with the databases.
Choose this options if you're using `orbitd-db` to develop **Desktop** (or "headless") applications, eg. with [Electron](https://electron.atom.io).
Choose this options if you're using `orbitd-db` to develop **backend** or **desktop** applications, eg. with [Electron](https://electron.atom.io).
## Usage
## API
- [orbitdb](#orbitdb)
- [kvstore(name)](#kvstorename)
- [put(key, value)](#kvstorename)
- [set(key, value)](#kvstorename)
- [get(key)](#kvstorename)
- [events](#kvstorename)
- [eventlog(name)](#eventlogname)
- [add(event)](#eventlogname)
- [get(hash)](#eventlogname)
- [iterator([options])](#eventlogname)
- [events](#eventlogname)
- [feed(name)](#feedname)
- [add(data)](#feedname)
- [get(hash)](#feedname)
- [iterator([options])](#feedname)
- [remove(hash)](#feedname)
- [events](#feedname)
- [docstore(name, options)](#docstorename-options)
- [put(doc)](#docstorename-options)
- [get(hash)](#docstorename-options)
- [query(mapper)](#docstorename-options)
- [del(key)](#docstorename-options)
- [events](#docstorename-options)
- [counter(name)](#countername)
- [value](#countername)
- [inc([amount])](#countername)
- [events](#countername)
- [disconnect()](#disconnect)
- [OrbitDB](#orbitdb)
- [constructor(ipfs, [directory], [options])](#constructoripfs-directory-options)
- [keyvalue(name|address)](#keyvaluenameaddress)
- [put(key, value)](#putkey-value)
- [set(key, value)](#setkey-value)
- [get(key)](#getkey)
- [log(name|address)](#lognameaddress)
- [add(event)](#addevent)
- [get(hash)](#gethash)
- [iterator([options])](#iteratoroptions)
- [feed(name|address)](#feednameaddress)
- [add(data)](#adddata)
- [get(hash)](#gethash-1)
- [remove(hash)](#removehash)
- [iterator([options])](#iteratoroptions)
- [docs(name|address, options)](#docsnameaddress-options)
- [put(doc)](#putdoc)
- [get(hash)](#getkey-1)
- [query(mapper)](#querymapper)
- [del(key)](#delkey)
- [counter(name|address)](#counternameaddress)
- [value](#value)
- [inc([value])](#incvalue)
- [stop()](#stop)
- [Store](#store)
- [load()](#load)
- [close()](#close)
- [drop()](#drop)
- [events](#events)
- [orbitdb](#events)
- [stores](#events)
- [key](#key)
- [type](#type)
## orbitdb
## OrbitDB
After creating an instance of `orbitd-db`, you can now access the different data stores.
### kvstore(name)
Package:
[orbit-db-kvstore](https://github.com/haadcode/orbit-db-kvstore)
### constructor(ipfs, [directory], [options])
```javascript
const db = orbitdb.kvstore('application.settings')
const IPFS = require('ipfs')
const OrbitDB = require('orbit-db')
const ipfs = new IPFS()
ipfs.on('ready', () => {
const orbitdb = new OrbitDB(ipfs)
})
```
- **put(key, value)**
After creating an `OrbitDB` instance , you can access the different data stores. Creating a database instance, eg. with `orbitdb.keyvalue(...)`, returns a *Promise* that resolves to a [database instance](#store). See the [Store](#store) section for details of common methods and properties.
```javascript
db.put('hello', { name: 'World' }).then(() => ...)
const db = await orbitdb.kvstore('profile')
```
- **set(key, value)**
### keyvalue(name|address)
Module: [orbit-db-kvstore](https://github.com/orbitdb/orbit-db-kvstore)
```javascript
db.set('hello', { name: 'Friend' }).then(() => ...)
const db = await orbitdb.keyvalue('application.settings')
// Or:
const db = await orbitdb.keyvalue(anotherkvdb.address)
```
- **get(key)**
**See the [Store](#store) section for details of common methods and properties.**
#### put(key, value)
```javascript
await db.put('hello', { name: 'World' })
```
#### set(key, value)
```javascript
await db.set('hello', { name: 'Friend' })
```
#### get(key)
```javascript
const value = db.get('hello')
// { name: 'Friend' }
```
- **load()**
### log(name|address)
Load the locally persisted database state to memory.
Module: [orbit-db-eventstore](https://github.com/orbitdb/orbit-db-eventstore)
```javascript
db.events.on('ready', () => {
/* query */
})
db.load()
const db = await orbitdb.eventlog('site.visitors')
// Or:
const db = await orbitdb.eventlog(anotherlogdb.address)
```
- **events**
**See the [Store](#store) section for details of common methods and properties.**
#### add(event)
```javascript
db.events.on('ready', () => /* local database loaded in memory */ )
db.events.on('synced', () => /* query for updated results */ )
const hash = await db.add({ name: 'User1' })
```
See [events](#events) for full description.
### eventlog(name)
Package:
[orbit-db-eventstore](https://github.com/haadcode/orbit-db-eventstore)
```javascript
const db = orbitdb.eventlog('site.visitors')
```
- **add(event)**
```javascript
db.add({ name: 'User1' }).then((hash) => ...)
```
- **get(hash)**
#### get(hash)
```javascript
const event = db.get(hash)
.map((e) => e.payload.value)
// { name: 'User1' }
```
- **iterator([options])**
#### iterator([options])
**options** : It is an object which supports the following properties
@ -177,48 +205,36 @@ After creating an instance of `orbitd-db`, you can now access the different data
// [{ name: 'User1' }]
```
- **load()**
### feed(name|address)
Load the locally persisted database state to memory.
Module: [orbit-db-feedstore](https://github.com/orbitdb/orbit-db-feedstore)
```javascript
db.events.on('ready', () => {
/* query */
})
db.load()
const db = await orbitdb.feed('orbit-db.issues')
// Or:
const db = await orbitdb.feed(anotherfeeddb.address)
```
- **events**
See the [Store](#store) section for details of common methods and properties.
#### add(data)
```javascript
db.events.on('ready', () => /* local database loaded in memory */ )
db.events.on('synced', () => /* query for updated results */ )
const hash = await db.add({ name: 'User1' })
```
See [events](#events) for full description.
### feed(name)
Package:
[orbit-db-feedstore](https://github.com/haadcode/orbit-db-feedstore)
```javascript
const db = orbitdb.feed('orbit-db.issues')
```
- **add(data)**
```javascript
db.add({ name: 'User1' }).then((hash) => ...)
```
- **get(hash)**
#### get(hash)
```javascript
const event = db.get(hash)
.map((e) => e.payload.value)
// { name: 'User1' }
```
- **iterator([options])**
#### remove(hash)
```javascript
const hash = await db.remove(hash)
```
#### iterator([options])
**options** : It is an object which supports the following properties
@ -233,6 +249,7 @@ After creating an instance of `orbitd-db`, you can now access the different data
`limit - (integer)` Limiting the entries of result
`reverse - (boolean)` If set to true will result in reversing the result.
```javascript
const all = db.iterator({ limit: -1 })
.collect()
@ -240,172 +257,177 @@ After creating an instance of `orbitd-db`, you can now access the different data
// [{ name: 'User1' }]
```
- **remove(hash)**
```javascript
db.remove(hash).then((removed) => ...)
```
### docs(name|address, options)
- **load()**
Load the locally persisted database state to memory.
Module: [orbit-db-docstore](https://github.com/orbitdb/orbit-db-docstore)
```javascript
db.events.on('ready', () => {
/* query */
})
db.load()
const db = await orbitdb.docs('orbit.users.shamb0t.profile')
// Or:
const db = await orbitdb.docs(anotherdocdb.address)
```
- **events**
By default, documents are indexed by field `_id`. You can also specify the field to index by:
```javascript
db.events.on('ready', () => /* local database loaded in memory */ )
db.events.on('synced', () => /* query for updated results */ )
const db = await orbitdb.docs('orbit.users.shamb0t.profile', { indexBy: 'name' })
```
See [events](#events) for full description.
### docstore(name, options)
Package:
[orbit-db-docstore](https://github.com/shamb0t/orbit-db-docstore)
**See the [Store](#store) section for details of common methods and properties.**
#### put(doc)
```javascript
const db = orbitdb.docstore('orbit.users.shamb0t.profile')
const hash = await db.put({ _id: 'QmAwesomeIpfsHash', name: 'shamb0t', followers: 500 })
```
By default, documents are indexed by field '_id'. You can also specify the field to index by:
```javascript
const db = orbitdb.docstore('orbit.users.shamb0t.profile', { indexBy: 'name' })
```
- **put(doc)**
```javascript
db.put({ _id: 'QmAwesomeIpfsHash', name: 'shamb0t', followers: 500 }).then((hash) => ...)
```
- **get(key)**
#### get(key)
```javascript
const profile = db.get('shamb0t')
.map((e) => e.payload.value)
// [{ _id: 'shamb0t', name: 'shamb0t', followers: 500 }]
```
- **query(mapper)**
#### query(mapper)
```javascript
const all = db.query((doc) => doc.followers >= 500)
// [{ _id: 'shamb0t', name: 'shamb0t', followers: 500 }]
```
- **del(key)**
#### del(key)
```javascript
db.del('shamb0t').then((removed) => ...)
const hash = await db.del('shamb0t')
```
- **load()**
### counter(name|address)
Load the locally persisted database state to memory.
Module: [orbit-db-counterstore](https://github.com/orbitdb/orbit-db-counterstore)
```javascript
db.events.on('ready', () => {
/* query */
})
db.load()
const counter = await orbitdb.counter('song_123.play_count')
// Or:
const counter = await orbitdb.counter(anothercounterdb.address)
```
- **events**
**See the [Store](#store) section for details of common methods and properties.**
```javascript
db.events.on('ready', () => /* local database loaded in memory */ )
db.events.on('synced', () => /* query for updated results */ )
```
See [events](#events) for full description.
### counter(name)
Package:
[orbit-db-counterstore](https://github.com/haadcode/orbit-db-counterstore)
```javascript
const counter = orbitdb.counter('song_123.play_count')
```
- **value**
#### value
```javascript
counter.value // 0
```
- **inc([value])**
#### inc([value])
```javascript
counter.inc()
await counter.inc()
counter.value // 1
counter.inc(7)
await counter.inc(7)
counter.value // 8
counter.inc(-2)
await counter.inc(-2)
counter.value // 8
```
- **load()**
### stop()
Stop OrbitDB, close databases and disconnect the databases from the network.
```javascript
orbitdb.stop()
```
## Store
Every database (store) has the following methods available in addition to their specific methods.
#### load()
Load the locally persisted database state to memory.
With events:
```javascript
db.events.on('ready', () => {
/* query */
/* database is now ready to be queried */
})
db.load()
```
- **events**
Async:
```javascript
db.events.on('ready', () => /* local database loaded in memory */ )
db.events.on('synced', () => /* query for updated results */ )
await db.load()
/* database is now ready to be queried */
```
See [events](#events) for full description.
#### close()
### disconnect()
Close the database.
Async:
```javascript
orbitdb.disconnect()
await db.close()
```
### events
#### drop()
- **stores**
Each database in `orbit-db` contains an `events` ([EventEmitter](https://nodejs.org/api/events.html)) object that emits events that describe what's happening in the database.
- `synced` - (dbname)
Emitted when an update happens in the databases. Eg. when the database was synchronized with a peer. This is usually a good place to requery the database
for updated results, eg. if a value of a key was changed or if there are new
events in an event log.
Remove the database locally. This does not delete any data from peers.
```javascript
db.events.on('synced', () => ... )
await db.drop()
```
- `sync` - (dbname)
#### key
Emitted before starting a database sync with a peer.
The [keypair]([orbit-db-keystore]()) used to access the database.
#### type
The type of the database as a string.
#### events
Each database in `orbit-db` contains an `events` ([EventEmitter](https://nodejs.org/api/events.html)) object that emits events that describe what's happening in the database. Events can be listened to with:
```javascript
db.events.on(name, callback)
```
- **`replicated`** - (address)
Emitted when a the database was synced with another peer. This is usually a good place to re-query the database for updated results, eg. if a value of a key was changed or if there are new events in an event log.
```javascript
db.events.on('sync', (dbname) => ... )
db.events.on('replicated', (address) => ... )
```
- `load` - (dbname)
- **`replicate`** - (address)
Emitted before loading the local database.
Emitted before replicating a part of the database with a peer.
```javascript
db.events.on('replicate', (address) => ... )
```
- **`replicate.progress`** - (address, hash, entry, progress, have)
Emitted while replicating a database. *address* is id of the database that emitted the event. *hash* is the multihash of the entry that was just loaded. *entry* is the database operation entry. *progress* is the current progress. *have* is a map of database pieces we have.
```javascript
db.events.on('replicate.progress', (address, hash, entry, progress, have) => ... )
```
- **`load`** - (dbname)
Emitted before loading the database.
```javascript
db.events.on('load', (dbname) => ... )
```
- `ready` - (dbname)
- **`load.progress`** - (address, hash, entry, progress, total)
Emitted while loading the local database, once for each entry. *dbname* is the name of the database that emitted the event. *hash* is the multihash of the entry that was just loaded. *entry* is the database operation entry. *progress* is a sequential number starting from 0 upon calling `load()`.
```javascript
db.events.on('load.progress', (address, hash, entry, progress, total) => ... )
```
- **`ready`** - (dbname)
Emitted after fully loading the local database.
@ -413,26 +435,10 @@ After creating an instance of `orbitd-db`, you can now access the different data
db.events.on('ready', (dbname) => ... )
```
- `write` - (dbname, hash, entry)
- **`write`** - (dbname, hash, entry)
Emitted after an entry was added locally to the database. *hash* is the IPFS hash of the latest state of the database. *entry* is the added database op.
```javascript
db.events.on('write', (dbname, hash, entry) => ... )
```
- `load.progress` - (dbname, hash, entry, progress)
Emitted while loading the local database, once for each entry. *dbname* is the name of the database that emitted the event. *hash* is the multihash of the entry that was just loaded. *entry* is the database operation entry. *progress* is a sequential number starting from 0 upon calling `load()`.
```javascript
db.events.on('load.progress', (dbname, hash, entry, progress) => ... )
```
- `error` - (error)
Emitted on an error.
```javascript
db.events.on('error', (err) => ... )
```

9
CHANGELOG.md
View File

@ -1,4 +1,13 @@
# Changelog
## v2.0.0
- addresses
- write-permissions
- replication
- performance
- use ipfs, no more ipfs-daemon
- async/await
- logging
## v0.12.0
- IPFS pubsub

336
GUIDE.md Normal file
View File

@ -0,0 +1,336 @@
# Getting Started with OrbitDB
This guide will get you familiar using OrbitDB in your JavaScript application. OrbitDB and IPFS both work in Node.js applications as well as in browser applications.
This guide is still being worked on and we would love to get [feedback and suggestions](https://github.com/orbitdb/orbit-db/issues) on how to improve it!
## Table of Contents
- [Background](#background)
- [Install](#install)
- [Setup](#setup)
- [Create a database](#create-a-database)
- [Address](#address)
- [Manifest](#manifest)
- [Keys](#keys)
- [Access Control](#access-control)
- [Public databases](#public-databases)
- [Add an entry](#add-an-entry)
- [Get an entry](#get-an-entry)
- [Persistency](#persistency)
- [Replicating a database](#replicating-a-database)
## Background
OrbitDB is a peer-to-peer database meaning that each peer has its own instance of a specific database. A database is replicated between the peers automatically resulting in an up-to-date view of the database upon updates from any peer. That is to say, the database gets pulled to the clients.
This means that each application contains the full database that they're using. This in turn changes the data modeling as compared to client-server model where there's usually one big database for all entries: in OrbitDB, the data should be stored, "partitioned" or "sharded" based on the access rights for that data. For example, in a twitter-like application, tweets would not be saved in a global "tweets" database to which millions of users write concurretnly, but rather, ***each user would have their own database*** for their tweets. To follow a user, a peer would subscribe to a user's feed, ie. replicate their feed database.
OrbitDB supports multiple data models (see more details below) and as such the developer has a variety ways to structure data. Combined with the peer-to-peer paradigm, the data modeling is important factor to build scalable decentralized applications.
This may not be intuitive or you might not be sure what the best approach would be and we'd be happy to help you decide on your data modeling and application needs, [feel free to reach out](https://github.com/orbitdb/orbit-db/issues)!
## Install
Install [orbit-db](https://github.com/orbitdb/orbit-db) and [ipfs](https://www.npmjs.com/package/ipfs) from npm:
```
npm install orbit-db ipfs
```
## Setup
Require OrbitDB and IPFS in your program and create the instances:
```javascript
const IPFS = require('ipfs')
const OrbitDB = require('orbit-db')
const ipfs = new IPFS()
ipfs.on('ready', () => {
const orbitdb = new OrbitDB(ipfs)
})
```
`orbitdb` is now the [OrbitDB](#orbitdb) instance we can use to interact with the databases.
## Create a database
First, choose the data model you want to use. The available data models are:
- [Key-Value](https://github.com/orbitdb/orbit-db/blob/master/API.md##keyvaluenameaddress)
- [Log](https://github.com/orbitdb/orbit-db/blob/master/API.md#lognameaddress) (append-only log)
- [Feed](https://github.com/orbitdb/orbit-db/blob/master/API.md#feednameaddress) (same as log database but entries can be removed)
- [Documents](https://github.com/orbitdb/orbit-db/blob/master/API.md#docsnameaddress-options) (store indexed JSON documents)
- [Counters](https://github.com/orbitdb/orbit-db/blob/master/API.md#counternameaddress)
Then, create a database instance (we'll use Key-Value database in this example):
```javascript
const ipfs = new IPFS()
ipfs.on('ready', async () => {
const orbitdb = new OrbitDB(ipfs)
const db = await orbitdb.keyvalue('first-database')
})
```
### Address
When a database is created, it will be assigned an address by OrbitDB. The address consists of three parts:
```
/orbitdb/Qmd8TmZrWASypEp4Er9tgWP4kCNQnW4ncSnvjvyHQ3EVSU/first-database
```
The first part, `/orbitdb`, specifies the protocol in use. The second part, an IPFS multihash `Qmd8TmZrWASypEp4Er9tgWP4kCNQnW4ncSnvjvyHQ3EVSU`, is the database manifest which contains the database info such as the name and type, and a pointer to the access controller. The last part, `first-database`, is the name of the database.
In order to replicate the database with peers, the address is what you need to give to other peers in order for them to start replicating the database.
The database address can be accessed as `db.address` from the database instance:
```
const address = db.address
// address == '/orbitdb/Qmdgwt7w4uBsw8LXduzCd18zfGXeTmBsiR8edQ1hSfzcJC/first-database'
```
For example:
```javascript
const ipfs = new IPFS()
ipfs.on('ready', async () => {
const orbitdb = new OrbitDB(ipfs)
const db = await orbitdb.keyvalue('first-database')
console.log(db.address.toString())
// /orbitdb/Qmd8TmZrWASypEp4Er9tgWP4kCNQnW4ncSnvjvyHQ3EVSU/first-database
})
```
#### Manifest
The second part of the address, the IPFS multihash `Qmdgwt7w4uBsw8LXduzCd18zfGXeTmBsiR8edQ1hSfzcJC`, is the manifest of a database. It's an IPFS object that contains information about the database.
The database manifest can be fetched from IPFS and it looks like this:
```json
{
"Data": "{\"name\":\"a\",\"type\":\"feed\",\"accessController\":\"/ipfs/QmdjrCN7SqGxRapsm6LuoS4HrWmLeQHVM6f1Zk5A3UveqA\"}",
"Hash": "Qmdgwt7w4uBsw8LXduzCd18zfGXeTmBsiR8edQ1hSfzcJC",
"Size": 102,
"Links": []
}
```
### Keys
Each entry in a database is signed by who created that entry. The signing key, the key that a peer uses to sign entries, can be accessed as a member variable of the database instance:
```
db.key
```
The key contains the keypair used to sign the database entries. The public key can be retrieved with:
```
db.key.getPublic('hex')
```
The key can also be accessed from the OrbitDB instance: `orbitdb.key.getPublic('hex')`.
If you want to give access to other peers to write to a database, you need to get their the public key in hex and add it to the access controller upon creating the database. If you want others to give you the access to write, you'll need to give them your public key (output of `orbitdb.key.getPublic('hex')`). For more information, see: [Access Control](https://github.com/orbitdb/orbit-db/blob/master/GUIDE.md#access-control).
### Access Control
You can specify the peers that have write-access to a database. You can define a set of peers that can write to a database or allow anyone write to a database. **By default and if not specified otherwise, the only creator of the database will be given write-access**.
***Note!*** *OrbitDB currently supports only write-access and the keys of the writers need to be known when creating a database. That is, the access rights can't be changed after a database has been created. In the future we'll support read access control and dynamic access control in a way that access rights can be added and removed to a database at any point in time without changing the database address. At the moment, if access rights need to be changed, the address of the database will change.*
Access rights are setup by passing an `access` object that defines the access rights of the database when created. OrbitDB currently supports write-access. The access rights are specified as an array of public keys of the peers who can write to the database.
```javascript
const ipfs = new IPFS()
ipfs.on('ready', async () => {
const orbitdb = new OrbitDB(ipfs)
const access = {
// Give write access to ourselves
write: [orbitdb.key.getPublic('hex')],
}
const db = await orbitdb.keyvalue('first-database', access)
console.log(db.address.toString())
// /orbitdb/Qmd8TmZrWASypEp4Er9tgWP4kCNQnW4ncSnvjvyHQ3EVSU/first-database
})
```
To give write access to another peer, you'll need to get their public key with some means. They'll need to give you the output of their OrbitDB instance's key: `orbitdb.key.getPublic('hex')`.
The keys look like this:
`042c07044e7ea51a489c02854db5e09f0191690dc59db0afd95328c9db614a2976e088cab7c86d7e48183191258fc59dc699653508ce25bf0369d67f33d5d77839`
Give access to another peer to write to the database:
```javascript
const ipfs = new IPFS()
ipfs.on('ready', async () => {
const orbitdb = new OrbitDB(ipfs)
const access = {
// Setup write access
write: [
// Give access to ourselves
orbitdb.key.getPublic('hex'),
// Give access to the second peer
'042c07044e7ea51a489c02854db5e09f0191690dc59db0afd95328c9db614a2976e088cab7c86d7e48183191258fc59dc699653508ce25bf0369d67f33d5d77839',
],
}
const db = await orbitdb.keyvalue('first-database', access)
console.log(db.address.toString())
// /orbitdb/Qmdgwt7w4uBsw8LXduzCd18zfGXeTmBsiR8edQ1hSfzcJC/first-database
// Second peer opens the database from the address
const db2 = await orbitdb.keyvalue(db1.address.toString())
})
```
#### Public databases
The access control mechanism also support "public" databases to which anyone can write to.
This can be done by adding a `*` to the write access array:
```javascript
const ipfs = new IPFS()
ipfs.on('ready', async () => {
const orbitdb = new OrbitDB(ipfs)
const access = {
// Give write access to everyone
write: ['*'],
}
const db = await orbitdb.keyvalue('first-database', access)
console.log(db.address.toString())
// /orbitdb/QmRrauSxaAvNjpZcm2Cq6y9DcrH8wQQWGjtokF4tgCUxGP/first-database
})
```
Note how the access controller hash is different compared to the previous example!
## Add an entry
To add an entry to the database, we simply call `db.put(key, value)`.
```javascript
const ipfs = new IPFS()
ipfs.on('ready', async () => {
const orbitdb = new OrbitDB(ipfs)
const db = await orbitdb.keyvalue('first-database')
await db.put('name', 'hello')
})
```
For adding entries to other databases, see:
- [log.add()](https://github.com/orbitdb/orbit-db/blob/master/API.md#addevent)
- [feed.add()](https://github.com/orbitdb/orbit-db/blob/master/API.md#adddata)
- [docs.put()](https://github.com/orbitdb/orbit-db/blob/master/API.md#putdoc)
- [counter.inc()](https://github.com/orbitdb/orbit-db/blob/master/API.md#incvalue)
**Parallelism**
We currently don't support parallel updates. Updates to a database need to be executed in a sequential manner. The write throughput is several hundreds or thousands of writes per second (depending on your platform and hardware, YMMV), so this shouldn't slow down your app too much. If it does, [lets us know](https://github.com/orbitdb/orbit-db/issues)!
Update the database one after another:
```javascript
await db.put('key1', 'hello1')
await db.put('key2', 'hello2')
await db.put('key3', 'hello3')
```
Not:
```javascript
// This is not supported atm!
Promise.all([
db.put('key1', 'hello1'),
db.put('key2', 'hello2'),
db.put('key3', 'hello3')
])
```
## Get an entry
To get a value or entry from the database, we call the appropriate query function which is different per database type.
Key-Value:
```javascript
const ipfs = new IPFS()
ipfs.on('ready', async () => {
const orbitdb = new OrbitDB(ipfs)
const db = await orbitdb.keyvalue('first-database')
await db.put('name', 'hello')
const value = db.get('name')
})
```
Other databases, see:
- [log.iterator()](https://github.com/orbitdb/orbit-db/blob/master/API.md#iteratoroptions)
- [feed.iterator()](https://github.com/orbitdb/orbit-db/blob/master/API.md#iteratoroptions-1)
- [docs.get()](https://github.com/orbitdb/orbit-db/blob/master/API.md#getkey-1)
- [docs.query()](https://github.com/orbitdb/orbit-db/blob/master/API.md#querymapper)
- [counter.value](https://github.com/orbitdb/orbit-db/blob/master/API.md#value)
## Persistency
OrbitDB saves the state of the database automatically on disk. This means that upon opening a database, the developer can choose to load locally the persisted before using the database. **Loading the database locally before using it is highly recommended!**
```javascript
const ipfs = new IPFS()
ipfs.on('ready', async () => {
const orbitdb = new OrbitDB(ipfs)
const db1 = await orbitdb.keyvalue('first-database')
await db1.put('name', 'hello')
await db1.close()
const db2 = await orbitdb.keyvalue('first-database')
await db2.load()
const value = db2.get('name')
// 'hello'
})
```
If the developer doesn't call `load()`, the database will be operational but will not have the persisted data available immediately. Instead, OrbitDB will load the data on the background as new updates come in from peers.
## Replicating a database
In order to have the same data, ie. a query returns the same result for all peers, an OrbitDB database must be replicated between the peers. This happens automatically in OrbitDB in a way that a peer only needs to open an OrbitDB from an address and it'll start replicating the database.
To know when database was updated, we can listen for the `replicated` event of a database: `db2.events.on('replicated', () => ...)`. When the `replicated` event is fired, it means we received updates for the database from a peer. This is a good time to query the database for new results.
Replicate a database between two nodes:
```javascript
// Create the first peer
const ipfs1 = new IPFS({ repo: './ipfs1' })
ipfs1.on('ready', async () => {
// Create the database
const orbitdb1 = new OrbitDB(ipfs1, './orbitdb1')
const db1 = await orbitdb.log('events')
// Create the second peer
const ipfs2 = new IPFS({ repo: './ipfs2' })
ipfs2.on('ready', async () => {
// Open the first database for the second peer,
// ie. replicate the database
const orbitdb2 = new OrbitDB(ipfs2, './orbitdb2')
const db2 = await orbitdb2.log(db1.address.toString())
// When the second database replicated new heads, query the database
db2.events.on('replicated', () => {
const result = db2.iterator({ limit: -1 }).collect()
console.log(result.join('\n'))
})
// Start adding entries to the first database
setInterval(async () => {
await db1.add({ time: new Date().getTime() })
}, 1000)
})
})
```
## More information
Is this guide missing something you'd like to understand or found an error? Please [open an issue](https://github.com/orbitdb/orbit-db/issues) and let us know what's missing!

6
Makefile
View File

@ -10,13 +10,13 @@ build: test
npm run build
mkdir -p examples/browser/lib/
cp dist/orbitdb.min.js examples/browser/lib/orbitdb.min.js
cp node_modules/ipfs-daemon/dist/ipfs-browser-daemon.min.js examples/browser/lib/ipfs-browser-daemon.min.js
cp node_modules/ipfs/dist/index.min.js examples/browser/lib/ipfs.min.js
@echo "Build success!"
@echo "Output: 'dist/', 'examples/browser/'"
clean:
rm -rf orbit-db/
rm -rf ipfs/
rm -rf orbitdb/
rm -rf node_modules/
rm package-lock.json
.PHONY: test build

114
README.md
View File

@ -1,28 +1,31 @@
# orbit-db
[![npm version](https://badge.fury.io/js/orbit-db.svg)](https://badge.fury.io/js/orbit-db)
[![CircleCI Status](https://circleci.com/gh/orbitdb/orbit-db.svg?style=shield)](https://circleci.com/gh/orbitdb/orbit-db)
[![](https://img.shields.io/badge/freenode-%23orbitdb-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23orbitdb)
[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://ipn.io)
[![Project Status](https://badge.waffle.io/haadcode/orbit.svg?label=In%20Progress&title=In%20Progress)](https://waffle.io/haadcode/orbit?source=haadcode%2Forbit-db,haadcode%2Forbit-db-counterstore,haadcode%2Forbit-db-eventstore,haadcode%2Forbit-db-feedstore,haadcode%2Forbit-db-kvstore,haadcode%2Forbit-db-store,haadcode%2Fipfs-log)
[![CircleCI Status](https://circleci.com/gh/orbitdb/orbit-db.svg?style=shield)](https://circleci.com/gh/orbitdb/orbit-db)
[![npm version](https://badge.fury.io/js/orbit-db.svg)](https://www.npmjs.com/package/orbit-db)
[![node](https://img.shields.io/node/v/orbit-db.svg)](https://www.npmjs.com/package/orbit-db)
[![Project Status](https://badge.waffle.io/orbitdb/orbit-db.svg?columns=In%20Progress&title=In%20Progress)](https://waffle.io/orbitdb/orbit-db)
> Distributed, peer-to-peer database for the decentralized web.
> A peer-to-peer database for the decentralized web
`orbit-db` is a serverless, distributed, peer-to-peer database. `orbit-db` uses [IPFS](https://ipfs.io) as its data storage and [IPFS Pubsub](https://github.com/ipfs/go-ipfs/blob/master/core/commands/pubsub.go#L23) to automatically sync databases with peers. It's an eventually consistent database that uses [CRDTs](https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type) for conflict-free database merges making `orbit-db` and excellent choice for offline-first applications.
Data in `orbit-db` can be stored in a
- **[Key-Value Store](https://github.com/orbitdb/orbit-db-kvstore)**
- **[Eventlog](https://github.com/orbitdb/orbit-db-eventstore)** (append-only log)
- **[Feed](https://github.com/orbitdb/orbit-db-feedstore)** (add and remove log)
- **[Documents](https://github.com/orbitdb/orbit-db-docstore)** (indexed by custom fields)
- **[Counters](https://github.com/orbitdb/orbit-db-counterstore)**
- **[Key-Value Store](https://github.com/orbitdb/orbit-db/blob/master/API.md#keyvaluenameaddress)**
- **[Log Database](https://github.com/orbitdb/orbit-db/blob/master/API.md#lognameaddress)** (append-only log)
- **[Feed](https://github.com/orbitdb/orbit-db/blob/master/API.md#feednameaddress)** (same as log database but entries can be removed)
- **[Document Store](https://github.com/orbitdb/orbit-db/blob/master/API.md#docsnameaddress-options)** (store indexed JSON documents)
- **[Counters](https://github.com/orbitdb/orbit-db/blob/master/API.md#counternameaddress)**
This is the Javascript implementation and it works both in **Node.js** and **Browsers**.
To get started, try out the **[OrbitDB CLI](https://github.com/orbitdb/orbit-db-cli)** or check the [live demo](https://ipfs.io/ipfs/QmUETzzv9FxBwPn4H6q3i6QXTzicvV3MMuKN53JQU3yMSG/)!
To get started, try the **[OrbitDB CLI](https://github.com/orbitdb/orbit-db-cli)**, read the **[Getting Started Guide](https://github.com/orbitdb/orbit-db/blob/master/GUIDE.md)** or check **[Live demo 1](https://ipfs.io/ipfs/QmRosp97r8GGUEdj5Wvivrn5nBkuyajhRXFUcWCp5Zubbo/)** and **[Live demo 2](https://ipfs.io/ipfs/QmasHFRj6unJ3nSmtPn97tWDaQWEZw3W9Eh3gUgZktuZDZ/)**!
<p align="left">
<img src="https://raw.githubusercontent.com/orbitdb/orbit-db/feat/write-access/screenshots/example1.png" width="28%">
<a href="https://asciinema.org/a/JdTmmdBCZarkBkPqbueicwMrG" target="_blank"><img src="https://asciinema.org/a/JdTmmdBCZarkBkPqbueicwMrG.png" width="50%"/></a>
</p>
## Table of Contents
@ -36,6 +39,8 @@ To get started, try out the **[OrbitDB CLI](https://github.com/orbitdb/orbit-db-
## Usage
Read the **[GETTING STARTED](https://github.com/orbitdb/orbit-db/blob/master/GUIDE.md)** guide for a more in-depth tutorial and to understand how OrbitDB works.
### CLI
For the CLI tool to manage orbit-db database, see **[OrbitDB CLI](https://github.com/orbitdb/orbit-db-cli)**.
@ -46,51 +51,49 @@ It can be installed from Npm with:
npm install orbit-db-cli -g
```
### Library
### As a library
`orbit-db` can be used in your Javascript programs as a module. This works both in Node.js as well as in the browsers.
To start, install the module with:
Install dependencies:
```
npm install orbit-db ipfs-daemon
npm install orbit-db ipfs
```
Use it as a module:
```javascript
const IPFS = require('ipfs-daemon/src/ipfs-node-daemon')
const IPFS = require('ipfs')
const OrbitDB = require('orbit-db')
const ipfs = new IPFS()
ipfs.on('error', (e) => console.error(e))
ipfs.on('ready', (e) => {
ipfs.on('ready', async () => {
// Create a database
const orbitdb = new OrbitDB(ipfs)
const db = orbitdb.eventlog("feed name")
db.add("hello world")
.then(() => {
const db = await orbitdb.log('database name')
// Add an entry to the database
const hash = await db.add('hello world')
// Get last 5 entries
const latest = db.iterator({ limit: 5 }).collect()
console.log(JSON.stringify(latest, null, 2))
})
})
```
*For more details, see examples for [kvstore](https://github.com/orbitdb/orbit-db-kvstore#usage), [eventlog](https://github.com/orbitdb/orbit-db-eventstore#usage), [feed](https://github.com/orbitdb/orbit-db-feedstore#usage), [docstore](https://github.com/shamb0t/orbit-db-docstore#usage) and [counter](https://github.com/orbitdb/orbit-db-counterstore#usage).*
*For more details, see examples for [kvstore](https://github.com/orbitdb/orbit-db-kvstore#usage), [eventlog](https://github.com/haadcode/orbit-db-eventstore#usage), [feed](https://github.com/haadcode/orbit-db-feedstore#usage), [docstore](https://github.com/shamb0t/orbit-db-docstore#usage) and [counter](https://github.com/haadcode/orbit-db-counterstore#usage).*
## API
See [API documentation](https://github.com/orbitdb/orbit-db/blob/master/API.md#orbit-db-api-documentation) for the full documentation.
- [Getting Started](https://github.com/orbitdb/orbit-db/blob/master/API.md#getting-started)
- [orbitdb](https://github.com/orbitdb/orbit-db/blob/master/API.md#orbitdb)
- [kvstore(name)](https://github.com/orbitdb/orbit-db/blob/master/API.md#kvstorename)
- [eventlog(name)](https://github.com/orbitdb/orbit-db/blob/master/API.md#eventlogname)
- [feed(name)](https://github.com/orbitdb/orbit-db/blob/master/API.md#feedname)
- [docstore(name, options)](https://github.com/orbitdb/orbit-db/blob/master/API.md#docstorename-options)
- [counter(name)](https://github.com/orbitdb/orbit-db/blob/master/API.md#countername)
- [disconnect()](https://github.com/orbitdb/orbit-db/blob/master/API.md#disconnect)
- [events](https://github.com/orbitdb/orbit-db/blob/master/API.md#events)
- [OrbitDB](https://github.com/orbitdb/orbit-db/blob/master/API.md#orbitdb)
- [keyvalue](https://github.com/orbitdb/orbit-db/blob/master/API.md#keyvaluenameaddress)
- [log](https://github.com/orbitdb/orbit-db/blob/master/API.md#lognameaddress)
- [feed](https://github.com/orbitdb/orbit-db/blob/master/API.md#feednameaddress)
- [docstore](https://github.com/orbitdb/orbit-db/blob/master/API.md#docsnameaddress-options)
- [counter](https://github.com/orbitdb/orbit-db/blob/master/API.md#counternameaddress)
- [common](https://github.com/orbitdb/orbit-db/blob/master/API.md#store)
## Examples
@ -105,13 +108,15 @@ npm install
### Browser example
```
npm run build:examples
npm run build
npm run examples:browser
```
<img src="https://raw.githubusercontent.com/orbitdb/orbit-db/feat/ipfs-pubsub/screenshots/orbit-db-demo1.gif" width="33%">
<p align="left">
<img src="https://raw.githubusercontent.com/orbitdb/orbit-db/feat/write-access/screenshots/example1.png" width="33%">
</p>
Check the code in [examples/browser/browser.html](https://github.com/orbitdb/orbit-db/blob/master/examples/browser/browser.html).
Check the code in [examples/browser/browser.html](https://github.com/orbitdb/orbit-db/blob/master/examples/browser/browser.html) and try the [live example](https://ipfs.io/ipfs/QmRosp97r8GGUEdj5Wvivrn5nBkuyajhRXFUcWCp5Zubbo/).
### Node.js example
@ -123,17 +128,12 @@ npm run examples:node
**Eventlog**
Check the code in [examples/eventlog.js](https://github.com/orbitdb/orbit-db/blob/master/examples/eventlog.js) and run it with:
See the code in [examples/eventlog.js](https://github.com/orbitdb/orbit-db/blob/master/examples/eventlog.js) and run it with:
```
LOG=debug node examples/eventlog.js
node examples/eventlog.js
```
**Key-Value**
Check the code in [examples/keyvalue.js](https://github.com/orbitdb/orbit-db/blob/master/examples/keyvalue.js) and run it with:
```
LOG=debug node examples/keyvalue.js
```
More examples at [examples](https://github.com/orbitdb/orbit-db/tree/master/examples).
## Development
@ -149,19 +149,27 @@ npm run build
#### Benchmark
```
node examples/benchmark.js
node benchmarks/benchmark-add.js
```
#### Logging
To enable OrbitDB's logging output, set a global ENV variable called `LOG` to `debug`,`warn` or `error`:
```
LOG=debug node <file>
```
## Background
Check out a visualization of the data flow at https://github.com/haadcode/proto2.
OrbitDB uses an append-only log as its operations log, implemented in [ipfs-log](https://github.com/haadcode/ipfs-log).
**TODO**
To understand a little bit about the architecture, check out a visualization of the data flow at https://github.com/haadcode/proto2 or a live demo: http://celebdil.benet.ai:8080/ipfs/Qmezm7g8mBpWyuPk6D84CNcfLKJwU6mpXuEN5GJZNkX3XK/.
**TODO:**
- list of modules used
- orbit-db-pubsub
- crdts
- ipfs-log
## Contributing
@ -169,6 +177,14 @@ We would be happy to accept PRs! If you want to work on something, it'd be good
A good place to start are the issues labelled ["help wanted"](https://github.com/orbitdb/orbit-db/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22+sort%3Areactions-%2B1-desc) or the project's [status board](https://waffle.io/orbitdb/orbit-db).
## Sponsors
The development of OrbitDB has been sponsored by:
* [Protocol Labs](https://protocol.ai/)
If you want to sponsor developers to work on OrbitDB, please reach out to @haadcode.
## License
[MIT](LICENSE) ©️ 2016 Haadcode
[MIT](LICENSE) ©️ 2017 Haadcode

69
benchmarks/benchmark-add.js Normal file
View File

@ -0,0 +1,69 @@
'use strict'
const IPFS = require('ipfs')
const IPFSRepo = require('ipfs-repo')
const DatastoreLevel = require('datastore-level')
const OrbitDB = require('../src/OrbitDB')
// Metrics
let totalQueries = 0
let seconds = 0
let queriesPerSecond = 0
let lastTenSeconds = 0
// Main loop
const queryLoop = async (db) => {
await db.add(totalQueries)
totalQueries ++
lastTenSeconds ++
queriesPerSecond ++
setImmediate(() => queryLoop(db))
}
// Start
console.log("Starting IPFS daemon...")
const repoConf = {
storageBackends: {
blocks: DatastoreLevel,
},
}
const ipfs = new IPFS({
repo: new IPFSRepo('./orbitdb/benchmarks/ipfs', repoConf),
start: false,
EXPERIMENTAL: {
pubsub: false,
sharding: false,
dht: false,
},
})
ipfs.on('error', (err) => console.error(err))
ipfs.on('ready', async () => {
try {
const orbit = new OrbitDB(ipfs, './orbitdb/benchmarks')
const db = await orbit.eventlog('orbit-db.benchmark', {
replicate: false,
})
// Metrics output
setInterval(() => {
seconds ++
if(seconds % 10 === 0) {
console.log(`--> Average of ${lastTenSeconds/10} q/s in the last 10 seconds`)
if(lastTenSeconds === 0)
throw new Error("Problems!")
lastTenSeconds = 0
}
console.log(`${queriesPerSecond} queries per second, ${totalQueries} queries in ${seconds} seconds (Oplog: ${db._oplog.length})`)
queriesPerSecond = 0
}, 1000)
// Start the main loop
queryLoop(db)
} catch (e) {
console.log(e)
process.exit(1)
}
})

155
benchmarks/benchmark-replication.js Normal file
View File

@ -0,0 +1,155 @@
'use strict'
const IPFS = require('ipfs')
const IPFSRepo = require('ipfs-repo')
const DatastoreLevel = require('datastore-level')
const OrbitDB = require('../src/OrbitDB')
const startIpfs = require('../test/utils/start-ipfs')
const pMapSeries = require('p-map-series')
// Metrics
let metrics1 = {
totalQueries: 0,
seconds: 0,
queriesPerSecond: 0,
lastTenSeconds: 0,
}
let metrics2 = {
totalQueries: 0,
seconds: 0,
queriesPerSecond: 0,
lastTenSeconds: 0,
}
const ipfsConf = {
Addresses: {
API: '/ip4/127.0.0.1/tcp/0',
Swarm: ['/ip4/0.0.0.0/tcp/0'],
Gateway: '/ip4/0.0.0.0/tcp/0'
},
Bootstrap: [],
}
const repoConf = {
storageBackends: {
blocks: DatastoreLevel,
},
}
const defaultConfig = Object.assign({}, {
start: true,
EXPERIMENTAL: {
pubsub: true,
sharding: false,
dht: false,
},
config: ipfsConf
})
const conf1 = Object.assign({}, defaultConfig, {
repo: new IPFSRepo('./orbitdb/benchmarks/replication/client1/ipfs', repoConf)
})
const conf2 = Object.assign({}, defaultConfig, {
repo: new IPFSRepo('./orbitdb/benchmarks/replication/client2/ipfs', repoConf)
})
// Write loop
const queryLoop = async (db) => {
if (metrics1.totalQueries < updateCount) {
try {
await db.add(metrics1.totalQueries)
} catch (e) {
console.error("!!", e)
}
metrics1.totalQueries ++
metrics1.lastTenSeconds ++
metrics1.queriesPerSecond ++
setImmediate(() => queryLoop(db))
}
}
// Metrics output function
const outputMetrics = (name, db, metrics) => {
metrics.seconds ++
console.log(`[${name}] ${metrics.queriesPerSecond} queries per second, ${metrics.totalQueries} queries in ${metrics.seconds} seconds (Oplog: ${db._oplog.length})`)
metrics.queriesPerSecond = 0
if(metrics.seconds % 10 === 0) {
console.log(`[${name}] --> Average of ${metrics.lastTenSeconds/10} q/s in the last 10 seconds`)
metrics.lastTenSeconds = 0
}
}
const database = 'benchmark-replication'
const updateCount = 2000
// Start
console.log("Starting IPFS daemons...")
pMapSeries([conf1, conf2], d => startIpfs(d))
.then(async ([ipfs1, ipfs2]) => {
try {
// Create the databases
const orbit1 = new OrbitDB(ipfs1, './orbitdb/benchmarks/replication/client1')
const orbit2 = new OrbitDB(ipfs2, './orbitdb/benchmarks/replication/client2')
const db1 = await orbit1.eventlog(database, { overwrite: true })
const db2 = await orbit2.eventlog(db1.address.toString())
let db1Connected = false
let db2Connected = false
console.log('Waiting for peers to connect...')
db1.events.on('peer', () => {
db1Connected = true
console.log('Peer 1 connected')
})
db2.events.on('peer', () => {
db2Connected = true
console.log('Peer 2 connected')
})
const startInterval = setInterval(() => {
if (db1Connected && db2Connected) {
clearInterval(startInterval)
// Start the write loop
queryLoop(db1)
// Metrics output for the writer, once/sec
const writeInterval = setInterval(() => {
outputMetrics("WRITE", db1, metrics1)
if (metrics1.totalQueries === updateCount) {
clearInterval(writeInterval)
}
}, 1000)
// Metrics output for the reader
let prevCount = 0
setInterval(() => {
try {
const result = db2.iterator({ limit: -1 }).collect()
metrics2.totalQueries = result.length
metrics2.queriesPerSecond = metrics2.totalQueries - prevCount
metrics2.lastTenSeconds += metrics2.queriesPerSecond
prevCount = metrics2.totalQueries
outputMetrics("READ", db2, metrics2)
if (db2._oplog.length === updateCount) {
console.log("Finished")
process.exit(0)
}
} catch (e) {
console.error("!", e)
}
}, 1000)
}
}, 100)
} catch (e) {
console.log(e)
process.exit(1)
}
})

78
benchmarks/browser/benchmark-add.html Normal file
View File

@ -0,0 +1,78 @@
<html>
<head>
<meta charset="utf-8">
</head>
<body>
<h1>OrbitDB - Benchmark log.add()</h1>
<h2>Description</h2>
<div>Add an entry to a log database. Measure throughput in write operations per second.</div>
<h2>Results</h2>
<pre id="output"></pre>
<script type="text/javascript" src="../../dist/orbitdb.min.js" charset="utf-8"></script>
<script type="text/javascript" src="../../node_modules/ipfs/dist/index.min.js" charset="utf-8"></script>
<script type="text/javascript">
let ipfs
// Metrics
let totalQueries = 0
let seconds = 0
let queriesPerSecond = 0
let lastTenSeconds = 0
const queryLoop = async (db) => {
await db.add(totalQueries)
totalQueries ++
lastTenSeconds ++
queriesPerSecond ++
setImmediate(() => queryLoop(db))
}
let run = (() => {
ipfs = new Ipfs({
repo: '/orbitdb/benchmarks/browser/benchmark-add',
start: false,
EXPERIMENTAL: {
pubsub: true,
sharding: false,
dht: false,
},
})
ipfs.on('error', (err) => console.error(err))
ipfs.on('ready', async () => {
const outputElm = document.getElementById('output')
try {
const orbit = new OrbitDB(ipfs, './orbitdb/benchmarks/browser')
const db = await orbit.eventlog('orbit-db.benchmark.add', {
replicate: false,
})
// Metrics output
setInterval(() => {
seconds ++
if(seconds % 10 === 0) {
outputElm.innerHTML = `--> Average of ${lastTenSeconds/10} q/s in the last 10 seconds<br>` + outputElm.innerHTML
console.log(`--> Average of ${lastTenSeconds/10} q/s in the last 10 seconds`)
if(lastTenSeconds === 0)
throw new Error("Problems!")
lastTenSeconds = 0
}
outputElm.innerHTML = `${queriesPerSecond} queries per second, ${totalQueries} queries in ${seconds} seconds (Entries: ${db._oplog.length})<br>` + outputElm.innerHTML
// console.log(`${queriesPerSecond} queries per second, ${totalQueries} queries in ${seconds} seconds (Entries: ${db._oplog.length})`)
queriesPerSecond = 0
}, 1000)
// Start the main loop
queryLoop(db)
} catch (e) {
console.error(e)
}
})
})()
</script>
</body>
</html>

135
benchmarks/browser/benchmark-replication-sender.html Normal file
View File

@ -0,0 +1,135 @@
<html>
<head>
<meta charset="utf-8">
</head>
<body>
<h1>OrbitDB - Benchmark Replication</h1>
<h2>Description</h2>
<div>Two peers</div>
<h2>Results</h2>
<pre id="output"></pre>
<script type="text/javascript" src="../../dist/orbitdb.min.js" charset="utf-8"></script>
<script type="text/javascript" src="../../node_modules/ipfs/dist/index.min.js" charset="utf-8"></script>
<script type="text/javascript">
// Metrics
let metrics1 = {
totalQueries: 0,
seconds: 0,
queriesPerSecond: 0,
lastTenSeconds: 0,
}
const ipfsConf = {
Addresses: {
Swarm: [
// Use IPFS dev signal server
'/dns4/star-signal.cloud.ipfs.team/wss/p2p-webrtc-star',
// '/ip4/0.0.0.0/tcp/9090/wss/p2p-webrtc-star',
]
},
Bootstrap: [],
Discovery: {
MDNS: {
Enabled: false,
Interval: 10
},
webRTCStar: {
Enabled: false
}
},
}
const defaultConfig = Object.assign({}, {
start: true,
EXPERIMENTAL: {
pubsub: true,
sharding: false,
dht: false,
},
config: ipfsConf
})
const conf1 = Object.assign({}, defaultConfig, {
repo: './orbitdb/benchmarks/2replication3/client2/ipfs'
})
// Write loop
const queryLoop = async (db) => {
if (metrics1.totalQueries < updateCount) {
await db.add(metrics1.totalQueries)
metrics1.totalQueries ++
metrics1.lastTenSeconds ++
metrics1.queriesPerSecond ++
setImmediate(() => queryLoop(db))
}
}
// Metrics output function
const outputMetrics = (name, db, metrics) => {
metrics.seconds ++
console.log(`[${name}] ${metrics.queriesPerSecond} queries per second, ${metrics.totalQueries} queries in ${metrics.seconds} seconds (Oplog: ${db._oplog.length})`)
metrics.queriesPerSecond = 0
if(metrics.seconds % 10 === 0) {
console.log(`[${name}] --> Average of ${metrics.lastTenSeconds/10} q/s in the last 10 seconds`)
metrics.lastTenSeconds = 0
}
}
const startIpfs = (config = {}) => {
return new Promise((resolve, reject) => {
const ipfs = new Ipfs(config)
ipfs.on('error', reject)
ipfs.on('ready', () => resolve(ipfs))
})
}
const database = 'benchmark-replication'
const updateCount = 2000
// Start
console.log("Starting IPFS daemons...")
let run = (async () => {
let ipfs1 = await startIpfs(conf1)
try {
// Create the databases
const orbit1 = new OrbitDB(ipfs1, './orbitdb/benchmarks/replication/client1')
const db1 = await orbit1.eventlog(database, { overwrite: true, sync: false })
console.log("Database address is:", db1.address.toString())
db1.events.on('peer', peer => console.log("PEER1!", peer))
let db1Connected = false
db1.events.on('peer', () => {
db1Connected = true
})
const startInterval = setInterval(() => {
if (db1Connected) {
clearInterval(startInterval)
// Start the write loop
queryLoop(db1)
// Metrics output for the writer, once/sec
const writeInterval = setInterval(() => {
outputMetrics("WRITE", db1, metrics1)
if (metrics1.totalQueries === updateCount) {
clearInterval(writeInterval)
}
}, 1000)
}
}, 100)
} catch (e) {
console.error(e)
}
})()
</script>
</body>
</html>

150
benchmarks/browser/benchmark-replication.html Normal file
View File

@ -0,0 +1,150 @@
<html>
<head>
<meta charset="utf-8">
</head>
<body>
<h1>OrbitDB - Benchmark Replication</h1>
<h2>Description</h2>
<div>Two peers</div>
<h2>Results</h2>
<pre id="output"></pre>
<script type="text/javascript" src="../../dist/orbitdb.min.js" charset="utf-8"></script>
<script type="text/javascript" src="../../node_modules/ipfs/dist/index.min.js" charset="utf-8"></script>
<script type="text/javascript">
// Metrics
let metrics1 = {
totalQueries: 0,
seconds: 0,
queriesPerSecond: 0,
lastTenSeconds: 0,
}
let metrics2 = {
totalQueries: 0,
seconds: 0,
queriesPerSecond: 0,
lastTenSeconds: 0,
}
const ipfsConf = {
Addresses: {
Swarm: [
// Use IPFS dev signal server
'/dns4/star-signal.cloud.ipfs.team/wss/p2p-webrtc-star',
// '/ip4/0.0.0.0/tcp/9090/wss/p2p-webrtc-star',
]
},
Bootstrap: [],
Discovery: {
MDNS: {
Enabled: false,
Interval: 10
},
webRTCStar: {
Enabled: false
}
},
}
const defaultConfig = Object.assign({}, {
start: true,
EXPERIMENTAL: {
pubsub: true,
sharding: false,
dht: false,
},
config: ipfsConf
})
const conf1 = Object.assign({}, defaultConfig, {
repo: './orbitdb/benchmarks/2replication3/client1/ipfs'
})
const conf2 = Object.assign({}, defaultConfig, {
repo: './orbitdb/benchmarks/2replication3/client2/ipfs'
})
// Write loop
const queryLoop = async (db) => {
if (metrics1.totalQueries < updateCount) {
await db.add(metrics1.totalQueries)
metrics1.totalQueries ++
metrics1.lastTenSeconds ++
metrics1.queriesPerSecond ++
setImmediate(() => queryLoop(db))
}
}
// Metrics output function
const outputMetrics = (name, db, metrics) => {
metrics.seconds ++
console.log(`[${name}] ${metrics.queriesPerSecond} queries per second, ${metrics.totalQueries} queries in ${metrics.seconds} seconds (Oplog: ${db._oplog.length})`)
metrics.queriesPerSecond = 0
if(metrics.seconds % 10 === 0) {
console.log(`[${name}] --> Average of ${metrics.lastTenSeconds/10} q/s in the last 10 seconds`)
metrics.lastTenSeconds = 0
}
}
const startIpfs = (config = {}) => {
return new Promise((resolve, reject) => {
const ipfs = new Ipfs(config)
ipfs.on('error', reject)
ipfs.on('ready', () => resolve(ipfs))
})
}
const database = 'benchmark-replication'
const updateCount = 2000
// Start
console.log("Starting IPFS daemons...")
let run = (async () => {
let ipfs2 = await startIpfs(conf2)
try {
// Create the databases
const address = '/orbitdb/QmcPCAwwV1rw7cLQU7VcCaUXEuLYSCH8uUf6NPDLYbL6JT/benchmark-replication'
const orbit2 = new OrbitDB(ipfs2, './orbitdb/benchmarks/replication/client3')
const db2 = await orbit2.eventlog(address)
db2.events.on('peer', peer => console.log("PEER2!", peer))
let db2Connected = false
db2.events.on('peer', () => {
db2Connected = true
})
const startInterval = setInterval(() => {
if (db2Connected) {
clearInterval(startInterval)
// Metrics output for the reader
let prevCount = 0
setInterval(() => {
const result = db2.iterator({ limit: -1 }).collect()
metrics2.totalQueries = result.length
metrics2.queriesPerSecond = metrics2.totalQueries - prevCount
metrics2.lastTenSeconds += metrics2.queriesPerSecond
prevCount = metrics2.totalQueries
outputMetrics("READ", db2, metrics2)
if (db2._oplog.length === updateCount) {
console.log("Finished")
}
}, 1000)
}
}, 100)
} catch (e) {
console.error(e)
}
})()
</script>
</body>
</html>

2
circle.yml
View File

@ -1,3 +1,3 @@
machine:
node:
version: 6.1.0
version: 8.2.0

27
conf/webpack.config.js
View File

@ -1,7 +1,8 @@
'use strict'
const webpack = require('webpack')
const path = require('path')
const webpack = require('webpack')
const Uglify = require('uglifyjs-webpack-plugin')
module.exports = {
entry: './src/OrbitDB.js',
@ -10,7 +11,23 @@ module.exports = {
library: 'OrbitDB',
filename: './dist/orbitdb.min.js'
},
devtool: 'source-map',
target: 'web',
devtool: 'none',
externals: {
fs: '{}',
},
node: {
console: false,
Buffer: true
},
plugins: [
new webpack.DefinePlugin({
'process.env': {
'NODE_ENV': JSON.stringify(process.env.NODE_ENV)
}
}),
new Uglify(),
],
resolve: {
modules: [
'node_modules',
@ -24,10 +41,4 @@ module.exports = {
],
moduleExtensions: ['-loader']
},
node: {
console: false,
Buffer: true
},
plugins: [],
target: 'web'
}

37
conf/webpack.debug.config.js Normal file
View File

@ -0,0 +1,37 @@
'use strict'
const path = require('path')
const Uglify = require('uglifyjs-webpack-plugin')
module.exports = {
entry: './src/OrbitDB.js',
output: {
libraryTarget: 'var',
library: 'OrbitDB',
filename: './dist/orbitdb.js'
},
target: 'web',
devtool: 'none',
externals: {
fs: '{}',
},
node: {
console: false,
Buffer: true
},
plugins: [
],
resolve: {
modules: [
'node_modules',
path.resolve(__dirname, '../node_modules')
]
},
resolveLoader: {
modules: [
'node_modules',
path.resolve(__dirname, '../node_modules')
],
moduleExtensions: ['-loader']
},
}

59
conf/webpack.example.config.js
View File

@ -1,43 +1,43 @@
const webpack = require('webpack')
'use strict'
const path = require('path')
const webpack = require('webpack')
const Uglify = require('uglifyjs-webpack-plugin')
const uglifyOptions = {
uglifyOptions: {
mangle: false,
},
}
module.exports = {
entry: './examples/browser/index.js',
entry: './examples/browser/browser-webpack-example/index.js',
output: {
filename: './examples/browser/bundle.js'
},
devtool: 'sourcemap',
stats: {
colors: true,
cached: false
filename: './examples/browser/browser-webpack-example/bundle.js'
},
target: 'web',
devtool: 'none',
node: {
console: false,
Buffer: true,
process: 'mock',
Buffer: true
},
externals: {
fs: '{}',
},
plugins: [
new webpack.optimize.UglifyJsPlugin({
mangle: false,
compress: { warnings: false }
})
new webpack.DefinePlugin({
'process.env': {
'NODE_ENV': JSON.stringify(process.env.NODE_ENV)
}
}),
new Uglify(uglifyOptions),
],
resolve: {
modules: [
'node_modules',
path.resolve(__dirname, '../node_modules')
],
alias: {
// These are needed because node-libs-browser depends on outdated
// versions
//
// Can be dropped once https://github.com/devongovett/browserify-zlib/pull/18
// is shipped
zlib: 'browserify-zlib-next',
// Can be dropped once https://github.com/webpack/node-libs-browser/pull/41
// is shipped
http: 'stream-http'
}
},
resolveLoader: {
modules: [
@ -46,15 +46,4 @@ module.exports = {
],
moduleExtensions: ['-loader']
},
module: {
rules: [{
test: /\.json$/,
loader: 'json-loader'
}]
},
node: {
Buffer: true
},
plugins: [],
target: 'web'
}

14498
dist/orbitdb.js vendored
View File

File diff suppressed because it is too large Load Diff

9
dist/orbitdb.min.js vendored
View File

File diff suppressed because one or more lines are too long

50
examples/benchmark.js
View File

@ -1,50 +0,0 @@
'use strict'
const IpfsDaemon = require('ipfs-daemon/src/ipfs-node-daemon')
const OrbitDB = require('../src/OrbitDB')
// Metrics
let totalQueries = 0
let seconds = 0
let queriesPerSecond = 0
let lastTenSeconds = 0
// Main loop
const queryLoop = (db) => {
db.add(totalQueries)
.then(() => {
totalQueries ++
lastTenSeconds ++
queriesPerSecond ++
setImmediate(() => queryLoop(db))
})
.catch((e) => console.error(e))
}
// Start
console.log("Starting IPFS daemon...")
const ipfs = new IpfsDaemon()
ipfs.on('error', (err) => console.error(err))
ipfs.on('ready', () => {
const orbit = new OrbitDB(ipfs, 'benchmark')
const db = orbit.eventlog('orbit-db.benchmark', { maxHistory: 100 })
// Metrics output
setInterval(() => {
seconds ++
if(seconds % 10 === 0) {
console.log(`--> Average of ${lastTenSeconds/10} q/s in the last 10 seconds`)
if(lastTenSeconds === 0)
throw new Error("Problems!")
lastTenSeconds = 0
}
console.log(`${queriesPerSecond} queries per second, ${totalQueries} queries in ${seconds} seconds`)
queriesPerSecond = 0
}, 1000)
// Start the main loop
queryLoop(db)
})

41
examples/browser/browser-webpack-example/browser.css Normal file
View File

@ -0,0 +1,41 @@
body {
font-family: 'Abel', sans-serif;
font-size: 0.8em;
}
#logo {
border-top: 1px dotted black;
border-bottom: 1px dotted black;
}
#status {
border-top: 1px dotted black;
border-bottom: 1px dotted black;
padding: 0.5em 0em;
text-align: center;
}
#results {
border: 1px dotted black;
padding: 0.5em;
}
#writerText {
padding-top: 0.5em;
}
pre {
text-align: center;
}
input {
padding: 0.5em;
}
h2 {
margin-bottom: 0.2em;
}
h3 {
margin-top: 0.8em;
margin-bottom: 0.2em;
}

50
examples/browser/browser-webpack-example/index.html Normal file
View File

@ -0,0 +1,50 @@
<html>
<head>
<meta charset="utf-8">
<link href="browser.css" rel="stylesheet">
</head>
<body>
<div id="logo">
<pre>
_ _ _ _ _
| | (_) | | | |
___ _ __| |__ _| |_ __| | |__
/ _ \| '__| '_ \| | __| / _\` | '_\
| (_) | | | |_) | | |_ | (_| | |_) |
\___/|_| |_.__/|_|\__| \__,_|_.__/
Peer-to-Peer Database for the Decentralized Web
<a href="https://github.com/orbitdb/orbit-db" target="_blank">https://github.com/orbitdb/orbit-db</a>
</pre>
</div>
<h2>Open or Create Local Database</h2>
<i>Open a database locally and create it if the database doesn't exist.</i>
<br><br>
<input id="dbname" type="text" placeholder="Database name"/>
<button id="create" type="button" disabled>Open</button>
<select id="type">
<option value="eventlog">Eventlog</option>
<option value="feed">Feed</option>
<option value="keyvalue">Key-Value</option>
<option value="docstore">DocumentDB</option>
<option value="counter">Counter</option>
</select>
<input id="public" type="checkbox" checked> Public
<h2>Open Remote Database</h2>
<i>Open a database from an OrbitDB address, eg. /orbitdb/QmfY3udPcWUD5NREjrUV351Cia7q4DXNcfyRLJzUPL3wPD/hello</i>
<br>
<i><b>Note!</b> Open the remote database in an Incognito Window or in a different browser. It won't work if you don't.</i>
<br><br>
<input id="dbaddress" type="text" placeholder="Address"/>
<button id="open" type="button" disabled>Open</button>
<input id="readonly" type="checkbox" checked> Read-only
<br><br>
<div id="status">Init</div>
<div id="output"></div>
<div id="writerText"></div>
<script type="text/javascript" src="bundle.js" charset="utf-8"></script>
<link href="https://fonts.googleapis.com/css?family=Abel" rel="stylesheet">
</body>
</html>

19
examples/browser/browser-webpack-example/index.js Normal file
View File

@ -0,0 +1,19 @@
'use strict'
/*
This is the entry point for Webpack to build the bundle from.
We use the same example code as the html browser example,
but we inject the Node.js modules of OrbitDB and IPFS into
the example.
In the html example, IPFS and OrbitDB are loaded from the
minified distribution builds (in '../lib')
*/
const IPFS = require('ipfs')
const OrbitDB = require('../../../src/OrbitDB')
const example = require('../example')
// Call the start function and pass in the
// IPFS and OrbitDB modules
example(IPFS, OrbitDB)

41
examples/browser/browser.css Normal file
View File

@ -0,0 +1,41 @@
body {
font-family: 'Abel', sans-serif;
font-size: 0.8em;
}
#logo {
border-top: 1px dotted black;
border-bottom: 1px dotted black;
}
#status {
border-top: 1px dotted black;
border-bottom: 1px dotted black;
padding: 0.5em 0em;
text-align: center;
}
#results {
border: 1px dotted black;
padding: 0.5em;
}
#writerText {
padding-top: 0.5em;
}
pre {
text-align: center;
}
input {
padding: 0.5em;
}
h2 {
margin-bottom: 0.2em;
}
h3 {
margin-top: 0.8em;
margin-bottom: 0.2em;
}

155
examples/browser/browser.html
View File

@ -1,122 +1,57 @@
<html>
<head>
<meta charset="utf-8">
<link href="browser.css" rel="stylesheet">
<link href="https://fonts.googleapis.com/css?family=Abel" rel="stylesheet">
</head>
<body>
<input id="dbname" type="text" placeholder="Database name"/>
<button id="open" type="button">Open</button>
<div id="logo">
<pre>
_ _ _ _ _
| | (_) | | | |
___ _ __| |__ _| |_ __| | |__
/ _ \| '__| '_ \| | __| / _\` | '_\
| (_) | | | |_) | | |_ | (_| | |_) |
\___/|_| |_.__/|_|\__| \__,_|_.__/
Peer-to-Peer Database for the Decentralized Web
<a href="https://github.com/orbitdb/orbit-db" target="_blank">https://github.com/orbitdb/orbit-db</a>
</pre>
</div>
<h2>Open or Create Local Database</h2>
<i>Open a database locally and create it if the database doesn't exist.</i>
<br><br>
<input id="dbname" type="text" placeholder="Database name"/>
<button id="create" type="button" disabled>Open</button>
<select id="type">
<option value="eventlog">Eventlog</option>
<option value="feed">Feed</option>
<option value="keyvalue">Key-Value</option>
<option value="docstore">DocumentDB</option>
<option value="counter">Counter</option>
</select>
<input id="public" type="checkbox" checked> Public
<h2>Open Remote Database</h2>
<i>Open a database from an OrbitDB address, eg. /orbitdb/QmfY3udPcWUD5NREjrUV351Cia7q4DXNcfyRLJzUPL3wPD/hello</i>
<br>
<i><b>Note!</b> Open the remote database in an Incognito Window or in a different browser. It won't work if you don't.</i>
<br><br>
<input id="dbaddress" type="text" placeholder="Address"/>
<button id="open" type="button" disabled>Open</button>
<input id="readonly" type="checkbox" checked> Read-only
<br><br>
<div id="status">Init</div>
<div id="output"></div>
<div id="writerText"></div>
<script type="text/javascript" src="lib/orbitdb.min.js" charset="utf-8"></script>
<script type="text/javascript" src="lib/ipfs-browser-daemon.min.js" charset="utf-8"></script>
<script type="text/javascript">
const elm = document.getElementById("output")
const dbnameField = document.getElementById("dbname")
const openButton = document.getElementById("open")
const openDatabase = () => {
openButton.disabled = true
elm.innerHTML = "Starting IPFS..."
const dbname = dbnameField.value
const username = new Date().getTime()
const key = 'greeting'
const ipfs = new IpfsDaemon({
IpfsDataDir: '/orbit-db-/examples/browser',
SignalServer: 'star-signal.cloud.ipfs.team', // IPFS dev server
})
function handleError(e) {
console.error(e.stack)
elm.innerHTML = e.message
}
ipfs.on('error', (e) => handleError(e))
ipfs.on('ready', () => {
elm.innerHTML = "Loading database..."
const orbit = new OrbitDB(ipfs, username)
const db = orbit.kvstore(dbname, { maxHistory: 5, syncHistory: false, cachePath: '/orbit-db' })
const log = orbit.eventlog(dbname + ".log", { maxHistory: 5, syncHistory: false, cachePath: '/orbit-db' })
const counter = orbit.counter(dbname + ".count", { maxHistory: 5, syncHistory: false, cachePath: '/orbit-db' })
const creatures = ['👻', '🐙', '🐷', '🐬', '🐞', '🐈', '🙉', '🐸', '🐓']
const idx = Math.floor(Math.random() * creatures.length)
const creature = creatures[idx]
const interval = Math.floor((Math.random() * 5000) + 3000)
let count = 0
const query = () => {
const value = "GrEEtinGs from " + username + " " + creature + ": Hello #" + count + " (" + new Date().getTime() + ")"
// Set a key-value pair
count ++
db.put(key, value)
.then(() => counter.inc()) // Increase the counter by one
.then(() => log.add(value)) // Add an event to 'latest visitors' log
.then(() => getData())
.catch((e) => handleError(e))
}
const getData = () => {
const result = db.get(key)
const latest = log.iterator({ limit: 5 }).collect()
const count = counter.value
ipfs.pubsub.peers(dbname + ".log")
.then((peers) => {
const output = `
<b>You are:</b> ${username} ${creature}<br>
<b>Peers:</b> ${peers.length}<br>
<b>Database:</b> ${dbname}<br>
<br><b>Writing to database every ${interval} milliseconds...</b><br><br>
<b>Key-Value Store</b>
-------------------------------------------------------
Key | Value
-------------------------------------------------------
${key} | ${result}
-------------------------------------------------------
<b>Eventlog</b>
-------------------------------------------------------
Latest Updates
-------------------------------------------------------
${latest.reverse().map((e) => e.payload.value).join('\n')}
<b>Counter</b>
-------------------------------------------------------
Visitor Count: ${count}
-------------------------------------------------------
`
elm.innerHTML = output.split("\n").join("<br>")
})
}
db.events.on('synced', () => getData())
counter.events.on('synced', () => getData())
log.events.on('synced', () => getData())
db.events.on('ready', () => getData())
counter.events.on('ready', () => getData())
log.events.on('ready', () => getData())
// Start query loop when the databse has loaded its history
db.load(10)
.then(() => counter.load(10))
.then(() => log.load(10))
.then(() => {
count = counter.value
setInterval(query, interval)
})
})
}
openButton.addEventListener('click', openDatabase)
<script type="text/javascript" src="lib/ipfs.min.js" charset="utf-8"></script>
<script type="text/javascript" src="example.js" charset="utf-8"></script>
<script type="text/javascript" charset="utf-8">
// Start the example
main()
</script>
</body>
</html>

276
examples/browser/example.js Normal file
View File

@ -0,0 +1,276 @@
const creatures = [
'🐙', '🐷', '🐬', '🐞',
'🐈', '🙉', '🐸', '🐓',
'🐊', '🕷', '🐠', '🐘',
'🐼', '🐰', '🐶', '🐥'
]
const elm = document.getElementById("output")
const statusElm = document.getElementById("status")
const dbnameField = document.getElementById("dbname")
const dbAddressField = document.getElementById("dbaddress")
const createButton = document.getElementById("create")
const openButton = document.getElementById("open")
const createType = document.getElementById("type")
const writerText = document.getElementById("writerText")
const publicCheckbox = document.getElementById("public")
const readonlyCheckbox = document.getElementById("readonly")
function handleError(e) {
console.error(e.stack)
statusElm.innerHTML = e.message
}
const main = (IPFS, ORBITDB) => {
let ipfsReady = false
let orbitdb, db
let count = 0
let interval = Math.floor((Math.random() * 300) + (Math.random() * 2000))
let updateInterval
// If we're building with Webpack, use the injected IPFS module.
// Otherwise use 'Ipfs' which is exposed by ipfs.min.js
if (IPFS)
Ipfs = IPFS
// If we're building with Webpack, use the injected OrbitDB module.
// Otherwise use 'OrbitDB' which is exposed by orbitdb.min.js
if (ORBITDB)
OrbitDB = ORBITDB
// Init UI
openButton.disabled = true
createButton.disabled = true
statusElm.innerHTML = "Starting IPFS..."
// Create IPFS instance
const ipfs = new Ipfs({
// repo: '/orbitdb/examples/browser/ipfs' + new Date().getTime(),
repo: '/orbitdb/examples/browser/new/ipfs',
EXPERIMENTAL: {
pubsub: true,
},
config: {
Addresses: {
Swarm: [
// Use IPFS dev signal server
'/dns4/star-signal.cloud.ipfs.team/wss/p2p-webrtc-star',
// Use local signal server
// '/ip4/0.0.0.0/tcp/9090/wss/p2p-webrtc-star',
]
},
Bootstrap: [],
Discovery: {
MDNS: {
Enabled: true,
Interval: 10
},
webRTCStar: {
Enabled: true
}
},
}
})
ipfs.on('error', (e) => handleError(e))
ipfs.on('ready', () => {
openButton.disabled = false
createButton.disabled = false
statusElm.innerHTML = "IPFS Started"
orbitdb = new OrbitDB(ipfs)
})
const load = async (db, statusText) => {
// Set the status text
statusElm.innerHTML = statusText
// When the database is ready (ie. loaded), display results
db.events.on('ready', () => queryAndRender(db))
// When database gets replicated with a peer, display results
db.events.on('replicated', () => queryAndRender(db))
// When we update the database, display result
db.events.on('write', () => queryAndRender(db))
// Hook up to the load progress event and render the progress
let maxTotal = 0, loaded = 0
db.events.on('load.progress', (address, hash, entry, progress, total) => {
loaded ++
maxTotal = Math.max.apply(null, [maxTotal, progress, 0])
statusElm.innerHTML = `Loading database... ${maxTotal} / ${total}`
})
db.events.on('ready', () => {
// Set the status text
setTimeout(() => {
statusElm.innerHTML = 'Database is ready'
}, 1000)
})
// Load locally persisted database
await db.load()
}
const startWriter = async (db, interval) => {
// Set the status text
writerText.innerHTML = `Writing to database every ${interval} milliseconds...`
// Start update/insert loop
updateInterval = setInterval(async () => {
try {
await update(db)
} catch (e) {
console.error(e.toString())
if (e.toString() === 'Error: Not allowed to write') {
writerText.innerHTML = '<span style="color: red">' + e.toString() + '</span>'
clearInterval(updateInterval)
}
}
}, interval)
}
const resetDatabase = async (db) => {
writerText.innerHTML = ""
elm.innerHTML = ""
clearInterval(updateInterval)
if (db) {
await db.close()
}
interval = Math.floor((Math.random() * 300) + (Math.random() * 2000))
}
const createDatabase = async () => {
await resetDatabase(db)
openButton.disabled = true
createButton.disabled = true
try {
const name = dbnameField.value
const type = createType.value
const publicAccess = publicCheckbox.checked
db = await orbitdb.open(name, {
// If database doesn't exist, create it
create: true,
overwrite: true,
// Load only the local version of the database,
// don't load the latest from the network yet
localOnly: false,
type: type,
// If "Public" flag is set, allow anyone to write to the database,
// otherwise only the creator of the database can write
write: publicAccess ? ['*'] : [],
})
await load(db, 'Creating database...')
startWriter(db, interval)
} catch (e) {
console.error(e)
}
openButton.disabled = false
createButton.disabled = false
}
const openDatabase = async () => {
const address = dbAddressField.value
await resetDatabase(db)
openButton.disabled = true
createButton.disabled = true
try {
statusElm.innerHTML = "Connecting to peers..."
db = await orbitdb.open(address, { sync: true })
await load(db, 'Loading database...')
if (!readonlyCheckbox.checked) {
startWriter(db, interval)
} else {
writerText.innerHTML = `Listening for updates to the database...`
}
} catch (e) {
console.error(e)
}
openButton.disabled = false
createButton.disabled = false
}
const update = async (db) => {
count ++
const time = new Date().toISOString()
const idx = Math.floor(Math.random() * creatures.length)
const creature = creatures[idx]
if (db.type === 'eventlog') {
const value = "GrEEtinGs from " + orbitdb.id + " " + creature + ": Hello #" + count + " (" + time + ")"
await db.add(value)
} else if (db.type === 'feed') {
const value = "GrEEtinGs from " + orbitdb.id + " " + creature + ": Hello #" + count + " (" + time + ")"
await db.add(value)
} else if (db.type === 'docstore') {
const value = { _id: 'peer1', avatar: creature, updated: time }
await db.put(value)
} else if (db.type === 'keyvalue') {
await db.set('mykey', creature)
} else if (db.type === 'counter') {
await db.inc(1)
} else {
throw new Error("Unknown datatbase type: ", db.type)
}
}
const query = (db) => {
if (db.type === 'eventlog')
return db.iterator({ limit: 5 }).collect()
else if (db.type === 'feed')
return db.iterator({ limit: 5 }).collect()
else if (db.type === 'docstore')
return db.get('peer1')
else if (db.type === 'keyvalue')
return db.get('mykey')
else if (db.type === 'counter')
return db.value
else
throw new Error("Unknown datatbase type: ", db.type)
}
const queryAndRender = async (db) => {
const networkPeers = await ipfs.swarm.peers()
const databasePeers = await ipfs.pubsub.peers(db.address.toString())
const result = query(db)
const output = `
<h2>${db.type.toUpperCase()}</h2>
<h3 id="remoteAddress">${db.address}</h3>
<div><i>Copy this address and use the 'Open Remote Database' in another browser to replicate this database between peers.</i></div>
<br>
<div><b>Peer ID:</b> ${orbitdb.id}</div>
<div><b>Peers (database/network):</b> ${databasePeers.length} / ${networkPeers.length}</div>
<div><b>Oplog Size:</b> ${db._oplog.length} / ${db._replicationInfo.max}</div>
<h2>Results</h2>
<div id="results">
<div>
${result && Array.isArray(result) && result.length > 0 && db.type !== 'docstore' && db.type !== 'keyvalue'
? result.slice().reverse().map((e) => e.payload.value).join('<br>\n')
: db.type === 'docstore'
? JSON.stringify(result, null, 2)
: result ? result.toString().replace('"', '').replace('"', '') : result
}
</div>
</div>
`
elm.innerHTML = output
}
openButton.addEventListener('click', openDatabase)
createButton.addEventListener('click', createDatabase)
}
if (typeof module !== 'undefined' && typeof module.exports !== 'undefined')
module.exports = main

12
examples/browser/index.html
View File

@ -1,12 +0,0 @@
<html>
<head>
<meta charset="utf-8">
</head>
<body>
<input id="dbname" type="text" placeholder="Database name"/>
<button id="open" type="button">Open</button>
<br><br>
<div id="output"></div>
<script type="text/javascript" src="bundle.js" charset="utf-8"></script>
</body>
</html>

110
examples/browser/index.js
View File

@ -1,110 +0,0 @@
'use strict'
const IPFS = require('ipfs-daemon/src/ipfs-browser-daemon')
const OrbitDB = require('../../src/OrbitDB')
const elm = document.getElementById("output")
const dbnameField = document.getElementById("dbname")
const openButton = document.getElementById("open")
const openDatabase = () => {
openButton.disabled = true
elm.innerHTML = "Starting IPFS..."
const dbname = dbnameField.value
const username = new Date().getTime()
const key = 'greeting'
const ipfs = new IPFS({
IpfsDataDir: '/orbit-db-/examples/browser',
SignalServer: 'star-signal.cloud.ipfs.team', // IPFS dev server
})
function handleError(e) {
console.error(e.stack)
elm.innerHTML = e.message
}
ipfs.on('error', (e) => handleError(e))
ipfs.on('ready', () => {
elm.innerHTML = "Loading database..."
const orbit = new OrbitDB(ipfs, username)
const db = orbit.kvstore(dbname, { maxHistory: 5, syncHistory: false, cachePath: '/orbit-db' })
const log = orbit.eventlog(dbname + ".log", { maxHistory: 5, syncHistory: false, cachePath: '/orbit-db' })
const counter = orbit.counter(dbname + ".count", { maxHistory: 5, syncHistory: false, cachePath: '/orbit-db' })
const creatures = ['👻', '🐙', '🐷', '🐬', '🐞', '🐈', '🙉', '🐸', '🐓']
const idx = Math.floor(Math.random() * creatures.length)
const creature = creatures[idx]
const interval = Math.floor((Math.random() * 5000) + 3000)
let count = 0
const query = () => {
const value = "GrEEtinGs from " + username + " " + creature + ": Hello #" + count + " (" + new Date().getTime() + ")"
// Set a key-value pair
count ++
db.put(key, value)
.then(() => counter.inc()) // Increase the counter by one
.then(() => log.add(value)) // Add an event to 'latest visitors' log
.then(() => getData())
.catch((e) => handleError(e))
}
const getData = () => {
const result = db.get(key)
const latest = log.iterator({ limit: 5 }).collect()
const count = counter.value
ipfs.pubsub.peers(dbname + ".log")
.then((peers) => {
const output = `
<b>You are:</b> ${username} ${creature}<br>
<b>Peers:</b> ${peers.length}<br>
<b>Database:</b> ${dbname}<br>
<br><b>Writing to database every ${interval} milliseconds...</b><br><br>
<b>Key-Value Store</b>
-------------------------------------------------------
Key | Value
-------------------------------------------------------
${key} | ${result}
-------------------------------------------------------
<b>Eventlog</b>
-------------------------------------------------------
Latest Updates
-------------------------------------------------------
${latest.reverse().map((e) => e.payload.value).join('\n')}
<b>Counter</b>
-------------------------------------------------------
Visitor Count: ${count}
-------------------------------------------------------
`
elm.innerHTML = output.split("\n").join("<br>")
})
}
db.events.on('synced', () => getData())
counter.events.on('synced', () => getData())
log.events.on('synced', () => getData())
db.events.on('ready', () => getData())
counter.events.on('ready', () => getData())
log.events.on('ready', () => getData())
// Start query loop when the databse has loaded its history
db.load(10)
.then(() => counter.load(10))
.then(() => log.load(10))
.then(() => {
count = counter.value
setInterval(query, interval)
})
})
}
openButton.addEventListener('click', openDatabase)

67
examples/eventlog.js
View File

@ -1,46 +1,51 @@
'use strict'
const IpfsDaemon = require('ipfs-daemon')
const IPFS = require('ipfs')
const OrbitDB = require('../src/OrbitDB')
const userId = Math.floor(Math.random() * 1000)
const conf = {
IpfsDataDir: '/tmp/' + userId,
Addresses: {
API: '/ip4/127.0.0.1/tcp/0',
Swarm: ['/ip4/0.0.0.0/tcp/0'],
Gateway: '/ip4/0.0.0.0/tcp/0'
},
}
console.log("Starting...")
const ipfs = new IpfsDaemon(conf)
ipfs.on('error', (err) => console.error(err))
ipfs.on('ready', () => {
const orbitdb = new OrbitDB(ipfs, userId)
const db = orbitdb.eventlog("|orbit-db|examples|eventlog-example")
const creatures = ['🐙', '🐷', '🐬', '🐞', '🐈', '🙉', '🐸', '🐓']
const query = () => {
console.log("Starting...")
const ipfs = new IPFS({
repo: './orbitdb/examples/ipfs',
start: true,
EXPERIMENTAL: {
pubsub: true,
},
})
ipfs.on('error', (err) => console.error(err))
ipfs.on('ready', async () => {
let db
try {
const orbitdb = new OrbitDB(ipfs, './orbitdb/examples/eventlog')
db = await orbitdb.eventlog('example', { overwrite: true })
} catch (e) {
console.error(e)
process.exit(1)
}
const query = async () => {
const index = Math.floor(Math.random() * creatures.length)
db.add({ avatar: creatures[index], userId: userId })
.then(() => {
const userId = Math.floor(Math.random() * 900 + 100)
try {
await db.add({ avatar: creatures[index], userId: userId })
const latest = db.iterator({ limit: 5 }).collect()
let output = ``
output += `[Latest Visitors]\n`
output += `--------------------\n`
output += `Latest Visitors\n`
output += `ID | Visitor\n`
output += `--------------------\n`
output += latest.reverse().map((e) => e.payload.value.avatar + " (userId: " + e.payload.value.userId + ")").join('\n') + `\n`
output += latest.reverse().map((e) => e.payload.value.userId + ' | ' + e.payload.value.avatar + ')').join('\n') + `\n`
console.log(output)
})
.catch((e) => {
console.error(e.stack)
})
} catch (e) {
console.error(e)
process.exit(1)
}
}
setInterval(query, 1000)

79
examples/keyvalue.js
View File

@ -1,50 +1,63 @@
'use strict'
const IpfsDaemon = require('ipfs-daemon')
const IPFS = require('ipfs')
const OrbitDB = require('../src/OrbitDB')
const userId = Math.floor(Math.random() * 1000)
const conf = {
IpfsDataDir: '/tmp/' + userId,
Addresses: {
API: '/ip4/127.0.0.1/tcp/0',
Swarm: ['/ip4/0.0.0.0/tcp/0'],
Gateway: '/ip4/0.0.0.0/tcp/0'
},
}
console.log("Starting...")
const ipfs = new IpfsDaemon(conf)
ipfs.on('error', (err) => console.error(err))
ipfs.on('ready', () => {
const orbitdb = new OrbitDB(ipfs, userId)
const db = orbitdb.kvstore("|orbit-db|examples|kvstore-example")
const userId = 1
const creatures = ['🐙', '🐬', '🐋', '🐠', '🐡', '🦀', '🐢', '🐟', '🐳']
const query = () => {
const index = Math.floor(Math.random() * creatures.length)
db.put(userId, { avatar: creatures[index], updated: new Date().getTime() })
.then(() => {
const user = db.get(userId)
let output = `\n`
const output = (user) => {
let output = ``
output += `----------------------\n`
output += `User\n`
output += `----------------------\n`
output += `Id: ${userId}\n`
output += `Avatar: ${user.avatar}\n`
output += `Updated: ${user.updated}\n`
output += `----------------------`
output += `----------------------\n`
console.log(output)
})
.catch((e) => {
console.error(e.stack)
})
}
console.log("Starting...")
const ipfs = new IPFS({
repo: './orbitdb/examples/ipfs',
start: true,
EXPERIMENTAL: {
pubsub: true,
},
})
ipfs.on('error', (err) => console.error(err))
ipfs.on('ready', async () => {
let db
try {
const orbitdb = new OrbitDB(ipfs, './orbitdb/examples/eventlog')
db = await orbitdb.kvstore('example', { overwrite: true })
await db.load()
// Query immediately after loading
const user = db.get(userId)
output(user)
} catch (e) {
console.error(e)
process.exit(1)
}
const query = async () => {
// Randomly select an avatar
const index = Math.floor(Math.random() * creatures.length)
// Set the key to the newly selected avatar and update the timestamp
await db.put(userId, { avatar: creatures[index], updated: new Date().getTime() })
// Get the value of the key
const user = db.get(userId)
// Display the value
output(user)
}
console.log("Starting update loop...")
setInterval(query, 1000)
})

10305
package-lock.json generated Normal file
View File

File diff suppressed because it is too large Load Diff

55
package.json
View File

@ -1,51 +1,56 @@
{
"name": "orbit-db",
"version": "0.17.3",
"version": "1.0.0",
"description": "Distributed p2p database on IPFS",
"author": "Haad",
"license": "MIT",
"repository": {
"type": "git",
"url": "https://github.com/haadcode/orbit-db"
"url": "https://github.com/orbitdb/orbit-db"
},
"engines": {
"node": "^6.x.x"
"node": ">=8.0.0"
},
"browser": {
"fs-pull-blob-store": "idb-pull-blob-store"
},
"main": "src/OrbitDB.js",
"dependencies": {
"libp2p-floodsub": "github:libp2p/js-libp2p-floodsub#orbit-floodsub",
"logplease": "^1.2.12",
"orbit-db-counterstore": "~0.2.0",
"orbit-db-docstore": "~0.2.0",
"orbit-db-eventstore": "~0.2.0",
"orbit-db-feedstore": "~0.2.0",
"orbit-db-kvstore": "~0.2.0",
"orbit-db-pubsub": "~0.2.1"
"logplease": "^1.2.14",
"multihashes": "^0.4.12",
"orbit-db-cache": "~0.0.7",
"orbit-db-counterstore": "~1.0.0",
"orbit-db-docstore": "~1.0.1",
"orbit-db-eventstore": "~1.0.0",
"orbit-db-feedstore": "~1.0.0",
"orbit-db-keystore": "~0.0.2",
"orbit-db-kvstore": "~1.0.0",
"orbit-db-pubsub": "~0.3.1"
},
"devDependencies": {
"babel-core": "^6.24.0",
"babel-loader": "^6.4.1",
"babel-plugin-transform-runtime": "^6.22.0",
"babel-polyfill": "^6.22.0",
"babel-preset-es2015": "^6.24.0",
"ipfs-daemon": "~0.3.1",
"json-loader": "^0.5.4",
"mocha": "^3.2.0",
"babel-core": "^6.26.0",
"babel-loader": "^7.1.2",
"babel-plugin-transform-runtime": "^6.23.0",
"babel-polyfill": "^6.26.0",
"babel-preset-es2015": "^6.24.1",
"datastore-level": "~0.7.0",
"ipfs": "~0.26.0",
"ipfs-repo": "~0.18.0",
"mocha": "^4.0.1",
"p-each-series": "^1.0.0",
"rimraf": "^2.6.1",
"uglify-js": "github:mishoo/UglifyJS2#harmony",
"webpack": "^2.3.1"
"p-map-series": "^1.0.0",
"rimraf": "^2.6.2",
"uglifyjs-webpack-plugin": "~1.1.0",
"webpack": "^3.8.1"
},
"scripts": {
"examples": "npm run examples:node",
"examples:node": "node examples/eventlog.js",
"examples:browser": "open examples/browser/index.html && LOG=debug node examples/start-daemon.js",
"examples:browser": "open examples/browser/browser.html",
"test": "mocha",
"build": "npm run build:examples && npm run build:dist",
"build": "npm run build:dist && npm run build:examples",
"build:examples": "webpack --config conf/webpack.example.config.js --sort-modules-by size",
"build:dist": "webpack -p --config conf/webpack.config.js --sort-modules-by size"
"build:dist": "webpack --config conf/webpack.config.js --sort-modules-by size && cp dist/orbitdb.min.js examples/browser/lib/orbitdb.min.js",
"build:debug": "webpack --config conf/webpack.debug.config.js --sort-modules-by size && cp dist/orbitdb.js examples/browser/lib/orbitdb.min.js"
}
}

BIN
screenshots/example1.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 185 KiB

4
scripts/post_install.sh
View File

@ -1,4 +0,0 @@
# !/bin/sh
rm -rf ./node_modules/ipfs/node_modules/ipfs-api
rm -rf ./node_modules/ipfsd-ctl/node_modules/go-ipfs-dep
rm -rf ./node_modules/ipfsd-ctl/node_modules/ipfs-api

362
src/OrbitDB.js
View File

@ -1,99 +1,361 @@
'use strict'
const EventEmitter = require('events').EventEmitter
const path = require('path')
const EventStore = require('orbit-db-eventstore')
const FeedStore = require('orbit-db-feedstore')
const KeyValueStore = require('orbit-db-kvstore')
const CounterStore = require('orbit-db-counterstore')
const DocumentStore = require('orbit-db-docstore')
const Pubsub = require('orbit-db-pubsub')
const Cache = require('orbit-db-cache')
const Keystore = require('orbit-db-keystore')
const AccessController = require('./ipfs-access-controller')
const OrbitDBAddress = require('./orbit-db-address')
const defaultNetworkName = 'Orbit DEV Network'
const Logger = require('logplease')
const logger = Logger.create("orbit-db")
Logger.setLogLevel('NONE')
const validTypes = ['eventlog', 'feed', 'docstore', 'counter', 'keyvalue']
class OrbitDB {
constructor(ipfs, id = 'default', options = {}) {
constructor(ipfs, directory, options = {}) {
this._ipfs = ipfs
this._pubsub = options && options.broker ? new options.broker(ipfs) : new Pubsub(ipfs)
this.user = { id: id }
this.network = { name: defaultNetworkName }
this.events = new EventEmitter()
this.id = options.peerId || (this._ipfs._peerInfo ? this._ipfs._peerInfo.id._idB58String : 'default')
this._pubsub = options && options.broker
? new options.broker(this._ipfs)
: new Pubsub(this._ipfs, this.id)
this.stores = {}
this.types = validTypes
this.directory = directory || './orbitdb'
this.keystore = new Keystore(path.join(this.directory, this.id, '/keystore'))
this.key = this.keystore.getKey(this.id) || this.keystore.createKey(this.id)
}
/* Databases */
feed(dbname, options) {
return this._createStore(FeedStore, dbname, options)
async feed (address, options = {}) {
options = Object.assign({ create: true, type: 'feed' }, options)
return this.open(address, options)
}
eventlog(dbname, options) {
return this._createStore(EventStore, dbname, options)
async log (address, options) {
options = Object.assign({ create: true, type: 'eventlog' }, options)
return this.open(address, options)
}
kvstore(dbname, options) {
return this._createStore(KeyValueStore, dbname, options)
async eventlog (address, options = {}) {
return this.log(address, options)
}
counter(dbname, options) {
return this._createStore(CounterStore, dbname, options)
async keyvalue (address, options) {
options = Object.assign({ create: true, type: 'keyvalue' }, options)
return this.open(address, options)
}
docstore(dbname, options) {
return this._createStore(DocumentStore, dbname, options)
async kvstore (address, options) {
return this.keyvalue(address, options)
}
close(dbname) {
if(this._pubsub) this._pubsub.unsubscribe(dbname)
if (this.stores[dbname]) {
this.stores[dbname].events.removeAllListeners('write')
delete this.stores[dbname]
}
async counter (address, options = {}) {
options = Object.assign({ create: true, type: 'counter' }, options)
return this.open(address, options)
}
disconnect() {
Object.keys(this.stores).forEach((e) => this.close(e))
if (this._pubsub) this._pubsub.disconnect()
async docs (address, options = {}) {
options = Object.assign({ create: true, type: 'docstore' }, options)
return this.open(address, options)
}
async docstore (address, options = {}) {
return this.docs(address, options)
}
async disconnect () {
// Close all open databases
const databases = Object.values(this.stores)
for (let db of databases) {
await db.close()
delete this.stores[db.address.toString()]
}
// Disconnect from pubsub
if (this._pubsub)
this._pubsub.disconnect()
// Remove all databases from the state
this.stores = {}
this.user = null
this.network = null
}
// Alias for disconnect()
async stop () {
await this.disconnect()
}
/* Private methods */
_createStore(Store, dbname, options) {
const opts = Object.assign({ replicate: true }, options)
async _createStore (Store, address, options) {
const addr = address.toString()
const store = new Store(this._ipfs, this.user.id, dbname, opts)
let accessController
if (options.accessControllerAddress) {
accessController = new AccessController(this._ipfs)
await accessController.load(options.accessControllerAddress)
}
const opts = Object.assign({ replicate: true }, options, {
accessController: accessController,
keystore: this.keystore,
cache: this._cache,
})
const store = new Store(this._ipfs, this.id, address, opts)
store.events.on('write', this._onWrite.bind(this))
store.events.on('ready', this._onReady.bind(this))
store.events.on('closed', this._onClosed.bind(this))
this.stores[dbname] = store
this.stores[addr] = store
if(opts.replicate && this._pubsub)
this._pubsub.subscribe(dbname, this._onMessage.bind(this))
this._pubsub.subscribe(addr, this._onMessage.bind(this), this._onPeerConnected.bind(this))
return store
}
/* Replication request from the message broker */
_onMessage(dbname, heads) {
// console.log(".MESSAGE", dbname, heads)
const store = this.stores[dbname]
store.sync(heads)
}
/* Data events */
_onWrite(dbname, hash, entry, heads) {
// 'New entry written to database...', after adding a new db entry locally
// console.log(".WRITE", dbname, hash, this.user.username)
// Callback for local writes to the database. We the update to pubsub.
_onWrite (address, entry, heads) {
if(!heads) throw new Error("'heads' not defined")
if(this._pubsub) setImmediate(() => this._pubsub.publish(dbname, heads))
if(this._pubsub) setImmediate(() => this._pubsub.publish(address, heads))
}
_onReady(dbname, heads) {
// if(heads && this._pubsub) setImmediate(() => this._pubsub.publish(dbname, heads))
if(heads && this._pubsub) {
setTimeout(() => this._pubsub.publish(dbname, heads), 1000)
// Callback for receiving a message from the network
async _onMessage (address, heads) {
const store = this.stores[address]
try {
logger.debug(`Received heads for '${address}':\n`, JSON.stringify(heads, null, 2))
await store.sync(heads)
} catch (e) {
logger.error(e)
}
}
// Callback for when a peer connected to a database
_onPeerConnected (address, peer, room) {
logger.debug(`New peer '${peer}' connected to '${address}'`)
const store = this.stores[address]
if (store) {
// Send the newly connected peer our latest heads
let heads = store._oplog.heads
if (heads.length > 0) {
logger.debug(`Send latest heads:\n`, JSON.stringify(heads, null, 2))
room.sendTo(peer, new Buffer(JSON.stringify(heads)))
}
store.events.emit('peer', peer)
}
}
// Callback when database was closed
_onClosed (address) {
logger.debug(`Database '${address}' was closed`)
// Remove the callback from the database
this.stores[address].events.removeAllListeners('closed')
// Unsubscribe from pubsub
if(this._pubsub)
this._pubsub.unsubscribe(address)
delete this.stores[address]
}
/* Create and Open databases */
/*
options = {
admin: [], // array of keys that are the admins of this database (same as write access)
write: [], // array of keys that can write to this database
directory: './orbitdb', // directory in which to place the database files
overwrite: false, // whether we should overwrite the existing database if it exists
}
*/
async create (name, type, options = {}) {
if (!OrbitDB.isValidType(type))
throw new Error(`Invalid database type '${type}'`)
// The directory to look databases from can be passed in as an option
const directory = options.directory || this.directory
logger.debug(`Creating database '${name}' as ${type} in '${directory}'`)
if (OrbitDBAddress.isValid(name))
throw new Error(`Given database name is an address. Please give only the name of the database!`)
// Create an AccessController
const accessController = new AccessController(this._ipfs)
/* Disabled temporarily until we do something with the admin keys */
// Add admins of the database to the access controller
// if (options && options.admin) {
// options.admin.forEach(e => accessController.add('admin', e))
// } else {
// // Default is to add ourselves as the admin of the database
// accessController.add('admin', this.key.getPublic('hex'))
// }
// Add keys that can write to the database
if (options && options.write) {
options.write.forEach(e => accessController.add('write', e))
} else {
// Default is to add ourselves as the admin of the database
accessController.add('write', this.key.getPublic('hex'))
}
// Save the Access Controller in IPFS
const accessControllerAddress = await accessController.save()
// Creates a DB manifest file
const createDBManifest = () => {
return {
name: name,
type: type,
accessController: path.join('/ipfs', accessControllerAddress),
}
}
// Save the manifest to IPFS
const manifest = createDBManifest()
const dag = await this._ipfs.object.put(new Buffer(JSON.stringify(manifest)))
const manifestHash = dag.toJSON().multihash.toString()
// Create the database address
const address = path.join('/orbitdb', manifestHash, name)
const dbAddress = OrbitDBAddress.parse(address)
// Load local cache
try {
const cacheFilePath = path.join(dbAddress.root, dbAddress.path)
this._cache = new Cache(path.join(directory), cacheFilePath)
await this._cache.load()
} catch (e) {
logger.warn("Couldn't load Cache:", e)
}
// Check if we already have the database
let localData = await this._cache.get(dbAddress.toString())
if (localData && localData.manifest && !options.overwrite)
throw new Error(`Database '${dbAddress}' already exists!`)
// Save the database locally
localData = Object.assign({}, this._cache.get(dbAddress.toString()), {
manifest: dbAddress.root
})
await this._cache.set(dbAddress.toString(), localData)
logger.debug(`Saved manifest to IPFS as '${dbAddress.root}'`)
logger.debug(`Created database '${dbAddress}'`)
// Open the database
return this.open(dbAddress, options)
}
/*
options = {
localOnly: false // if set to true, throws an error if database can't be found locally
create: false // wether to create the database
type: TODO
overwrite: TODO
}
*/
async open (address, options = {}) {
options = Object.assign({ localOnly: false, create: false }, options)
logger.debug(`Open database '${address}'`)
// The directory to look databases from can be passed in as an option
const directory = options.directory || this.directory
logger.debug(`Look from '${directory}'`)
// If address is just the name of database, check the options to crate the database
if (!OrbitDBAddress.isValid(address)) {
if (!options.create) {
throw new Error(`'options.create' set to 'false'. If you want to create a database, set 'options.create' to 'true'.`)
} else if (options.create && !options.type) {
throw new Error(`Database type not provided! Provide a type with 'options.type' (${validTypes.join('|')})`)
} else {
logger.warn(`Not a valid OrbitDB address '${address}', creating the database`)
options.overwrite = options.overwrite ? options.overwrite : true
return this.create(address, options.type, options)
}
}
// Parse the database address
const dbAddress = OrbitDBAddress.parse(address)
// Load local cache
try {
const cacheFilePath = path.join(dbAddress.root, dbAddress.path)
this._cache = new Cache(path.join(directory), cacheFilePath)
await this._cache.load()
} catch (e) {
console.warn(e)
logger.warn("Couldn't load Cache:", e)
}
// Check if we have the database
let localData = await this._cache.get(dbAddress.toString())
const haveDB = localData && localData.manifest
logger.debug((haveDB ? 'Found' : 'Didn\'t find') + ` database '${dbAddress}'`)
// If we want to try and open the database local-only, throw an error
// if we don't have the database locally
if (options.localOnly && !haveDB) {
logger.error(`Database '${dbAddress}' doesn't exist!`)
throw new Error(`Database '${dbAddress}' doesn't exist!`)
}
logger.debug(`Loading Manifest for '${dbAddress}'`)
// Get the database manifest from IPFS
const dag = await this._ipfs.object.get(dbAddress.root)
const manifest = JSON.parse(dag.toJSON().data)
logger.debug(`Manifest for '${dbAddress}':\n${JSON.stringify(manifest, null, 2)}`)
// Make sure the type from the manifest matches the type that was given as an option
if (options.type && manifest.type !== options.type)
throw new Error(`Database '${dbAddress}' is type '${manifest.type}' but was opened as '${options.type}'`)
// Save the database locally
localData = Object.assign({}, this._cache.get(dbAddress.toString()), {
manifest: dbAddress.root
})
await this._cache.set(dbAddress.toString(), localData)
logger.debug(`Saved manifest to IPFS as '${dbAddress.root}'`)
// Open the the database
options = Object.assign({}, options, { accessControllerAddress: manifest.accessController })
return this._openDatabase(dbAddress, manifest.type, options)
}
async _openDatabase (address, type, options) {
if (type === 'counter')
return this._createStore(CounterStore, address, options)
else if (type === 'eventlog')
return this._createStore(EventStore, address, options)
else if (type === 'feed')
return this._createStore(FeedStore, address, options)
else if (type === 'docstore')
return this._createStore(DocumentStore, address, options)
else if (type === 'keyvalue')
return this._createStore(KeyValueStore, address, options)
else
throw new Error(`Invalid database type '${type}'`)
}
static isValidType (type) {
return validTypes.includes(type)
}
static create () {
return new Error('Not implemented yet!')
}
static open () {
return new Error('Not implemented yet!')
}
}
module.exports = OrbitDB

85
src/access-controller.js Normal file
View File

@ -0,0 +1,85 @@
'use strict'
class AccessController {
constructor () {
this._access = {
admin: [],
write: [],
read: [], // Not used atm
}
}
/* Overridable functions */
async load (address) {}
async save () {}
/* Properties */
get admin () {
return this._access.admin
}
get write () {
// Both admins and write keys can write
return this._access.write.concat(this._access.admin)
}
// Not used atm
get read () {
return this._access.read
}
/* Public Methods */
add (access, key) {
// if(!Object.keys(this._access).includes(access))
// throw new Error(`unknown access level: ${access}`)
// if (!this._access[access].includes(key))
// this._access[access].push(key)
// TODO: uniques only
switch (access) {
case 'admin':
this._access.admin.push(key)
break
case 'write':
this._access.write.push(key)
break
case 'read':
this._access.read.push(key)
break
default:
break
}
}
remove (access, key) {
const without = (arr, e) => {
const reducer = (res, val) => {
if (val !== key)
res.push(val)
return res
}
return arr.reduce(reducer, [])
}
// if(!Object.keys(this._access).includes(access))
// throw new Error(`unknown access level: ${access}`)
// if (this._access[access].includes(key))
// this._access[access] = without(this._access[access], key)
switch (access) {
case 'admin':
this._access.admin = without(this._access.admin, key)
break
case 'write':
this._access.write = without(this._access.write, key)
break
case 'read':
this._access.read = without(this._access.read, key)
break
default:
break
}
}
}
module.exports = AccessController

39
src/ipfs-access-controller.js Normal file
View File

@ -0,0 +1,39 @@
'use strict'
const AccessController = require('./access-controller')
class IPFSAccessController extends AccessController {
constructor (ipfs) {
super()
this._ipfs = ipfs
}
async load (address) {
// Transform '/ipfs/QmPFtHi3cmfZerxtH9ySLdzpg1yFhocYDZgEZywdUXHxFU'
// to 'QmPFtHi3cmfZerxtH9ySLdzpg1yFhocYDZgEZywdUXHxFU'
if (address.indexOf('/ipfs') === 0)
address = address.split('/')[2]
try {
const dag = await this._ipfs.object.get(address)
const obj = JSON.parse(dag.toJSON().data)
this._access = obj
} catch (e) {
console.log("ACCESS ERROR:", e)
}
}
async save () {
let hash
try {
const access = JSON.stringify(this._access, null, 2)
const dag = await this._ipfs.object.put(new Buffer(access))
hash = dag.toJSON().multihash.toString()
} catch (e) {
console.log("ACCESS ERROR:", e)
}
return hash
}
}
module.exports = IPFSAccessController

48
src/orbit-db-address.js Normal file
View File

@ -0,0 +1,48 @@
'use strict'
const path = require('path')
const multihash = require('multihashes')
class OrbitDBAddress {
constructor (root, path) {
this.root = root
this.path = path
}
toString () {
return path.join('/orbitdb', this.root, this.path)
}
static isValid (address) {
const parts = address.toString()
.split('/')
.filter((e, i) => !((i === 0 || i === 1) && address.toString().indexOf('/orbit') === 0 && e === 'orbitdb'))
.filter(e => e !== '' && e !== ' ')
const accessControllerHash = parts[0].indexOf('Qm') > -1 ? multihash.fromB58String(parts[0]) : null
try {
multihash.validate(accessControllerHash)
} catch (e) {
return false
}
return accessControllerHash !== null
}
static parse (address) {
if (!address)
throw new Error(`Not a valid OrbitDB address: ${address}`)
if (!OrbitDBAddress.isValid(address))
throw new Error(`Not a valid OrbitDB address: ${address}`)
const parts = address.toString()
.split('/')
.filter((e, i) => !((i === 0 || i === 1) && address.toString().indexOf('/orbit') === 0 && e === 'orbitdb'))
.filter(e => e !== '' && e !== ' ')
return new OrbitDBAddress(parts[0], parts.slice(1, parts.length).join('/'))
}
}
module.exports = OrbitDBAddress

205
test/counterdb.test.js
View File

@ -1,127 +1,112 @@
// 'use strict'
'use strict'
// const path = require('path')
// const assert = require('assert')
// const Promise = require('bluebird')
// const rmrf = require('rimraf')
// const IpfsNodeDaemon = require('ipfs-daemon/src/ipfs-node-daemon')
// const IpfsNativeDaemon = require('ipfs-daemon/src/ipfs-native-daemon')
// const OrbitDB = require('../src/OrbitDB')
const path = require('path')
const assert = require('assert')
const rmrf = require('rimraf')
const mapSeries = require('p-each-series')
const OrbitDB = require('../src/OrbitDB')
const config = require('./utils/config')
const startIpfs = require('./utils/start-ipfs')
const waitForPeers = require('./utils/wait-for-peers')
// const username = 'testrunner'
// const username2 = 'rennurtset'
// const cacheFile = path.join(process.cwd(), '/tmp/orbit-db-tests/cache.json')
const dbPath1 = './orbitdb/tests/counters/peer1'
const dbPath2 = './orbitdb/tests/counters/peer2'
const ipfsPath1 = './orbitdb/tests/counters/peer1/ipfs'
const ipfsPath2 = './orbitdb/tests/counters/peer2/ipfs'
// const daemonConfs = require('./ipfs-daemons.conf.js')
describe('CounterStore', function() {
this.timeout(config.timeout)
// const waitForPeers = (ipfs, peersToWait, topic, callback) => {
// const i = setInterval(() => {
// ipfs.pubsub.peers(topic, (err, peers) => {
// if (err) {
// return callback(err)
// }
let orbitdb1, orbitdb2
let ipfs1, ipfs2
// const hasAllPeers = peersToWait.map((e) => peers.includes(e)).filter((e) => e === false).length === 0
// if (hasAllPeers) {
// clearInterval(i)
// callback(null)
// }
// })
// }, 1000)
// }
before(async () => {
rmrf.sync(dbPath1)
rmrf.sync(dbPath2)
config.daemon1.repo = ipfsPath1
config.daemon2.repo = ipfsPath2
ipfs1 = await startIpfs(config.daemon1)
ipfs2 = await startIpfs(config.daemon2)
})
// [IpfsNodeDaemon].forEach((IpfsDaemon) => {
// let ipfs, ipfsDaemon
after(async () => {
if (orbitdb1)
orbitdb1.stop()
// describe('CounterStore', function() {
// this.timeout(20000)
// let client1, client2
// let daemon1, daemon2
if (orbitdb2)
orbitdb2.stop()
// before((done) => {
// rmrf.sync(cacheFile)
// daemon1 = new IpfsDaemon(daemonConfs.daemon1)
// daemon1.on('ready', () => {
// daemon2 = new IpfsDaemon(daemonConfs.daemon2)
// daemon2.on('ready', () => {
// ipfs = [daemon1, daemon2]
// done()
// })
// })
// })
if (ipfs1)
await ipfs1.stop()
// after((done) => {
// daemon1.stop()
// daemon2.stop()
// rmrf.sync(cacheFile)
// done()
// })
if (ipfs2)
await ipfs2.stop()
})
// beforeEach(() => {
// client1 = new OrbitDB(ipfs[0])
// client2 = new OrbitDB(ipfs[1])
// })
beforeEach(() => {
orbitdb1 = new OrbitDB(ipfs1, './orbitdb/1')
orbitdb2 = new OrbitDB(ipfs2, './orbitdb/2')
})
// afterEach(() => {
// if (client1) client1.disconnect()
// if (client2) client2.disconnect()
// })
afterEach(() => {
if (orbitdb1)
orbitdb1.stop()
// describe('counters', function() {
// it('increases a counter value', function(done) {
// const timeout = setTimeout(() => done(new Error('event was not fired')), 2000)
// const counter = client1.counter('counter test', { subscribe: false, cacheFile: cacheFile })
// counter.events.on('ready', () => {
// Promise.map([13, 1], (f) => counter.inc(f), { concurrency: 1, cacheFile: cacheFile })
// .then(() => {
// assert.equal(counter.value, 14)
// clearTimeout(timeout)
// client1.disconnect()
// done()
// })
// .catch(done)
// })
// })
if (orbitdb2)
orbitdb2.stop()
})
// it.skip('creates a new counter from cached data', function(done) {
// const timeout = setTimeout(() => done(new Error('event was not fired')), 2000)
// const counter = client1.counter('counter test', { subscribe: false, cacheFile: cacheFile })
// counter.events.on('ready', () => {
// assert.equal(counter.value, 14)
// clearTimeout(timeout)
// client1.disconnect()
// done()
// })
// })
describe('counters', function() {
let address
// it.only('syncs counters', (done) => {
// const name = new Date().getTime()
// const counter1 = client1.counter(name)
// const counter2 = client2.counter(name)
// const numbers = [[13, 10], [2, 5]]
// // const res1 = ([13, 10]).map((f) => counter1.inc(f))//, { concurrency: 1 })
// // const res2 = ([2, 5]).map((f) => counter2.inc(f))//, { concurrency: 1 })
it('increases a counter value', async () => {
const counter = await orbitdb1.counter('counter test', { path: dbPath1 })
address = counter.address.toString()
await mapSeries([13, 1], (f) => counter.inc(f))
assert.equal(counter.value, 14)
await counter.close()
})
// waitForPeers(daemon1, [daemon2.PeerId], name, (err, res) => {
// waitForPeers(daemon2, [daemon1.PeerId], name, (err, res) => {
// console.log("load!!!")
// const increaseCounter = (counter, i) => numbers[i].map((e) => counter.inc(e))
// Promise.map([counter1, counter2], increaseCounter, { concurrency: 1 })
// .then((res) => {
// console.log("..", res)
// // wait for a while to make sure db's have been synced
// setTimeout(() => {
// assert.equal(counter2.value, 30)
// assert.equal(counter1.value, 30)
// done()
// }, 2000)
// })
// .catch(done)
// })
// })
// })
it('opens a saved counter', async () => {
const counter = await orbitdb1.counter(address, { path: dbPath1 })
await counter.load()
assert.equal(counter.value, 14)
await counter.close()
})
// })
// })
it('syncs counters', async () => {
let options = {
// Set write access for both clients
write: [
orbitdb1.key.getPublic('hex'),
orbitdb2.key.getPublic('hex')
],
}
// })
const numbers = [[13, 10], [2, 5]]
const increaseCounter = (counterDB, i) => mapSeries(numbers[i], n => counterDB.inc(n))
// Create a new counter database in the first client
options = Object.assign({}, options, { path: dbPath1 })
const counter1 = await orbitdb1.counter(new Date().getTime().toString(), options)
// Open the database in the second client
options = Object.assign({}, options, { path: dbPath2, sync: true })
const counter2 = await orbitdb2.counter(counter1.address.toString(), options)
// Wait for peers to connect first
await waitForPeers(ipfs1, [orbitdb2.id], counter1.address.toString())
await waitForPeers(ipfs2, [orbitdb1.id], counter1.address.toString())
// Increase the counters sequentially
await mapSeries([counter1, counter2], increaseCounter)
return new Promise(resolve => {
// Wait for a while to make sure db's have been synced
setTimeout(() => {
assert.equal(counter1.value, 30)
assert.equal(counter2.value, 30)
resolve()
}, 2000)
})
})
})
})

238
test/create-open.test.js Normal file
View File

@ -0,0 +1,238 @@
'use strict'
const assert = require('assert')
const fs = require('fs')
const path = require('path')
const rmrf = require('rimraf')
const mapSeries = require('p-map-series')
const OrbitDB = require('../src/OrbitDB')
const OrbitDBAddress = require('../src/orbit-db-address')
const { first, last } = require('./utils/test-utils')
const config = require('./utils/config')
const startIpfs = require('./utils/start-ipfs')
const stopIpfs = require('./utils/stop-ipfs')
const dbPath = './orbitdb/tests/create-open'
const ipfsPath = './orbitdb/tests/create-open/ipfs'
describe('orbit-db - Create & Open', function() {
this.timeout(config.timeout)
let ipfs, orbitdb, db, address
let localDataPath
before(async () => {
config.daemon1.repo = ipfsPath
rmrf.sync(config.daemon1.repo)
rmrf.sync(dbPath)
ipfs = await startIpfs(config.daemon1)
orbitdb = new OrbitDB(ipfs, dbPath)
})
after(async () => {
if(orbitdb)
orbitdb.stop()
if (ipfs)
await stopIpfs(ipfs)
})
describe('Create', function() {
describe('Errors', function() {
it('throws an error if given an invalid database type', async () => {
let err
try {
db = await orbitdb.create('first', 'invalid-type')
} catch (e) {
err = e.toString()
}
assert.equal(err, 'Error: Invalid database type \'invalid-type\'')
})
it('throws an error if given an address instead of name', async () => {
let err
try {
db = await orbitdb.create('/orbitdb/Qmc9PMho3LwTXSaUXJ8WjeBZyXesAwUofdkGeadFXsqMzW/first', 'feed')
} catch (e) {
err = e.toString()
}
assert.equal(err, 'Error: Given database name is an address. Please give only the name of the database!')
})
it('throws an error if database already exists', async () => {
let err
try {
db = await orbitdb.create('first', 'feed')
db = await orbitdb.create('first', 'feed')
} catch (e) {
err = e.toString()
}
assert.equal(err, `Error: Database '${db.address}' already exists!`)
})
it('throws an error if database type doesn\'t match', async () => {
let err, log, kv
try {
log = await orbitdb.kvstore('keyvalue')
kv = await orbitdb.eventlog(log.address.toString())
} catch (e) {
err = e.toString()
}
assert.equal(err, `Error: Database '${log.address}' is type 'keyvalue' but was opened as 'eventlog'`)
})
})
describe('Success', function() {
before(async () => {
db = await orbitdb.create('second', 'feed')
localDataPath = path.join(dbPath, db.address.root, db.address.path + '.orbitdb')
})
it('creates a feed database', async () => {
assert.notEqual(db, null)
})
it('database has the correct address', async () => {
assert.equal(db.address.toString().indexOf('/orbitdb'), 0)
assert.equal(db.address.toString().indexOf('Qm'), 9)
assert.equal(db.address.toString().indexOf('second'), 56)
})
it('saves the database locally', async () => {
assert.equal(fs.existsSync(localDataPath), true)
})
it('saves database manifest reference locally', async () => {
const buffer = JSON.parse(fs.readFileSync(localDataPath))
const data = buffer[db.address.toString()]
assert.equal(data.manifest, db.address.root)
assert.equal(db.address.path, 'second')
})
it('saves database manifest file locally', async () => {
const dag = await ipfs.object.get(db.address.root)
const manifest = JSON.parse(dag.toJSON().data)
assert.notEqual(manifest, )
assert.equal(manifest.name, 'second')
assert.equal(manifest.type, 'feed')
assert.notEqual(manifest.accessController, null)
assert.equal(manifest.accessController.indexOf('/ipfs'), 0)
})
it('can pass local database directory as an option', async () => {
const dir = './orbitdb/tests/another-feed'
db = await orbitdb.create('third', 'feed', { directory: dir })
localDataPath = path.join(dir, db.address.root, db.address.path + '.orbitdb')
assert.equal(fs.existsSync(localDataPath), true)
})
describe('Access Controller', function() {
before(async () => {
if (db) {
await db.close()
await db.drop()
}
})
afterEach(async () => {
if (db) {
await db.close()
await db.drop()
}
})
it('creates an access controller and adds ourselves as writer by default', async () => {
db = await orbitdb.create('fourth', 'feed')
assert.deepEqual(db.access.write, [orbitdb.key.getPublic('hex')])
})
it('creates an access controller and adds writers', async () => {
db = await orbitdb.create('fourth', 'feed', { write: ['another-key', 'yet-another-key', orbitdb.key.getPublic('hex')] })
assert.deepEqual(db.access.write, ['another-key', 'yet-another-key', orbitdb.key.getPublic('hex')])
})
it('creates an access controller and doesn\'t add an admin', async () => {
db = await orbitdb.create('sixth', 'feed')
assert.deepEqual(db.access.admin, [])
})
it('creates an access controller and doesn\'t add read access keys', async () => {
db = await orbitdb.create('seventh', 'feed', { read: ['one', 'two'] })
assert.deepEqual(db.access.read, [])
})
})
})
})
describe('Open', function() {
before(async () => {
db = await orbitdb.open('abc', { create: true, type: 'feed' })
})
it('throws an error if trying to open a database with name only and \'create\' is not set to \'true\'', async () => {
let err
try {
db = await orbitdb.open('XXX', { create: false })
} catch (e) {
err = e.toString()
}
assert.equal(err, "Error: 'options.create' set to 'false'. If you want to create a database, set 'options.create' to 'true'.")
})
it('throws an error if trying to open a database with name only and \'create\' is not set to true', async () => {
let err
try {
db = await orbitdb.open('YYY', { create: true })
} catch (e) {
err = e.toString()
}
assert.equal(err, "Error: Database type not provided! Provide a type with 'options.type' (eventlog|feed|docstore|counter|keyvalue)")
})
it('opens a database - name only', async () => {
db = await orbitdb.open('abc', { create: true, type: 'feed', overwrite: true })
assert.equal(db.address.toString().indexOf('/orbitdb'), 0)
assert.equal(db.address.toString().indexOf('Qm'), 9)
assert.equal(db.address.toString().indexOf('abc'), 56)
})
it('opens the same database - from an address', async () => {
db = await orbitdb.open(db.address)
assert.equal(db.address.toString().indexOf('/orbitdb'), 0)
assert.equal(db.address.toString().indexOf('Qm'), 9)
assert.equal(db.address.toString().indexOf('abc'), 56)
})
it('doesn\'t open a database if we don\'t have it locally', async () => {
const address = new OrbitDBAddress(db.address.root.slice(0, -1) + 'A', 'non-existent')
return new Promise((resolve, reject) => {
setTimeout(resolve, 1000)
orbitdb.open(address)
.then(() => reject(new Error('Shouldn\'t open the database')))
})
})
it('throws an error if trying to open a database locally and we don\'t have it', () => {
const address = new OrbitDBAddress(db.address.root.slice(0, -1) + 'A', 'second')
return orbitdb.open(address, { localOnly: true })
.then(() => new Error('Shouldn\'t open the database'))
.catch(e => {
assert.equal(e.toString(), `Error: Database '${address}' doesn't exist!`)
})
})
it('open the database and it has the added entries', async () => {
db = await orbitdb.open('ZZZ', { create: true, type: 'feed' })
await db.add('hello1')
await db.add('hello2')
db = await orbitdb.open(db.address)
await db.load()
const res = db.iterator({ limit: -1 }).collect()
assert.equal(res.length, 2)
assert.equal(res[0].payload.value, 'hello1')
assert.equal(res[1].payload.value, 'hello2')
})
})
})

179
test/docstore.test.js
View File

@ -2,155 +2,161 @@
const assert = require('assert')
const rmrf = require('rimraf')
const IpfsNodeDaemon = require('ipfs-daemon/src/ipfs-node-daemon')
const IpfsNativeDaemon = require('ipfs-daemon/src/ipfs-native-daemon')
const OrbitDB = require('../src/OrbitDB')
const hasIpfsApiWithPubsub = require('./test-utils').hasIpfsApiWithPubsub
const config = require('./test-config')
const config = require('./utils/config')
const startIpfs = require('./utils/start-ipfs')
config.daemons.forEach((IpfsDaemon) => {
const dbPath = './orbitdb/tests/docstore'
const ipfsPath = './orbitdb/tests/docstore/ipfs'
describe('orbit-db - Document Store', function() {
this.timeout(config.timeout)
let ipfs, client1, client2, db
let ipfs, orbitdb1, orbitdb2, db
before(function (done) {
rmrf.sync(config.defaultIpfsDirectory)
rmrf.sync(config.defaultOrbitDBDirectory)
ipfs = new IpfsDaemon()
ipfs.on('error', done)
ipfs.on('ready', () => {
assert.equal(hasIpfsApiWithPubsub(ipfs), true)
client1 = new OrbitDB(ipfs, 'A')
client2 = new OrbitDB(ipfs, 'B')
done()
})
before(async () => {
config.daemon1.repo = ipfsPath
rmrf.sync(config.daemon1.repo)
ipfs = await startIpfs(config.daemon1)
orbitdb1 = new OrbitDB(ipfs, dbPath + '/1')
orbitdb2 = new OrbitDB(ipfs, dbPath + '/2')
})
after(() => {
if(client1) client1.disconnect()
if(client2) client2.disconnect()
if(orbitdb1)
orbitdb1.disconnect()
if(orbitdb2)
orbitdb2.disconnect()
if (ipfs)
ipfs.stop()
rmrf.sync(config.defaultOrbitDBDirectory)
rmrf.sync(config.defaultIpfsDirectory)
})
it('creates and opens a database', async () => {
db = await orbitdb1.docstore('first doc database')
db = await orbitdb1.docstore('first doc database')
})
describe('Default index \'_id\'', function() {
beforeEach(() => {
db = client1.docstore(config.dbname, { replicate: false, maxHistory: 0 })
beforeEach(async () => {
const options = {
replicate: false,
maxHistory: 0,
path: dbPath,
}
db = await orbitdb1.docstore(config.dbname, options)
})
it('put', () => {
afterEach(async () => {
await db.drop()
})
it('put', async () => {
const doc = { _id: 'hello world', doc: 'all the things'}
return db.put(doc)
.then(() => {
await db.put(doc)
const value = db.get('hello world')
assert.deepEqual(value, [doc])
})
})
it('get - partial term match', () => {
it('get - partial term match', async () => {
const doc1 = { _id: 'hello world', doc: 'some things'}
const doc2 = { _id: 'hello universe', doc: 'all the things'}
const doc3 = { _id: 'sup world', doc: 'other things'}
return db.put(doc1)
.then(() => db.put(doc2))
.then(() => db.put(doc3))
.then(() => {
await db.put(doc1)
await db.put(doc2)
await db.put(doc3)
const value = db.get('hello')
assert.deepEqual(value, [doc1, doc2])
})
})
it('get after delete', () => {
it('get after delete', async () => {
const doc1 = { _id: 'hello world', doc: 'some things'}
const doc2 = { _id: 'hello universe', doc: 'all the things'}
const doc3 = { _id: 'sup world', doc: 'other things'}
return db.put(doc1)
.then(() => db.put(doc2))
.then(() => db.put(doc3))
.then(() => db.del('hello universe'))
.then(() => {
await db.put(doc1)
await db.put(doc2)
await db.put(doc3)
await db.del('hello universe')
const value1 = db.get('hello')
const value2 = db.get('sup')
assert.deepEqual(value1, [doc1])
assert.deepEqual(value2, [doc3])
})
})
it('put updates a value', () => {
it('put updates a value', async () => {
const doc1 = { _id: 'hello world', doc: 'all the things'}
const doc2 = { _id: 'hello world', doc: 'some of the things'}
return db.put(doc1)
.then(() => db.put(doc2))
.then(() => {
await db.put(doc1)
await db.put(doc2)
const value = db.get('hello')
assert.deepEqual(value, [doc2])
})
})
it('query', () => {
it('query', async () => {
const doc1 = { _id: 'hello world', doc: 'all the things', views: 17}
const doc2 = { _id: 'sup world', doc: 'some of the things', views: 10}
const doc3 = { _id: 'hello other world', doc: 'none of the things', views: 5}
const doc4 = { _id: 'hey universe', doc: ''}
return db.put(doc1)
.then(() => db.put(doc2))
.then(() => db.put(doc3))
.then(() => db.put(doc4))
.then(() => {
await db.put(doc1)
await db.put(doc2)
await db.put(doc3)
await db.put(doc4)
const value1 = db.query((e) => e.views > 5)
const value2 = db.query((e) => e.views > 10)
const value3 = db.query((e) => e.views > 17)
assert.deepEqual(value1, [doc1, doc2])
assert.deepEqual(value2, [doc1])
assert.deepEqual(value3, [])
})
})
it('query after delete', () => {
it('query after delete', async () => {
const doc1 = { _id: 'hello world', doc: 'all the things', views: 17}
const doc2 = { _id: 'sup world', doc: 'some of the things', views: 10}
const doc3 = { _id: 'hello other world', doc: 'none of the things', views: 5}
const doc4 = { _id: 'hey universe', doc: ''}
return db.put(doc1)
.then(() => db.put(doc2))
.then(() => db.put(doc3))
.then(() => db.del('hello world'))
.then(() => db.put(doc4))
.then(() => {
await db.put(doc1)
await db.put(doc2)
await db.put(doc3)
await db.del('hello world')
await db.put(doc4)
const value1 = db.query((e) => e.views >= 5)
const value2 = db.query((e) => e.views >= 10)
assert.deepEqual(value1, [doc2, doc3])
assert.deepEqual(value2, [doc2])
})
})
})
describe('Specified index', function() {
beforeEach(() => {
db = client1.docstore(config.dbname, { indexBy: 'doc', replicate: false, maxHistory: 0 })
beforeEach(async () => {
const options = {
indexBy: 'doc',
replicate: false,
maxHistory: 0
}
db = await orbitdb1.docstore(config.dbname, options)
})
it('put', () => {
afterEach(async () => {
await db.drop()
})
it('put', async () => {
const doc = { _id: 'hello world', doc: 'all the things'}
return db.put(doc)
.then(() => {
await db.put(doc)
const value = db.get('all')
assert.deepEqual(value, [doc])
})
})
it('get - matches specified index', () => {
it('get - matches specified index', async () => {
const doc1 = { _id: 'hello world', doc: 'all the things'}
const doc2 = { _id: 'hello world', doc: 'some things'}
return db.put(doc1)
.then(() => db.put(doc2))
.then(() => {
await db.put(doc1)
await db.put(doc2)
const value1 = db.get('all')
const value2 = db.get('some')
assert.deepEqual(value1, [doc1])
@ -158,34 +164,3 @@ config.daemons.forEach((IpfsDaemon) => {
})
})
})
describe('Sync', function() {
const doc1 = { _id: 'hello world', doc: 'all the things'}
const doc2 = { _id: 'moi moi', doc: 'everything'}
const options = {
replicate: false,
maxHistory: 0,
}
it('syncs databases', (done) => {
const db1 = client1.docstore(config.dbname, options)
const db2 = client2.docstore(config.dbname, options)
db2.events.on('write', (dbname, hash, entry, heads) => {
assert.deepEqual(db1.get('hello world'), [])
db1.sync(heads)
})
db1.events.on('synced', () => {
const value = db1.get(doc1._id)
assert.deepEqual(value, [doc1])
done()
})
db2.put(doc1)
.catch(done)
})
})
})
})

125
test/eventlog.test.js
View File

@ -2,99 +2,96 @@
const assert = require('assert')
const rmrf = require('rimraf')
const mapSeries = require('./promise-map-series')
const mapSeries = require('p-map-series')
const OrbitDB = require('../src/OrbitDB')
const hasIpfsApiWithPubsub = require('./test-utils').hasIpfsApiWithPubsub
const first = require('./test-utils').first
const last = require('./test-utils').last
const config = require('./test-config')
const first = require('./utils/test-utils').first
const last = require('./utils/test-utils').last
const config = require('./utils/config')
const startIpfs = require('./utils/start-ipfs')
config.daemons.forEach((IpfsDaemon) => {
const dbPath = './orbitdb/tests/eventlog'
const ipfsPath = './orbitdb/tests/eventlog/ipfs'
describe('orbit-db - Eventlog', function() {
this.timeout(config.timeout)
let ipfs, client1, client2, db
let ipfs, orbitdb1, orbitdb2, db
before(function (done) {
rmrf.sync(config.defaultIpfsDirectory)
rmrf.sync(config.defaultOrbitDBDirectory)
ipfs = new IpfsDaemon()
ipfs.on('error', done)
ipfs.on('ready', () => {
assert.equal(hasIpfsApiWithPubsub(ipfs), true)
client1 = new OrbitDB(ipfs, 'A')
client2 = new OrbitDB(ipfs, 'B')
done()
})
before(async () => {
config.daemon1.repo = ipfsPath
rmrf.sync(config.daemon1.repo)
rmrf.sync(dbPath)
ipfs = await startIpfs(config.daemon1)
orbitdb1 = new OrbitDB(ipfs, dbPath + '/1')
orbitdb2 = new OrbitDB(ipfs, dbPath + '/2')
})
after(() => {
if(client1) client1.disconnect()
if(client2) client2.disconnect()
ipfs.stop()
rmrf.sync(config.defaultOrbitDBDirectory)
rmrf.sync(config.defaultIpfsDirectory)
after(async () => {
if(orbitdb1)
orbitdb1.stop()
if(orbitdb2)
orbitdb2.stop()
if (ipfs)
await ipfs.stop()
})
describe('Eventlog', function () {
it('returns the added entry\'s hash, 1 entry', () => {
db = client1.eventlog(config.dbname, { replicate: false, maxHistory: 0 })
return db.add('hello1')
.then((hash) => {
it('creates and opens a database', async () => {
db = await orbitdb1.eventlog('first database')
db = await orbitdb1.eventlog('first database')
const items = db.iterator({ limit: -1 }).collect()
assert.equal(items.length, 0)
})
it('returns the added entry\'s hash, 1 entry', async () => {
db = await orbitdb1.eventlog('first database')
const hash = await db.add('hello1')
const items = db.iterator({ limit: -1 }).collect()
assert.notEqual(hash, null)
assert.equal(hash, last(items).hash)
assert.equal(items.length, 1)
})
})
it('returns the added entry\'s hash, 2 entries', () => {
it('returns the added entry\'s hash, 2 entries', async () => {
const prevHash = db.iterator().collect()[0].hash
return db.add('hello2')
.then((hash) => {
const hash = await db.add('hello2')
const items = db.iterator({ limit: -1 }).collect()
assert.equal(items.length, 2)
assert.notEqual(hash, null)
assert.notEqual(hash, prevHash)
assert.equal(hash, last(items).hash)
})
})
it('adds five items', () => {
db = client1.eventlog(config.dbname, { replicate: false, maxHistory: 0 })
return mapSeries([1, 2, 3, 4, 5], (i) => db.add('hello' + i))
.then(() => {
it('adds five items', async () => {
db = await orbitdb1.eventlog('second database')
await mapSeries([1, 2, 3, 4, 5], (i) => db.add('hello' + i))
const items = db.iterator({ limit: -1 }).collect()
assert.equal(items.length, 5)
assert.equal(first(items.map((f) => f.payload.value)), 'hello1')
assert.equal(last(items.map((f) => f.payload.value)), 'hello5')
})
})
it('adds an item that is > 256 bytes', () => {
db = client1.eventlog(config.dbname, { replicate: false, maxHistory: 0 })
it('adds an item that is > 256 bytes', async () => {
db = await orbitdb1.eventlog('third database')
let msg = new Buffer(1024)
msg.fill('a')
return db.add(msg.toString())
.then((hash) => {
const hash = await db.add(msg.toString())
assert.notEqual(hash, null)
assert.equal(hash.startsWith('Qm'), true)
assert.equal(hash.length, 46)
})
})
})
describe('Iterator', function() {
let items = []
const itemCount = 5
beforeEach(() => {
before(async () => {
items = []
db = client1.eventlog(config.dbname, { replicate: false, maxHistory: 0 })
return mapSeries([0, 1, 2, 3, 4], (i) => db.add('hello' + i))
.then((res) => items = res)
db = await orbitdb1.eventlog('iterator tests')
items = await mapSeries([0, 1, 2, 3, 4], (i) => db.add('hello' + i))
})
describe('Defaults', function() {
@ -352,36 +349,4 @@ config.daemons.forEach((IpfsDaemon) => {
})
})
})
describe('sync', () => {
const options = {
replicate: false,
}
it('syncs databases', (done) => {
const db1 = client1.eventlog(config.dbname, options)
const db2 = client2.eventlog(config.dbname, options)
db1.events.on('error', (e) => {
console.log(e.stack())
done(e)
})
db2.events.on('write', (dbname, hash, entry, heads) => {
assert.equal(db1.iterator({ limit: -1 }).collect().length, 0)
db1.sync(heads)
})
db1.events.on('synced', () => {
const items = db1.iterator({ limit: -1 }).collect()
assert.equal(items.length, 1)
assert.equal(items[0].payload.value, 'hello2')
done()
})
db2.add('hello2')
.catch(done)
})
})
})
})

162
test/feed.test.js
View File

@ -2,124 +2,118 @@
const assert = require('assert')
const rmrf = require('rimraf')
const mapSeries = require('./promise-map-series')
const mapSeries = require('p-map-series')
const OrbitDB = require('../src/OrbitDB')
const hasIpfsApiWithPubsub = require('./test-utils').hasIpfsApiWithPubsub
const first = require('./test-utils').first
const last = require('./test-utils').last
const config = require('./test-config')
const first = require('./utils/test-utils').first
const last = require('./utils/test-utils').last
const config = require('./utils/config')
const startIpfs = require('./utils/start-ipfs')
config.daemons.forEach((IpfsDaemon) => {
const dbPath = './orbitdb/tests/feed'
const ipfsPath = './orbitdb/tests/feed/ipfs'
describe('orbit-db - Feed', function() {
this.timeout(config.timeout)
let ipfs, client1, client2, db
let ipfs, orbitdb1, orbitdb2, db, address
before(function (done) {
rmrf.sync(config.defaultIpfsDirectory)
rmrf.sync(config.defaultOrbitDBDirectory)
ipfs = new IpfsDaemon()
ipfs.on('error', done)
ipfs.on('ready', () => {
assert.equal(hasIpfsApiWithPubsub(ipfs), true)
client1 = new OrbitDB(ipfs, 'A')
client2 = new OrbitDB(ipfs, 'B')
done()
})
before(async () => {
config.daemon1.repo = ipfsPath
rmrf.sync(config.daemon1.repo)
rmrf.sync(dbPath)
ipfs = await startIpfs(config.daemon1)
orbitdb1 = new OrbitDB(ipfs, dbPath + '/1')
orbitdb2 = new OrbitDB(ipfs, dbPath + '/2')
})
after(() => {
if(client1) client1.disconnect()
if(client2) client2.disconnect()
ipfs.stop()
rmrf.sync(config.defaultOrbitDBDirectory)
rmrf.sync(config.defaultIpfsDirectory)
after(async () => {
if(orbitdb1)
orbitdb1.stop()
if(orbitdb2)
orbitdb2.stop()
if (ipfs)
await ipfs.stop()
})
describe('Feed', function() {
it('returns the added entry\'s hash, 1 entry', () => {
db = client1.feed(config.dbname, { replicate: false, maxHistory: 0 })
return db.add('hello1')
.then((hash) => {
it('creates and opens a database', async () => {
db = await orbitdb1.feed('first database')
db = await orbitdb1.feed('first database')
const items = db.iterator({ limit: -1 }).collect()
assert.equal(items.length, 0)
})
it('returns the added entry\'s hash, 1 entry', async () => {
db = await orbitdb1.feed('first')
address = db.address.toString()
const hash = await db.add('hello1')
const items = db.iterator({ limit: -1 }).collect()
assert.notEqual(hash, null)
assert.equal(hash, last(items).hash)
assert.equal(items.length, 1)
})
})
it('returns the added entry\'s hash, 2 entries', () => {
it('returns the added entry\'s hash, 2 entries', async () => {
db = await orbitdb1.feed(address)
await db.load()
const prevHash = db.iterator().collect()[0].hash
return db.add('hello2')
.then((hash) => {
const hash = await db.add('hello2')
const items = db.iterator({ limit: -1 }).collect()
assert.equal(items.length, 2)
assert.notEqual(hash, null)
assert.notEqual(hash, prevHash)
assert.equal(hash, last(items).hash)
})
})
it('adds five items', () => {
db = client1.feed(config.dbname, { replicate: false, maxHistory: 0 })
return mapSeries([1, 2, 3, 4, 5], (i) => db.add('hello' + i))
.then(() => {
it('adds five items', async () => {
db = await orbitdb1.feed('second')
await mapSeries([1, 2, 3, 4, 5], (i) => db.add('hello' + i))
const items = db.iterator({ limit: -1 }).collect()
assert.equal(items.length, 5)
assert.equal(first(items.map((f) => f.payload.value)), 'hello1')
assert.equal(last(items.map((f) => f.payload.value)), 'hello5')
})
})
it('adds an item that is > 256 bytes', () => {
db = client1.feed(config.dbname, { replicate: false, maxHistory: 0 })
it('adds an item that is > 256 bytes', async () => {
db = await orbitdb1.feed('third')
let msg = new Buffer(1024)
msg.fill('a')
return db.add(msg.toString())
.then((hash) => {
const hash = await db.add(msg.toString())
assert.notEqual(hash, null)
assert.equal(hash.startsWith('Qm'), true)
assert.equal(hash.length, 46)
})
})
it('deletes an item when only one item in the database', () => {
db = client1.feed(config.dbname, { replicate: false, maxHistory: 0 })
return db.add('hello3')
.then((hash) => db.remove(hash))
.then((delopHash) => {
it('deletes an item when only one item in the database', async () => {
db = await orbitdb1.feed('fourth')
const hash = await db.add('hello3')
const delopHash = await db.remove(hash)
const items = db.iterator().collect()
assert.equal(delopHash.startsWith('Qm'), true)
assert.equal(items.length, 0)
})
})
it('deletes an item when two items in the database', () => {
db = client1.feed(config.dbname, { replicate: false, maxHistory: 0 })
it('deletes an item when two items in the database', async () => {
db = await orbitdb1.feed('fifth')
return db.add('hello1')
.then(() => db.add('hello2'))
.then((hash) => db.remove(hash))
.then(() => {
await db.add('hello1')
const hash = await db.add('hello2')
await db.remove(hash)
const items = db.iterator({ limit: -1 }).collect()
assert.equal(items.length, 1)
assert.equal(first(items).payload.value, 'hello1')
})
})
it('deletes an item between adds', () => {
db = client1.feed(config.dbname, { replicate: false, maxHistory: 0 })
it('deletes an item between adds', async () => {
db = await orbitdb1.feed('sixth')
const hash = await db.add('hello1')
await db.add('hello2')
await db.remove(hash)
await db.add('hello3')
let hash
return db.add('hello1')
.then((res) => hash = res)
.then(() => db.add('hello2'))
.then(() => db.remove(hash))
.then(() => db.add('hello3'))
.then(() => {
const items = db.iterator({ limit: -1 }).collect()
assert.equal(items.length, 2)
@ -131,17 +125,15 @@ config.daemons.forEach((IpfsDaemon) => {
assert.equal(secondItem.payload.value, 'hello3')
})
})
})
describe('Iterator', function() {
let items = []
const itemCount = 5
beforeEach(() => {
before(async () => {
items = []
db = client1.feed(config.dbname, { replicate: false, maxHistory: 0 })
return mapSeries([0, 1, 2, 3, 4], (i) => db.add('hello' + i))
.then((res) => items = res)
db = await orbitdb1.feed('feed-iterator')
items = await mapSeries([0, 1, 2, 3, 4], (i) => db.add('hello' + i))
})
describe('Defaults', function() {
@ -399,30 +391,4 @@ config.daemons.forEach((IpfsDaemon) => {
})
})
})
describe('sync', () => {
const options = {
replicate: false,
}
it('syncs databases', (done) => {
const db1 = client1.feed(config.dbname, options)
const db2 = client2.feed(config.dbname, options)
db2.events.on('write', (dbname, hash, entry, heads) => {
assert.equal(db1.iterator({ limit: -1 }).collect().length, 0)
db1.sync(heads)
})
db1.events.on('synced', () => {
const items = db1.iterator({ limit: -1 }).collect()
assert.equal(items.length, 1)
assert.equal(items[0].payload.value, 'hello2')
done()
})
db2.add('hello2')
.catch(done)
})
})
})
})

38
test/ipfs-daemons.conf.js
View File

@ -1,38 +0,0 @@
module.exports = {
daemon1: {
IpfsDataDir: '/tmp/orbit-db-tests-1',
Addresses: {
API: '/ip4/127.0.0.1/tcp/0',
Swarm: ['/ip4/0.0.0.0/tcp/0'],
Gateway: '/ip4/0.0.0.0/tcp/0'
},
Bootstrap: [],
Discovery: {
MDNS: {
Enabled: true,
Interval: 10
},
webRTCStar: {
Enabled: false
}
}
},
daemon2: {
IpfsDataDir: '/tmp/orbit-db-tests-2',
Addresses: {
API: '/ip4/127.0.0.1/tcp/0',
Swarm: ['/ip4/0.0.0.0/tcp/0'],
Gateway: '/ip4/0.0.0.0/tcp/0'
},
Bootstrap: [],
Discovery: {
MDNS: {
Enabled: true,
Interval: 10
},
webRTCStar: {
Enabled: false
}
}
}
}

158
test/kvstore.test.js
View File

@ -3,79 +3,79 @@
const assert = require('assert')
const rmrf = require('rimraf')
const OrbitDB = require('../src/OrbitDB')
const hasIpfsApiWithPubsub = require('./test-utils').hasIpfsApiWithPubsub
const config = require('./test-config')
const config = require('./utils/config')
const startIpfs = require('./utils/start-ipfs')
config.daemons.forEach((IpfsDaemon) => {
const dbPath = './orbitdb/tests/kvstore'
const ipfsPath = './orbitdb/tests/kvstore/ipfs'
describe('orbit-db - Key-Value Store', function() {
this.timeout(config.timeout)
let ipfs, client1, client2, db
let ipfs, orbitdb1, orbitdb2, db
before(function (done) {
rmrf.sync(config.defaultIpfsDirectory)
rmrf.sync(config.defaultOrbitDBDirectory)
ipfs = new IpfsDaemon()
ipfs.on('error', done)
ipfs.on('ready', () => {
assert.equal(hasIpfsApiWithPubsub(ipfs), true)
client1 = new OrbitDB(ipfs, 'A')
client2 = new OrbitDB(ipfs, 'B')
done()
})
before(async () => {
config.daemon1.repo = ipfsPath
rmrf.sync(config.daemon1.repo)
rmrf.sync(dbPath)
ipfs = await startIpfs(config.daemon1)
orbitdb1 = new OrbitDB(ipfs, dbPath + '/1')
orbitdb2 = new OrbitDB(ipfs, dbPath + '/2')
})
after(() => {
if(client1) client1.disconnect()
if(client2) client2.disconnect()
ipfs.stop()
rmrf.sync(config.defaultOrbitDBDirectory)
rmrf.sync(config.defaultIpfsDirectory)
after(async () => {
if(orbitdb1)
orbitdb1.stop()
if(orbitdb2)
orbitdb2.stop()
if (ipfs)
await ipfs.stop()
})
beforeEach(() => {
db = client1.kvstore(config.dbname, { replicate: false, maxHistory: 0 })
beforeEach(async () => {
db = await orbitdb1.kvstore(config.dbname, { path: dbPath })
})
it('put', () => {
return db.put('key1', 'hello1')
.then(() => {
afterEach(async () => {
await db.drop()
})
it('creates and opens a database', async () => {
db = await orbitdb1.keyvalue('first kv database')
db = await orbitdb1.keyvalue('first kv database')
})
it('put', async () => {
await db.put('key1', 'hello1')
const value = db.get('key1')
assert.equal(value, 'hello1')
})
})
it('get', () => {
return db.put('key1', 'hello2')
.then(() => {
it('get', async () => {
await db.put('key1', 'hello2')
const value = db.get('key1')
assert.equal(value, 'hello2')
})
})
it('put updates a value', () => {
return db.put('key1', 'hello3')
.then(() => db.put('key1', 'hello4'))
.then(() => {
it('put updates a value', async () => {
await db.put('key1', 'hello3')
await db.put('key1', 'hello4')
const value = db.get('key1')
assert.equal(value, 'hello4')
})
})
it('set is an alias for put', () => {
return db.set('key1', 'hello5')
.then(() => {
it('set is an alias for put', async () => {
await db.set('key1', 'hello5')
const value = db.get('key1')
assert.equal(value, 'hello5')
})
})
it('put/get - multiple keys', () => {
return db.put('key1', 'hello1')
.then(() => db.put('key2', 'hello2'))
.then(() => db.put('key3', 'hello3'))
.then(() => {
it('put/get - multiple keys', async () => {
await db.put('key1', 'hello1')
await db.put('key2', 'hello2')
await db.put('key3', 'hello3')
const v1 = db.get('key1')
const v2 = db.get('key2')
const v3 = db.get('key3')
@ -83,81 +83,41 @@ config.daemons.forEach((IpfsDaemon) => {
assert.equal(v2, 'hello2')
assert.equal(v3, 'hello3')
})
})
it('deletes a key', () => {
return db.put('key1', 'hello!')
.then(() => db.del('key1'))
.then(() => {
it('deletes a key', async () => {
await db.put('key1', 'hello!')
await db.del('key1')
const value = db.get('key1')
assert.equal(value, null)
})
})
it('deletes a key after multiple updates', () => {
return db.put('key1', 'hello1')
.then(() => db.put('key1', 'hello2'))
.then(() => db.put('key1', 'hello3'))
.then(() => db.del('key1'))
.then(() => {
it('deletes a key after multiple updates', async () => {
await db.put('key1', 'hello1')
await db.put('key1', 'hello2')
await db.put('key1', 'hello3')
await db.del('key1')
const value = db.get('key1')
assert.equal(value, null)
})
})
it('get - integer value', () => {
it('get - integer value', async () => {
const val = 123
return db.put('key1', val)
.then(() => {
await db.put('key1', val)
const v1 = db.get('key1')
assert.equal(v1, val)
})
})
it('get - object value', () => {
it('get - object value', async () => {
const val = { one: 'first', two: 2 }
return db.put('key1', val)
.then(() => {
await db.put('key1', val)
const v1 = db.get('key1')
assert.deepEqual(v1, val)
})
})
it('get - array value', () => {
it('get - array value', async () => {
const val = [1, 2, 3, 4, 5]
return db.put('key1', val)
.then(() => {
await db.put('key1', val)
const v1 = db.get('key1')
assert.deepEqual(v1, val)
})
})
describe('sync', () => {
const options = {
replicate: false,
}
it('syncs databases', (done) => {
const db1 = client1.kvstore(config.dbname, options)
const db2 = client2.kvstore(config.dbname, options)
db1.events.on('error', done)
db2.events.on('write', (dbname, hash, entry, heads) => {
assert.equal(db1.get('key1'), null)
assert.equal(db2.get('key1'), 'hello1')
db1.sync(heads)
})
db1.events.on('synced', () => {
const value = db1.get('key1')
assert.equal(value, 'hello1')
done()
})
db2.put('key1', 'hello1')
.catch(done)
})
})
})
})

1
test/mocha.opts
View File

@ -1,3 +1,4 @@
--reporter spec
--colors
--recursive
--exit

265
test/network-stress.tests.js Normal file
View File

@ -0,0 +1,265 @@
'use strict'
const fs = require('fs')
const rmrf = require('rimraf')
const path = require('path')
const assert = require('assert')
const pMap = require('p-map')
const pEachSeries = require('p-each-series')
const pWhilst = require('p-whilst')
const OrbitDB = require('../src/OrbitDB')
const startIpfs = require('./utils/start-ipfs')
// Settings for the test ipfs daemons
const config = require('./utils/config.js')
describe.skip('OrbitDB - Network Stress Tests', function() {
// We need a huge timeout since we're running
// very long-running tests (takes minutes)
this.timeout(1000 * 60 * 60) // 1 hour
const tests = [
{
description: '1 update - 2 peers - as fast as possible',
updates: 1,
maxInterval: -1,
minInterval: 0,
sequential: false,
content: 'Hello #',
clients: [
{ name: 'daemon1' },
{ name: 'daemon2' },
// { name: 'daemon3' },
// { name: 'daemon4' },
// { name: 'daemon5' },
// { name: 'daemon6' },
// Don't go beyond 6...
// { name: 'daemon7' },
// { name: 'daemon8' },
],
},
{
description: '32 update - concurrent - 2 peers - random interval',
updates: 32,
maxInterval: 2000,
minInterval: 10,
sequential: false,
content: 'Hello random! ',
clients: [
{ name: 'daemon1' },
{ name: 'daemon2' },
],
},
{
description: '1000 update concurrently - 2 peers - as fast as possible',
updates: 1000,
maxInterval: -1,
minInterval: 0,
sequential: false,
content: 'Hello #',
clients: [
{ name: 'daemon1' },
{ name: 'daemon2' },
],
},
{
description: '200 update as Buffers sequentially - 2 peers - as fast as possible',
updates: 200,
maxInterval: -1,
minInterval: 0,
sequential: true,
content: Buffer.from('👻'),
clients: [
{ name: 'daemon1' },
{ name: 'daemon2' },
],
},
{
description: '50 update over a period long time - 6 peers - slow, random write intervals',
updates: 50,
maxInterval: 3000,
minInterval: 1000,
sequential: false,
content: 'Terve! ',
clients: [
{ name: 'daemon1' },
{ name: 'daemon2' },
{ name: 'daemon3' },
{ name: 'daemon4' },
{ name: 'daemon5' },
{ name: 'daemon6' },
],
},
{
description: '50 update over a period long time - 8 peers - slow, random write intervals',
updates: 100,
maxInterval: 3000,
minInterval: 1000,
sequential: false,
content: 'Terve! ',
clients: [
{ name: 'daemon1' },
{ name: 'daemon2' },
{ name: 'daemon3' },
{ name: 'daemon4' },
{ name: 'daemon5' },
{ name: 'daemon6' },
{ name: 'daemon7' },
{ name: 'daemon8' },
],
},
]
const rootPath = './orbitdb/network-tests/'
const channelName = 'orbitdb-network-stress-tests'
tests.forEach(test => {
it(test.description, (done) => {
const updateCount = test.updates
const maxInterval = test.maxInterval || -1
const minInterval = test.minInterval || 0
const sequential = test.sequential
const clientData = test.clients
rmrf.sync(rootPath)
// Create IPFS instances
const createIpfsInstance = (c) => {
const repoPath = path.join(rootPath, c.name, '/ipfs' + new Date().getTime())
console.log("Starting IPFS instance <<>>", repoPath)
return startIpfs(Object.assign({}, config.defaultIpfsConfig, {
repo: repoPath,
start: true,
}))
}
const createOrbitDB = async (databaseConfig, ipfs) => {
const orbitdb = new OrbitDB(ipfs, path.join('./orbitdb/network-tests/', databaseConfig.name))
const db = await orbitdb.eventlog(databaseConfig.address, {
write: ['*']
})
return db
}
let allTasks = []
const setupAllTasks = (databases) => {
// Create the payloads
let texts = []
for (let i = 1; i < updateCount + 1; i ++) {
texts.push(test.content + i)
}
const setupUpdates = (client) => texts.reduce((res, acc) => {
return res.concat([{ db: client, content: acc }])
}, [])
allTasks = databases.map(db => {
return {
name: db.id,
tasks: setupUpdates(db),
}
})
}
const runAllTasks = () => {
if (sequential) {
return pEachSeries(allTasks, e => pEachSeries(e.tasks, writeToDB))
.then(() => console.log())
} else {
return pMap(allTasks, e => pEachSeries(e.tasks, writeToDB))
.then(() => console.log())
}
}
let i = 0
const writeToDB = (task) => {
return new Promise((resolve, reject) => {
if (maxInterval === -1) {
task.db.add(task.content)
.then(() => process.stdout.write(`\rUpdates (${databases.length} peers): ${Math.floor(++i)} / ${updateCount}`))
.then(resolve)
.catch(reject)
} else {
setTimeout(() => {
task.db.add(task.content)
.then(() => process.stdout.write(`\rUpdates (${databases.length} peers): ${Math.floor(++i)} / ${updateCount}`))
.then(resolve)
.catch(reject)
}, Math.floor(Math.random() * maxInterval) + minInterval)
}
})
}
const waitForAllTasks = (address) => {
let msgCount = 0
return pWhilst(
() => msgCount < databases.length * databases.length * updateCount,
() => new Promise(resolve => {
return queryDatabases(address)
.then(res => {
msgCount = res.reduce((val, acc) => val += acc.length, 0)
})
.then(() => process.stdout.write(`\rUpdated (${databases.length} peers): ` + msgCount.toString() + ' / ' + (updateCount * databases.length * databases.length)))
.then(() => setTimeout(resolve, 100))
})
)
.then(() => process.stdout.write(`\rUpdated (${databases.length} peers): ` + msgCount.toString() + ' / ' + (updateCount * databases.length * databases.length) + '\n'))
}
const queryDatabases = () => {
return pMap(databases, db => db.iterator({ limit: -1 }).collect(), { concurrency: 2 })
}
// All our databases instances
let databases = []
let addr
// Start the test
pMap(clientData, (c, idx) => {
return createIpfsInstance(c)
.then(async (ipfs) => {
let db
if (idx === 0 && !addr) {
c.address = channelName
db = await createOrbitDB(c, ipfs)
addr = db.address.toString()
} else if (addr) {
c.address = addr
db = await createOrbitDB(c, ipfs)
} else {
console.error("Address not defined!")
}
return db
})
}, { concurrency: 1 })
.then((result) => databases = result)
.then(() => setupAllTasks(databases))
.then(() => console.log(`Applying ${updateCount} updates per peer. This will take a while...`))
.then(() => runAllTasks())
.then(() => console.log('Done. Waiting for all updates to reach the peers...'))
.then(() => waitForAllTasks(addr))
.then(() => queryDatabases())
.then((result) => {
// Both databases have the same amount of entries
result.forEach(entries => {
assert.equal(entries.length, updateCount * databases.length)
})
// Both databases have the same entries in the same order
result.reduce((prev, entries) => {
assert.deepEqual(entries, prev)
return entries
}, result[0])
// Success! Cleanup and finish
pEachSeries(databases, db => {
db.close()
db._ipfs.stop()
})
.then(() => done())
})
.catch(done)
})
})
})

211
test/persistency.js
View File

@ -1,95 +1,184 @@
'use strict'
const assert = require('assert')
const mapSeries = require('./promise-map-series')
const mapSeries = require('p-map-series')
const rmrf = require('rimraf')
const hasIpfsApiWithPubsub = require('./test-utils').hasIpfsApiWithPubsub
const OrbitDB = require('../src/OrbitDB')
const config = require('./test-config')
const config = require('./utils/config')
const startIpfs = require('./utils/start-ipfs')
// Daemon settings
const daemonsConf = require('./ipfs-daemons.conf.js')
// orbit-db path
const testDataDir = './orbit-db'
config.daemons.forEach((IpfsDaemon) => {
const dbPath = './orbitdb/tests/persistency'
const ipfsPath = './orbitdb/tests/persistency/ipfs'
describe('orbit-db - Persistency', function() {
this.timeout(config.timeout)
let ipfs1, ipfs2, client1, client2, db1, db2
const entryCount = 100
const removeDirectories = () => {
rmrf.sync(daemonsConf.daemon1.IpfsDataDir)
rmrf.sync(daemonsConf.daemon2.IpfsDataDir)
rmrf.sync(config.defaultIpfsDirectory)
rmrf.sync(config.defaultOrbitDBDirectory)
rmrf.sync(testDataDir)
}
let ipfs, orbitdb1, db, address
before(function (done) {
removeDirectories()
ipfs1 = new IpfsDaemon(daemonsConf.daemon1)
ipfs1.on('error', done)
ipfs1.on('ready', () => {
assert.equal(hasIpfsApiWithPubsub(ipfs1), true)
ipfs2 = new IpfsDaemon(daemonsConf.daemon2)
ipfs2.on('error', done)
ipfs2.on('ready', () => {
assert.equal(hasIpfsApiWithPubsub(ipfs2), true)
client1 = new OrbitDB(ipfs1, "one")
client2 = new OrbitDB(ipfs2, "two")
done()
})
})
before(async () => {
config.daemon1.repo = ipfsPath
rmrf.sync(config.daemon1.repo)
rmrf.sync(dbPath)
ipfs = await startIpfs(config.daemon1)
orbitdb1 = new OrbitDB(ipfs, dbPath + '/1')
})
after(() => {
ipfs1.stop()
ipfs2.stop()
removeDirectories()
after(async () => {
if(orbitdb1)
orbitdb1.stop()
if (ipfs)
await ipfs.stop()
})
describe('load', function() {
it('loads database from local cache', function(done) {
const entryCount = 100
beforeEach(async () => {
const dbName = new Date().getTime().toString()
const entryArr = []
for (let i = 0; i < entryCount; i ++)
entryArr.push(i)
const options = {
replicate: false,
maxHistory: -1,
cachePath: testDataDir,
}
let db = client1.eventlog(config.dbname, options)
db.events.on('error', done)
db.load().then(function () {
mapSeries(entryArr, (i) => db.add('hello' + i))
.then(function() {
db = await orbitdb1.eventlog(dbName)
address = db.address.toString()
await mapSeries(entryArr, (i) => db.add('hello' + i))
await db.close()
db = null
db = client1.eventlog(config.dbname, options)
db.events.on('error', done)
db.events.on('ready', () => {
try {
})
afterEach(async () => {
await db.drop()
})
it('loads database from local cache', async () => {
db = await orbitdb1.eventlog(address)
await db.load()
const items = db.iterator({ limit: -1 }).collect()
assert.equal(items.length, entryCount)
assert.equal(items[0].payload.value, 'hello0')
assert.equal(items[entryCount - 1].payload.value, 'hello99')
done()
})
it('loading a database emits \'ready\' event', async () => {
db = await orbitdb1.eventlog(address)
return new Promise(async (resolve) => {
db.events.on('ready', () => {
const items = db.iterator({ limit: -1 }).collect()
assert.equal(items.length, entryCount)
assert.equal(items[0].payload.value, 'hello0')
assert.equal(items[entryCount - 1].payload.value, 'hello99')
resolve()
})
await db.load()
})
})
it('loading a database emits \'load.progress\' event', async () => {
db = await orbitdb1.eventlog(address)
return new Promise(async (resolve, reject) => {
let count = 0
db.events.on('load.progress', (address, hash, entry, progress, total) => {
count ++
try {
assert.equal(address, db.address.toString())
assert.equal(total, entryCount)
assert.equal(progress, count)
assert.notEqual(hash, null)
assert.notEqual(entry, null)
if (progress === entryCount && count === entryCount) {
resolve()
}
} catch (e) {
done(e)
reject(e)
}
})
db.load()
.catch(done)
// Start loading the database
await db.load()
})
.catch(done)
}).catch(done)
})
})
describe('load from snapshot', function() {
beforeEach(async () => {
const dbName = new Date().getTime().toString()
const entryArr = []
for (let i = 0; i < entryCount; i ++)
entryArr.push(i)
db = await orbitdb1.eventlog(dbName)
address = db.address.toString()
await mapSeries(entryArr, (i) => db.add('hello' + i))
await db.saveSnapshot()
await db.close()
db = null
})
afterEach(async () => {
await db.drop()
})
it('loads database from snapshot', async () => {
db = await orbitdb1.eventlog(address)
await db.loadFromSnapshot()
const items = db.iterator({ limit: -1 }).collect()
assert.equal(items.length, entryCount)
assert.equal(items[0].payload.value, 'hello0')
assert.equal(items[entryCount - 1].payload.value, 'hello99')
})
it('throws an error when trying to load a missing snapshot', async () => {
db = await orbitdb1.eventlog(address)
await db.drop()
db = null
db = await orbitdb1.eventlog(address)
let err
try {
await db.loadFromSnapshot()
} catch (e) {
err = e.toString()
}
assert.equal(err, `Error: Snapshot for ${address} not found!`)
})
it('loading a database emits \'ready\' event', async () => {
db = await orbitdb1.eventlog(address)
return new Promise(async (resolve) => {
db.events.on('ready', () => {
const items = db.iterator({ limit: -1 }).collect()
assert.equal(items.length, entryCount)
assert.equal(items[0].payload.value, 'hello0')
assert.equal(items[entryCount - 1].payload.value, 'hello99')
resolve()
})
await db.loadFromSnapshot()
})
})
it('loading a database emits \'load.progress\' event', async () => {
db = await orbitdb1.eventlog(address)
return new Promise(async (resolve, reject) => {
let count = 0
db.events.on('load.progress', (address, hash, entry, progress, total) => {
count ++
try {
assert.equal(address, db.address.toString())
assert.equal(total, entryCount)
assert.equal(progress, count)
assert.notEqual(hash, null)
assert.notEqual(entry, null)
if (progress === entryCount && count === entryCount) {
resolve()
}
} catch (e) {
reject(e)
}
})
// Start loading the database
await db.loadFromSnapshot()
})
})
})

16
test/promise-map-series.js
View File

@ -1,16 +0,0 @@
'use strict'
// https://gist.github.com/dignifiedquire/dd08d2f3806a7b87f45b00c41fe109b7
module.exports = function mapSeries (list, func) {
const res = []
return list.reduce((acc, next) => {
return acc.then((val) => {
res.push(val)
return func(next)
})
}, Promise.resolve(null)).then((val) => {
res.push(val)
return res.slice(1)
})
}

153
test/replicate-and-load.test.js Normal file
View File

@ -0,0 +1,153 @@
'use strict'
const assert = require('assert')
const mapSeries = require('p-each-series')
const rmrf = require('rimraf')
const OrbitDB = require('../src/OrbitDB')
const config = require('./utils/config')
const startIpfs = require('./utils/start-ipfs')
const stopIpfs = require('./utils/stop-ipfs')
const waitForPeers = require('./utils/wait-for-peers')
const dbPath1 = './orbitdb/tests/replicate-and-load/1'
const dbPath2 = './orbitdb/tests/replicate-and-load/2'
const ipfsPath1 = './orbitdb/tests/replicate-and-load/1/ipfs'
const ipfsPath2 = './orbitdb/tests/replicate-and-load/2/ipfs'
describe('orbit-db - Replicate and Load', function() {
this.timeout(config.timeout)
let ipfs1, ipfs2, orbitdb1, orbitdb2, db1, db2
before(async () => {
config.daemon1.repo = ipfsPath1
config.daemon2.repo = ipfsPath2
rmrf.sync(config.daemon1.repo)
rmrf.sync(config.daemon2.repo)
rmrf.sync(dbPath1)
rmrf.sync(dbPath2)
ipfs1 = await startIpfs(config.daemon1)
ipfs2 = await startIpfs(config.daemon2)
orbitdb1 = new OrbitDB(ipfs1, dbPath1)
orbitdb2 = new OrbitDB(ipfs2, dbPath2)
})
after(async () => {
if(orbitdb1)
await orbitdb1.stop()
if(orbitdb2)
await orbitdb2.stop()
if (ipfs1)
await stopIpfs(ipfs1)
if (ipfs2)
await stopIpfs(ipfs2)
})
describe('two peers', function() {
// Opens two databases db1 and db2 and gives write-access to both of the peers
const openDatabases1 = async (options) => {
// Set write access for both clients
options.write = [
orbitdb1.key.getPublic('hex'),
orbitdb2.key.getPublic('hex')
],
options = Object.assign({}, options, { path: dbPath1 })
db1 = await orbitdb1.eventlog('replicate-and-load-tests', options)
// Set 'localOnly' flag on and it'll error if the database doesn't exist locally
options = Object.assign({}, options, { path: dbPath2 })
db2 = await orbitdb2.eventlog(db1.address.toString(), options)
}
const openDatabases = async (options) => {
// Set write access for both clients
options.write = [
orbitdb1.key.getPublic('hex'),
orbitdb2.key.getPublic('hex')
],
options = Object.assign({}, options, { path: dbPath1, create: true })
db1 = await orbitdb1.eventlog('tests', options)
// Set 'localOnly' flag on and it'll error if the database doesn't exist locally
options = Object.assign({}, options, { path: dbPath2 })
db2 = await orbitdb2.eventlog(db1.address.toString(), options)
}
beforeEach(async () => {
await openDatabases({ sync: true })
assert.equal(db1.address.toString(), db2.address.toString())
console.log("Waiting for peers...")
await waitForPeers(ipfs1, [orbitdb2.id], db1.address.toString())
await waitForPeers(ipfs2, [orbitdb1.id], db1.address.toString())
console.log("Found peers")
})
afterEach(async () => {
await db1.drop()
await db2.drop()
})
it('replicates database of 100 entries and loads it from the disk', async () => {
const entryCount = 100
const entryArr = []
let timer
for (let i = 0; i < entryCount; i ++)
entryArr.push(i)
await mapSeries(entryArr, (i) => db1.add('hello' + i))
return new Promise((resolve, reject) => {
timer = setInterval(async () => {
const items = db2.iterator({ limit: -1 }).collect()
if (items.length === entryCount) {
clearInterval(timer)
assert.equal(items.length, entryCount)
assert.equal(items[0].payload.value, 'hello0')
assert.equal(items[items.length - 1].payload.value, 'hello99')
db2 = null
try {
// Set write access for both clients
let options = {
write: [
orbitdb1.key.getPublic('hex'),
orbitdb2.key.getPublic('hex')
],
}
// Get the previous address to make sure nothing mutates it
const addr = db1.address.toString()
// Open the database again (this time from the disk)
options = Object.assign({}, options, { path: dbPath1, create: false })
db1 = await orbitdb1.eventlog(addr, options)
// Set 'localOnly' flag on and it'll error if the database doesn't exist locally
options = Object.assign({}, options, { path: dbPath2, localOnly: true })
db2 = await orbitdb2.eventlog(addr, options)
await db1.load()
await db2.load()
// Make sure we have all the entries in the databases
const result1 = db1.iterator({ limit: -1 }).collect()
const result2 = db2.iterator({ limit: -1 }).collect()
assert.equal(result1.length, entryCount)
assert.equal(result2.length, entryCount)
} catch (e) {
reject(e)
}
resolve()
}
}, 100)
})
})
})
})

100
test/replicate-automatically.test.js Normal file
View File

@ -0,0 +1,100 @@
'use strict'
const assert = require('assert')
const mapSeries = require('p-each-series')
const rmrf = require('rimraf')
const OrbitDB = require('../src/OrbitDB')
const config = require('./utils/config')
const startIpfs = require('./utils/start-ipfs')
const stopIpfs = require('./utils/stop-ipfs')
const waitForPeers = require('./utils/wait-for-peers')
const dbPath1 = './orbitdb/tests/replicate-automatically/1'
const dbPath2 = './orbitdb/tests/replicate-automatically/2'
const ipfsPath1 = './orbitdb/tests/replicate-automatically/1/ipfs'
const ipfsPath2 = './orbitdb/tests/replicate-automatically/2/ipfs'
describe('orbit-db - Automatic Replication', function() {
this.timeout(config.timeout)
let ipfs1, ipfs2, orbitdb1, orbitdb2, db1, db2
before(async () => {
config.daemon1.repo = ipfsPath1
config.daemon2.repo = ipfsPath2
rmrf.sync(config.daemon1.repo)
rmrf.sync(config.daemon2.repo)
rmrf.sync(dbPath1)
rmrf.sync(dbPath2)
ipfs1 = await startIpfs(config.daemon1)
ipfs2 = await startIpfs(config.daemon2)
orbitdb1 = new OrbitDB(ipfs1, dbPath1)
orbitdb2 = new OrbitDB(ipfs2, dbPath2)
})
after(async () => {
if(orbitdb1)
await orbitdb1.stop()
if(orbitdb2)
await orbitdb2.stop()
if (ipfs1)
await stopIpfs(ipfs1)
if (ipfs2)
await stopIpfs(ipfs2)
})
beforeEach(async () => {
let options = {}
// Set write access for both clients
options.write = [
orbitdb1.key.getPublic('hex'),
orbitdb2.key.getPublic('hex')
],
options = Object.assign({}, options, { path: dbPath1 })
db1 = await orbitdb1.eventlog('replicate-automatically-tests', options)
})
afterEach(async () => {
await db1.drop()
await db2.drop()
})
it('starts replicating the database when peers connect', async () => {
const entryCount = 10
const entryArr = []
let options = {}
let timer
// Create the entries in the first database
for (let i = 0; i < entryCount; i ++)
entryArr.push(i)
await mapSeries(entryArr, (i) => db1.add('hello' + i))
// Open the second database
options = Object.assign({}, options, { path: dbPath2, sync: true })
db2 = await orbitdb2.eventlog(db1.address.toString(), options)
// Listen for the 'replicated' events and check that all the entries
// were replicated to the second database
return new Promise((resolve, reject) => {
db2.events.on('replicated', (address) => {
try {
const result1 = db1.iterator({ limit: -1 }).collect()
const result2 = db2.iterator({ limit: -1 }).collect()
// Make sure we have all the entries
if (result1.length === entryCount && result2.length === entryCount) {
assert.deepEqual(result1, result2)
resolve()
}
} catch (e) {
reject(e)
}
})
})
})
})

159
test/replicate.test.js
View File

@ -3,99 +3,90 @@
const assert = require('assert')
const mapSeries = require('p-each-series')
const rmrf = require('rimraf')
const hasIpfsApiWithPubsub = require('./test-utils').hasIpfsApiWithPubsub
const OrbitDB = require('../src/OrbitDB')
const config = require('./test-config')
const config = require('./utils/config')
const startIpfs = require('./utils/start-ipfs')
const stopIpfs = require('./utils/stop-ipfs')
const waitForPeers = require('./utils/wait-for-peers')
// Daemon settings
const daemonsConf = require('./ipfs-daemons.conf.js')
// Shared database name
const waitForPeers = (ipfs, channel) => {
return new Promise((resolve, reject) => {
console.log("Waiting for peers...")
const interval = setInterval(() => {
ipfs.pubsub.peers(channel)
.then((peers) => {
if (peers.length > 0) {
console.log("Found peers, running tests...")
clearInterval(interval)
resolve()
}
})
.catch((e) => {
clearInterval(interval)
reject(e)
})
}, 1000)
})
}
config.daemons.forEach((IpfsDaemon) => {
const dbPath1 = './orbitdb/tests/replication/1'
const dbPath2 = './orbitdb/tests/replication/2'
const ipfsPath1 = './orbitdb/tests/replication/1/ipfs'
const ipfsPath2 = './orbitdb/tests/replication/2/ipfs'
describe('orbit-db - Replication', function() {
this.timeout(config.timeout)
let ipfs1, ipfs2, client1, client2, db1, db2
let ipfs1, ipfs2, orbitdb1, orbitdb2, db1, db2
const removeDirectories = () => {
rmrf.sync(daemonsConf.daemon1.IpfsDataDir)
rmrf.sync(daemonsConf.daemon2.IpfsDataDir)
rmrf.sync(config.defaultIpfsDirectory)
rmrf.sync(config.defaultOrbitDBDirectory)
rmrf.sync('/tmp/daemon1')
rmrf.sync('/tmp/daemon2')
}
before(function (done) {
removeDirectories()
ipfs1 = new IpfsDaemon(daemonsConf.daemon1)
ipfs1.on('error', done)
ipfs1.on('ready', () => {
assert.equal(hasIpfsApiWithPubsub(ipfs1), true)
ipfs2 = new IpfsDaemon(daemonsConf.daemon2)
ipfs2.on('error', done)
ipfs2.on('ready', () => {
assert.equal(hasIpfsApiWithPubsub(ipfs2), true)
client1 = new OrbitDB(ipfs1, "one")
client2 = new OrbitDB(ipfs2, "two")
done()
})
})
before(async () => {
config.daemon1.repo = ipfsPath1
config.daemon2.repo = ipfsPath2
rmrf.sync(config.daemon1.repo)
rmrf.sync(config.daemon2.repo)
rmrf.sync(dbPath1)
rmrf.sync(dbPath2)
ipfs1 = await startIpfs(config.daemon1)
ipfs2 = await startIpfs(config.daemon2)
orbitdb1 = new OrbitDB(ipfs1, dbPath1)
orbitdb2 = new OrbitDB(ipfs2, dbPath2)
})
after((done) => {
if (client1) client1.disconnect()
if (client2) client2.disconnect()
if (ipfs1) ipfs1.stop()
if (ipfs2) ipfs2.stop()
removeDirectories()
setTimeout(() => done(), 10000)
after(async () => {
if(orbitdb1)
await orbitdb1.stop()
if(orbitdb2)
await orbitdb2.stop()
if (ipfs1)
await stopIpfs(ipfs1)
if (ipfs2)
await stopIpfs(ipfs2)
})
describe('two peers', function() {
beforeEach(() => {
db1 = client1.eventlog(config.dbname, { maxHistory: 1, cachePath: '/tmp/daemon1' })
db2 = client2.eventlog(config.dbname, { maxHistory: 1, cachePath: '/tmp/daemon2' })
beforeEach(async () => {
let options = {
// Set write access for both clients
write: [
orbitdb1.key.getPublic('hex'),
orbitdb2.key.getPublic('hex')
],
}
options = Object.assign({}, options, { path: dbPath1 })
db1 = await orbitdb1.eventlog('replication tests', options)
// Set 'sync' flag on. It'll prevent creating a new local database and rather
// fetch the database from the network
options = Object.assign({}, options, { path: dbPath2, sync: true })
db2 = await orbitdb2.eventlog(db1.address.toString(), options)
assert.equal(db1.address.toString(), db2.address.toString())
await waitForPeers(ipfs1, [orbitdb2.id], db1.address.toString())
await waitForPeers(ipfs2, [orbitdb1.id], db1.address.toString())
})
it('replicates database of 1 entry', (done) => {
waitForPeers(ipfs1, config.dbname)
.then(() => {
db2.events.once('error', done)
db2.events.once('synced', (db) => {
afterEach(async () => {
await db1.drop()
await db2.drop()
})
it('replicates database of 1 entry', async () => {
await db1.add('hello')
return new Promise(resolve => {
setTimeout(() => {
const items = db2.iterator().collect()
assert.equal(items.length, 1)
assert.equal(items[0].payload.value, 'hello')
done()
resolve()
}, 1000)
})
db1.add('hello')
.catch(done)
})
.catch(done)
})
it('replicates database of 100 entries', (done) => {
it('replicates database of 100 entries', async () => {
const entryCount = 100
const entryArr = []
let timer
@ -103,31 +94,19 @@ config.daemons.forEach((IpfsDaemon) => {
for (let i = 0; i < entryCount; i ++)
entryArr.push(i)
waitForPeers(ipfs1, config.dbname)
.then(() => {
let count = 0
db2.events.once('error', done)
db2.events.on('synced', (d) => {
if (count === entryCount && !timer) {
await mapSeries(entryArr, (i) => db1.add('hello' + i))
return new Promise(resolve => {
timer = setInterval(() => {
const items = db2.iterator({ limit: -1 }).collect()
if (items.length === count) {
if (items.length === entryCount) {
clearInterval(timer)
assert.equal(items.length, entryCount)
assert.equal(items[0].payload.value, 'hello0')
assert.equal(items[items.length - 1].payload.value, 'hello99')
setTimeout(done, 5000)
resolve()
}
}, 1000)
}
})
db1.events.on('write', () => count++)
mapSeries(entryArr, (i) => db1.add('hello' + i))
.catch(done)
})
.catch(done)
})
})
})

18
test/test-config.js
View File

@ -1,18 +0,0 @@
'use strict'
const IpfsNodeDaemon = require('ipfs-daemon/src/ipfs-node-daemon')
const IpfsNativeDaemon = require('ipfs-daemon/src/ipfs-native-daemon')
const testDaemons = require('./test-daemons')
// Set logplease logging level if in the browser
if (typeof window !== 'undefined')
window.LOG = 'ERROR'
// Config
module.exports = {
daemons: testDaemons,
timeout: 60000,
defaultIpfsDirectory: './ipfs',
defaultOrbitDBDirectory: './orbit-db',
dbname: 'abcdefghijklmn',
}

9
test/test-daemons.js
View File

@ -1,9 +0,0 @@
'use strict'
const IpfsNodeDaemon = require('ipfs-daemon/src/ipfs-node-daemon')
const IpfsNativeDaemon = require('ipfs-daemon/src/ipfs-native-daemon')
// module.exports = [IpfsNodeDaemon]
// module.exports = [IpfsNativeDaemon]
// module.exports = [IpfsNativeDaemon, IpfsNodeDaemon]
module.exports = [IpfsNodeDaemon, IpfsNativeDaemon]

75
test/utils/config.js Normal file
View File

@ -0,0 +1,75 @@
module.exports = {
timeout: 60000,
dbname: 'orbit-db-tests',
defaultIpfsConfig: {
start: true,
EXPERIMENTAL: {
pubsub: true
},
config: {
Addresses: {
API: '/ip4/127.0.0.1/tcp/0',
Swarm: ['/ip4/0.0.0.0/tcp/0'],
Gateway: '/ip4/0.0.0.0/tcp/0'
},
Bootstrap: [],
Discovery: {
MDNS: {
Enabled: true,
Interval: 10
},
webRTCStar: {
Enabled: false
}
},
}
},
daemon1: {
repo: './ipfs/orbitdb/tests/daemon1',
start: true,
EXPERIMENTAL: {
pubsub: true
},
config: {
Addresses: {
API: '/ip4/127.0.0.1/tcp/0',
Swarm: ['/ip4/0.0.0.0/tcp/0'],
Gateway: '/ip4/0.0.0.0/tcp/0'
},
Bootstrap: [],
Discovery: {
MDNS: {
Enabled: true,
Interval: 10
},
webRTCStar: {
Enabled: false
}
},
},
},
daemon2: {
repo: './ipfs/orbitdb/tests/daemon2',
start: true,
EXPERIMENTAL: {
pubsub: true
},
config: {
Addresses: {
API: '/ip4/127.0.0.1/tcp/0',
Swarm: ['/ip4/0.0.0.0/tcp/0'],
Gateway: '/ip4/0.0.0.0/tcp/0'
},
Bootstrap: [],
Discovery: {
MDNS: {
Enabled: true,
Interval: 10
},
webRTCStar: {
Enabled: false
}
},
},
},
}

18
test/utils/start-ipfs.js Normal file
View File

@ -0,0 +1,18 @@
'use strict'
const IPFS = require('ipfs')
/**
* Start an IPFS instance
* @param {Object} config [IPFS configuration to use]
* @return {[Promise<IPFS>]} [IPFS instance]
*/
const startIpfs = (config = {}) => {
return new Promise((resolve, reject) => {
const ipfs = new IPFS(config)
ipfs.on('error', reject)
ipfs.on('ready', () => resolve(ipfs))
})
}
module.exports = startIpfs

23
test/utils/stop-ipfs.js Normal file
View File

@ -0,0 +1,23 @@
'use strict'
const IPFS = require('ipfs')
/**
* Stop an IPFS instance
* @param {[IPFS]} ipfs [IPFS instance to stop]
* @return {[Promise]} [Empty]
*/
const stopIpfs = (ipfs) => {
return new Promise((resolve, reject) => {
// TODO: ipfs.stop() should return a Promise, PR it in github/js-ipfs
ipfs.stop((err) => {
if (err) {
reject(err)
} else {
resolve()
}
})
})
}
module.exports = stopIpfs

0
test/test-utils.js → test/utils/test-utils.js
View File

16
test/utils/wait-for-peers.js Normal file
View File

@ -0,0 +1,16 @@
'use strict'
const waitForPeers = (ipfs, peersToWait, topic, callback) => {
return new Promise((resolve, reject) => {
const i = setInterval(async () => {
const peers = await ipfs.pubsub.peers(topic)
const hasAllPeers = peersToWait.map((e) => peers.includes(e)).filter((e) => e === false).length === 0
if (hasAllPeers) {
clearInterval(i)
resolve()
}
}, 500)
})
}
module.exports = waitForPeers

221
test/write-permissions.test.js Normal file
View File

@ -0,0 +1,221 @@
'use strict'
const assert = require('assert')
const rmrf = require('rimraf')
const OrbitDB = require('../src/OrbitDB')
const config = require('./utils/config')
const startIpfs = require('./utils/start-ipfs')
const dbPath = './orbitdb/tests/sync'
const ipfsPath = './orbitdb/tests/feed/ipfs'
const databases = [
{
type: 'eventlog',
create: (orbitdb, name, options) => orbitdb.eventlog(name, options),
tryInsert: (db) => db.add('hello'),
query: (db) => db.iterator({ limit: -1 }).collect(),
getTestValue: (db) => db.iterator({ limit: -1 }).collect()[0].payload.value,
expectedValue: 'hello',
},
{
type: 'feed',
create: (orbitdb, name, options) => orbitdb.feed(name, options),
tryInsert: (db) => db.add('hello'),
query: (db) => db.iterator({ limit: -1 }).collect(),
getTestValue: (db) => db.iterator({ limit: -1 }).collect()[0].payload.value,
expectedValue: 'hello',
},
{
type: 'key-value',
create: (orbitdb, name, options) => orbitdb.kvstore(name, options),
tryInsert: (db) => db.set('one', 'hello'),
query: (db) => [],
getTestValue: (db) => db.get('one'),
expectedValue: 'hello',
},
{
type: 'documents',
create: (orbitdb, name, options) => orbitdb.docstore(name, options),
tryInsert: (db) => db.put({ _id: 'hello world', doc: 'all the things'}),
query: (db) => [],
getTestValue: (db) => db.get('hello world'),
expectedValue: [{ _id: 'hello world', doc: 'all the things'}],
},
{
type: 'counter',
create: (orbitdb, name, options) => orbitdb.counter(name, options),
tryInsert: (db) => db.inc(8),
query: (db) => [],
getTestValue: (db) => db.value,
expectedValue: 8,
},
]
describe('orbit-db - Write Permissions', function() {
this.timeout(20000)
let ipfs, orbitdb1, orbitdb2
before(async () => {
config.daemon1.repo = ipfsPath
rmrf.sync(config.daemon1.repo)
rmrf.sync(dbPath)
ipfs = await startIpfs(config.daemon1)
orbitdb1 = new OrbitDB(ipfs, dbPath + '/1')
orbitdb2 = new OrbitDB(ipfs, dbPath + '/2')
})
after(async () => {
if(orbitdb1)
orbitdb1.stop()
if(orbitdb2)
orbitdb2.stop()
if (ipfs)
await ipfs.stop()
})
describe('allows multiple peers to write to the databases', function() {
databases.forEach(async (database) => {
it(database.type + ' allows multiple writers', async () => {
let options = {
// Set write access for both clients
write: [
orbitdb1.key.getPublic('hex'),
orbitdb2.key.getPublic('hex')
],
}
const db1 = await database.create(orbitdb1, 'sync-test', options)
options = Object.assign({}, options, { sync: true })
const db2 = await database.create(orbitdb2, db1.address.toString(), options)
await database.tryInsert(db1)
await database.tryInsert(db2)
assert.deepEqual(database.getTestValue(db1), database.expectedValue)
assert.deepEqual(database.getTestValue(db2), database.expectedValue)
})
})
})
describe('syncs databases', function() {
databases.forEach(async (database) => {
it(database.type + ' syncs', async () => {
let options = {
// Set write access for both clients
write: [
orbitdb1.key.getPublic('hex'),
orbitdb2.key.getPublic('hex')
],
}
const db1 = await database.create(orbitdb1, 'sync-test', options)
options = Object.assign({}, options, { sync: true })
const db2 = await database.create(orbitdb2, db1.address.toString(), options)
await database.tryInsert(db2)
assert.equal(database.query(db1).length, 0)
db1.sync(db2._oplog.heads)
return new Promise(resolve => {
setTimeout(() => {
const value = database.getTestValue(db1)
assert.deepEqual(value, database.expectedValue)
resolve()
}, 1000)
})
})
})
})
describe('syncs databases that anyone can write to', function() {
databases.forEach(async (database) => {
it(database.type + ' syncs', async () => {
let options = {
// Set write permission for everyone
write: ['*'],
}
const db1 = await database.create(orbitdb1, 'sync-test-public-dbs', options)
options = Object.assign({}, options, { sync: true })
const db2 = await database.create(orbitdb2, db1.address.toString(), options)
await database.tryInsert(db2)
assert.equal(database.query(db1).length, 0)
db1.sync(db2._oplog.heads)
return new Promise(resolve => {
setTimeout(() => {
const value = database.getTestValue(db1)
assert.deepEqual(value, database.expectedValue)
resolve()
}, 1000)
})
})
})
})
describe('doesn\'t sync if peer is not allowed to write to the database', function() {
databases.forEach(async (database) => {
it(database.type + ' doesn\'t sync', async () => {
let options = {
// No write access (only creator of the database can write)
write: [],
}
options = Object.assign({}, options, { path: dbPath + '/sync-test/1' })
const db1 = await database.create(orbitdb1, 'write error test 1', options)
options = Object.assign({}, options, { path: dbPath + '/sync-test/2', sync: true })
const db2 = await database.create(orbitdb2, 'write error test 1', options)
db1.events.on('replicated', () => {
throw new Error('Shouldn\'t replicate!')
})
try {
await database.tryInsert(db2)
} catch (e) {
assert.equal(e.toString(), 'Error: Not allowed to write')
}
assert.equal(database.query(db1).length, 0)
db1.sync(db2._oplog.heads)
return new Promise(resolve => {
setTimeout(() => {
assert.equal(database.query(db1).length, 0)
resolve()
}, 500)
})
})
})
})
describe('throws an error if peer is not allowed to write to the database', function() {
databases.forEach(async (database) => {
it(database.type + ' throws an error', async () => {
let options = {
// No write access (only creator of the database can write)
write: [],
}
let err
try {
const db1 = await database.create(orbitdb1, 'write error test 2', options)
options = Object.assign({}, options, { sync: true })
const db2 = await database.create(orbitdb2, db1.address.toString(), options)
await database.tryInsert(db2)
} catch (e) {
err = e.toString()
}
assert.equal(err, 'Error: Not allowed to write')
})
})
})
})