Mirroristas/orbitdb
2
0
Fork 0
mirror of https://github.com/orbitdb/orbitdb.git synced 2025-03-30 15:08:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

Pre release (#85)

Browse Source 
* docs: Update README to match new version.

* docs: Update events example to use new API.

* docs: Correctly print out db query results.

* test: Remove concurrent.

* test: Remove unimplemented and 3rd party AC tests.

* test: Remove unimplemented and 3rd party identity tests.

* docs: Move jsdoc config to conf directory.

* Point package.json main at index.js to access all exported functions.

* docs: Vetted AC docs; these examples should work if implemented in code. Explicitly show orbit-db function imports.

* docs: Fix incorrectly declared write objects.

* docs: Improved canAppend documentation. Better JS syntax highlighting.

* docs: wss and define filters for localhost separately.

* docs: Simplified webSockets implementation with filters.

* docs: Return manifest json only (no hash). JS highlighting.

* docs: Remove operations documentation.

* docs: Update heading levels.

* docs: Differentiate between db types which expose put/add function.

* docs: Correctly import IPFS and pass config.

* docs: A simple method for full db replication.

* docs: Link to existing examples of db implementation.

* docs: Update heading.

* docs: JS code formatting. import statements.

* docs: Expand on the concepts of identities and identity management.

* docs: Describe head sync-ing and full replication.

* docs: Comprehensive explanation of setting up a db and sync-ing/replicating data across peers. Examples can be run in node.js.

* docs: Syntax highlighting. Correct code implementation for custom/3rd party storage implementations.

* docs: Getting started cleanup.

* docs: Manifest as an IPLD data strcture.
This commit is contained in:
Hayden Young 2023-06-18 02:13:54 +08:00 committed by GitHub
parent 0c01bd22b7
commit 85e6848f4c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
7 changed files with 502 additions and 172 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

104
docs/ACCESS_CONTROLLERS.md
View File

@ -6,8 +6,17 @@ An access controller is passed when a database is opened for the first time. Onc
Different access controllers can be assigned to the database using the `AccessController` param and passing it to OrbitDB's `open` function.
```
const orbitdb = await OrbitDB()
```js
import { create } from 'ipfs-core'
import { OrbitDB, getAccessController } from 'orbit-db'
const ipfs = create({ options })
const orbitdb = await OrbitDB({ ipfs })
// SomeAccessController must already be available in the AC list.
const SomeAccessController = getAccessController('some-access-controller')
const db = orbitdb.open('my-db', { AccessController: SomeAccessController() })
```
@ -17,65 +26,99 @@ OrbitDB is bundled with two AccessControllers; IPFSAccessController, an immutabl
By default, the database `db` will use the IPFSAccessController and allow only the creator to write to the database.
```
const orbitdb = await OrbitDB()
```js
const orbitdb = await OrbitDB({ ipfs })
const db = orbitdb.open('my-db')
await db.add('hello world') // only orbitdb.identity.id can write to the db.
```
To change write access, pass the IPFSAccessController with the `write ` parameter and an array of one or more Identity ids:
To change write access, pass the IPFSAccessController with the `write` parameter and an array of one or more Identity ids:
```js
import { create } from 'ipfs-core'
import { OrbitDB, Identities, getAccessController } from 'orbit-db'
const ipfs = create({ options })
```
const identities = await Identities()
const identity1 = identities.createIdentity('userA')
const identity2 = identities.createIdentity('userB')
const anotherIdentity = identities.createIdentity('userB')
const orbitdb = await OrbitDB()
const db = orbitdb.open('my-db', { AccessController: IPFSAccessController(write: [identity1.id, identity2.id]) })
// OrbitDB will create an identity using the id 'UserA'.
const orbitdb = await OrbitDB({ ipfs, id: 'userA' })
// Retrieve the access controller from the list of preloaded ACs.
const IPFSAccessController = getAccessController('ipfs')
// Open a db with write access for userA and userB.
const db = orbitdb.open('my-db', { AccessController: IPFSAccessController({ write: [orbitdb.identity.id, anotherIdentity.id]) })
```
To allow anyone to write to the database, specify the wildcard '*':
```js
import { create } from 'ipfs-core'
import { OrbitDB, Identities, getAccessController } from 'orbit-db'
const ipfs = create({ options })
const orbitdb = await OrbitDB({ ipfs })
const IPFSAccessController = getAccessController('ipfs')
const db = orbitdb.open('my-db', { AccessController: IPFSAccessController({ write: ['*'] }) })
```
const orbitdb = await OrbitDB()
const db = orbitdb.open('my-db', { AccessController: IPFSAccessController(write: ['*']) })
```
**The access information cannot be changed after the initial setup (as it is immutable).** If different write access is needed, you will need to set up a new database and associated IPFSAccessController. It is important to note that this will change the address of the database.
## OrbitDB Access Controller
The OrbitDB access controller provides configurable write access using grant and revoke.
```
```js
import { create } from 'ipfs-core'
import { OrbitDB, Identities, getAccessController } from 'orbit-db'
const ipfs = create({ options })
const orbitdb = await OrbitDB({ ipfs })
const identities = await Identities()
const identity1 = identities.createIdentity('userA')
const identity2 = identities.createIdentity('userB')
const anotherIdentity = identities.createIdentity('userB')
const orbitdb = await OrbitDB()
const db = orbitdb.open('my-db', { AccessController: OrbitDBAccessController(write: [identity1.id]) })
// Retrieve the access controller from the list of preloaded ACs.
const OrbitDBAccessController = getAccessController('orbitdb')
db.access.grant('write', identity2.id)
db.access.revoke('write', identity2.id)
const db = orbitdb.open('my-db', { AccessController: OrbitDBAccessController({ write: [orbitdb.identity.id, anotherIdentity.id]) })
db.access.grant('write', anotherIdentity.id)
db.access.revoke('write', anotherIdentity.id)
```
When granting or revoking access, a capability and the identity's id must be defined.
Grant and revoke are not limited to 'write' access only. A custom access capability can be specified, for example, `db.access.grant('custom-access', identity1.id)`.
The OrbitDBAccessController is a mutable access controller. Granting and revoking access does not change the address of the database.
## Custom Access Controller
Access can be customized by implementing a custom access controller. To implement a custom access controller, specify:
- A curried function with the function signature `async ({ orbitdb, identities, address })`,
- A partial function with the function signature `async ({ orbitdb, identities, address })`,
- A `type` constant,
- A canAppend function with the param `entry`.
- A canAppend function with the param `entry` and a boolean return type.
```
The canAppend function must return true if the entry can be appended to the log or false otherwise.
```js
const type = 'custom'
const CustomAccessController = () => async ({ orbitdb, identities, address }) => {
address = '/custom/access-controller'
const canAppend = (entry) => {
// return true if the entry can be appended to the log, false otherwise.
}
}
@ -84,7 +127,7 @@ CustomAccessController.type = type
Additional configuration can be passed to the access controller by adding one or more parameters to the `CustomAccessController` function. For example, passing a configurable object parameter with the variable `write`:
```
```js
const CustomAccessController = ({ write }) => async ({ orbitdb, identities, address }) => {
}
```
@ -95,7 +138,7 @@ The main driver of the access controller is the canAppend function. This specifi
How the custom access controller evaluates access will be determined by its use case, but in most instances, the canAppend function will want to check whether the entry being created can be written to the database (and underlying operations log). Therefore, the entry's identity will need to be used to retrieve the identity's id:
```
```js
write = [identity.id]
const canAppend = async (entry) => {
@ -119,8 +162,11 @@ In the above example, the `entry.identity` will be the hash of the identity. Usi
Before passing the custom access controller to the `open` function, it must be added to OrbitDB's AccessControllers:
```
AccessControllers.add(CustomAccessController)
const orbitdb = await OrbitDB()
```js
import { create } from 'ipfs-core'
import { OrbitDB, addAccessController } from 'orbit-db'
addAccessController(CustomAccessController)
const orbitdb = await OrbitDB({ ipfs })
const db = await orbitdb.open('my-db', { AccessController: CustomAccessController(params) })
```

46
docs/CONNECTING_PEERS.md
View File

@ -7,20 +7,24 @@ OrbitDB peers connect to one another using js-libp2p. Connection settings will v
Node.js allows libp2p to open connections with other Node.js daemons.
```javascript
ipfs1 = await IPFS.create({ repo: './ipfs1' })
ipfs2 = await IPFS.create({ repo: './ipfs2' })
import { create } from 'ipfs-core'
const ipfs1 = await create({ repo: './ipfs1' })
const ipfs2 = await create({ repo: './ipfs2' })
const cid = await ipfs1.block.put('here is some data')
const block = await ipfs2.block.get(cid)
```
On localhost or a local network, both ipfs nodes should discover each other quickly enough that ipfs2 will retrieve the block added to ipfs1.
On localhost or a local network, both ipfs nodes should discover each other quickly enough so that ipfs2 will retrieve the block added to ipfs1.
In remote networks, retrieval of content across peers may take significantly longer. To speed up communication between the two peers, connect one peer to another directly using the swarm API and a peer's publicly accessible address. For example, assuming ipfs1 is listening on the address /ip4/1.2.3.4/tcp/12345/p2p/ipfs1-peer-hash:
In remote networks, retrieval of content across peers may take significantly longer. To speed up communication between the two peers, one peer can be directly connected to another using the swarm API and a peer's publicly accessible address. For example, assuming ipfs1 is listening on the address /ip4/1.2.3.4/tcp/12345/p2p/ipfs1-peer-hash:
```javascript
ipfs1 = await IPFS.create({ repo: './ipfs1' })
ipfs2 = await IPFS.create({ repo: './ipfs2' })
import { create } from 'ipfs-core'
const ipfs1 = await create({ repo: './ipfs1' })
const ipfs2 = await create({ repo: './ipfs2' })
await ipfs2.swarm.connect('/ip4/1.2.3.4/tcp/12345/p2p/ipfs1-peer-hash')
@ -35,17 +39,17 @@ For various security reasons, a browser cannot dial another peer over a raw TCP
On the server, listen for incoming websocket connections:
```javascript
import { WebSockets } from '@libp2p/websockets'
import { webSockets } from '@libp2p/websockets'
import { create } from 'ipfs-core'
ipfs1 = await IPFS.create({
ipfs1 = await create({
libp2p: {
addresses: {
listen: [
'/ip4/0.0.0.0/tcp/0/ws'
'/ip4/0.0.0.0/tcp/0/wss'
]
},
transports: [new WebSockets()]
transports: [webSockets()]
},
repo: './ipfs1'
})
@ -55,21 +59,27 @@ Within the browser, dial into the server using the server's exposed web socket:
```javascript
// import the following libraries if using a build environment such as vite.
import { WebSockets } from '@libp2p/websockets'
import { webSockets } from '@libp2p/websockets'
import { create } from 'ipfs-core'
import { all } from '@libp2p/websockets/filters'
// uncomment { filter: all } if no tls certificate is deployed. Only do this in development environments.
const ws = new webSockets(/* { filter: all } */)
const ws = new webSockets()
ipfs1 = await IPFS.create({
ipfs1 = await create({
libp2p: {
transports: [new webSockets()]
transports: [ws]
}},
repo: './ipfs1'
})
```
You may find IPFS is unable to connect to a local WebRTC Star server. This will likely to due to the local WebSockets transport being insecure (ws instead of wss). To solve this issue, pass the `all` filter to the `webSockets` function:
```
import { all } from '@libp2p/websockets/filters'
const ws = webSockets({ filter: all })
```
## Browser to Browser and Node Daemon to Browser
A connection cannot be made directly to a browser node. Browsers do not listen for incoming connections, they operate in a server/client environment where the server address is known and the browser connects to the server using the known address. Therefore, for a browser to respond to an incoming connection a relay is required to "listen" on the browser's behalf. The relay assigns and advertises multi addresses on behalf of the browser nodes, allowing the nodes to create a direct connection between each other.
@ -86,7 +96,7 @@ In the first browser peer, configure
import { create } from 'ipfs-core'
import { multiaddr } from 'multiaddr'
ipfs = await IPFS.create({
ipfs = await create({
config: {
Addresses: {
Swarm: ['/ip4/0.0.0.0/tcp/12345/ws/p2p-webrtc-star']
@ -101,7 +111,7 @@ Configure the second browser node in the same way as the first, then dial in to
import { create } from 'ipfs-core'
import { multiaddr } from 'multiaddr'
ipfs = await IPFS.create({
ipfs = await create({
config: {
Addresses: {
Swarm: ['/ip4/0.0.0.0/tcp/12345/ws/p2p-webrtc-star']

135
docs/DATABASES.md
View File

@ -1,4 +1,4 @@
# DB
# Databases
DB provides a variety of different data stores with a common interface.
@ -13,14 +13,14 @@ OrbitDB provides four types of data stores:
The type of database can be specified when calling OrbitDB's `open` function by using the `type` parameter:
```
```js
const type = 'documents'
orbitdb.open('my-db', { type })
```
If no type is specified, Events will the default database type.
### Address
## Address
When a database is created, it is assigned an address by OrbitDB. The address consists of three parts:
@ -32,7 +32,7 @@ The first part, `/orbitdb`, specifies the protocol in use. The second part, an I
In order to replicate the database with peers, the address is what you need to give to other peers in order for them to start replicating the database.
```javascript
```js
import IPFS from 'ipfs-core'
import OrbitDB from 'orbit-db'
@ -43,7 +43,7 @@ console.log(db.address)
// /orbitdb/zdpuAmrcSRUhkQcnRQ6p4bphs7DJWGBkqczSGFYynX6moTcDL
```
### Manifest
## Manifest
The second part of the address, the IPFS multihash `zdpuAmrcSRUhkQcnRQ6p4bphs7DJWGBkqczSGFYynX6moTcDL`, is also the hash of the database's manifest. The manifest contains information about the database such as name, type and other metadata. It also contains a reference to the access controller, which is made up of the type and the hash of the access controller object.
@ -51,70 +51,68 @@ An example of a manifest is given below:
```json
{
hash: 'zdpuAzzxCWEzRffxFrxNNVkcVFbkmA1EQdpZJJPc3wpjojkAT',
manifest: {
name: 'my-db',
type: 'events',
accessController: '/ipfs/zdpuB1TUuF5E81MFChDbRsZZ1A3Kz2piLJwKQ2ddnfZLEBx64'
}
name: 'my-db',
type: 'events',
accessController: '/ipfs/zdpuB1TUuF5E81MFChDbRsZZ1A3Kz2piLJwKQ2ddnfZLEBx64'
}
```
## Operations
The manifest is simply an [IPLD data structure](https://ipld.io/docs/) which can be retrived from IPFS just like any other hash:
Operations are of either type "PUT" or "DEL".
```js
import { create } from 'ipfs-core'
import * as Block from 'multiformats/block'
import OrbitDB from 'orbit-db'
A PUT operation describes a record which has been created or edited. If operations share the same key or id, they are assumed to be related and the operation which was created after all other operations with the same key will be the latest version of the record.
const ipfs = await create()
A DEL operation describes a record which has been removed. It will share the same key as a previous PUT operation and will indicate that the record that was PUT is now deleted.
// Create the db then close.
const orbitdb = await OrbitDB({ ipfs })
const db = await orbitdb.open('my-db')
await db.close()
A PUT record might look like:
// Get the db address.
const addr = OrbitDBAddress(db.address)
// Extract the hash from the full db path.
const bytes = await ipfs.get(addr.path)
// Defines how we serialize/hash the data.
const codec = dagCbor
const hasher = sha256
// Retrieve the block data, decoding it to human-readable JSON text.
const { value } = await Block.decode({ bytes, codec, hasher })
console.log(value)
```
{
id: 'log-1',
payload: { op: 'PUT', key: 4, value: 'Some data' },
next: [ '3' ],
refs: [
'2',
'1'
],
clock: Clock {
id: '038cc50a92f10c39f74394a1779dffb2c79ddc6b7d1bbef8c484bd4bbf8330c426',
time: 4
},
v: 2
}
```
In the above example, payload holds the information about the record. `op` is the operation carried out, in this case PUT (the other option is DEL). `key` holds a unique identifier for the record and value contains some data. In the above example, data is a string but it could be a number, XML or even the JSON representation of an object.
## Opening a new database
Opening a default event store:
```
```js
const orbitdb = await OrbitDB()
await orbitdb.open('my-db')
```
Opening a documents database:
```
```js
const orbitdb = await OrbitDB()
await orbitdb.open('my-db', { type: 'documents' })
```
Opening a keyvalue database:
```
```js
const orbitdb = await OrbitDB()
await orbitdb.open('my-db', { type: 'keyvalue' })
```
Opening a database and adding meta
```
```js
const meta = { description: 'A database with metadata.' }
const orbitdb = await OrbitDB()
await orbitdb.open('my-db', { meta })
@ -122,10 +120,10 @@ await orbitdb.open('my-db', { meta })
## Loading an existing database
```
```js
const orbitdb = await OrbitDB()
const db = await orbitdb.open('my-db')
db.close()
await db.close()
const dbReopened = await orbitdb.open(db.address)
```
@ -133,25 +131,17 @@ const dbReopened = await orbitdb.open(db.address)
### Adding/Putting items in a database
All databases expose a common `put` function which is used to add items to the database.
Database types such as **documents** and **keyvalue** expose the `put` function which is used to add items as a key/value combination to the database.
```
```js
const orbitdb = await OrbitDB()
const db = await orbitdb.open('my-db', { type: keyvalue })
const hash = await db.put('key', 'value')
```
For databases such as Events which is an append-only data store, a `null` key will need to be used:
Alternatively, append-only database types such as **events** expose the `add` function which adds a value to the database:
```
const orbitdb = await OrbitDB()
const db = await orbitdb.open('my-db')
const hash = await db.put(null, 'event')
```
Alternatively, append-only databases can implement the convenience function `add`:
```
```js
const orbitdb = await OrbitDB()
const db = await orbitdb.open('my-db')
const hash = await db.add('event')
@ -159,9 +149,9 @@ const hash = await db.add('event')
### Removing/Deleting items from a database
To delete an item from a databse, use the `del` function:
To delete an item from a database, use the `del` function:
```
```js
const orbitdb = await OrbitDB()
const db = await orbitdb.open('my-db', { type: keyvalue })
const hash = await db.put('key', 'value')
@ -170,24 +160,49 @@ await db.del(hash)
## Replicating a database across peers
```
import * as IPFS from 'ipfs-core'
The power of OrbitDB lies in its ability to replicate databases across distributed systems that may not always be connected.
const ipfs1 = await IPFS.create({ config1, repo: './ipfs1' })
const ipfs2 = await IPFS.create({ config2, repo: './ipfs2' })
A simple replication process between two databases can be accomplished by listening for updates and iterating over the record set as those updates occur.
```js
import { create } from 'ipfs-core'
import { OrbitDB } from 'orbit-db'
const ipfs1 = await create({ config1, repo: './ipfs1' })
const ipfs2 = await create({ config2, repo: './ipfs2' })
orbitdb1 = await OrbitDB({ ipfs: ipfs1, id: 'user1', directory: './orbitdb1' })
orbitdb2 = await OrbitDB({ ipfs: ipfs2, id: 'user2', directory: './orbitdb2' })
const db1 = await orbitdb1.open('my-db')
await db1.add('hello world')
// Opening a db by address will start the synchronization process but only the
// database heads will be synchronized.
const db2 = await orbitdb2.open(db1.address)
// We only have the heads of db1. To replicate all of db1's records, we will
// need to iterate over db1's entire record set.
// We can determine when heads have been synchronized from db1 to db2 by
// listening for the "update" event and iterating over the record set.
db2.events.on('update', async (entry) => {
for await (const record of db2.iterator()) {
console.log(record)
}
// we can use the convenience function db.all() instead of iterating over
// db2's records.
// await db2.all()
})
```
To learn more, check out [OrbitDB's sychronization protocol](https://orbitdb.org/api/module-Sync.html) and the [OrbitDB replication documentation](./REPLICATION.md).
## Building a custom database
OrbitDB can be extended to use custom or third party data stores. To implement a custom database, ensure the Database object is extended and that the OrbitDB database interface is implement. The database will also require a unique type.
```
```js
const CustomStore = async ({ OpLog, Database, ipfs, identity, address, name, access, directory, storage, meta, syncAutomatically, indexBy = '_id' }) => {
const database = await Database({ OpLog, ipfs, identity, address, name, access, directory, storage, meta, syncAutomatically })
@ -229,4 +244,6 @@ const CustomStore = async ({ OpLog, Database, ipfs, identity, address, name, acc
iterator
}
}
```
```
[Documents](../src/db/documents.js), [Events](../src/db/events.js) and [KeyValue](../src/db/keyvalue.js) provide good examples of how a database is implemented in OrbitDB.

199
docs/GETTING_STARTED.md
View File

@ -6,52 +6,211 @@ This guide will help you get up and running with a simple OrbitDB database that
Install OrbitDB:
```
```sh
npm i orbit-db
```
You will also need IPFS for replication:
```
```sh
npm i ipfs-core
```
## Creating a simple database
## Creating a standalone database
To create a database, launch an instance of OrbitDB call the `open` function with a unique database name:
To create a database on a single machine, launch an instance of OrbitDB. Once launched, you can open a new database.
Assuming you have a Node.js development environment installed, create a new project using the command line:
```sh
mkdir orbitdb-app
cd orbitdb-app
npm init
```
const ipfs = await IPFS.create()
Create a file in your project called index.js and add the following code to it:
```js
import { OrbitDB } from 'orbit-db'
import { create } from 'ipfs-core'
// Create an IPFS instance with defaults.
const ipfs = await create()
const orbitdb = await OrbitDB({ ipfs })
const db = await orbitdb.open('my-db')
console.log('my-db address', db.address)
// Add some records to the db.
await db.add('hello world 1')
await db.add('hello world 2')
// Print out the above records.
console.log(await db.all())
// Close your db and stop OrbitDB and IPFS.
await db.close()
await orbitdb.stop()
await ipfs.stop()
```
Run index.js to create your new OrbitDB database:
```sh
node index.js
```
You should see the address of your new database and the records you have added
to it.
Without a type, OrbitDB defaults to a database type of 'events'. To change the database type, pass the `type` parameter with a valid database type.
Update:
```js
const db = await orbitdb.open('my-db')
```
Once opened, your new database will reside on the system it was created on.
to read:
Without a type, OrbitDB defaults to a database type of 'events'. To change the database type, pass a `type` with a valid database type:
```js
const db = await orbitdb.open('my-documents-db', { 'documents '})
```
Also replace:
```js
await db.add('hello world 1')
await db.add('hello world 2')
```
const type = 'documents'
const ipfs = await IPFS.create()
const orbitdb = await OrbitDB({ ipfs })
const db = await orbitdb.open('my-db', { type })
with:
```js
await db.put('doc1', { hello: "world 1", hits: 5 })
await db.put('doc2', { hello: "world 2", hits: 2 })
```
Run index.js again:
```sh
node index.js
```
You will have a new database of type 'documents'. Note that you can create all
kinds of data stores using OrbitDB. The 'documents' database allows for more complex data types such as JSON.
## Replicating a database
A database created on one peer can be replicated on another by opening the database by its address rather than by its name:
OrbitDB's power lies in its ability to replicate data between peers distributed across multiple devices and networks; peers that may not always be connected.
```
const address = '/orbitdb/zdpuAzzxCWEzRffxFrxNNVkcVFbkmA1EQdpZJJPc3wpjojkAT'
const ipfs = await IPFS.create()
const orbitdb = await OrbitDB({ ipfs })
const db = await db.open(address)
To create an OrbitDB database peer, create a new project called `orbitdb-peer`:
```sh
mkdir orbitdb-peer
cd orbitdb-peer
npm init
```
IPFS is required for carrying out the underlying replication.
Create a new file called index.js and paste in the following code:
More information about replication is available in the [Replication](./REPLICATION.md) documentation.
```js
import { OrbitDB, getAccessController } from 'orbit-db'
import { create } from 'ipfs-core'
const main = async () => {
// create a random directory to avoid IPFS and OrbitDB conflicts.
let randDir = (Math.random() + 1).toString(36).substring(2);
const config = {
Addresses: {
API: '/ip4/127.0.0.1/tcp/0',
Swarm: ['/ip4/0.0.0.0/tcp/0'],
Gateway: '/ip4/0.0.0.0/tcp/0'
}
}
// This will create an IPFS repo in ./[randDir]/ipfs.
const ipfs = await create({ config, repo: './' + randDir + '/ipfs'})
// This will create all OrbitDB-related databases (keystore, my-db, etc) in
// ./[randDir]/ipfs.
const orbitdb = await OrbitDB({ ipfs, directory: './' + randDir + '/orbitdb' })
// Get the IPFS AccessController function. We will need it to ensure everyone
// can write to the database.
const AccessController = getAccessController('ipfs')
let db
if (process.argv[2]) {
db = await orbitdb.open(process.argv[2])
} else {
// When we open a new database, write access is only available to the
// db creator. When replicating a database on a remote peer, the remote
// peer must also have write access. Here, we are simply allowing anyone
// to write to the database. A more robust solution would use the
// OrbitDBAccessController to provide "fine-grain" access using grant and
// revoke.
db = await orbitdb.open('my-db', { AccessController: AccessController({ write: ['*']})})
}
// Copy this output if you want to connect a peer to another.
console.log('my-db address', db.address)
// Add some records to the db when another peers joins.
db.events.on('join', async (peerId, heads) => {
await db.add('hello world 1')
await db.add('hello world 2')
})
db.events.on('update', async (entry) => {
console.log('entry', entry)
// To complete full replication, fetch all the records from the other peer.
await db.all()
})
// Clean up when stopping this app using ctrl+c
process.on('SIGINT', async () => {
// Close your db and stop OrbitDB and IPFS.
await db.close()
await orbitdb.stop()
await ipfs.stop()
process.exit()
})
}
main()
```
Open two consoles in your command line terminal.
In terminal 1, run the first peer:
```sh
node index.js
```
When running, you should see the address of the database, for example:
```sh
my-db address /orbitdb/zdpuB2aYUCnZ7YUBrDkCWpRLQ8ieUbqJEVRZEd5aDhJBDpBqj
```
Copy the database's address from terminal 1 and, in terminal 2, run:
```sh
node index.js /orbitdb/zdpuB2aYUCnZ7YUBrDkCWpRLQ8ieUbqJEVRZEd5aDhJBDpBqj
```
Upon connection, you should see the records being created in terminal 1's database received by the database running in terminal 2.
## Further Reading
The [Databases](./DATABASES.md) documentation covers replication and data entry in more detail.
[Databases](./DATABASES.md) covers database management and data entry in more detail.
[Replication](./REPLICATION.md) provides a comprehensive overview of how to perform data replication across multiple peers.

38
docs/IDENTITIES.md
View File

@ -1,26 +1,48 @@
# Identities
An identity is a cryptographically signed identifier or "id" and can be used to sign and verify various data. Within OrbitDB, the main objective of an identity is verify write access to a database's log and, if allowed, to sign each entry as it is added to the log.
An identity is a cryptographically signed public key which can be used to sign and verify various data. Within OrbitDB, the main objective of an identity is verify write access to a database's log and, if allowed, to sign each entry as it is added to the log.
Identities provides methods to manage one or more identities and includes functionality for creating, retrieving, signing and verifying an identity as well as signing and verifying messages using an existing identity.
`Identities` provides methods to manage one or more identities and includes functionality for creating, retrieving, signing and verifying an identity as well as signing and verifying messages using an existing identity.
## Creating an identity
```
An identity can be created by using the `createIdentity` function.
A root key is used to create a new key with the "id" of the root key's public key, Using the derived private key, the root public key is signed. This is known as the "signed message".
A new identity is signed using the root key's private key. The identity is consists of the signed message and the derived public key concatenated together ("signed identity")
A "signatures object" is then created to hold both the signed message and signed identity.
Finally, a new identity consisting of the root public key and derived public key plus the signatures object is generated and stored to the Identities storage.
```js
import { Identities } from 'orbit-db'
const id = 'userA'
const identities = await Identities()
const identities = await Identities()
const identity = identities.createIdentity({ id })
```
Once created, the identity can be passed to OrbitDB:
The `id` parameter that is passed to createIdentity is used to reference the root key pair in the PublicKeyIdentityProvider. The id can be any arbitrary text, e.g. 'bob', 'My-Key-123', etc.
The PublicKeyIdentityProvider stores the id and the root keys as a key/value pair in the key store. Other providers may not store root keys in the same manner and so the `id` parameter may not always be required.
Once created, `identities` and the associated `id` can be passed to OrbitDB:
```js
const orbitdb = await OrbitDB({ identities, id: 'userA' })
```
const orbitdb = await OrbitDB({ identity })
```
This identity can now be used by OrbitDB to control access to database actions such as write.
## Specifying a keystore
```
An existing keystore can be passed to `Identities`:
```js
import { Identities, KeyStore } from 'orbit-db'
const keystore = await KeyStore()
const id = 'userA'
const identities = await Identities({ keystore })

125
docs/REPLICATION.md
View File

@ -3,36 +3,111 @@
Below is a simple replication example. Both peers run within the same Node daemon.
```
const waitFor = async (valueA, toBeValueB, pollInterval = 100) => {
return new Promise((resolve) => {
const interval = setInterval(async () => {
if (await valueA() === await toBeValueB()) {
clearInterval(interval)
resolve()
}
}, pollInterval)
})
import { OrbitDB } from 'orbit-db'
import { create } from 'ipfs-core'
// The config will set up a TCP connection when dialling other node.js peers.
// You can find out more about peer connectivity at https://connectivity.libp2p.io/.
const config1 = {
Addresses: {
API: '/ip4/127.0.0.1/tcp/0',
Swarm: ['/ip4/0.0.0.0/tcp/0'],
Gateway: '/ip4/0.0.0.0/tcp/0'
},
Bootstrap: [],
Discovery: {
MDNS: {
Enabled: true,
Interval: 0
},
webRTCStar: {
Enabled: false
}
}
}
let connected1 = false
let connected2 = false
const onConnected1 = async (peerId, heads) => {
connected1 = true
const config2 = {
Addresses: {
API: '/ip4/127.0.0.1/tcp/0',
Swarm: ['/ip4/0.0.0.0/tcp/0'],
Gateway: '/ip4/0.0.0.0/tcp/0'
},
Bootstrap: [],
Discovery: {
MDNS: {
Enabled: true,
Interval: 0
},
webRTCStar: {
Enabled: false
}
}
}
const onConnected2 = async (peerId, heads) => {
connected2 = true
}
const ipfs1 = await create({ config: config1, repo: './ipfs/1' })
const ipfs2 = await create({ config: config2, repo: './ipfs/2' })
db1.events.on('join', onConnected1)
db2.events.on('join', onConnected2)
// The decentralized nature if IPFS can make it slow for peers to find one
// another. You can speed up a connection between two peers by "dialling-in"
// to one peer from another.
// const ipfs1PeerId = await ipfs1.id()
// await ipfs2.swarm.connect(ipfs1PeerId.id)
await db1.put({ _id: 1, msg: 'record 1 on db 1' })
await db2.put({ _id: 2, msg: 'record 2 on db 2' })
await db1.put({ _id: 3, msg: 'record 3 on db 1' })
await db2.put({ _id: 4, msg: 'record 4 on db 2' })
const orbitdb1 = await OrbitDB({ ipfs: ipfs1, id: 'userA', directory: './orbitdb/1' })
const orbitdb2 = await OrbitDB({ ipfs: ipfs2, id: 'userB', directory: './orbitdb/2' })
await waitFor(() => connected1, () => true)
await waitFor(() => connected2, () => true)
// This opens a new db. Default db type will be 'events'.
const db1 = await orbitdb1.open('my-db')
// We connect to the first db using its address. This initiates a
// synchronization of the heads between db1 and db2.
const db2 = await orbitdb2.open(db1.address)
// We write some data to db1. This will not be replicated on db2 until we
// explicitly request these records using db2's iterator or all() convenience
// function.
await db1.add('hello world 1')
await db1.add('hello world 2')
await db1.add('hello world 3')
await db1.add('hello world 4')
let db2Updated = false
// Listen for the connection of ipfs1 to ipfs2.
// If we want to listen for connections from ipfs2 to ipfs1, add a "join"
// listener to db1.
db2.events.on('join', async (peerId, heads) => {
// The peerId of the ipfs1 node.
console.log(peerId, (await ipfs1.id()).id)
})
// Listen for any updates to db2. This is especially useful when listening for
// new heads that are available on db1.
// If we want to listen for new data on db2, add an "update" listener to db1.
db2.events.on('update', async (entry) => {
// Full replication is achieved by explicitly retrieving all records from db1.
console.log(await db2.all())
db2Updated = true
})
// wait for db2 to complete updating.
await new Promise((resolve, reject) => {
setInterval(() => {
if (db2Updated) {
resolve()
}
}, 1000)
})
// Close db1 and its underlying ipfs peer.
await db1.close()
await orbitdb1.stop()
await ipfs1.stop()
// Close db2 and its underlying ipfs peer.
await db2.close()
await orbitdb2.stop()
await ipfs2.stop()
```
Refer to the API for more information about [OrbitDB's synchronization protocol](https://orbitdb.org/api/module-Sync.html).

27
docs/STORAGE.md
View File

@ -2,6 +2,8 @@
OrbitDB is all about storage, and storage can be configured to best meet the needs of the implementation. Storage is also designed to be hierarchical, allowing for a variety of storage mechanisms to be used together.
Which storage strategy is chosen depends on the requirements of the application. OrbitDB's storage customization allows for trade-offs between memory usage and speed.
## Storage types
OrbitDB is bundled with the following storage:
@ -10,7 +12,7 @@ OrbitDB is bundled with the following storage:
- LevelStorage: LevelDB-based storage,
- LRUStorage: A Least Recently Used cache,
- MemoryStorage: A memory only array,
- ComposedStorage: A storage mechanism combining two other storage objects.
- ComposedStorage: Combines two storages, eg. LRUStorage and IPFSBlockStorage.
All storage objects expose two common functions, `put` for adding a record and `get` for retrieving a record. This allows for storage to be easily swapped in and out based on the needs of the database solution.
@ -20,7 +22,7 @@ ComposedStorage combines two of the above storage objects. This reduces the need
To use composed storage, create two storage objects and then pass them to an instance of `ComposedStorage`:
```
```js
const memoryStorage = await MemoryStorage()
const levelStorage = await LevelStorage()
@ -31,27 +33,26 @@ The order in which primary storage is passed to ComposedStorage is important. Wh
## Customizing Storage
By default, OrbitDB uses `ComposedStorage`, but storage can be customized across most functionality. For example, to permanently store database operations in OpLog, the default `MemoryStorage` can be replaced with `LevelStorage`:
To override OrbitDB's default storage, alternative storages can be specified when a database is opened:
```
const identities = await Identities()
const identity = identities.createIdentity({ id: 'userA' })
```js
const entryStorage = await MemoryStorage()
const headsStorage = await MemoryStorage()
const indexStorage = await MemoryStorage()
const log = await Log(identity, { entryStorage, headsStorage, indexStorage })
await log.append('An operation')
const db = await orbitdb.open('my-db', { entryStorage, headsStorage })
```
## Implementing a third party storage solution
Any storage mechanism can be used with OrbitDB provided it implements the OrbitDB storage interface. Once created, simply pass the storage instance to OrbitDB:
```
const identities = await Identities()
const identity = identities.createIdentity({ id: 'userA' })
```js
// Perhaps some kind of locally developed storage implementation.
import CustomStorage from './custom-storage.js'
const entryStorage = await CustomStorage()
const headsStorage = await CustomStorage()
const indexStorage = await CustomStorage()
const log = await Log(identity, { entryStorage, headsStorage, indexStorage })
const db = await orbitdb.open('my-db', { entryStorage, headsStorage, indexStorage })
```
See the [various storage implementations](../src/storage) to see how custom storage should be structured for compatibility with OrbitDB.