Mirroristas/bigchaindb
2
0
Fork 0
mirror of https://github.com/bigchaindb/bigchaindb.git synced 2024-10-13 13:34:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #1 from bigchaindb/master

Browse Source 
Marge from Origin Repository
This commit is contained in:
IgorKalaidjan 2018-11-29 10:52:00 +02:00 committed by GitHub
commit c31b5497fb
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
155 changed files with 5170 additions and 2419 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

108
CHANGELOG.md
View File

@ -24,6 +24,114 @@ For reference, the possible headings are:
* **Known Issues**
* **Notes**
## [2.0 Beta 9] - 2018-11-27
### Changed
Removed support for TLSv1 and TLSv1.1 in all NGINX config files. Kept support for TLSv1.2 and added support for TLSv1.3. [Pull Request #2601](https://github.com/bigchaindb/bigchaindb/pull/2601)
### Fixed
Fixed two issues with schema validation. Pull requests [#2606](https://github.com/bigchaindb/bigchaindb/pull/2606) & [#2607](https://github.com/bigchaindb/bigchaindb/pull/2607)
### External Contributors
[@gamjapark](https://github.com/gamjapark) and team translated all the [BigchainDB root docs](https://docs.bigchaindb.com/en/latest/korean/index.html) into Korean. [Pull Request #2603](https://github.com/bigchaindb/bigchaindb/pull/2603)
## [2.0 Beta 8] - 2018-11-03
### Changed
* Revised the [Simple Deployment Template](http://docs.bigchaindb.com/projects/server/en/latest/simple-deployment-template/index.html) in the docs. Added NGINX to the mix. Pull Requests [#2578](https://github.com/bigchaindb/bigchaindb/pull/2578) and [#2579](https://github.com/bigchaindb/bigchaindb/pull/2579)
* Revised `nginx/nginx.conf` to enable CORS. [Pull Request #2580](https://github.com/bigchaindb/bigchaindb/pull/2580)
### Fixed
* Fixed a typo in the Kubernetes ConfigMap template. [Pull Request #2583](https://github.com/bigchaindb/bigchaindb/pull/2583)
### External Contributors
[@gamjapark](https://github.com/gamjapark) translated the main `README.md` file into Korean. [Pull Request #2592](https://github.com/bigchaindb/bigchaindb/pull/2592)
## [2.0 Beta 7] - 2018-09-28
Tag name: v2.0.0b7
### Added
Completed the implementation of chain-migration elections (BEP-42). Pull requests [#2553](https://github.com/bigchaindb/bigchaindb/pull/2553), [#2556](https://github.com/bigchaindb/bigchaindb/pull/2556), [#2558](https://github.com/bigchaindb/bigchaindb/pull/2558), [#2563](https://github.com/bigchaindb/bigchaindb/pull/2563) and [#2566](https://github.com/bigchaindb/bigchaindb/pull/2566)
### Changed
* Code that used the Python driver's (deprecated) transactions.send() method now uses its transactions.send_commit() method instead. [Pull request #2547](https://github.com/bigchaindb/bigchaindb/pull/2547)
* Code that implied pluggable "consensus" now implies pluggable transaction "validation" (a more accurate word). [Pull request #2561](https://github.com/bigchaindb/bigchaindb/pull/2561)
### Removed
Benchmark logs. [Pull request #2565](https://github.com/bigchaindb/bigchaindb/pull/2565)
### Fixed
A bug caused by an incorrect MongoDB query. [Pull request #2567](https://github.com/bigchaindb/bigchaindb/pull/2567)
### Notes
There's now better documentation about logs, log rotation, and the `server.bind` config setting. Pull requests [#2546](https://github.com/bigchaindb/bigchaindb/pull/2546) and [#2575](https://github.com/bigchaindb/bigchaindb/pull/2575)
## [2.0 Beta 6] - 2018-09-17
Tag name: v2.0.0b6
### Added
* [New documentation about privacy and handling private data](https://docs.bigchaindb.com/en/latest/private-data.html). [Pull request #2437](https://github.com/bigchaindb/bigchaindb/pull/2437)
* New documentation about log rotation. Also rotate Tendermint logs if started using Monit. [Pull request #2528](https://github.com/bigchaindb/bigchaindb/pull/2528)
* Began implementing one of the migration strategies outlined in [BEP-42](https://github.com/bigchaindb/BEPs/tree/master/42). That involved creating a more general-purpose election process and commands. Pull requests [#2488](https://github.com/bigchaindb/bigchaindb/pull/2488), [#2495](https://github.com/bigchaindb/bigchaindb/pull/2495), [#2498](https://github.com/bigchaindb/bigchaindb/pull/2498), [#2515](https://github.com/bigchaindb/bigchaindb/pull/2515), [#2535](https://github.com/bigchaindb/bigchaindb/pull/2535)
* Used memoization to avoid doing some validation checks multiple times. [Pull request #2490](https://github.com/bigchaindb/bigchaindb/pull/2490)
* Created an all-in-one Docker image containing BigchainDB Server, Tendermint and MongoDB. It was created for a particular user and is not recommended for production use unless you really know what you're doing. [Pull request #2424](https://github.com/bigchaindb/bigchaindb/pull/2424)
### Changed
* The supported versions of Tendermint are now hard-wired into BigchainDB Server: it checks to see what version the connected Tendermint has, and if it's not compatible, BigchainDB Server exits with an error message. [Pull request #2541](https://github.com/bigchaindb/bigchaindb/pull/2541)
* The docs no longer say to install the highest version of Tendermint: they say to install a specific version. [Pull request #2524](https://github.com/bigchaindb/bigchaindb/pull/2524)
* The setup docs include more recommended settings for `config.toml`. [Pull request #2516](https://github.com/bigchaindb/bigchaindb/pull/2516)
* The process to add, remove or update the voting power of a validator at run time (using the `bigchaindb upsert-validator` subcommands) was completely changed and is now fully working. See [issue #2372](https://github.com/bigchaindb/bigchaindb/issues/2372) and all the pull requests it references. Pull requests [#2439](https://github.com/bigchaindb/bigchaindb/pull/2439) and [#2440](https://github.com/bigchaindb/bigchaindb/pull/2440)
* The license on the documentation was changed from CC-BY-SA-4 to CC-BY-4. [Pull request #2427](https://github.com/bigchaindb/bigchaindb/pull/2427)
* Re-activated and/or updated some unit tests that had been deacivated during the migration to Tendermint. Pull requests [#2390](https://github.com/bigchaindb/bigchaindb/pull/2390), [#2415](https://github.com/bigchaindb/bigchaindb/pull/2415), [#2452](https://github.com/bigchaindb/bigchaindb/pull/24), [#2456](https://github.com/bigchaindb/bigchaindb/pull/2456)
* Updated RapidJSON to a newer, faster version. [Pull request #2470](https://github.com/bigchaindb/bigchaindb/pull/2470)
* The Java driver is now officially supported. [Pull request #2478](https://github.com/bigchaindb/bigchaindb/pull/2478)
* The MongoDB indexes on transaction id and block height were changed to be [unique indexes](https://docs.mongodb.com/manual/core/index-unique/). [Pull request #2492](https://github.com/bigchaindb/bigchaindb/pull/2492)
* Updated the required `cryptoconditions` package to a newer one. [Pull request #2494](https://github.com/bigchaindb/bigchaindb/pull/2494)
### Removed
* Removed some old code and tests. Pull requests
[#2374](https://github.com/bigchaindb/bigchaindb/pull/2374),
[#2452](https://github.com/bigchaindb/bigchaindb/pull/2452),
[#2474](https://github.com/bigchaindb/bigchaindb/pull/2474),
[#2476](https://github.com/bigchaindb/bigchaindb/pull/2476),
[#2491](https://github.com/bigchaindb/bigchaindb/pull/2491)
### Fixed
* Fixed the Events API so that it only sends valid transactions to subscribers. Also changed how it works internally, so now it is more reliable. [Pull request #2529](https://github.com/bigchaindb/bigchaindb/pull/2529)
* Fixed a bug where MongoDB database initialization would abort if a collection already existed. [Pull request #2520](https://github.com/bigchaindb/bigchaindb/pull/2520)
* Fixed a unit test that was failing randomly. [Pull request #2423](https://github.com/bigchaindb/bigchaindb/pull/2423)
* Fixed the validator curl port. [Pull request #2447](https://github.com/bigchaindb/bigchaindb/pull/2447)
* Fixed an error in the docs about the HTTP POST /transactions endpoint. [Pull request #2481](https://github.com/bigchaindb/bigchaindb/pull/2481)
* Fixed a unit test that could loop forever. [Pull requqest #2486](https://github.com/bigchaindb/bigchaindb/pull/2486)
* Fixed a bug when validating a CREATE + TRANSFER. [Pull request #2487](https://github.com/bigchaindb/bigchaindb/pull/2487)
* Fixed the HTTP response when posting a transaction in commit mode. [Pull request #2510](https://github.com/bigchaindb/bigchaindb/pull/2510)
* Fixed a crash that happened when attempting to restart BigchainDB at Tendermint block height 1. [Pull request#2519](https://github.com/bigchaindb/bigchaindb/pull/2519)
### External Contributors
@danacr - [Pull request #2447](https://github.com/bigchaindb/bigchaindb/pull/2447)
### Notes
The docs section titled "Production Deployment Template" was renamed to "Kubernetes Deployment Template" and we no longer consider it the go-to deployment template. The "Simple Deployment Template" is simpler, easier to understand, and less expensive (unless you are with an organization that already has a big Kubernetes cluster).
## [2.0 Beta 5] - 2018-08-01
Tag name: v2.0.0b5

65
README_kor.md Normal file
View File

@ -0,0 +1,65 @@
[![Codecov branch](https://img.shields.io/codecov/c/github/bigchaindb/bigchaindb/master.svg)](https://codecov.io/github/bigchaindb/bigchaindb?branch=master)
[![Latest release](https://img.shields.io/github/release/bigchaindb/bigchaindb/all.svg)](https://github.com/bigchaindb/bigchaindb/releases)
[![Status on PyPI](https://img.shields.io/pypi/status/bigchaindb.svg)](https://pypi.org/project/BigchainDB/)
[![Travis branch](https://img.shields.io/travis/bigchaindb/bigchaindb/master.svg)](https://travis-ci.org/bigchaindb/bigchaindb)
[![Documentation Status](https://readthedocs.org/projects/bigchaindb-server/badge/?version=latest)](https://docs.bigchaindb.com/projects/server/en/latest/)
[![Join the chat at https://gitter.im/bigchaindb/bigchaindb](https://badges.gitter.im/bigchaindb/bigchaindb.svg)](https://gitter.im/bigchaindb/bigchaindb?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
# BigchainDB 서버
BigchaingDB는 블록체인 데이터베이스입니다. 이 저장소는 _BigchaingDB 서버_를 위한 저장소입니다.
### 기본 사항
* [빠른 시작 사용해보기](https://docs.bigchaindb.com/projects/server/en/latest/quickstart.html)
* [BigchainDB 2.0 백서 읽기](https://www.bigchaindb.com/whitepaper/)
* [BigchainDB에 대한 _Hitchiker's Guide_를 확인십시오.](https://www.bigchaindb.com/developers/guide/)
### `master` Branch에서 BigchainDB 서버 실행 및 테스트
BigchaingDB 서버의 최신 버전을 실행하고 테스트하는 것은 어렵지 않습니다. [Docker Compose](https://docs.docker.com/compose/install/)의 최신 버전이 설치되어 있는지 확인하십시오. 준비가 되었다면, 터미널에서 다음을 실행하십시오.
```text
git clone https://github.com/bigchaindb/bigchaindb.git
cd bigchaindb
make run
```
이제 BigchainDB는 `http://localhost:9984/`에 연결되어야 합니다.
또한, 실행시키기 위한 다른 명령어들도 있습니다.
* `make start` : 소스로부터 BigchainDB를 실행하고 데몬화합니다. \(이는 `make stop` 을 하면 중지합니다.\)
* `make stop` : BigchainDB를 중지합니다.
* `make logs` : 로그에 첨부합니다.
* `make text` : 모든 유닛과 허가 테스트를 실행합니다.
* `make test-unit-watch` : 모든 테스트를 수행하고 기다립니다. 코드를 변경할 때마다 테스트는 다시 실행될 것입니다.
* `make cov` : 코드 커버리지를 확인하고 브라우저에서 결과를 엽니다.
* `make doc` : HTML 문서를 만들고, 브라우저에서 엽니다.
* `make clean` : 모든 빌드와 테스트, 커버리지 및 파이썬 아티팩트를 제거합니다.
* `make reset` : 모든 컨테이너들을 중지하고 제거합니다. 경고 : BigchainDB에 저장된 모든 데이터를 잃을 수 있습니다.
사용 가능한 모든 명령어를 보기 위해서는 `make` 를 실행하십시오.
### 모두를 위한 링크들
* [BigchainDB.com ](https://www.bigchaindb.com/)- 뉴스 레터 가입을 포함하는 BigchainDB 주요 웹 사이트
* [로드맵](https://github.com/bigchaindb/org/blob/master/ROADMAP.md)
* [블로그](https://medium.com/the-bigchaindb-blog)
* [트위터](https://twitter.com/BigchainDB)
### 개발자들을 위한 링크들
* [모든 BigchainDB 문서](https://docs.bigchaindb.com/en/latest/)
* [BigchainDB 서버 문서](https://docs.bigchaindb.com/projects/server/en/latest/index.html)
* [CONTRIBUTING.md](https://github.com/bigchaindb/bigchaindb/blob/master/.github/CONTRIBUTING.md) - 기여를 하는 방법
* [커뮤니티 가이드라인](https://github.com/bigchaindb/bigchaindb/blob/master/CODE_OF_CONDUCT.md)
* [이슈 작성](https://github.com/bigchaindb/bigchaindb/issues)
* [pull request 하기](https://github.com/bigchaindb/bigchaindb/pulls)
* [Gitter 채팅방](https://gitter.im/bigchaindb/bigchaindb)
### 합법
* [라이선스](https://github.com/bigchaindb/bigchaindb/blob/master/LICENSES.md) - 오픈 소스 & 오픈 콘텐츠
* [발행](https://www.bigchaindb.com/imprint/)
* [연락처](https://www.bigchaindb.com/contact/)

13
RELEASE_PROCESS.md
View File

@ -45,6 +45,7 @@ The following steps are what we do to release a new version of _BigchainDB Serve
- In `bigchaindb/version.py`:
- update `__version__` to e.g. `0.9.0` (with no `.dev` on the end)
- update `__short_version__` to e.g. `0.9` (with no `.dev` on the end)
- In the docs about installing BigchainDB (and Tendermint), and in the associated scripts, recommend/install a version of Tendermint that _actually works_ with the soon-to-be-released version of BigchainDB. You can find all such references by doing a search for the previously-recommended version number, such as `0.22.8`.
- In `setup.py`, _maybe_ update the development status item in the `classifiers` list. For example, one allowed value is `"Development Status :: 5 - Production/Stable"`. The [allowed values are listed at pypi.python.org](https://pypi.python.org/pypi?%3Aaction=list_classifiers).
1. **Wait for all the tests to pass!**
@ -59,17 +60,23 @@ The following steps are what we do to release a new version of _BigchainDB Serve
1. Click "Publish release" to publish the release on GitHub.
1. On your local computer, make sure you're on the `master` branch and that it's up-to-date with the `master` branch in the bigchaindb/bigchaindb repository (e.g. `git pull upstream master`). We're going to use that to push a new `bigchaindb` package to PyPI.
1. Make sure you have a `~/.pypirc` file containing credentials for PyPI.
1. Do `make release` to build and publish the new `bigchaindb` package on PyPI.
1. Do `make release` to build and publish the new `bigchaindb` package on PyPI. For this step you need to have `twine` installed. If you get an error like `Makefile:135: recipe for target 'clean-pyc' failed` then try doing
```text
sudo chown -R $(whoami):$(whoami) .
```
1. [Log in to readthedocs.org](https://readthedocs.org/accounts/login/) and go to the **BigchainDB Server** project, then:
- Click on "Builds", select "latest" from the drop-down menu, then click the "Build Version:" button.
- Wait for the build of "latest" to finish. This can take a few minutes.
- Go to Admin --> Advanced Settings
and make sure that "Default branch:" (i.e. what "latest" points to)
is set to the new release's tag, e.g. `v0.9.1`.
(Don't miss the `v` in front.)
(It won't be an option if you didn't wait for the build of "latest" to finish.)
Then scroll to the bottom and click "Save".
- Go to Admin --> Versions
and under **Choose Active Versions**, do these things:
1. Make sure that the new version's tag is "Active" and "Public"
1. Make sure the **stable** branch is _not_ active.
1. Scroll to the bottom of the page and click the "Submit" button.
1. Scroll to the bottom of the page and click "Save".
1. Go to [Docker Hub](https://hub.docker.com/) and sign in, then:
- Click on "Organizations"
- Click on "bigchaindb"

2
acceptance/python/Dockerfile
View File

@ -5,5 +5,5 @@ RUN pip install --upgrade \
pycco \
websocket-client~=0.47.0 \
pytest~=3.0 \
bigchaindb-driver==0.5.1 \
bigchaindb-driver~=0.6.2 \
blns

4
acceptance/python/src/test_basic.py
View File

@ -61,7 +61,7 @@ def test_basic():
bike_id = fulfilled_creation_tx['id']
# Now she is ready to send it to the BigchainDB Network.
sent_transfer_tx = bdb.transactions.send(fulfilled_creation_tx)
sent_transfer_tx = bdb.transactions.send_commit(fulfilled_creation_tx)
# And just to be 100% sure, she also checks if she can retrieve
# it from the BigchainDB node.
@ -107,7 +107,7 @@ def test_basic():
private_keys=alice.private_key)
# She finally sends the transaction to a BigchainDB node.
sent_transfer_tx = bdb.transactions.send(fulfilled_transfer_tx)
sent_transfer_tx = bdb.transactions.send_commit(fulfilled_transfer_tx)
# And just to be 100% sure, she also checks if she can retrieve
# it from the BigchainDB node.

7
acceptance/python/src/test_divisible_asset.py
View File

@ -74,7 +74,7 @@ def test_divisible_assets():
prepared_token_tx,
private_keys=alice.private_key)
bdb.transactions.send(fulfilled_token_tx, mode='commit')
bdb.transactions.send_commit(fulfilled_token_tx)
# We store the `id` of the transaction to use it later on.
bike_token_id = fulfilled_token_tx['id']
@ -116,8 +116,7 @@ def test_divisible_assets():
prepared_transfer_tx,
private_keys=bob.private_key)
sent_transfer_tx = bdb.transactions.send(fulfilled_transfer_tx,
mode='commit')
sent_transfer_tx = bdb.transactions.send_commit(fulfilled_transfer_tx)
# First, Bob checks if the transaction was successful.
assert bdb.transactions.retrieve(
@ -167,7 +166,7 @@ def test_divisible_assets():
# Remember Bob, last time you spent 3 tokens already,
# so you only have 7 left.
with pytest.raises(BadRequest) as error:
bdb.transactions.send(fulfilled_transfer_tx, mode='commit')
bdb.transactions.send_commit(fulfilled_transfer_tx)
# Now Bob gets an error saying that the amount he wanted to spent is
# higher than the amount of tokens he has left.

2
acceptance/python/src/test_double_spend.py
View File

@ -30,7 +30,7 @@ def test_double_create():
def send_and_queue(tx):
try:
bdb.transactions.send(tx)
bdb.transactions.send_commit(tx)
results.put('OK')
except bigchaindb_driver.exceptions.TransportError as e:
results.put('FAIL')

5
acceptance/python/src/test_multiple_owners.py
View File

@ -64,7 +64,7 @@ def test_multiple_owners():
prepared_dw_tx,
private_keys=[alice.private_key, bob.private_key])
bdb.transactions.send(fulfilled_dw_tx, mode='commit')
bdb.transactions.send_commit(fulfilled_dw_tx)
# We store the `id` of the transaction to use it later on.
dw_id = fulfilled_dw_tx['id']
@ -109,8 +109,7 @@ def test_multiple_owners():
prepared_transfer_tx,
private_keys=[alice.private_key, bob.private_key])
sent_transfer_tx = bdb.transactions.send(fulfilled_transfer_tx,
mode='commit')
sent_transfer_tx = bdb.transactions.send_commit(fulfilled_transfer_tx)
# They check if the transaction was successful.
assert bdb.transactions.retrieve(

2
acceptance/python/src/test_naughty_strings.py
View File

@ -54,7 +54,7 @@ def send_naughty_tx(asset, metadata):
# The fulfilled tx gets sent to the BDB network
try:
sent_transaction = bdb.transactions.send(fulfilled_transaction)
sent_transaction = bdb.transactions.send_commit(fulfilled_transaction)
except BadRequest as e:
sent_transaction = e

2
acceptance/python/src/test_stream.py
View File

@ -100,7 +100,7 @@ def test_stream():
# transactions to be in the shared queue: this is a two phase test,
# first we send a bunch of transactions, then we check if they are
# valid (and, in this case, they should).
bdb.transactions.send(tx, mode='async')
bdb.transactions.send_async(tx)
# The `id` of every sent transaction is then stored in a list.
sent.append(tx['id'])

4
bigchaindb/README.md
View File

@ -18,9 +18,9 @@ The `BigchainDB` class is defined here. Most node-level operations and database
`Block`, `Transaction`, and `Asset` classes are defined here. The classes mirror the block and transaction structure from the [documentation](https://docs.bigchaindb.com/projects/server/en/latest/data-models/index.html), but also include methods for validation and signing.
### [`consensus.py`](./consensus.py)
### [`validation.py`](./validation.py)
Base class for consensus methods (verification of votes, blocks, and transactions). The actual logic is mostly found in `transaction` and `block` models, defined in [`models.py`](./models.py).
Base class for validation methods (verification of votes, blocks, and transactions). The actual logic is mostly found in `transaction` and `block` models, defined in [`models.py`](./models.py).
### [`processes.py`](./processes.py)

8
bigchaindb/__init__.py
View File

@ -7,6 +7,7 @@ import logging
from bigchaindb.log import DEFAULT_LOGGING_CONFIG as log_config
from bigchaindb.lib import BigchainDB # noqa
from bigchaindb.migrations.chain_migration_election import ChainMigrationElection
from bigchaindb.version import __version__ # noqa
from bigchaindb.core import App # noqa
@ -94,9 +95,10 @@ _config = copy.deepcopy(config)
from bigchaindb.common.transaction import Transaction # noqa
from bigchaindb import models # noqa
from bigchaindb.upsert_validator import ValidatorElection # noqa
from bigchaindb.upsert_validator import ValidatorElectionVote # noqa
from bigchaindb.elections.vote import Vote # noqa
Transaction.register_type(Transaction.CREATE, models.Transaction)
Transaction.register_type(Transaction.TRANSFER, models.Transaction)
Transaction.register_type(ValidatorElection.VALIDATOR_ELECTION, ValidatorElection)
Transaction.register_type(ValidatorElectionVote.VALIDATOR_ELECTION_VOTE, ValidatorElectionVote)
Transaction.register_type(ValidatorElection.OPERATION, ValidatorElection)
Transaction.register_type(ChainMigrationElection.OPERATION, ChainMigrationElection)
Transaction.register_type(Vote.OPERATION, Vote)

2
bigchaindb/backend/localmongodb/connection.py
View File

@ -62,7 +62,7 @@ class LocalMongoDBConnection(Connection):
try:
try:
return query.run(self.conn)
except pymongo.errors.AutoReconnect as exc:
except pymongo.errors.AutoReconnect:
logger.warning('Lost connection to the database, '
'retrying query.')
return query.run(self.conn)

112
bigchaindb/backend/localmongodb/query.py
View File

@ -91,9 +91,10 @@ def get_assets(conn, asset_ids):
@register_query(LocalMongoDBConnection)
def get_spent(conn, transaction_id, output):
query = {'inputs.fulfills': {
'transaction_id': transaction_id,
'output_index': output}}
query = {'inputs':
{'$elemMatch':
{'$and': [{'fulfills.transaction_id': transaction_id},
{'fulfills.output_index': output}]}}}
return conn.run(
conn.collection('transactions')
@ -182,15 +183,18 @@ def get_owned_ids(conn, owner):
@register_query(LocalMongoDBConnection)
def get_spending_transactions(conn, inputs):
transaction_ids = [i['transaction_id'] for i in inputs]
output_indexes = [i['output_index'] for i in inputs]
query = {'inputs':
{'$elemMatch':
{'$and':
[
{'fulfills.transaction_id': {'$in': transaction_ids}},
{'fulfills.output_index': {'$in': output_indexes}}
]}}}
cursor = conn.run(
conn.collection('transactions').aggregate([
{'$match': {
'inputs.fulfills': {
'$in': inputs,
},
}},
{'$project': {'_id': False}}
]))
conn.collection('transactions').find(query, {'_id': False}))
return cursor
@ -236,7 +240,7 @@ def store_unspent_outputs(conn, *unspent_outputs):
def delete_unspent_outputs(conn, *unspent_outputs):
if unspent_outputs:
return conn.run(
conn.collection('utxos').remove({
conn.collection('utxos').delete_many({
'$or': [{
'$and': [
{'transaction_id': unspent_output['transaction_id']},
@ -257,18 +261,15 @@ def get_unspent_outputs(conn, *, query=None):
@register_query(LocalMongoDBConnection)
def store_pre_commit_state(conn, state):
commit_id = state['commit_id']
return conn.run(
conn.collection('pre_commit')
.update({'commit_id': commit_id}, state, upsert=True)
.replace_one({}, state, upsert=True)
)
@register_query(LocalMongoDBConnection)
def get_pre_commit_state(conn, commit_id):
return conn.run(conn.collection('pre_commit')
.find_one({'commit_id': commit_id},
projection={'_id': False}))
def get_pre_commit_state(conn):
return conn.run(conn.collection('pre_commit').find_one())
@register_query(LocalMongoDBConnection)
@ -283,6 +284,41 @@ def store_validator_set(conn, validators_update):
)
@register_query(LocalMongoDBConnection)
def delete_validator_set(conn, height):
return conn.run(
conn.collection('validators').delete_many({'height': height})
)
@register_query(LocalMongoDBConnection)
def store_election(conn, election_id, height, is_concluded):
return conn.run(
conn.collection('elections').replace_one(
{'election_id': election_id,
'height': height},
{'election_id': election_id,
'height': height,
'is_concluded': is_concluded},
upsert=True,
)
)
@register_query(LocalMongoDBConnection)
def store_elections(conn, elections):
return conn.run(
conn.collection('elections').insert_many(elections)
)
@register_query(LocalMongoDBConnection)
def delete_elections(conn, height):
return conn.run(
conn.collection('elections').delete_many({'height': height})
)
@register_query(LocalMongoDBConnection)
def get_validator_set(conn, height=None):
query = {}
@ -296,20 +332,19 @@ def get_validator_set(conn, height=None):
.limit(1)
)
return list(cursor)[0]
return next(cursor, None)
@register_query(LocalMongoDBConnection)
def get_validator_set_by_election_id(conn, election_id):
def get_election(conn, election_id):
query = {'election_id': election_id}
cursor = conn.run(
conn.collection('validators')
.find(query, projection={'_id': False})
return conn.run(
conn.collection('elections')
.find_one(query, projection={'_id': False},
sort=[('height', DESCENDING)])
)
return next(cursor, None)
@register_query(LocalMongoDBConnection)
def get_asset_tokens_for_public_key(conn, asset_id, public_key):
@ -322,3 +357,30 @@ def get_asset_tokens_for_public_key(conn, asset_id, public_key):
{'$project': {'_id': False}}
]))
return cursor
@register_query(LocalMongoDBConnection)
def store_abci_chain(conn, height, chain_id, is_synced=True):
return conn.run(
conn.collection('abci_chains').replace_one(
{'height': height},
{'height': height, 'chain_id': chain_id,
'is_synced': is_synced},
upsert=True,
)
)
@register_query(LocalMongoDBConnection)
def delete_abci_chain(conn, height):
return conn.run(
conn.collection('abci_chains').delete_many({'height': height})
)
@register_query(LocalMongoDBConnection)
def get_latest_abci_chain(conn):
return conn.run(
conn.collection('abci_chains')
.find_one(projection={'_id': False}, sort=[('height', DESCENDING)])
)

151
bigchaindb/backend/localmongodb/schema.py
View File

@ -7,9 +7,9 @@
import logging
from pymongo import ASCENDING, DESCENDING, TEXT
from pymongo.errors import CollectionInvalid
from bigchaindb import backend
from bigchaindb.common import exceptions
from bigchaindb.backend.utils import module_dispatch_registrar
from bigchaindb.backend.localmongodb.connection import LocalMongoDBConnection
@ -18,12 +18,48 @@ logger = logging.getLogger(__name__)
register_schema = module_dispatch_registrar(backend.schema)
INDEXES = {
'transactions': [
('id', dict(unique=True, name='transaction_id')),
('asset.id', dict(name='asset_id')),
('outputs.public_keys', dict(name='outputs')),
([('inputs.fulfills.transaction_id', ASCENDING),
('inputs.fulfills.output_index', ASCENDING)], dict(name='inputs')),
],
'assets': [
('id', dict(name='asset_id', unique=True)),
([('$**', TEXT)], dict(name='text')),
],
'blocks': [
([('height', DESCENDING)], dict(name='height', unique=True)),
],
'metadata': [
('id', dict(name='transaction_id', unique=True)),
([('$**', TEXT)], dict(name='text')),
],
'utxos': [
([('transaction_id', ASCENDING),
('output_index', ASCENDING)], dict(name='utxo', unique=True)),
],
'pre_commit': [
('height', dict(name='height', unique=True)),
],
'elections': [
([('height', DESCENDING), ('election_id', ASCENDING)],
dict(name='election_id_height', unique=True)),
],
'validators': [
('height', dict(name='height', unique=True)),
],
'abci_chains': [
('height', dict(name='height', unique=True)),
('chain_id', dict(name='chain_id', unique=True)),
],
}
@register_schema(LocalMongoDBConnection)
def create_database(conn, dbname):
if dbname in conn.conn.database_names():
raise exceptions.DatabaseAlreadyExists('Database `{}` already exists'
.format(dbname))
logger.info('Create database `%s`.', dbname)
# TODO: read and write concerns can be declared here
conn.conn.get_database(dbname)
@ -32,107 +68,22 @@ def create_database(conn, dbname):
@register_schema(LocalMongoDBConnection)
def create_tables(conn, dbname):
for table_name in backend.schema.TABLES:
logger.info('Create `%s` table.', table_name)
# create the table
# TODO: read and write concerns can be declared here
conn.conn[dbname].create_collection(table_name)
try:
logger.info(f'Create `{table_name}` table.')
conn.conn[dbname].create_collection(table_name)
except CollectionInvalid:
logger.info(f'Collection {table_name} already exists.')
create_indexes(conn, dbname, table_name, INDEXES[table_name])
@register_schema(LocalMongoDBConnection)
def create_indexes(conn, dbname):
create_transactions_secondary_index(conn, dbname)
create_assets_secondary_index(conn, dbname)
create_blocks_secondary_index(conn, dbname)
create_metadata_secondary_index(conn, dbname)
create_utxos_secondary_index(conn, dbname)
create_pre_commit_secondary_index(conn, dbname)
create_validators_secondary_index(conn, dbname)
def create_indexes(conn, dbname, collection, indexes):
logger.info(f'Ensure secondary indexes for `{collection}`.')
for fields, kwargs in indexes:
conn.conn[dbname][collection].create_index(fields, **kwargs)
@register_schema(LocalMongoDBConnection)
def drop_database(conn, dbname):
conn.conn.drop_database(dbname)
def create_transactions_secondary_index(conn, dbname):
logger.info('Create `transactions` secondary index.')
# to query the transactions for a transaction id, this field is unique
conn.conn[dbname]['transactions'].create_index('id',
unique=True,
name='transaction_id')
# secondary index for asset uuid, this field is unique
conn.conn[dbname]['transactions']\
.create_index('asset.id', name='asset_id')
# secondary index on the public keys of outputs
conn.conn[dbname]['transactions']\
.create_index('outputs.public_keys',
name='outputs')
# secondary index on inputs/transaction links (transaction_id, output)
conn.conn[dbname]['transactions']\
.create_index([
('inputs.fulfills.transaction_id', ASCENDING),
('inputs.fulfills.output_index', ASCENDING),
], name='inputs')
def create_assets_secondary_index(conn, dbname):
logger.info('Create `assets` secondary index.')
# unique index on the id of the asset.
# the id is the txid of the transaction that created the asset
conn.conn[dbname]['assets'].create_index('id',
name='asset_id',
unique=True)
# full text search index
conn.conn[dbname]['assets'].create_index([('$**', TEXT)], name='text')
def create_blocks_secondary_index(conn, dbname):
conn.conn[dbname]['blocks']\
.create_index([('height', DESCENDING)], name='height', unique=True)
def create_metadata_secondary_index(conn, dbname):
logger.info('Create `assets` secondary index.')
# the id is the txid of the transaction where metadata was defined
conn.conn[dbname]['metadata'].create_index('id',
name='transaction_id',
unique=True)
# full text search index
conn.conn[dbname]['metadata'].create_index([('$**', TEXT)], name='text')
def create_utxos_secondary_index(conn, dbname):
logger.info('Create `utxos` secondary index.')
conn.conn[dbname]['utxos'].create_index(
[('transaction_id', ASCENDING), ('output_index', ASCENDING)],
name='utxo',
unique=True,
)
def create_pre_commit_secondary_index(conn, dbname):
logger.info('Create `pre_commit` secondary index.')
conn.conn[dbname]['pre_commit'].create_index('commit_id',
name='pre_commit_id',
unique=True)
def create_validators_secondary_index(conn, dbname):
logger.info('Create `validators` secondary index.')
conn.conn[dbname]['validators'].create_index('height',
name='height',
unique=True,)
conn.conn[dbname]['validators'].create_index('election_id',
name='election_id',
unique=True,)

83
bigchaindb/backend/query.py
View File

@ -8,9 +8,6 @@ from functools import singledispatch
from bigchaindb.backend.exceptions import OperationError
VALIDATOR_UPDATE_ID = 'a_unique_id_string'
PRE_COMMIT_ID = 'a_unique_id_string'
@singledispatch
def store_asset(connection, asset):
@ -316,12 +313,11 @@ def get_unspent_outputs(connection, *, query=None):
@singledispatch
def store_pre_commit_state(connection, commit_id, state):
"""Store pre-commit state in a document with `id` as `commit_id`.
def store_pre_commit_state(connection, state):
"""Store pre-commit state.
Args:
commit_id (string): `id` of document where `state` should be stored.
state (dict): commit state.
state (dict): pre-commit state.
Returns:
The result of the operation.
@ -331,14 +327,11 @@ def store_pre_commit_state(connection, commit_id, state):
@singledispatch
def get_pre_commit_state(connection, commit_id):
"""Get pre-commit state where `id` is `commit_id`.
Args:
commit_id (string): `id` of document where `state` should be stored.
def get_pre_commit_state(connection):
"""Get pre-commit state.
Returns:
Document with `id` as `commit_id`
Document representing the pre-commit state.
"""
raise NotImplementedError
@ -351,6 +344,34 @@ def store_validator_set(conn, validator_update):
raise NotImplementedError
@singledispatch
def delete_validator_set(conn, height):
"""Delete the validator set at the given height."""
raise NotImplementedError
@singledispatch
def store_election(conn, election_id, height, is_concluded):
"""Store election record"""
raise NotImplementedError
@singledispatch
def store_elections(conn, elections):
"""Store election records in bulk"""
raise NotImplementedError
@singledispatch
def delete_elections(conn, height):
"""Delete all election records at the given height"""
raise NotImplementedError
@singledispatch
def get_validator_set(conn, height):
"""Get validator set for a given `height`, if `height` is not specified
@ -361,22 +382,48 @@ def get_validator_set(conn, height):
@singledispatch
def get_validator_set_by_election_id(conn, election_id):
"""Return a validator set change with the specified election_id
def get_election(conn, election_id):
"""Return the election record
"""
raise NotImplementedError
@singledispatch
def get_asset_tokens_for_public_key(connection, asset_id,
public_key, operation):
def get_asset_tokens_for_public_key(connection, asset_id, public_key):
"""Retrieve a list of tokens of type `asset_id` that are owned by the `public_key`.
Args:
asset_id (str): Id of the token.
public_key (str): base58 encoded public key
operation: filter transaction based on `operation`
Returns:
Iterator of transaction that list given owner in conditions.
"""
raise NotImplementedError
@singledispatch
def store_abci_chain(conn, height, chain_id, is_synced=True):
"""Create or update an ABCI chain at the given height.
Usually invoked in the beginning of the ABCI communications (height=0)
or when ABCI client (like Tendermint) is migrated (any height).
Args:
is_synced: True if the chain is known by both ABCI client and server
"""
raise NotImplementedError
@singledispatch
def delete_abci_chain(conn, height):
"""Delete the ABCI chain at the given height."""
raise NotImplementedError
@singledispatch
def get_latest_abci_chain(conn):
"""Returns the ABCI chain stored at the biggest height, if any,
None otherwise.
"""
raise NotImplementedError

28
bigchaindb/backend/schema.py
View File

@ -10,13 +10,13 @@ import logging
import bigchaindb
from bigchaindb.backend.connection import connect
from bigchaindb.common.exceptions import ValidationError
from bigchaindb.common.utils import validate_all_values_for_key
from bigchaindb.common.utils import validate_all_values_for_key_in_obj, validate_all_values_for_key_in_list
logger = logging.getLogger(__name__)
# Tables/collections that every backend database must create
TABLES = ('transactions', 'blocks', 'assets', 'metadata',
'validators', 'pre_commit', 'utxos')
'validators', 'elections', 'pre_commit', 'utxos', 'abci_chains')
VALID_LANGUAGES = ('danish', 'dutch', 'english', 'finnish', 'french', 'german',
'hungarian', 'italian', 'norwegian', 'portuguese', 'romanian',
@ -31,10 +31,6 @@ def create_database(connection, dbname):
Args:
dbname (str): the name of the database to create.
Raises:
:exc:`~DatabaseAlreadyExists`: If the given :attr:`dbname` already
exists as a database.
"""
raise NotImplementedError
@ -51,17 +47,6 @@ def create_tables(connection, dbname):
raise NotImplementedError
@singledispatch
def create_indexes(connection, dbname):
"""Create the indexes to be used by BigchainDB.
Args:
dbname (str): the name of the database to create indexes for.
"""
raise NotImplementedError
@singledispatch
def drop_database(connection, dbname):
"""Drop the database used by BigchainDB.
@ -90,10 +75,6 @@ def init_database(connection=None, dbname=None):
dbname (str): the name of the database to create.
Defaults to the database name given in the BigchainDB
configuration.
Raises:
:exc:`~DatabaseAlreadyExists`: If the given :attr:`dbname` already
exists as a database.
"""
connection = connection or connect()
@ -101,7 +82,6 @@ def init_database(connection=None, dbname=None):
create_database(connection, dbname)
create_tables(connection, dbname)
create_indexes(connection, dbname)
def validate_language_key(obj, key):
@ -121,7 +101,9 @@ def validate_language_key(obj, key):
if backend == 'localmongodb':
data = obj.get(key, {})
if isinstance(data, dict):
validate_all_values_for_key(data, 'language', validate_language)
validate_all_values_for_key_in_obj(data, 'language', validate_language)
elif isinstance(data, list):
validate_all_values_for_key_in_list(data, 'language', validate_language)
def validate_language(value):

207
bigchaindb/commands/bigchaindb.py
View File

@ -13,21 +13,23 @@ import copy
import json
import sys
from bigchaindb.core import rollback
from bigchaindb.migrations.chain_migration_election import ChainMigrationElection
from bigchaindb.utils import load_node_key
from bigchaindb.common.exceptions import (DatabaseAlreadyExists,
DatabaseDoesNotExist,
from bigchaindb.common.exceptions import (DatabaseDoesNotExist,
ValidationError)
from bigchaindb.elections.vote import Vote
import bigchaindb
from bigchaindb import (backend, ValidatorElection,
BigchainDB, ValidatorElectionVote)
BigchainDB)
from bigchaindb.backend import schema
from bigchaindb.backend import query
from bigchaindb.backend.query import PRE_COMMIT_ID
from bigchaindb.commands import utils
from bigchaindb.commands.utils import (configure_bigchaindb,
input_on_stderr)
from bigchaindb.log import setup_logging
from bigchaindb.tendermint_utils import public_key_from_base64, public_key_to_base64
from bigchaindb.tendermint_utils import public_key_from_base64
from bigchaindb.commands.election_types import elections
from bigchaindb.version import __tm_supported_versions__
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
@ -100,41 +102,27 @@ def run_configure(args):
@configure_bigchaindb
def run_upsert_validator(args):
"""Initiate and manage elections to change the validator set"""
def run_election(args):
"""Initiate and manage elections"""
b = BigchainDB()
# Call the function specified by args.action, as defined above
globals()[f'run_upsert_validator_{args.action}'](args, b)
globals()[f'run_election_{args.action}'](args, b)
def run_upsert_validator_new(args, bigchain):
"""Initiates an election to add/update/remove a validator to an existing BigchainDB network
def run_election_new(args, bigchain):
election_type = args.election_type.replace('-', '_')
globals()[f'run_election_new_{election_type}'](args, bigchain)
:param args: dict
args = {
'public_key': the public key of the proposed peer, (str)
'power': the proposed validator power for the new peer, (str)
'node_id': the node_id of the new peer (str)
'sk': the path to the private key of the node calling the election (str)
}
:param bigchain: an instance of BigchainDB
:return: election_id or `False` in case of failure
"""
new_validator = {
'public_key': public_key_from_base64(args.public_key),
'power': args.power,
'node_id': args.node_id
}
def create_new_election(sk, bigchain, election_class, data):
try:
key = load_node_key(args.sk)
voters = ValidatorElection.recipients(bigchain)
election = ValidatorElection.generate([key.public_key],
voters,
new_validator, None).sign([key.private_key])
key = load_node_key(sk)
voters = election_class.recipients(bigchain)
election = election_class.generate([key.public_key],
voters,
data, None).sign([key.private_key])
election.validate(bigchain)
except ValidationError as e:
logger.error(e)
@ -152,8 +140,46 @@ def run_upsert_validator_new(args, bigchain):
return False
def run_upsert_validator_approve(args, bigchain):
"""Approve an election to add/update/remove a validator to an existing BigchainDB network
def run_election_new_upsert_validator(args, bigchain):
"""Initiates an election to add/update/remove a validator to an existing BigchainDB network
:param args: dict
args = {
'public_key': the public key of the proposed peer, (str)
'power': the proposed validator power for the new peer, (str)
'node_id': the node_id of the new peer (str)
'sk': the path to the private key of the node calling the election (str)
}
:param bigchain: an instance of BigchainDB
:return: election_id or `False` in case of failure
"""
new_validator = {
'public_key': {'value': public_key_from_base64(args.public_key),
'type': 'ed25519-base16'},
'power': args.power,
'node_id': args.node_id
}
return create_new_election(args.sk, bigchain, ValidatorElection, new_validator)
def run_election_new_chain_migration(args, bigchain):
"""Initiates an election to halt block production
:param args: dict
args = {
'sk': the path to the private key of the node calling the election (str)
}
:param bigchain: an instance of BigchainDB
:return: election_id or `False` in case of failure
"""
return create_new_election(args.sk, bigchain, ChainMigrationElection, {})
def run_election_approve(args, bigchain):
"""Approve an election
:param args: dict
args = {
@ -175,9 +201,9 @@ def run_upsert_validator_approve(args, bigchain):
inputs = [i for i in tx.to_inputs() if key.public_key in i.owners_before]
election_pub_key = ValidatorElection.to_public_key(tx.id)
approval = ValidatorElectionVote.generate(inputs,
[([election_pub_key], voting_power)],
tx.id).sign([key.private_key])
approval = Vote.generate(inputs,
[([election_pub_key], voting_power)],
tx.id).sign([key.private_key])
approval.validate(bigchain)
resp = bigchain.write_transaction(approval, 'broadcast_tx_commit')
@ -190,8 +216,8 @@ def run_upsert_validator_approve(args, bigchain):
return False
def run_upsert_validator_show(args, bigchain):
"""Retrieves information about an upsert-validator election
def run_election_show(args, bigchain):
"""Retrieves information about an election
:param args: dict
args = {
@ -205,14 +231,7 @@ def run_upsert_validator_show(args, bigchain):
logger.error(f'No election found with election_id {args.election_id}')
return
new_validator = election.asset['data']
public_key = public_key_to_base64(new_validator['public_key'])
power = new_validator['power']
node_id = new_validator['node_id']
status = election.get_status(bigchain)
response = f'public_key={public_key}\npower={power}\nnode_id={node_id}\nstatus={status}'
response = election.show_election(bigchain)
logger.info(response)
@ -228,14 +247,7 @@ def _run_init():
@configure_bigchaindb
def run_init(args):
"""Initialize the database"""
# TODO Provide mechanism to:
# 1. prompt the user to inquire whether they wish to drop the db
# 2. force the init, (e.g., via -f flag)
try:
_run_init()
except DatabaseAlreadyExists:
print('The database already exists.', file=sys.stderr)
print('If you wish to re-initialize it, first drop it.', file=sys.stderr)
_run_init()
@configure_bigchaindb
@ -257,16 +269,7 @@ def run_drop(args):
def run_recover(b):
pre_commit = query.get_pre_commit_state(b.connection, PRE_COMMIT_ID)
# Initially the pre-commit collection would be empty
if pre_commit:
latest_block = query.get_latest_block(b.connection)
# NOTE: the pre-commit state can only be ahead of the commited state
# by 1 block
if latest_block and (latest_block['height'] < pre_commit['height']):
query.delete_transactions(b.connection, pre_commit['transactions'])
rollback(b)
@configure_bigchaindb
@ -279,16 +282,22 @@ def run_start(args):
logger.info('BigchainDB Version %s', bigchaindb.__version__)
run_recover(bigchaindb.lib.BigchainDB())
try:
if not args.skip_initialize_database:
logger.info('Initializing database')
_run_init()
except DatabaseAlreadyExists:
pass
if not args.skip_initialize_database:
logger.info('Initializing database')
_run_init()
logger.info('Starting BigchainDB main process.')
from bigchaindb.start import start
start()
start(args)
def run_tendermint_version(args):
"""Show the supported Tendermint version(s)"""
supported_tm_ver = {
'description': 'BigchainDB supports the following Tendermint version(s)',
'tendermint': __tm_supported_versions__,
}
print(json.dumps(supported_tm_ver, indent=4, sort_keys=True))
def create_parser():
@ -315,41 +324,38 @@ def create_parser():
help='The backend to use. It can only be '
'"localmongodb", currently.')
# parser for managing validator elections
validator_parser = subparsers.add_parser('upsert-validator',
help='Add/update/delete a validator.')
# parser for managing elections
election_parser = subparsers.add_parser('election',
help='Manage elections.')
validator_subparser = validator_parser.add_subparsers(title='Action',
dest='action')
election_subparser = election_parser.add_subparsers(title='Action',
dest='action')
new_election_parser = validator_subparser.add_parser('new',
help='Calls a new election.')
new_election_parser = election_subparser.add_parser('new',
help='Calls a new election.')
new_election_parser.add_argument('public_key',
help='Public key of the validator to be added/updated/removed.')
new_election_subparser = new_election_parser.add_subparsers(title='Election_Type',
dest='election_type')
new_election_parser.add_argument('power',
type=int,
help='The proposed power for the validator. '
'Setting to 0 will remove the validator.')
# Parser factory for each type of new election, so we get a bunch of commands that look like this:
# election new <some_election_type> <args>...
for name, data in elections.items():
args = data['args']
generic_parser = new_election_subparser.add_parser(name, help=data['help'])
for arg, kwargs in args.items():
generic_parser.add_argument(arg, **kwargs)
new_election_parser.add_argument('node_id',
help='The node_id of the validator.')
new_election_parser.add_argument('--private-key',
dest='sk',
help='Path to the private key of the election initiator.')
approve_election_parser = validator_subparser.add_parser('approve',
help='Approve the election.')
approve_election_parser = election_subparser.add_parser('approve',
help='Approve the election.')
approve_election_parser.add_argument('election_id',
help='The election_id of the election.')
approve_election_parser.add_argument('--private-key',
dest='sk',
required=True,
help='Path to the private key of the election initiator.')
show_election_parser = validator_subparser.add_parser('show',
help='Provides information about an election.')
show_election_parser = election_subparser.add_parser('show',
help='Provides information about an election.')
show_election_parser.add_argument('election_id',
help='The transaction id of the election you wish to query.')
@ -375,6 +381,15 @@ def create_parser():
action='store_true',
help='Skip database initialization')
subparsers.add_parser('tendermint-version',
help='Show the Tendermint supported versions')
start_parser.add_argument('--experimental-parallel-validation',
dest='experimental_parallel_validation',
default=False,
action='store_true',
help='💀 EXPERIMENTAL: parallelize validation for better throughput 💀')
return parser

31
bigchaindb/commands/election_types.py Normal file
View File

@ -0,0 +1,31 @@
elections = {
'upsert-validator': {
'help': 'Propose a change to the validator set',
'args': {
'public_key': {
'help': 'Public key of the validator to be added/updated/removed.'
},
'power': {
'type': int,
'help': 'The proposed power for the validator. Setting to 0 will remove the validator.'},
'node_id': {
'help': 'The node_id of the validator.'
},
'--private-key': {
'dest': 'sk',
'required': True,
'help': 'Path to the private key of the election initiator.'
}
}
},
'chain-migration': {
'help': 'Call for a halt to block production to allow for a version change across breaking changes.',
'args': {
'--private-key': {
'dest': 'sk',
'required': True,
'help': 'Path to the private key of the election initiator.'
}
}
}
}

8
bigchaindb/common/exceptions.py
View File

@ -11,10 +11,6 @@ class ConfigurationError(BigchainDBError):
"""Raised when there is a problem with server configuration"""
class DatabaseAlreadyExists(BigchainDBError):
"""Raised when trying to create the database but the db is already there"""
class DatabaseDoesNotExist(BigchainDBError):
"""Raised when trying to delete the database but the db is not there"""
@ -112,3 +108,7 @@ class UnequalValidatorSet(ValidationError):
class InvalidPowerChange(ValidationError):
"""Raised if proposed power change in validator set is >=1/3 total power"""
class InvalidPublicKey(ValidationError):
"""Raised if public key doesn't match the encoding type"""

58
bigchaindb/common/memoize.py Normal file
View File

@ -0,0 +1,58 @@
import functools
import codecs
from functools import lru_cache
class HDict(dict):
def __hash__(self):
return hash(codecs.decode(self['id'], 'hex'))
@lru_cache(maxsize=16384)
def from_dict(func, *args, **kwargs):
return func(*args, **kwargs)
def memoize_from_dict(func):
@functools.wraps(func)
def memoized_func(*args, **kwargs):
if args[1].get('id', None):
args = list(args)
args[1] = HDict(args[1])
new_args = tuple(args)
return from_dict(func, *new_args, **kwargs)
else:
return func(*args, **kwargs)
return memoized_func
class ToDictWrapper():
def __init__(self, tx):
self.tx = tx
def __eq__(self, other):
return self.tx.id == other.tx.id
def __hash__(self):
return hash(self.tx.id)
@lru_cache(maxsize=16384)
def to_dict(func, tx_wrapped):
return func(tx_wrapped.tx)
def memoize_to_dict(func):
@functools.wraps(func)
def memoized_func(*args, **kwargs):
if args[0].id:
return to_dict(func, ToDictWrapper(args[0]))
else:
return func(*args, **kwargs)
return memoized_func

6
bigchaindb/common/schema/__init__.py
View File

@ -37,8 +37,10 @@ _, TX_SCHEMA_TRANSFER = _load_schema('transaction_transfer_' +
_, TX_SCHEMA_VALIDATOR_ELECTION = _load_schema('transaction_validator_election_' +
TX_SCHEMA_VERSION)
_, TX_SCHEMA_VALIDATOR_ELECTION_VOTE = _load_schema('transaction_validator_election_vote_' +
TX_SCHEMA_VERSION)
_, TX_SCHEMA_CHAIN_MIGRATION_ELECTION = _load_schema('transaction_chain_migration_election_' +
TX_SCHEMA_VERSION)
_, TX_SCHEMA_VOTE = _load_schema('transaction_vote_' + TX_SCHEMA_VERSION)
def _validate_schema(schema, body):

44
bigchaindb/common/schema/transaction_chain_migration_election_v2.0.yaml Normal file
View File

@ -0,0 +1,44 @@
# Copyright BigchainDB GmbH and BigchainDB contributors
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
---
"$schema": "http://json-schema.org/draft-04/schema#"
type: object
title: Chain Migration Election Schema - Propose a halt in block production to allow for a version change
required:
- operation
- asset
- outputs
properties:
operation:
type: string
value: "CHAIN_MIGRATION_ELECTION"
asset:
additionalProperties: false
properties:
data:
additionalProperties: false
properties:
seed:
type: string
required:
- data
outputs:
type: array
items:
"$ref": "#/definitions/output"
definitions:
output:
type: object
properties:
condition:
type: object
required:
- uri
properties:
uri:
type: string
pattern: "^ni:///sha-256;([a-zA-Z0-9_-]{0,86})[?]\
(fpt=ed25519-sha-256(&)?|cost=[0-9]+(&)?|\
subtypes=ed25519-sha-256(&)?){2,3}$"

3
bigchaindb/common/schema/transaction_v2.0.yaml
View File

@ -63,7 +63,8 @@ definitions:
- CREATE
- TRANSFER
- VALIDATOR_ELECTION
- VALIDATOR_ELECTION_VOTE
- CHAIN_MIGRATION_ELECTION
- VOTE
asset:
type: object
additionalProperties: false

17
bigchaindb/common/schema/transaction_validator_election_v2.0.yaml
View File

@ -22,8 +22,23 @@ properties:
properties:
node_id:
type: string
public_key:
seed:
type: string
public_key:
type: object
additionalProperties: false
required:
- value
- type
properties:
value:
type: string
type:
type: string
enum:
- ed25519-base16
- ed25519-base32
- ed25519-base64
power:
"$ref": "#/definitions/positiveInteger"
required:

4
bigchaindb/common/schema/transaction_validator_election_vote_v2.0.yaml → bigchaindb/common/schema/transaction_vote_v2.0.yaml
View File

@ -5,14 +5,14 @@
---
"$schema": "http://json-schema.org/draft-04/schema#"
type: object
title: Validator Election Vote Schema - Vote on a validator set change
title: Vote Schema - Vote on an election
required:
- operation
- outputs
properties:
operation:
type: string
value: "VALIDATOR_ELECTION_VOTE"
value: "VOTE"
outputs:
type: array
items:

33
bigchaindb/common/transaction.py
View File

@ -12,7 +12,8 @@ Attributes:
"""
from collections import namedtuple
from copy import deepcopy
from functools import reduce
from functools import reduce, lru_cache
import rapidjson
import base58
from cryptoconditions import Fulfillment, ThresholdSha256, Ed25519Sha256
@ -27,6 +28,7 @@ from bigchaindb.common.exceptions import (KeypairMismatchException,
AmountError, AssetIdMismatch,
ThresholdTooDeep)
from bigchaindb.common.utils import serialize
from .memoize import memoize_from_dict, memoize_to_dict
UnspentOutput = namedtuple(
@ -82,6 +84,11 @@ class Input(object):
# TODO: If `other !== Fulfillment` return `False`
return self.to_dict() == other.to_dict()
# NOTE: This function is used to provide a unique key for a given
# Input to suppliment memoization
def __hash__(self):
return hash((self.fulfillment, self.fulfills))
def to_dict(self):
"""Transforms the object to a Python dictionary.
@ -94,7 +101,7 @@ class Input(object):
"""
try:
fulfillment = self.fulfillment.serialize_uri()
except (TypeError, AttributeError, ASN1EncodeError):
except (TypeError, AttributeError, ASN1EncodeError, ASN1DecodeError):
fulfillment = _fulfillment_to_details(self.fulfillment)
try:
@ -161,7 +168,7 @@ def _fulfillment_to_details(fulfillment):
if fulfillment.type_name == 'ed25519-sha-256':
return {
'type': 'ed25519-sha-256',
'public_key': base58.b58encode(fulfillment.public_key),
'public_key': base58.b58encode(fulfillment.public_key).decode(),
}
if fulfillment.type_name == 'threshold-sha-256':
@ -500,7 +507,7 @@ class Transaction(object):
VERSION = '2.0'
def __init__(self, operation, asset, inputs=None, outputs=None,
metadata=None, version=None, hash_id=None):
metadata=None, version=None, hash_id=None, tx_dict=None):
"""The constructor allows to create a customizable Transaction.
Note:
@ -553,6 +560,7 @@ class Transaction(object):
self.outputs = outputs or []
self.metadata = metadata
self._id = hash_id
self.tx_dict = tx_dict
@property
def unspent_outputs(self):
@ -990,7 +998,7 @@ class Transaction(object):
raise ValueError('Inputs and '
'output_condition_uris must have the same count')
tx_dict = self.to_dict()
tx_dict = self.tx_dict if self.tx_dict else self.to_dict()
tx_dict = Transaction._remove_signatures(tx_dict)
tx_dict['id'] = None
tx_serialized = Transaction._to_str(tx_dict)
@ -1003,6 +1011,7 @@ class Transaction(object):
return all(validate(i, cond)
for i, cond in enumerate(output_condition_uris))
@lru_cache(maxsize=16384)
def _input_valid(self, input_, operation, message, output_condition_uri=None):
"""Validates a single Input against a single Output.
@ -1048,6 +1057,11 @@ class Transaction(object):
ffill_valid = parsed_ffill.validate(message=message.digest())
return output_valid and ffill_valid
# This function is required by `lru_cache` to create a key for memoization
def __hash__(self):
return hash(self.id)
@memoize_to_dict
def to_dict(self):
"""Transforms the object to a Python dictionary.
@ -1150,7 +1164,9 @@ class Transaction(object):
tx_body (dict): The Transaction to be transformed.
"""
# NOTE: Remove reference to avoid side effects
tx_body = deepcopy(tx_body)
# tx_body = deepcopy(tx_body)
tx_body = rapidjson.loads(rapidjson.dumps(tx_body))
try:
proposed_tx_id = tx_body['id']
except KeyError:
@ -1167,6 +1183,7 @@ class Transaction(object):
raise InvalidHash(err_msg.format(proposed_tx_id))
@classmethod
@memoize_from_dict
def from_dict(cls, tx, skip_schema_validation=True):
"""Transforms a Python dictionary to a Transaction object.
@ -1178,13 +1195,15 @@ class Transaction(object):
"""
operation = tx.get('operation', Transaction.CREATE) if isinstance(tx, dict) else Transaction.CREATE
cls = Transaction.resolve_class(operation)
if not skip_schema_validation:
cls.validate_id(tx)
cls.validate_schema(tx)
inputs = [Input.from_dict(input_) for input_ in tx['inputs']]
outputs = [Output.from_dict(output) for output in tx['outputs']]
return cls(tx['operation'], tx['asset'], inputs, outputs,
tx['metadata'], tx['version'], hash_id=tx['id'])
tx['metadata'], tx['version'], hash_id=tx['id'], tx_dict=tx)
@classmethod
def from_db(cls, bigchain, tx_dict_list):

32
bigchaindb/common/utils.py
View File

@ -76,10 +76,20 @@ def validate_txn_obj(obj_name, obj, key, validation_fun):
if backend == 'localmongodb':
data = obj.get(key, {})
if isinstance(data, dict):
validate_all_keys(obj_name, data, validation_fun)
validate_all_keys_in_obj(obj_name, data, validation_fun)
elif isinstance(data, list):
validate_all_items_in_list(obj_name, data, validation_fun)
def validate_all_keys(obj_name, obj, validation_fun):
def validate_all_items_in_list(obj_name, data, validation_fun):
for item in data:
if isinstance(item, dict):
validate_all_keys_in_obj(obj_name, item, validation_fun)
elif isinstance(item, list):
validate_all_items_in_list(obj_name, item, validation_fun)
def validate_all_keys_in_obj(obj_name, obj, validation_fun):
"""Validate all (nested) keys in `obj` by using `validation_fun`.
Args:
@ -97,10 +107,12 @@ def validate_all_keys(obj_name, obj, validation_fun):
for key, value in obj.items():
validation_fun(obj_name, key)
if isinstance(value, dict):
validate_all_keys(obj_name, value, validation_fun)
validate_all_keys_in_obj(obj_name, value, validation_fun)
elif isinstance(value, list):
validate_all_items_in_list(obj_name, value, validation_fun)
def validate_all_values_for_key(obj, key, validation_fun):
def validate_all_values_for_key_in_obj(obj, key, validation_fun):
"""Validate value for all (nested) occurrence of `key` in `obj`
using `validation_fun`.
@ -117,7 +129,17 @@ def validate_all_values_for_key(obj, key, validation_fun):
if vkey == key:
validation_fun(value)
elif isinstance(value, dict):
validate_all_values_for_key(value, key, validation_fun)
validate_all_values_for_key_in_obj(value, key, validation_fun)
elif isinstance(value, list):
validate_all_values_for_key_in_list(value, key, validation_fun)
def validate_all_values_for_key_in_list(input_list, key, validation_fun):
for item in input_list:
if isinstance(item, dict):
validate_all_values_for_key_in_obj(item, key, validation_fun)
elif isinstance(item, list):
validate_all_values_for_key_in_list(item, key, validation_fun)
def validate_key(obj_name, key):

22
bigchaindb/config_utils.py
View File

@ -28,7 +28,7 @@ from bigchaindb.common import exceptions
import bigchaindb
from bigchaindb.consensus import BaseConsensusRules
from bigchaindb.validation import BaseValidationRules
# TODO: move this to a proper configuration file for logging
logging.getLogger('requests').setLevel(logging.WARNING)
@ -258,38 +258,38 @@ def autoconfigure(filename=None, config=None, force=False):
@lru_cache()
def load_consensus_plugin(name=None):
"""Find and load the chosen consensus plugin.
def load_validation_plugin(name=None):
"""Find and load the chosen validation plugin.
Args:
name (string): the name of the entry_point, as advertised in the
setup.py of the providing package.
Returns:
an uninstantiated subclass of ``bigchaindb.consensus.AbstractConsensusRules``
an uninstantiated subclass of ``bigchaindb.validation.AbstractValidationRules``
"""
if not name:
return BaseConsensusRules
return BaseValidationRules
# TODO: This will return the first plugin with group `bigchaindb.consensus`
# TODO: This will return the first plugin with group `bigchaindb.validation`
# and name `name` in the active WorkingSet.
# We should probably support Requirements specs in the config, e.g.
# consensus_plugin: 'my-plugin-package==0.0.1;default'
# validation_plugin: 'my-plugin-package==0.0.1;default'
plugin = None
for entry_point in iter_entry_points('bigchaindb.consensus', name):
for entry_point in iter_entry_points('bigchaindb.validation', name):
plugin = entry_point.load()
# No matching entry_point found
if not plugin:
raise ResolutionError(
'No plugin found in group `bigchaindb.consensus` with name `{}`'.
'No plugin found in group `bigchaindb.validation` with name `{}`'.
format(name))
# Is this strictness desireable?
# It will probably reduce developer headaches in the wild.
if not issubclass(plugin, (BaseConsensusRules,)):
if not issubclass(plugin, (BaseValidationRules,)):
raise TypeError('object of type "{}" does not implement `bigchaindb.'
'consensus.BaseConsensusRules`'.format(type(plugin)))
'validation.BaseValidationRules`'.format(type(plugin)))
return plugin

177
bigchaindb/core.py
View File

@ -6,6 +6,7 @@
with Tendermint.
"""
import logging
import sys
from abci.application import BaseApplication
from abci.types_pb2 import (
@ -19,12 +20,14 @@ from abci.types_pb2 import (
)
from bigchaindb import BigchainDB
from bigchaindb.elections.election import Election
from bigchaindb.version import __tm_supported_versions__
from bigchaindb.utils import tendermint_version_is_compatible
from bigchaindb.tendermint_utils import (decode_transaction,
calculate_hash)
from bigchaindb.lib import Block, PreCommitState
from bigchaindb.backend.query import PRE_COMMIT_ID
from bigchaindb.upsert_validator import ValidatorElection
from bigchaindb.lib import Block
import bigchaindb.upsert_validator.validator_utils as vutils
from bigchaindb.events import EventTypes, Event
CodeTypeOk = 0
@ -36,33 +39,95 @@ class App(BaseApplication):
"""Bridge between BigchainDB and Tendermint.
The role of this class is to expose the BigchainDB
transactional logic to the Tendermint Consensus
State Machine.
transaction logic to Tendermint Core.
"""
def __init__(self, bigchaindb=None):
def __init__(self, bigchaindb=None, events_queue=None):
self.events_queue = events_queue
self.bigchaindb = bigchaindb or BigchainDB()
self.block_txn_ids = []
self.block_txn_hash = ''
self.block_transactions = []
self.validators = None
self.new_height = None
self.chain = self.bigchaindb.get_latest_abci_chain()
def log_abci_migration_error(self, chain_id, validators):
logger.error(f'An ABCI chain migration is in process. ' +
'Download the new ABCI client and configure it with ' +
'chain_id={chain_id} and validators={validators}.')
def abort_if_abci_chain_is_not_synced(self):
if self.chain is None or self.chain['is_synced']:
return
validators = self.bigchaindb.get_validators()
self.log_abci_migration_error(self.chain['chain_id'], validators)
sys.exit(1)
def init_chain(self, genesis):
"""Initialize chain with block of height 0"""
"""Initialize chain upon genesis or a migration"""
validator_set = [vutils.decode_validator(v) for v in genesis.validators]
block = Block(app_hash='', height=0, transactions=[])
app_hash = ''
height = 0
known_chain = self.bigchaindb.get_latest_abci_chain()
if known_chain is not None:
chain_id = known_chain['chain_id']
if known_chain['is_synced']:
msg = f'Got invalid InitChain ABCI request ({genesis}) - ' + \
'the chain {chain_id} is already synced.'
logger.error(msg)
sys.exit(1)
if chain_id != genesis.chain_id:
validators = self.bigchaindb.get_validators()
self.log_abci_migration_error(chain_id, validators)
sys.exit(1)
# set migration values for app hash and height
block = self.bigchaindb.get_latest_block()
app_hash = '' if block is None else block['app_hash']
height = 0 if block is None else block['height'] + 1
known_validators = self.bigchaindb.get_validators()
validator_set = [vutils.decode_validator(v)
for v in genesis.validators]
if known_validators and known_validators != validator_set:
self.log_abci_migration_error(known_chain['chain_id'],
known_validators)
sys.exit(1)
block = Block(app_hash=app_hash, height=height, transactions=[])
self.bigchaindb.store_block(block._asdict())
self.bigchaindb.store_validator_set(1, validator_set, None)
self.bigchaindb.store_validator_set(height + 1, validator_set)
abci_chain_height = 0 if known_chain is None else known_chain['height']
self.bigchaindb.store_abci_chain(abci_chain_height,
genesis.chain_id, True)
self.chain = {'height': abci_chain_height, 'is_synced': True,
'chain_id': genesis.chain_id}
return ResponseInitChain()
def info(self, request):
"""Return height of the latest committed block."""
self.abort_if_abci_chain_is_not_synced()
# Check if BigchainDB supports the Tendermint version
if not (hasattr(request, 'version') and tendermint_version_is_compatible(request.version)):
logger.error(f'Unsupported Tendermint version: {getattr(request, "version", "no version")}.'
f' Currently, BigchainDB only supports {__tm_supported_versions__}. Exiting!')
sys.exit(1)
logger.info(f"Tendermint version: {request.version}")
r = ResponseInfo()
block = self.bigchaindb.get_latest_block()
if block:
r.last_block_height = block['height']
chain_shift = 0 if self.chain is None else self.chain['height']
r.last_block_height = block['height'] - chain_shift
r.last_block_app_hash = block['app_hash'].encode('utf-8')
else:
r.last_block_height = 0
@ -77,16 +142,15 @@ class App(BaseApplication):
raw_tx: a raw string (in bytes) transaction.
"""
logger.benchmark('CHECK_TX_INIT')
self.abort_if_abci_chain_is_not_synced()
logger.debug('check_tx: %s', raw_transaction)
transaction = decode_transaction(raw_transaction)
if self.bigchaindb.is_valid_transaction(transaction):
logger.debug('check_tx: VALID')
logger.benchmark('CHECK_TX_END, tx_id:%s', transaction['id'])
return ResponseCheckTx(code=CodeTypeOk)
else:
logger.debug('check_tx: INVALID')
logger.benchmark('CHECK_TX_END, tx_id:%s', transaction['id'])
return ResponseCheckTx(code=CodeTypeError)
def begin_block(self, req_begin_block):
@ -95,9 +159,12 @@ class App(BaseApplication):
req_begin_block: block object which contains block header
and block hash.
"""
logger.benchmark('BEGIN BLOCK, height:%s, num_txs:%s',
req_begin_block.header.height,
req_begin_block.header.num_txs)
self.abort_if_abci_chain_is_not_synced()
chain_shift = 0 if self.chain is None else self.chain['height']
logger.debug('BEGIN BLOCK, height:%s, num_txs:%s',
req_begin_block.header.height + chain_shift,
req_begin_block.header.num_txs)
self.block_txn_ids = []
self.block_transactions = []
@ -109,6 +176,9 @@ class App(BaseApplication):
Args:
raw_tx: a raw string (in bytes) transaction.
"""
self.abort_if_abci_chain_is_not_synced()
logger.debug('deliver_tx: %s', raw_transaction)
transaction = self.bigchaindb.is_valid_transaction(
decode_transaction(raw_transaction), self.block_transactions)
@ -130,8 +200,20 @@ class App(BaseApplication):
height (int): new height of the chain.
"""
height = request_end_block.height
self.abort_if_abci_chain_is_not_synced()
chain_shift = 0 if self.chain is None else self.chain['height']
height = request_end_block.height + chain_shift
self.new_height = height
# store pre-commit state to recover in case there is a crash during
# `end_block` or `commit`
logger.debug(f'Updating pre-commit state: {self.new_height}')
pre_commit_state = dict(height=self.new_height,
transactions=self.block_txn_ids)
self.bigchaindb.store_pre_commit_state(pre_commit_state)
block_txn_hash = calculate_hash(self.block_txn_ids)
block = self.bigchaindb.get_latest_block()
@ -140,38 +222,57 @@ class App(BaseApplication):
else:
self.block_txn_hash = block['app_hash']
# Check if the current block concluded any validator elections and
# update the locally tracked validator set
validator_updates = ValidatorElection.get_validator_update(self.bigchaindb,
self.new_height,
self.block_transactions)
validator_update = Election.process_block(self.bigchaindb,
self.new_height,
self.block_transactions)
# Store pre-commit state to recover in case there is a crash
# during `commit`
pre_commit_state = PreCommitState(commit_id=PRE_COMMIT_ID,
height=self.new_height,
transactions=self.block_txn_ids)
logger.debug('Updating PreCommitState: %s', self.new_height)
self.bigchaindb.store_pre_commit_state(pre_commit_state._asdict())
return ResponseEndBlock(validator_updates=validator_updates)
return ResponseEndBlock(validator_updates=validator_update)
def commit(self):
"""Store the new height and along with block hash."""
self.abort_if_abci_chain_is_not_synced()
data = self.block_txn_hash.encode('utf-8')
# register a new block only when new transactions are received
if self.block_txn_ids:
self.bigchaindb.store_bulk_transactions(self.block_transactions)
block = Block(app_hash=self.block_txn_hash,
height=self.new_height,
transactions=self.block_txn_ids)
# NOTE: storing the block should be the last operation during commit
# this effects crash recovery. Refer BEP#8 for details
self.bigchaindb.store_block(block._asdict())
block = Block(app_hash=self.block_txn_hash,
height=self.new_height,
transactions=self.block_txn_ids)
# NOTE: storing the block should be the last operation during commit
# this effects crash recovery. Refer BEP#8 for details
self.bigchaindb.store_block(block._asdict())
logger.debug('Commit-ing new block with hash: apphash=%s ,'
'height=%s, txn ids=%s', data, self.new_height,
self.block_txn_ids)
logger.benchmark('COMMIT_BLOCK, height:%s', self.new_height)
if self.events_queue:
event = Event(EventTypes.BLOCK_VALID, {
'height': self.new_height,
'transactions': self.block_transactions
})
self.events_queue.put(event)
return ResponseCommit(data=data)
def rollback(b):
pre_commit = b.get_pre_commit_state()
if pre_commit is None:
# the pre_commit record is first stored in the first `end_block`
return
latest_block = b.get_latest_block()
if latest_block is None:
logger.error('Found precommit state but no blocks!')
sys.exit(1)
# NOTE: the pre-commit state is always at most 1 block ahead of the commited state
if latest_block['height'] < pre_commit['height']:
Election.rollback(b, pre_commit['height'], pre_commit['transactions'])
b.delete_transactions(pre_commit['transactions'])

0
bigchaindb/elections/__init__.py Normal file
View File

354
bigchaindb/elections/election.py Normal file
View File

@ -0,0 +1,354 @@
# Copyright BigchainDB GmbH and BigchainDB contributors
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
from collections import OrderedDict
import base58
from uuid import uuid4
from bigchaindb import backend
from bigchaindb.elections.vote import Vote
from bigchaindb.common.exceptions import (InvalidSignature,
MultipleInputsError,
InvalidProposer,
UnequalValidatorSet,
DuplicateTransaction)
from bigchaindb.tendermint_utils import key_from_base64, public_key_to_base64
from bigchaindb.common.crypto import (public_key_from_ed25519_key)
from bigchaindb.common.transaction import Transaction
from bigchaindb.common.schema import (_validate_schema,
TX_SCHEMA_COMMON,
TX_SCHEMA_CREATE)
class Election(Transaction):
"""Represents election transactions.
To implement a custom election, create a class deriving from this one
with OPERATION set to the election operation, ALLOWED_OPERATIONS
set to (OPERATION,), CREATE set to OPERATION.
"""
OPERATION = None
# Custom validation schema
TX_SCHEMA_CUSTOM = None
# Election Statuses:
ONGOING = 'ongoing'
CONCLUDED = 'concluded'
INCONCLUSIVE = 'inconclusive'
# Vote ratio to approve an election
ELECTION_THRESHOLD = 2 / 3
@classmethod
def get_validator_change(cls, bigchain):
"""Return the validator set from the most recent approved block
:return: {
'height': <block_height>,
'validators': <validator_set>
}
"""
latest_block = bigchain.get_latest_block()
if latest_block is None:
return None
return bigchain.get_validator_change(latest_block['height'])
@classmethod
def get_validators(cls, bigchain, height=None):
"""Return a dictionary of validators with key as `public_key` and
value as the `voting_power`
"""
validators = {}
for validator in bigchain.get_validators(height):
# NOTE: we assume that Tendermint encodes public key in base64
public_key = public_key_from_ed25519_key(key_from_base64(validator['public_key']['value']))
validators[public_key] = validator['voting_power']
return validators
@classmethod
def recipients(cls, bigchain):
"""Convert validator dictionary to a recipient list for `Transaction`"""
recipients = []
for public_key, voting_power in cls.get_validators(bigchain).items():
recipients.append(([public_key], voting_power))
return recipients
@classmethod
def is_same_topology(cls, current_topology, election_topology):
voters = {}
for voter in election_topology:
if len(voter.public_keys) > 1:
return False
[public_key] = voter.public_keys
voting_power = voter.amount
voters[public_key] = voting_power
# Check whether the voters and their votes is same to that of the
# validators and their voting power in the network
return current_topology == voters
def validate(self, bigchain, current_transactions=[]):
"""Validate election transaction
NOTE:
* A valid election is initiated by an existing validator.
* A valid election is one where voters are validators and votes are
allocated according to the voting power of each validator node.
Args:
:param bigchain: (BigchainDB) an instantiated bigchaindb.lib.BigchainDB object.
:param current_transactions: (list) A list of transactions to be validated along with the election
Returns:
Election: a Election object or an object of the derived Election subclass.
Raises:
ValidationError: If the election is invalid
"""
input_conditions = []
duplicates = any(txn for txn in current_transactions if txn.id == self.id)
if bigchain.is_committed(self.id) or duplicates:
raise DuplicateTransaction('transaction `{}` already exists'
.format(self.id))
if not self.inputs_valid(input_conditions):
raise InvalidSignature('Transaction signature is invalid.')
current_validators = self.get_validators(bigchain)
# NOTE: Proposer should be a single node
if len(self.inputs) != 1 or len(self.inputs[0].owners_before) != 1:
raise MultipleInputsError('`tx_signers` must be a list instance of length one')
# NOTE: Check if the proposer is a validator.
[election_initiator_node_pub_key] = self.inputs[0].owners_before
if election_initiator_node_pub_key not in current_validators.keys():
raise InvalidProposer('Public key is not a part of the validator set')
# NOTE: Check if all validators have been assigned votes equal to their voting power
if not self.is_same_topology(current_validators, self.outputs):
raise UnequalValidatorSet('Validator set much be exactly same to the outputs of election')
return self
@classmethod
def generate(cls, initiator, voters, election_data, metadata=None):
# Break symmetry in case we need to call an election with the same properties twice
uuid = uuid4()
election_data['seed'] = str(uuid)
(inputs, outputs) = cls.validate_create(initiator, voters, election_data, metadata)
election = cls(cls.OPERATION, {'data': election_data}, inputs, outputs, metadata)
cls.validate_schema(election.to_dict())
return election
@classmethod
def validate_schema(cls, tx):
"""Validate the election transaction. Since `ELECTION` extends `CREATE` transaction, all the validations for
`CREATE` transaction should be inherited
"""
_validate_schema(TX_SCHEMA_COMMON, tx)
_validate_schema(TX_SCHEMA_CREATE, tx)
if cls.TX_SCHEMA_CUSTOM:
_validate_schema(cls.TX_SCHEMA_CUSTOM, tx)
@classmethod
def create(cls, tx_signers, recipients, metadata=None, asset=None):
raise NotImplementedError
@classmethod
def transfer(cls, tx_signers, recipients, metadata=None, asset=None):
raise NotImplementedError
@classmethod
def to_public_key(cls, election_id):
return base58.b58encode(bytes.fromhex(election_id)).decode()
@classmethod
def count_votes(cls, election_pk, transactions, getter=getattr):
votes = 0
for txn in transactions:
if getter(txn, 'operation') == Vote.OPERATION:
for output in getter(txn, 'outputs'):
# NOTE: We enforce that a valid vote to election id will have only
# election_pk in the output public keys, including any other public key
# along with election_pk will lead to vote being not considered valid.
if len(getter(output, 'public_keys')) == 1 and [election_pk] == getter(output, 'public_keys'):
votes = votes + int(getter(output, 'amount'))
return votes
def get_commited_votes(self, bigchain, election_pk=None):
if election_pk is None:
election_pk = self.to_public_key(self.id)
txns = list(backend.query.get_asset_tokens_for_public_key(bigchain.connection,
self.id,
election_pk))
return self.count_votes(election_pk, txns, dict.get)
def has_concluded(self, bigchain, current_votes=[]):
"""Check if the election can be concluded or not.
* Elections can only be concluded if the validator set has not changed
since the election was initiated.
* Elections can be concluded only if the current votes form a supermajority.
Custom elections may override this function and introduce additional checks.
"""
if self.has_validator_set_changed(bigchain):
return False
election_pk = self.to_public_key(self.id)
votes_committed = self.get_commited_votes(bigchain, election_pk)
votes_current = self.count_votes(election_pk, current_votes)
total_votes = sum(output.amount for output in self.outputs)
if (votes_committed < (2/3) * total_votes) and \
(votes_committed + votes_current >= (2/3)*total_votes):
return True
return False
def get_status(self, bigchain):
election = self.get_election(self.id, bigchain)
if election and election['is_concluded']:
return self.CONCLUDED
return self.INCONCLUSIVE if self.has_validator_set_changed(bigchain) else self.ONGOING
def has_validator_set_changed(self, bigchain):
latest_change = self.get_validator_change(bigchain)
if latest_change is None:
return False
latest_change_height = latest_change['height']
election = self.get_election(self.id, bigchain)
return latest_change_height > election['height']
def get_election(self, election_id, bigchain):
return bigchain.get_election(election_id)
def store(self, bigchain, height, is_concluded):
bigchain.store_election(self.id, height, is_concluded)
def show_election(self, bigchain):
data = self.asset['data']
if 'public_key' in data.keys():
data['public_key'] = public_key_to_base64(data['public_key']['value'])
response = ''
for k, v in data.items():
if k != 'seed':
response += f'{k}={v}\n'
response += f'status={self.get_status(bigchain)}'
return response
@classmethod
def _get_initiated_elections(cls, height, txns):
elections = []
for tx in txns:
if not isinstance(tx, Election):
continue
elections.append({'election_id': tx.id, 'height': height,
'is_concluded': False})
return elections
@classmethod
def _get_votes(cls, txns):
elections = OrderedDict()
for tx in txns:
if not isinstance(tx, Vote):
continue
election_id = tx.asset['id']
if election_id not in elections:
elections[election_id] = []
elections[election_id].append(tx)
return elections
@classmethod
def process_block(cls, bigchain, new_height, txns):
"""Looks for election and vote transactions inside the block, records
and processes elections.
Every election is recorded in the database.
Every vote has a chance to conclude the corresponding election. When
an election is concluded, the corresponding database record is
marked as such.
Elections and votes are processed in the order in which they
appear in the block. Elections are concluded in the order of
appearance of their first votes in the block.
For every election concluded in the block, calls its `on_approval`
method. The returned value of the last `on_approval`, if any,
is a validator set update to be applied in one of the following blocks.
`on_approval` methods are implemented by elections of particular type.
The method may contain side effects but should be idempotent. To account
for other concluded elections, if it requires so, the method should
rely on the database state.
"""
# elections initiated in this block
initiated_elections = cls._get_initiated_elections(new_height, txns)
if initiated_elections:
bigchain.store_elections(initiated_elections)
# elections voted for in this block and their votes
elections = cls._get_votes(txns)
validator_update = None
for election_id, votes in elections.items():
election = bigchain.get_transaction(election_id)
if election is None:
continue
if not election.has_concluded(bigchain, votes):
continue
validator_update = election.on_approval(bigchain, new_height)
election.store(bigchain, new_height, is_concluded=True)
return [validator_update] if validator_update else []
@classmethod
def rollback(cls, bigchain, new_height, txn_ids):
"""Looks for election and vote transactions inside the block and
cleans up the database artifacts possibly created in `process_blocks`.
Part of the `end_block`/`commit` crash recovery.
"""
# delete election records for elections initiated at this height and
# elections concluded at this height
bigchain.delete_elections(new_height)
txns = [bigchain.get_transaction(tx_id) for tx_id in txn_ids]
elections = cls._get_votes(txns)
for election_id in elections:
election = bigchain.get_transaction(election_id)
election.on_rollback(bigchain, new_height)
def on_approval(self, bigchain, new_height):
"""Override to update the database state according to the
election rules. Consider the current database state to account for
other concluded elections, if required.
"""
raise NotImplementedError
def on_rollback(self, bigchain, new_height):
"""Override to clean up the database artifacts possibly created
in `on_approval`. Part of the `end_block`/`commit` crash recovery.
"""
raise NotImplementedError

26
bigchaindb/upsert_validator/validator_election_vote.py → bigchaindb/elections/vote.py
View File

@ -6,16 +6,18 @@ from bigchaindb.common.transaction import Transaction
from bigchaindb.common.schema import (_validate_schema,
TX_SCHEMA_COMMON,
TX_SCHEMA_TRANSFER,
TX_SCHEMA_VALIDATOR_ELECTION_VOTE)
TX_SCHEMA_VOTE)
class ValidatorElectionVote(Transaction):
class Vote(Transaction):
VALIDATOR_ELECTION_VOTE = 'VALIDATOR_ELECTION_VOTE'
OPERATION = 'VOTE'
# NOTE: This class inherits TRANSFER txn type. The `TRANSFER` property is
# overriden to re-use methods from parent class
TRANSFER = VALIDATOR_ELECTION_VOTE
ALLOWED_OPERATIONS = (VALIDATOR_ELECTION_VOTE,)
TRANSFER = OPERATION
ALLOWED_OPERATIONS = (OPERATION,)
# Custom validation schema
TX_SCHEMA_CUSTOM = TX_SCHEMA_VOTE
def validate(self, bigchain, current_transactions=[]):
"""Validate election vote transaction
@ -28,7 +30,7 @@ class ValidatorElectionVote(Transaction):
bigchain (BigchainDB): an instantiated bigchaindb.lib.BigchainDB object.
Returns:
ValidatorElectionVote object
Vote: a Vote object
Raises:
ValidationError: If the election vote is invalid
@ -39,20 +41,18 @@ class ValidatorElectionVote(Transaction):
@classmethod
def generate(cls, inputs, recipients, election_id, metadata=None):
(inputs, outputs) = cls.validate_transfer(inputs, recipients, election_id, metadata)
election_vote = cls(cls.VALIDATOR_ELECTION_VOTE, {'id': election_id}, inputs, outputs, metadata)
cls.validate_schema(election_vote.to_dict(), skip_id=True)
election_vote = cls(cls.OPERATION, {'id': election_id}, inputs, outputs, metadata)
cls.validate_schema(election_vote.to_dict())
return election_vote
@classmethod
def validate_schema(cls, tx, skip_id=False):
"""Validate the validator election vote transaction. Since `VALIDATOR_ELECTION_VOTE` extends `TRANFER`
def validate_schema(cls, tx):
"""Validate the validator election vote transaction. Since `VOTE` extends `TRANSFER`
transaction, all the validations for `CREATE` transaction should be inherited
"""
if not skip_id:
cls.validate_id(tx)
_validate_schema(TX_SCHEMA_COMMON, tx)
_validate_schema(TX_SCHEMA_TRANSFER, tx)
_validate_schema(TX_SCHEMA_VALIDATOR_ELECTION_VOTE, tx)
_validate_schema(cls.TX_SCHEMA_CUSTOM, tx)
@classmethod
def create(cls, tx_signers, recipients, metadata=None, asset=None):

89
bigchaindb/event_stream.py
View File

@ -1,89 +0,0 @@
# Copyright BigchainDB GmbH and BigchainDB contributors
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
import asyncio
import json
import logging
import time
import aiohttp
from bigchaindb import config
from bigchaindb.common.utils import gen_timestamp
from bigchaindb.events import EventTypes, Event
from bigchaindb.tendermint_utils import decode_transaction_base64
HOST = config['tendermint']['host']
PORT = config['tendermint']['port']
URL = 'ws://{}:{}/websocket'.format(HOST, PORT)
logger = logging.getLogger(__name__)
@asyncio.coroutine
def connect_and_recv(event_queue):
session = aiohttp.ClientSession()
ws = yield from session.ws_connect(URL)
logger.info('Connected to tendermint ws server')
stream_id = 'bigchaindb_stream_{}'.format(gen_timestamp())
yield from subscribe_events(ws, stream_id)
while True:
msg = yield from ws.receive()
process_event(event_queue, msg.data, stream_id)
if msg.type in (aiohttp.WSMsgType.CLOSED,
aiohttp.WSMsgType.ERROR):
session.close()
raise aiohttp.ClientConnectionError()
def process_event(event_queue, event, stream_id):
event_stream_id = stream_id + '#event'
event = json.loads(event)
if (event['id'] == event_stream_id and event['result']['query'] == 'tm.event=\'NewBlock\''):
block = event['result']['data']['value']['block']
block_id = block['header']['height']
block_txs = block['data']['txs']
# Only push non empty blocks
if block_txs:
block_txs = [decode_transaction_base64(txn) for txn in block_txs]
new_block = {'height': block_id, 'transactions': block_txs}
event = Event(EventTypes.BLOCK_VALID, new_block)
event_queue.put(event)
@asyncio.coroutine
def subscribe_events(ws, stream_id):
payload = {
'method': 'subscribe',
'jsonrpc': '2.0',
'params': ['tm.event=\'NewBlock\''],
'id': stream_id
}
yield from ws.send_str(json.dumps(payload))
@asyncio.coroutine
def try_connect_and_recv(event_queue):
try:
yield from connect_and_recv(event_queue)
except Exception as e:
logger.warning('WebSocket connection failed with exception %s', e)
time.sleep(3)
yield from try_connect_and_recv(event_queue)
def start(event_queue):
loop = asyncio.get_event_loop()
try:
loop.run_until_complete(try_connect_and_recv(event_queue))
except (KeyboardInterrupt, SystemExit):
logger.info('Shutting down Tendermint event stream connection')

143
bigchaindb/lib.py
View File

@ -9,6 +9,7 @@ MongoDB.
import logging
from collections import namedtuple
from uuid import uuid4
import rapidjson
try:
from hashlib import sha3_256
@ -26,7 +27,7 @@ from bigchaindb.common.exceptions import (SchemaValidationError,
DoubleSpend)
from bigchaindb.tendermint_utils import encode_transaction, merkleroot
from bigchaindb import exceptions as core_exceptions
from bigchaindb.consensus import BaseConsensusRules
from bigchaindb.validation import BaseValidationRules
logger = logging.getLogger(__name__)
@ -55,19 +56,20 @@ class BigchainDB(object):
A connection to the database.
"""
config_utils.autoconfigure()
self.mode_commit = 'broadcast_tx_commit'
self.mode_list = ('broadcast_tx_async',
'broadcast_tx_sync',
'broadcast_tx_commit')
self.mode_commit)
self.tendermint_host = bigchaindb.config['tendermint']['host']
self.tendermint_port = bigchaindb.config['tendermint']['port']
self.endpoint = 'http://{}:{}/'.format(self.tendermint_host, self.tendermint_port)
consensusPlugin = bigchaindb.config.get('consensus_plugin')
validationPlugin = bigchaindb.config.get('validation_plugin')
if consensusPlugin:
self.consensus = config_utils.load_consensus_plugin(consensusPlugin)
if validationPlugin:
self.validation = config_utils.load_validation_plugin(validationPlugin)
else:
self.consensus = BaseConsensusRules
self.validation = BaseValidationRules
self.connection = connection if connection else backend.connect(**bigchaindb.config['database'])
@ -77,10 +79,11 @@ class BigchainDB(object):
raise ValidationError('Mode must be one of the following {}.'
.format(', '.join(self.mode_list)))
tx_dict = transaction.tx_dict if transaction.tx_dict else transaction.to_dict()
payload = {
'method': mode,
'jsonrpc': '2.0',
'params': [encode_transaction(transaction.to_dict())],
'params': [encode_transaction(tx_dict)],
'id': str(uuid4())
}
# TODO: handle connection errors!
@ -94,38 +97,38 @@ class BigchainDB(object):
def _process_post_response(self, response, mode):
logger.debug(response)
if response.get('error') is not None:
return (500, 'Internal error')
error = response.get('error')
if error:
status_code = 500
message = error.get('message', 'Internal Error')
data = error.get('data', '')
if 'Tx already exists in cache' in data:
status_code = 400
return (status_code, message + ' - ' + data)
result = response['result']
if mode == self.mode_commit:
check_tx_code = result.get('check_tx', {}).get('code', 0)
deliver_tx_code = result.get('deliver_tx', {}).get('code', 0)
error_code = check_tx_code or deliver_tx_code
else:
error_code = result.get('code', 0)
if error_code:
return (500, 'Transaction validation failed')
return (202, '')
# result = response['result']
# if mode == self.mode_list[2]:
# return self._process_commit_mode_response(result)
# else:
# status_code = result['code']
# return self._process_status_code(status_code,
# 'Error while processing transaction')
# def _process_commit_mode_response(self, result):
# check_tx_status_code = result['check_tx']['code']
# if check_tx_status_code == 0:
# deliver_tx_status_code = result['deliver_tx']['code']
# return self._process_status_code(deliver_tx_status_code,
# 'Error while commiting the transaction')
# else:
# return (500, 'Error while validating the transaction')
def process_status_code(self, status_code, failure_msg):
return (202, '') if status_code == 0 else (500, failure_msg)
def store_bulk_transactions(self, transactions):
txns = []
assets = []
txn_metadatas = []
for transaction_obj in transactions:
# self.update_utxoset(transaction)
transaction = transaction_obj.to_dict()
if transaction['operation'] == transaction_obj.CREATE:
for t in transactions:
transaction = t.tx_dict if t.tx_dict else rapidjson.loads(rapidjson.dumps(t.to_dict()))
if transaction['operation'] == t.CREATE:
asset = transaction.pop('asset')
asset['id'] = transaction['id']
assets.append(asset)
@ -140,6 +143,9 @@ class BigchainDB(object):
backend.query.store_assets(self.connection, assets)
return backend.query.store_transactions(self.connection, txns)
def delete_transactions(self, txs):
return backend.query.delete_transactions(self.connection, txs)
def update_utxoset(self, transaction):
"""Update the UTXO set given ``transaction``. That is, remove
the outputs that the given ``transaction`` spends, and add the
@ -224,6 +230,10 @@ class BigchainDB(object):
return backend.query.delete_unspent_outputs(
self.connection, *unspent_outputs)
def is_committed(self, transaction_id):
transaction = backend.query.get_transaction(self.connection, transaction_id)
return bool(transaction)
def get_transaction(self, transaction_id):
transaction = backend.query.get_transaction(self.connection, transaction_id)
@ -244,6 +254,9 @@ class BigchainDB(object):
return transaction
def get_transactions(self, txn_ids):
return backend.query.get_transactions(self.connection, txn_ids)
def get_transactions_filtered(self, asset_id, operation=None):
"""Get a list of transactions filtered on some criteria
"""
@ -426,29 +439,71 @@ class BigchainDB(object):
def get_validators(self, height=None):
result = self.get_validator_change(height)
validators = result['validators']
return validators
return [] if result is None else result['validators']
def get_validators_by_election_id(self, election_id):
result = backend.query.get_validator_set_by_election_id(self.connection, election_id)
return result
def get_election(self, election_id):
return backend.query.get_election(self.connection, election_id)
def delete_validator_update(self):
return backend.query.delete_validator_update(self.connection)
def get_pre_commit_state(self):
return backend.query.get_pre_commit_state(self.connection)
def store_pre_commit_state(self, state):
return backend.query.store_pre_commit_state(self.connection, state)
def store_validator_set(self, height, validators, election_id):
def store_validator_set(self, height, validators):
"""Store validator set at a given `height`.
NOTE: If the validator set already exists at that `height` then an
exception will be raised.
"""
return backend.query.store_validator_set(self.connection, {'height': height,
'validators': validators,
'election_id': election_id})
'validators': validators})
def delete_validator_set(self, height):
return backend.query.delete_validator_set(self.connection, height)
def store_abci_chain(self, height, chain_id, is_synced=True):
return backend.query.store_abci_chain(self.connection, height,
chain_id, is_synced)
def delete_abci_chain(self, height):
return backend.query.delete_abci_chain(self.connection, height)
def get_latest_abci_chain(self):
return backend.query.get_latest_abci_chain(self.connection)
def migrate_abci_chain(self):
"""Generate and record a new ABCI chain ID. New blocks are not
accepted until we receive an InitChain ABCI request with
the matching chain ID and validator set.
Chain ID is generated based on the current chain and height.
`chain-X` => `chain-X-migrated-at-height-5`.
`chain-X-migrated-at-height-5` => `chain-X-migrated-at-height-21`.
If there is no known chain (we are at genesis), the function returns.
"""
latest_chain = self.get_latest_abci_chain()
if latest_chain is None:
return
block = self.get_latest_block()
suffix = '-migrated-at-height-'
chain_id = latest_chain['chain_id']
block_height_str = str(block['height'])
new_chain_id = chain_id.split(suffix)[0] + suffix + block_height_str
self.store_abci_chain(block['height'] + 1, new_chain_id, False)
def store_election(self, election_id, height, is_concluded):
return backend.query.store_election(self.connection, election_id,
height, is_concluded)
def store_elections(self, elections):
return backend.query.store_elections(self.connection, elections)
def delete_elections(self, height):
return backend.query.delete_elections(self.connection, height)
Block = namedtuple('Block', ('app_hash', 'height', 'transactions'))
PreCommitState = namedtuple('PreCommitState', ('commit_id', 'height', 'transactions'))

30
bigchaindb/log.py
View File

@ -11,8 +11,6 @@ import os
DEFAULT_LOG_DIR = os.getcwd()
BENCHMARK_LOG_LEVEL = 15
DEFAULT_LOGGING_CONFIG = {
'version': 1,
@ -29,11 +27,6 @@ DEFAULT_LOGGING_CONFIG = {
'format': ('[%(asctime)s] [%(levelname)s] (%(name)s) '
'%(message)s (%(processName)-10s - pid: %(process)d)'),
'datefmt': '%Y-%m-%d %H:%M:%S',
},
'benchmark': {
'class': 'logging.Formatter',
'format': ('%(asctime)s, %(levelname)s, %(message)s'),
'datefmt': '%Y-%m-%d %H:%M:%S',
}
},
'handlers': {
@ -59,31 +52,16 @@ DEFAULT_LOGGING_CONFIG = {
'backupCount': 5,
'formatter': 'file',
'level': logging.ERROR,
},
'benchmark': {
'class': 'logging.handlers.RotatingFileHandler',
'filename': os.path.join(DEFAULT_LOG_DIR, 'bigchaindb-benchmark.log'),
'mode': 'w',
'maxBytes': 209715200,
'backupCount': 5,
'formatter': 'benchmark',
'level': BENCHMARK_LOG_LEVEL,
}
},
'loggers': {},
'root': {
'level': logging.DEBUG,
'handlers': ['console', 'file', 'errors', 'benchmark'],
'handlers': ['console', 'file', 'errors'],
},
}
def benchmark(self, message, *args, **kws):
# Yes, logger takes its '*args' as 'args'.
if self.isEnabledFor(BENCHMARK_LOG_LEVEL):
self._log(BENCHMARK_LOG_LEVEL, message, args, **kws)
def _normalize_log_level(level):
try:
return level.upper()
@ -104,11 +82,6 @@ def setup_logging():
"""
# Add a new logging level for logging benchmark
logging.addLevelName(BENCHMARK_LOG_LEVEL, 'BENCHMARK')
logging.BENCHMARK = BENCHMARK_LOG_LEVEL
logging.Logger.benchmark = benchmark
logging_configs = DEFAULT_LOGGING_CONFIG
new_logging_configs = bigchaindb.config['log']
@ -127,7 +100,6 @@ def setup_logging():
if 'level_logfile' in new_logging_configs:
level = _normalize_log_level(new_logging_configs['level_logfile'])
logging_configs['handlers']['file']['level'] = level
logging_configs['handlers']['benchmark']['level'] = level
if 'fmt_console' in new_logging_configs:
fmt = new_logging_configs['fmt_console']

0
bigchaindb/migrations/__init__.py Normal file
View File

48
bigchaindb/migrations/chain_migration_election.py Normal file
View File

@ -0,0 +1,48 @@
import json
from bigchaindb.common.schema import TX_SCHEMA_CHAIN_MIGRATION_ELECTION
from bigchaindb.elections.election import Election
class ChainMigrationElection(Election):
OPERATION = 'CHAIN_MIGRATION_ELECTION'
CREATE = OPERATION
ALLOWED_OPERATIONS = (OPERATION,)
TX_SCHEMA_CUSTOM = TX_SCHEMA_CHAIN_MIGRATION_ELECTION
def has_concluded(self, bigchaindb, *args, **kwargs):
chain = bigchaindb.get_latest_abci_chain()
if chain is not None and not chain['is_synced']:
# do not conclude the migration election if
# there is another migration in progress
return False
return super().has_concluded(bigchaindb, *args, **kwargs)
def on_approval(self, bigchain, *args, **kwargs):
bigchain.migrate_abci_chain()
def show_election(self, bigchain):
output = super().show_election(bigchain)
chain = bigchain.get_latest_abci_chain()
if chain is None or chain['is_synced']:
return output
output += f'\nchain_id={chain["chain_id"]}'
block = bigchain.get_latest_block()
output += f'\napp_hash={block["app_hash"]}'
validators = [
{
'pub_key': {
'type': 'tendermint/PubKeyEd25519',
'value': k,
},
'power': v,
} for k, v in self.get_validators(bigchain).items()
]
output += f'\nvalidators={json.dumps(validators, indent=4)}'
return output
def on_rollback(self, bigchain, new_height):
bigchain.delete_abci_chain(new_height)

4
bigchaindb/models.py
View File

@ -27,7 +27,7 @@ class Transaction(Transaction):
if self.operation == Transaction.CREATE:
duplicates = any(txn for txn in current_transactions if txn.id == self.id)
if bigchain.get_transaction(self.to_dict()['id']) or duplicates:
if bigchain.is_committed(self.id) or duplicates:
raise DuplicateTransaction('transaction `{}` already exists'
.format(self.id))
@ -45,11 +45,11 @@ class Transaction(Transaction):
@classmethod
def validate_schema(cls, tx_body):
cls.validate_id(tx_body)
validate_transaction_schema(tx_body)
validate_txn_obj('asset', tx_body['asset'], 'data', validate_key)
validate_txn_obj('metadata', tx_body, 'metadata', validate_key)
validate_language_key(tx_body['asset'], 'data')
validate_language_key(tx_body, 'metadata')
class FastTransaction:

127
bigchaindb/parallel_validation.py Normal file
View File

@ -0,0 +1,127 @@
# Copyright BigchainDB GmbH and BigchainDB contributors
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
import multiprocessing as mp
from collections import defaultdict
from abci.types_pb2 import ResponseCheckTx, ResponseDeliverTx
from bigchaindb import BigchainDB, App
from bigchaindb.tendermint_utils import decode_transaction
CodeTypeOk = 0
class ParallelValidationApp(App):
def __init__(self, bigchaindb=None, events_queue=None):
super().__init__(bigchaindb, events_queue)
self.parallel_validator = ParallelValidator()
self.parallel_validator.start()
def check_tx(self, raw_transaction):
return ResponseCheckTx(code=CodeTypeOk)
def deliver_tx(self, raw_transaction):
self.parallel_validator.validate(raw_transaction)
return ResponseDeliverTx(code=CodeTypeOk)
def end_block(self, request_end_block):
result = self.parallel_validator.result(timeout=30)
for transaction in result:
if transaction:
self.block_txn_ids.append(transaction.id)
self.block_transactions.append(transaction)
return super().end_block(request_end_block)
RESET = 'reset'
EXIT = 'exit'
class ParallelValidator:
def __init__(self, number_of_workers=mp.cpu_count()):
self.number_of_workers = number_of_workers
self.transaction_index = 0
self.routing_queues = [mp.Queue() for _ in range(self.number_of_workers)]
self.workers = []
self.results_queue = mp.Queue()
def start(self):
for routing_queue in self.routing_queues:
worker = ValidationWorker(routing_queue, self.results_queue)
process = mp.Process(target=worker.run)
process.start()
self.workers.append(process)
def stop(self):
for routing_queue in self.routing_queues:
routing_queue.put(EXIT)
def validate(self, raw_transaction):
dict_transaction = decode_transaction(raw_transaction)
index = int(dict_transaction['id'], 16) % self.number_of_workers
self.routing_queues[index].put((self.transaction_index, dict_transaction))
self.transaction_index += 1
def result(self, timeout=None):
result_buffer = [None] * self.transaction_index
for _ in range(self.transaction_index):
index, transaction = self.results_queue.get(timeout=timeout)
result_buffer[index] = transaction
self.transaction_index = 0
for routing_queue in self.routing_queues:
routing_queue.put(RESET)
return result_buffer
class ValidationWorker:
"""Run validation logic in a loop. This Worker is suitable for a Process
life: no thrills, just a queue to get some values, and a queue to return results.
Note that a worker is expected to validate multiple transactions in
multiple rounds, and it needs to keep in memory all transactions already
validated, until a new round starts. To trigger a new round of validation,
a ValidationWorker expects a `RESET` message. To exit the infinite loop the
worker is in, it expects an `EXIT` message.
"""
def __init__(self, in_queue, results_queue):
self.in_queue = in_queue
self.results_queue = results_queue
self.bigchaindb = BigchainDB()
self.reset()
def reset(self):
# We need a place to store already validated transactions,
# in case of dependant transactions in the same block.
# `validated_transactions` maps an `asset_id` with the list
# of all other transactions sharing the same asset.
self.validated_transactions = defaultdict(list)
def validate(self, dict_transaction):
try:
asset_id = dict_transaction['asset']['id']
except KeyError:
asset_id = dict_transaction['id']
transaction = self.bigchaindb.is_valid_transaction(
dict_transaction,
self.validated_transactions[asset_id])
if transaction:
self.validated_transactions[asset_id].append(transaction)
return transaction
def run(self):
while True:
message = self.in_queue.get()
if message == RESET:
self.reset()
elif message == EXIT:
return
else:
index, transaction = message
self.results_queue.put((index, self.validate(transaction)))

17
bigchaindb/start.py
View File

@ -8,8 +8,8 @@ import setproctitle
import bigchaindb
from bigchaindb.lib import BigchainDB
from bigchaindb.core import App
from bigchaindb.parallel_validation import ParallelValidationApp
from bigchaindb.web import server, websocket_server
from bigchaindb import event_stream
from bigchaindb.events import Exchange, EventTypes
from bigchaindb.utils import Process
@ -35,7 +35,7 @@ BANNER = """
"""
def start():
def start(args):
# Exchange object for event stream api
logger.info('Starting BigchainDB')
exchange = Exchange()
@ -47,7 +47,6 @@ def start():
p_webapi = Process(name='bigchaindb_webapi', target=app_server.run, daemon=True)
p_webapi.start()
# start message
logger.info(BANNER.format(bigchaindb.config['server']['bind']))
# start websocket server
@ -57,13 +56,6 @@ def start():
args=(exchange.get_subscriber_queue(EventTypes.BLOCK_VALID),))
p_websocket_server.start()
# connect to tendermint event stream
p_websocket_client = Process(name='bigchaindb_ws_to_tendermint',
target=event_stream.start,
daemon=True,
args=(exchange.get_publisher_queue(),))
p_websocket_client.start()
p_exchange = Process(name='bigchaindb_exchange', target=exchange.run, daemon=True)
p_exchange.start()
@ -75,7 +67,10 @@ def start():
setproctitle.setproctitle('bigchaindb')
# Start the ABCIServer
app = ABCIServer(app=App())
if args.experimental_parallel_validation:
app = ABCIServer(app=ParallelValidationApp(events_queue=exchange.get_publisher_queue()))
else:
app = ABCIServer(app=App(events_queue=exchange.get_publisher_queue()))
app.run()

1
bigchaindb/upsert_validator/__init__.py
View File

@ -3,5 +3,4 @@
# Code is Apache-2.0 and docs are CC-BY-4.0
from bigchaindb.upsert_validator.validator_election_vote import ValidatorElectionVote # noqa
from bigchaindb.upsert_validator.validator_election import ValidatorElection # noqa

271
bigchaindb/upsert_validator/validator_election.py
View File

@ -2,263 +2,66 @@
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
import base58
from bigchaindb import backend
from bigchaindb.common.exceptions import (InvalidSignature,
MultipleInputsError,
InvalidProposer,
UnequalValidatorSet,
InvalidPowerChange,
DuplicateTransaction)
from bigchaindb.tendermint_utils import key_from_base64
from bigchaindb.common.crypto import (public_key_from_ed25519_key)
from bigchaindb.common.transaction import Transaction
from bigchaindb.common.schema import (_validate_schema,
TX_SCHEMA_VALIDATOR_ELECTION,
TX_SCHEMA_COMMON,
TX_SCHEMA_CREATE)
from . import ValidatorElectionVote
from .validator_utils import (new_validator_set, encode_validator)
from bigchaindb.common.exceptions import InvalidPowerChange
from bigchaindb.elections.election import Election
from bigchaindb.common.schema import TX_SCHEMA_VALIDATOR_ELECTION
from .validator_utils import (new_validator_set, encode_validator, validate_asset_public_key)
class ValidatorElection(Transaction):
class ValidatorElection(Election):
VALIDATOR_ELECTION = 'VALIDATOR_ELECTION'
OPERATION = 'VALIDATOR_ELECTION'
# NOTE: this transaction class extends create so the operation inheritence is achieved
# by renaming CREATE to VALIDATOR_ELECTION
CREATE = VALIDATOR_ELECTION
ALLOWED_OPERATIONS = (VALIDATOR_ELECTION,)
# Election Statuses:
ONGOING = 'ongoing'
CONCLUDED = 'concluded'
INCONCLUSIVE = 'inconclusive'
ELECTION_THRESHOLD = 2 / 3
def __init__(self, operation, asset, inputs, outputs,
metadata=None, version=None, hash_id=None):
# operation `CREATE` is being passed as argument as `VALIDATOR_ELECTION` is an extension
# of `CREATE` and any validation on `CREATE` in the parent class should apply to it
super().__init__(operation, asset, inputs, outputs, metadata, version, hash_id)
@classmethod
def get_validator_change(cls, bigchain, height=None):
"""Return the latest change to the validator set
:return: {
'height': <block_height>,
'asset': {
'height': <block_height>,
'validators': <validator_set>,
'election_id': <election_id_that_approved_the_change>
}
}
"""
return bigchain.get_validator_change(height)
@classmethod
def get_validators(cls, bigchain, height=None):
"""Return a dictionary of validators with key as `public_key` and
value as the `voting_power`
"""
validators = {}
for validator in bigchain.get_validators(height):
# NOTE: we assume that Tendermint encodes public key in base64
public_key = public_key_from_ed25519_key(key_from_base64(validator['pub_key']['data']))
validators[public_key] = validator['voting_power']
return validators
@classmethod
def recipients(cls, bigchain):
"""Convert validator dictionary to a recipient list for `Transaction`"""
recipients = []
for public_key, voting_power in cls.get_validators(bigchain).items():
recipients.append(([public_key], voting_power))
return recipients
@classmethod
def is_same_topology(cls, current_topology, election_topology):
voters = {}
for voter in election_topology:
if len(voter.public_keys) > 1:
return False
[public_key] = voter.public_keys
voting_power = voter.amount
voters[public_key] = voting_power
# Check whether the voters and their votes is same to that of the
# validators and their voting power in the network
return (current_topology == voters)
CREATE = OPERATION
ALLOWED_OPERATIONS = (OPERATION,)
TX_SCHEMA_CUSTOM = TX_SCHEMA_VALIDATOR_ELECTION
def validate(self, bigchain, current_transactions=[]):
"""Validate election transaction
For more details refer BEP-21: https://github.com/bigchaindb/BEPs/tree/master/21
NOTE:
* A valid election is initiated by an existing validator.
* A valid election is one where voters are validators and votes are
alloacted according to the voting power of each validator node.
Args:
bigchain (BigchainDB): an instantiated bigchaindb.lib.BigchainDB object.
Returns:
ValidatorElection object
Raises:
ValidationError: If the election is invalid
"""For more details refer BEP-21: https://github.com/bigchaindb/BEPs/tree/master/21
"""
input_conditions = []
duplicates = any(txn for txn in current_transactions if txn.id == self.id)
if bigchain.get_transaction(self.id) or duplicates:
raise DuplicateTransaction('transaction `{}` already exists'
.format(self.id))
if not self.inputs_valid(input_conditions):
raise InvalidSignature('Transaction signature is invalid.')
current_validators = self.get_validators(bigchain)
# NOTE: Proposer should be a single node
if len(self.inputs) != 1 or len(self.inputs[0].owners_before) != 1:
raise MultipleInputsError('`tx_signers` must be a list instance of length one')
super(ValidatorElection, self).validate(bigchain, current_transactions=current_transactions)
# NOTE: change more than 1/3 of the current power is not allowed
if self.asset['data']['power'] >= (1/3)*sum(current_validators.values()):
raise InvalidPowerChange('`power` change must be less than 1/3 of total power')
# NOTE: Check if the proposer is a validator.
[election_initiator_node_pub_key] = self.inputs[0].owners_before
if election_initiator_node_pub_key not in current_validators.keys():
raise InvalidProposer('Public key is not a part of the validator set')
# NOTE: Check if all validators have been assigned votes equal to their voting power
if not self.is_same_topology(current_validators, self.outputs):
raise UnequalValidatorSet('Validator set much be exactly same to the outputs of election')
return self
@classmethod
def generate(cls, initiator, voters, election_data, metadata=None):
(inputs, outputs) = cls.validate_create(initiator, voters, election_data, metadata)
election = cls(cls.VALIDATOR_ELECTION, {'data': election_data}, inputs, outputs, metadata)
cls.validate_schema(election.to_dict(), skip_id=True)
return election
def validate_schema(cls, tx):
super(ValidatorElection, cls).validate_schema(tx)
validate_asset_public_key(tx['asset']['data']['public_key'])
@classmethod
def validate_schema(cls, tx, skip_id=False):
"""Validate the validator election transaction. Since `VALIDATOR_ELECTION` extends `CREATE`
transaction, all the validations for `CREATE` transaction should be inherited
"""
if not skip_id:
cls.validate_id(tx)
_validate_schema(TX_SCHEMA_COMMON, tx)
_validate_schema(TX_SCHEMA_CREATE, tx)
_validate_schema(TX_SCHEMA_VALIDATOR_ELECTION, tx)
def has_concluded(self, bigchain, *args, **kwargs):
latest_block = bigchain.get_latest_block()
if latest_block is not None:
latest_block_height = latest_block['height']
latest_validator_change = bigchain.get_validator_change()['height']
@classmethod
def create(cls, tx_signers, recipients, metadata=None, asset=None):
raise NotImplementedError
# TODO change to `latest_block_height + 3` when upgrading to Tendermint 0.24.0.
if latest_validator_change == latest_block_height + 2:
# do not conclude the election if there is a change assigned already
return False
@classmethod
def transfer(cls, tx_signers, recipients, metadata=None, asset=None):
raise NotImplementedError
return super().has_concluded(bigchain, *args, **kwargs)
@classmethod
def to_public_key(cls, election_id):
return base58.b58encode(bytes.fromhex(election_id))
def on_approval(self, bigchain, new_height):
validator_updates = [self.asset['data']]
curr_validator_set = bigchain.get_validators(new_height)
updated_validator_set = new_validator_set(curr_validator_set,
validator_updates)
@classmethod
def count_votes(cls, election_pk, transactions, getter=getattr):
votes = 0
for txn in transactions:
if getter(txn, 'operation') == 'VALIDATOR_ELECTION_VOTE':
for output in getter(txn, 'outputs'):
# NOTE: We enforce that a valid vote to election id will have only
# election_pk in the output public keys, including any other public key
# along with election_pk will lead to vote being not considered valid.
if len(getter(output, 'public_keys')) == 1 and [election_pk] == getter(output, 'public_keys'):
votes = votes + int(getter(output, 'amount'))
return votes
updated_validator_set = [v for v in updated_validator_set
if v['voting_power'] > 0]
def get_commited_votes(self, bigchain, election_pk=None):
if election_pk is None:
election_pk = self.to_public_key(self.id)
txns = list(backend.query.get_asset_tokens_for_public_key(bigchain.connection,
self.id,
election_pk))
return self.count_votes(election_pk, txns, dict.get)
# TODO change to `new_height + 2` when upgrading to Tendermint 0.24.0.
bigchain.store_validator_set(new_height + 1, updated_validator_set)
return encode_validator(self.asset['data'])
@classmethod
def has_concluded(cls, bigchain, election_id, current_votes=[], height=None):
"""Check if the given `election_id` can be concluded or not
NOTE:
* Election is concluded iff the current validator set is exactly equal
to the validator set encoded in election outputs
* Election can concluded only if the current votes achieves a supermajority
"""
election = bigchain.get_transaction(election_id)
if election:
election_pk = election.to_public_key(election.id)
votes_commited = election.get_commited_votes(bigchain, election_pk)
votes_current = election.count_votes(election_pk, current_votes)
current_validators = election.get_validators(bigchain, height)
if election.is_same_topology(current_validators, election.outputs):
total_votes = sum(current_validators.values())
if (votes_commited < (2/3)*total_votes) and \
(votes_commited + votes_current >= (2/3)*total_votes):
return election
return False
@classmethod
def get_validator_update(cls, bigchain, new_height, txns):
votes = {}
for txn in txns:
if not isinstance(txn, ValidatorElectionVote):
continue
election_id = txn.asset['id']
election_votes = votes.get(election_id, [])
election_votes.append(txn)
votes[election_id] = election_votes
election = cls.has_concluded(bigchain, election_id, election_votes, new_height)
# Once an election concludes any other conclusion for the same
# or any other election is invalidated
if election:
# The new validator set comes into effect from height = new_height+1
validator_updates = [election.asset['data']]
curr_validator_set = bigchain.get_validators(new_height)
updated_validator_set = new_validator_set(curr_validator_set,
validator_updates)
updated_validator_set = [v for v in updated_validator_set if v['voting_power'] > 0]
bigchain.store_validator_set(new_height+1, updated_validator_set, election.id)
return [encode_validator(election.asset['data'])]
return []
def get_validator_update_by_election_id(self, election_id, bigchain):
result = bigchain.get_validators_by_election_id(election_id)
return result
def get_status(self, bigchain):
concluded = self.get_validator_update_by_election_id(self.id, bigchain)
if concluded:
return self.CONCLUDED
latest_change = self.get_validator_change(bigchain)
latest_change_height = latest_change['height']
election_height = bigchain.get_block_containing_tx(self.id)[0]
if latest_change_height >= election_height:
return self.INCONCLUSIVE
else:
return self.ONGOING
def on_rollback(self, bigchaindb, new_height):
# TODO change to `new_height + 2` when upgrading to Tendermint 0.24.0.
bigchaindb.delete_validator_set(new_height + 1)

56
bigchaindb/upsert_validator/validator_utils.py
View File

@ -1,12 +1,14 @@
import codecs
import base64
import binascii
from abci.types_pb2 import (Validator,
PubKey)
from bigchaindb.tendermint_utils import public_key_to_base64
from bigchaindb.common.exceptions import InvalidPublicKey
def encode_validator(v):
ed25519_public_key = v['public_key']
ed25519_public_key = v['public_key']['value']
# NOTE: tendermint expects public to be encoded in go-amino format
pub_key = PubKey(type='ed25519',
data=bytes.fromhex(ed25519_public_key))
@ -16,22 +18,60 @@ def encode_validator(v):
def decode_validator(v):
return {'pub_key': {'type': v.pub_key.type,
'data': codecs.encode(v.pub_key.data, 'base64').decode().rstrip('\n')},
return {'public_key': {'type': 'ed25519-base64',
'value': codecs.encode(v.pub_key.data, 'base64').decode().rstrip('\n')},
'voting_power': v.power}
def new_validator_set(validators, updates):
validators_dict = {}
for v in validators:
validators_dict[v['pub_key']['data']] = v
validators_dict[v['public_key']['value']] = v
updates_dict = {}
for u in updates:
public_key64 = public_key_to_base64(u['public_key'])
updates_dict[public_key64] = {'pub_key': {'type': 'ed25519',
'data': public_key64},
decoder = get_public_key_decoder(u['public_key'])
public_key64 = base64.b64encode(decoder(u['public_key']['value'])).decode('utf-8')
updates_dict[public_key64] = {'public_key': {'type': 'ed25519-base64',
'value': public_key64},
'voting_power': u['power']}
new_validators_dict = {**validators_dict, **updates_dict}
return list(new_validators_dict.values())
def encode_pk_to_base16(validator):
pk = validator['public_key']
decoder = get_public_key_decoder(pk)
public_key16 = base64.b16encode(decoder(pk['value'])).decode('utf-8')
validator['public_key']['value'] = public_key16
return validator
def validate_asset_public_key(pk):
pk_binary = pk['value'].encode('utf-8')
decoder = get_public_key_decoder(pk)
try:
pk_decoded = decoder(pk_binary)
if len(pk_decoded) != 32:
raise InvalidPublicKey('Public key should be of size 32 bytes')
except binascii.Error:
raise InvalidPublicKey('Invalid `type` specified for public key `value`')
def get_public_key_decoder(pk):
encoding = pk['type']
decoder = base64.b64decode
if encoding == 'ed25519-base16':
decoder = base64.b16decode
elif encoding == 'ed25519-base32':
decoder = base64.b32decode
elif encoding == 'ed25519-base64':
decoder = base64.b64decode
else:
raise InvalidPublicKey('Invalid `type` specified for public key `value`')
return decoder

23
bigchaindb/utils.py
View File

@ -9,7 +9,8 @@ import multiprocessing as mp
import json
import setproctitle
from packaging import version
from bigchaindb.version import __tm_supported_versions__
from bigchaindb.tendermint_utils import key_from_base64
from bigchaindb.common.crypto import key_pair_from_ed25519_key
@ -185,3 +186,23 @@ def load_node_key(path):
priv_key = priv_validator['priv_key']['value']
hex_private_key = key_from_base64(priv_key)
return key_pair_from_ed25519_key(hex_private_key)
def tendermint_version_is_compatible(running_tm_ver):
"""
Check Tendermint compatability with BigchainDB server
:param running_tm_ver: Version number of the connected Tendermint instance
:type running_tm_ver: str
:return: True/False depending on the compatability with BigchainDB server
:rtype: bool
"""
# Splitting because version can look like this e.g. 0.22.8-40d6dc2e
tm_ver = running_tm_ver.split('-')
if not tm_ver:
return False
for ver in __tm_supported_versions__:
if version.parse(ver) == version.parse(tm_ver[0]):
return True
return False

15
bigchaindb/consensus.py → bigchaindb/validation.py
View File

@ -3,23 +3,22 @@
# Code is Apache-2.0 and docs are CC-BY-4.0
class BaseConsensusRules():
"""Base consensus rules for Bigchain.
class BaseValidationRules():
"""Base validation rules for BigchainDB.
A consensus plugin must expose a class inheriting from this one via an entry_point.
A validation plugin must expose a class inheriting from this one via an entry_point.
All methods listed below must be implemented.
"""
@staticmethod
def validate_transaction(bigchain, transaction):
def validate_transaction(bigchaindb, transaction):
"""See :meth:`bigchaindb.models.Transaction.validate`
for documentation.
"""
return transaction.validate(bigchain)
return transaction.validate(bigchaindb)
@staticmethod
def validate_block(bigchain, block):
def validate_block(bigchaindb, block):
"""See :meth:`bigchaindb.models.Block.validate` for documentation."""
return block.validate(bigchain)
return block.validate(bigchaindb)

7
bigchaindb/version.py
View File

@ -2,5 +2,8 @@
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
__version__ = '2.0.0b5'
__short_version__ = '2.0b5'
__version__ = '2.0.0b9'
__short_version__ = '2.0b9'
# Supported Tendermint versions
__tm_supported_versions__ = ["0.22.8"]

1
bigchaindb/web/views/info.py
View File

@ -47,6 +47,7 @@ def get_api_v1_info(api_prefix):
return {
'docs': ''.join(docs_url),
'transactions': '{}transactions/'.format(api_prefix),
'blocks': '{}blocks/'.format(api_prefix),
'assets': '{}assets/'.format(api_prefix),
'outputs': '{}outputs/'.format(api_prefix),
'streams': websocket_root,

12
bigchaindb/web/websocket_server.py
View File

@ -50,13 +50,13 @@ def _multiprocessing_to_asyncio(in_queue, out_queue, loop):
def eventify_block(block):
for tx in block['transactions']:
try:
asset_id = tx['asset']['id']
except KeyError:
asset_id = tx['id']
if tx.asset:
asset_id = tx.asset.get('id', tx.id)
else:
asset_id = tx.id
yield {'height': block['height'],
'asset_id': asset_id,
'transaction_id': tx['id']}
'transaction_id': tx.id}
class Dispatcher:
@ -134,7 +134,7 @@ def websocket_handler(request):
except RuntimeError as e:
logger.debug('Websocket exception: %s', str(e))
break
except CancelledError as e:
except CancelledError:
logger.debug('Websocket closed')
break
if msg.type == aiohttp.WSMsgType.CLOSED:

3
docs/contributing/requirements.txt
View File

@ -1,4 +1,5 @@
Sphinx>=1.4.8
Sphinx~=1.0
recommonmark>=0.4.0
sphinx-rtd-theme>=0.1.9
wget
packaging~=18.0

2
docs/contributing/source/dev-setup-coding-and-contribution-process/run-node-as-processes.md
View File

@ -25,7 +25,7 @@ After the installation of MongoDB is complete, run MongoDB using `sudo mongod`
### Installing a Tendermint Executable
Find [the version number of the latest Tendermint release](https://github.com/tendermint/tendermint/releases) and install it using the following, where 0.22.8 should be replaced by the latest released version number:
The version of BigchainDB Server described in these docs only works well with Tendermint 0.22.8 (not a higher version number). Install that:
```bash
$ sudo apt install -y unzip

3
docs/root/requirements.txt
View File

@ -1,5 +1,6 @@
Sphinx>=1.4.8
Sphinx~=1.0
recommonmark>=0.4.0
sphinx-rtd-theme>=0.1.9
sphinxcontrib-napoleon>=0.4.4
sphinxcontrib-httpdomain>=1.5.0
packaging~=18.0

4
docs/root/source/decentralized.md
View File

@ -12,10 +12,6 @@ Ideally, each node in a BigchainDB network is owned and controlled by a differen
We use the phrase "BigchainDB consortium" (or just "consortium") to refer to the set of people and/or organizations who run the nodes of a BigchainDB network. A consortium requires some form of governance to make decisions such as membership and policies. The exact details of the governance process are determined by each consortium, but it can be very decentralized.
If sharding is turned on (i.e. if the number of shards is larger than one), then the actual data is decentralized in that no one node stores all the data.
Every node has its own locally-stored list of the public keys of other consortium members: the so-called keyring. There's no centrally-stored or centrally-shared keyring.
A consortium can increase its decentralization (and its resilience) by increasing its jurisdictional diversity, geographic diversity, and other kinds of diversity. This idea is expanded upon in [the section on node diversity](diversity.html).
There’s no node that has a long-term special position in the BigchainDB network. All nodes run the same software and perform the same duties.

1
docs/root/source/index.rst
View File

@ -98,3 +98,4 @@ More About BigchainDB
permissions
private-data
Data Models <https://docs.bigchaindb.com/projects/server/en/latest/data-models/index.html>
korean/index

BIN
docs/root/source/korean/_static/CREATE_and_TRANSFER_example.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.1 KiB

BIN
docs/root/source/korean/_static/CREATE_example.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.8 KiB

BIN
docs/root/source/korean/_static/schemaDB.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 166 KiB

21
docs/root/source/korean/assets_ko.md Normal file
View File

@ -0,0 +1,21 @@
<!-- Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0 -->
BigchainDB가 자산 등록 및 전송에 적합한 방법
==========================================================
BigchainDB는 모든 종류의 데이터를 저장할 수 있지만 자산 등록 및 전송을 저장하는 데 특히 유용합니다.:
* BigchainDB 네트워크에 전송되어 체크되고 저장되는 (있는 경우) 트랜잭션은 기본적으로 CREATE 트랜잭션과 TRANSFER 트랜잭션의 두 가지가 있습니다.
* CREATE 트랜잭션은 임의의 메타 데이터와 함께 모든 종류의 자산 (나눌 수 없거나 분할 할 수없는)을 등록하는 데 사용할 수 있습니다.
* 저작물에는 0 명, 1 명 또는 여러 명의 소유자가있을 수 있습니다.
* 자산 소유자는 자산을 신규 소유자에게 양도하려는 사람이 만족해야하는 조건을 지정할 수 있습니다. 예를 들어 5 명의 현재 소유자 중 최소 3 명이 TRANSFER 트랜잭션에 암호를 사용해야합니다.
* BigchainDB는 TRANSFER 트랜잭션의 유효성을 검사하는 과정에서 조건이 충족되었는지 확인합니다. (또한 누구나 만족하는지 확인할 수 있습니다.)
* BigchainDB는 자산의 이중 지출을 방지합니다.
* 유효성이 검증 된 트랜잭션은 [변경불가능](https://github.com/bigchaindb/bigchaindb/blob/master/docs/root/source/korean/immutable-ko.md) 입니다.
Note
우리는 "소유자"라는 단어를 다소 느슨하게 사용했습니다. **보다 정확한 단어**는 이행자, 서명자, 조정자 또는 이전 가능 요소 일 수 있습니다. 관련 [BigchainDB Transaction Spec](https://github.com/bigchaindb/BEPs/tree/master/tx-specs/)의 Owners에 대한 참고 사항 절을 참조하십시오.

12
docs/root/source/korean/bft-ko.md Normal file
View File

@ -0,0 +1,12 @@
<!---
Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# BigchainDB와 Byzantine Fault Tolerance
[BigchainDB Server](https://docs.bigchaindb.com/projects/server/en/latest/index.html)
는 블록체인 합의와 트랜잭션 복제에 [Tendermint](https://tendermint.com/)를 사용합니다.
그리고 Tendermint 는 [Byzantine Fault Tolerant (BFT)](https://en.wikipedia.org/wiki/Byzantine_fault_tolerance).

23
docs/root/source/korean/decentralized_kor.md Normal file
View File

@ -0,0 +1,23 @@
<!---
Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# BigchainDB 분산 방식
분산이란 모든 것을 소유하거나 통제하는 사람이 없고, 단 하나의 실패 지점이 없다는 것을 의미합니다.
이상적으로, BigchainDB 네트워크에서 각각의 노드는 다른 개인 또는 조직에 의해 소유되고 관리됩니다. 네트워크가 한 조직 내에 상주하고 있더라도, 각 노드를 다른 사용자나 부서로 제어하는 것이 좋습니다.
우리는 "BigchainDB 컨소시엄" (또는 단지 "컨소시엄")은 BigchainDB 네트워크의 노드를 구동하는 사람들 혹은 조직을 의미합니다. 컨소시엄은 회원제나 정책과 같은 결정을 내리기 위한 어떠한 형태의 거버넌스 요구합니다. 거버넌스 프로세스의 정확한 세부사항은 각 컨소시엄에 의해 결정되지만, 상당히 분산될 수 있습니다.
컨소시엄은 관할의 다양성과 지리적 다양성 및 기타 종류의 다양성을 증가시킴으로써 분산화(및 탄력성)를 증가시킬 수 있습니다. 이 아이디어는 [노드 다양성 부문](Dvolio.html)에서 확장됩니다.
BigchainDB 네트워크에는 오래된 특정한 위치를 가지는 노드가 없습니다. 모든 노드들은 동일한 소프트웨어를 실행하고 동일한 작업을 수행합니다.
만약 노드에 대한 관리자 접근 권한이 있는 경우, 해당 노드를 제거할 수 있지만(예: 해당 노드에 저장된 데이터 변경 또는 삭제), 이러한 변경 사항은 해당 노드에 분리된 상태로 유지되어야 합니다. BigchainDB 네트워크는 노드의 3분의 1 이상이 손상된 경우에만 손상될 수 있습니다. 자세한 내용은 [Tendermint 문서](https://tendermint.com/docs/introduction/introduction.html)을 참조하십시오.
노드의 관리자나 슈퍼 유저도 자산을 전송할 수 없다는 점에 유의하십시오. 유효한 전송 트랜잭션을 생성하는 유일한 방법은 자산에 대한 현재 암호화 조건을 충족하는 것이며 관리자/슈퍼사용자는 필요한 정보를 가지고 있지 않기 때문에 이 작업을 수행할 수 없습니다(예: 개인 키).
노드의 관리자나 슈퍼 사용자도 자산을 전송할 수는 없다는 점을 유의하십시오. 타당한 전송 트랜잭션을 만드는 유일한 방법은 자산에 대한 현재 암호화 조건을 충족시키는 것이며, 관리자 또는 슈퍼 사용자는 필요한 정보를 가지고 있지 않기 때문에 이 작업을 수행할 수 없습니다. (예: 개인 키)

17
docs/root/source/korean/diversity-ko.md Normal file
View File

@ -0,0 +1,17 @@
<!---
Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# 노드 다양성의 종류
한 명의 유저나 이벤트가 노드의 "상당수" 부분을 제어하거나 손상시키는 것을 어렵게 만드는 절차가 수행되어야 합니다.(BigchainDB 서버는 Tendermint를 사용하기 때문에 여기서 "상당수"는 노드의 1/3을 말합니다.) 아래에 쓰여진 여러 가지의 다양성을 고려해야 합니다. 모든 종류에 있어서 높은 다양성을 갖는 것은 꽤 어려운 일입니다.
1. **관할권 다양성.** 노드는 여러 합법적 관할권 내의 참여자들이 제어합니다. 이는 어떤 일을 수행하기에 이 수단들을 사용하기 어렵게 할 수 있습니다.
1. **지리적 다양성.** 서버는 지리적으로 여러 곳에 물리적으로 위치합니다. 이는 자연 재해(홍수 또는 지진 등)가 문제를 야기할 만큼 손상되기 어렵게 합니다.
1. **호스팅 다양성.** 서버는 여러 호스팅 공급자(ex. Amazon Web Services, Microsoft Azure, Digital Oceure, Rackspace)가 호스팅해야 합니다. 이는 하나의 호스팅 공급자가 '상당 수'의 노드에 영향을 미치기가 어려워집니다.
1. **일반적인 의미의 다양성.** 일반적으로 멤버십 다양성은 컨소시엄에 많은 이점을 줍니다. 예를 들어, 그것은 문제 해결에 필요한 다양한 아이디어 소스를 컨소시엄에 제공합니다.
참고: 모든 노드가 동일한 코드(ex. BigchainDB의 동일한 구현)를 실행하고 있는 경우, 해당 코드의 버그를 사용하여 모든 노드를 손상시킬 수 있습니다. 이상적으로는 BigchainDB 서버(예: Python, Go 등)에서 운영되고 있는 다양한 구현이 있어, 컨소시엄은 다양한 서버 구현을 할 수 있을 것입니다. 운영 체제에 대해서도 이는 유사하게 적용됩니다.

26
docs/root/source/korean/immutable-ko.md Normal file
View File

@ -0,0 +1,26 @@
<!---
Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# 어떻게 BigchainDB는 불변성을 갖는가
*Imunable*이라는 단어는 "시간 경과에 따른 불변성"을 의미합니다. 예를 들어, π의 10진수 값은 변경할 수 없습니다(3.14159...).
블록체인 커뮤니티는 종종 블록체인을 "불변하다"고 설명합니다. 우리가 그 단어를 문자 그대로 해석한다면, 그것은 블록체인 데이터가 변경할 수 없거나 영구적이라는 것을 의미하는데, 이것은 말이 안됩니다. 데이터는 *변경 될 수 있습니다.* 예를 들어, 전염병이 인류를 멸종 시킬 수도 있는 것처럼 데이터는 수분에 의한 손상, 온도에 의한 손상, 엔트로피의 일반적인 증가로 인해 시간이 지남에 따라 손상될 수 있습니다.
블록체인 데이터가 일반적인 경우보다 변경(혹은 삭제)하기가 더 어려운 것은 사실입니다. 블록체인 데이터는 단순히 (의도적인) "변형 방지"에 그치지 않고 하드 드라이브의 데이터 손상과 같은 비의도적으로 발생할 수 있는 무작위 변경에도 대응합니다. 따라서 블록체인에서 "불변한다"라는 단어를 우리는 어떤 모든 의도와 목적이 *실제적으로* 불변한 것으로 해석합니다. (언어학자들은 "불변한다"라는 단어가 블록체인 커뮤니티에서 쓰이는 *기술적 용어*라고 말할 것입니다.)
블록체인 데이터는 여러 가지 방법을 통해 불변성을 가질 수 있습니다:
1. **데이터 변경 또는 삭제를 위한 API 없음.** Blockchain 소프트웨어는 일반적으로 블록체인에 저장된 데이터를 변경하거나 삭제하기 위한 API를 제공하지 않습니다. BigchainDB 역시 관련한 API를 제공하지 않습니다 . 이것은 변경이나 삭제가 *다른 방식*으로 일어나는 것을 막지 못합니다. 이것은 단지 하나의 방어선일 뿐입니다.
1. **복제.** 모든 데이터는 여러 곳에 복제(복사)됩니다. 복제 팩터가 높을수록, 모든 복제본을 변경하거나 삭제하기가 더 어려워집니다.
1. **내부 감시 장치.** 모든 노드가 모든 변경 사항을 모니터링하고 허용되지 않은 변경 사항이 발생하면 적절한 조치를 취할 수 있습니다.
1. **외부 감시 장치.** 컨소시엄은 부정행위를 찾아 데이터를 모니터링하고 감사할 수 있는 검증된 제 3자를 선택할 수 있습니다. 공개적으로 읽을 수 있는 데이터를 가진 컨소시엄의 경우, 대중은 감사자 역할을 할 수 있습니다.
1. **경제적 인센티브.** 일부 블록체인 시스템은 기존의 저장된 데이터를 변경하는 데 많은 비용이 들게 합니다. 그 예로 작업 증명 및 지분 증명 시스템이 있습니다. BigchainDB의 경우에는 이런 인센티브를 사용하지 않습니다.
1. 변화에 대한 손쉬운 실행 취소를 위해 오류 수정 코드와 같은 고급 기술을 사용해 데이터를 저장할 수 있습니다
1. **암호화폐의 표식**은 종종 메시지(예: 트랜잭션)가 도중에 손상되었는지 확인하고 메시지에 서명한 사용자를 확인하는 방법으로 사용됩니다. BigchainDB에서는 각 트랜잭션에 한 명 이상의 당사자가 서명해야 합니다
1. **전체 또는 부분적 백업**은 때때로 자기 테이프 저장소, 기타 블록체인, 인쇄물 등에 기록될 수 있습니다.
1. **강력한 보안** 노드 소유자는 강력한 보안 정책을 채택하고 적용할 수 있습니다.
1. **노드 다양성.** 다양성으로 인해서 한 가지 요소(예: 자연 재해 또는 운영 체제 버그)가 상당 수의 노드를 손상시킬 수 없도록 합니다. [노드 다양성의 종류에 대한 섹션](https://github.com/bigchaindb/bigchaindb/blob/master/docs/root/source/korean/diversity-ko.md)을 참조하세요.

97
docs/root/source/korean/index.rst Normal file
View File

@ -0,0 +1,97 @@

.. Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
BigchainDB 문서
========================
블록체인 데이터베이스인 BigchainDB를 만나보세요.
`분산형 <https://github.com/bigchaindb/bigchaindb/blob/master/docs/root/source/korean/decentralized_kor.md>`_, `불변성 <https://github.com/bigchaindb/bigchaindb/blob/master/docs/root/source/korean/immutable-ko.md>`_ 및 `자산에 대한 네이티브 지원 <https://github.com/bigchaindb/bigchaindb/blob/master/docs/root/source/korean/assets_ko.md>`_ 을 포함한 일부 데이터베이스 특성들과 블록체인 특성을 가지고 있습니다.
높은 수준에서, 사용자는 BigchainDB HTTP API를 사용하는 BigchainDB 네트워크(노드 집합) 또는 BigchainDB 파이썬 드라이버와 같은 API용 래퍼로 통신할 수 있습니다. 각 BigchainDB 노드는 BigchainDB Server 및 다양한 다른 소프트웨어를 실행합니다. 더 자세한 사항은 용어 페이지에서 이러한 용어 중 일부를 설명합니다.
.. raw:: html
<style media="screen" type="text/css">
.button {
border-top: 1px solid #96d1f8;
background: #65a9d7;
background: -webkit-gradient(linear, left top, left bottom, from(#3e779d), to(#65a9d7));
background: -webkit-linear-gradient(top, #3e779d, #65a9d7);
background: -moz-linear-gradient(top, #3e779d, #65a9d7);
background: -ms-linear-gradient(top, #3e779d, #65a9d7);
background: -o-linear-gradient(top, #3e779d, #65a9d7);
padding: 8.5px 17px;
-webkit-border-radius: 3px;
-moz-border-radius: 3px;
border-radius: 3px;
-webkit-box-shadow: rgba(0,0,0,1) 0 1px 0;
-moz-box-shadow: rgba(0,0,0,1) 0 1px 0;
box-shadow: rgba(0,0,0,1) 0 1px 0;
text-shadow: rgba(0,0,0,.4) 0 1px 0;
color: white;
font-size: 16px;
font-family: Arial, Sans-Serif;
text-decoration: none;
vertical-align: middle;
}
.button:hover {
border-top-color: #28597a;
background: #28597a;
color: #ccc;
}
.button:active {
border-top-color: #1b435e;
background: #1b435e;
}
a.button:visited {
color: white
}
.buttondiv {
margin-bottom: 1.5em;
}
</style>
<div class="buttondiv">
<a class="button" href="http://bigchaindb.com/http-api">HTTP API 문서</a>
</div>
<div class="buttondiv">
<a class="button" href="http://docs.bigchaindb.com/projects/contributing/en/latest/index.html">BigchainDB에 기여하는 법</a>
</div>
<div class="buttondiv">
<a class="button" href="http://docs.bigchaindb.com/projects/py-driver/en/latest/index.html">파이썬 드라이버 문서</a>
</div>
<div class="buttondiv">
<a class="button" href="https://docs.bigchaindb.com/projects/js-driver/en/latest/index.html">자바스크립트 드라이버 문서</a>
</div>
<div class="buttondiv">
<a class="button" href="http://docs.bigchaindb.com/projects/server/en/latest/index.html">서버 문서</a>
</div>
<div class="buttondiv">
<a class="button" href="http://docs.bigchaindb.com/projects/server/en/latest/quickstart.html">서버 빠른 시작</a>
</div>
BigchainDB에 대한 추가 정보
-------------------------------------------------------
.. toctree::
:maxdepth: 1
BigchainDB Docs Home <self>
production-ready_kor
terminology_kor
decentralized_kor
diversity-ko
immutable-ko
bft-ko
query-ko
assets_ko
smart-contracts_ko
transaction-concepts_ko
store-files_ko
permissions-ko
private-data-ko
Data Models <https://docs.bigchaindb.com/projects/server/en/latest/data-models/index.html>

58
docs/root/source/korean/permissions-ko.md Normal file
View File

@ -0,0 +1,58 @@
<!---
Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# BigchainDB 사용 권한
BigchainDB를 사용하면 다른 사용자가 할 수 있는 것을 어느 정도 제어할 수 있습니다.
이 능력은 \*nix환경에서의 "권한", SQL에서의 "특권", 보안 환경에서의 "액세스 제어"와 유사합니다.
## 출력 지출/이전 권한
BigchainDB에서, 모든 출력에는 연관된 조건(crypto-condition)이 있습니다.
사용되지 않은 출력을 쓰거나 전송하려면, 사용자(또는 사용자 그룹)이 조건을 충족시켜야 합니다.
특정 사용자만이 출력을 보낼 권한이 있다는 뜻입니다. 가장 단순한 조건은, "공용 키에 해당하는 개인 키를 가진 사람만이 출력을 보낼 수 있습니다." 훨씬 더 정교한 조건들도 가능합니다, 예를 들어 “이 출력을 사용하려면,…"
- "…회계 그룹의 모든 사람이 서명 할 수 있습니다."
- "…네 명 중 세 명이 서명해야 합니다."
- "…Bob이 반드시 서명해야 하거나 Tom과 Sylvia 둘 모두가 서명해야 합니다."
자세한 내용은, [BigchainDB Transactions Spec](https://github.com/bigchaindb/BEPs/tree/master/tx-specs/)관련 **트랜잭션 구성요소:조건** 섹션을 참조하세요.
출력이 한번 소비되면 다시 사용할 수 없습니다: *아무도* 그렇게 할 권한이 없습니다. 즉, BigchainDB는 누구나 출력을 "이중 소비" 하도록 허용 하지 않습니다.
## 쓰기 권한
누군가 TRANSFER 트랜잭션을 만들면, `metadata` 필드에 임의의 JSON 객체를 넣을 수 있다. (적정 범위 내에서; 실제 BigchainDB 네트워크는 트랜잭션의 크기에 제한을 둔다.) 즉, TRANSFER 트랜잭션에서 원하는 모든 것을 쓸 수 있다.
BigchainDB에서 "쓰기 권한"이 없다는 의미인가요? 아닙니다!!
TRANSFER 트랜잭션은 입력이 이전 출력을 충족시키는 경우에만 유효(허용)합니다. 이 출력들에 대한 조건은 누가 유효한 TRANSFER 트랜잭션을 할 수 있는지 조절 할 것입니다. 즉, 출력에 대한 조건은 특정 사용자에게 관련 자산 내역에 무엇인가 쓸 수 있는 "쓰기 권한"을 부여하는 것과 같습니다.
예를 들어, 당신은 BigchainDB를 사용하여 오직 당신만이 쓰기권한이 있는 공용 저널을 작성 할 수 있습니다. 방법은 다음과 같습니다: 먼저 하나의 출력으로 `asset.data` 을 통해 `{"title": "The Journal of John Doe"}` 와 같이 되도록 CREATE 트랜잭션을 생성합니다. 이 출력에는 금액 1과 사용자(개인 키를 가진)만이 출력을 보낼 수 있는 조건이 있습니다. 저널에 무엇인가를 추가하고 싶을 때마다, `metadata` 같은 필드에 최신 항목을 넣은 TRANSFER 트랜잭션을 새로 만들어야 합니다.
```json
{"timestamp": "1508319582",
"entry": "I visited Marmot Lake with Jane."}
```
TRANSFER 트랜잭션에는 하나의 출력이 있습니다. 이 출력에는 금액1과 사용자(개인키를 가진)만이 출력을 보낼 수 있는 조건이 있습니다. 기타 등등. 당신만이 자산 내역(당신의 저널)에 덧붙일 수 있습니다.
이와 같은 기술은 공학 노트북,공급망 기록,정부 회의록 등에도 사용 될 수 있습니다.
또한 더 정교한 것들도 할 수 있습니다. 예를 들어, 누군가가 TRANSFER 트랜잭션을 작성할 때마다, *다른 누군가*에게 사용 권한을 부여하여 일종의 작성자-전달 혹은 연쇄 편지를 설정한다.
Note
누구나 CREATE 트랜잭션의 `asset.data` 필드에 있는 JSON(조건하에)을 쓸 수 있습니다. 허가가 필요하지 않습니다.
## 읽기 권한
다음 페이지를 참고하세요, [:doc:BigchainDB, Privacy and Private Data](https://github.com/bigchaindb/bigchaindb/blob/master/docs/root/source/korean/private-data-ko.md).
## 역할 기반 액세스 제어(RBAC)
2017년 9월에, 우리는 [BigchainDB RBAC 하부 시스템을 정의 할 수 있는 방법에 대한 블로그 게시물](https://blog.bigchaindb.com/role-based-access-control-for-bigchaindb-assets-b7cada491997)을 게재 했습니다. 글을 쓴 시점(2018년 1월)에는 플러그인을 사용해야 해서, 표준 BigchainDB(다음에서 사용가능한 [BigchainDB Testnet](https://testnet.bigchaindb.com/) 를 사용 할 수 없었습니다. 이는 미래에 바뀔 수 있습니다. 만약 관심이 있다면, [BigchainDB로 연락하십시요.](https://www.bigchaindb.com/contact/)

101
docs/root/source/korean/private-data-ko.md Normal file
View File

@ -0,0 +1,101 @@
<!---
Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# BigchainDB, 개인정보 및 개인 데이터
## 기본 정보
1. 한도 내에서 BigchainDB 네트워크에 임의의 데이터(암호화 된 데이터 포함)를 저장 할 수 있습니다. 모든 트랜잭션에는 거의 모든 유니코드 문자열(최대 길이까지)을 저장 할 수 있는 `metadata` 섹션이 있습니다. 마찬가지로, 모든 CREATE 트랜잭션에는 거의 모든 유니코드 문자열을 저장 할 수 있는 `asset.data` 섹션이 있습니다.
2. 특정 BigchainDB 거래 필드에 저장된 데이터는 암호화 해서는 안됩니다, 예를 들어 공용키 및 자산과 같이. BigchainDB는 Zcoin과 비슷한 개인 거래를 제공하지 않습니다.
3. 데이터가 BigchinDB 네트워크에 저장되면 변경 또는 삭제 될 수 없다고 가정하는 것이 좋습니다.
4. BigchainDB 네트워크의 모든 노드에는 저장된 모든 데이터의 전체 복사본이 있습니다.
5. BigchainDB 네트워크의 모든 노드는 저장된 모든 데이터를 읽을 수 있습니다.
6. BigchainDB 노드(예를 들어 노드이 sysadmin)에 대한 전체 액세스 권한을 가진 모든 사용자는해당 노드에 저장된 모든 데이터를 읽을 수 있습니다.
7. BigchainDB HTTP API를 통해 노드에 접근하는 모든 사용자는 BigchainDB에 저장된 모든 데이터를 찾고 읽을 수 있습니다. 액세스 권한이 있는 사람들의 목록은 매우 짧을 수 있습니다.
8. 외부 사용자와 BigchainDB 노드 사이의 연결이(예를 들어 HTTPS를 사용하여) 암호화되 않으면도청자는 전송중인 모든 HTTP 요청 및 응답을 읽을 수 있습니다.
9. 만약 누군가가 평문에 접근 할 수 있다면(어디에서 가져왔는지 관계없이), 원칙적으로 이것을 전 세계와 공유 할 수 있습니다. 그렇게 하는 것을 어렵게 만들 수 있습니다, 예를 들어 데이터가 많고 방을 나갈 떄 검색되는 안전한 방 안에만 들어 갈 수 있는 것과 같습니다.
## 오프 체인에서 개인 데이터 저장
시스템은 제3자 데이터베이스, 문서 저장소 또는 CMS(컨텐츠 관리 시스템)와 같은 오프 체인 데이터를 저장할 수 있으며, BigchinDB를 사용하여 다음 작업을 수행할 수 있습니다:
- 제3자 시스템에 읽기 권한 또는 기타 권한이 있는 사용자를 추적합니다. 이 작업을 수행하는 방법의 예는 아래에 있습니다.
- 제3자 시스템에 대한 모든 요청을 영구적으로 기록합니다.
- 모든 문서의 변경 사항을 감지 할 수 있도록, 다른 곳에 저장된 문서의 해시를 저장합니다.
- 암호화 된 터널을 설정했다는 것을 증명할 수 있도록 두 개의 오프 체인 파티(예:Diffie-Hellman 키 교환) 간의 모든 핸드셰이크 설정 요청 및 응답을 기록합니다(독자가 해당 터널에 액세스하지 않고). 이 아이디어에 대한 자세한 내용은 [the BigchainDB Privacy Protocols 저장소](https://github.com/bigchaindb/privacy-protocols)에 있습니다.
특정 문서에 대한 읽기 권한을 가진 사람을 기록하는 간단한 방법은 제 3자 시스템(“Docpile“)이 모든 문서+사용자 쌍에 대해 BigchinDB 네트워크에 CREATE 트랜잭션을 저장하여 해당 사용자가 그 문서에 대한 읽기 권한을 가지고 있음을 나타낼 수 있습니다. 트랜잭션은 Docpile에 의해 서명 될 수 있습니다(또는 문서 소유자에 의해). 자산 데이터 필드는 1)사용자의 고유 ID 및 2)문서의 고유 ID를 포함합니다. CREATE 트랜잭션의 한 출력은 DocPile(또는 문서 소유자)에 의해서만 전송/소비 될 수 있습니다.
읽기 권한을 취소하기 위해, DocPile은 원래 사용자가 더 이상 해당 문서에 대한 읽기 권한을 가지고 있지 않다고 하는 메타 데이터 필드를 사용하여, 원래의 CREATE 트랜잭션에서 하나의 출력을 보내기 위한 TRANSFER 트랜잭션을 생성 할 수 있습니다.
이는 무한정으로 수행될 수 있습니다,즉.사용자가 다시 읽기 권한을 가지고 있음을 나타내기 위해 다른 TRANSFER 트랜잭션을 DocPile에서 작성할 수 있습니다.
DocPile은 CREATE → TRANSFER → TRANSFER → 사용자+문서 쌍에 대한 etc.chain 과정에서 사용자의 마지막 트랜잭션을 읽음으로써 주어진 문서에 대한 읽기 권한을 가지고 있는지 파악할 수 있습니다.
여기에 같은 일을 하는 다른 방법들이 있다. 위는 단지 하나의 예시이다.
위의 예시에서는 사용자가 소유한(통제 된)자산으로 “읽기 권한“을 취급하지 않았다는 것을 알 수 있습니다, 왜냐하면 사용 권한 자산이 사용자에게 주어 지면(사용자에 의해 양도되거나 사용자에 의해 생성된 경우) 사용자가 다시 Docpile로 전송 할 때까지 어떠한 것도 제어 할 수 없기 때문입니다(Docpile에 의해).
## 체인에서 암호화 된 개인 데이터 저장
체인상에서 개인 데이터를 암호화하여 저장하는 방법에는 여러 가지가 있습니다. 모든 유스 케이스에는 고유한 목표와 제약이 있으며, 최상의 해결책은 유스 케이스에 달려있다.
[BigchainDB 컨설팅 팀](https://www.bigchaindb.com/services/), 우리의 파트너와 함께, 당신의유스 케이스에 가장 적합한 솔루션을 설계하는 데 도움을 줄 수 있습니다.
아래에서는 다양한 암호화 기본 설정을 사용하여 가능한 시스템을 설정하는 예제를 설명합니다.
참고 사항:
- Ed25519 키 쌍은 [메시지 암호화 및 암호 해독이 아닌](https://crypto.stackexchange.com/questions/27866/why-curve25519-for-encryption-but-ed25519-for-signatures) 암호화 서명 및 확인을 위해 설계되었습니다. 암호화의 경우, X25519와 같은 암호화를 위해 설계된 키 쌍을 사용해야 합니다.
- 누군가(또는 어떤 그룹)이 체인상의 암호화 된 데이터를 해독하는 방법을 발표하면 암호화 된 데이터에 액세스 할 수 있는 모든 사람이 평문을 가져올 수 있습니다. 데이터는 삭제할 수 없습니다.
- 암호화 된 데이터는 MongoDM에서 색인을 생성하거나 검색 할 수 없습니다.(암호문을 색인화하고 검색 할 수 있지만 유용하지는 않습니다.) 암호화 된 데이터를 색인화하고 검색하기 위해 준 유사 암호를 사용할 수 있지만, MongoDB는 이를 지원할 계획이 없습니다. 색인화 또는 키워드 검색이 필요한 경우 `asset.data`의 몇가지 필드 또는 `metadata`객체를 일반 텍스트로 남겨두고 민감한 정보를 암호화 된 하위 객체에 저장할 수 있습니다.
### 시스템 예시 1
대칭 키로 데이터를 암호화하고 체인에(`metadata` 또는 `asset.data` 에서) 암호문을 저장하십시오. 키를 제 3자에게 알리려면, 공용 키를 사용하여 대칭 키를 암호화하고 암호화 키를 보냅니다. 개인 키로 대칭 키의 암호를 해독한 다음 대칭 키를 사용하여 on-chain 암호문의 암호를 해독할 수 있습니다.
공용 키/ 개인 키 쌍과 함께 대칭 키를 사용하는 이유는 암호문을 한 번만 저장하면 되기 때문입니다.
### 시스템 예시 2
이 예시에서는 [프록시 재-암호화](https://en.wikipedia.org/wiki/Proxy_re-encryption) 를 사용합니다:
1. MegaCorp는 자체 공용 키를 사용하여 일부 데이터를 암호화 한 후 암호화 된 데이터(암호문1)을 BigchainDB 네트워크에 저장합니다.
2. MegaCorp는 다른 사람들이 암호화 된 데이터를 읽을 수 있게 하고 싶지만, 공용 키를 공유하지 않고 모든 새로운 수신자에 대해 스스로를 다시 암호화 할 필요가 없습니다. 대신 프록시 재 암호화 서비스를 제공하기 위해 Moxie라는 “프록시“를 찾습니다.
3. Zorban은 MegaCorp에 연결하여 데이터 읽기 권한을 요청합니다.
4. MegaCorp는 Zorban에게 공용 키를 요청합니다.
5. MegaCorp “재 암호화 키“를 생성하여 프록시 Moxie로 전송합니다.
6. Moxie (프록시)는 재 암호화 키를 사용하여 암호문 1을 암호화하고 암호문 2를 만듭니다.
7. Moxie는 Zorban(또는 Zorban에게 전달하는 MegaCorp)에게 암호문 2를 보냅니다.
8. Zorban은 개인 키를 사용하여 암호문 2를 해독해서 원본 암호화되지 않은 데이터를 가져옵니다.
참고:
- 프록시는 암호문만 볼 수 있습니다. 암호화 되지 않은 데이터는 볼 수 없습니다.
- Zorban은 암호문 1, 즉 체인 상의 데이터를 해독 할 수 있는 능력이 없습니다.
- 위의 흐름에는 다양한 변형이 있습니다.
## 시스템 예시 3
이 예시는 [삭제 코딩](https://en.wikipedia.org/wiki/Erasure_code)을 사용합니다:
1. 데이터를 n개의 조각으로 삭제하십시오.
2. 서로 다른 암호화 키로 n개의 조각을 암호화 하십시오.
3. n 개의 암호화 된 부분을 체인에 저장합니다 (예: n개의 별도 트랜잭션).
4. n 개의 암호 해독 키 각각을 다른 당사자와 공유하십시오.
만약 k< N 인 키홀더가 k개의 조각들을 가져와서 해독한다면, 그것들은 원본 텍스트를 다시 만들 수 있습니다. k미만이면 충분하지 않습니다.
### 시스템 예시 4
이 설정은 특수 노드가 데이터의 일부를 볼 수 있어야 하지만, 다른 노드는 볼 수 없어야 하는 기업용 블록 체인 시나리오에서 사용할 수 있습니다.
- 특수 노드는 X25519 키 쌍 (또는 유사한 비대칭 *암호화*키 쌍)을 생성합니다 .
- BigchainDB 최종 사용자는 특수 노드의 X25519 공용 키(암호화 키)를 찾습니다.
-최종 사용자는 위에서 언급 한 공용 키를 사용하여, asset.data 또는 메타 데이터(또는 모두)를 사용하여 유효한 BigchainDB 트랜잭션을 생성합니다.
- 이는 asset.data 또는 메타 데이터의 내용이 유효성 검증에 중요하지 않은 트랜잭션에 대해서만 수행되므로, 모든 노드 운영자가 트랜잭션을 검증 할 수 있습니다.
- 특수 노드는 암호화 된 데이터를 해독 할 수 있지만, 다른 노드 운영자와 다른 최종 사용자는 할 수 없습니다.

11
docs/root/source/korean/production-ready_kor.md Normal file
View File

@ -0,0 +1,11 @@
<!---
Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# 배포 - 준비
경우에 따라, BigchainDB는 배포-준비가 될 수도 있고 되지 않을 수도 있습니다. 서비스 공급자에게 문의해야 합니다. 만약 BigchainDB를 (배포로) 전환하고자 한다면, 서비스 공급자에게 문의하십시오.
참고 : BigchainDB는 "보증 없음" 섹션을 가지는 오픈소스 라이센스이며, 이는 전형적인 오픈소스 라이센스입니다. 이는 소프트웨어 산업의 표준입니다. 예를 들어, 리눅스 커널은 라이센스에 "보증 없음" 섹션을 가지고 있지만, 수십억 대의 시스템에 의해 배포되어 사용됩니다. 보증은 대개 서비스 공급자가 소프트웨어 라이센스 수준 이상으로 제공합니다.

201
docs/root/source/korean/query-ko.md Normal file
View File

@ -0,0 +1,201 @@
<!--
Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
-->
BigchainDB 쿼리
===================
노드 operator는 MongoDB의 쿼리 엔진의 최대 성능을 사용하여 모든 트랜잭션, 자산 및 메타데이터를 포함하여 저장된 모든 데이터를 검색하고 쿼리할 수 있습니다. 노드 operator는 외부 사용자에게 얼마나 많은 쿼리 파워를 송출할지 스스로 결정할 수 있습니다.
예제 쿼리가 포함된 블로그 게시물
------------------------------
BigchainDB 블로그에 MongoDB 도구를 사용하여 BigchainDB 노드의 MongoDB 데이터베이스를 쿼리하는 방법에 대한 게시물을 올렸습니다. 데이터에 대한 일부 특정 예제 쿼리가 주요 내용입니다. [여기서 확인하세요](https://blog.bigchaindb.com/using-mongodb-to-query-bigchaindb-data-3fc651e0861b)
MongoDB에 연결하기
-------------------------
MongoDB 데이터베이스를 쿼리하려면 먼저 데이터베이스에 연결해야 합니다. 그러기 위해선 호스트 이름과 포트를 알아야 합니다.
개발 및 테스트를 위해 지역 컴퓨터에서 BigchainDB 노드를 실행 중인 경우 호스트 이름은 "로컬 호스트"여야 하며 이러한 값을 변경하지 않는 한 포트는 "27017"이어야 합니다. 원격 시스템에서 BigchainDB 노드를 실행 중이며 해당 시스템에 SSH할 수 있는 경우에도 마찬가지입니다.
원격 시스템에서 BigchainDB 노드를 실행하고 MongoDB를 auth를 사용하고 공개적으로 액세스할 수 있도록 구성한 경우(권한이 있는 사용자에게) 호스트 이름과 포트를 확인할 수 있습니다.
쿼리하기
------------
BigchainDB 노드 운영자는 로컬 MongoDB 인스턴스에 대한 전체 액세스 권한을 가지므로 실행하는데 MongoDB의 다음의 API를 사용할 수 있습니다:
- [the Mongo Shell](https://docs.mongodb.com/manual/mongo/)
- [MongoDB Compass](https://www.mongodb.com/products/compass)
- one of [the MongoDB drivers](https://docs.mongodb.com/ecosystem/drivers/), such as [PyMongo](https://api.mongodb.com/python/current/), or
- MongoDB 쿼리에 대한 서드파티툴, RazorSQL, Studio 3T, Mongo Management Studio, NoSQLBooster for MongoDB, or Dr. Mongo.
Note
SQL을 이용해 mongoDB 데이터베이스를 쿼리할 수 있습니다. 예를 들어:
* Studio 3T: "[How to Query MongoDB with SQL](https://studio3t.com/whats-new/how-to-query-mongodb-with-sql/)"
* NoSQLBooster for MongoDB: "[How to Query MongoDB with SQL SELECT](https://mongobooster.com/blog/query-mongodb-with-sql/)"
예를 들어, 기본 BigchainDB 노드를 실행하는 시스템에 있는 경우 Mongo Shell (``mongo``)을 사용하여 연결하고 다음과 같이 볼 수 있습니다.
$ mongo
MongoDB shell version v3.6.5
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.6.4
...
> show dbs
admin 0.000GB
bigchain 0.000GB
config 0.000GB
local 0.000GB
> use bigchain
switched to db bigchain
> show collections
abci_chains
assets
blocks
elections
metadata
pre_commit
transactions
utxos
validators
위 예제는 몇 가지 상황을 보여줍니다:
- 호스트 이름이나 포트를 지정하지 않으면 Mongo Shell은 각각 `localhost`와 `27017`으로 가정합니다. (`localhost`는 우분투에 IP주소를 127.0.0.1로 설정했습니다.)
* BigchainDB는 데이터를 `bigchain`이라는 데이터베이스에 저장합니다.
* `bigchain` 데이터베이스에는 여러 [collections](https://docs.mongodb.com/manual/core/databases-and-collections/)가 포함되어 있습니다.
* 어떤 컬렉션에도 투표가 저장되지 않습니다. 이런 데이터는 모두 자체(LevelDB) 데이터베이스에 의해 처리되고 저장됩니다.
컬렉션에 대한 예시 문서
---------------------------------------
``bigchain`` 데이터베이스의 가장 흥미로운 부분은 아래와 같습니다:
- transactions
- assets
- metadata
- blocks
`db.assets.findOne()` 은 MongoDB 쿼리를 사용하여 이러한 컬렉션들을 탐색할 수 있습니다.
### 트랜잭션에 대한 예시 문서
transaction 컬렉션에서 CREATE 트랜잭션에는 추가 `"_id"` 필드(MongoDB에 추가됨)가 포함되며 `"asset"`과 `"metadata"` 필드에는 데이터가 저장되어 있지 않습니다.
{
"_id":ObjectId("5b17b9fa6ce88300067b6804"),
"inputs":[…],
"outputs":[…],
"operation":"CREATE",
"version":"2.0",
"id":"816c4dd7…851af1629"
}
A TRANSFER transaction from the transactions collection is similar, but it keeps its `"asset"` field.
{
"_id":ObjectId("5b17b9fa6ce88300067b6807"),
"inputs":[…],
"outputs":[…],
"operation":"TRANSFER",
"asset":{
"id":"816c4dd7ae…51af1629"
},
"version":"2.0",
"id":"985ee697d…a3296b9"
}
### assets에 대한 예시 문서
assets에 대한 기술에는 MongoDB가 추가한 `"_id"` 분야와 CREATE 거래에서 나온 `asset.data` 그리고 `"id"` 세 가지 최상위 분야로 구성되어 있습니다.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
{
"_id":ObjectId("5b17b9fe6ce88300067b6823"),
"data":{
"type":"cow",
"name":"Mildred"
},
"id":"96002ef8740…45869959d8"
}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
### metadata에 대한 예시 문서
metadata 컬렉션의 문서는 MongoDB가 추가한 `"_id"`필드와 거래에서 나온 `asset.data`그리고 거래에서 나온 ``"id"`` 세 가지 최상위 분야로 구성되어 있습니다.
{
"_id":ObjectId("5b17ba006ce88300067b683d"),
"metadata":{
"transfer_time":1058568256
},
"id":"53cba620e…ae9fdee0"
}
### blocks에 대한 예시 문서
{
"_id":ObjectId("5b212c1ceaaa420006f41c57"),
"app_hash":"2b0b75c2c2…7fb2652ce26c6",
"height":17,
"transactions":[
"5f1f2d6b…ed98c1e"
]
}
## 노드 operator가 외부 유저에게 보낼 수 있는 것
각 노드 operator는 외부 사용자가 자신의 로컬 MongoDB 데이터베이스에서 정보를 얻는 방법을 결정할 수 있습니다. 그들은 다음과 같은 것들을 보낼 수 있습니다:
- 외부유저를 쿼리 처리하는 로컬 MongoDB 데이터베이스 한된 제한된 권한을 가진 역할을 가진 MongoDB 사용자 예) read-only
- 제한된 미리 정의된 쿼리 집합을 허용하는 제한된 HTTP API, [BigchainDB 서버에서 제공하는 HTTP API](http://bigchaindb.com/http-api), 혹은Django, Express, Ruby on Rails, or ASP.NET.를 이용해 구현된 커스텀 HTTP API
- 다른 API(예: GraphQL API) 제3자의 사용자 정의 코드 또는 코드를 사용하여 수행할 수 있습니다..
각 노드 operator는 로컬 MongoDB 데이터베이스에 대한 다른 레벨 또는 유형의 액세스를 노출할 수 있습니다.
예를 들어, 한 노드 operator가 최적화된 [공간 쿼리](https://docs.mongodb.com/manual/reference/operator/query-geospatial/)를 전문으로 제공하기로 정할 수 있습니다.
보안 고려사항
-----------------------
BigchainDB 버전 1.3.0 이전 버전에서는 하나의 MongoDB 논리 데이터베이스가 있었기 때문에 외부 사용자에게 데이터베이스를 노출하는 것은 매우 위험했으며 권장되지 않습니다. "Drop database"는 공유된 MongoDB 데이터베이스를 삭제합니다.
BigchainDB 버전 2.0.0 이상에선 각 노드에 고유한 독립 로컬 MongoDB 데이터베이스가 존재합니다. 노드 간 통신은 아래 그림 1에서와 같이 MongoDB 프로토콜이 아닌 Tendermint 프로토콜을 사용하여 수행됩니다. 노드의 로컬 MongoDB 데이터베이스가 손상되어도 다른 노드는 영향을 받지 않습니다.
![image](https://user-images.githubusercontent.com/36066656/48752907-f1dcd600-ecce-11e8-95f4-3cdeaa1dc4c6.png)
Figure 1: A Four-Node BigchainDB 2.0 Network
퍼포먼스 및 요금 고려사항
-----------------------------------
쿼리 프로세싱은 상당히 많은 리소스를 소모할 수 있으므로, BigchainDB 서버 및 Tendermint Core와 별도의 컴퓨터에서 MongoDB를 실행하는 것이 좋습니다.
노드 operator 는 조회에 사용되는 리소스를 측정하여 조회를 요청한 사람은 누구든지 요금을 청구할 수 있습니다.
일부 쿼리는 너무 오래 걸리거나 리소스를 너무 많이 사용할 수 있습니다. 노드 operator는 사용할 수 있는 리소스에 상한을 두고, 초과된다면 중지(또는 차단)해야 합니다.
MongoDB 쿼리를 더욱 효율적으로 만들기 위해 [인덱스](https://docs.mongodb.com/manual/indexes/)를 만들 수 있습니다. 이러한 인덱스는 노드 operator 또는 일부 외부 사용자가 생성할 수 있습니다(노드 운영자가 허용하는 경우). 인덱스는 비어 있지 않습니다. 새 데이터를 컬렉션에 추가할 때마다 해당 인덱스를 업데이트해야 합니다. 노드 운영자는 이러한 요금을 인덱스를 생성한 사람에게 전달하고자 할 수 있습니다. mongoDB에서는 [단일 컬렉션은 64개 이하의 인덱스를 가질 수 있습니다](https://docs.mongodb.com/manual/reference/limits/#Number-of-Indexes-per-Collection).
Tendermint voting 파워가 0인 노드인 추종자 노드를 생성할 수 있다. 여전히 모든 데이터의 복사본이 있으므로 읽기 전용 노드로 사용할 수 있습니다. Follower 노드는 투표 검증자의 작업 부하에 영향을 미치지 않고 서비스로 전문화된 쿼리를 제공할 수 있습니다(쓰기도 가능). 팔로워의 팔로워들도 있을 수 있습니다.
자바스크립트 쿼리 코드 예시
------------------------------
[MongoDB node.js 드라이버](https://mongodb.github.io/node-mongodb-native/?jmp=docs)와 같은 MongoDB 드라이버를 사용하여 다음 중 하나를 사용하여 노드의 MongoDB 데이터베이스에 연결할 수 있습니다. 여기 자바스크립트 쿼리 코드에 대한 링크가 있습니다.
- [The BigchainDB JavaScript/Node.js driver source code](https://github.com/bigchaindb/js-bigchaindb-driver)
- [Example code by @manolodewiner](https://github.com/manolodewiner/query-mongodb-bigchaindb/blob/master/queryMongo.js)
- [More example code by @manolodewiner](https://github.com/bigchaindb/bigchaindb/issues/2315#issuecomment-392724279)

16
docs/root/source/korean/smart-contracts_ko.md Normal file
View File

@ -0,0 +1,16 @@
<!--
Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
-->
BigchainDB 및 스마트계약
==============================
BigchainDB에는 스마트 계약 (즉, 컴퓨터 프로그램)의 소스 코드를 저장할 수 있지만 BigchainDB는 임의의 스마트 계약을 실행하지 않습니다.
BigchainDB는 대체 가능한 자산과 대체 할 수없는 자산 모두를 전송할 수있는 권한을 가진 사람을 시행하는 데 사용할 수 있습니다. 이중 지출을 막을 것입니다. 즉, ERC-20 (대체 가능한 토큰) 또는 ERC-721 (대체 할 수없는 토큰) 스마트 계약 대신 BigchainDB 네트워크를 사용할 수 있습니다.
자산 이전 권한은 쓰기 권한으로 해석 될 수 있으므로 로그, 저널 또는 감사 내역에 기록 할 수있는 사람을 제어하는데 사용할 수 있습니다. [BigchainDB의 사용 권한](https://github.com/bigchaindb/bigchaindb/blob/master/docs/root/source/korean/permissions-ko.md)에 대한 자세한 내용은 페이지에서 확인하십시오.
BigchainDB 네트워크는 oracles 또는 체인 간 통신 프로토콜을 통해 다른 블록 체인 네트워크에 연결할 수 있습니다. 이는 BigchainDB를 다른 블록 체인을 사용하여 임의의 스마트 계약을 실행하는 솔루션의 일부로 사용할 수 있음을 의미합니다.

13
docs/root/source/korean/store-files_ko.md Normal file
View File

@ -0,0 +1,13 @@
<!---
Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# BigchainDB에 파일을 저장하는 방법
BigchainDB 네트워크에 파일을 저장할 수는 있지만 그렇게하지 않는 것이 좋습니다. 파일이 아닌 구조화 된 데이터를 저장, 인덱싱 및 쿼리하는 데 가장 적합합니다.
분산 된 파일 저장소를 원하면 Storj, Sia, Swarm 또는 IPFS / Filecoin을 확인하십시오. 파일 URL, 해시 또는 기타 메타 데이터를 BigchainDB 네트워크에 저장할 수 있습니다.
BigchainDB 네트워크에 파일을 저장해야하는 경우,이를 수행하는 한 가지 방법은 긴 Base64 문자열로 변환 한 다음 해당 문자열을 하나 이상의 BigchainDB 트랜잭션 (CREATE 트랜잭션의 `asset.data`)에 저장하는 것입니다 , 또는 어떤 거래의 `메타데이터` 일 수도있다.

25
docs/root/source/korean/terminology_kor.md Normal file
View File

@ -0,0 +1,25 @@
<!---
Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# 용어
BigchainDB와 관련돈 몇 가지 전문화된 용어가 있습니다. 시작하기에 앞서, 최소한 다음과 같은 사항을 알아야합니다.
## BigchainDB 노드
**BigchainDB 노드**는 [BigchainDB 서버](https://docs.bigchaindb.com/projects/server/en/latest/introduction.html) 및 관련된 소프트웨어를 실행하는 시스템(또는 논리적인 시스템)입니다. 각각의 노드는 한 개인이나 조직에 의해 제어될 수 있습니다.
## BigchainDB 네트워크
BigchainDB 노드들의 집합은 서로 연결하여 **BigchainDB 네트워크**를 형성할 수 있습니다. 해당 네트워크에서 각각의 노드는 동일한 소프트웨어를 실행합니다. BigchainDB 네트워크는 모니터링 같은 것들을 하기 위한 추가적인 시스템이 있을 수 있습니다.
## BigchainDB 컨소시엄
BigchainDB 네트워크에 노드들을 실행하는 사람과 조직은 **BigchainDB 컨소시엄**(즉, 다른 조직)에 속합니다. 컨소시엄은 결정을 하기 위해 일종의 거버넌스 구조를 가져야합니다. 만약 BigchainDB 네트워크가 단 하나의 회사에 의해서 운영된다면, "컨소시엄"은 단지 그 회사일 뿐입니다.
**BigchainDB 네트워크와 컨소시엄의 차이는 무엇일까요?**
BigchainDB 네트워크는 단지 연결된 노드들의 집합입니다. 컨소시엄은 하나의 BigchainDB 네트워크를 가지는 조직이며, 해당 네트워크에서 각각의 노드는 다른 운영자를 가집니다.

60
docs/root/source/korean/transaction-concepts_ko.md Normal file
View File

@ -0,0 +1,60 @@
<!---
Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# 트랜잭션 개념
*트랜잭션*은 물건 (예 : 자산)을 등록, 발행, 생성 또는 전송하는 데 사용됩니다.
트랜잭션은 BigchainDB가 저장하는 가장 기본적인 종류의 레코드입니다. CREATE 트랜잭션과 TRANSFER 트랜잭션의 두 종류가 있습니다.
## 트랜잭션 생성
CREATE 트랜잭션은 BigchainDB에서 한 가지 (또는 자산)의 이력을 등록, 발행, 생성 또는 다른 방법으로 시작하는 데 사용될 수 있습니다. 예를 들어, 신원이나 창작물을 등록 할 수 있습니다. 이러한 것들을 종종 "자산"이라고 부르지만 literal 자산이 아닐 수도 있습니다.
BigchainDB는 BigchainDB Server v0.8.0부터 나눌 수있는 자산을 지원합니다. 이는 "공유"의 초기 숫자로 자산을 생성 / 등록 할 수 있음을 의미합니다. 예를 들어, CREATE 트랜잭션은 50 개의 오크 나무로 된 트럭로드를 등록 할 수 있습니다. 분할 가능한 자산의 각 주식은 서로 공유 할 수 있어야합니다. 주식은 대체 가능해야합니다.
CREATE 트랜잭션은 하나 이상의 출력을 가질 수 있습니다. 각 출력에는 관련 금액이 있습니다. 출력에 연결된 공유 수입니다. 예를 들어 자산이 50 개의 오크 나무로 구성되어있는 경우 한 출력에는 한 소유자 세트에 35 개의 오크 나무가 있고 다른 출력에는 다른 소유자 세트에는 15 개의 오크 나무가있을 수 있습니다.
또한 각 출력에는 연관된 조건이 있습니다. 출력을 전송 / 소비하기 위해 충족되어야하는 조건 (TRANSFER 트랜잭션에 의해). BigchainDB는 다양한 조건을 지원합니다. 자세한 내용은 관련 [BigchainDB 트랜잭션 Spec](https://github.com/bigchaindb/BEPs/tree/master/tx-specs/)과 관련된 **트랜잭션 구성 요소 : 조건 섹션**을 참조하십시오.
![Example BigchainDB CREATE transaction](./_static/CREATE_example.png)
위의 예제에서는 BigchainDB CREATE 트랜잭션 다이어그램을 보여줍니다. Pam은 자산 3 주를 소유 / 통제하고 다른 주식은 없습니다 (다른 산출물이 없으므로).
각 출력에는 해당 출력의 조건과 연관된 모든 공개 키 목록이 있습니다. 다시 말하면, 그 목록은 "소유자"의 목록으로 해석 될 수 있습니다.보다 정확한 단어는 이행자, 서명자, 컨트롤러 또는 이전 가능 요소 일 수 있습니다. 관련 [BigchainDB Transactions Spec](https://github.com/bigchaindb/BEPs/tree/master/tx-specs/) **소유자에 관한 참고 사항** 섹션을 참조하십시오.
CREATE 트랜잭션은 모든 소유자가 서명해야합니다. (만약 당신이 그 서명을 원한다면, 그것은 인코딩되었지만 하나의 입력의 "이행"에있다.)
## 트랜잭션 이전
트랜잭션 이전은 다른 트랜잭션 (CREATE 트랜잭션 또는 다른 TRANSFER 트랜잭션)에서 하나 이상의 출력을 전송 / 소비 할 수 있습니다. 이러한 출력물은 모두 동일한 자산과 연결되어야합니다. TRANSFER 트랜잭션은 한 번에 하나의 자산의 공유 만 전송할 수 있습니다.
트랜잭션 이전의 각 입력은 다른 트랜잭션의 한 출력에 연결됩니다. 각 입력은 전송 / 소비하려는 출력의 조건을 충족해야합니다.
트랜잭션 이전은 위에서 설명한 CREATE 트랜잭션과 마찬가지로 하나 이상의 출력을 가질 수 있습니다. 투입물에 들어오는 총 주식 수는 산출물에서 나가는 총 주식 수와 같아야합니다.
![Example BigchainDB transactions](./_static/CREATE_and_TRANSFER_example.png)
위 그림은 두 개의 BigchainDB 트랜잭션, CREATE 트랜잭션 및 TRANSFER 트랜잭션의 다이어그램을 보여줍니다. CREATE 트랜잭션은 이전 다이어그램과 동일합니다. TRANSFER 트랜잭션은 Pam의 출력을 소비하므로 TRANSFER 트랜잭션의 입력에는 Pam의 유효한 서명 (즉, 유효한 이행)이 포함되어야합니다. TRANSFER 트랜잭션에는 두 개의 출력이 있습니다. Jim은 하나의 공유를 가져오고 Pam은 나머지 두 개의 공유를 가져옵니다.
용어 : "Pam, 3"출력을 "소비 된 트랜잭션 출력"이라고하며 "Jim, 1"및 "Pam, 2"출력을 "사용되지 않은 트랜잭션 출력"(UTXO)이라고합니다.
**예제 1:** 빨간 차가 Joe가 소유하고 관리한다고 가정합니다. 자동차의 현재 전송 조건에서 Joe가 유효한 전송을 서명해야한다고 가정합니다. Joe는 Joe의 서명 (현재 출력 조건을 충족시키기 위해)과 Rae가 유효한 전송을 서명해야한다는 새로운 출력 조건을 포함하는 입력을 포함하는 TRANSFER 트랜잭션을 작성할 수 있습니다.
**예제 2:** 예를 들어 동일한 자산 유형의 이전에 전송되지 않은 4 개의 자산에 대한 출력 조건을 충족하는 TRANSFER 트랜잭션을 생성 할 수 있습니다. 종이 클립. 총 금액은 20, 10, 45 및 25 일 수 있으며, 말하자면 총 100 개의 클립입니다. 또한 TRANSFER 트랜잭션은 새로운 전송 조건을 설정합니다. 예를 들어, Gertrude가 서명하는 경우에만 60 개의 클립 클립이 전송 될 수 있으며 Jack과 Kelly가 서명하는 경우에만 40 개의 클립 클립이 전송 될 수 있습니다. 들어오는 클립 클립의 합계가 나가는 클립 클립의 합계와 같아야합니다 (100).
## 트랜잭션 유효성
언제 트랜잭션이 유효한지 유효성을 검사하는 것에 관해 해당 블로그에 게시되어있습니다. *The BigchainDB Blog*:
["What is a Valid Transaction in BigchainDB?"](https://blog.bigchaindb.com/what-is-a-valid-transaction-in-bigchaindb-9a1a075a9598) (Note: That post was about BigchainDB Server v1.0.0.)
Each [BigchainDB Transactions Spec](https://github.com/bigchaindb/BEPs/tree/master/tx-specs/) documents the conditions for a transaction (of that version) to be valid.
## 트랜잭션 예시
아래의 [HTTP API 문서](https://docs.bigchaindb.com/projects/server/en/latest/http-client-server-api.html)와 [the Python 드라이버 문서](https://docs.bigchaindb.com/projects/py-driver/en/latest/usage.html)에는 예제 BigchainDB 트랜잭션이 있습니다.
.

2
docs/root/source/permissions.rst
View File

@ -3,6 +3,8 @@
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
.. _permissions-in-bigchaindb:
Permissions in BigchainDB
-------------------------

2
docs/root/source/query.rst
View File

@ -61,8 +61,10 @@ For example, if you're on a machine that's running a default BigchainDB node, th
> use bigchain
switched to db bigchain
> show collections
abci_chains
assets
blocks
elections
metadata
pre_commit
transactions

14
docs/root/source/smart-contracts.rst
View File

@ -8,16 +8,8 @@ BigchainDB and Smart Contracts
One can store the source code of any smart contract (i.e. a computer program) in BigchainDB, but BigchainDB won't run arbitrary smart contracts.
BigchainDB will run the subset of smart contracts expressible using `Crypto-Conditions <https://tools.ietf.org/html/draft-thomas-crypto-conditions-03>`_.
BigchainDB can be used to enforce who has permission to transfer assets, both fungible assets and non-fungible assets. It will prevent double-spending. In other words, a BigchainDB network could be used instead of an ERC-20 (fungible token) or ERC-721 (non-fungible token) smart contract.
The owners of an asset can impose conditions on it that must be met for the asset to be transferred to new owners. Examples of possible conditions (crypto-conditions) include:
Asset transfer permissions can also be interpreted as write permissions, so they can be used to control who can write to a log, journal or audit trail. There is more about that idea in :ref:`the page about permissions in BigchainDB <permissions-in-bigchaindb>`.
- The current owner must sign the transfer transaction (one which transfers ownership to new owners).
- Three out of five current owners must sign the transfer transaction.
- (Shannon and Kelly) or Morgan must sign the transfer transaction.
Crypto-conditions can be quite complex. They can't include loops or recursion and therefore will always run/check in finite time.
.. note::
We used the word "owners" somewhat loosely above. A more accurate word might be fulfillers, signers, controllers, or transfer-enablers. See the section titled **A Note about Owners** in the relevant `BigchainDB Transactions Spec <https://github.com/bigchaindb/BEPs/tree/master/tx-specs/>`_.
A BigchainDB network can be connected to other blockchain networks, via oracles or inter-chain communications protocols. That means BigchainDB can be used as part of a solution that uses *other* blockchains to run arbitrary smart contracts.

5
docs/server/requirements.txt
View File

@ -1,7 +1,8 @@
Sphinx>=1.4.8
Sphinx~=1.0
recommonmark>=0.4.0
sphinx-rtd-theme>=0.1.9
sphinxcontrib-napoleon>=0.4.4
sphinxcontrib-httpdomain>=1.5.0
pyyaml>=3.12
pyyaml~=3.12
aafigure>=0.6
packaging~=18.0

4
docs/server/source/appendices/index.rst
View File

@ -11,12 +11,10 @@ Appendices
json-serialization
cryptography
the-bigchaindb-class
backend
commands
aws-setup
generate-key-pair-for-ssh
firewall-notes
ntp-notes
licenses
all-in-one-bigchaindb
log-rotation

49
docs/server/source/appendices/log-rotation.md Normal file
View File

@ -0,0 +1,49 @@
<!---
Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# Logging and Log Rotation
Each BigchainDB node runs:
- MongoDB
- BigchainDB Server
- Tendermint
When running a BigchainDB node for long periods
of time, we need to consider doing log rotation, i.e. we do not want the logs taking
up large amounts of storage and making the node unresponsive or getting it into a bad state.
## MongoDB Logging and Log Rotation
See the MongoDB docs about
[logging](https://docs.mongodb.com/v3.6/administration/monitoring/#monitoring-standard-loggging)
and [log rotation](https://docs.mongodb.com/v3.6/tutorial/rotate-log-files/).
## BigchainDB Server Logging and Log Rotation
BigchainDB Server writes its logs to two files: normal logs and error logs. The names of those files, and their locations, are set as part of the BigchainDB configuration settings. The default names and locations are:
- `~/bigchaindb.log`
- `~/bigchaindb-errors.log`
Log rotation is baked into BigchainDB Server using Python's `logging` module. The logs for BigchainDB Server are rotated when any of the above mentioned files exceeds 209715200 bytes (i.e. approximately 209 MB).
For more information, see the docs about [the BigchainDB Server configuration settings related to logging](../server-reference/configuration.html#log).
## Tendermint Logging and Log Rotation
Tendermint writes its logs to the files:
- `tendermint.out.log`
- `tendermint.err.log`
If you started BigchainDB Server and Tendermint using Monit, as suggested by our guide on
[How to Set Up a BigchainDB Network](../simple-deployment-template/network-setup.html),
then the logs will be written to `$HOME/.bigchaindb-monit/logs/`.
Moreover, if you started BigchainDB Server and Tendermint using Monit,
then Monit monitors the Tendermint log files.
Tendermint logs are rotated if any of the above mentioned log files exceeds 200 MB.

0
docs/server/source/appendices/backend.rst → docs/server/source/code-reference/backend.rst
View File

0
docs/server/source/appendices/commands.rst → docs/server/source/code-reference/commands.rst
View File

24
docs/server/source/code-reference/index.rst Normal file
View File

@ -0,0 +1,24 @@
.. Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
Code Reference
==============
This section contains auto-generated documentation of various functions, classes and methods
in the BigchainDB Server code, based on Python docstrings in the code itself.
.. warning::
While we try to keep docstrings accurate,
if you want to know *for sure* what the code does,
then you have to read the code itself.
.. toctree::
:maxdepth: 1
the-bigchaindb-class
backend
commands

0
docs/server/source/appendices/the-bigchaindb-class.rst → docs/server/source/code-reference/the-bigchaindb-class.rst
View File

6
docs/server/source/conf.py
View File

@ -60,9 +60,9 @@ extensions = [
# autodoc settings
autodoc_member_order = 'bysource'
autodoc_default_flags = [
'members',
]
autodoc_default_options = {
'members': None,
}
todo_include_todos = True

4
docs/server/source/events/websocket-event-stream-api.rst
View File

@ -90,7 +90,7 @@ Valid Transactions
Streams an event for any newly valid transactions committed to a block. Message
bodies contain the transaction's ID, associated asset ID, and containing
block's ID.
block's height.
Example message:
@ -99,7 +99,7 @@ Example message:
{
"transaction_id": "<sha3-256 hash>",
"asset_id": "<sha3-256 hash>",
"block_id": "<int>"
"height": <int>
}

18
docs/server/source/http-client-server-api.rst
View File

@ -154,7 +154,15 @@ Transactions
is in a committed block.
.. note::
In the async and sync modes, after a successful HTTP response is returned, the transaction may still be rejected later on. All the transactions are recorded internally by Tendermint in WAL (Write-Ahead Log) before the HTTP response is returned. Nevertheless, the following should be noted:
- Transactions in WAL including the failed ones are not exposed in any of the BigchainDB or Tendermint APIs.
- Transactions are never fetched from WAL. WAL is never replayed.
- A critical failure (e.g. the system is out of disk space) may occur preventing transactions from being stored in WAL, even when the HTTP response indicates a success.
- If a transaction fails the validation because it conflicts with the other transactions of the same block, Tendermint includes it into its block, but BigchainDB does not store these transactions and does not offer any information about them in the APIs.
.. note::
The posted transaction should be valid.
The relevant
`BigchainDB Transactions Spec <https://github.com/bigchaindb/BEPs/tree/master/tx-specs/>`_
@ -608,14 +616,8 @@ Validators
:statuscode 200: The query was executed successfully and validators set was returned.
Advanced Usage
--------------------------------
The following endpoints are more advanced
and meant for debugging and transparency purposes.
Blocks
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
------
.. http:get:: /api/v1/blocks/{block_height}

1
docs/server/source/index.rst
View File

@ -23,4 +23,5 @@ BigchainDB Server Documentation
data-models/index
k8s-deployment-template/index
release-notes
code-reference/index
appendices/index

13
docs/server/source/quickstart.md
View File

@ -51,16 +51,15 @@ Code is Apache-2.0 and docs are CC-BY-4.0
Create a BigchainDB transaction and post it to a BigchainDB network in 20 seconds or less:
<div class="buttondiv">
<a class="button" href="https://www.bigchaindb.com/getstarted/">Try BigchainDB Now</a>
<a class="button" href="https://www.bigchaindb.com/developers/getstarted/">Try BigchainDB Now</a>
</div>
## Develop an App
To develop an app that talks to a BigchainDB network, you'll want a test network to test it against. The BigchainDB Testnet is a free-to-use, publicly-available BigchainDB network that you can test against.
To develop an app that talks to a BigchainDB network, you'll want a test network to test it against. You have a few options:
<div class="buttondiv">
<a class="button" href="https://testnet.bigchaindb.com/">Get started with the BigchainDB Testnet</a>
</div>
1. The BigchainDB Test Network (or "Testnet") is a free-to-use, publicly-available BigchainDB network that you can test against. You can find out more about the BigchainDB Testnet on [the BigchainDB "Get Started" page](https://www.bigchaindb.com/developers/getstarted/#server).
1. You could also run a BigchainDB node on you local machine. One way is to use the [Simple Deployment Template](./simple-deployment-template/index.html), with a one-node "network." Another way is to use one of the deployment methods listed in the [the docs about contributing to BigchainDB](https://docs.bigchaindb.com/projects/contributing/en/latest/index.html).
Regardless of which BigchainDB network you use, you'll probably use one of the [BigchainDB drivers or tools](https://www.bigchaindb.com/getstarted/#drivers).
@ -68,10 +67,6 @@ Regardless of which BigchainDB network you use, you'll probably use one of the [
To help develop BigchainDB Server (the core software in each BigchainDB node), see [the docs about contributing to BigchainDB](https://docs.bigchaindb.com/projects/contributing/en/latest/index.html).
## Old Quickstart
If you want something like the old Quickstart page, i.e. some command-line instructions to set up and run a BigchainDB node, then see [the notes on running a local dev node in the docs about contributing to BigchainDB](https://docs.bigchaindb.com/projects/contributing/en/latest/dev-setup-coding-and-contribution-process/index.html).
<hr>
<br>

146
docs/server/source/server-reference/bigchaindb-cli.md
View File

@ -81,77 +81,125 @@ configuration file as documented under
[Configuration Settings](configuration.html).
## bigchaindb upsert-validator
## bigchaindb election
Manage elections to add, update, or remove a validator from the validators set. The upsert-validator subcommands implement [BEP-21](https://github.com/bigchaindb/BEPs/tree/master/21), please refer it for more details.
Manage elections to govern the BigchainDB network. The specifics of the election process are defined in [BEP-18](https://github.com/bigchaindb/BEPs/tree/master/18).
Election management is broken into several subcommands. Below is the command line syntax for each,
Election management is broken into several subcommands. Below is the command line syntax for each of them.
#### upsert-validator new
### election new
Create a new election which proposes a change to the validator set. An election can be used to add/update/remove a validator from the validator set.
Create a new election which proposes a change to the BigchainDB network.
If the command succeeds, it will post an election transaction and output `election_id`.
The election proposal consists of vote tokens allocated to every current validator proportional to his voting power. Validators spend their votes to approve the election using the [election-approve command](#election-approve).
Every election has a type. Currently supported types are `upsert-validator` and `chain-migration`. Their transaction operations are `VALIDATOR_ELECTION` and `CHAIN_MIGRATION` accordingly. See below for how to create an election of a particular type.
Note that elections can only be proposed and approved by existing validators.
#### election new upsert-validator
Create an election to add, update, or remove a validator.
Below is the command line syntax and the return value,
```bash
$ bigchaindb upsert-validator new E_PUBKEY E_POWER E_NODE_ID --private-key PATH_TO_YOUR_PRIVATE_KEY
[SUCCESS] Submitted proposal with id: <election_id>
$ bigchaindb election new upsert-validator <public-key> <power> <node-id> --private-key <path-to-the-private-key>
```
- `E_PUBKEY`: Public key of the node to be added/updated/removed.
- `E_POWER`: The new power for the `E_PUBKEY`. NOTE, if power is set to `0` then `E_PUBKEY` will be removed from the validator set when the election concludes.
- `E_NODE_ID`: Node id of `E_PUBKEY`. The node operator of `E_PUBKEY` can generate the node id via `tendermint show_node_id`.
- `--private-key`: The path to Tendermint's private key which can be generally found at `/home/user/.tendermint/config/priv_validator.json`. For example, to add a new validator, provide the public key and node id for some node not already in the validator set, along with whatever voting power you'd like them to have. To remove an existing validator, provide their public key and node id, and set `E_POWER` to `0`. Please note that the private key provided here is of the node which is generating this election i.e.
- `<public-key>` is the public key of the node to be added/updated/removed. The encoding and type of the key have to match those specified in `genesis.json` in the supported Tendermint version.
- `<power>` is the new power for the validator. To remove the validator, set the power to `0`.
- `<node-id>` is the node identifier from Tendermint. A node operator can learn his node identifier by executing `tendermint show_node_id`.
- `<path-to-the-private-key>` is the path to the private key of the validator who proposes the election. Tendermint places it at `.tendermint/config/priv_validator.json`.
NOTE: A change to the validator set can only be proposed by one of the exisitng validators.
Example usage,
Example:
```bash
$ bigchaindb upsert-validator new HHG0IQRybpT6nJMIWWFWhMczCLHt6xcm7eP52GnGuPY= 1 fb7140f03a4ffad899fabbbf655b97e0321add66 --private-key /home/user/.tendermint/config/priv_validator.json
$ bigchaindb election new upsert-validator HHG0IQRybpT6nJMIWWFWhMczCLHt6xcm7eP52GnGuPY= 1 fb7140f03a4ffad899fabbbf655b97e0321add66 --private-key /home/user/.tendermint/config/priv_validator.json
[SUCCESS] Submitted proposal with id: 04a067582cf03eba2b53b82e4adb5ece424474cbd4f7183780855a93ac5e3caa
```
If the command succeeds, it will create an election and return an `election_id`. A successful execution of the above command **doesn't** imply that the validator set will be immediately updated but rather it means the proposal has been succcessfully accepted by the network. Once the `election_id` has been generated the node operator should share this `election_id` with other validators in the network and urge them to approve the proposal. Note that the node operator should themsleves also approve the proposal.
A successful execution of the above command does not imply the validator set has been updated but rather the proposal has been accepted by the network.
Once `election_id` has been generated, the proposer should share it with other validators of the network (e.g. via email) and ask them to approve the proposal.
Note that election proposers do not automatically approve elections by proposing them.
**NOTE**: The election proposal consists of vote tokens allocated to each current validator as per their voting power. Validators then cast their votes to approve the change to the validator set by spending their vote tokens.
For more details about how validator set changes work, refer to [BEP-21](https://github.com/bigchaindb/BEPs/tree/master/21).
#### election new chain-migration
#### upsert-validator approve
Create an election to halt block production, to coordinate on making a Tendermint upgrade with a backwards-incompatible chain.
Approve an election by voting for it. The propsal generated by executing `bigchaindb upsert-valdiator approve ...` can approved by the validators using this command. The validator who is approving the proposal will spend all their votes i.e. if the validator has a network power of `10` then they will cast `10` votes for the proposal.`
Below is the command line syntax and the return value,
```bash
$ bigchaindb upsert-validator approve <election_id> --private-key PATH_TO_YOUR_PRIVATE_KEY
[SUCCESS] Your vote has been submitted
```
- `election_id` is the transaction id of the election the approval should be given for.
- `--private-key` should be the path to Tendermint's private key which can be generally found at `/home/user/.tendermint/config/priv_validator.json`.
Example usage,
```bash
$ bigchaindb upsert-validator approve 04a067582cf03eba2b53b82e4adb5ece424474cbd4f7183780855a93ac5e3caa --private-key /home/user/.tendermint/config/priv_validator.json
[SUCCESS] Your vote has been submitted
```
If the command succeeds a message will be returned stating that the vote was submitted successfully. Once a proposal has been approved by sufficent validators (more than `2/3` of the total voting power) then the proposed change is applied to the network. For example, consider a network wherein the total power is `90` then the proposed changed applied only after `60` (`2/3 * 90`) have been received.
#### upsert-validator show
Retrieves information about an election initiated by `upsert-validator new`.
Below is the command line syntax and the return value,
```bash
$ bigchaindb upsert-validator show ELECTION_ID
public_key=<e_pub_key>
power=<e_power>
node_id=<e_node_id>
$ bigchaindb election new chain-migration --private-key <path-to-the-private-key>
```
- `<path-to-the-private-key>` is the path to the private key of the validator who proposes the election. Tendermint places it at `.tendermint/config/priv_validator.json`.
Example:
```bash
$ bigchaindb election new migration --private-key /home/user/.tendermint/config/priv_validator.json
[SUCCESS] Submitted proposal with id: 04a067582cf03eba2b53b82e4adb5ece424474cbd4f7183780855a93ac5e3caa
```
Concluded chain migration elections halt block production at whichever block height they are approved.
Afterwards, validators are supposed to upgrade Tendermint, set new `chain_id`, `app_hash`, and `validators` (to learn these values, use the [election show](#election-show) command) in `genesis.json`, make and save a MongoDB dump, and restart the system.
For more details about how chain migrations work, refer to [Type 3 scenarios in BEP-42](https://github.com/bigchaindb/BEPs/tree/master/42).
### election approve
Approve an election by voting for it. The command places a `VOTE` transaction, spending all of the validator's vote tokens to the election address.
```bash
$ bigchaindb election approve <election-id> --private-key <path-to-the-private-key>
```
- `election-id` is the election identifier the approval is given for.
- `<path-to-the-private-key>` is the path to the private key of the validator who votes for the election. Tendermint places it at `.tendermint/config/priv_validator.json`.
Example:
```bash
$ bigchaindb election approve 04a067582cf03eba2b53b82e4adb5ece424474cbd4f7183780855a93ac5e3caa --private-key /home/user/.tendermint/config/priv_validator.json
[SUCCESS] Your vote has been submitted
```
Once a proposal has been approved by the sufficient amount of validators (contributing more than `2/3` of the total voting power), the proposed change is applied to the network.
### election show
Retrieves the information about elections.
```bash
$ bigchaindb election show <election-id>
status=<status>
```
The `public_key`, `power`, and `node_id` are the same values used in the `upsert-validator new` command that originally triggered the election. `status` takes three possible values, `ongoing`, if the election has not yet reached a 2/3 majority, `concluded`, if the election reached the 2/3 majority needed to pass, or `inconclusive`, if the validator set changed while the election was in process, rendering it undecidable.
`status` has three possible values:
- `ongoing`, if the election can be concluded but has not yet collected enough votes,
- `concluded`, if the election has been concluded,
- `inconclusive`, if the validator set changed while the election was in process, rendering it undecidable.
After a chain migration is concluded, the `show` command also outputs `chain_id`, `app_hash`, and `validators` for `genesis.json` of the new chain.
## bigchaindb tendermint-version
Show the Tendermint versions supported by BigchainDB server.
```bash
$ bigchaindb tendermint-version
{
"description": "BigchainDB supports the following Tendermint version(s)",
"tendermint": [
"0.22.8"
]
}
```

23
docs/server/source/server-reference/configuration.md
View File

@ -110,7 +110,18 @@ If (no environment variables were set and there's no local config file), or you
`server.bind`, `server.loglevel` and `server.workers`
are settings for the [Gunicorn HTTP server](http://gunicorn.org/), which is used to serve the [HTTP client-server API](../http-client-server-api.html).
`server.bind` is where to bind the Gunicorn HTTP server socket. It's a string. It can be any valid value for [Gunicorn's bind setting](http://docs.gunicorn.org/en/stable/settings.html#bind). If you want to allow IPv4 connections from anyone, on port 9984, use `0.0.0.0:9984`. In a production setting, we recommend you use Gunicorn behind a reverse proxy server. If Gunicorn and the reverse proxy are running on the same machine, then use `localhost:PORT` where PORT is _not_ 9984 (because the reverse proxy needs to listen on port 9984). Maybe use PORT=9983 in that case because we know 9983 isn't used. If Gunicorn and the reverse proxy are running on different machines, then use `A.B.C.D:9984` where A.B.C.D is the IP address of the reverse proxy. There's [more information about deploying behind a reverse proxy in the Gunicorn documentation](http://docs.gunicorn.org/en/stable/deploy.html). (They call it a proxy.)
`server.bind` is where to bind the Gunicorn HTTP server socket. It's a string. It can be any valid value for [Gunicorn's bind setting](http://docs.gunicorn.org/en/stable/settings.html#bind). For example:
* If you want to allow IPv4 connections from anyone, on port 9984, use `0.0.0.0:9984`
* If you want to allow IPv6 connections from anyone, on port 9984, use `[::]:9984`
In a production setting, we recommend you use Gunicorn behind a reverse proxy server such as NGINX. If Gunicorn and the reverse proxy are running on the same machine, then you can use `localhost:9984` (the default value), meaning Gunicorn will talk to the reverse proxy on port 9984. The reverse proxy could then be bound to port 80 (for HTTP) or port 443 (for HTTPS), so that external clients would connect using that port. For example:
[External clients]---(port 443)---[NGINX]---(port 9984)---[Gunicorn / BigchainDB Server]
If Gunicorn and the reverse proxy are running on different machines, then `server.bind` should be `hostname:9984`, where hostname is the IP address or [FQDN](https://en.wikipedia.org/wiki/Fully_qualified_domain_name) of the reverse proxy.
There's [more information about deploying behind a reverse proxy in the Gunicorn documentation](http://docs.gunicorn.org/en/stable/deploy.html). (They call it a proxy.)
`server.loglevel` sets the log level of Gunicorn's Error log outputs. See
[Gunicorn's documentation](http://docs.gunicorn.org/en/latest/settings.html#loglevel)
@ -269,11 +280,11 @@ The full path to the file where logs should be written.
The user running `bigchaindb` must have write access to the
specified path.
**Log rotation:** Log files have a size limit of 200 MB
**Log rotation:** Log files have a size limit of about 200 MB
and will be rotated up to five times.
For example, if we `log.file` is set to `"~/bigchain.log"`, then
For example, if `log.file` is set to `"~/bigchain.log"`, then
logs would always be written to `bigchain.log`. Each time the file
`bigchain.log` reaches 200 MB it would be closed and renamed
`bigchain.log` reaches 200 MB it will be closed and renamed
`bigchain.log.1`. If `bigchain.log.1` and `bigchain.log.2` already exist they
would be renamed `bigchain.log.2` and `bigchain.log.3`. This pattern would be
applied up to `bigchain.log.5` after which `bigchain.log.5` would be
@ -292,7 +303,7 @@ defined by [Python](https://docs.python.org/3.6/library/logging.html#levels),
but case-insensitive for the sake of convenience:
```text
"critical", "error", "warning", "info", "benchmark", "debug", "notset"
"critical", "error", "warning", "info", "debug", "notset"
```
### log.level_logfile
@ -302,7 +313,7 @@ defined by [Python](https://docs.python.org/3.6/library/logging.html#levels),
but case-insensitive for the sake of convenience:
```text
"critical", "error", "warning", "info", "benchmark", "debug", "notset"
"critical", "error", "warning", "info", "debug", "notset"
```
### log.datefmt_console

69
docs/server/source/simple-deployment-template/deploy-a-machine.md Normal file
View File

@ -0,0 +1,69 @@
<!---
Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# Deploy a Machine for Your BigchainDB Node
The first step is to deploy a machine for your BigchainDB node.
It might be a virtual machine (VM) or a real machine.
If you follow this simple deployment template, all your node's
software will run on that one machine.
We don't make any assumptions about _where_ you run the machine.
It might be in Azure, AWS, your data center or a Raspberry Pi.
## IP Addresses
The following instructions assume all the nodes
in the network (including yours) have public IP addresses.
(A BigchainDB network _can_ be run inside a private network,
using private IP addresses, but we don't cover that here.)
## Operating System
**Use Ubuntu 18.04 or Ubuntu Server 18.04 as the operating system.**
Similar instructions will work on other versions of Ubuntu,
and other recent Debian-like Linux distros,
but you may have to change the names of the packages,
or install more packages.
## Network Security Group
If your machine is in AWS or Azure, for example, _and_
you want users to connect to BigchainDB via HTTPS,
then you should configure its network security group
to allow all incoming and outgoing traffic for:
* TCP on port 22 (SSH)
* TCP on port 80 (HTTP)
* TCP on port 443 (HTTPS)
* Any protocol on port 26656 (Tendermint P2P)
If you don't care about HTTPS, then forget about port 443,
and replace port 80 with port 9984 (the default BigchainDB HTTP port).
## Update Your System
SSH into your machine and update all its OS-level packages:
```
sudo apt update
sudo apt full-upgrade
```
## Node Security
If you're going to use your node in production,
then you should take additional steps to secure it.
We don't cover that here; there are many books and websites
about securing Linux machines.
## DNS Setup
* Register a domain name for your BigchainDB node, such as `example.com`
* Pick a subdomain of that domain for your BigchainDB node, such as `bnode.example.com`
* Create a DNS "A Record" pointing your chosen subdomain (such as `bnode.example.com`)
at your machine's IP address.

21
docs/server/source/simple-deployment-template/index.rst
View File

@ -8,13 +8,30 @@
Simple Deployment Template
==========================
This section describes *one* way to deploy a BigchainDB network.
You can modify it as you see fit.
This section describes *one* way to deploy a BigchainDB network
(i.e. a set of connected BigchainDB nodes).
You can modify this simple deployment template as you see fit.
It's "simple" in the sense that each BigchainDB node is installed
and run on a single virtual machine (or real machine).
We also have a :ref:`kubernetes-deployment-template` (not simple).
**Note 1:** These instructions will also work for a "network" with only one node.
If you want your network to be able to handle the failure or misbehavior
of one node, then your network must have at least four nodes.
Nodes can be added or removed from a network after is it up and running.
**Note 2:** If you want to set up a node or network
so that you can contribute to developing and testing the BigchainDB code,
then see
`the docs about contributing to BigchainDB <https://docs.bigchaindb.com/projects/contributing/en/latest/index.html>`_
.
.. toctree::
:maxdepth: 1
deploy-a-machine
set-up-nginx
set-up-node-software
network-setup
tips
troubleshooting

314
docs/server/source/simple-deployment-template/network-setup.md
View File

@ -6,132 +6,23 @@ Code is Apache-2.0 and docs are CC-BY-4.0
# How to Set Up a BigchainDB Network
Note 1: These instructions will also work for a "network" with only one node.
Until now, everything could be done by a node operator, by themselves.
Now the node operators, also called **Members**, must share some information
with each other, so they can form a network.
Note 2: You might not need to set up your own network yet. You should start by creating a proof-of-concept app that writes to [the BigchainDB Testnet](https://testnet.bigchaindb.com/), and if that goes well, then you can look into setting up your own network.
There is one special Member who helps coordinate everyone: the **Coordinator**.
Note 3: If you want to set up a node or network so that you can contribute to developing and testing the BigchainDB code, then see [the docs about contributing to BigchainDB](https://docs.bigchaindb.com/projects/contributing/en/latest/index.html).
## Member: Share hostname, pub_key.value and node_id
<hr>
Each BigchainDB node is identified by its:
The process to create a network is both *social* and *technical*: *social* because someone (that we will call **Coordinator**) needs to find at least three other **Members** willing to join the network, and coordinate the effort; *technical* because each member of the network needs to set up a machine running BigchainDB. (Note: a Coordinator is a Member as well.)
* `hostname`, i.e. the node's DNS subdomain, such as `bnode.example.com`, or its IP address, such as `46.145.17.32`
* Tendermint `pub_key.value`
* Tendermint `node_id`
A **BigchainDB Network** (or just *Network*) is a set of **4 or more BigchainDB Nodes** (or *Nodes*). Every Node is independently managed by a Member, and runs an instance of the [BigchainDB Server software][bdb:software]. At the **Genesis** of a Network, there **MUST** be at least **4** Nodes ready to connect. After the Genesis, a Network can dynamically add new Nodes or remove old Nodes.
A Network will stop working if more than one third of the Nodes are down or faulty _in any way_. The bigger a Network, the more failures it can handle. A Network of size 4 can tolerate only 1 failure, so if 3 out of 4 Nodes are online, everything will work as expected. Eventually, the Node that was offline will automatically sync with the others.
## Before We Start
This tutorial assumes you have basic knowledge on how to manage a GNU/Linux machine.
**Please note: The commands on this page work on Ubuntu 18.04. Similar commands will work on other versions of Ubuntu, and other recent Debian-like Linux distros, but you may have to change the names of the packages, or install more packages.**
We don't make any assumptions about **where** you run the Node.
You can run BigchainDB Server on a Virtual Machine on the cloud, on a machine in your data center, or even on a Raspberry Pi. Just make sure that your Node is reachable by the other Nodes. Here's a **non-exhaustive list of examples**:
- **good**: all Nodes running in the cloud using public IPs.
- **bad**: some Nodes running in the cloud using public IPs, some Nodes in a private network.
- **good**: all Nodes running in a private network.
The rule of thumb is: if Nodes can ping each other, then you are good to go.
The next sections are labelled with **Member** or **Coordinator**, depending on who should follow the instructions. Remember, a Coordinator is also a Member.
## Member: Set Up a Node
Every Member in the Network **must** set up its own Node. The process consists of installing three components, BigchainDB Server, Tendermint Core, and MongoDB, and configuring the firewall.
**Important note on security: it's up to the Member to harden their system.**
### Install the Required Software
Make sure your system is up to date.
```
sudo apt update
sudo apt full-upgrade
```
#### Install BigchainDB Server
BigchainDB Server requires **Python 3.6+**, so make sure your system has it. Install the required packages:
```
# For Ubuntu 18.04:
sudo apt install -y python3-pip libssl-dev
# Ubuntu 16.04, and other Linux distros, may require other packages or more packages
```
Now install the latest version of BigchainDB. You can find the latest version by going to the [BigchainDB project release history page on PyPI][bdb:pypi]. For example, to install version 2.0.0b3, you would do:
```
# Change 2.0.0b3 to the latest version as explained above:
sudo pip3 install bigchaindb==2.0.0b3
```
Check that you installed the correct version of BigchainDB Server using `bigchaindb --version`.
#### Install (and Start) MongoDB
Install a recent version of MongoDB. BigchainDB Server requires version 3.4 or newer.
```
sudo apt install mongodb
```
If you install MongoDB using the above command (which installs the `mongodb` package), it also configures MongoDB, starts MongoDB (in the background), and installs a MongoDB startup script (so that MongoDB will be started automatically when the machine is restarted).
Note: The `mongodb` package is _not_ the official MongoDB package from MongoDB the company. If you want to install the official MongoDB package, please see [the MongoDB documentation](https://docs.mongodb.com/manual/installation/). Note that installing the official package _doesn't_ also start MongoDB.
#### Install Tendermint
Install a [recent version of Tendermint][tendermint:releases]. BigchainDB Server requires version 0.22.8 or newer.
```
sudo apt install -y unzip
wget https://github.com/tendermint/tendermint/releases/download/v0.22.8/tendermint_0.22.8_linux_amd64.zip
unzip tendermint_0.22.8_linux_amd64.zip
rm tendermint_0.22.8_linux_amd64.zip
sudo mv tendermint /usr/local/bin
```
### Set Up the Firewall
Make sure to accept inbound connections on ports `9984`, `9985`, and `26656`. You might also want to add port `22` so that you can continue to access the machine via SSH.
```
sudo ufw allow 22/tcp
sudo ufw allow 9984/tcp
sudo ufw allow 9985/tcp
sudo ufw allow 26656/tcp
sudo ufw enable
```
Some cloud providers, like Microsoft Azure, require you to change "security groups" (virtual firewalls) using their portal or other APIs (such as their CLI).
## Member: Configure BigchainDB Server
To configure BigchainDB Server, run:
```
bigchaindb configure
```
The first question is ``API Server bind? (default `localhost:9984`)``. To expose the API to the public, bind the API Server to `0.0.0.0:9984`. Unless you have specific needs, you can keep the default value for all other questions.
## Member: Generate the Private Key and Node id
A Node is identified by the triplet `<hostname, node_id, public_key>`.
As a Member, it's your duty to create and store securely your private key, and share your `hostname`, `node_id`, and `public_key` with the other members of the network.
To generate all of that, run:
```
tendermint init
```
The `public_key` is stored in the file `.tendermint/config/priv_validator.json`, and it should look like:
The Tendermint `pub_key.value` is stored
in the file `$HOME/.tendermint/config/priv_validator.json`.
That file should look like:
```json
{
@ -150,7 +41,7 @@ The `public_key` is stored in the file `.tendermint/config/priv_validator.json`,
}
```
To extract your `node_id`, run the command:
To get your Tendermint `node_id`, run the command:
```
tendermint show_node_id
@ -158,62 +49,60 @@ tendermint show_node_id
An example `node_id` is `9b989cd5ac65fec52652a457aed6f5fd200edc22`.
An example hostname is `charlie5.cloudservers.company.com`. You can also use a public IP addres, like `46.145.17.32`, instead of a hostname, but make sure that IP address won't change.
**Share your `hostname`, `pub_key.value` and `node_id` with all other Members.**
Share the `node_id`, `pub_key.value` and hostname of your Node with all other Members.
## Coordinator: Create & Share the genesis.json File
**Important note on security: each Member should take extra steps to verify the public keys they receive from the other Members have not been tampered with, e.g. a key signing party would be one way.**
## Coordinator: Initialize the Network
At this point the Coordinator should have received the data from all the Members, and should combine them in the `.tendermint/config/genesis.json` file:
At this point the Coordinator should have received the data
from all the Members, and should combine them in the file
`$HOME/.tendermint/config/genesis.json`:
```json
{
{
"genesis_time":"0001-01-01T00:00:00Z",
"chain_id":"test-chain-la6HSr",
"consensus_params":{
"block_size_params":{
"consensus_params":{
"block_size_params":{
"max_bytes":"22020096",
"max_txs":"10000",
"max_gas":"-1"
},
"tx_size_params":{
"tx_size_params":{
"max_bytes":"10240",
"max_gas":"-1"
},
"block_gossip_params":{
"block_gossip_params":{
"block_part_size_bytes":"65536"
},
"evidence_params":{
"evidence_params":{
"max_age":"100000"
}
},
"validators":[
{
"pub_key":{
"validators":[
{
"pub_key":{
"type":"AC26791624DE60",
"value":"<Member 1 public key>"
},
"power":10,
"name":"<Member 1 name>"
},
{
"pub_key":{
{
"pub_key":{
"type":"AC26791624DE60",
"value":"<Member 2 public key>"
},
"power":10,
"name":"<Member 2 name>"
},
{
"...":{
{
"...":{
},
},
{
"pub_key":{
{
"pub_key":{
"type":"AC26791624DE60",
"value":"<Member N public key>"
},
@ -225,9 +114,12 @@ At this point the Coordinator should have received the data from all the Members
}
```
**Note:** `consensus_params` in the `genesis.json` are default values for Tendermint consensus.
**Note:** The above `consensus_params` in the `genesis.json`
are default values.
The new `genesis.json` file contains the data that describes the Network. The key `name` is the Member's moniker; it can be any valid string, but put something human-readable like `"Alice's Node Shop"`.
The new `genesis.json` file contains the data that describes the Network.
The key `name` is the Member's moniker; it can be any valid string,
but put something human-readable like `"Alice's Node Shop"`.
At this point, the Coordinator must share the new `genesis.json` file with all Members.
@ -235,23 +127,32 @@ At this point, the Coordinator must share the new `genesis.json` file with all M
At this point the Member should have received the `genesis.json` file.
**Important note on security: each Member should verify that the `genesis.json` file contains the correct public keys.**
The Member must copy the `genesis.json` file
into their local `$HOME/.tendermint/config` directory.
Every Member now shares the same `chain_id` and `genesis_time` (used to identify the Network),
and the same list of `validators`.
The Member must copy the `genesis.json` file in the local `.tendermint/config` directory. Every Member now shares the same `chain_id`, `genesis_time`, used to identify the Network, and the same list of `validators`.
The Member must edit the `.tendermint/config/config.toml` file and make the following changes:
Each Member must edit their `$HOME/.tendermint/config/config.toml` file
and make the following changes:
```
...
moniker = "Name of our node"
create_empty_blocks = false
...
log_level = "main:info,state:info,*:error"
persistent_peers = "<Member 1 node id>@<Member 1 hostname>:26656,\
<Member 2 node id>@<Member 2 hostname>:26656,\
<Member N node id>@<Member N hostname>:26656,"
send_rate = 102400000
recv_rate = 102400000
recheck = false
```
Note: The list of `persistent_peers` doesn't have to include all nodes
in the network.
## Member: Start MongoDB
If you installed MongoDB using `sudo apt install mongodb`, then MongoDB should already be running in the background. You can check using `systemctl status mongodb`.
@ -260,28 +161,9 @@ If MongoDB isn't running, then you can start it using the command `mongod`, but
If you installed MongoDB using `sudo apt install mongodb`, then a MongoDB startup script should already be installed (so MongoDB will start automatically when the machine is restarted). Otherwise, you should install a startup script for MongoDB.
## Member: Start BigchainDB and Tendermint
## Member: Start BigchainDB and Tendermint Using Monit
If you want to use a process manager, jump to the [next section](member-start-bigchaindb-and-tendermint-using-monit).
To start BigchainDB, one uses the command `bigchaindb start` but that will run it in the foreground. If you want to run it in the background (so it will continue running after you logout), you can use `nohup`, `tmux`, or `screen`. For example, `nohup bigchaindb start 2>&1 > bigchaindb.log &`
The _recommended_ approach is to create a startup script for BigchainDB, so it will start right after the boot of the operating system. (As mentioned earlier, MongoDB should already have a startup script.)
To start Tendermint, one uses the command `tendermint node` but that will run it in the foreground. If you want to run it in the background (so it will continue running after you logout), you can use `nohup`, `tmux`, or `screen`. For example, `nohup tendermint node 2>&1 > tendermint.log &`
The _recommended_ approach is to create a startup script for Tendermint, so it will start right after the boot of the operating system.
Note: We'll share some example startup scripts in the future. This document is a work in progress.
If you followed the recommended approach and created startup scripts for BigchainDB and Tendermint, then you can reboot the machine now. MongoDB, BigchainDB and Tendermint should all start.
### Member: Start BigchainDB and Tendermint using Monit
This section describes how to manage the BigchainDB and Tendermint processes using [Monit][monit] - a small open-source utility for managing and monitoring Unix processes.
This section assumes that you followed the guide down to the [start MongoDB section](#member-start-mongodb) inclusive.
This section describes how to manage the BigchainDB and Tendermint processes using [Monit][monit], a small open-source utility for managing and monitoring Unix processes. BigchainDB and Tendermint are managed together, because if BigchainDB is stopped (or crashes) and is restarted, *Tendermint won't try reconnecting to it*. (That's not a bug. It's just how Tendermint works.)
Install Monit:
@ -289,103 +171,33 @@ Install Monit:
sudo apt install monit
```
If you installed the `bigchaindb` Python package, you should have the `bigchaindb-monit-config` script in your `PATH` now.
Run the script:
If you installed the `bigchaindb` Python package as above, you should have the `bigchaindb-monit-config` script in your `PATH` now. Run the script to build a configuration file for Monit:
```
bigchaindb-monit-config
```
The script builds a configuration file for Monit.
Run Monit as a daemon, instructing it to wake up every second to check on processes:
```
monit -d 1
```
It will run the processes and restart them when they crash. If the root `bigchaindb_` process crashes, Monit will also restart the Tendermint process.
Monit will run the BigchainDB and Tendermint processes and restart them when they crash. If the root `bigchaindb_` process crashes, Monit will also restart the Tendermint process.
Check the status by running `monit status` or `monit summary`.
You can check the status by running `monit status` or `monit summary`.
By default, it will collect program logs into the `~/.bigchaindb-monit/logs` folder.
Consult `monit -h` or [the Monit documentation][monit-manual] to know more about the operational power you've just got the taste of.
To learn more about Monit, use `monit -h` (help) or read [the Monit documentation][monit-manual].
Check `bigchaindb-monit-config -h` if you want to arrange a different folder for logs or some of the Monit internal artifacts.
If you want to start and manage the BigchainDB and Tendermint processes yourself, then look inside the file [bigchaindb/pkg/scripts/bigchaindb-monit-config](https://github.com/bigchaindb/bigchaindb/blob/master/pkg/scripts/bigchaindb-monit-config) to see how *it* starts BigchainDB and Tendermint.
## How Others Can Access Your Node
If you followed the above instructions, then your node should be publicly-accessible with BigchainDB Root URL `http://hostname:9984` (where hostname is something like `bdb7.canada.vmsareus.net` or `17.122.200.76`). That is, anyone can interact with your node using the [BigchainDB HTTP API](http-client-server-api.html) exposed at that address. The most common way to do that is to use one of the [BigchainDB Drivers](./drivers-clients/index.html).
## Troubleshooting
To check which nodes your node is connected to (via Tendermint protocols), do:
```text
# if you don't jq installed, then install it
sudo apt install jq
# then do
curl -s localhost:26657/net_info | jq ".result.peers[].node_info | {id, listen_addr, moniker}"
```
Tendermint has other endpoints besides `/net_info`: see [the Tendermint RPC docs](https://tendermint.github.io/slate/?shell#introduction).
## Refreshing Your Node
If you want to refresh your node back to a fresh empty state, then your best bet is to terminate it and deploy a new virtual machine, but if that's not an option, then you can:
- drop the `bigchain` database in MongoDB using `bigchaindb drop` (but that only works if MongoDB is running)
- reset Tendermint using `tendermint unsafe_reset_all`
- delete the directory `$HOME/.tendermint`
## Shutting down BigchainDB
If you want to stop/kill BigchainDB, you can do so by sending `SIGINT`, `SIGQUIT` or `SIGTERM` to the running BigchainDB
process(es). Depending on how you started BigchainDB i.e. foreground or background. e.g. you started BigchainDB in the background as mentioned above in the guide:
```bash
$ nohup bigchaindb start 2>&1 > bigchaindb.log &
$ # Check the PID of the main BigchainDB process
$ ps -ef | grep bigchaindb
<user> *<pid> <ppid> <C> <STIME> <tty> <time> bigchaindb
<user> <pid> <ppid>* <C> <STIME> <tty> <time> gunicorn: master [bigchaindb_gunicorn]
<user> <pid> <ppid>* <C> <STIME> <tty> <time> bigchaindb_ws
<user> <pid> <ppid>* <C> <STIME> <tty> <time> bigchaindb_ws_to_tendermint
<user> <pid> <ppid>* <C> <STIME> <tty> <time> bigchaindb_exchange
<user> <pid> <ppid> <C> <STIME> <tty> <time> gunicorn: worker [bigchaindb_gunicorn]
<user> <pid> <ppid> <C> <STIME> <tty> <time> gunicorn: worker [bigchaindb_gunicorn]
<user> <pid> <ppid> <C> <STIME> <tty> <time> gunicorn: worker [bigchaindb_gunicorn]
<user> <pid> <ppid> <C> <STIME> <tty> <time> gunicorn: worker [bigchaindb_gunicorn]
<user> <pid> <ppid> <C> <STIME> <tty> <time> gunicorn: worker [bigchaindb_gunicorn]
...
$ # Send any of the above mentioned signals to the parent/root process(marked with `*` for clarity)
# Sending SIGINT
$ kill -2 <bigchaindb_parent_pid>
$ # OR
# Sending SIGTERM
$ kill -15 <bigchaindb_parent_pid>
$ # OR
# Sending SIGQUIT
$ kill -3 <bigchaindb_parent_pid>
# If you want to kill all the processes by name yourself
$ pgrep bigchaindb | xargs kill -9
```
If you started BigchainDB in the foreground, a `Ctrl + C` or `Ctrl + Z` would shut down BigchainDB.
## Member: Dynamically Add a New Member to the Network
TBD.
If you followed the above instructions, then your node should be publicly-accessible with BigchainDB Root URL `https://hostname` or `http://hostname:9984`. That is, anyone can interact with your node using the [BigchainDB HTTP API](../../http-client-server-api.html) exposed at that address. The most common way to do that is to use one of the [BigchainDB Drivers](../../drivers-clients/index.html).
[bdb:software]: https://github.com/bigchaindb/bigchaindb/
[bdb:pypi]: https://pypi.org/project/BigchainDB/#history

42
docs/server/source/simple-deployment-template/set-up-nginx.md Normal file
View File

@ -0,0 +1,42 @@
<!---
Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# Set Up NGINX
If you don't want HTTPS
(for communications between the external world and your node),
then you can skip all the NGINX steps on this page.
Note: This simple deployment template uses NGINX for more than just HTTPS.
For example, it also does basic rate limiting.
## Install NGINX
SSH into your machine and install NGINX:
```
sudo apt update
sudo apt install nginx
```
## Configure & Reload NGINX
Get an SSL certificate for your node's subdomain (such as `bnode.example.com`).
* Copy the SSL private key into `/etc/nginx/ssl/cert.key`
* Create a "PEM file" (text file) by concatenating your SSL certificate with all intermediate certificates (_in that order, with the intermediate certs last_).
* Copy that PEM file into `/etc/nginx/ssl/cert.pem`
* In the
[bigchaindb/bigchaindb repository on GitHub](https://github.com/bigchaindb/bigchaindb),
find the file `nginx/nginx.conf` and copy its contents to
`/etc/nginx/nginx.conf` on your machine (i.e. replace the existing file there).
* Edit that file (`/etc/nginx/nginx.conf`): replace the two instances of
the string `example.testnet2.com`
with your chosen subdomain (such as `bnode.example.com`).
* Reload NGINX by doing:
```
sudo service nginx reload
```

104
docs/server/source/simple-deployment-template/set-up-node-software.md Normal file
View File

@ -0,0 +1,104 @@
<!---
Copyright BigchainDB GmbH and BigchainDB contributors
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# Set Up BigchainDB, MongoDB and Tendermint
We now install and configure software that must run
in every BigchainDB node: BigchainDB Server,
MongoDB and Tendermint.
## Install BigchainDB Server
BigchainDB Server requires **Python 3.6+**, so make sure your system has it.
Install the required OS-level packages:
```
# For Ubuntu 18.04:
sudo apt install -y python3-pip libssl-dev
# Ubuntu 16.04, and other Linux distros, may require other packages or more packages
```
Now install the latest version of BigchainDB Server.
You can find the latest version by going
to the [BigchainDB project release history page on PyPI][bdb:pypi].
For example, to install version 2.0.0b9, you would do:
```
# Change 2.0.0b9 to the latest version as explained above:
sudo pip3 install bigchaindb==2.0.0b9
```
Check that you installed the correct version of BigchainDB Server using `bigchaindb --version`.
## Configure BigchainDB Server
To configure BigchainDB Server, run:
```
bigchaindb configure
```
The first question is ``API Server bind? (default `localhost:9984`)``.
* If you're using NGINX (e.g. if you want HTTPS),
then accept the default value (`localhost:9984`).
* If you're not using NGINX, then enter the value `0.0.0.0:9984`
You can accept the default value for all other BigchainDB config settings.
If you're using NGINX, then you should edit your BigchainDB config file
(in `$HOME/.bigchaindb` by default) and set the following values
under `"wsserver"`:
```
"advertised_scheme": "wss",
"advertised_host": "bnode.example.com",
"advertised_port": 443
```
where `bnode.example.com` should be replaced by your node's actual subdomain.
## Install (and Start) MongoDB
Install a recent version of MongoDB.
BigchainDB Server requires version 3.4 or newer.
```
sudo apt install mongodb
```
If you install MongoDB using the above command (which installs the `mongodb` package),
it also configures MongoDB, starts MongoDB (in the background),
and installs a MongoDB startup script
(so that MongoDB will be started automatically when the machine is restarted).
Note: The `mongodb` package is _not_ the official MongoDB package
from MongoDB the company. If you want to install the official MongoDB package,
please see
[the MongoDB documentation](https://docs.mongodb.com/manual/installation/).
Note that installing the official package _doesn't_ also start MongoDB.
## Install Tendermint
The version of BigchainDB Server described in these docs only works well
with Tendermint 0.22.8 (not a higher version number). Install that:
```
sudo apt install -y unzip
wget https://github.com/tendermint/tendermint/releases/download/v0.22.8/tendermint_0.22.8_linux_amd64.zip
unzip tendermint_0.22.8_linux_amd64.zip
rm tendermint_0.22.8_linux_amd64.zip
sudo mv tendermint /usr/local/bin
```
## Start Configuring Tendermint
You won't be able to finish configuring Tendermint until you have some information
from the other nodes in the network, but you can start by doing:
```
tendermint init
```

59
docs/server/source/simple-deployment-template/tips.md Normal file
View File

@ -0,0 +1,59 @@
# Tips
## Refreshing Your Node
If you want to refresh your node back to a fresh empty state, then your best bet is to terminate it and deploy a new machine, but if that's not an option, then you can:
* drop the `bigchain` database in MongoDB using `bigchaindb drop` (but that only works if MongoDB is running)
* reset Tendermint using `tendermint unsafe_reset_all`
* delete the directory `$HOME/.tendermint`
## Shutting Down BigchainDB
If you want to stop/kill BigchainDB, you can do so by sending `SIGINT`, `SIGQUIT` or `SIGTERM` to the running BigchainDB
process(es). Depending on how you started BigchainDB i.e. foreground or background. e.g. you started BigchainDB in the background as mentioned above in the guide:
```bash
$ nohup bigchaindb start 2>&1 > bigchaindb.log &
$ # Check the PID of the main BigchainDB process
$ ps -ef | grep bigchaindb
<user> *<pid> <ppid> <C> <STIME> <tty> <time> bigchaindb
<user> <pid> <ppid>* <C> <STIME> <tty> <time> gunicorn: master [bigchaindb_gunicorn]
<user> <pid> <ppid>* <C> <STIME> <tty> <time> bigchaindb_ws
<user> <pid> <ppid>* <C> <STIME> <tty> <time> bigchaindb_ws_to_tendermint
<user> <pid> <ppid>* <C> <STIME> <tty> <time> bigchaindb_exchange
<user> <pid> <ppid> <C> <STIME> <tty> <time> gunicorn: worker [bigchaindb_gunicorn]
<user> <pid> <ppid> <C> <STIME> <tty> <time> gunicorn: worker [bigchaindb_gunicorn]
<user> <pid> <ppid> <C> <STIME> <tty> <time> gunicorn: worker [bigchaindb_gunicorn]
<user> <pid> <ppid> <C> <STIME> <tty> <time> gunicorn: worker [bigchaindb_gunicorn]
<user> <pid> <ppid> <C> <STIME> <tty> <time> gunicorn: worker [bigchaindb_gunicorn]
...
$ # Send any of the above mentioned signals to the parent/root process(marked with `*` for clarity)
# Sending SIGINT
$ kill -2 <bigchaindb_parent_pid>
$ # OR
# Sending SIGTERM
$ kill -15 <bigchaindb_parent_pid>
$ # OR
# Sending SIGQUIT
$ kill -3 <bigchaindb_parent_pid>
# If you want to kill all the processes by name yourself
$ pgrep bigchaindb | xargs kill -9
```
If you started BigchainDB in the foreground, a `Ctrl + C` or `Ctrl + Z` would shut down BigchainDB.
## Member: Dynamically Add or Remove Validators
One member can make a proposal to call an election to add a validator, remove a validator, or change the voting power of a validator. They then share the election/proposal ID with all the other members. Once more than 2/3 of the voting power votes yes, the proposed change comes into effect. The commands to create a new election/proposal, to approve an election/proposal, and to get the current status of an election/proposal can be found in the documentation about the [bigchaindb election](../server-reference/bigchaindb-cli.html#bigchaindb-election) subcommands.
## Logging and Log Rotation
See the page in the Appendices about [logging and log rotation](../appendices/log-rotation.html).

32
docs/server/source/simple-deployment-template/troubleshooting.md Normal file
View File

@ -0,0 +1,32 @@
# Troubleshooting
## General Tips
- Check the BigchainDB, Tendermint and MongoDB logs.
For help with that, see the page about [Logging and Log Rotation](../appendices/log-rotation.html).
- Try Googling the error message.
## Resolving Tendermint Connectivity Problems
To check which nodes your node is connected to (via Tendermint protocols), do:
```text
# if you don't have jq installed, then install it
sudo apt install jq
# then do
curl -s localhost:26657/net_info | jq ".result.peers[].node_info | {id, listen_addr, moniker}"
```
Note: Tendermint has other endpoints besides `/net_info`: see [the Tendermint RPC docs](https://tendermint.github.io/slate/?shell#introduction).
If you're running your network inside a [private network](https://en.wikipedia.org/wiki/Private_network), e.g. with IP addresses of the form 192.168.x.y, then you may have to change the following setting in `config.toml`:
```text
addr_book_strict = false
```
## Other Problems
See the [Tendermint tips in the vrde/notes repository](https://github.com/vrde/notes/tree/master/tendermint).
If you're stuck, maybe [file a new issue on GitHub](https://github.com/bigchaindb/bigchaindb/issues/new). If your problem occurs often enough, we'll write about it here.

2
k8s/bigchaindb/bigchaindb-ss.yaml
View File

@ -158,7 +158,7 @@ spec:
timeoutSeconds: 15
# BigchainDB container
- name: bigchaindb
image: bigchaindb/bigchaindb:2.0.0-beta5
image: bigchaindb/bigchaindb:2.0.0-beta9
imagePullPolicy: Always
args:
- start

2
k8s/dev-setup/bigchaindb.yaml
View File

@ -38,7 +38,7 @@ spec:
terminationGracePeriodSeconds: 10
containers:
- name: bigchaindb
image: bigchaindb/bigchaindb:2.0.0-beta5
image: bigchaindb/bigchaindb:2.0.0-beta9
imagePullPolicy: Always
args:
- start

2
k8s/nginx-https-web-proxy/container/nginx.conf.template
View File

@ -62,7 +62,7 @@ http {
server_name "PROXY_FQDN";
ssl_certificate /etc/nginx/ssl/cert.pem;
ssl_certificate_key /etc/nginx/ssl/cert.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
underscores_in_headers on;

2
k8s/nginx-https/container/nginx.conf.template
View File

@ -66,7 +66,7 @@ http {
ssl_certificate /etc/nginx/ssl/cert.pem;
ssl_certificate_key /etc/nginx/ssl/cert.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
underscores_in_headers on;

2
k8s/nginx-https/container/nginx.conf.threescale.template
View File

@ -68,7 +68,7 @@ http {
ssl_certificate /etc/nginx/ssl/cert.pem;
ssl_certificate_key /etc/nginx/ssl/cert.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
underscores_in_headers on;

Some files were not shown because too many files have changed in this diff Show More