Mirroristas/planetmint
2
0
Fork 0
mirror of https://github.com/planetmint/planetmint.git synced 2025-06-02 20:26:38 +00:00
Code Issues Packages Projects Releases Wiki Activity

Initial Migration

Browse Source
This commit is contained in:
Sangat Das 2022-01-13 16:29:35 +00:00
parent 3ad513ed14
commit ed1d4b82ad
454 changed files with 42542 additions and 723 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
.ci/entrypoint.sh Executable file
View File

@ -0,0 +1,14 @@
#!/bin/bash
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
set -e -x
if [[ ${PLANETMINT_CI_ABCI} == 'enable' ]]; then
sleep 3600
else
bigchaindb -l DEBUG start
fi

12
.ci/travis-after-success.sh Executable file
View File

@ -0,0 +1,12 @@
#!/bin/bash
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
set -e -x
if [[ -z ${TOXENV} ]] && [[ ${PLANETMINT_CI_ABCI} != 'enable' ]] && [[ ${PLANETMINT_ACCEPTANCE_TEST} != 'enable' ]]; then
codecov -v -f htmlcov/coverage.xml
fi

16
.ci/travis-before-install.sh Executable file
View File

@ -0,0 +1,16 @@
#!/bin/bash
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
if [[ -z ${TOXENV} ]]; then
sudo apt-get update
sudo apt-get -y -o Dpkg::Options::="--force-confnew" install docker-ce
sudo rm /usr/local/bin/docker-compose
curl -L https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-`uname -s`-`uname -m` > docker-compose
chmod +x docker-compose
sudo mv docker-compose /usr/local/bin
fi

18
.ci/travis-before-script.sh Executable file
View File

@ -0,0 +1,18 @@
#!/bin/bash
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
set -e -x
if [[ -z ${TOXENV} ]]; then
if [[ ${PLANETMINT_CI_ABCI} == 'enable' ]]; then
docker-compose up -d bigchaindb
else
docker-compose up -d bdb
fi
fi

21
.ci/travis-install.sh Executable file
View File

@ -0,0 +1,21 @@
#!/bin/bash
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
set -e -x
pip install --upgrade pip
if [[ -n ${TOXENV} ]]; then
pip install --upgrade tox
elif [[ ${PLANETMINT_CI_ABCI} == 'enable' ]]; then
docker-compose build --no-cache --build-arg abci_status=enable bigchaindb
elif [[ $PLANETMINT_INTEGRATION_TEST == 'enable' ]]; then
docker-compose build bigchaindb python-driver
else
docker-compose build --no-cache bigchaindb
pip install --upgrade codecov
fi

18
.ci/travis_script.sh Executable file
View File

@ -0,0 +1,18 @@
#!/bin/bash
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
set -e -x
if [[ -n ${TOXENV} ]]; then
tox -e ${TOXENV}
elif [[ ${PLANETMINT_CI_ABCI} == 'enable' ]]; then
docker-compose exec bigchaindb pytest -v -m abci
elif [[ ${PLANETMINT_ACCEPTANCE_TEST} == 'enable' ]]; then
./run-acceptance-test.sh
else
docker-compose exec bigchaindb pytest -v --cov=bigchaindb --cov-report xml:htmlcov/coverage.xml
fi

9
.dockerignore Normal file
View File

@ -0,0 +1,9 @@
.cache/
.coverage
.eggs/
.git/
.gitignore
.ropeproject/
.travis.yml
Planetmint.egg-info/
dist/

14
.github/CONTRIBUTING.md vendored Normal file
View File

@ -0,0 +1,14 @@
<!---
Copyright © 2020 Interplanetary Database Association e.V.,
Planetmint and IPDB software contributors.
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# How to Contribute to the Planetmint Project
There are many ways you can contribute to the Planetmint project, some very easy and others more involved.
All of that is documented elsewhere: go to the "[Contributing to Planetmint" docs on ReadTheDocs](https://docs.bigchaindb.com/projects/contributing/en/latest/index.html).
Note: GitHub automatically links to this file (`.github/CONTRIBUTING.md`) when a contributor creates a new issue or pull request, so you shouldn't delete it. Just use it to point people to full and proper help elsewhere.

20
.github/ISSUE_TEMPLATE.md vendored Normal file
View File

@ -0,0 +1,20 @@
# Do you want to:
- make a bug report? Then read below about what should go in a bug report.
- make a feature request or proposal? Then read [the page about how to make a feature request or proposal](https://docs.bigchaindb.com/projects/contributing/en/latest/ways-can-contribute/make-a-feature-request-or-proposal.html).
- ask a question about Planetmint? Then [go to Gitter](https://gitter.im/bigchaindb/bigchaindb) (our chat room) and ask it there.
- share your neat idea or realization? Then [go to Gitter](https://gitter.im/bigchaindb/bigchaindb) (our chat room) and share it there.
# What Should Go in a Bug Report
- What computer are you on (hardware)?
- What operating system are you using, including version. e.g. Ubuntu 14.04? Fedora 23?
- What version of Planetmint software were you using? Is that the latest version?
- What, exactly, did you do to get to the point where you got stuck? Describe all the steps so we can get there too. Show screenshots or copy-and-paste text to GitHub.
- Show what actually happened.
- Say what you tried to do to resolve the problem.
- Provide details to convince us that it matters to you. Is it for a school project, a job, a contract with a deadline, a child who needs it for Christmas?
We will do our best but please understand that we don't have time to help everyone, especially people who don't care to help us help them. "It doesn't work." is not going to get any reaction from us. We need _details_.
Tip: Use Github code block formatting to make code render pretty in GitHub. To do that, put three backticks followed by a string to set the type of code (e.g. `Python`), then the code, and then end with three backticks. There's more information about [inserting code blocks](https://help.github.com/articles/creating-and-highlighting-code-blocks/) in the GitHub help pages.

30
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@ -0,0 +1,30 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
**Expected behavior**
A clear and concise description of what you expected to happen.
**Logs or terminal output**
If applicable, add add textual content to help explain your problem.
**Desktop (please complete the following information):**
- Distribution: [e.g. Ubuntu 18.04]
- Bigchaindb version:
- Tendermint version:
- Mongodb version:
- Python full version: [e.g. Python 3.6.6]
**Additional context**
Add any other context about the problem here.

18
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View File

@ -0,0 +1,18 @@
Make sure the title of this pull request has the form:
**Problem: A short statement of the problem.**
## Solution
A short statement about how this PR solves the **Problem**.
## Issues Resolved
What issues does this PR resolve, if any? Please include lines like the following (i.e. "Resolves #NNNN), so that when this PR gets merged, GitHub will automatically close those issues.
Resolves #NNNN
Resolves #MMMM
## BEPs Implemented
What [BEPs](https://github.com/bigchaindb/beps) does this pull request implement, if any?

92
.gitignore vendored
View File

@ -6,8 +6,24 @@ __pycache__/
# C extensions
*.so
# Swap -- copypasta from https://github.com/github/gitignore/blob/master/Global/Vim.gitignore
[._]*.s[a-v][a-z]
[._]*.sw[a-p]
[._]s[a-v][a-z]
[._]sw[a-p]
# Session
Session.vim
# Temporary
.netrwhist
*~
# Auto-generated tag files
tags
# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
@ -19,13 +35,9 @@ lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
@ -40,16 +52,14 @@ pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
.pytest_cache/
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
# Translations
*.mo
@ -58,72 +68,32 @@ coverage.xml
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
docs/build/
# PyBuilder
target/
# Jupyter Notebook
# Ipython Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
.python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# Just in time documentation
docs/server/source/http-samples
# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/
# Terraform state files
# See https://stackoverflow.com/a/41482391
terraform.tfstate
terraform.tfstate.backup
# Celery stuff
celerybeat-schedule
celerybeat.pid
# tendermint data
tmdata/data
network/*/data
# SageMath parsed files
*.sage.py
# Docs that are fetched at build time
docs/contributing/source/cross-project-policies/*.md
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
.DS_Store

25
.pre-commit-config.yaml Normal file
View File

@ -0,0 +1,25 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
repos:
- repo: git://github.com/pre-commit/pre-commit-hooks
sha: v1.1.1
hooks:
- id: trailing-whitespace
args: ['--no-markdown-linebreak-ext']
- id: check-merge-conflict
- id: debug-statements
- id: check-added-large-files
- id: flake8
- repo: git://github.com/chewse/pre-commit-mirrors-pydocstyle
sha: v2.1.1
hooks:
- id: pydocstyle
# list of error codes to check, see: http://www.pydocstyle.org/en/latest/error_codes.html
args: ['--select=D204,D201,D209,D210,D212,D300,D403']
# negate the exclude to only apply the hooks to 'bigchaindb' and 'tests' folder
exclude: '^(?!bigchaindb/)(?!tests/)(?!acceptance/)'

11
.readthedocs.yml Normal file
View File

@ -0,0 +1,11 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
build:
image: latest
python:
version: 3.6
pip_install: true

77
.travis.yml Normal file
View File

@ -0,0 +1,77 @@
# Copyright © 2020, 2021 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
sudo: required
dist: focal
services:
- docker
language: python
cache: pip
python:
- 3.6
- 3.7
- 3.8
env:
global:
- DOCKER_COMPOSE_VERSION=1.29.2
matrix:
- TOXENV=flake8
- TOXENV=docsroot
matrix:
fast_finish: true
include:
- python: 3.6
env:
- PLANETMINT_DATABASE_BACKEND=localmongodb
- PLANETMINT_DATABASE_SSL=
- python: 3.6
env:
- PLANETMINT_DATABASE_BACKEND=localmongodb
- PLANETMINT_DATABASE_SSL=
- PLANETMINT_CI_ABCI=enable
- python: 3.6
env:
- PLANETMINT_ACCEPTANCE_TEST=enable
- python: 3.7
env:
- PLANETMINT_DATABASE_BACKEND=localmongodb
- PLANETMINT_DATABASE_SSL=
- python: 3.7
env:
- PLANETMINT_DATABASE_BACKEND=localmongodb
- PLANETMINT_DATABASE_SSL=
- PLANETMINT_CI_ABCI=enable
- python: 3.7
env:
- PLANETMINT_ACCEPTANCE_TEST=enable
- python: 3.8
env:
- PLANETMINT_DATABASE_BACKEND=localmongodb
- PLANETMINT_DATABASE_SSL=
- python: 3.8
env:
- PLANETMINT_DATABASE_BACKEND=localmongodb
- PLANETMINT_DATABASE_SSL=
- PLANETMINT_CI_ABCI=enable
- python: 3.8
env:
- PLANETMINT_ACCEPTANCE_TEST=enable
before_install: sudo .ci/travis-before-install.sh
install: .ci/travis-install.sh
before_script: .ci/travis-before-script.sh
script: .ci/travis_script.sh
after_success: .ci/travis-after-success.sh

1162
CHANGELOG.md Normal file
View File

File diff suppressed because it is too large Load Diff

57
CODE_OF_CONDUCT.md Normal file
View File

@ -0,0 +1,57 @@
<!---
Copyright © 2020 Interplanetary Database Association e.V.,
Planetmint and IPDB software contributors.
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# Contributor Code of Conduct
As contributors and maintainers of this project, and in the interest of
fostering an open and welcoming community, we pledge to respect all people who
contribute to the project.
We are committed to making participation in this project a harassment-free
experience for everyone, regardless of level of experience, gender, gender
identity and expression, sexual orientation, disability, personal appearance,
body size, race, ethnicity, age, religion, nationality, or species--no picking on Wrigley for being a buffalo!
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery
* Personal attacks
* Trolling or insulting/derogatory comments
* Public or private harassment
* Publishing other's private information, such as physical or electronic
addresses, without explicit permission
* Deliberate intimidation
* Other unethical or unprofessional conduct
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
By adopting this Code of Conduct, project maintainers commit themselves to
fairly and consistently applying these principles to every aspect of managing
this project. Project maintainers who do not follow or enforce the Code of
Conduct may be permanently removed from the project team.
This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community.
Instances of abusive, harassing, or otherwise unacceptable behavior directed at yourself or another community member may be
reported by contacting a project maintainer at [contact@bigchaindb.com](mailto:contact@bigchaindb.com). All
complaints will be reviewed and investigated and will result in a response that
is appropriate to the circumstances. Maintainers are
obligated to maintain confidentiality with regard to the reporter of an
incident.
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 1.3.0, available at
[http://contributor-covenant.org/version/1/3/0/][version]
[homepage]: http://contributor-covenant.org
[version]: http://contributor-covenant.org/version/1/3/0/

24
Dockerfile Normal file
View File

@ -0,0 +1,24 @@
FROM python:3.6
LABEL maintainer "contact@ipdb.global"
RUN mkdir -p /usr/src/app
COPY . /usr/src/app/
WORKDIR /usr/src/app
RUN apt-get -qq update \
&& apt-get -y upgrade \
&& apt-get install -y jq \
&& pip install . \
&& apt-get autoremove \
&& apt-get clean
VOLUME ["/data", "/certs"]
ENV PYTHONUNBUFFERED 0
ENV PLANETMINT_CONFIG_PATH /data/.bigchaindb
ENV PLANETMINT_SERVER_BIND 0.0.0.0:9984
ENV PLANETMINT_WSSERVER_HOST 0.0.0.0
ENV PLANETMINT_WSSERVER_SCHEME ws
ENV PLANETMINT_WSSERVER_ADVERTISED_HOST 0.0.0.0
ENV PLANETMINT_WSSERVER_ADVERTISED_SCHEME ws
ENV PLANETMINT_WSSERVER_ADVERTISED_PORT 9985
ENTRYPOINT ["bigchaindb"]
CMD ["start"]

51
Dockerfile-all-in-one Normal file
View File

@ -0,0 +1,51 @@
FROM alpine:3.9
LABEL maintainer "contact@ipdb.global"
ARG TM_VERSION=v0.31.5
RUN mkdir -p /usr/src/app
ENV HOME /root
COPY . /usr/src/app/
WORKDIR /usr/src/app
RUN apk --update add sudo bash \
&& apk --update add python3 openssl ca-certificates git \
&& apk --update add --virtual build-dependencies python3-dev \
libffi-dev openssl-dev build-base jq \
&& apk add --no-cache libstdc++ dpkg gnupg \
&& pip3 install --upgrade pip cffi \
&& pip install -e . \
&& apk del build-dependencies \
&& rm -f /var/cache/apk/*
# Install mongodb and monit
RUN apk --update add mongodb monit
# Install Tendermint
RUN wget https://github.com/tendermint/tendermint/releases/download/${TM_VERSION}/tendermint_${TM_VERSION}_linux_amd64.zip \
&& unzip tendermint_${TM_VERSION}_linux_amd64.zip \
&& mv tendermint /usr/local/bin/ \
&& rm tendermint_${TM_VERSION}_linux_amd64.zip
ENV TMHOME=/tendermint
# Set permissions required for mongodb
RUN mkdir -p /data/db /data/configdb \
&& chown -R mongodb:mongodb /data/db /data/configdb
# Planetmint enviroment variables
ENV PLANETMINT_DATABASE_PORT 27017
ENV PLANETMINT_DATABASE_BACKEND localmongodb
ENV PLANETMINT_SERVER_BIND 0.0.0.0:9984
ENV PLANETMINT_WSSERVER_HOST 0.0.0.0
ENV PLANETMINT_WSSERVER_SCHEME ws
ENV PLANETMINT_WSSERVER_ADVERTISED_HOST 0.0.0.0
ENV PLANETMINT_WSSERVER_ADVERTISED_SCHEME ws
ENV PLANETMINT_TENDERMINT_PORT 26657
VOLUME /data/db /data/configdb /tendermint
EXPOSE 27017 28017 9984 9985 26656 26657 26658
WORKDIR $HOME
ENTRYPOINT ["/usr/src/app/pkg/scripts/all-in-one.bash"]

30
Dockerfile-alpine Normal file
View File

@ -0,0 +1,30 @@
FROM alpine:latest
LABEL maintainer "contact@ipdb.global"
RUN mkdir -p /usr/src/app
COPY . /usr/src/app/
WORKDIR /usr/src/app
RUN apk --update add sudo \
&& apk --update add python3 py-pip openssl ca-certificates git\
&& apk --update add --virtual build-dependencies python3-dev \
libffi-dev openssl-dev build-base \
&& apk add --no-cache libstdc++ \
&& pip3 install --upgrade pip cffi \
&& pip install -e . \
&& apk del build-dependencies \
&& rm -f /var/cache/apk/*
# When developing with Python in a docker container, we are using PYTHONBUFFERED
# to force stdin, stdout and stderr to be totally unbuffered and to capture logs/outputs
ENV PYTHONUNBUFFERED 0
ENV PLANETMINT_DATABASE_PORT 27017
ENV PLANETMINT_DATABASE_BACKEND $backend
ENV PLANETMINT_SERVER_BIND 0.0.0.0:9984
ENV PLANETMINT_WSSERVER_HOST 0.0.0.0
ENV PLANETMINT_WSSERVER_SCHEME ws
ENV PLANETMINT_WSSERVER_ADVERTISED_HOST 0.0.0.0
ENV PLANETMINT_WSSERVER_ADVERTISED_SCHEME ws
ENV PLANETMINT_TENDERMINT_PORT 26657
ARG backend
RUN bigchaindb -y configure "$backend"

35
Dockerfile-dev Normal file
View File

@ -0,0 +1,35 @@
ARG python_version=3.6
FROM python:${python_version}
LABEL maintainer "contact@ipdb.global"
RUN apt-get update \
&& apt-get install -y git \
&& pip install -U pip \
&& apt-get autoremove \
&& apt-get clean
ARG backend
ARG abci_status
# When developing with Python in a docker container, we are using PYTHONBUFFERED
# to force stdin, stdout and stderr to be totally unbuffered and to capture logs/outputs
ENV PYTHONUNBUFFERED 0
ENV PLANETMINT_DATABASE_PORT 27017
ENV PLANETMINT_DATABASE_BACKEND $backend
ENV PLANETMINT_SERVER_BIND 0.0.0.0:9984
ENV PLANETMINT_WSSERVER_HOST 0.0.0.0
ENV PLANETMINT_WSSERVER_SCHEME ws
ENV PLANETMINT_WSSERVER_ADVERTISED_HOST 0.0.0.0
ENV PLANETMINT_WSSERVER_ADVERTISED_SCHEME ws
ENV PLANETMINT_TENDERMINT_PORT 26657
ENV PLANETMINT_CI_ABCI ${abci_status}
RUN mkdir -p /usr/src/app
COPY . /usr/src/app/
WORKDIR /usr/src/app
RUN pip install -e .[dev]
RUN bigchaindb -y configure

862
LICENSE
View File

@ -1,661 +1,201 @@
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU Affero General Public License is a free, copyleft license for
software and other kinds of works, specifically designed to ensure
cooperation with the community in the case of network server software.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
our General Public Licenses are intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
Developers that use our General Public Licenses protect your rights
with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
A secondary benefit of defending all users' freedom is that
improvements made in alternate versions of the program, if they
receive widespread use, become available for other developers to
incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
The GNU Affero General Public License is designed specifically to
ensure that, in such cases, the modified source code becomes available
to the community. It requires the operator of a network server to
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
An older license, called the Affero General Public License and
published by Affero, was designed to accomplish similar goals. This is
a different license, not a version of the Affero GPL, but Affero has
released a new version of the Affero GPL which permits relicensing under
this license.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU Affero General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published
by the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If your software can interact with users remotely through a computer
network, you should also make sure that it provides a way for users to
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive
of the code. There are many ways you could offer source, and different
solutions will be better for different programs; see section 13 for the
specific requirements.
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU AGPL, see
<https://www.gnu.org/licenses/>.
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

24
LICENSES.md Normal file
View File

@ -0,0 +1,24 @@
# Copyrights and Licenses
## Copyrights
For all the code and documentation in this repository, the copyright is owned by one or more of the following:
- Planetmint GmbH
- A Planetmint contributor who agreed to a Planetmint Contributor License Agreement (CLA) with Planetmint GmbH. (See [BEP-16](https://github.com/bigchaindb/BEPs/tree/master/16).)
- A Planetmint contributor who signed off on the Developer Certificate of Origin (DCO) for all their contributions. (See [BEP-24](https://github.com/bigchaindb/BEPs/tree/master/24).)
- (Rarely, see the **Exceptions Section** below) A third pary who licensed the code in question under an open source license.
## Code Licenses
All code in this repository, including short code snippets in the documentation, but not including the **Exceptions** noted below, is licensed under the Apache License, Version 2.0, the full text of which can be found at [http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0).
For the licenses on all other Planetmint-related code (i.e. in other repositories), see the LICENSE file in the associated repository.
## Documentation Licenses
The official Planetmint documentation, _except for the short code snippets embedded within it_, is licensed under a Creative Commons Attribution 4.0 International license, the full text of which can be found at [http://creativecommons.org/licenses/by/4.0/legalcode](http://creativecommons.org/licenses/by/4.0/legalcode).
## Exceptions
The contents of the `k8s/nginx-openresty/` directory are licensed as described in the `LICENSE.md` file in that directory.

144
Makefile Normal file
View File

@ -0,0 +1,144 @@
.PHONY: help run start stop logs test test-unit test-unit-watch test-acceptance cov doc doc-acceptance clean reset release dist check-deps clean-build clean-pyc clean-test
.DEFAULT_GOAL := help
#############################
# Open a URL in the browser #
#############################
define BROWSER_PYSCRIPT
import os, webbrowser, sys
try:
from urllib import pathname2url
except:
from urllib.request import pathname2url
webbrowser.open("file://" + pathname2url(os.path.abspath(sys.argv[1])))
endef
export BROWSER_PYSCRIPT
##################################
# Display help for this makefile #
##################################
define PRINT_HELP_PYSCRIPT
import re, sys
print("Planetmint 2.0 developer toolbox")
print("--------------------------------")
print("Usage: make COMMAND")
print("")
print("Commands:")
for line in sys.stdin:
match = re.match(r'^([a-zA-Z_-]+):.*?## (.*)$$', line)
if match:
target, help = match.groups()
print(" %-16s %s" % (target, help))
endef
export PRINT_HELP_PYSCRIPT
##################
# Basic commands #
##################
DOCKER := docker
DC := docker-compose
BROWSER := python -c "$$BROWSER_PYSCRIPT"
HELP := python -c "$$PRINT_HELP_PYSCRIPT"
ECHO := /usr/bin/env echo
IS_DOCKER_COMPOSE_INSTALLED := $(shell command -v docker-compose 2> /dev/null)
################
# Main targets #
################
help: ## Show this help
@$(HELP) < $(MAKEFILE_LIST)
run: check-deps ## Run Planetmint from source (stop it with ctrl+c)
# although bigchaindb has tendermint and mongodb in depends_on,
# launch them first otherwise tendermint will get stuck upon sending yet another log
# due to some docker-compose issue; does not happen when containers are run as daemons
@$(DC) up --no-deps mongodb tendermint bigchaindb
start: check-deps ## Run Planetmint from source and daemonize it (stop with `make stop`)
@$(DC) up -d bigchaindb
stop: check-deps ## Stop Planetmint
@$(DC) stop
logs: check-deps ## Attach to the logs
@$(DC) logs -f bigchaindb
test: check-deps test-unit test-acceptance ## Run unit and acceptance tests
test-unit: check-deps ## Run all tests once
@$(DC) up -d bdb
@$(DC) exec bigchaindb pytest
test-unit-watch: check-deps ## Run all tests and wait. Every time you change code, tests will be run again
@$(DC) run --rm --no-deps bigchaindb pytest -f
test-acceptance: check-deps ## Run all acceptance tests
@./run-acceptance-test.sh
cov: check-deps ## Check code coverage and open the result in the browser
@$(DC) run --rm bigchaindb pytest -v --cov=bigchaindb --cov-report html
$(BROWSER) htmlcov/index.html
doc: check-deps ## Generate HTML documentation and open it in the browser
@$(DC) run --rm --no-deps bdocs make -C docs/root html
@$(DC) run --rm --no-deps bdocs make -C docs/server html
@$(DC) run --rm --no-deps bdocs make -C docs/contributing html
$(BROWSER) docs/root/build/html/index.html
doc-acceptance: check-deps ## Create documentation for acceptance tests
@$(DC) run --rm python-acceptance pycco -i -s /src -d /docs
$(BROWSER) acceptance/python/docs/index.html
clean: clean-build clean-pyc clean-test ## Remove all build, test, coverage and Python artifacts
@$(ECHO) "Cleaning was successful."
reset: check-deps ## Stop and REMOVE all containers. WARNING: you will LOSE all data stored in Planetmint.
@$(DC) down
release: dist ## package and upload a release
twine upload dist/*
dist: clean ## builds source (and not for now, wheel package)
python setup.py sdist
# python setup.py bdist_wheel
ls -l dist
###############
# Sub targets #
###############
check-deps:
ifndef IS_DOCKER_COMPOSE_INSTALLED
@$(ECHO) "Error: docker-compose is not installed"
@$(ECHO)
@$(ECHO) "You need docker-compose to run this command. Check out the official docs on how to install it in your system:"
@$(ECHO) "- https://docs.docker.com/compose/install/"
@$(ECHO)
@$(DC) # docker-compose is not installed, so we call it to generate an error and exit
endif
clean-build: # Remove build artifacts
@rm -fr build/
@rm -fr dist/
@rm -fr .eggs/
@find . -name '*.egg-info' -exec rm -fr {} +
@find . -name '*.egg' -exec rm -f {} +
clean-pyc: # Remove Python file artifacts
@find . -name '*.pyc' -exec rm -f {} +
@find . -name '*.pyo' -exec rm -f {} +
@find . -name '*~' -exec rm -f {} +
@find . -name '__pycache__' -exec rm -fr {} +
clean-test: # Remove test and coverage artifacts
@find . -name '.pytest_cache' -exec rm -fr {} +
@rm -fr .tox/
@rm -f .coverage
@rm -fr htmlcov/

97
PYTHON_STYLE_GUIDE.md Normal file
View File

@ -0,0 +1,97 @@
<!---
Copyright © 2020 Interplanetary Database Association e.V.,
Planetmint and IPDB software contributors.
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# Python Style Guide
This guide starts out with our general Python coding style guidelines and ends with a section on how we write & run (Python) tests.
## General Python Coding Style Guidelines
Our starting point is [PEP8](https://www.python.org/dev/peps/pep-0008/), the standard "Style Guide for Python Code." Many Python IDEs will check your code against PEP8. (Note that PEP8 isn't frozen; it actually changes over time, but slowly.)
Planetmint uses Python 3.5+, so you can ignore all PEP8 guidelines specific to Python 2.
We use [pre-commit](http://pre-commit.com/) to check some of the rules below before every commit but not everything is realized yet.
The hooks we use can be found in the [.pre-commit-config.yaml](https://github.com/bigchaindb/bigchaindb/blob/master/.pre-commit-config.yaml) file.
### Python Docstrings
PEP8 says some things about docstrings, but not what to put in them or how to structure them. [PEP257](https://www.python.org/dev/peps/pep-0257/) was one proposal for docstring conventions, but we prefer [Google-style docstrings](https://google.github.io/styleguide/pyguide.html?showone=Comments#Comments) instead: they're easier to read and the [napoleon extension](http://www.sphinx-doc.org/en/stable/ext/napoleon.html) for Sphinx lets us turn them into nice-looking documentation. Here are some references on Google-style docstrings:
* [Google's docs on Google-style docstrings](https://google.github.io/styleguide/pyguide.html?showone=Comments#Comments)
* [napoleon's docs include an overview of Google-style docstrings](http://sphinxcontrib-napoleon.readthedocs.org/en/latest/index.html)
* [Example Google-style docstrings](http://sphinxcontrib-napoleon.readthedocs.org/en/latest/example_google.html) (from napoleon's docs)
### Maximum Line Length
PEP8 has some [maximum line length guidelines](https://www.python.org/dev/peps/pep-0008/#id17), starting with "Limit all lines to a maximum of 79 characters" but "for flowing long blocks of text with fewer structural restrictions (docstrings or comments), the line length should be limited to 72 characters."
We discussed this at length, and it seems that the consensus is: _try_ to keep line lengths less than 79/72 characters, unless you have a special situation where longer lines would improve readability. (The basic reason is that 79/72 works for everyone, and Planetmint is an open source project.) As a hard limit, keep all lines less than 119 characters (which is the width of GitHub code review).
### Single or Double Quotes?
Python lets you use single or double quotes. PEP8 says you can use either, as long as you're consistent. We try to stick to using single quotes, except in cases where using double quotes is more readable. For example:
```python
print('This doesn\'t look so nice.')
print("Doesn't this look nicer?")
```
### Breaking Strings Across Multiple Lines
Should we use parentheses or slashes (`\`) to break strings across multiple lines, i.e.
```python
my_string = ('This is a very long string, so long that it will not fit into just one line '
'so it must be split across multiple lines.')
# or
my_string = 'This is a very long string, so long that it will not fit into just one line ' \
'so it must be split across multiple lines.'
```
It seems the preference is for slashes, but using parentheses is okay too. (There are good arguments either way. Arguing about it seems like a waste of time.)
### How to Format Long import Statements
If you need to `import` lots of names from a module or package, and they won't all fit in one line (without making the line too long), then use parentheses to spread the names across multiple lines, like so:
```python
from Tkinter import (
Tk, Frame, Button, Entry, Canvas, Text,
LEFT, DISABLED, NORMAL, RIDGE, END,
)
# Or
from Tkinter import (Tk, Frame, Button, Entry, Canvas, Text,
LEFT, DISABLED, NORMAL, RIDGE, END)
```
For the rationale, see [PEP 328](https://www.python.org/dev/peps/pep-0328/#rationale-for-parentheses).
### Using the % operator or `format()` to Format Strings
Given the choice:
```python
x = 'name: %s; score: %d' % (name, n)
# or
x = 'name: {}; score: {}'.format(name, n)
```
we use the `format()` version. The [official Python documentation says](https://docs.python.org/2/library/stdtypes.html#str.format), "This method of string formatting is the new standard in Python 3, and should be preferred to the % formatting described in String Formatting Operations in new code."
## Running the Flake8 Style Checker
We use [Flake8](http://flake8.pycqa.org/en/latest/index.html) to check our Python code style. Once you have it installed, you can run it using:
```text
flake8 --max-line-length 119 bigchaindb/
```
## Writing and Running (Python) Tests
The content of this section was moved to [`bigchaindb/tests/README.md`](https://github.com/bigchaindb/bigchaindb/blob/master/tests/README.md).
Note: We automatically run all tests on all pull requests (using Travis CI), so you should definitely run all tests locally before you submit a pull request. See the above-linked README file for instructions.

78
README.md
View File

@ -1 +1,77 @@
# planetmint
<!---
Copyright © 2020 Interplanetary Database Association e.V.,
Planetmint and IPDB software contributors.
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
<!--- There is no shield to get the latest version
(including pre-release versions) from PyPI,
so show the latest GitHub release instead.
--->
[![Codecov branch](https://img.shields.io/codecov/c/github/bigchaindb/bigchaindb/master.svg)](https://codecov.io/github/bigchaindb/bigchaindb?branch=master)
[![Latest release](https://img.shields.io/github/release/bigchaindb/bigchaindb/all.svg)](https://github.com/bigchaindb/bigchaindb/releases)
[![Status on PyPI](https://img.shields.io/pypi/status/bigchaindb.svg)](https://pypi.org/project/Planetmint/)
[![Travis branch](https://img.shields.io/travis/bigchaindb/bigchaindb/master.svg)](https://travis-ci.com/bigchaindb/bigchaindb)
[![Documentation Status](https://readthedocs.org/projects/bigchaindb-server/badge/?version=latest)](https://docs.bigchaindb.com/projects/server/en/latest/)
[![Join the chat at https://gitter.im/bigchaindb/bigchaindb](https://badges.gitter.im/bigchaindb/bigchaindb.svg)](https://gitter.im/bigchaindb/bigchaindb?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
# Planetmint Server
Planetmint is the blockchain database. This repository is for _BigchainDB Server_.
## The Basics
* [Try the Quickstart](https://docs.bigchaindb.com/projects/server/en/latest/quickstart.html)
* [Read the Planetmint 2.0 whitepaper](https://www.bigchaindb.com/whitepaper/)
* [Check out the _Hitchiker's Guide to BigchainDB_](https://www.bigchaindb.com/developers/guide/)
## Run and Test Planetmint Server from the `master` Branch
Running and testing the latest version of Planetmint Server is easy. Make sure you have a recent version of [Docker Compose](https://docs.docker.com/compose/install/) installed. When you are ready, fire up a terminal and run:
```text
git clone https://github.com/bigchaindb/bigchaindb.git
cd bigchaindb
make run
```
Planetmint should be reachable now on `http://localhost:9984/`.
There are also other commands you can execute:
* `make start`: Run Planetmint from source and daemonize it (stop it with `make stop`).
* `make stop`: Stop Planetmint.
* `make logs`: Attach to the logs.
* `make test`: Run all unit and acceptance tests.
* `make test-unit-watch`: Run all tests and wait. Every time you change code, tests will be run again.
* `make cov`: Check code coverage and open the result in the browser.
* `make doc`: Generate HTML documentation and open it in the browser.
* `make clean`: Remove all build, test, coverage and Python artifacts.
* `make reset`: Stop and REMOVE all containers. WARNING: you will LOSE all data stored in Planetmint.
To view all commands available, run `make`.
## Links for Everyone
* [Planetmint.com](https://www.bigchaindb.com/) - the main Planetmint website, including newsletter signup
* [Roadmap](https://github.com/bigchaindb/org/blob/master/ROADMAP.md)
* [Blog](https://medium.com/the-bigchaindb-blog)
* [Twitter](https://twitter.com/Planetmint)
## Links for Developers
* [All Planetmint Documentation](https://docs.bigchaindb.com/en/latest/)
* [Planetmint Server Documentation](https://docs.bigchaindb.com/projects/server/en/latest/index.html)
* [CONTRIBUTING.md](.github/CONTRIBUTING.md) - how to contribute
* [Community guidelines](CODE_OF_CONDUCT.md)
* [Open issues](https://github.com/bigchaindb/bigchaindb/issues)
* [Open pull requests](https://github.com/bigchaindb/bigchaindb/pulls)
* [Gitter chatroom](https://gitter.im/bigchaindb/bigchaindb)
## Legal
* [Licenses](LICENSES.md) - open source & open content
* [Imprint](https://www.bigchaindb.com/imprint/)
* [Contact Us](https://www.bigchaindb.com/contact/)

77
README_cn.md Normal file
View File

@ -0,0 +1,77 @@
<!---
Copyright © 2020 Interplanetary Database Association e.V.,
Planetmint and IPDB software contributors.
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
<!--- There is no shield to get the latest version
(including pre-release versions) from PyPI,
so show the latest GitHub release instead.
--->
[![Codecov branch](https://img.shields.io/codecov/c/github/bigchaindb/bigchaindb/master.svg)](https://codecov.io/github/bigchaindb/bigchaindb?branch=master)
[![Latest release](https://img.shields.io/github/release/bigchaindb/bigchaindb/all.svg)](https://github.com/bigchaindb/bigchaindb/releases)
[![Status on PyPI](https://img.shields.io/pypi/status/bigchaindb.svg)](https://pypi.org/project/Planetmint/)
[![Travis branch](https://img.shields.io/travis/bigchaindb/bigchaindb/master.svg)](https://travis-ci.com/bigchaindb/bigchaindb)
[![Documentation Status](https://readthedocs.org/projects/bigchaindb-server/badge/?version=latest)](https://docs.bigchaindb.com/projects/server/en/latest/)
[![Join the chat at https://gitter.im/bigchaindb/bigchaindb](https://badges.gitter.im/bigchaindb/bigchaindb.svg)](https://gitter.im/bigchaindb/bigchaindb?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
# Planetmint 服务器
Planetmint 是区块链数据库. 这是 _BigchainDB 服务器_ 的仓库.
## 基础知识
* [尝试快速开始](https://docs.bigchaindb.com/projects/server/en/latest/quickstart.html)
* [阅读 Planetmint 2.0 白皮书](https://www.bigchaindb.com/whitepaper/)
* [查阅漫游指南](https://www.bigchaindb.com/developers/guide/)
## 运行和测试 `master` 分支的 Planetmint 服务器
运行和测试最新版本的 Planetmint 服务器非常简单. 确认你有安装最新版本的 [Docker Compose](https://docs.docker.com/compose/install/). 当你准备好了, 打开一个终端并运行:
```text
git clone https://github.com/bigchaindb/bigchaindb.git
cd bigchaindb
make run
```
Planetmint 应该可以通过 `http://localhost:9984/` 访问.
这里也有一些其他的命令你可以运行:
* `make start`: 通过源码和守护进程的方式运行 Planetmint (通过 `make stop` 停止).
* `make stop`: 停止运行 Planetmint.
* `make logs`: 附在日志上.
* `make test`: 运行所有单元和验收测试.
* `make test-unit-watch`: 运行所有测试并等待. 每次更改代码时都会再次运行测试.
* `make cov`: 检查代码覆盖率并在浏览器中打开结果.
* `make doc`: 生成 HTML 文档并在浏览器中打开它.
* `make clean`: 删除所有构建, 测试, 覆盖和 Python 生成物.
* `make reset`: 停止并移除所有容器. 警告: 您将丢失存储在 Planetmint 中的所有数据.
查看所有可用命令, 请运行 `make`.
## 一般人员链接
* [Planetmint.com](https://www.bigchaindb.com/) - Planetmint 主网站, 包括新闻订阅
* [路线图](https://github.com/bigchaindb/org/blob/master/ROADMAP.md)
* [博客](https://medium.com/the-bigchaindb-blog)
* [推特](https://twitter.com/Planetmint)
## 开发人员链接
* [所有的 Planetmint 文档](https://docs.bigchaindb.com/en/latest/)
* [Planetmint 服务器 文档](https://docs.bigchaindb.com/projects/server/en/latest/index.html)
* [CONTRIBUTING.md](.github/CONTRIBUTING.md) - how to contribute
* [社区指南](CODE_OF_CONDUCT.md)
* [公开问题](https://github.com/bigchaindb/bigchaindb/issues)
* [公开的 pull request](https://github.com/bigchaindb/bigchaindb/pulls)
* [Gitter 聊天室](https://gitter.im/bigchaindb/bigchaindb)
## 法律声明
* [许可](LICENSES.md) - 开源代码 & 开源内容
* [印记](https://www.bigchaindb.com/imprint/)
* [联系我们](https://www.bigchaindb.com/contact/)

65
README_kor.md Normal file
View File

@ -0,0 +1,65 @@
[![Codecov branch](https://img.shields.io/codecov/c/github/bigchaindb/bigchaindb/master.svg)](https://codecov.io/github/bigchaindb/bigchaindb?branch=master)
[![Latest release](https://img.shields.io/github/release/bigchaindb/bigchaindb/all.svg)](https://github.com/bigchaindb/bigchaindb/releases)
[![Status on PyPI](https://img.shields.io/pypi/status/bigchaindb.svg)](https://pypi.org/project/Planetmint/)
[![Travis branch](https://img.shields.io/travis/bigchaindb/bigchaindb/master.svg)](https://travis-ci.org/bigchaindb/bigchaindb)
[![Documentation Status](https://readthedocs.org/projects/bigchaindb-server/badge/?version=latest)](https://docs.bigchaindb.com/projects/server/en/latest/)
[![Join the chat at https://gitter.im/bigchaindb/bigchaindb](https://badges.gitter.im/bigchaindb/bigchaindb.svg)](https://gitter.im/bigchaindb/bigchaindb?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
# Planetmint 서버
BigchaingDB는 블록체인 데이터베이스입니다. 이 저장소는 _BigchaingDB 서버_를 위한 저장소입니다.
### 기본 사항
* [빠른 시작 사용해보기](https://docs.bigchaindb.com/projects/server/en/latest/quickstart.html)
* [Planetmint 2.0 백서 읽기](https://www.bigchaindb.com/whitepaper/)
* [BigchainDB에 대한 _Hitchiker's Guide_를 확인십시오.](https://www.bigchaindb.com/developers/guide/)
### `master` Branch에서 Planetmint 서버 실행 및 테스트
BigchaingDB 서버의 최신 버전을 실행하고 테스트하는 것은 어렵지 않습니다. [Docker Compose](https://docs.docker.com/compose/install/)의 최신 버전이 설치되어 있는지 확인하십시오. 준비가 되었다면, 터미널에서 다음을 실행하십시오.
```text
git clone https://github.com/bigchaindb/bigchaindb.git
cd bigchaindb
make run
```
이제 BigchainDB는 `http://localhost:9984/`에 연결되어야 합니다.
또한, 실행시키기 위한 다른 명령어들도 있습니다.
* `make start` : 소스로부터 BigchainDB를 실행하고 데몬화합니다. \(이는 `make stop` 을 하면 중지합니다.\)
* `make stop` : BigchainDB를 중지합니다.
* `make logs` : 로그에 첨부합니다.
* `make text` : 모든 유닛과 허가 테스트를 실행합니다.
* `make test-unit-watch` : 모든 테스트를 수행하고 기다립니다. 코드를 변경할 때마다 테스트는 다시 실행될 것입니다.
* `make cov` : 코드 커버리지를 확인하고 브라우저에서 결과를 엽니다.
* `make doc` : HTML 문서를 만들고, 브라우저에서 엽니다.
* `make clean` : 모든 빌드와 테스트, 커버리지 및 파이썬 아티팩트를 제거합니다.
* `make reset` : 모든 컨테이너들을 중지하고 제거합니다. 경고 : BigchainDB에 저장된 모든 데이터를 잃을 수 있습니다.
사용 가능한 모든 명령어를 보기 위해서는 `make` 를 실행하십시오.
### 모두를 위한 링크들
* [Planetmint.com ](https://www.bigchaindb.com/)- 뉴스 레터 가입을 포함하는 Planetmint 주요 웹 사이트
* [로드맵](https://github.com/bigchaindb/org/blob/master/ROADMAP.md)
* [블로그](https://medium.com/the-bigchaindb-blog)
* [트위터](https://twitter.com/Planetmint)
### 개발자들을 위한 링크들
* [모든 Planetmint 문서](https://docs.bigchaindb.com/en/latest/)
* [Planetmint 서버 문서](https://docs.bigchaindb.com/projects/server/en/latest/index.html)
* [CONTRIBUTING.md](https://github.com/bigchaindb/bigchaindb/blob/master/.github/CONTRIBUTING.md) - 기여를 하는 방법
* [커뮤니티 가이드라인](https://github.com/bigchaindb/bigchaindb/blob/master/CODE_OF_CONDUCT.md)
* [이슈 작성](https://github.com/bigchaindb/bigchaindb/issues)
* [pull request 하기](https://github.com/bigchaindb/bigchaindb/pulls)
* [Gitter 채팅방](https://gitter.im/bigchaindb/bigchaindb)
### 합법
* [라이선스](https://github.com/bigchaindb/bigchaindb/blob/master/LICENSES.md) - 오픈 소스 & 오픈 콘텐츠
* [발행](https://www.bigchaindb.com/imprint/)
* [연락처](https://www.bigchaindb.com/contact/)

101
RELEASE_PROCESS.md Normal file
View File

@ -0,0 +1,101 @@
<!---
Copyright © 2020 Interplanetary Database Association e.V.,
Planetmint and IPDB software contributors.
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# Our Release Process
## Notes
Planetmint follows
[the Python form of Semantic Versioning](https://packaging.python.org/tutorials/distributing-packages/#choosing-a-versioning-scheme)
(i.e. MAJOR.MINOR.PATCH),
which is almost identical
to [regular semantic versioning](http://semver.org/), but there's no hyphen, e.g.
- `0.9.0` for a typical final release
- `4.5.2a1` not `4.5.2-a1` for the first Alpha release
- `3.4.5rc2` not `3.4.5-rc2` for Release Candidate 2
**Note 1:** For Git tags (which are used to identify releases on GitHub), we append a `v` in front. For example, the Git tag for version `2.0.0a1` was `v2.0.0a1`.
**Note 2:** For Docker image tags (e.g. on Docker Hub), we use longer version names for Alpha, Beta and Release Candidate releases. For example, the Docker image tag for version `2.0.0a2` was `2.0.0-alpha2`.
We use `0.9` and `0.9.0` as example version and short-version values below. You should replace those with the correct values for your new version.
We follow [BEP-1](https://github.com/bigchaindb/BEPs/tree/master/1), which is our variant of C4, the Collective Code Construction Contract, so a release is just a [tagged commit](https://git-scm.com/book/en/v2/Git-Basics-Tagging) on the `master` branch, i.e. a label for a particular Git commit.
The following steps are what we do to release a new version of _BigchainDB Server_. The steps to release the Python Driver are similar but not the same.
## Steps
1. Create a pull request where you make the following changes:
- Update `CHANGELOG.md`
- Update all Docker image tags in all Kubernetes YAML files (in the `k8s/` directory).
For example, in the files:
- `k8s/bigchaindb/bigchaindb-ss.yaml` and
- `k8s/dev-setup/bigchaindb.yaml`
find the line of the form `image: bigchaindb/bigchaindb:0.8.1` and change the version number to the new version number, e.g. `0.9.0`. (This is the Docker image that Kubernetes should pull from Docker Hub.)
Keep in mind that this is a _Docker image tag_ so our naming convention is
a bit different; see Note 2 in the **Notes** section above.
- In `bigchaindb/version.py`:
- update `__version__` to e.g. `0.9.0` (with no `.dev` on the end)
- update `__short_version__` to e.g. `0.9` (with no `.dev` on the end)
- In the docs about installing Planetmint (and Tendermint), and in the associated scripts, recommend/install a version of Tendermint that _actually works_ with the soon-to-be-released version of Planetmint. You can find all such references by doing a search for the previously-recommended version number, such as `0.31.5`.
- In `setup.py`, _maybe_ update the development status item in the `classifiers` list. For example, one allowed value is `"Development Status :: 5 - Production/Stable"`. The [allowed values are listed at pypi.python.org](https://pypi.python.org/pypi?%3Aaction=list_classifiers).
2. **Wait for all the tests to pass!**
3. Merge the pull request into the `master` branch.
4. Go to the [bigchaindb/bigchaindb Releases page on GitHub](https://github.com/bigchaindb/bigchaindb/releases)
and click the "Draft a new release" button.
5. Fill in the details:
- **Tag version:** version number preceded by `v`, e.g. `v0.9.1`
- **Target:** the last commit that was just merged. In other words, that commit will get a Git tag with the value given for tag version above.
- **Title:** Same as tag version above, e.g `v0.9.1`
- **Description:** The body of the changelog entry (Added, Changed, etc.)
6. Click "Publish release" to publish the release on GitHub.
7. On your local computer, make sure you're on the `master` branch and that it's up-to-date with the `master` branch in the bigchaindb/bigchaindb repository (e.g. `git pull upstream master`). We're going to use that to push a new `bigchaindb` package to PyPI.
8. Make sure you have a `~/.pypirc` file containing credentials for PyPI.
9. Do `make release` to build and publish the new `bigchaindb` package on PyPI. For this step you need to have `twine` installed. If you get an error like `Makefile:135: recipe for target 'clean-pyc' failed` then try doing
```text
sudo chown -R $(whoami):$(whoami) .
```
10. [Log in to readthedocs.org](https://readthedocs.org/accounts/login/) and go to the **Planetmint Server** project, then:
- Click on "Builds", select "latest" from the drop-down menu, then click the "Build Version:" button.
- Wait for the build of "latest" to finish. This can take a few minutes.
- Go to Admin --> Advanced Settings
and make sure that "Default branch:" (i.e. what "latest" points to)
is set to the new release's tag, e.g. `v0.9.1`.
(It won't be an option if you didn't wait for the build of "latest" to finish.)
Then scroll to the bottom and click "Save".
- Go to Admin --> Versions
and under **Choose Active Versions**, do these things:
1. Make sure that the new version's tag is "Active" and "Public"
2. Make sure the **stable** branch is _not_ active.
3. Scroll to the bottom of the page and click "Save".
11. Go to [Docker Hub](https://hub.docker.com/) and sign in, then:
- Click on "Organizations"
- Click on "bigchaindb"
- Click on "bigchaindb/bigchaindb"
- Click on "Build Settings"
- Find the row where "Docker Tag Name" equals `latest`
and change the value of "Name" to the name (Git tag)
of the new release, e.g. `v0.9.0`.
- If the release is an Alpha, Beta or Release Candidate release,
then a new row must be added.
You can do that by clicking the green "+" (plus) icon.
The contents of the new row should be similar to the existing rows
of previous releases like that.
- Click on "Tags"
- Delete the "latest" tag (so we can rebuild it)
- Click on "Build Settings" again
- Click on the "Trigger" button for the "latest" tag and make sure it worked by clicking on "Tags" again
- If the release is an Alpha, Beta or Release Candidate release,
then click on the "Trigger" button for that tag as well.
Congratulations, you have released a new version of Planetmint Server!

14
ROADMAP.md Normal file
View File

@ -0,0 +1,14 @@
<!---
Copyright © 2020 Interplanetary Database Association e.V.,
Planetmint and IPDB software contributors.
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# Planetmint Roadmap
We moved the Planetmint Roadmap to the bigchaindb/org repository; see:
[https://github.com/bigchaindb/org/blob/master/ROADMAP.md](https://github.com/bigchaindb/org/blob/master/ROADMAP.md)
(We kept this file here to avoid breaking some links.)

27
acceptance/README.md Normal file
View File

@ -0,0 +1,27 @@
<!---
Copyright © 2020 Interplanetary Database Association e.V.,
Planetmint and IPDB software contributors.
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# Acceptance test suite
This directory contains the acceptance test suite for Planetmint.
The suite uses Docker Compose to set up a single Planetmint node, run all tests, and finally stop the node. In the future we will add support for a four node network setup.
## Running the tests
It should be as easy as `make test-acceptance`.
Note that `make test-acceptance` will take some time to start the node and shutting it down. If you are developing a test, or you wish to run a specific test in the acceptance test suite, first start the node with `make start`. After the node is running, you can run `pytest` inside the `python-acceptance` container with:
```bash
docker-compose run --rm python-acceptance pytest <use whatever option you need>
```
## Writing and documenting the tests
Tests are sometimes difficult to read. For acceptance tests, we try to be really explicit on what the test is doing, so please write code that is *simple* and easy to understand. We decided to use literate-programming documentation. To generate the documentation run:
```bash
make doc-acceptance
```

1
acceptance/python/.gitignore vendored Normal file
View File

@ -0,0 +1 @@
docs

9
acceptance/python/Dockerfile Normal file
View File

@ -0,0 +1,9 @@
FROM python:3.6.3
RUN mkdir -p /src
RUN pip install --upgrade \
pycco \
websocket-client~=0.47.0 \
pytest~=3.0 \
bigchaindb-driver~=0.6.2 \
blns

125
acceptance/python/src/test_basic.py Normal file
View File

@ -0,0 +1,125 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
# # Basic Acceptance Test
# Here we check that the primitives of the system behave as expected.
# As you will see, this script tests basic stuff like:
#
# - create a transaction
# - check if the transaction is stored
# - check for the outputs of a given public key
# - transfer the transaction to another key
#
# We run a series of checks for each steps, that is retrieving the transaction from
# the remote system, and also checking the `outputs` of a given public key.
#
# This acceptance test is a rip-off of our
# [tutorial](https://docs.bigchaindb.com/projects/py-driver/en/latest/usage.html).
# ## Imports
# We need some utils from the `os` package, we will interact with
# env variables.
import os
# For this test case we import and use the Python Driver.
from bigchaindb_driver import BigchainDB
from bigchaindb_driver.crypto import generate_keypair
def test_basic():
# ## Set up a connection to Planetmint
# To use BighainDB we need a connection. Here we create one. By default we
# connect to localhost, but you can override this value using the env variable
# called `PLANETMINT_ENDPOINT`, a valid value must include the schema:
# `https://example.com:9984`
bdb = BigchainDB(os.environ.get('PLANETMINT_ENDPOINT'))
# ## Create keypairs
# This test requires the interaction between two actors with their own keypair.
# The two keypairs will be called—drum roll—Alice and Bob.
alice, bob = generate_keypair(), generate_keypair()
# ## Alice registers her bike in Planetmint
# Alice has a nice bike, and here she creates the "digital twin"
# of her bike.
bike = {'data': {'bicycle': {'serial_number': 420420}}}
# She prepares a `CREATE` transaction...
prepared_creation_tx = bdb.transactions.prepare(
operation='CREATE',
signers=alice.public_key,
asset=bike)
# ... and she fulfills it with her private key.
fulfilled_creation_tx = bdb.transactions.fulfill(
prepared_creation_tx,
private_keys=alice.private_key)
# We will use the `id` of this transaction several time, so we store it in
# a variable with a short and easy name
bike_id = fulfilled_creation_tx['id']
# Now she is ready to send it to the Planetmint Network.
sent_transfer_tx = bdb.transactions.send_commit(fulfilled_creation_tx)
# And just to be 100% sure, she also checks if she can retrieve
# it from the Planetmint node.
assert bdb.transactions.retrieve(bike_id), 'Cannot find transaction {}'.format(bike_id)
# Alice is now the proud owner of one unspent asset.
assert len(bdb.outputs.get(alice.public_key, spent=False)) == 1
assert bdb.outputs.get(alice.public_key)[0]['transaction_id'] == bike_id
# ## Alice transfers her bike to Bob
# After registering her bike, Alice is ready to transfer it to Bob.
# She needs to create a new `TRANSFER` transaction.
# A `TRANSFER` transaction contains a pointer to the original asset. The original asset
# is identified by the `id` of the `CREATE` transaction that defined it.
transfer_asset = {'id': bike_id}
# Alice wants to spend the one and only output available, the one with index `0`.
output_index = 0
output = fulfilled_creation_tx['outputs'][output_index]
# Here, she defines the `input` of the `TRANSFER` transaction. The `input` contains
# several keys:
#
# - `fulfillment`, taken from the previous `CREATE` transaction.
# - `fulfills`, that specifies which condition she is fulfilling.
# - `owners_before`.
transfer_input = {'fulfillment': output['condition']['details'],
'fulfills': {'output_index': output_index,
'transaction_id': fulfilled_creation_tx['id']},
'owners_before': output['public_keys']}
# Now that all the elements are set, she creates the actual transaction...
prepared_transfer_tx = bdb.transactions.prepare(
operation='TRANSFER',
asset=transfer_asset,
inputs=transfer_input,
recipients=bob.public_key)
# ... and signs it with her private key.
fulfilled_transfer_tx = bdb.transactions.fulfill(
prepared_transfer_tx,
private_keys=alice.private_key)
# She finally sends the transaction to a Planetmint node.
sent_transfer_tx = bdb.transactions.send_commit(fulfilled_transfer_tx)
# And just to be 100% sure, she also checks if she can retrieve
# it from the Planetmint node.
assert bdb.transactions.retrieve(fulfilled_transfer_tx['id']) == sent_transfer_tx
# Now Alice has zero unspent transactions.
assert len(bdb.outputs.get(alice.public_key, spent=False)) == 0
# While Bob has one.
assert len(bdb.outputs.get(bob.public_key, spent=False)) == 1
# Bob double checks what he got was the actual bike.
bob_tx_id = bdb.outputs.get(bob.public_key, spent=False)[0]['transaction_id']
assert bdb.transactions.retrieve(bob_tx_id) == sent_transfer_tx

181
acceptance/python/src/test_divisible_asset.py Normal file
View File

@ -0,0 +1,181 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
# # Divisible assets integration testing
# This test checks if we can successfully divide assets.
# The script tests various things like:
#
# - create a transaction with a divisible asset and issue them to someone
# - check if the transaction is stored and has the right amount of tokens
# - spend some tokens
# - try to spend more tokens than available
#
# We run a series of checks for each step, that is retrieving
# the transaction from the remote system, and also checking the `amount`
# of a given transaction.
#
# This integration test is a rip-off of our
# [tutorial](https://docs.bigchaindb.com/projects/py-driver/en/latest/usage.html).
# ## Imports
# We need some utils from the `os` package, we will interact with
# env variables.
# We need the `pytest` package to catch the `BadRequest` exception properly.
# And of course, we also need the `BadRequest`.
import os
import pytest
from bigchaindb_driver.exceptions import BadRequest
# For this test case we import and use the Python Driver.
from bigchaindb_driver import BigchainDB
from bigchaindb_driver.crypto import generate_keypair
def test_divisible_assets():
# ## Set up a connection to Planetmint
# Check [test_basic.py](./test_basic.html) to get some more details
# about the endpoint.
bdb = BigchainDB(os.environ.get('PLANETMINT_ENDPOINT'))
# Oh look, it is Alice again and she brought her friend Bob along.
alice, bob = generate_keypair(), generate_keypair()
# ## Alice creates a time sharing token
# Alice wants to go on vacation, while Bobs bike just broke down.
# Alice decides to rent her bike to Bob while she is gone.
# So she prepares a `CREATE` transaction to issues 10 tokens.
# First, she prepares an asset for a time sharing token. As you can see in
# the description, Bob and Alice agree that each token can be used to ride
# the bike for one hour.
bike_token = {
'data': {
'token_for': {
'bike': {
'serial_number': 420420
}
},
'description': 'Time share token. Each token equals one hour of riding.',
},
}
# She prepares a `CREATE` transaction and issues 10 tokens.
# Here, Alice defines in a tuple that she wants to assign
# these 10 tokens to Bob.
prepared_token_tx = bdb.transactions.prepare(
operation='CREATE',
signers=alice.public_key,
recipients=[([bob.public_key], 10)],
asset=bike_token)
# She fulfills and sends the transaction.
fulfilled_token_tx = bdb.transactions.fulfill(
prepared_token_tx,
private_keys=alice.private_key)
bdb.transactions.send_commit(fulfilled_token_tx)
# We store the `id` of the transaction to use it later on.
bike_token_id = fulfilled_token_tx['id']
# Let's check if the transaction was successful.
assert bdb.transactions.retrieve(bike_token_id), \
'Cannot find transaction {}'.format(bike_token_id)
# Bob owns 10 tokens now.
assert bdb.transactions.retrieve(bike_token_id)['outputs'][0][
'amount'] == '10'
# ## Bob wants to use the bike
# Now that Bob got the tokens and the sun is shining, he wants to get out
# with the bike for three hours.
# To use the bike he has to send the tokens back to Alice.
# To learn about the details of transferring a transaction check out
# [test_basic.py](./test_basic.html)
transfer_asset = {'id': bike_token_id}
output_index = 0
output = fulfilled_token_tx['outputs'][output_index]
transfer_input = {'fulfillment': output['condition']['details'],
'fulfills': {'output_index': output_index,
'transaction_id': fulfilled_token_tx[
'id']},
'owners_before': output['public_keys']}
# To use the tokens Bob has to reassign 7 tokens to himself and the
# amount he wants to use to Alice.
prepared_transfer_tx = bdb.transactions.prepare(
operation='TRANSFER',
asset=transfer_asset,
inputs=transfer_input,
recipients=[([alice.public_key], 3), ([bob.public_key], 7)])
# He signs and sends the transaction.
fulfilled_transfer_tx = bdb.transactions.fulfill(
prepared_transfer_tx,
private_keys=bob.private_key)
sent_transfer_tx = bdb.transactions.send_commit(fulfilled_transfer_tx)
# First, Bob checks if the transaction was successful.
assert bdb.transactions.retrieve(
fulfilled_transfer_tx['id']) == sent_transfer_tx
# There are two outputs in the transaction now.
# The first output shows that Alice got back 3 tokens...
assert bdb.transactions.retrieve(
fulfilled_transfer_tx['id'])['outputs'][0]['amount'] == '3'
# ... while Bob still has 7 left.
assert bdb.transactions.retrieve(
fulfilled_transfer_tx['id'])['outputs'][1]['amount'] == '7'
# ## Bob wants to ride the bike again
# It's been a week and Bob wants to right the bike again.
# Now he wants to ride for 8 hours, that's a lot Bob!
# He prepares the transaction again.
transfer_asset = {'id': bike_token_id}
# This time we need an `output_index` of 1, since we have two outputs
# in the `fulfilled_transfer_tx` we created before. The first output with
# index 0 is for Alice and the second output is for Bob.
# Since Bob wants to spend more of his tokens he has to provide the
# correct output with the correct amount of tokens.
output_index = 1
output = fulfilled_transfer_tx['outputs'][output_index]
transfer_input = {'fulfillment': output['condition']['details'],
'fulfills': {'output_index': output_index,
'transaction_id': fulfilled_transfer_tx['id']},
'owners_before': output['public_keys']}
# This time Bob only provides Alice in the `recipients` because he wants
# to spend all his tokens
prepared_transfer_tx = bdb.transactions.prepare(
operation='TRANSFER',
asset=transfer_asset,
inputs=transfer_input,
recipients=[([alice.public_key], 8)])
fulfilled_transfer_tx = bdb.transactions.fulfill(
prepared_transfer_tx,
private_keys=bob.private_key)
# Oh Bob, what have you done?! You tried to spend more tokens than you had.
# Remember Bob, last time you spent 3 tokens already,
# so you only have 7 left.
with pytest.raises(BadRequest) as error:
bdb.transactions.send_commit(fulfilled_transfer_tx)
# Now Bob gets an error saying that the amount he wanted to spent is
# higher than the amount of tokens he has left.
assert error.value.args[0] == 400
message = 'Invalid transaction (AmountError): The amount used in the ' \
'inputs `7` needs to be same as the amount used in the ' \
'outputs `8`'
assert error.value.args[2]['message'] == message
# We have to stop this test now, I am sorry, but Bob is pretty upset
# about his mistake. See you next time :)

48
acceptance/python/src/test_double_spend.py Normal file
View File

@ -0,0 +1,48 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
# # Double Spend testing
# This test challenge the system with double spends.
import os
from uuid import uuid4
from threading import Thread
import queue
import bigchaindb_driver.exceptions
from bigchaindb_driver import BigchainDB
from bigchaindb_driver.crypto import generate_keypair
def test_double_create():
bdb = BigchainDB(os.environ.get('PLANETMINT_ENDPOINT'))
alice = generate_keypair()
results = queue.Queue()
tx = bdb.transactions.fulfill(
bdb.transactions.prepare(
operation='CREATE',
signers=alice.public_key,
asset={'data': {'uuid': str(uuid4())}}),
private_keys=alice.private_key)
def send_and_queue(tx):
try:
bdb.transactions.send_commit(tx)
results.put('OK')
except bigchaindb_driver.exceptions.TransportError as e:
results.put('FAIL')
t1 = Thread(target=send_and_queue, args=(tx, ))
t2 = Thread(target=send_and_queue, args=(tx, ))
t1.start()
t2.start()
results = [results.get(timeout=2), results.get(timeout=2)]
assert results.count('OK') == 1
assert results.count('FAIL') == 1

126
acceptance/python/src/test_multiple_owners.py Normal file
View File

@ -0,0 +1,126 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
# # Multiple owners integration testing
# This test checks if we can successfully create and transfer a transaction
# with multiple owners.
# The script tests various things like:
#
# - create a transaction with multiple owners
# - check if the transaction is stored and has the right amount of public keys
# - transfer the transaction to a third person
#
# We run a series of checks for each step, that is retrieving
# the transaction from the remote system, and also checking the public keys
# of a given transaction.
#
# This integration test is a rip-off of our
# [tutorial](https://docs.bigchaindb.com/projects/py-driver/en/latest/usage.html).
# ## Imports
# We need some utils from the `os` package, we will interact with
# env variables.
import os
# For this test case we import and use the Python Driver.
from bigchaindb_driver import BigchainDB
from bigchaindb_driver.crypto import generate_keypair
def test_multiple_owners():
# ## Set up a connection to Planetmint
# Check [test_basic.py](./test_basic.html) to get some more details
# about the endpoint.
bdb = BigchainDB(os.environ.get('PLANETMINT_ENDPOINT'))
# Hey Alice and Bob, nice to see you again!
alice, bob = generate_keypair(), generate_keypair()
# ## Alice and Bob create a transaction
# Alice and Bob just moved into a shared flat, no one can afford these
# high rents anymore. Bob suggests to get a dish washer for the
# kitchen. Alice agrees and here they go, creating the asset for their
# dish washer.
dw_asset = {
'data': {
'dish washer': {
'serial_number': 1337
}
}
}
# They prepare a `CREATE` transaction. To have multiple owners, both
# Bob and Alice need to be the recipients.
prepared_dw_tx = bdb.transactions.prepare(
operation='CREATE',
signers=alice.public_key,
recipients=(alice.public_key, bob.public_key),
asset=dw_asset)
# Now they both sign the transaction by providing their private keys.
# And send it afterwards.
fulfilled_dw_tx = bdb.transactions.fulfill(
prepared_dw_tx,
private_keys=[alice.private_key, bob.private_key])
bdb.transactions.send_commit(fulfilled_dw_tx)
# We store the `id` of the transaction to use it later on.
dw_id = fulfilled_dw_tx['id']
# Let's check if the transaction was successful.
assert bdb.transactions.retrieve(dw_id), \
'Cannot find transaction {}'.format(dw_id)
# The transaction should have two public keys in the outputs.
assert len(
bdb.transactions.retrieve(dw_id)['outputs'][0]['public_keys']) == 2
# ## Alice and Bob transfer a transaction to Carol.
# Alice and Bob save a lot of money living together. They often go out
# for dinner and don't cook at home. But now they don't have any dishes to
# wash, so they decide to sell the dish washer to their friend Carol.
# Hey Carol, nice to meet you!
carol = generate_keypair()
# Alice and Bob prepare the transaction to transfer the dish washer to
# Carol.
transfer_asset = {'id': dw_id}
output_index = 0
output = fulfilled_dw_tx['outputs'][output_index]
transfer_input = {'fulfillment': output['condition']['details'],
'fulfills': {'output_index': output_index,
'transaction_id': fulfilled_dw_tx[
'id']},
'owners_before': output['public_keys']}
# Now they create the transaction...
prepared_transfer_tx = bdb.transactions.prepare(
operation='TRANSFER',
asset=transfer_asset,
inputs=transfer_input,
recipients=carol.public_key)
# ... and sign it with their private keys, then send it.
fulfilled_transfer_tx = bdb.transactions.fulfill(
prepared_transfer_tx,
private_keys=[alice.private_key, bob.private_key])
sent_transfer_tx = bdb.transactions.send_commit(fulfilled_transfer_tx)
# They check if the transaction was successful.
assert bdb.transactions.retrieve(
fulfilled_transfer_tx['id']) == sent_transfer_tx
# The owners before should include both Alice and Bob.
assert len(
bdb.transactions.retrieve(fulfilled_transfer_tx['id'])['inputs'][0][
'owners_before']) == 2
# While the new owner is Carol.
assert bdb.transactions.retrieve(fulfilled_transfer_tx['id'])[
'outputs'][0]['public_keys'][0] == carol.public_key

101
acceptance/python/src/test_naughty_strings.py Normal file
View File

@ -0,0 +1,101 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
# ## Testing potentially hazardous strings
# This test uses a library of `naughty` strings (code injections, weird unicode chars., etc.) as both keys and values.
# We look for either a successful tx, or in the case that we use a naughty string as a key, and it violates some key
# constraints, we expect to receive a well formatted error message.
# ## Imports
# We need some utils from the `os` package, we will interact with
# env variables.
import os
# Since the naughty strings get encoded and decoded in odd ways,
# we'll use a regex to sweep those details under the rug.
import re
# We'll use a nice library of naughty strings...
from blns import blns
# And parameterize our test so each one is treated as a separate test case
import pytest
# For this test case we import and use the Python Driver.
from bigchaindb_driver import BigchainDB
from bigchaindb_driver.crypto import generate_keypair
from bigchaindb_driver.exceptions import BadRequest
naughty_strings = blns.all()
# This is our base test case, but we'll reuse it to send naughty strings as both keys and values.
def send_naughty_tx(asset, metadata):
# ## Set up a connection to Planetmint
# Check [test_basic.py](./test_basic.html) to get some more details
# about the endpoint.
bdb = BigchainDB(os.environ.get('PLANETMINT_ENDPOINT'))
# Here's Alice.
alice = generate_keypair()
# Alice is in a naughty mood today, so she creates a tx with some naughty strings
prepared_transaction = bdb.transactions.prepare(
operation='CREATE',
signers=alice.public_key,
asset=asset,
metadata=metadata)
# She fulfills the transaction
fulfilled_transaction = bdb.transactions.fulfill(
prepared_transaction,
private_keys=alice.private_key)
# The fulfilled tx gets sent to the BDB network
try:
sent_transaction = bdb.transactions.send_commit(fulfilled_transaction)
except BadRequest as e:
sent_transaction = e
# If her key contained a '.', began with a '$', or contained a NUL character
regex = '.*\..*|\$.*|.*\x00.*'
key = next(iter(metadata))
if re.match(regex, key):
# Then she expects a nicely formatted error code
status_code = sent_transaction.status_code
error = sent_transaction.error
regex = (
r'\{\s*\n*'
r'\s*"message":\s*"Invalid transaction \(ValidationError\):\s*'
r'Invalid key name.*The key name cannot contain characters.*\n*'
r'\s*"status":\s*400\n*'
r'\s*\}\n*')
assert status_code == 400
assert re.fullmatch(regex, error), sent_transaction
# Otherwise, she expects to see her transaction in the database
elif 'id' in sent_transaction.keys():
tx_id = sent_transaction['id']
assert bdb.transactions.retrieve(tx_id)
# If neither condition was true, then something weird happened...
else:
raise TypeError(sent_transaction)
@pytest.mark.parametrize("naughty_string", naughty_strings, ids=naughty_strings)
def test_naughty_keys(naughty_string):
asset = {'data': {naughty_string: 'nice_value'}}
metadata = {naughty_string: 'nice_value'}
send_naughty_tx(asset, metadata)
@pytest.mark.parametrize("naughty_string", naughty_strings, ids=naughty_strings)
def test_naughty_values(naughty_string):
asset = {'data': {'nice_key': naughty_string}}
metadata = {'nice_key': naughty_string}
send_naughty_tx(asset, metadata)

132
acceptance/python/src/test_stream.py Normal file
View File

@ -0,0 +1,132 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
# # Stream Acceptance Test
# This test checks if the event stream works correctly. The basic idea of this
# test is to generate some random **valid** transaction, send them to a
# Planetmint node, and expect those transactions to be returned by the valid
# transactions Stream API. During this test, two threads work together,
# sharing a queue to exchange events.
#
# - The *main thread* first creates and sends the transactions to Planetmint;
# then it run through all events in the shared queue to check if all
# transactions sent have been validated by Planetmint.
# - The *listen thread* listens to the events coming from Planetmint and puts
# them in a queue shared with the main thread.
import os
import queue
import json
from threading import Thread, Event
from uuid import uuid4
# For this script, we need to set up a websocket connection, that's the reason
# we import the
# [websocket](https://github.com/websocket-client/websocket-client) module
from websocket import create_connection
from bigchaindb_driver import BigchainDB
from bigchaindb_driver.crypto import generate_keypair
def test_stream():
# ## Set up the test
# We use the env variable `BICHAINDB_ENDPOINT` to know where to connect.
# Check [test_basic.py](./test_basic.html) for more information.
BDB_ENDPOINT = os.environ.get('PLANETMINT_ENDPOINT')
# *That's pretty bad, but let's do like this for now.*
WS_ENDPOINT = 'ws://{}:9985/api/v1/streams/valid_transactions'.format(BDB_ENDPOINT.rsplit(':')[0])
bdb = BigchainDB(BDB_ENDPOINT)
# Hello to Alice again, she is pretty active in those tests, good job
# Alice!
alice = generate_keypair()
# We need few variables to keep the state, specifically we need `sent` to
# keep track of all transactions Alice sent to Planetmint, while `received`
# are the transactions Planetmint validated and sent back to her.
sent = []
received = queue.Queue()
# In this test we use a websocket. The websocket must be started **before**
# sending transactions to Planetmint, otherwise we might lose some
# transactions. The `ws_ready` event is used to synchronize the main thread
# with the listen thread.
ws_ready = Event()
# ## Listening to events
# This is the function run by the complementary thread.
def listen():
# First we connect to the remote endpoint using the WebSocket protocol.
ws = create_connection(WS_ENDPOINT)
# After the connection has been set up, we can signal the main thread
# to proceed (continue reading, it should make sense in a second.)
ws_ready.set()
# It's time to consume all events coming from the Planetmint stream API.
# Every time a new event is received, it is put in the queue shared
# with the main thread.
while True:
result = ws.recv()
received.put(result)
# Put `listen` in a thread, and start it. Note that `listen` is a local
# function and it can access all variables in the enclosing function.
t = Thread(target=listen, daemon=True)
t.start()
# ## Pushing the transactions to Planetmint
# After starting the listen thread, we wait for it to connect, and then we
# proceed.
ws_ready.wait()
# Here we prepare, sign, and send ten different `CREATE` transactions. To
# make sure each transaction is different from the other, we generate a
# random `uuid`.
for _ in range(10):
tx = bdb.transactions.fulfill(
bdb.transactions.prepare(
operation='CREATE',
signers=alice.public_key,
asset={'data': {'uuid': str(uuid4())}}),
private_keys=alice.private_key)
# We don't want to wait for each transaction to be in a block. By using
# `async` mode, we make sure that the driver returns as soon as the
# transaction is pushed to the Planetmint API. Remember: we expect all
# transactions to be in the shared queue: this is a two phase test,
# first we send a bunch of transactions, then we check if they are
# valid (and, in this case, they should).
bdb.transactions.send_async(tx)
# The `id` of every sent transaction is then stored in a list.
sent.append(tx['id'])
# ## Check the valid transactions coming from Planetmint
# Now we are ready to check if Planetmint did its job. A simple way to
# check if all sent transactions have been processed is to **remove** from
# `sent` the transactions we get from the *listen thread*. At one point in
# time, `sent` should be empty, and we exit the test.
while sent:
# To avoid waiting forever, we have an arbitrary timeout of 5
# seconds: it should be enough time for Planetmint to create
# blocks, in fact a new block is created every second. If we hit
# the timeout, then game over ¯\\\_(ツ)\_/¯
try:
event = received.get(timeout=5)
txid = json.loads(event)['transaction_id']
except queue.Empty:
assert False, 'Did not receive all expected transactions'
# Last thing is to try to remove the `txid` from the set of sent
# transactions. If this test is running in parallel with others, we
# might get a transaction id of another test, and `remove` can fail.
# It's OK if this happens.
try:
sent.remove(txid)
except ValueError:
pass

42
bigchaindb/README.md Normal file
View File

@ -0,0 +1,42 @@
<!---
Copyright © 2020 Interplanetary Database Association e.V.,
Planetmint and IPDB software contributors.
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# Overview
A high-level description of the files and subdirectories of Planetmint.
## Files
### [`lib.py`](lib.py)
The `Planetmint` class is defined here. Most node-level operations and database interactions are found in this file. This is the place to start if you are interested in implementing a server API, since many of these class methods concern Planetmint interacting with the outside world.
### [`models.py`](./models.py)
`Block`, `Transaction`, and `Asset` classes are defined here. The classes mirror the block and transaction structure from the [documentation](https://docs.bigchaindb.com/projects/server/en/latest/data-models/index.html), but also include methods for validation and signing.
### [`validation.py`](./validation.py)
Base class for validation methods (verification of votes, blocks, and transactions). The actual logic is mostly found in `transaction` and `block` models, defined in [`models.py`](./models.py).
### [`processes.py`](./processes.py)
Entry point for the Planetmint process, after initialization. All subprocesses are started here: processes to handle new blocks, votes, etc.
### [`config_utils.py`](./config_utils.py)
Methods for managing the configuration, including loading configuration files, automatically generating the configuration, and keeping the configuration consistent across Planetmint instances.
## Folders
### [`commands`](./commands)
Contains code for the [CLI](https://docs.bigchaindb.com/projects/server/en/latest/server-reference/bigchaindb-cli.html) for Planetmint.
### [`db`](./db)
Code for building the database connection, creating indexes, and other database setup tasks.

106
bigchaindb/__init__.py Normal file
View File

@ -0,0 +1,106 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
import copy
import logging
from bigchaindb.log import DEFAULT_LOGGING_CONFIG as log_config
from bigchaindb.lib import BigchainDB # noqa
from bigchaindb.migrations.chain_migration_election import ChainMigrationElection
from bigchaindb.version import __version__ # noqa
from bigchaindb.core import App # noqa
# from functools import reduce
# PORT_NUMBER = reduce(lambda x, y: x * y, map(ord, 'BigchainDB')) % 2**16
# basically, the port number is 9984
# The following variable is used by `bigchaindb configure` to
# prompt the user for database values. We cannot rely on
# _base_database_localmongodb.keys() because dicts are unordered.
# I tried to configure
_database_keys_map = {
'localmongodb': ('host', 'port', 'name'),
}
_base_database_localmongodb = {
'host': 'localhost',
'port': 27017,
'name': 'bigchain',
'replicaset': None,
'login': None,
'password': None,
}
_database_localmongodb = {
'backend': 'localmongodb',
'connection_timeout': 5000,
'max_tries': 3,
'ssl': False,
'ca_cert': None,
'certfile': None,
'keyfile': None,
'keyfile_passphrase': None,
'crlfile': None,
}
_database_localmongodb.update(_base_database_localmongodb)
_database_map = {
'localmongodb': _database_localmongodb,
}
config = {
'server': {
# Note: this section supports all the Gunicorn settings:
# - http://docs.gunicorn.org/en/stable/settings.html
'bind': 'localhost:9984',
'loglevel': logging.getLevelName(
log_config['handlers']['console']['level']).lower(),
'workers': None, # if None, the value will be cpu_count * 2 + 1
},
'wsserver': {
'scheme': 'ws',
'host': 'localhost',
'port': 9985,
'advertised_scheme': 'ws',
'advertised_host': 'localhost',
'advertised_port': 9985,
},
'tendermint': {
'host': 'localhost',
'port': 26657,
'version': 'v0.31.5', # look for __tm_supported_versions__
},
# FIXME: hardcoding to localmongodb for now
'database': _database_map['localmongodb'],
'log': {
'file': log_config['handlers']['file']['filename'],
'error_file': log_config['handlers']['errors']['filename'],
'level_console': logging.getLevelName(
log_config['handlers']['console']['level']).lower(),
'level_logfile': logging.getLevelName(
log_config['handlers']['file']['level']).lower(),
'datefmt_console': log_config['formatters']['console']['datefmt'],
'datefmt_logfile': log_config['formatters']['file']['datefmt'],
'fmt_console': log_config['formatters']['console']['format'],
'fmt_logfile': log_config['formatters']['file']['format'],
'granular_levels': {},
},
}
# We need to maintain a backup copy of the original config dict in case
# the user wants to reconfigure the node. Check ``bigchaindb.config_utils``
# for more info.
_config = copy.deepcopy(config)
from bigchaindb.common.transaction import Transaction # noqa
from bigchaindb import models # noqa
from bigchaindb.upsert_validator import ValidatorElection # noqa
from bigchaindb.elections.vote import Vote # noqa
Transaction.register_type(Transaction.CREATE, models.Transaction)
Transaction.register_type(Transaction.TRANSFER, models.Transaction)
Transaction.register_type(ValidatorElection.OPERATION, ValidatorElection)
Transaction.register_type(ChainMigrationElection.OPERATION, ChainMigrationElection)
Transaction.register_type(Vote.OPERATION, Vote)

51
bigchaindb/backend/README.md Normal file
View File

@ -0,0 +1,51 @@
<!---
Copyright © 2020 Interplanetary Database Association e.V.,
Planetmint and IPDB software contributors.
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# Backend Interfaces
## Structure
- [`connection.py`](./connection.py): Database connection-related interfaces
- [`query.py`](./query.py): Database query-related interfaces, dispatched through single-dispatch
- [`schema.py`](./schema.py): Database setup and schema-related interfaces, dispatched through
single-dispatch
Built-in implementations (e.g. [MongoDB's](./localmongodb)) are provided in sub-directories and
have their connection type's location exposed as `BACKENDS` in [`connection.py`](./connection.py).
## Single-Dispatched Interfaces
The architecture of this module is based heavily upon Python's newly-introduced [single-dispatch
generic functions](https://www.python.org/dev/peps/pep-0443/). Single-dispatch is convenient,
because it allows Python, rather than something we design ourselves, to manage the dispatching of
generic functions based on certain conditions being met (e.g. the database backend to use).
To see what this looks like in Planetmint, first note that our backend interfaces have been
configured to dispatch based on a backend's **connection type**.
Call `bigchaindb.backend.connect()` to create an instance of a `Connection`:
```python
from bigchaindb.backend import connect
connection = connect() # By default, uses the current configuration for backend, host, port, etc.
```
Then, we can call a backend function by directly calling its interface:
```python
from bigchaindb.backend import query
query.write_transaction(connection, ...)
```
Notice that we don't need to care about which backend implementation to use or how to access it.
Code can simply call the base interface function with a `Connection` instance, and single-dispatch
will handle routing the call to the actual implementation.
Planetmint will load and register the configured backend's implementation automatically (see
`bigchaindb.backend.connect()`), so you should always just be able to call an interface function if
you have a `Connection` instance. A few helper utilities (see [`backend/utils.py`](./utils.py)) are
also provided to make registering new backend implementations easier.

17
bigchaindb/backend/__init__.py Normal file
View File

@ -0,0 +1,17 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
"""Generic backend database interfaces expected by Planetmint.
The interfaces in this module allow Planetmint to be agnostic about its
database backend. One can configure Planetmint to use different databases as
its data store by setting the ``database.backend`` property in the
configuration or the ``PLANETMINT_DATABASE_BACKEND`` environment variable.
"""
# Include the backend interfaces
from bigchaindb.backend import schema, query # noqa
from bigchaindb.backend.connection import connect # noqa

170
bigchaindb/backend/connection.py Normal file
View File

@ -0,0 +1,170 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
import logging
from importlib import import_module
from itertools import repeat
import bigchaindb
from bigchaindb.backend.exceptions import ConnectionError
from bigchaindb.backend.utils import get_bigchaindb_config_value, get_bigchaindb_config_value_or_key_error
from bigchaindb.common.exceptions import ConfigurationError
BACKENDS = {
'localmongodb': 'bigchaindb.backend.localmongodb.connection.LocalMongoDBConnection',
}
logger = logging.getLogger(__name__)
def connect(backend=None, host=None, port=None, name=None, max_tries=None,
connection_timeout=None, replicaset=None, ssl=None, login=None, password=None,
ca_cert=None, certfile=None, keyfile=None, keyfile_passphrase=None,
crlfile=None):
"""Create a new connection to the database backend.
All arguments default to the current configuration's values if not
given.
Args:
backend (str): the name of the backend to use.
host (str): the host to connect to.
port (int): the port to connect to.
name (str): the name of the database to use.
replicaset (str): the name of the replica set (only relevant for
MongoDB connections).
Returns:
An instance of :class:`~bigchaindb.backend.connection.Connection`
based on the given (or defaulted) :attr:`backend`.
Raises:
:exc:`~ConnectionError`: If the connection to the database fails.
:exc:`~ConfigurationError`: If the given (or defaulted) :attr:`backend`
is not supported or could not be loaded.
:exc:`~AuthenticationError`: If there is a OperationFailure due to
Authentication failure after connecting to the database.
"""
backend = backend or get_bigchaindb_config_value_or_key_error('backend')
host = host or get_bigchaindb_config_value_or_key_error('host')
port = port or get_bigchaindb_config_value_or_key_error('port')
dbname = name or get_bigchaindb_config_value_or_key_error('name')
# Not sure how to handle this here. This setting is only relevant for
# mongodb.
# I added **kwargs for both RethinkDBConnection and MongoDBConnection
# to handle these these additional args. In case of RethinkDBConnection
# it just does not do anything with it.
#
# UPD: RethinkDBConnection is not here anymore cause we no longer support RethinkDB.
# The problem described above might be reconsidered next time we introduce a backend,
# if it ever happens.
replicaset = replicaset or get_bigchaindb_config_value('replicaset')
ssl = ssl if ssl is not None else get_bigchaindb_config_value('ssl', False)
login = login or get_bigchaindb_config_value('login')
password = password or get_bigchaindb_config_value('password')
ca_cert = ca_cert or get_bigchaindb_config_value('ca_cert')
certfile = certfile or get_bigchaindb_config_value('certfile')
keyfile = keyfile or get_bigchaindb_config_value('keyfile')
keyfile_passphrase = keyfile_passphrase or get_bigchaindb_config_value('keyfile_passphrase', None)
crlfile = crlfile or get_bigchaindb_config_value('crlfile')
try:
module_name, _, class_name = BACKENDS[backend].rpartition('.')
Class = getattr(import_module(module_name), class_name)
except KeyError:
raise ConfigurationError('Backend `{}` is not supported. '
'Planetmint currently supports {}'.format(backend, BACKENDS.keys()))
except (ImportError, AttributeError) as exc:
raise ConfigurationError('Error loading backend `{}`'.format(backend)) from exc
logger.debug('Connection: {}'.format(Class))
return Class(host=host, port=port, dbname=dbname,
max_tries=max_tries, connection_timeout=connection_timeout,
replicaset=replicaset, ssl=ssl, login=login, password=password,
ca_cert=ca_cert, certfile=certfile, keyfile=keyfile,
keyfile_passphrase=keyfile_passphrase, crlfile=crlfile)
class Connection:
"""Connection class interface.
All backend implementations should provide a connection class that inherits
from and implements this class.
"""
def __init__(self, host=None, port=None, dbname=None,
connection_timeout=None, max_tries=None,
**kwargs):
"""Create a new :class:`~.Connection` instance.
Args:
host (str): the host to connect to.
port (int): the port to connect to.
dbname (str): the name of the database to use.
connection_timeout (int, optional): the milliseconds to wait
until timing out the database connection attempt.
Defaults to 5000ms.
max_tries (int, optional): how many tries before giving up,
if 0 then try forever. Defaults to 3.
**kwargs: arbitrary keyword arguments provided by the
configuration's ``database`` settings
"""
dbconf = bigchaindb.config['database']
self.host = host or dbconf['host']
self.port = port or dbconf['port']
self.dbname = dbname or dbconf['name']
self.connection_timeout = connection_timeout if connection_timeout is not None \
else dbconf['connection_timeout']
self.max_tries = max_tries if max_tries is not None else dbconf['max_tries']
self.max_tries_counter = range(self.max_tries) if self.max_tries != 0 else repeat(0)
self._conn = None
@property
def conn(self):
if self._conn is None:
self.connect()
return self._conn
def run(self, query):
"""Run a query.
Args:
query: the query to run
Raises:
:exc:`~DuplicateKeyError`: If the query fails because of a
duplicate key constraint.
:exc:`~OperationFailure`: If the query fails for any other
reason.
:exc:`~ConnectionError`: If the connection to the database
fails.
"""
raise NotImplementedError()
def connect(self):
"""Try to connect to the database.
Raises:
:exc:`~ConnectionError`: If the connection to the database
fails.
"""
attempt = 0
for i in self.max_tries_counter:
attempt += 1
try:
self._conn = self._connect()
except ConnectionError as exc:
logger.warning('Attempt %s/%s. Connection to %s:%s failed after %sms.',
attempt, self.max_tries if self.max_tries != 0 else '∞',
self.host, self.port, self.connection_timeout)
if attempt == self.max_tries:
logger.critical('Cannot connect to the Database. Giving up.')
raise ConnectionError() from exc
else:
break

22
bigchaindb/backend/exceptions.py Normal file
View File

@ -0,0 +1,22 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
from bigchaindb.exceptions import BigchainDBError
class BackendError(BigchainDBError):
"""Top level exception for any backend exception."""
class ConnectionError(BackendError):
"""Exception raised when the connection to the backend fails."""
class OperationError(BackendError):
"""Exception raised when a backend operation fails."""
class DuplicateKeyError(OperationError):
"""Exception raised when an insert fails because the key is not unique"""

28
bigchaindb/backend/localmongodb/__init__.py Normal file
View File

@ -0,0 +1,28 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
"""MongoDB backend implementation.
Contains a MongoDB-specific implementation of the
:mod:`~bigchaindb.backend.schema` and :mod:`~bigchaindb.backend.query` interfaces.
You can specify Planetmint to use MongoDB as its database backend by either
setting ``database.backend`` to ``'localmongodb'`` in your configuration file, or
setting the ``PLANETMINT_DATABASE_BACKEND`` environment variable to
``'localmongodb'``.
MongoDB is the default database backend for Planetmint.
If configured to use MongoDB, Planetmint will automatically return instances
of :class:`~bigchaindb.backend.localmongodb.LocalMongoDBConnection` for
:func:`~bigchaindb.backend.connection.connect` and dispatch calls of the
generic backend interfaces to the implementations in this module.
"""
# Register the single dispatched modules on import.
from bigchaindb.backend.localmongodb import schema, query # noqa
# MongoDBConnection should always be accessed via
# ``bigchaindb.backend.connect()``.

136
bigchaindb/backend/localmongodb/connection.py Normal file
View File

@ -0,0 +1,136 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
import logging
from ssl import CERT_REQUIRED
import pymongo
from bigchaindb.backend.connection import Connection
from bigchaindb.backend.exceptions import (DuplicateKeyError,
OperationError,
ConnectionError)
from bigchaindb.backend.utils import get_bigchaindb_config_value
from bigchaindb.common.exceptions import ConfigurationError
from bigchaindb.utils import Lazy
logger = logging.getLogger(__name__)
class LocalMongoDBConnection(Connection):
def __init__(self, replicaset=None, ssl=None, login=None, password=None,
ca_cert=None, certfile=None, keyfile=None,
keyfile_passphrase=None, crlfile=None, **kwargs):
"""Create a new Connection instance.
Args:
replicaset (str, optional): the name of the replica set to
connect to.
**kwargs: arbitrary keyword arguments provided by the
configuration's ``database`` settings
"""
super().__init__(**kwargs)
self.replicaset = replicaset or get_bigchaindb_config_value('replicaset')
self.ssl = ssl if ssl is not None else get_bigchaindb_config_value('ssl', False)
self.login = login or get_bigchaindb_config_value('login')
self.password = password or get_bigchaindb_config_value('password')
self.ca_cert = ca_cert or get_bigchaindb_config_value('ca_cert')
self.certfile = certfile or get_bigchaindb_config_value('certfile')
self.keyfile = keyfile or get_bigchaindb_config_value('keyfile')
self.keyfile_passphrase = keyfile_passphrase or get_bigchaindb_config_value('keyfile_passphrase')
self.crlfile = crlfile or get_bigchaindb_config_value('crlfile')
@property
def db(self):
return self.conn[self.dbname]
def query(self):
return Lazy()
def collection(self, name):
"""Return a lazy object that can be used to compose a query.
Args:
name (str): the name of the collection to query.
"""
return self.query()[self.dbname][name]
def run(self, query):
try:
try:
return query.run(self.conn)
except pymongo.errors.AutoReconnect:
logger.warning('Lost connection to the database, '
'retrying query.')
return query.run(self.conn)
except pymongo.errors.AutoReconnect as exc:
raise ConnectionError from exc
except pymongo.errors.DuplicateKeyError as exc:
raise DuplicateKeyError from exc
except pymongo.errors.OperationFailure as exc:
print(f'DETAILS: {exc.details}')
raise OperationError from exc
def _connect(self):
"""Try to connect to the database.
Raises:
:exc:`~ConnectionError`: If the connection to the database
fails.
:exc:`~AuthenticationError`: If there is a OperationFailure due to
Authentication failure after connecting to the database.
:exc:`~ConfigurationError`: If there is a ConfigurationError while
connecting to the database.
"""
try:
# FYI: the connection process might raise a
# `ServerSelectionTimeoutError`, that is a subclass of
# `ConnectionFailure`.
# The presence of ca_cert, certfile, keyfile, crlfile implies the
# use of certificates for TLS connectivity.
if self.ca_cert is None or self.certfile is None or \
self.keyfile is None or self.crlfile is None:
client = pymongo.MongoClient(self.host,
self.port,
replicaset=self.replicaset,
serverselectiontimeoutms=self.connection_timeout,
ssl=self.ssl,
**MONGO_OPTS)
if self.login is not None and self.password is not None:
client[self.dbname].authenticate(self.login, self.password)
else:
logger.info('Connecting to MongoDB over TLS/SSL...')
client = pymongo.MongoClient(self.host,
self.port,
replicaset=self.replicaset,
serverselectiontimeoutms=self.connection_timeout,
ssl=self.ssl,
ssl_ca_certs=self.ca_cert,
ssl_certfile=self.certfile,
ssl_keyfile=self.keyfile,
ssl_pem_passphrase=self.keyfile_passphrase,
ssl_crlfile=self.crlfile,
ssl_cert_reqs=CERT_REQUIRED,
**MONGO_OPTS)
if self.login is not None:
client[self.dbname].authenticate(self.login,
mechanism='MONGODB-X509')
return client
except (pymongo.errors.ConnectionFailure,
pymongo.errors.OperationFailure) as exc:
logger.info('Exception in _connect(): {}'.format(exc))
raise ConnectionError(str(exc)) from exc
except pymongo.errors.ConfigurationError as exc:
raise ConfigurationError from exc
MONGO_OPTS = {
'socketTimeoutMS': 20000,
}

377
bigchaindb/backend/localmongodb/query.py Normal file
View File

@ -0,0 +1,377 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
"""Query implementation for MongoDB"""
from pymongo import DESCENDING
from bigchaindb import backend
from bigchaindb.backend.exceptions import DuplicateKeyError
from bigchaindb.backend.utils import module_dispatch_registrar
from bigchaindb.backend.localmongodb.connection import LocalMongoDBConnection
from bigchaindb.common.transaction import Transaction
register_query = module_dispatch_registrar(backend.query)
@register_query(LocalMongoDBConnection)
def store_transactions(conn, signed_transactions):
return conn.run(conn.collection('transactions')
.insert_many(signed_transactions))
@register_query(LocalMongoDBConnection)
def get_transaction(conn, transaction_id):
return conn.run(
conn.collection('transactions')
.find_one({'id': transaction_id}, {'_id': 0}))
@register_query(LocalMongoDBConnection)
def get_transactions(conn, transaction_ids):
try:
return conn.run(
conn.collection('transactions')
.find({'id': {'$in': transaction_ids}},
projection={'_id': False}))
except IndexError:
pass
@register_query(LocalMongoDBConnection)
def store_metadatas(conn, metadata):
return conn.run(
conn.collection('metadata')
.insert_many(metadata, ordered=False))
@register_query(LocalMongoDBConnection)
def get_metadata(conn, transaction_ids):
return conn.run(
conn.collection('metadata')
.find({'id': {'$in': transaction_ids}},
projection={'_id': False}))
@register_query(LocalMongoDBConnection)
def store_asset(conn, asset):
try:
return conn.run(
conn.collection('assets')
.insert_one(asset))
except DuplicateKeyError:
pass
@register_query(LocalMongoDBConnection)
def store_assets(conn, assets):
return conn.run(
conn.collection('assets')
.insert_many(assets, ordered=False))
@register_query(LocalMongoDBConnection)
def get_asset(conn, asset_id):
try:
return conn.run(
conn.collection('assets')
.find_one({'id': asset_id}, {'_id': 0, 'id': 0}))
except IndexError:
pass
@register_query(LocalMongoDBConnection)
def get_assets(conn, asset_ids):
return conn.run(
conn.collection('assets')
.find({'id': {'$in': asset_ids}},
projection={'_id': False}))
@register_query(LocalMongoDBConnection)
def get_spent(conn, transaction_id, output):
query = {'inputs':
{'$elemMatch':
{'$and': [{'fulfills.transaction_id': transaction_id},
{'fulfills.output_index': output}]}}}
return conn.run(
conn.collection('transactions')
.find(query, {'_id': 0}))
@register_query(LocalMongoDBConnection)
def get_latest_block(conn):
return conn.run(
conn.collection('blocks')
.find_one(projection={'_id': False},
sort=[('height', DESCENDING)]))
@register_query(LocalMongoDBConnection)
def store_block(conn, block):
try:
return conn.run(
conn.collection('blocks')
.insert_one(block))
except DuplicateKeyError:
pass
@register_query(LocalMongoDBConnection)
def get_txids_filtered(conn, asset_id, operation=None, last_tx=None):
match = {
Transaction.CREATE: {'operation': 'CREATE', 'id': asset_id},
Transaction.TRANSFER: {'operation': 'TRANSFER', 'asset.id': asset_id},
None: {'$or': [{'asset.id': asset_id}, {'id': asset_id}]},
}[operation]
cursor = conn.run(conn.collection('transactions').find(match))
if last_tx:
cursor = cursor.sort([('$natural', DESCENDING)]).limit(1)
return (elem['id'] for elem in cursor)
@register_query(LocalMongoDBConnection)
def text_search(conn, search, *, language='english', case_sensitive=False,
diacritic_sensitive=False, text_score=False, limit=0, table='assets'):
cursor = conn.run(
conn.collection(table)
.find({'$text': {
'$search': search,
'$language': language,
'$caseSensitive': case_sensitive,
'$diacriticSensitive': diacritic_sensitive}},
{'score': {'$meta': 'textScore'}, '_id': False})
.sort([('score', {'$meta': 'textScore'})])
.limit(limit))
if text_score:
return cursor
return (_remove_text_score(obj) for obj in cursor)
def _remove_text_score(asset):
asset.pop('score', None)
return asset
@register_query(LocalMongoDBConnection)
def get_owned_ids(conn, owner):
cursor = conn.run(
conn.collection('transactions').aggregate([
{'$match': {'outputs.public_keys': owner}},
{'$project': {'_id': False}}
]))
return cursor
@register_query(LocalMongoDBConnection)
def get_spending_transactions(conn, inputs):
transaction_ids = [i['transaction_id'] for i in inputs]
output_indexes = [i['output_index'] for i in inputs]
query = {'inputs':
{'$elemMatch':
{'$and':
[
{'fulfills.transaction_id': {'$in': transaction_ids}},
{'fulfills.output_index': {'$in': output_indexes}}
]}}}
cursor = conn.run(
conn.collection('transactions').find(query, {'_id': False}))
return cursor
@register_query(LocalMongoDBConnection)
def get_block(conn, block_id):
return conn.run(
conn.collection('blocks')
.find_one({'height': block_id},
projection={'_id': False}))
@register_query(LocalMongoDBConnection)
def get_block_with_transaction(conn, txid):
return conn.run(
conn.collection('blocks')
.find({'transactions': txid},
projection={'_id': False, 'height': True}))
@register_query(LocalMongoDBConnection)
def delete_transactions(conn, txn_ids):
conn.run(conn.collection('assets').delete_many({'id': {'$in': txn_ids}}))
conn.run(conn.collection('metadata').delete_many({'id': {'$in': txn_ids}}))
conn.run(conn.collection('transactions').delete_many({'id': {'$in': txn_ids}}))
@register_query(LocalMongoDBConnection)
def store_unspent_outputs(conn, *unspent_outputs):
if unspent_outputs:
try:
return conn.run(
conn.collection('utxos').insert_many(
unspent_outputs,
ordered=False,
)
)
except DuplicateKeyError:
# TODO log warning at least
pass
@register_query(LocalMongoDBConnection)
def delete_unspent_outputs(conn, *unspent_outputs):
if unspent_outputs:
return conn.run(
conn.collection('utxos').delete_many({
'$or': [{
'$and': [
{'transaction_id': unspent_output['transaction_id']},
{'output_index': unspent_output['output_index']},
],
} for unspent_output in unspent_outputs]
})
)
@register_query(LocalMongoDBConnection)
def get_unspent_outputs(conn, *, query=None):
if query is None:
query = {}
return conn.run(conn.collection('utxos').find(query,
projection={'_id': False}))
@register_query(LocalMongoDBConnection)
def store_pre_commit_state(conn, state):
return conn.run(
conn.collection('pre_commit')
.replace_one({}, state, upsert=True)
)
@register_query(LocalMongoDBConnection)
def get_pre_commit_state(conn):
return conn.run(conn.collection('pre_commit').find_one())
@register_query(LocalMongoDBConnection)
def store_validator_set(conn, validators_update):
height = validators_update['height']
return conn.run(
conn.collection('validators').replace_one(
{'height': height},
validators_update,
upsert=True
)
)
@register_query(LocalMongoDBConnection)
def delete_validator_set(conn, height):
return conn.run(
conn.collection('validators').delete_many({'height': height})
)
@register_query(LocalMongoDBConnection)
def store_election(conn, election_id, height, is_concluded):
return conn.run(
conn.collection('elections').replace_one(
{'election_id': election_id,
'height': height},
{'election_id': election_id,
'height': height,
'is_concluded': is_concluded},
upsert=True,
)
)
@register_query(LocalMongoDBConnection)
def store_elections(conn, elections):
return conn.run(
conn.collection('elections').insert_many(elections)
)
@register_query(LocalMongoDBConnection)
def delete_elections(conn, height):
return conn.run(
conn.collection('elections').delete_many({'height': height})
)
@register_query(LocalMongoDBConnection)
def get_validator_set(conn, height=None):
query = {}
if height is not None:
query = {'height': {'$lte': height}}
cursor = conn.run(
conn.collection('validators')
.find(query, projection={'_id': False})
.sort([('height', DESCENDING)])
.limit(1)
)
return next(cursor, None)
@register_query(LocalMongoDBConnection)
def get_election(conn, election_id):
query = {'election_id': election_id}
return conn.run(
conn.collection('elections')
.find_one(query, projection={'_id': False},
sort=[('height', DESCENDING)])
)
@register_query(LocalMongoDBConnection)
def get_asset_tokens_for_public_key(conn, asset_id, public_key):
query = {'outputs.public_keys': [public_key],
'asset.id': asset_id}
cursor = conn.run(
conn.collection('transactions').aggregate([
{'$match': query},
{'$project': {'_id': False}}
]))
return cursor
@register_query(LocalMongoDBConnection)
def store_abci_chain(conn, height, chain_id, is_synced=True):
return conn.run(
conn.collection('abci_chains').replace_one(
{'height': height},
{'height': height, 'chain_id': chain_id,
'is_synced': is_synced},
upsert=True,
)
)
@register_query(LocalMongoDBConnection)
def delete_abci_chain(conn, height):
return conn.run(
conn.collection('abci_chains').delete_many({'height': height})
)
@register_query(LocalMongoDBConnection)
def get_latest_abci_chain(conn):
return conn.run(
conn.collection('abci_chains')
.find_one(projection={'_id': False}, sort=[('height', DESCENDING)])
)

90
bigchaindb/backend/localmongodb/schema.py Normal file
View File

@ -0,0 +1,90 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
"""Utils to initialize and drop the database."""
import logging
from pymongo import ASCENDING, DESCENDING, TEXT
from pymongo.errors import CollectionInvalid
from bigchaindb import backend
from bigchaindb.backend.utils import module_dispatch_registrar
from bigchaindb.backend.localmongodb.connection import LocalMongoDBConnection
logger = logging.getLogger(__name__)
register_schema = module_dispatch_registrar(backend.schema)
INDEXES = {
'transactions': [
('id', dict(unique=True, name='transaction_id')),
('asset.id', dict(name='asset_id')),
('outputs.public_keys', dict(name='outputs')),
([('inputs.fulfills.transaction_id', ASCENDING),
('inputs.fulfills.output_index', ASCENDING)], dict(name='inputs')),
],
'assets': [
('id', dict(name='asset_id', unique=True)),
([('$**', TEXT)], dict(name='text')),
],
'blocks': [
([('height', DESCENDING)], dict(name='height', unique=True)),
],
'metadata': [
('id', dict(name='transaction_id', unique=True)),
([('$**', TEXT)], dict(name='text')),
],
'utxos': [
([('transaction_id', ASCENDING),
('output_index', ASCENDING)], dict(name='utxo', unique=True)),
],
'pre_commit': [
('height', dict(name='height', unique=True)),
],
'elections': [
([('height', DESCENDING), ('election_id', ASCENDING)],
dict(name='election_id_height', unique=True)),
],
'validators': [
('height', dict(name='height', unique=True)),
],
'abci_chains': [
('height', dict(name='height', unique=True)),
('chain_id', dict(name='chain_id', unique=True)),
],
}
@register_schema(LocalMongoDBConnection)
def create_database(conn, dbname):
logger.info('Create database `%s`.', dbname)
# TODO: read and write concerns can be declared here
conn.conn.get_database(dbname)
@register_schema(LocalMongoDBConnection)
def create_tables(conn, dbname):
for table_name in backend.schema.TABLES:
# create the table
# TODO: read and write concerns can be declared here
try:
logger.info(f'Create `{table_name}` table.')
conn.conn[dbname].create_collection(table_name)
except CollectionInvalid:
logger.info(f'Collection {table_name} already exists.')
create_indexes(conn, dbname, table_name, INDEXES[table_name])
def create_indexes(conn, dbname, collection, indexes):
logger.info(f'Ensure secondary indexes for `{collection}`.')
for fields, kwargs in indexes:
conn.conn[dbname][collection].create_index(fields, **kwargs)
@register_schema(LocalMongoDBConnection)
def drop_database(conn, dbname):
conn.conn.drop_database(dbname)

430
bigchaindb/backend/query.py Normal file
View File

@ -0,0 +1,430 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
"""Query interfaces for backends."""
from functools import singledispatch
from bigchaindb.backend.exceptions import OperationError
@singledispatch
def store_asset(connection, asset):
"""Write an asset to the asset table.
Args:
asset (dict): the asset.
Returns:
The result of the operation.
"""
raise NotImplementedError
@singledispatch
def store_assets(connection, assets):
"""Write a list of assets to the assets table.
Args:
assets (list): a list of assets to write.
Returns:
The database response.
"""
raise NotImplementedError
@singledispatch
def store_metadatas(connection, metadata):
"""Write a list of metadata to metadata table.
Args:
metadata (list): list of metadata.
Returns:
The result of the operation.
"""
raise NotImplementedError
@singledispatch
def store_transactions(connection, signed_transactions):
"""Store the list of transactions."""
raise NotImplementedError
@singledispatch
def get_transaction(connection, transaction_id):
"""Get a transaction from the transactions table.
Args:
transaction_id (str): the id of the transaction.
Returns:
The result of the operation.
"""
raise NotImplementedError
@singledispatch
def get_transactions(connection, transaction_ids):
"""Get transactions from the transactions table.
Args:
transaction_ids (list): list of transaction ids to fetch
Returns:
The result of the operation.
"""
raise NotImplementedError
@singledispatch
def get_asset(connection, asset_id):
"""Get a transaction from the transactions table.
Args:
asset_id (str): the id of the asset
Returns:
The result of the operation.
"""
raise NotImplementedError
@singledispatch
def get_spent(connection, transaction_id, condition_id):
"""Check if a `txid` was already used as an input.
A transaction can be used as an input for another transaction. Bigchain
needs to make sure that a given `txid` is only used once.
Args:
transaction_id (str): The id of the transaction.
condition_id (int): The index of the condition in the respective
transaction.
Returns:
The transaction that used the `txid` as an input else `None`
"""
raise NotImplementedError
@singledispatch
def get_spending_transactions(connection, inputs):
"""Return transactions which spend given inputs
Args:
inputs (list): list of {txid, output}
Returns:
Iterator of (block_ids, transaction) for transactions that
spend given inputs.
"""
raise NotImplementedError
@singledispatch
def get_owned_ids(connection, owner):
"""Retrieve a list of `txids` that can we used has inputs.
Args:
owner (str): base58 encoded public key.
Returns:
Iterator of (block_id, transaction) for transactions
that list given owner in conditions.
"""
raise NotImplementedError
@singledispatch
def get_block(connection, block_id):
"""Get a block from the bigchain table.
Args:
block_id (str): block id of the block to get
Returns:
block (dict): the block or `None`
"""
raise NotImplementedError
@singledispatch
def get_block_with_transaction(connection, txid):
"""Get a block containing transaction id `txid`
Args:
txid (str): id of transaction to be searched.
Returns:
block_id (int): the block id or `None`
"""
raise NotImplementedError
@singledispatch
def get_metadata(connection, transaction_ids):
"""Get a list of metadata from the metadata table.
Args:
transaction_ids (list): a list of ids for the metadata to be retrieved from
the database.
Returns:
metadata (list): the list of returned metadata.
"""
raise NotImplementedError
@singledispatch
def get_assets(connection, asset_ids):
"""Get a list of assets from the assets table.
Args:
asset_ids (list): a list of ids for the assets to be retrieved from
the database.
Returns:
assets (list): the list of returned assets.
"""
raise NotImplementedError
@singledispatch
def get_txids_filtered(connection, asset_id, operation=None):
"""Return all transactions for a particular asset id and optional operation.
Args:
asset_id (str): ID of transaction that defined the asset
operation (str) (optional): Operation to filter on
"""
raise NotImplementedError
@singledispatch
def text_search(conn, search, *, language='english', case_sensitive=False,
diacritic_sensitive=False, text_score=False, limit=0, table=None):
"""Return all the assets that match the text search.
The results are sorted by text score.
For more information about the behavior of text search on MongoDB see
https://docs.mongodb.com/manual/reference/operator/query/text/#behavior
Args:
search (str): Text search string to query the text index
language (str, optional): The language for the search and the rules for
stemmer and tokenizer. If the language is ``None`` text search uses
simple tokenization and no stemming.
case_sensitive (bool, optional): Enable or disable case sensitive
search.
diacritic_sensitive (bool, optional): Enable or disable case sensitive
diacritic search.
text_score (bool, optional): If ``True`` returns the text score with
each document.
limit (int, optional): Limit the number of returned documents.
Returns:
:obj:`list` of :obj:`dict`: a list of assets
Raises:
OperationError: If the backend does not support text search
"""
raise OperationError('This query is only supported when running '
'Planetmint with MongoDB as the backend.')
@singledispatch
def get_latest_block(conn):
"""Get the latest commited block i.e. block with largest height"""
raise NotImplementedError
@singledispatch
def store_block(conn, block):
"""Write a new block to the `blocks` table
Args:
block (dict): block with current height and block hash.
Returns:
The result of the operation.
"""
raise NotImplementedError
@singledispatch
def store_unspent_outputs(connection, unspent_outputs):
"""Store unspent outputs in ``utxo_set`` table."""
raise NotImplementedError
@singledispatch
def delete_unspent_outputs(connection, unspent_outputs):
"""Delete unspent outputs in ``utxo_set`` table."""
raise NotImplementedError
@singledispatch
def delete_transactions(conn, txn_ids):
"""Delete transactions from database
Args:
txn_ids (list): list of transaction ids
Returns:
The result of the operation.
"""
raise NotImplementedError
@singledispatch
def get_unspent_outputs(connection, *, query=None):
"""Retrieves unspent outputs.
Args:
query (dict): An optional parameter to filter the result set.
Defaults to ``None``, which means that all UTXO records
will be returned.
Returns:
Generator yielding unspent outputs (UTXO set) according to the
given query.
"""
raise NotImplementedError
@singledispatch
def store_pre_commit_state(connection, state):
"""Store pre-commit state.
Args:
state (dict): pre-commit state.
Returns:
The result of the operation.
"""
raise NotImplementedError
@singledispatch
def get_pre_commit_state(connection):
"""Get pre-commit state.
Returns:
Document representing the pre-commit state.
"""
raise NotImplementedError
@singledispatch
def store_validator_set(conn, validator_update):
"""Store updated validator set"""
raise NotImplementedError
@singledispatch
def delete_validator_set(conn, height):
"""Delete the validator set at the given height."""
raise NotImplementedError
@singledispatch
def store_election(conn, election_id, height, is_concluded):
"""Store election record"""
raise NotImplementedError
@singledispatch
def store_elections(conn, elections):
"""Store election records in bulk"""
raise NotImplementedError
@singledispatch
def delete_elections(conn, height):
"""Delete all election records at the given height"""
raise NotImplementedError
@singledispatch
def get_validator_set(conn, height):
"""Get validator set for a given `height`, if `height` is not specified
then return the latest validator set
"""
raise NotImplementedError
@singledispatch
def get_election(conn, election_id):
"""Return the election record
"""
raise NotImplementedError
@singledispatch
def get_asset_tokens_for_public_key(connection, asset_id, public_key):
"""Retrieve a list of tokens of type `asset_id` that are owned by the `public_key`.
Args:
asset_id (str): Id of the token.
public_key (str): base58 encoded public key
Returns:
Iterator of transaction that list given owner in conditions.
"""
raise NotImplementedError
@singledispatch
def store_abci_chain(conn, height, chain_id, is_synced=True):
"""Create or update an ABCI chain at the given height.
Usually invoked in the beginning of the ABCI communications (height=0)
or when ABCI client (like Tendermint) is migrated (any height).
Args:
is_synced: True if the chain is known by both ABCI client and server
"""
raise NotImplementedError
@singledispatch
def delete_abci_chain(conn, height):
"""Delete the ABCI chain at the given height."""
raise NotImplementedError
@singledispatch
def get_latest_abci_chain(conn):
"""Returns the ABCI chain stored at the biggest height, if any,
None otherwise.
"""
raise NotImplementedError

128
bigchaindb/backend/schema.py Normal file
View File

@ -0,0 +1,128 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
"""Database creation and schema-providing interfaces for backends."""
from functools import singledispatch
import logging
import bigchaindb
from bigchaindb.backend.connection import connect
from bigchaindb.common.exceptions import ValidationError
from bigchaindb.common.utils import validate_all_values_for_key_in_obj, validate_all_values_for_key_in_list
logger = logging.getLogger(__name__)
# Tables/collections that every backend database must create
TABLES = ('transactions', 'blocks', 'assets', 'metadata',
'validators', 'elections', 'pre_commit', 'utxos', 'abci_chains')
VALID_LANGUAGES = ('danish', 'dutch', 'english', 'finnish', 'french', 'german',
'hungarian', 'italian', 'norwegian', 'portuguese', 'romanian',
'russian', 'spanish', 'swedish', 'turkish', 'none',
'da', 'nl', 'en', 'fi', 'fr', 'de', 'hu', 'it', 'nb', 'pt',
'ro', 'ru', 'es', 'sv', 'tr')
@singledispatch
def create_database(connection, dbname):
"""Create database to be used by Planetmint.
Args:
dbname (str): the name of the database to create.
"""
raise NotImplementedError
@singledispatch
def create_tables(connection, dbname):
"""Create the tables to be used by Planetmint.
Args:
dbname (str): the name of the database to create tables for.
"""
raise NotImplementedError
@singledispatch
def drop_database(connection, dbname):
"""Drop the database used by Planetmint.
Args:
dbname (str): the name of the database to drop.
Raises:
:exc:`~DatabaseDoesNotExist`: If the given :attr:`dbname` does not
exist as a database.
"""
raise NotImplementedError
def init_database(connection=None, dbname=None):
"""Initialize the configured backend for use with Planetmint.
Creates a database with :attr:`dbname` with any required tables
and supporting indexes.
Args:
connection (:class:`~bigchaindb.backend.connection.Connection`): an
existing connection to use to initialize the database.
Creates one if not given.
dbname (str): the name of the database to create.
Defaults to the database name given in the Planetmint
configuration.
"""
connection = connection or connect()
dbname = dbname or bigchaindb.config['database']['name']
create_database(connection, dbname)
create_tables(connection, dbname)
def validate_language_key(obj, key):
"""Validate all nested "language" key in `obj`.
Args:
obj (dict): dictionary whose "language" key is to be validated.
Returns:
None: validation successful
Raises:
ValidationError: will raise exception in case language is not valid.
"""
backend = bigchaindb.config['database']['backend']
if backend == 'localmongodb':
data = obj.get(key, {})
if isinstance(data, dict):
validate_all_values_for_key_in_obj(data, 'language', validate_language)
elif isinstance(data, list):
validate_all_values_for_key_in_list(data, 'language', validate_language)
def validate_language(value):
"""Check if `value` is a valid language.
https://docs.mongodb.com/manual/reference/text-search-languages/
Args:
value (str): language to validated
Returns:
None: validation successful
Raises:
ValidationError: will raise exception in case language is not valid.
"""
if value not in VALID_LANGUAGES:
error_str = ('MongoDB does not support text search for the '
'language "{}". If you do not understand this error '
'message then please rename key/field "language" to '
'something else like "lang".').format(value)
raise ValidationError(error_str)

39
bigchaindb/backend/utils.py Normal file
View File

@ -0,0 +1,39 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
import bigchaindb
class ModuleDispatchRegistrationError(Exception):
"""Raised when there is a problem registering dispatched functions for a
module
"""
def module_dispatch_registrar(module):
def dispatch_wrapper(obj_type):
def wrapper(func):
func_name = func.__name__
try:
dispatch_registrar = getattr(module, func_name)
return dispatch_registrar.register(obj_type)(func)
except AttributeError as ex:
raise ModuleDispatchRegistrationError(
('`{module}` does not contain a single-dispatchable '
'function named `{func}`. The module being registered '
'was not implemented correctly!').format(
func=func_name, module=module.__name__)) from ex
return wrapper
return dispatch_wrapper
def get_bigchaindb_config_value(key, default_value=None):
return bigchaindb.config['database'].get(key, default_value)
def get_bigchaindb_config_value_or_key_error(key):
return bigchaindb.config['database'][key]

0
bigchaindb/commands/__init__.py Normal file
View File

398
bigchaindb/commands/bigchaindb.py Normal file
View File

@ -0,0 +1,398 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
"""Implementation of the `bigchaindb` command,
the command-line interface (CLI) for Planetmint Server.
"""
import os
import logging
import argparse
import copy
import json
import sys
from bigchaindb.core import rollback
from bigchaindb.migrations.chain_migration_election import ChainMigrationElection
from bigchaindb.utils import load_node_key
from bigchaindb.common.transaction_mode_types import BROADCAST_TX_COMMIT
from bigchaindb.common.exceptions import (DatabaseDoesNotExist,
ValidationError)
from bigchaindb.elections.vote import Vote
import bigchaindb
from bigchaindb import (backend, ValidatorElection,
BigchainDB)
from bigchaindb.backend import schema
from bigchaindb.commands import utils
from bigchaindb.commands.utils import (configure_bigchaindb,
input_on_stderr)
from bigchaindb.log import setup_logging
from bigchaindb.tendermint_utils import public_key_from_base64
from bigchaindb.commands.election_types import elections
from bigchaindb.version import __tm_supported_versions__
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
# Note about printing:
# We try to print to stdout for results of a command that may be useful to
# someone (or another program). Strictly informational text, or errors,
# should be printed to stderr.
@configure_bigchaindb
def run_show_config(args):
"""Show the current configuration"""
# TODO Proposal: remove the "hidden" configuration. Only show config. If
# the system needs to be configured, then display information on how to
# configure the system.
config = copy.deepcopy(bigchaindb.config)
del config['CONFIGURED']
print(json.dumps(config, indent=4, sort_keys=True))
@configure_bigchaindb
def run_configure(args):
"""Run a script to configure the current node."""
config_path = args.config or bigchaindb.config_utils.CONFIG_DEFAULT_PATH
config_file_exists = False
# if the config path is `-` then it's stdout
if config_path != '-':
config_file_exists = os.path.exists(config_path)
if config_file_exists and not args.yes:
want = input_on_stderr('Config file `{}` exists, do you want to '
'override it? (cannot be undone) [y/N]: '.format(config_path))
if want != 'y':
return
conf = copy.deepcopy(bigchaindb.config)
# select the correct config defaults based on the backend
print('Generating default configuration for backend {}'
.format(args.backend), file=sys.stderr)
database_keys = bigchaindb._database_keys_map[args.backend]
conf['database'] = bigchaindb._database_map[args.backend]
if not args.yes:
for key in ('bind', ):
val = conf['server'][key]
conf['server'][key] = input_on_stderr('API Server {}? (default `{}`): '.format(key, val), val)
for key in ('scheme', 'host', 'port'):
val = conf['wsserver'][key]
conf['wsserver'][key] = input_on_stderr('WebSocket Server {}? (default `{}`): '.format(key, val), val)
for key in database_keys:
val = conf['database'][key]
conf['database'][key] = input_on_stderr('Database {}? (default `{}`): '.format(key, val), val)
for key in ('host', 'port'):
val = conf['tendermint'][key]
conf['tendermint'][key] = input_on_stderr('Tendermint {}? (default `{}`)'.format(key, val), val)
if config_path != '-':
bigchaindb.config_utils.write_config(conf, config_path)
else:
print(json.dumps(conf, indent=4, sort_keys=True))
print('Configuration written to {}'.format(config_path), file=sys.stderr)
print('Ready to go!', file=sys.stderr)
@configure_bigchaindb
def run_election(args):
"""Initiate and manage elections"""
b = BigchainDB()
# Call the function specified by args.action, as defined above
globals()[f'run_election_{args.action}'](args, b)
def run_election_new(args, bigchain):
election_type = args.election_type.replace('-', '_')
globals()[f'run_election_new_{election_type}'](args, bigchain)
def create_new_election(sk, bigchain, election_class, data):
try:
key = load_node_key(sk)
voters = election_class.recipients(bigchain)
election = election_class.generate([key.public_key],
voters,
data, None).sign([key.private_key])
election.validate(bigchain)
except ValidationError as e:
logger.error(e)
return False
except FileNotFoundError as fd_404:
logger.error(fd_404)
return False
resp = bigchain.write_transaction(election, BROADCAST_TX_COMMIT)
if resp == (202, ''):
logger.info('[SUCCESS] Submitted proposal with id: {}'.format(election.id))
return election.id
else:
logger.error('Failed to commit election proposal')
return False
def run_election_new_upsert_validator(args, bigchain):
"""Initiates an election to add/update/remove a validator to an existing Planetmint network
:param args: dict
args = {
'public_key': the public key of the proposed peer, (str)
'power': the proposed validator power for the new peer, (str)
'node_id': the node_id of the new peer (str)
'sk': the path to the private key of the node calling the election (str)
}
:param bigchain: an instance of Planetmint
:return: election_id or `False` in case of failure
"""
new_validator = {
'public_key': {'value': public_key_from_base64(args.public_key),
'type': 'ed25519-base16'},
'power': args.power,
'node_id': args.node_id
}
return create_new_election(args.sk, bigchain, ValidatorElection, new_validator)
def run_election_new_chain_migration(args, bigchain):
"""Initiates an election to halt block production
:param args: dict
args = {
'sk': the path to the private key of the node calling the election (str)
}
:param bigchain: an instance of Planetmint
:return: election_id or `False` in case of failure
"""
return create_new_election(args.sk, bigchain, ChainMigrationElection, {})
def run_election_approve(args, bigchain):
"""Approve an election
:param args: dict
args = {
'election_id': the election_id of the election (str)
'sk': the path to the private key of the signer (str)
}
:param bigchain: an instance of Planetmint
:return: success log message or `False` in case of error
"""
key = load_node_key(args.sk)
tx = bigchain.get_transaction(args.election_id)
voting_powers = [v.amount for v in tx.outputs if key.public_key in v.public_keys]
if len(voting_powers) > 0:
voting_power = voting_powers[0]
else:
logger.error('The key you provided does not match any of the eligible voters in this election.')
return False
inputs = [i for i in tx.to_inputs() if key.public_key in i.owners_before]
election_pub_key = ValidatorElection.to_public_key(tx.id)
approval = Vote.generate(inputs,
[([election_pub_key], voting_power)],
tx.id).sign([key.private_key])
approval.validate(bigchain)
resp = bigchain.write_transaction(approval, BROADCAST_TX_COMMIT)
if resp == (202, ''):
logger.info('[SUCCESS] Your vote has been submitted')
return approval.id
else:
logger.error('Failed to commit vote')
return False
def run_election_show(args, bigchain):
"""Retrieves information about an election
:param args: dict
args = {
'election_id': the transaction_id for an election (str)
}
:param bigchain: an instance of Planetmint
"""
election = bigchain.get_transaction(args.election_id)
if not election:
logger.error(f'No election found with election_id {args.election_id}')
return
response = election.show_election(bigchain)
logger.info(response)
return response
def _run_init():
bdb = bigchaindb.BigchainDB()
schema.init_database(connection=bdb.connection)
@configure_bigchaindb
def run_init(args):
"""Initialize the database"""
_run_init()
@configure_bigchaindb
def run_drop(args):
"""Drop the database"""
dbname = bigchaindb.config['database']['name']
if not args.yes:
response = input_on_stderr('Do you want to drop `{}` database? [y/n]: '.format(dbname))
if response != 'y':
return
conn = backend.connect()
try:
schema.drop_database(conn, dbname)
except DatabaseDoesNotExist:
print("Cannot drop '{name}'. The database does not exist.".format(name=dbname), file=sys.stderr)
def run_recover(b):
rollback(b)
@configure_bigchaindb
def run_start(args):
"""Start the processes to run the node"""
# Configure Logging
setup_logging()
logger.info('Planetmint Version %s', bigchaindb.__version__)
run_recover(bigchaindb.lib.BigchainDB())
if not args.skip_initialize_database:
logger.info('Initializing database')
_run_init()
logger.info('Starting Planetmint main process.')
from bigchaindb.start import start
start(args)
def run_tendermint_version(args):
"""Show the supported Tendermint version(s)"""
supported_tm_ver = {
'description': 'Planetmint supports the following Tendermint version(s)',
'tendermint': __tm_supported_versions__,
}
print(json.dumps(supported_tm_ver, indent=4, sort_keys=True))
def create_parser():
parser = argparse.ArgumentParser(
description='Control your Planetmint node.',
parents=[utils.base_parser])
# all the commands are contained in the subparsers object,
# the command selected by the user will be stored in `args.command`
# that is used by the `main` function to select which other
# function to call.
subparsers = parser.add_subparsers(title='Commands',
dest='command')
# parser for writing a config file
config_parser = subparsers.add_parser('configure',
help='Prepare the config file.')
config_parser.add_argument('backend',
choices=['localmongodb'],
default='localmongodb',
const='localmongodb',
nargs='?',
help='The backend to use. It can only be '
'"localmongodb", currently.')
# parser for managing elections
election_parser = subparsers.add_parser('election',
help='Manage elections.')
election_subparser = election_parser.add_subparsers(title='Action',
dest='action')
new_election_parser = election_subparser.add_parser('new',
help='Calls a new election.')
new_election_subparser = new_election_parser.add_subparsers(title='Election_Type',
dest='election_type')
# Parser factory for each type of new election, so we get a bunch of commands that look like this:
# election new <some_election_type> <args>...
for name, data in elections.items():
args = data['args']
generic_parser = new_election_subparser.add_parser(name, help=data['help'])
for arg, kwargs in args.items():
generic_parser.add_argument(arg, **kwargs)
approve_election_parser = election_subparser.add_parser('approve',
help='Approve the election.')
approve_election_parser.add_argument('election_id',
help='The election_id of the election.')
approve_election_parser.add_argument('--private-key',
dest='sk',
required=True,
help='Path to the private key of the election initiator.')
show_election_parser = election_subparser.add_parser('show',
help='Provides information about an election.')
show_election_parser.add_argument('election_id',
help='The transaction id of the election you wish to query.')
# parsers for showing/exporting config values
subparsers.add_parser('show-config',
help='Show the current configuration')
# parser for database-level commands
subparsers.add_parser('init',
help='Init the database')
subparsers.add_parser('drop',
help='Drop the database')
# parser for starting Planetmint
start_parser = subparsers.add_parser('start',
help='Start Planetmint')
start_parser.add_argument('--no-init',
dest='skip_initialize_database',
default=False,
action='store_true',
help='Skip database initialization')
subparsers.add_parser('tendermint-version',
help='Show the Tendermint supported versions')
start_parser.add_argument('--experimental-parallel-validation',
dest='experimental_parallel_validation',
default=False,
action='store_true',
help='💀 EXPERIMENTAL: parallelize validation for better throughput 💀')
return parser
def main():
utils.start(create_parser(), sys.argv[1:], globals())

31
bigchaindb/commands/election_types.py Normal file
View File

@ -0,0 +1,31 @@
elections = {
'upsert-validator': {
'help': 'Propose a change to the validator set',
'args': {
'public_key': {
'help': 'Public key of the validator to be added/updated/removed.'
},
'power': {
'type': int,
'help': 'The proposed power for the validator. Setting to 0 will remove the validator.'},
'node_id': {
'help': 'The node_id of the validator.'
},
'--private-key': {
'dest': 'sk',
'required': True,
'help': 'Path to the private key of the election initiator.'
}
}
},
'chain-migration': {
'help': 'Call for a halt to block production to allow for a version change across breaking changes.',
'args': {
'--private-key': {
'dest': 'sk',
'required': True,
'help': 'Path to the private key of the election initiator.'
}
}
}
}

161
bigchaindb/commands/utils.py Normal file
View File

@ -0,0 +1,161 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
"""Utility functions and basic common arguments
for ``argparse.ArgumentParser``.
"""
import argparse
import builtins
import functools
import multiprocessing as mp
import sys
import bigchaindb
import bigchaindb.config_utils
from bigchaindb.version import __version__
def configure_bigchaindb(command):
"""Decorator to be used by command line functions, such that the
configuration of bigchaindb is performed before the execution of
the command.
Args:
command: The command to decorate.
Returns:
The command wrapper function.
"""
@functools.wraps(command)
def configure(args):
config_from_cmdline = None
try:
if args.log_level is not None:
config_from_cmdline = {
'log': {
'level_console': args.log_level,
'level_logfile': args.log_level,
},
'server': {'loglevel': args.log_level},
}
except AttributeError:
pass
bigchaindb.config_utils.autoconfigure(
filename=args.config, config=config_from_cmdline, force=True)
command(args)
return configure
def _convert(value, default=None, convert=None):
def convert_bool(value):
if value.lower() in ('true', 't', 'yes', 'y'):
return True
if value.lower() in ('false', 'f', 'no', 'n'):
return False
raise ValueError('{} cannot be converted to bool'.format(value))
if value == '':
value = None
if convert is None:
if default is not None:
convert = type(default)
else:
convert = str
if convert == bool:
convert = convert_bool
if value is None:
return default
else:
return convert(value)
# We need this because `input` always prints on stdout, while it should print
# to stderr. It's a very old bug, check it out here:
# - https://bugs.python.org/issue1927
def input_on_stderr(prompt='', default=None, convert=None):
"""Output a string to stderr and wait for input.
Args:
prompt (str): the message to display.
default: the default value to return if the user
leaves the field empty
convert (callable): a callable to be used to convert
the value the user inserted. If None, the type of
``default`` will be used.
"""
print(prompt, end='', file=sys.stderr)
value = builtins.input()
return _convert(value, default, convert)
def start(parser, argv, scope):
"""Utility function to execute a subcommand.
The function will look up in the ``scope``
if there is a function called ``run_<parser.args.command>``
and will run it using ``parser.args`` as first positional argument.
Args:
parser: an ArgumentParser instance.
argv: the list of command line arguments without the script name.
scope (dict): map containing (eventually) the functions to be called.
Raises:
NotImplementedError: if ``scope`` doesn't contain a function called
``run_<parser.args.command>``.
"""
args = parser.parse_args(argv)
if not args.command:
parser.print_help()
raise SystemExit()
# look up in the current scope for a function called 'run_<command>'
# replacing all the dashes '-' with the lowercase character '_'
func = scope.get('run_' + args.command.replace('-', '_'))
# if no command has been found, raise a `NotImplementedError`
if not func:
raise NotImplementedError('Command `{}` not yet implemented'.
format(args.command))
args.multiprocess = getattr(args, 'multiprocess', False)
if args.multiprocess is False:
args.multiprocess = 1
elif args.multiprocess is None:
args.multiprocess = mp.cpu_count()
return func(args)
base_parser = argparse.ArgumentParser(add_help=False, prog='bigchaindb')
base_parser.add_argument('-c', '--config',
help='Specify the location of the configuration file '
'(use "-" for stdout)')
# NOTE: this flag should not have any default value because that will override
# the environment variables provided to configure the logger.
base_parser.add_argument('-l', '--log-level',
type=str.upper, # convert to uppercase for comparison to choices
choices=['DEBUG', 'BENCHMARK', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'],
help='Log level')
base_parser.add_argument('-y', '--yes', '--yes-please',
action='store_true',
help='Assume "yes" as answer to all prompts and run '
'non-interactively')
base_parser.add_argument('-v', '--version',
action='version',
version='%(prog)s {}'.format(__version__))

0
bigchaindb/common/__init__.py Normal file
View File

55
bigchaindb/common/crypto.py Normal file
View File

@ -0,0 +1,55 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
# Separate all crypto code so that we can easily test several implementations
from collections import namedtuple
try:
from hashlib import sha3_256
except ImportError:
from sha3 import sha3_256
from cryptoconditions import crypto
CryptoKeypair = namedtuple('CryptoKeypair', ('private_key', 'public_key'))
def hash_data(data):
"""Hash the provided data using SHA3-256"""
return sha3_256(data.encode()).hexdigest()
def generate_key_pair():
"""Generates a cryptographic key pair.
Returns:
:class:`~bigchaindb.common.crypto.CryptoKeypair`: A
:obj:`collections.namedtuple` with named fields
:attr:`~bigchaindb.common.crypto.CryptoKeypair.private_key` and
:attr:`~bigchaindb.common.crypto.CryptoKeypair.public_key`.
"""
# TODO FOR CC: Adjust interface so that this function becomes unnecessary
return CryptoKeypair(
*(k.decode() for k in crypto.ed25519_generate_key_pair()))
PrivateKey = crypto.Ed25519SigningKey
PublicKey = crypto.Ed25519VerifyingKey
def key_pair_from_ed25519_key(hex_private_key):
"""Generate base58 encode public-private key pair from a hex encoded private key"""
priv_key = crypto.Ed25519SigningKey(bytes.fromhex(hex_private_key)[:32], encoding='bytes')
public_key = priv_key.get_verifying_key()
return CryptoKeypair(private_key=priv_key.encode(encoding='base58').decode('utf-8'),
public_key=public_key.encode(encoding='base58').decode('utf-8'))
def public_key_from_ed25519_key(hex_public_key):
"""Generate base58 public key from hex encoded public key"""
public_key = crypto.Ed25519VerifyingKey(bytes.fromhex(hex_public_key), encoding='bytes')
return public_key.encode(encoding='base58').decode('utf-8')

115
bigchaindb/common/exceptions.py Normal file
View File

@ -0,0 +1,115 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
"""Custom exceptions used in the `bigchaindb` package.
"""
from bigchaindb.exceptions import BigchainDBError
class ConfigurationError(BigchainDBError):
"""Raised when there is a problem with server configuration"""
class DatabaseDoesNotExist(BigchainDBError):
"""Raised when trying to delete the database but the db is not there"""
class StartupError(BigchainDBError):
"""Raised when there is an error starting up the system"""
class CyclicBlockchainError(BigchainDBError):
"""Raised when there is a cycle in the blockchain"""
class KeypairMismatchException(BigchainDBError):
"""Raised if the private key(s) provided for signing don't match any of the
current owner(s)
"""
class OperationError(BigchainDBError):
"""Raised when an operation cannot go through"""
################################################################################
# Validation errors
#
# All validation errors (which are handleable errors, not faults) should
# subclass ValidationError. However, where possible they should also have their
# own distinct type to differentiate them from other validation errors,
# especially for the purposes of testing.
class ValidationError(BigchainDBError):
"""Raised if there was an error in validation"""
class DoubleSpend(ValidationError):
"""Raised if a double spend is found"""
class InvalidHash(ValidationError):
"""Raised if there was an error checking the hash for a particular
operation
"""
class SchemaValidationError(ValidationError):
"""Raised if there was any error validating an object's schema"""
class InvalidSignature(ValidationError):
"""Raised if there was an error checking the signature for a particular
operation
"""
class AssetIdMismatch(ValidationError):
"""Raised when multiple transaction inputs related to different assets"""
class AmountError(ValidationError):
"""Raised when there is a problem with a transaction's output amounts"""
class InputDoesNotExist(ValidationError):
"""Raised if a transaction input does not exist"""
class TransactionOwnerError(ValidationError):
"""Raised if a user tries to transfer a transaction they don't own"""
class DuplicateTransaction(ValidationError):
"""Raised if a duplicated transaction is found"""
class ThresholdTooDeep(ValidationError):
"""Raised if threshold condition is too deep"""
class MultipleValidatorOperationError(ValidationError):
"""Raised when a validator update pending but new request is submited"""
class MultipleInputsError(ValidationError):
"""Raised if there were multiple inputs when only one was expected"""
class InvalidProposer(ValidationError):
"""Raised if the public key is not a part of the validator set"""
class UnequalValidatorSet(ValidationError):
"""Raised if the validator sets differ"""
class InvalidPowerChange(ValidationError):
"""Raised if proposed power change in validator set is >=1/3 total power"""
class InvalidPublicKey(ValidationError):
"""Raised if public key doesn't match the encoding type"""

58
bigchaindb/common/memoize.py Normal file
View File

@ -0,0 +1,58 @@
import functools
import codecs
from functools import lru_cache
class HDict(dict):
def __hash__(self):
return hash(codecs.decode(self['id'], 'hex'))
@lru_cache(maxsize=16384)
def from_dict(func, *args, **kwargs):
return func(*args, **kwargs)
def memoize_from_dict(func):
@functools.wraps(func)
def memoized_func(*args, **kwargs):
if args[1].get('id', None):
args = list(args)
args[1] = HDict(args[1])
new_args = tuple(args)
return from_dict(func, *new_args, **kwargs)
else:
return func(*args, **kwargs)
return memoized_func
class ToDictWrapper():
def __init__(self, tx):
self.tx = tx
def __eq__(self, other):
return self.tx.id == other.tx.id
def __hash__(self):
return hash(self.tx.id)
@lru_cache(maxsize=16384)
def to_dict(func, tx_wrapped):
return func(tx_wrapped.tx)
def memoize_to_dict(func):
@functools.wraps(func)
def memoized_func(*args, **kwargs):
if args[0].id:
return to_dict(func, ToDictWrapper(args[0]))
else:
return func(*args, **kwargs)
return memoized_func

54
bigchaindb/common/schema/README.md Normal file
View File

@ -0,0 +1,54 @@
<!---
Copyright © 2020 Interplanetary Database Association e.V.,
Planetmint and IPDB software contributors.
SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
Code is Apache-2.0 and docs are CC-BY-4.0
--->
# Introduction
This directory contains the schemas for the different JSON documents Planetmint uses.
The aim is to provide:
- a strict definition of the data structures used in Planetmint,
- a language-independent tool to validate the structure of incoming/outcoming
data. (There are several ready to use
[implementations](http://json-schema.org/implementations.html) written in
different languages.)
## Sources
The files defining the JSON Schema for transactions (`transaction_*.yaml`)
are based on the [Planetmint Transactions Specs](https://github.com/bigchaindb/BEPs/tree/master/tx-specs).
If you want to add a new transaction version,
you must write a spec for it first.
(You can't change the JSON Schema files for old versions.
Those were used to validate old transactions
and are needed to re-check those transactions.)
There used to be a file defining the JSON Schema for votes, named `vote.yaml`.
It was used by Planetmint version 1.3.0 and earlier.
If you want a copy of the latest `vote.yaml` file,
then you can get it from the version 1.3.0 release on GitHub, at
[https://github.com/bigchaindb/bigchaindb/blob/v1.3.0/bigchaindb/common/schema/vote.yaml](https://github.com/bigchaindb/bigchaindb/blob/v1.3.0/bigchaindb/common/schema/vote.yaml).
## Learn about JSON Schema
A good resource is [Understanding JSON Schema](http://spacetelescope.github.io/understanding-json-schema/index.html).
It provides a *more accessible documentation for JSON schema* than the [specs](http://json-schema.org/documentation.html).
## If it's supposed to be JSON, why's everything in YAML D:?
YAML is great for its conciseness and friendliness towards human-editing in comparision to JSON.
Although YAML is a superset of JSON, at the end of the day, JSON Schema processors, like
[json-schema](http://python-jsonschema.readthedocs.io/en/latest/), take in a native object (e.g.
Python dicts or JavaScript objects) as the schema used for validation. As long as we can serialize
the YAML into what the JSON Schema processor expects (almost always as simple as loading the YAML
like you would with a JSON file), it's the same as using JSON.
Specific advantages of using YAML:
- Legibility, especially when nesting
- Multi-line string literals, that make it easy to include descriptions that can be [auto-generated
into Sphinx documentation](/docs/server/generate_schema_documentation.py)

82
bigchaindb/common/schema/__init__.py Normal file
View File

@ -0,0 +1,82 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
"""Schema validation related functions and data"""
import os.path
import logging
import jsonschema
import yaml
import rapidjson
from bigchaindb.common.exceptions import SchemaValidationError
logger = logging.getLogger(__name__)
def _load_schema(name, path=__file__):
"""Load a schema from disk"""
path = os.path.join(os.path.dirname(path), name + '.yaml')
with open(path) as handle:
schema = yaml.safe_load(handle)
fast_schema = rapidjson.Validator(rapidjson.dumps(schema))
return path, (schema, fast_schema)
TX_SCHEMA_VERSION = 'v2.0'
TX_SCHEMA_PATH, TX_SCHEMA_COMMON = _load_schema('transaction_' +
TX_SCHEMA_VERSION)
_, TX_SCHEMA_CREATE = _load_schema('transaction_create_' +
TX_SCHEMA_VERSION)
_, TX_SCHEMA_TRANSFER = _load_schema('transaction_transfer_' +
TX_SCHEMA_VERSION)
_, TX_SCHEMA_VALIDATOR_ELECTION = _load_schema('transaction_validator_election_' +
TX_SCHEMA_VERSION)
_, TX_SCHEMA_CHAIN_MIGRATION_ELECTION = _load_schema('transaction_chain_migration_election_' +
TX_SCHEMA_VERSION)
_, TX_SCHEMA_VOTE = _load_schema('transaction_vote_' + TX_SCHEMA_VERSION)
def _validate_schema(schema, body):
"""Validate data against a schema"""
# Note
#
# Schema validation is currently the major CPU bottleneck of
# Planetmint. the `jsonschema` library validates python data structures
# directly and produces nice error messages, but validation takes 4+ ms
# per transaction which is pretty slow. The rapidjson library validates
# much faster at 1.5ms, however it produces _very_ poor error messages.
# For this reason we use both, rapidjson as an optimistic pathway and
# jsonschema as a fallback in case there is a failure, so we can produce
# a helpful error message.
try:
schema[1](rapidjson.dumps(body))
except ValueError as exc:
try:
jsonschema.validate(body, schema[0])
except jsonschema.ValidationError as exc2:
raise SchemaValidationError(str(exc2)) from exc2
logger.warning('code problem: jsonschema did not raise an exception, wheras rapidjson raised %s', exc)
raise SchemaValidationError(str(exc)) from exc
def validate_transaction_schema(tx):
"""Validate a transaction dict.
TX_SCHEMA_COMMON contains properties that are common to all types of
transaction. TX_SCHEMA_[TRANSFER|CREATE] add additional constraints on top.
"""
_validate_schema(TX_SCHEMA_COMMON, tx)
if tx['operation'] == 'TRANSFER':
_validate_schema(TX_SCHEMA_TRANSFER, tx)
else:
_validate_schema(TX_SCHEMA_CREATE, tx)

45
bigchaindb/common/schema/transaction_chain_migration_election_v2.0.yaml Normal file
View File

@ -0,0 +1,45 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
---
"$schema": "http://json-schema.org/draft-04/schema#"
type: object
title: Chain Migration Election Schema - Propose a halt in block production to allow for a version change
required:
- operation
- asset
- outputs
properties:
operation:
type: string
value: "CHAIN_MIGRATION_ELECTION"
asset:
additionalProperties: false
properties:
data:
additionalProperties: false
properties:
seed:
type: string
required:
- data
outputs:
type: array
items:
"$ref": "#/definitions/output"
definitions:
output:
type: object
properties:
condition:
type: object
required:
- uri
properties:
uri:
type: string
pattern: "^ni:///sha-256;([a-zA-Z0-9_-]{0,86})[?]\
(fpt=ed25519-sha-256(&)?|cost=[0-9]+(&)?|\
subtypes=ed25519-sha-256(&)?){2,3}$"

35
bigchaindb/common/schema/transaction_create_v1.0.yaml Normal file
View File

@ -0,0 +1,35 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
---
"$schema": "http://json-schema.org/draft-04/schema#"
type: object
title: Transaction Schema - CREATE/GENESIS specific constraints
required:
- asset
- inputs
properties:
asset:
additionalProperties: false
properties:
data:
anyOf:
- type: object
additionalProperties: true
- type: 'null'
required:
- data
inputs:
type: array
title: "Transaction inputs"
maxItems: 1
minItems: 1
items:
type: "object"
required:
- fulfills
properties:
fulfills:
type: "null"

35
bigchaindb/common/schema/transaction_create_v2.0.yaml Normal file
View File

@ -0,0 +1,35 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
---
"$schema": "http://json-schema.org/draft-04/schema#"
type: object
title: Transaction Schema - CREATE specific constraints
required:
- asset
- inputs
properties:
asset:
additionalProperties: false
properties:
data:
anyOf:
- type: object
additionalProperties: true
- type: 'null'
required:
- data
inputs:
type: array
title: "Transaction inputs"
maxItems: 1
minItems: 1
items:
type: "object"
required:
- fulfills
properties:
fulfills:
type: "null"

34
bigchaindb/common/schema/transaction_transfer_v1.0.yaml Normal file
View File

@ -0,0 +1,34 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
---
"$schema": "http://json-schema.org/draft-04/schema#"
type: object
title: Transaction Schema - TRANSFER specific properties
required:
- asset
properties:
asset:
additionalProperties: false
properties:
id:
"$ref": "#/definitions/sha3_hexdigest"
required:
- id
inputs:
type: array
title: "Transaction inputs"
minItems: 1
items:
type: "object"
required:
- fulfills
properties:
fulfills:
type: "object"
definitions:
sha3_hexdigest:
pattern: "[0-9a-f]{64}"
type: string

34
bigchaindb/common/schema/transaction_transfer_v2.0.yaml Normal file
View File

@ -0,0 +1,34 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
---
"$schema": "http://json-schema.org/draft-04/schema#"
type: object
title: Transaction Schema - TRANSFER specific properties
required:
- asset
properties:
asset:
additionalProperties: false
properties:
id:
"$ref": "#/definitions/sha3_hexdigest"
required:
- id
inputs:
type: array
title: "Transaction inputs"
minItems: 1
items:
type: "object"
required:
- fulfills
properties:
fulfills:
type: "object"
definitions:
sha3_hexdigest:
pattern: "[0-9a-f]{64}"
type: string

168
bigchaindb/common/schema/transaction_v1.0.yaml Normal file
View File

@ -0,0 +1,168 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
---
"$schema": "http://json-schema.org/draft-04/schema#"
type: object
additionalProperties: false
title: Transaction Schema
required:
- id
- inputs
- outputs
- operation
- metadata
- asset
- version
properties:
id:
anyOf:
- "$ref": "#/definitions/sha3_hexdigest"
- type: 'null'
operation:
"$ref": "#/definitions/operation"
asset:
"$ref": "#/definitions/asset"
inputs:
type: array
title: "Transaction inputs"
items:
"$ref": "#/definitions/input"
outputs:
type: array
items:
"$ref": "#/definitions/output"
metadata:
"$ref": "#/definitions/metadata"
version:
type: string
pattern: "^1\\.0$"
definitions:
offset:
type: integer
minimum: 0
base58:
pattern: "[1-9a-zA-Z^OIl]{43,44}"
type: string
public_keys:
anyOf:
- type: array
items:
"$ref": "#/definitions/base58"
- type: 'null'
sha3_hexdigest:
pattern: "[0-9a-f]{64}"
type: string
uuid4:
pattern: "[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}"
type: string
operation:
type: string
enum:
- CREATE
- TRANSFER
- GENESIS
asset:
type: object
additionalProperties: false
properties:
id:
"$ref": "#/definitions/sha3_hexdigest"
data:
anyOf:
- type: object
additionalProperties: true
- type: 'null'
output:
type: object
additionalProperties: false
required:
- amount
- condition
- public_keys
properties:
amount:
type: string
pattern: "^[0-9]{1,20}$"
condition:
type: object
additionalProperties: false
required:
- details
- uri
properties:
details:
"$ref": "#/definitions/condition_details"
uri:
type: string
pattern: "^ni:///sha-256;([a-zA-Z0-9_-]{0,86})[?]\
(fpt=(ed25519|threshold)-sha-256(&)?|cost=[0-9]+(&)?|\
subtypes=ed25519-sha-256(&)?){2,3}$"
public_keys:
"$ref": "#/definitions/public_keys"
input:
type: "object"
additionalProperties: false
required:
- owners_before
- fulfillment
properties:
owners_before:
"$ref": "#/definitions/public_keys"
fulfillment:
anyOf:
- type: string
pattern: "^[a-zA-Z0-9_-]*$"
- "$ref": "#/definitions/condition_details"
fulfills:
anyOf:
- type: 'object'
additionalProperties: false
required:
- output_index
- transaction_id
properties:
output_index:
"$ref": "#/definitions/offset"
transaction_id:
"$ref": "#/definitions/sha3_hexdigest"
- type: 'null'
metadata:
anyOf:
- type: object
additionalProperties: true
minProperties: 1
- type: 'null'
condition_details:
anyOf:
- type: object
additionalProperties: false
required:
- type
- public_key
properties:
type:
type: string
pattern: "^ed25519-sha-256$"
public_key:
"$ref": "#/definitions/base58"
- type: object
additionalProperties: false
required:
- type
- threshold
- subconditions
properties:
type:
type: "string"
pattern: "^threshold-sha-256$"
threshold:
type: integer
minimum: 1
maximum: 100
subconditions:
type: array
items:
"$ref": "#/definitions/condition_details"

170
bigchaindb/common/schema/transaction_v2.0.yaml Normal file
View File

@ -0,0 +1,170 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
---
"$schema": "http://json-schema.org/draft-04/schema#"
type: object
additionalProperties: false
title: Transaction Schema
required:
- id
- inputs
- outputs
- operation
- metadata
- asset
- version
properties:
id:
anyOf:
- "$ref": "#/definitions/sha3_hexdigest"
- type: 'null'
operation:
"$ref": "#/definitions/operation"
asset:
"$ref": "#/definitions/asset"
inputs:
type: array
title: "Transaction inputs"
items:
"$ref": "#/definitions/input"
outputs:
type: array
items:
"$ref": "#/definitions/output"
metadata:
"$ref": "#/definitions/metadata"
version:
type: string
pattern: "^2\\.0$"
definitions:
offset:
type: integer
minimum: 0
base58:
pattern: "[1-9a-zA-Z^OIl]{43,44}"
type: string
public_keys:
anyOf:
- type: array
items:
"$ref": "#/definitions/base58"
- type: 'null'
sha3_hexdigest:
pattern: "[0-9a-f]{64}"
type: string
uuid4:
pattern: "[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}"
type: string
operation:
type: string
enum:
- CREATE
- TRANSFER
- VALIDATOR_ELECTION
- CHAIN_MIGRATION_ELECTION
- VOTE
asset:
type: object
additionalProperties: false
properties:
id:
"$ref": "#/definitions/sha3_hexdigest"
data:
anyOf:
- type: object
additionalProperties: true
- type: 'null'
output:
type: object
additionalProperties: false
required:
- amount
- condition
- public_keys
properties:
amount:
type: string
pattern: "^[0-9]{1,20}$"
condition:
type: object
additionalProperties: false
required:
- details
- uri
properties:
details:
"$ref": "#/definitions/condition_details"
uri:
type: string
pattern: "^ni:///sha-256;([a-zA-Z0-9_-]{0,86})[?]\
(fpt=(ed25519|threshold)-sha-256(&)?|cost=[0-9]+(&)?|\
subtypes=ed25519-sha-256(&)?){2,3}$"
public_keys:
"$ref": "#/definitions/public_keys"
input:
type: "object"
additionalProperties: false
required:
- owners_before
- fulfillment
properties:
owners_before:
"$ref": "#/definitions/public_keys"
fulfillment:
anyOf:
- type: string
pattern: "^[a-zA-Z0-9_-]*$"
- "$ref": "#/definitions/condition_details"
fulfills:
anyOf:
- type: 'object'
additionalProperties: false
required:
- output_index
- transaction_id
properties:
output_index:
"$ref": "#/definitions/offset"
transaction_id:
"$ref": "#/definitions/sha3_hexdigest"
- type: 'null'
metadata:
anyOf:
- type: object
additionalProperties: true
minProperties: 1
- type: 'null'
condition_details:
anyOf:
- type: object
additionalProperties: false
required:
- type
- public_key
properties:
type:
type: string
pattern: "^ed25519-sha-256$"
public_key:
"$ref": "#/definitions/base58"
- type: object
additionalProperties: false
required:
- type
- threshold
- subconditions
properties:
type:
type: "string"
pattern: "^threshold-sha-256$"
threshold:
type: integer
minimum: 1
maximum: 100
subconditions:
type: array
items:
"$ref": "#/definitions/condition_details"

68
bigchaindb/common/schema/transaction_validator_election_v2.0.yaml Normal file
View File

@ -0,0 +1,68 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
---
"$schema": "http://json-schema.org/draft-04/schema#"
type: object
title: Validator Election Schema - Propose a change to validator set
required:
- operation
- asset
- outputs
properties:
operation:
type: string
value: "VALIDATOR_ELECTION"
asset:
additionalProperties: false
properties:
data:
additionalProperties: false
properties:
node_id:
type: string
seed:
type: string
public_key:
type: object
additionalProperties: false
required:
- value
- type
properties:
value:
type: string
type:
type: string
enum:
- ed25519-base16
- ed25519-base32
- ed25519-base64
power:
"$ref": "#/definitions/positiveInteger"
required:
- node_id
- public_key
- power
required:
- data
outputs:
type: array
items:
"$ref": "#/definitions/output"
definitions:
output:
type: object
properties:
condition:
type: object
required:
- uri
properties:
uri:
type: string
pattern: "^ni:///sha-256;([a-zA-Z0-9_-]{0,86})[?]\
(fpt=ed25519-sha-256(&)?|cost=[0-9]+(&)?|\
subtypes=ed25519-sha-256(&)?){2,3}$"

34
bigchaindb/common/schema/transaction_vote_v2.0.yaml Normal file
View File

@ -0,0 +1,34 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
---
"$schema": "http://json-schema.org/draft-04/schema#"
type: object
title: Vote Schema - Vote on an election
required:
- operation
- outputs
properties:
operation:
type: string
value: "VOTE"
outputs:
type: array
items:
"$ref": "#/definitions/output"
definitions:
output:
type: object
properties:
condition:
type: object
required:
- uri
properties:
uri:
type: string
pattern: "^ni:///sha-256;([a-zA-Z0-9_-]{0,86})[?]\
(fpt=ed25519-sha-256(&)?|cost=[0-9]+(&)?|\
subtypes=ed25519-sha-256(&)?){2,3}$"

1330
bigchaindb/common/transaction.py Normal file
View File

File diff suppressed because it is too large Load Diff

8
bigchaindb/common/transaction_mode_types.py Normal file
View File

@ -0,0 +1,8 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
BROADCAST_TX_COMMIT = 'broadcast_tx_commit'
BROADCAST_TX_ASYNC = 'broadcast_tx_async'
BROADCAST_TX_SYNC = 'broadcast_tx_sync'

165
bigchaindb/common/utils.py Normal file
View File

@ -0,0 +1,165 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
import time
import re
import rapidjson
import bigchaindb
from bigchaindb.common.exceptions import ValidationError
def gen_timestamp():
"""The Unix time, rounded to the nearest second.
See https://en.wikipedia.org/wiki/Unix_time
Returns:
str: the Unix time
"""
return str(round(time.time()))
def serialize(data):
"""Serialize a dict into a JSON formatted string.
This function enforces rules like the separator and order of keys.
This ensures that all dicts are serialized in the same way.
This is specially important for hashing data. We need to make sure that
everyone serializes their data in the same way so that we do not have
hash mismatches for the same structure due to serialization
differences.
Args:
data (dict): dict to serialize
Returns:
str: JSON formatted string
"""
return rapidjson.dumps(data, skipkeys=False, ensure_ascii=False,
sort_keys=True)
def deserialize(data):
"""Deserialize a JSON formatted string into a dict.
Args:
data (str): JSON formatted string.
Returns:
dict: dict resulting from the serialization of a JSON formatted
string.
"""
return rapidjson.loads(data)
def validate_txn_obj(obj_name, obj, key, validation_fun):
"""Validate value of `key` in `obj` using `validation_fun`.
Args:
obj_name (str): name for `obj` being validated.
obj (dict): dictionary object.
key (str): key to be validated in `obj`.
validation_fun (function): function used to validate the value
of `key`.
Returns:
None: indicates validation successful
Raises:
ValidationError: `validation_fun` will raise exception on failure
"""
backend = bigchaindb.config['database']['backend']
if backend == 'localmongodb':
data = obj.get(key, {})
if isinstance(data, dict):
validate_all_keys_in_obj(obj_name, data, validation_fun)
elif isinstance(data, list):
validate_all_items_in_list(obj_name, data, validation_fun)
def validate_all_items_in_list(obj_name, data, validation_fun):
for item in data:
if isinstance(item, dict):
validate_all_keys_in_obj(obj_name, item, validation_fun)
elif isinstance(item, list):
validate_all_items_in_list(obj_name, item, validation_fun)
def validate_all_keys_in_obj(obj_name, obj, validation_fun):
"""Validate all (nested) keys in `obj` by using `validation_fun`.
Args:
obj_name (str): name for `obj` being validated.
obj (dict): dictionary object.
validation_fun (function): function used to validate the value
of `key`.
Returns:
None: indicates validation successful
Raises:
ValidationError: `validation_fun` will raise this error on failure
"""
for key, value in obj.items():
validation_fun(obj_name, key)
if isinstance(value, dict):
validate_all_keys_in_obj(obj_name, value, validation_fun)
elif isinstance(value, list):
validate_all_items_in_list(obj_name, value, validation_fun)
def validate_all_values_for_key_in_obj(obj, key, validation_fun):
"""Validate value for all (nested) occurrence of `key` in `obj`
using `validation_fun`.
Args:
obj (dict): dictionary object.
key (str): key whose value is to be validated.
validation_fun (function): function used to validate the value
of `key`.
Raises:
ValidationError: `validation_fun` will raise this error on failure
"""
for vkey, value in obj.items():
if vkey == key:
validation_fun(value)
elif isinstance(value, dict):
validate_all_values_for_key_in_obj(value, key, validation_fun)
elif isinstance(value, list):
validate_all_values_for_key_in_list(value, key, validation_fun)
def validate_all_values_for_key_in_list(input_list, key, validation_fun):
for item in input_list:
if isinstance(item, dict):
validate_all_values_for_key_in_obj(item, key, validation_fun)
elif isinstance(item, list):
validate_all_values_for_key_in_list(item, key, validation_fun)
def validate_key(obj_name, key):
"""Check if `key` contains ".", "$" or null characters.
https://docs.mongodb.com/manual/reference/limits/#Restrictions-on-Field-Names
Args:
obj_name (str): object name to use when raising exception
key (str): key to validated
Returns:
None: validation successful
Raises:
ValidationError: will raise exception in case of regex match.
"""
if re.search(r'^[$]|\.|\x00', key):
error_str = ('Invalid key name "{}" in {} object. The '
'key name cannot contain characters '
'".", "$" or null characters').format(key, obj_name)
raise ValidationError(error_str)

308
bigchaindb/config_utils.py Normal file
View File

@ -0,0 +1,308 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
"""Utils for reading and setting configuration settings.
The value of each Planetmint Server configuration setting is
determined according to the following rules:
* If it's set by an environment variable, then use that value
* Otherwise, if it's set in a local config file, then use that
value
* Otherwise, use the default value (contained in
``bigchaindb.__init__``)
"""
import os
import copy
import json
import logging
import collections.abc
from functools import lru_cache
from pkg_resources import iter_entry_points, ResolutionError
from bigchaindb.common import exceptions
import bigchaindb
from bigchaindb.validation import BaseValidationRules
# TODO: move this to a proper configuration file for logging
logging.getLogger('requests').setLevel(logging.WARNING)
logger = logging.getLogger(__name__)
CONFIG_DEFAULT_PATH = os.environ.setdefault(
'PLANETMINT_CONFIG_PATH',
os.path.join(os.path.expanduser('~'), '.bigchaindb'),
)
CONFIG_PREFIX = 'PLANETMINT'
CONFIG_SEP = '_'
def map_leafs(func, mapping):
"""Map a function to the leafs of a mapping."""
def _inner(mapping, path=None):
if path is None:
path = []
for key, val in mapping.items():
if isinstance(val, collections.abc.Mapping):
_inner(val, path + [key])
else:
mapping[key] = func(val, path=path+[key])
return mapping
return _inner(copy.deepcopy(mapping))
# Thanks Alex <3
# http://stackoverflow.com/a/3233356/597097
def update(d, u):
"""Recursively update a mapping (i.e. a dict, list, set, or tuple).
Conceptually, d and u are two sets trees (with nodes and edges).
This function goes through all the nodes of u. For each node in u,
if d doesn't have that node yet, then this function adds the node from u,
otherwise this function overwrites the node already in d with u's node.
Args:
d (mapping): The mapping to overwrite and add to.
u (mapping): The mapping to read for changes.
Returns:
mapping: An updated version of d (updated by u).
"""
for k, v in u.items():
if isinstance(v, collections.abc.Mapping):
r = update(d.get(k, {}), v)
d[k] = r
else:
d[k] = u[k]
return d
def file_config(filename=None):
"""Returns the config values found in a configuration file.
Args:
filename (str): the JSON file with the configuration values.
If ``None``, CONFIG_DEFAULT_PATH will be used.
Returns:
dict: The config values in the specified config file (or the
file at CONFIG_DEFAULT_PATH, if filename == None)
"""
logger.debug('On entry into file_config(), filename = {}'.format(filename))
if filename is None:
filename = CONFIG_DEFAULT_PATH
logger.debug('file_config() will try to open `{}`'.format(filename))
with open(filename) as f:
try:
config = json.load(f)
except ValueError as err:
raise exceptions.ConfigurationError(
'Failed to parse the JSON configuration from `{}`, {}'.format(filename, err)
)
logger.info('Configuration loaded from `{}`'.format(filename))
return config
def env_config(config):
"""Return a new configuration with the values found in the environment.
The function recursively iterates over the config, checking if there is
a matching env variable. If an env variable is found, the func updates
the configuration with that value.
The name of the env variable is built combining a prefix (``PLANETMINT``)
with the path to the value. If the ``config`` in input is:
``{'database': {'host': 'localhost'}}``
this function will try to read the env variable ``PLANETMINT_DATABASE_HOST``.
"""
def load_from_env(value, path):
var_name = CONFIG_SEP.join([CONFIG_PREFIX] + list(map(lambda s: s.upper(), path)))
return os.environ.get(var_name, value)
return map_leafs(load_from_env, config)
def update_types(config, reference, list_sep=':'):
"""Return a new configuration where all the values types
are aligned with the ones in the default configuration
"""
def _coerce(current, value):
# Coerce a value to the `current` type.
try:
# First we try to apply current to the value, since it
# might be a function
return current(value)
except TypeError:
# Then we check if current is a list AND if the value
# is a string.
if isinstance(current, list) and isinstance(value, str):
# If so, we use the colon as the separator
return value.split(list_sep)
try:
# If we are here, we should try to apply the type
# of `current` to the value
return type(current)(value)
except TypeError:
# Worst case scenario we return the value itself.
return value
def _update_type(value, path):
current = reference
for elem in path:
try:
current = current[elem]
except KeyError:
return value
return _coerce(current, value)
return map_leafs(_update_type, config)
def set_config(config):
"""Set bigchaindb.config equal to the default config dict,
then update that with whatever is in the provided config dict,
and then set bigchaindb.config['CONFIGURED'] = True
Args:
config (dict): the config dict to read for changes
to the default config
Note:
Any previous changes made to ``bigchaindb.config`` will be lost.
"""
# Deep copy the default config into bigchaindb.config
bigchaindb.config = copy.deepcopy(bigchaindb._config)
# Update the default config with whatever is in the passed config
update(bigchaindb.config, update_types(config, bigchaindb.config))
bigchaindb.config['CONFIGURED'] = True
def update_config(config):
"""Update bigchaindb.config with whatever is in the provided config dict,
and then set bigchaindb.config['CONFIGURED'] = True
Args:
config (dict): the config dict to read for changes
to the default config
"""
# Update the default config with whatever is in the passed config
update(bigchaindb.config, update_types(config, bigchaindb.config))
bigchaindb.config['CONFIGURED'] = True
def write_config(config, filename=None):
"""Write the provided configuration to a specific location.
Args:
config (dict): a dictionary with the configuration to load.
filename (str): the name of the file that will store the new configuration. Defaults to ``None``.
If ``None``, the HOME of the current user and the string ``.bigchaindb`` will be used.
"""
if not filename:
filename = CONFIG_DEFAULT_PATH
with open(filename, 'w') as f:
json.dump(config, f, indent=4)
def is_configured():
return bool(bigchaindb.config.get('CONFIGURED'))
def autoconfigure(filename=None, config=None, force=False):
"""Run ``file_config`` and ``env_config`` if the module has not
been initialized.
"""
if not force and is_configured():
logger.debug('System already configured, skipping autoconfiguration')
return
# start with the current configuration
newconfig = bigchaindb.config
# update configuration from file
try:
newconfig = update(newconfig, file_config(filename=filename))
except FileNotFoundError as e:
if filename:
raise
else:
logger.info('Cannot find config file `%s`.' % e.filename)
# override configuration with env variables
newconfig = env_config(newconfig)
if config:
newconfig = update(newconfig, config)
set_config(newconfig) # sets bigchaindb.config
@lru_cache()
def load_validation_plugin(name=None):
"""Find and load the chosen validation plugin.
Args:
name (string): the name of the entry_point, as advertised in the
setup.py of the providing package.
Returns:
an uninstantiated subclass of ``bigchaindb.validation.AbstractValidationRules``
"""
if not name:
return BaseValidationRules
# TODO: This will return the first plugin with group `bigchaindb.validation`
# and name `name` in the active WorkingSet.
# We should probably support Requirements specs in the config, e.g.
# validation_plugin: 'my-plugin-package==0.0.1;default'
plugin = None
for entry_point in iter_entry_points('bigchaindb.validation', name):
plugin = entry_point.load()
# No matching entry_point found
if not plugin:
raise ResolutionError(
'No plugin found in group `bigchaindb.validation` with name `{}`'.
format(name))
# Is this strictness desireable?
# It will probably reduce developer headaches in the wild.
if not issubclass(plugin, (BaseValidationRules,)):
raise TypeError('object of type "{}" does not implement `bigchaindb.'
'validation.BaseValidationRules`'.format(type(plugin)))
return plugin
def load_events_plugins(names=None):
plugins = []
if names is None:
return plugins
for name in names:
for entry_point in iter_entry_points('bigchaindb.events', name):
plugins.append((name, entry_point.load()))
return plugins

271
bigchaindb/core.py Normal file
View File

@ -0,0 +1,271 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
"""This module contains all the goodness to integrate Planetmint
with Tendermint.
"""
import logging
import sys
from abci.application import BaseApplication
from abci import CodeTypeOk
from bigchaindb import BigchainDB
from bigchaindb.elections.election import Election
from bigchaindb.version import __tm_supported_versions__
from bigchaindb.utils import tendermint_version_is_compatible
from bigchaindb.tendermint_utils import (decode_transaction,
calculate_hash)
from bigchaindb.lib import Block
import bigchaindb.upsert_validator.validator_utils as vutils
from bigchaindb.events import EventTypes, Event
CodeTypeError = 1
logger = logging.getLogger(__name__)
class App(BaseApplication):
"""Bridge between Planetmint and Tendermint.
The role of this class is to expose the Planetmint
transaction logic to Tendermint Core.
"""
def __init__(self, abci, bigchaindb=None, events_queue=None,):
super().__init__(abci)
self.events_queue = events_queue
self.bigchaindb = bigchaindb or BigchainDB()
self.block_txn_ids = []
self.block_txn_hash = ''
self.block_transactions = []
self.validators = None
self.new_height = None
self.chain = self.bigchaindb.get_latest_abci_chain()
def log_abci_migration_error(self, chain_id, validators):
logger.error('An ABCI chain migration is in process. '
'Download the new ABCI client and configure it with '
f'chain_id={chain_id} and validators={validators}.')
def abort_if_abci_chain_is_not_synced(self):
if self.chain is None or self.chain['is_synced']:
return
validators = self.bigchaindb.get_validators()
self.log_abci_migration_error(self.chain['chain_id'], validators)
sys.exit(1)
def init_chain(self, genesis):
"""Initialize chain upon genesis or a migration"""
app_hash = ''
height = 0
known_chain = self.bigchaindb.get_latest_abci_chain()
if known_chain is not None:
chain_id = known_chain['chain_id']
if known_chain['is_synced']:
msg = (f'Got invalid InitChain ABCI request ({genesis}) - '
f'the chain {chain_id} is already synced.')
logger.error(msg)
sys.exit(1)
if chain_id != genesis.chain_id:
validators = self.bigchaindb.get_validators()
self.log_abci_migration_error(chain_id, validators)
sys.exit(1)
# set migration values for app hash and height
block = self.bigchaindb.get_latest_block()
app_hash = '' if block is None else block['app_hash']
height = 0 if block is None else block['height'] + 1
known_validators = self.bigchaindb.get_validators()
validator_set = [vutils.decode_validator(v)
for v in genesis.validators]
if known_validators and known_validators != validator_set:
self.log_abci_migration_error(known_chain['chain_id'],
known_validators)
sys.exit(1)
block = Block(app_hash=app_hash, height=height, transactions=[])
self.bigchaindb.store_block(block._asdict())
self.bigchaindb.store_validator_set(height + 1, validator_set)
abci_chain_height = 0 if known_chain is None else known_chain['height']
self.bigchaindb.store_abci_chain(abci_chain_height,
genesis.chain_id, True)
self.chain = {'height': abci_chain_height, 'is_synced': True,
'chain_id': genesis.chain_id}
return self.abci.ResponseInitChain()
def info(self, request):
"""Return height of the latest committed block."""
self.abort_if_abci_chain_is_not_synced()
# Check if Planetmint supports the Tendermint version
if not (hasattr(request, 'version') and tendermint_version_is_compatible(request.version)):
logger.error(f'Unsupported Tendermint version: {getattr(request, "version", "no version")}.'
f' Currently, Planetmint only supports {__tm_supported_versions__}. Exiting!')
sys.exit(1)
logger.info(f"Tendermint version: {request.version}")
r = self.abci.ResponseInfo()
block = self.bigchaindb.get_latest_block()
if block:
chain_shift = 0 if self.chain is None else self.chain['height']
r.last_block_height = block['height'] - chain_shift
r.last_block_app_hash = block['app_hash'].encode('utf-8')
else:
r.last_block_height = 0
r.last_block_app_hash = b''
return r
def check_tx(self, raw_transaction):
"""Validate the transaction before entry into
the mempool.
Args:
raw_tx: a raw string (in bytes) transaction.
"""
self.abort_if_abci_chain_is_not_synced()
logger.debug('check_tx: %s', raw_transaction)
transaction = decode_transaction(raw_transaction)
if self.bigchaindb.is_valid_transaction(transaction):
logger.debug('check_tx: VALID')
return self.abci.ResponseCheckTx(code=CodeTypeOk)
else:
logger.debug('check_tx: INVALID')
return self.abci.ResponseCheckTx(code=CodeTypeError)
def begin_block(self, req_begin_block):
"""Initialize list of transaction.
Args:
req_begin_block: block object which contains block header
and block hash.
"""
self.abort_if_abci_chain_is_not_synced()
chain_shift = 0 if self.chain is None else self.chain['height']
logger.debug('BEGIN BLOCK, height:%s, num_txs:%s',
req_begin_block.header.height + chain_shift,
req_begin_block.header.num_txs)
self.block_txn_ids = []
self.block_transactions = []
return self.abci.ResponseBeginBlock()
def deliver_tx(self, raw_transaction):
"""Validate the transaction before mutating the state.
Args:
raw_tx: a raw string (in bytes) transaction.
"""
self.abort_if_abci_chain_is_not_synced()
logger.debug('deliver_tx: %s', raw_transaction)
transaction = self.bigchaindb.is_valid_transaction(
decode_transaction(raw_transaction), self.block_transactions)
if not transaction:
logger.debug('deliver_tx: INVALID')
return self.abci.ResponseDeliverTx(code=CodeTypeError)
else:
logger.debug('storing tx')
self.block_txn_ids.append(transaction.id)
self.block_transactions.append(transaction)
return self.abci.ResponseDeliverTx(code=CodeTypeOk)
def end_block(self, request_end_block):
"""Calculate block hash using transaction ids and previous block
hash to be stored in the next block.
Args:
height (int): new height of the chain.
"""
self.abort_if_abci_chain_is_not_synced()
chain_shift = 0 if self.chain is None else self.chain['height']
height = request_end_block.height + chain_shift
self.new_height = height
# store pre-commit state to recover in case there is a crash during
# `end_block` or `commit`
logger.debug(f'Updating pre-commit state: {self.new_height}')
pre_commit_state = dict(height=self.new_height,
transactions=self.block_txn_ids)
self.bigchaindb.store_pre_commit_state(pre_commit_state)
block_txn_hash = calculate_hash(self.block_txn_ids)
block = self.bigchaindb.get_latest_block()
if self.block_txn_ids:
self.block_txn_hash = calculate_hash([block['app_hash'], block_txn_hash])
else:
self.block_txn_hash = block['app_hash']
validator_update = Election.process_block(self.bigchaindb,
self.new_height,
self.block_transactions)
return self.abci.ResponseEndBlock(validator_updates=validator_update)
def commit(self):
"""Store the new height and along with block hash."""
self.abort_if_abci_chain_is_not_synced()
data = self.block_txn_hash.encode('utf-8')
# register a new block only when new transactions are received
if self.block_txn_ids:
self.bigchaindb.store_bulk_transactions(self.block_transactions)
block = Block(app_hash=self.block_txn_hash,
height=self.new_height,
transactions=self.block_txn_ids)
# NOTE: storing the block should be the last operation during commit
# this effects crash recovery. Refer BEP#8 for details
self.bigchaindb.store_block(block._asdict())
logger.debug('Commit-ing new block with hash: apphash=%s ,'
'height=%s, txn ids=%s', data, self.new_height,
self.block_txn_ids)
if self.events_queue:
event = Event(EventTypes.BLOCK_VALID, {
'height': self.new_height,
'transactions': self.block_transactions
})
self.events_queue.put(event)
return self.abci.ResponseCommit(data=data)
def rollback(b):
pre_commit = b.get_pre_commit_state()
if pre_commit is None:
# the pre_commit record is first stored in the first `end_block`
return
latest_block = b.get_latest_block()
if latest_block is None:
logger.error('Found precommit state but no blocks!')
sys.exit(1)
# NOTE: the pre-commit state is always at most 1 block ahead of the commited state
if latest_block['height'] < pre_commit['height']:
Election.rollback(b, pre_commit['height'], pre_commit['transactions'])
b.delete_transactions(pre_commit['transactions'])

0
bigchaindb/elections/__init__.py Normal file
View File

355
bigchaindb/elections/election.py Normal file
View File

@ -0,0 +1,355 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
from collections import OrderedDict
import base58
from uuid import uuid4
from bigchaindb import backend
from bigchaindb.elections.vote import Vote
from bigchaindb.common.exceptions import (InvalidSignature,
MultipleInputsError,
InvalidProposer,
UnequalValidatorSet,
DuplicateTransaction)
from bigchaindb.tendermint_utils import key_from_base64, public_key_to_base64
from bigchaindb.common.crypto import (public_key_from_ed25519_key)
from bigchaindb.common.transaction import Transaction
from bigchaindb.common.schema import (_validate_schema,
TX_SCHEMA_COMMON,
TX_SCHEMA_CREATE)
class Election(Transaction):
"""Represents election transactions.
To implement a custom election, create a class deriving from this one
with OPERATION set to the election operation, ALLOWED_OPERATIONS
set to (OPERATION,), CREATE set to OPERATION.
"""
OPERATION = None
# Custom validation schema
TX_SCHEMA_CUSTOM = None
# Election Statuses:
ONGOING = 'ongoing'
CONCLUDED = 'concluded'
INCONCLUSIVE = 'inconclusive'
# Vote ratio to approve an election
ELECTION_THRESHOLD = 2 / 3
@classmethod
def get_validator_change(cls, bigchain):
"""Return the validator set from the most recent approved block
:return: {
'height': <block_height>,
'validators': <validator_set>
}
"""
latest_block = bigchain.get_latest_block()
if latest_block is None:
return None
return bigchain.get_validator_change(latest_block['height'])
@classmethod
def get_validators(cls, bigchain, height=None):
"""Return a dictionary of validators with key as `public_key` and
value as the `voting_power`
"""
validators = {}
for validator in bigchain.get_validators(height):
# NOTE: we assume that Tendermint encodes public key in base64
public_key = public_key_from_ed25519_key(key_from_base64(validator['public_key']['value']))
validators[public_key] = validator['voting_power']
return validators
@classmethod
def recipients(cls, bigchain):
"""Convert validator dictionary to a recipient list for `Transaction`"""
recipients = []
for public_key, voting_power in cls.get_validators(bigchain).items():
recipients.append(([public_key], voting_power))
return recipients
@classmethod
def is_same_topology(cls, current_topology, election_topology):
voters = {}
for voter in election_topology:
if len(voter.public_keys) > 1:
return False
[public_key] = voter.public_keys
voting_power = voter.amount
voters[public_key] = voting_power
# Check whether the voters and their votes is same to that of the
# validators and their voting power in the network
return current_topology == voters
def validate(self, bigchain, current_transactions=[]):
"""Validate election transaction
NOTE:
* A valid election is initiated by an existing validator.
* A valid election is one where voters are validators and votes are
allocated according to the voting power of each validator node.
Args:
:param bigchain: (Planetmint) an instantiated bigchaindb.lib.BigchainDB object.
:param current_transactions: (list) A list of transactions to be validated along with the election
Returns:
Election: a Election object or an object of the derived Election subclass.
Raises:
ValidationError: If the election is invalid
"""
input_conditions = []
duplicates = any(txn for txn in current_transactions if txn.id == self.id)
if bigchain.is_committed(self.id) or duplicates:
raise DuplicateTransaction('transaction `{}` already exists'
.format(self.id))
if not self.inputs_valid(input_conditions):
raise InvalidSignature('Transaction signature is invalid.')
current_validators = self.get_validators(bigchain)
# NOTE: Proposer should be a single node
if len(self.inputs) != 1 or len(self.inputs[0].owners_before) != 1:
raise MultipleInputsError('`tx_signers` must be a list instance of length one')
# NOTE: Check if the proposer is a validator.
[election_initiator_node_pub_key] = self.inputs[0].owners_before
if election_initiator_node_pub_key not in current_validators.keys():
raise InvalidProposer('Public key is not a part of the validator set')
# NOTE: Check if all validators have been assigned votes equal to their voting power
if not self.is_same_topology(current_validators, self.outputs):
raise UnequalValidatorSet('Validator set much be exactly same to the outputs of election')
return self
@classmethod
def generate(cls, initiator, voters, election_data, metadata=None):
# Break symmetry in case we need to call an election with the same properties twice
uuid = uuid4()
election_data['seed'] = str(uuid)
(inputs, outputs) = cls.validate_create(initiator, voters, election_data, metadata)
election = cls(cls.OPERATION, {'data': election_data}, inputs, outputs, metadata)
cls.validate_schema(election.to_dict())
return election
@classmethod
def validate_schema(cls, tx):
"""Validate the election transaction. Since `ELECTION` extends `CREATE` transaction, all the validations for
`CREATE` transaction should be inherited
"""
_validate_schema(TX_SCHEMA_COMMON, tx)
_validate_schema(TX_SCHEMA_CREATE, tx)
if cls.TX_SCHEMA_CUSTOM:
_validate_schema(cls.TX_SCHEMA_CUSTOM, tx)
@classmethod
def create(cls, tx_signers, recipients, metadata=None, asset=None):
raise NotImplementedError
@classmethod
def transfer(cls, tx_signers, recipients, metadata=None, asset=None):
raise NotImplementedError
@classmethod
def to_public_key(cls, election_id):
return base58.b58encode(bytes.fromhex(election_id)).decode()
@classmethod
def count_votes(cls, election_pk, transactions, getter=getattr):
votes = 0
for txn in transactions:
if getter(txn, 'operation') == Vote.OPERATION:
for output in getter(txn, 'outputs'):
# NOTE: We enforce that a valid vote to election id will have only
# election_pk in the output public keys, including any other public key
# along with election_pk will lead to vote being not considered valid.
if len(getter(output, 'public_keys')) == 1 and [election_pk] == getter(output, 'public_keys'):
votes = votes + int(getter(output, 'amount'))
return votes
def get_commited_votes(self, bigchain, election_pk=None):
if election_pk is None:
election_pk = self.to_public_key(self.id)
txns = list(backend.query.get_asset_tokens_for_public_key(bigchain.connection,
self.id,
election_pk))
return self.count_votes(election_pk, txns, dict.get)
def has_concluded(self, bigchain, current_votes=[]):
"""Check if the election can be concluded or not.
* Elections can only be concluded if the validator set has not changed
since the election was initiated.
* Elections can be concluded only if the current votes form a supermajority.
Custom elections may override this function and introduce additional checks.
"""
if self.has_validator_set_changed(bigchain):
return False
election_pk = self.to_public_key(self.id)
votes_committed = self.get_commited_votes(bigchain, election_pk)
votes_current = self.count_votes(election_pk, current_votes)
total_votes = sum(output.amount for output in self.outputs)
if (votes_committed < (2/3) * total_votes) and \
(votes_committed + votes_current >= (2/3)*total_votes):
return True
return False
def get_status(self, bigchain):
election = self.get_election(self.id, bigchain)
if election and election['is_concluded']:
return self.CONCLUDED
return self.INCONCLUSIVE if self.has_validator_set_changed(bigchain) else self.ONGOING
def has_validator_set_changed(self, bigchain):
latest_change = self.get_validator_change(bigchain)
if latest_change is None:
return False
latest_change_height = latest_change['height']
election = self.get_election(self.id, bigchain)
return latest_change_height > election['height']
def get_election(self, election_id, bigchain):
return bigchain.get_election(election_id)
def store(self, bigchain, height, is_concluded):
bigchain.store_election(self.id, height, is_concluded)
def show_election(self, bigchain):
data = self.asset['data']
if 'public_key' in data.keys():
data['public_key'] = public_key_to_base64(data['public_key']['value'])
response = ''
for k, v in data.items():
if k != 'seed':
response += f'{k}={v}\n'
response += f'status={self.get_status(bigchain)}'
return response
@classmethod
def _get_initiated_elections(cls, height, txns):
elections = []
for tx in txns:
if not isinstance(tx, Election):
continue
elections.append({'election_id': tx.id, 'height': height,
'is_concluded': False})
return elections
@classmethod
def _get_votes(cls, txns):
elections = OrderedDict()
for tx in txns:
if not isinstance(tx, Vote):
continue
election_id = tx.asset['id']
if election_id not in elections:
elections[election_id] = []
elections[election_id].append(tx)
return elections
@classmethod
def process_block(cls, bigchain, new_height, txns):
"""Looks for election and vote transactions inside the block, records
and processes elections.
Every election is recorded in the database.
Every vote has a chance to conclude the corresponding election. When
an election is concluded, the corresponding database record is
marked as such.
Elections and votes are processed in the order in which they
appear in the block. Elections are concluded in the order of
appearance of their first votes in the block.
For every election concluded in the block, calls its `on_approval`
method. The returned value of the last `on_approval`, if any,
is a validator set update to be applied in one of the following blocks.
`on_approval` methods are implemented by elections of particular type.
The method may contain side effects but should be idempotent. To account
for other concluded elections, if it requires so, the method should
rely on the database state.
"""
# elections initiated in this block
initiated_elections = cls._get_initiated_elections(new_height, txns)
if initiated_elections:
bigchain.store_elections(initiated_elections)
# elections voted for in this block and their votes
elections = cls._get_votes(txns)
validator_update = None
for election_id, votes in elections.items():
election = bigchain.get_transaction(election_id)
if election is None:
continue
if not election.has_concluded(bigchain, votes):
continue
validator_update = election.on_approval(bigchain, new_height)
election.store(bigchain, new_height, is_concluded=True)
return [validator_update] if validator_update else []
@classmethod
def rollback(cls, bigchain, new_height, txn_ids):
"""Looks for election and vote transactions inside the block and
cleans up the database artifacts possibly created in `process_blocks`.
Part of the `end_block`/`commit` crash recovery.
"""
# delete election records for elections initiated at this height and
# elections concluded at this height
bigchain.delete_elections(new_height)
txns = [bigchain.get_transaction(tx_id) for tx_id in txn_ids]
elections = cls._get_votes(txns)
for election_id in elections:
election = bigchain.get_transaction(election_id)
election.on_rollback(bigchain, new_height)
def on_approval(self, bigchain, new_height):
"""Override to update the database state according to the
election rules. Consider the current database state to account for
other concluded elections, if required.
"""
raise NotImplementedError
def on_rollback(self, bigchain, new_height):
"""Override to clean up the database artifacts possibly created
in `on_approval`. Part of the `end_block`/`commit` crash recovery.
"""
raise NotImplementedError

64
bigchaindb/elections/vote.py Normal file
View File

@ -0,0 +1,64 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
from bigchaindb.common.transaction import Transaction
from bigchaindb.common.schema import (_validate_schema,
TX_SCHEMA_COMMON,
TX_SCHEMA_TRANSFER,
TX_SCHEMA_VOTE)
class Vote(Transaction):
OPERATION = 'VOTE'
# NOTE: This class inherits TRANSFER txn type. The `TRANSFER` property is
# overriden to re-use methods from parent class
TRANSFER = OPERATION
ALLOWED_OPERATIONS = (OPERATION,)
# Custom validation schema
TX_SCHEMA_CUSTOM = TX_SCHEMA_VOTE
def validate(self, bigchain, current_transactions=[]):
"""Validate election vote transaction
NOTE: There are no additional validity conditions on casting votes i.e.
a vote is just a valid TRANFER transaction
For more details refer BEP-21: https://github.com/bigchaindb/BEPs/tree/master/21
Args:
bigchain (Planetmint): an instantiated bigchaindb.lib.BigchainDB object.
Returns:
Vote: a Vote object
Raises:
ValidationError: If the election vote is invalid
"""
self.validate_transfer_inputs(bigchain, current_transactions)
return self
@classmethod
def generate(cls, inputs, recipients, election_id, metadata=None):
(inputs, outputs) = cls.validate_transfer(inputs, recipients, election_id, metadata)
election_vote = cls(cls.OPERATION, {'id': election_id}, inputs, outputs, metadata)
cls.validate_schema(election_vote.to_dict())
return election_vote
@classmethod
def validate_schema(cls, tx):
"""Validate the validator election vote transaction. Since `VOTE` extends `TRANSFER`
transaction, all the validations for `CREATE` transaction should be inherited
"""
_validate_schema(TX_SCHEMA_COMMON, tx)
_validate_schema(TX_SCHEMA_TRANSFER, tx)
_validate_schema(cls.TX_SCHEMA_CUSTOM, tx)
@classmethod
def create(cls, tx_signers, recipients, metadata=None, asset=None):
raise NotImplementedError
@classmethod
def transfer(cls, tx_signers, recipients, metadata=None, asset=None):
raise NotImplementedError

109
bigchaindb/events.py Normal file
View File

@ -0,0 +1,109 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
from queue import Empty
from collections import defaultdict
from multiprocessing import Queue
POISON_PILL = 'POISON_PILL'
class EventTypes:
"""Container class that holds all the possible
events Planetmint manages.
"""
# If you add a new Event Type, make sure to add it
# to the docs in docs/server/source/event-plugin-api.rst
ALL = ~0
BLOCK_VALID = 1
BLOCK_INVALID = 2
# NEW_EVENT = 4
# NEW_EVENT = 8
# NEW_EVENT = 16...
class Event:
"""An Event."""
def __init__(self, event_type, event_data):
"""Creates a new event.
Args:
event_type (int): the type of the event, see
:class:`~bigchaindb.events.EventTypes`
event_data (obj): the data of the event.
"""
self.type = event_type
self.data = event_data
class Exchange:
"""Dispatch events to subscribers."""
def __init__(self):
self.publisher_queue = Queue()
self.started_queue = Queue()
# Map <event_types -> queues>
self.queues = defaultdict(list)
def get_publisher_queue(self):
"""Get the queue used by the publisher.
Returns:
a :class:`multiprocessing.Queue`.
"""
return self.publisher_queue
def get_subscriber_queue(self, event_types=None):
"""Create a new queue for a specific combination of event types
and return it.
Returns:
a :class:`multiprocessing.Queue`.
Raises:
RuntimeError if called after `run`
"""
try:
self.started_queue.get(timeout=1)
raise RuntimeError('Cannot create a new subscriber queue while Exchange is running.')
except Empty:
pass
if event_types is None:
event_types = EventTypes.ALL
queue = Queue()
self.queues[event_types].append(queue)
return queue
def dispatch(self, event):
"""Given an event, send it to all the subscribers.
Args
event (:class:`~bigchaindb.events.EventTypes`): the event to
dispatch to all the subscribers.
"""
for event_types, queues in self.queues.items():
if event.type & event_types:
for queue in queues:
queue.put(event)
def run(self):
"""Start the exchange"""
self.started_queue.put('STARTED')
while True:
event = self.publisher_queue.get()
if event == POISON_PILL:
return
else:
self.dispatch(event)

12
bigchaindb/exceptions.py Normal file
View File

@ -0,0 +1,12 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
class BigchainDBError(Exception):
"""Base class for Planetmint exceptions."""
class CriticalDoubleSpend(BigchainDBError):
"""Data integrity error that requires attention"""

50
bigchaindb/fastquery.py Normal file
View File

@ -0,0 +1,50 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
from bigchaindb.utils import condition_details_has_owner
from bigchaindb.backend import query
from bigchaindb.common.transaction import TransactionLink
class FastQuery():
"""Database queries that join on block results from a single node."""
def __init__(self, connection):
self.connection = connection
def get_outputs_by_public_key(self, public_key):
"""Get outputs for a public key"""
txs = list(query.get_owned_ids(self.connection, public_key))
return [TransactionLink(tx['id'], index)
for tx in txs
for index, output in enumerate(tx['outputs'])
if condition_details_has_owner(output['condition']['details'],
public_key)]
def filter_spent_outputs(self, outputs):
"""Remove outputs that have been spent
Args:
outputs: list of TransactionLink
"""
links = [o.to_dict() for o in outputs]
txs = list(query.get_spending_transactions(self.connection, links))
spends = {TransactionLink.from_dict(input_['fulfills'])
for tx in txs
for input_ in tx['inputs']}
return [ff for ff in outputs if ff not in spends]
def filter_unspent_outputs(self, outputs):
"""Remove outputs that have not been spent
Args:
outputs: list of TransactionLink
"""
links = [o.to_dict() for o in outputs]
txs = list(query.get_spending_transactions(self.connection, links))
spends = {TransactionLink.from_dict(input_['fulfills'])
for tx in txs
for input_ in tx['inputs']}
return [ff for ff in outputs if ff in spends]

514
bigchaindb/lib.py Normal file
View File

@ -0,0 +1,514 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
"""Module containing main contact points with Tendermint and
MongoDB.
"""
import logging
from collections import namedtuple
from uuid import uuid4
import rapidjson
try:
from hashlib import sha3_256
except ImportError:
# NOTE: needed for Python < 3.6
from sha3 import sha3_256
import requests
import bigchaindb
from bigchaindb import backend, config_utils, fastquery
from bigchaindb.models import Transaction
from bigchaindb.common.exceptions import (SchemaValidationError,
ValidationError,
DoubleSpend)
from bigchaindb.common.transaction_mode_types import (BROADCAST_TX_COMMIT,
BROADCAST_TX_ASYNC,
BROADCAST_TX_SYNC)
from bigchaindb.tendermint_utils import encode_transaction, merkleroot
from bigchaindb import exceptions as core_exceptions
from bigchaindb.validation import BaseValidationRules
logger = logging.getLogger(__name__)
class BigchainDB(object):
"""Bigchain API
Create, read, sign, write transactions to the database
"""
def __init__(self, connection=None):
"""Initialize the Bigchain instance
A Bigchain instance has several configuration parameters (e.g. host).
If a parameter value is passed as an argument to the Bigchain
__init__ method, then that is the value it will have.
Otherwise, the parameter value will come from an environment variable.
If that environment variable isn't set, then the value
will come from the local configuration file. And if that variable
isn't in the local configuration file, then the parameter will have
its default value (defined in bigchaindb.__init__).
Args:
connection (:class:`~bigchaindb.backend.connection.Connection`):
A connection to the database.
"""
config_utils.autoconfigure()
self.mode_commit = BROADCAST_TX_COMMIT
self.mode_list = (BROADCAST_TX_ASYNC,
BROADCAST_TX_SYNC,
self.mode_commit)
self.tendermint_host = bigchaindb.config['tendermint']['host']
self.tendermint_port = bigchaindb.config['tendermint']['port']
self.endpoint = 'http://{}:{}/'.format(self.tendermint_host, self.tendermint_port)
validationPlugin = bigchaindb.config.get('validation_plugin')
if validationPlugin:
self.validation = config_utils.load_validation_plugin(validationPlugin)
else:
self.validation = BaseValidationRules
self.connection = connection if connection else backend.connect(**bigchaindb.config['database'])
def post_transaction(self, transaction, mode):
"""Submit a valid transaction to the mempool."""
if not mode or mode not in self.mode_list:
raise ValidationError('Mode must be one of the following {}.'
.format(', '.join(self.mode_list)))
tx_dict = transaction.tx_dict if transaction.tx_dict else transaction.to_dict()
payload = {
'method': mode,
'jsonrpc': '2.0',
'params': [encode_transaction(tx_dict)],
'id': str(uuid4())
}
# TODO: handle connection errors!
return requests.post(self.endpoint, json=payload)
def write_transaction(self, transaction, mode):
# This method offers backward compatibility with the Web API.
"""Submit a valid transaction to the mempool."""
response = self.post_transaction(transaction, mode)
return self._process_post_response(response.json(), mode)
def _process_post_response(self, response, mode):
logger.debug(response)
error = response.get('error')
if error:
status_code = 500
message = error.get('message', 'Internal Error')
data = error.get('data', '')
if 'Tx already exists in cache' in data:
status_code = 400
return (status_code, message + ' - ' + data)
result = response['result']
if mode == self.mode_commit:
check_tx_code = result.get('check_tx', {}).get('code', 0)
deliver_tx_code = result.get('deliver_tx', {}).get('code', 0)
error_code = check_tx_code or deliver_tx_code
else:
error_code = result.get('code', 0)
if error_code:
return (500, 'Transaction validation failed')
return (202, '')
def store_bulk_transactions(self, transactions):
txns = []
assets = []
txn_metadatas = []
for t in transactions:
transaction = t.tx_dict if t.tx_dict else rapidjson.loads(rapidjson.dumps(t.to_dict()))
if transaction['operation'] == t.CREATE:
asset = transaction.pop('asset')
asset['id'] = transaction['id']
assets.append(asset)
metadata = transaction.pop('metadata')
txn_metadatas.append({'id': transaction['id'],
'metadata': metadata})
txns.append(transaction)
backend.query.store_metadatas(self.connection, txn_metadatas)
if assets:
backend.query.store_assets(self.connection, assets)
return backend.query.store_transactions(self.connection, txns)
def delete_transactions(self, txs):
return backend.query.delete_transactions(self.connection, txs)
def update_utxoset(self, transaction):
"""Update the UTXO set given ``transaction``. That is, remove
the outputs that the given ``transaction`` spends, and add the
outputs that the given ``transaction`` creates.
Args:
transaction (:obj:`~bigchaindb.models.Transaction`): A new
transaction incoming into the system for which the UTXO
set needs to be updated.
"""
spent_outputs = [
spent_output for spent_output in transaction.spent_outputs
]
if spent_outputs:
self.delete_unspent_outputs(*spent_outputs)
self.store_unspent_outputs(
*[utxo._asdict() for utxo in transaction.unspent_outputs]
)
def store_unspent_outputs(self, *unspent_outputs):
"""Store the given ``unspent_outputs`` (utxos).
Args:
*unspent_outputs (:obj:`tuple` of :obj:`dict`): Variable
length tuple or list of unspent outputs.
"""
if unspent_outputs:
return backend.query.store_unspent_outputs(
self.connection, *unspent_outputs)
def get_utxoset_merkle_root(self):
"""Returns the merkle root of the utxoset. This implies that
the utxoset is first put into a merkle tree.
For now, the merkle tree and its root will be computed each
time. This obviously is not efficient and a better approach
that limits the repetition of the same computation when
unnecesary should be sought. For instance, future optimizations
could simply re-compute the branches of the tree that were
affected by a change.
The transaction hash (id) and output index should be sufficient
to uniquely identify a utxo, and consequently only that
information from a utxo record is needed to compute the merkle
root. Hence, each node of the merkle tree should contain the
tuple (txid, output_index).
.. important:: The leaves of the tree will need to be sorted in
some kind of lexicographical order.
Returns:
str: Merkle root in hexadecimal form.
"""
utxoset = backend.query.get_unspent_outputs(self.connection)
# TODO Once ready, use the already pre-computed utxo_hash field.
# See common/transactions.py for details.
hashes = [
sha3_256(
'{}{}'.format(utxo['transaction_id'], utxo['output_index']).encode()
).digest() for utxo in utxoset
]
# TODO Notice the sorted call!
return merkleroot(sorted(hashes))
def get_unspent_outputs(self):
"""Get the utxoset.
Returns:
generator of unspent_outputs.
"""
cursor = backend.query.get_unspent_outputs(self.connection)
return (record for record in cursor)
def delete_unspent_outputs(self, *unspent_outputs):
"""Deletes the given ``unspent_outputs`` (utxos).
Args:
*unspent_outputs (:obj:`tuple` of :obj:`dict`): Variable
length tuple or list of unspent outputs.
"""
if unspent_outputs:
return backend.query.delete_unspent_outputs(
self.connection, *unspent_outputs)
def is_committed(self, transaction_id):
transaction = backend.query.get_transaction(self.connection, transaction_id)
return bool(transaction)
def get_transaction(self, transaction_id):
transaction = backend.query.get_transaction(self.connection, transaction_id)
if transaction:
asset = backend.query.get_asset(self.connection, transaction_id)
metadata = backend.query.get_metadata(self.connection, [transaction_id])
if asset:
transaction['asset'] = asset
if 'metadata' not in transaction:
metadata = metadata[0] if metadata else None
if metadata:
metadata = metadata.get('metadata')
transaction.update({'metadata': metadata})
transaction = Transaction.from_dict(transaction)
return transaction
def get_transactions(self, txn_ids):
return backend.query.get_transactions(self.connection, txn_ids)
def get_transactions_filtered(self, asset_id, operation=None, last_tx=None):
"""Get a list of transactions filtered on some criteria
"""
txids = backend.query.get_txids_filtered(self.connection, asset_id,
operation, last_tx)
for txid in txids:
yield self.get_transaction(txid)
def get_outputs_filtered(self, owner, spent=None):
"""Get a list of output links filtered on some criteria
Args:
owner (str): base58 encoded public_key.
spent (bool): If ``True`` return only the spent outputs. If
``False`` return only unspent outputs. If spent is
not specified (``None``) return all outputs.
Returns:
:obj:`list` of TransactionLink: list of ``txid`` s and ``output`` s
pointing to another transaction's condition
"""
outputs = self.fastquery.get_outputs_by_public_key(owner)
if spent is None:
return outputs
elif spent is True:
return self.fastquery.filter_unspent_outputs(outputs)
elif spent is False:
return self.fastquery.filter_spent_outputs(outputs)
def get_spent(self, txid, output, current_transactions=[]):
transactions = backend.query.get_spent(self.connection, txid,
output)
transactions = list(transactions) if transactions else []
if len(transactions) > 1:
raise core_exceptions.CriticalDoubleSpend(
'`{}` was spent more than once. There is a problem'
' with the chain'.format(txid))
current_spent_transactions = []
for ctxn in current_transactions:
for ctxn_input in ctxn.inputs:
if ctxn_input.fulfills and\
ctxn_input.fulfills.txid == txid and\
ctxn_input.fulfills.output == output:
current_spent_transactions.append(ctxn)
transaction = None
if len(transactions) + len(current_spent_transactions) > 1:
raise DoubleSpend('tx "{}" spends inputs twice'.format(txid))
elif transactions:
transaction = Transaction.from_db(self, transactions[0])
elif current_spent_transactions:
transaction = current_spent_transactions[0]
return transaction
def store_block(self, block):
"""Create a new block."""
return backend.query.store_block(self.connection, block)
def get_latest_block(self):
"""Get the block with largest height."""
return backend.query.get_latest_block(self.connection)
def get_block(self, block_id):
"""Get the block with the specified `block_id`.
Returns the block corresponding to `block_id` or None if no match is
found.
Args:
block_id (int): block id of the block to get.
"""
block = backend.query.get_block(self.connection, block_id)
latest_block = self.get_latest_block()
latest_block_height = latest_block['height'] if latest_block else 0
if not block and block_id > latest_block_height:
return
result = {'height': block_id,
'transactions': []}
if block:
transactions = backend.query.get_transactions(self.connection, block['transactions'])
result['transactions'] = [t.to_dict() for t in Transaction.from_db(self, transactions)]
return result
def get_block_containing_tx(self, txid):
"""Retrieve the list of blocks (block ids) containing a
transaction with transaction id `txid`
Args:
txid (str): transaction id of the transaction to query
Returns:
Block id list (list(int))
"""
blocks = list(backend.query.get_block_with_transaction(self.connection, txid))
if len(blocks) > 1:
logger.critical('Transaction id %s exists in multiple blocks', txid)
return [block['height'] for block in blocks]
def validate_transaction(self, tx, current_transactions=[]):
"""Validate a transaction against the current status of the database."""
transaction = tx
# CLEANUP: The conditional below checks for transaction in dict format.
# It would be better to only have a single format for the transaction
# throught the code base.
if isinstance(transaction, dict):
try:
transaction = Transaction.from_dict(tx)
except SchemaValidationError as e:
logger.warning('Invalid transaction schema: %s', e.__cause__.message)
return False
except ValidationError as e:
logger.warning('Invalid transaction (%s): %s', type(e).__name__, e)
return False
return transaction.validate(self, current_transactions)
def is_valid_transaction(self, tx, current_transactions=[]):
# NOTE: the function returns the Transaction object in case
# the transaction is valid
try:
return self.validate_transaction(tx, current_transactions)
except ValidationError as e:
logger.warning('Invalid transaction (%s): %s', type(e).__name__, e)
return False
def text_search(self, search, *, limit=0, table='assets'):
"""Return an iterator of assets that match the text search
Args:
search (str): Text search string to query the text index
limit (int, optional): Limit the number of returned documents.
Returns:
iter: An iterator of assets that match the text search.
"""
return backend.query.text_search(self.connection, search, limit=limit,
table=table)
def get_assets(self, asset_ids):
"""Return a list of assets that match the asset_ids
Args:
asset_ids (:obj:`list` of :obj:`str`): A list of asset_ids to
retrieve from the database.
Returns:
list: The list of assets returned from the database.
"""
return backend.query.get_assets(self.connection, asset_ids)
def get_metadata(self, txn_ids):
"""Return a list of metadata that match the transaction ids (txn_ids)
Args:
txn_ids (:obj:`list` of :obj:`str`): A list of txn_ids to
retrieve from the database.
Returns:
list: The list of metadata returned from the database.
"""
return backend.query.get_metadata(self.connection, txn_ids)
@property
def fastquery(self):
return fastquery.FastQuery(self.connection)
def get_validator_change(self, height=None):
return backend.query.get_validator_set(self.connection, height)
def get_validators(self, height=None):
result = self.get_validator_change(height)
return [] if result is None else result['validators']
def get_election(self, election_id):
return backend.query.get_election(self.connection, election_id)
def get_pre_commit_state(self):
return backend.query.get_pre_commit_state(self.connection)
def store_pre_commit_state(self, state):
return backend.query.store_pre_commit_state(self.connection, state)
def store_validator_set(self, height, validators):
"""Store validator set at a given `height`.
NOTE: If the validator set already exists at that `height` then an
exception will be raised.
"""
return backend.query.store_validator_set(self.connection, {'height': height,
'validators': validators})
def delete_validator_set(self, height):
return backend.query.delete_validator_set(self.connection, height)
def store_abci_chain(self, height, chain_id, is_synced=True):
return backend.query.store_abci_chain(self.connection, height,
chain_id, is_synced)
def delete_abci_chain(self, height):
return backend.query.delete_abci_chain(self.connection, height)
def get_latest_abci_chain(self):
return backend.query.get_latest_abci_chain(self.connection)
def migrate_abci_chain(self):
"""Generate and record a new ABCI chain ID. New blocks are not
accepted until we receive an InitChain ABCI request with
the matching chain ID and validator set.
Chain ID is generated based on the current chain and height.
`chain-X` => `chain-X-migrated-at-height-5`.
`chain-X-migrated-at-height-5` => `chain-X-migrated-at-height-21`.
If there is no known chain (we are at genesis), the function returns.
"""
latest_chain = self.get_latest_abci_chain()
if latest_chain is None:
return
block = self.get_latest_block()
suffix = '-migrated-at-height-'
chain_id = latest_chain['chain_id']
block_height_str = str(block['height'])
new_chain_id = chain_id.split(suffix)[0] + suffix + block_height_str
self.store_abci_chain(block['height'] + 1, new_chain_id, False)
def store_election(self, election_id, height, is_concluded):
return backend.query.store_election(self.connection, election_id,
height, is_concluded)
def store_elections(self, elections):
return backend.query.store_elections(self.connection, elections)
def delete_elections(self, height):
return backend.query.delete_elections(self.connection, height)
Block = namedtuple('Block', ('app_hash', 'height', 'transactions'))

130
bigchaindb/log.py Normal file
View File

@ -0,0 +1,130 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
import bigchaindb
import logging
from bigchaindb.common.exceptions import ConfigurationError
from logging.config import dictConfig as set_logging_config
import os
DEFAULT_LOG_DIR = os.getcwd()
DEFAULT_LOGGING_CONFIG = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'console': {
'class': 'logging.Formatter',
'format': ('[%(asctime)s] [%(levelname)s] (%(name)s) '
'%(message)s (%(processName)-10s - pid: %(process)d)'),
'datefmt': '%Y-%m-%d %H:%M:%S',
},
'file': {
'class': 'logging.Formatter',
'format': ('[%(asctime)s] [%(levelname)s] (%(name)s) '
'%(message)s (%(processName)-10s - pid: %(process)d)'),
'datefmt': '%Y-%m-%d %H:%M:%S',
}
},
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'formatter': 'console',
'level': logging.INFO,
},
'file': {
'class': 'logging.handlers.RotatingFileHandler',
'filename': os.path.join(DEFAULT_LOG_DIR, 'bigchaindb.log'),
'mode': 'w',
'maxBytes': 209715200,
'backupCount': 5,
'formatter': 'file',
'level': logging.INFO,
},
'errors': {
'class': 'logging.handlers.RotatingFileHandler',
'filename': os.path.join(DEFAULT_LOG_DIR, 'bigchaindb-errors.log'),
'mode': 'w',
'maxBytes': 209715200,
'backupCount': 5,
'formatter': 'file',
'level': logging.ERROR,
}
},
'loggers': {},
'root': {
'level': logging.DEBUG,
'handlers': ['console', 'file', 'errors'],
},
}
def _normalize_log_level(level):
try:
return level.upper()
except AttributeError as exc:
raise ConfigurationError('Log level must be a string!') from exc
def setup_logging():
"""Function to configure log hadlers.
.. important::
Configuration, if needed, should be applied before invoking this
decorator, as starting the subscriber process for logging will
configure the root logger for the child process based on the
state of :obj:`bigchaindb.config` at the moment this decorator
is invoked.
"""
logging_configs = DEFAULT_LOGGING_CONFIG
new_logging_configs = bigchaindb.config['log']
if 'file' in new_logging_configs:
filename = new_logging_configs['file']
logging_configs['handlers']['file']['filename'] = filename
if 'error_file' in new_logging_configs:
error_filename = new_logging_configs['error_file']
logging_configs['handlers']['errors']['filename'] = error_filename
if 'level_console' in new_logging_configs:
level = _normalize_log_level(new_logging_configs['level_console'])
logging_configs['handlers']['console']['level'] = level
if 'level_logfile' in new_logging_configs:
level = _normalize_log_level(new_logging_configs['level_logfile'])
logging_configs['handlers']['file']['level'] = level
if 'fmt_console' in new_logging_configs:
fmt = new_logging_configs['fmt_console']
logging_configs['formatters']['console']['format'] = fmt
if 'fmt_logfile' in new_logging_configs:
fmt = new_logging_configs['fmt_logfile']
logging_configs['formatters']['file']['format'] = fmt
if 'datefmt_console' in new_logging_configs:
fmt = new_logging_configs['datefmt_console']
logging_configs['formatters']['console']['datefmt'] = fmt
if 'datefmt_logfile' in new_logging_configs:
fmt = new_logging_configs['datefmt_logfile']
logging_configs['formatters']['file']['datefmt'] = fmt
log_levels = new_logging_configs.get('granular_levels', {})
for logger_name, level in log_levels.items():
level = _normalize_log_level(level)
try:
logging_configs['loggers'][logger_name]['level'] = level
except KeyError:
logging_configs['loggers'][logger_name] = {'level': level}
set_logging_config(logging_configs)

0
bigchaindb/migrations/__init__.py Normal file
View File

48
bigchaindb/migrations/chain_migration_election.py Normal file
View File

@ -0,0 +1,48 @@
import json
from bigchaindb.common.schema import TX_SCHEMA_CHAIN_MIGRATION_ELECTION
from bigchaindb.elections.election import Election
class ChainMigrationElection(Election):
OPERATION = 'CHAIN_MIGRATION_ELECTION'
CREATE = OPERATION
ALLOWED_OPERATIONS = (OPERATION,)
TX_SCHEMA_CUSTOM = TX_SCHEMA_CHAIN_MIGRATION_ELECTION
def has_concluded(self, bigchaindb, *args, **kwargs):
chain = bigchaindb.get_latest_abci_chain()
if chain is not None and not chain['is_synced']:
# do not conclude the migration election if
# there is another migration in progress
return False
return super().has_concluded(bigchaindb, *args, **kwargs)
def on_approval(self, bigchain, *args, **kwargs):
bigchain.migrate_abci_chain()
def show_election(self, bigchain):
output = super().show_election(bigchain)
chain = bigchain.get_latest_abci_chain()
if chain is None or chain['is_synced']:
return output
output += f'\nchain_id={chain["chain_id"]}'
block = bigchain.get_latest_block()
output += f'\napp_hash={block["app_hash"]}'
validators = [
{
'pub_key': {
'type': 'tendermint/PubKeyEd25519',
'value': k,
},
'power': v,
} for k, v in self.get_validators(bigchain).items()
]
output += f'\nvalidators={json.dumps(validators, indent=4)}'
return output
def on_rollback(self, bigchain, new_height):
bigchain.delete_abci_chain(new_height)

75
bigchaindb/models.py Normal file
View File

@ -0,0 +1,75 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
from bigchaindb.backend.schema import validate_language_key
from bigchaindb.common.exceptions import (InvalidSignature,
DuplicateTransaction)
from bigchaindb.common.schema import validate_transaction_schema
from bigchaindb.common.transaction import Transaction
from bigchaindb.common.utils import (validate_txn_obj, validate_key)
class Transaction(Transaction):
ASSET = 'asset'
METADATA = 'metadata'
DATA = 'data'
def validate(self, bigchain, current_transactions=[]):
"""Validate transaction spend
Args:
bigchain (Planetmint): an instantiated bigchaindb.Planetmint object.
Returns:
The transaction (Transaction) if the transaction is valid else it
raises an exception describing the reason why the transaction is
invalid.
Raises:
ValidationError: If the transaction is invalid
"""
input_conditions = []
if self.operation == Transaction.CREATE:
duplicates = any(txn for txn in current_transactions if txn.id == self.id)
if bigchain.is_committed(self.id) or duplicates:
raise DuplicateTransaction('transaction `{}` already exists'
.format(self.id))
if not self.inputs_valid(input_conditions):
raise InvalidSignature('Transaction signature is invalid.')
elif self.operation == Transaction.TRANSFER:
self.validate_transfer_inputs(bigchain, current_transactions)
return self
@classmethod
def from_dict(cls, tx_body):
return super().from_dict(tx_body, False)
@classmethod
def validate_schema(cls, tx_body):
validate_transaction_schema(tx_body)
validate_txn_obj(cls.ASSET, tx_body[cls.ASSET], cls.DATA, validate_key)
validate_txn_obj(cls.METADATA, tx_body, cls.METADATA, validate_key)
validate_language_key(tx_body[cls.ASSET], cls.DATA)
validate_language_key(tx_body, cls.METADATA)
class FastTransaction:
"""A minimal wrapper around a transaction dictionary. This is useful for
when validation is not required but a routine expects something that looks
like a transaction, for example during block creation.
Note: immutability could also be provided
"""
def __init__(self, tx_dict):
self.data = tx_dict
@property
def id(self):
return self.data['id']
def to_dict(self):
return self.data

124
bigchaindb/parallel_validation.py Normal file
View File

@ -0,0 +1,124 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
import multiprocessing as mp
from collections import defaultdict
from bigchaindb import App, BigchainDB
from bigchaindb.tendermint_utils import decode_transaction
from abci import CodeTypeOk
class ParallelValidationApp(App):
def __init__(self, bigchaindb=None, events_queue=None, abci=None):
super().__init__(bigchaindb, events_queue, abci=abci)
self.parallel_validator = ParallelValidator()
self.parallel_validator.start()
def check_tx(self, raw_transaction):
return self.abci.ResponseCheckTx(code=CodeTypeOk)
def deliver_tx(self, raw_transaction):
self.parallel_validator.validate(raw_transaction)
return self.abci.ResponseDeliverTx(code=CodeTypeOk)
def end_block(self, request_end_block):
result = self.parallel_validator.result(timeout=30)
for transaction in result:
if transaction:
self.block_txn_ids.append(transaction.id)
self.block_transactions.append(transaction)
return super().end_block(request_end_block)
RESET = 'reset'
EXIT = 'exit'
class ParallelValidator:
def __init__(self, number_of_workers=mp.cpu_count()):
self.number_of_workers = number_of_workers
self.transaction_index = 0
self.routing_queues = [mp.Queue() for _ in range(self.number_of_workers)]
self.workers = []
self.results_queue = mp.Queue()
def start(self):
for routing_queue in self.routing_queues:
worker = ValidationWorker(routing_queue, self.results_queue)
process = mp.Process(target=worker.run)
process.start()
self.workers.append(process)
def stop(self):
for routing_queue in self.routing_queues:
routing_queue.put(EXIT)
def validate(self, raw_transaction):
dict_transaction = decode_transaction(raw_transaction)
index = int(dict_transaction['id'], 16) % self.number_of_workers
self.routing_queues[index].put((self.transaction_index, dict_transaction))
self.transaction_index += 1
def result(self, timeout=None):
result_buffer = [None] * self.transaction_index
for _ in range(self.transaction_index):
index, transaction = self.results_queue.get(timeout=timeout)
result_buffer[index] = transaction
self.transaction_index = 0
for routing_queue in self.routing_queues:
routing_queue.put(RESET)
return result_buffer
class ValidationWorker:
"""Run validation logic in a loop. This Worker is suitable for a Process
life: no thrills, just a queue to get some values, and a queue to return results.
Note that a worker is expected to validate multiple transactions in
multiple rounds, and it needs to keep in memory all transactions already
validated, until a new round starts. To trigger a new round of validation,
a ValidationWorker expects a `RESET` message. To exit the infinite loop the
worker is in, it expects an `EXIT` message.
"""
def __init__(self, in_queue, results_queue):
self.in_queue = in_queue
self.results_queue = results_queue
self.bigchaindb = BigchainDB()
self.reset()
def reset(self):
# We need a place to store already validated transactions,
# in case of dependant transactions in the same block.
# `validated_transactions` maps an `asset_id` with the list
# of all other transactions sharing the same asset.
self.validated_transactions = defaultdict(list)
def validate(self, dict_transaction):
try:
asset_id = dict_transaction['asset']['id']
except KeyError:
asset_id = dict_transaction['id']
transaction = self.bigchaindb.is_valid_transaction(
dict_transaction,
self.validated_transactions[asset_id])
if transaction:
self.validated_transactions[asset_id].append(transaction)
return transaction
def run(self):
while True:
message = self.in_queue.get()
if message == RESET:
self.reset()
elif message == EXIT:
return
else:
index, transaction = message
self.results_queue.put((index, self.validate(transaction)))

90
bigchaindb/start.py Normal file
View File

@ -0,0 +1,90 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
import logging
import setproctitle
from abci import TmVersion, ABCI
import bigchaindb
from bigchaindb.lib import BigchainDB
from bigchaindb.core import App
from bigchaindb.parallel_validation import ParallelValidationApp
from bigchaindb.web import server, websocket_server
from bigchaindb.events import Exchange, EventTypes
from bigchaindb.utils import Process
logger = logging.getLogger(__name__)
BANNER = """
****************************************************************************
* *
* Planetmint 2.2.2 *
* codename "jumping sloth" *
* Initialization complete. Planetmint Server is ready and waiting. *
* *
* You can send HTTP requests via the HTTP API documented in the *
* Planetmint Server docs at: *
* https://bigchaindb.com/http-api *
* *
* Listening to client connections on: {:<15} *
* *
****************************************************************************
"""
def start(args):
# Exchange object for event stream api
logger.info('Starting Planetmint')
exchange = Exchange()
# start the web api
app_server = server.create_server(
settings=bigchaindb.config['server'],
log_config=bigchaindb.config['log'],
bigchaindb_factory=BigchainDB)
p_webapi = Process(name='bigchaindb_webapi', target=app_server.run, daemon=True)
p_webapi.start()
logger.info(BANNER.format(bigchaindb.config['server']['bind']))
# start websocket server
p_websocket_server = Process(name='bigchaindb_ws',
target=websocket_server.start,
daemon=True,
args=(exchange.get_subscriber_queue(EventTypes.BLOCK_VALID),))
p_websocket_server.start()
p_exchange = Process(name='bigchaindb_exchange', target=exchange.run, daemon=True)
p_exchange.start()
# We need to import this after spawning the web server
# because import ABCIServer will monkeypatch all sockets
# for gevent.
from abci.server import ABCIServer
setproctitle.setproctitle('bigchaindb')
# Start the ABCIServer
abci = ABCI(TmVersion(bigchaindb.config['tendermint']['version']))
if args.experimental_parallel_validation:
app = ABCIServer(
app=ParallelValidationApp(
abci=abci.types,
events_queue=exchange.get_publisher_queue(),
)
)
else:
app = ABCIServer(
app=App(
abci=abci.types,
events_queue=exchange.get_publisher_queue(),
)
)
app.run()
if __name__ == '__main__':
start()

100
bigchaindb/tendermint_utils.py Normal file
View File

@ -0,0 +1,100 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
import base64
import hashlib
import json
from binascii import hexlify
try:
from hashlib import sha3_256
except ImportError:
from sha3 import sha3_256
def encode_transaction(value):
"""Encode a transaction (dict) to Base64."""
return base64.b64encode(json.dumps(value).encode('utf8')).decode('utf8')
def decode_transaction(raw):
"""Decode a transaction from bytes to a dict."""
return json.loads(raw.decode('utf8'))
def decode_transaction_base64(value):
"""Decode a transaction from Base64."""
return json.loads(base64.b64decode(value.encode('utf8')).decode('utf8'))
def calculate_hash(key_list):
if not key_list:
return ''
full_hash = sha3_256()
for key in key_list:
full_hash.update(key.encode('utf8'))
return full_hash.hexdigest()
def merkleroot(hashes):
"""Computes the merkle root for a given list.
Args:
hashes (:obj:`list` of :obj:`bytes`): The leaves of the tree.
Returns:
str: Merkle root in hexadecimal form.
"""
# XXX TEMPORARY -- MUST REVIEW and possibly CHANGE
# The idea here is that the UTXO SET would be empty and this function
# would be invoked to compute the merkle root, and since there is nothing,
# i.e. an empty list, then the hash of the empty string is returned.
# This seems too easy but maybe that is good enough? TO REVIEW!
if not hashes:
return sha3_256(b'').hexdigest()
# XXX END TEMPORARY -- MUST REVIEW ...
if len(hashes) == 1:
return hexlify(hashes[0]).decode()
if len(hashes) % 2 == 1:
hashes.append(hashes[-1])
parent_hashes = [
sha3_256(hashes[i] + hashes[i+1]).digest()
for i in range(0, len(hashes)-1, 2)
]
return merkleroot(parent_hashes)
def public_key64_to_address(base64_public_key):
"""Note this only compatible with Tendermint 0.19.x"""
ed25519_public_key = public_key_from_base64(base64_public_key)
encoded_public_key = amino_encoded_public_key(ed25519_public_key)
return hashlib.new('ripemd160', encoded_public_key).hexdigest().upper()
def public_key_from_base64(base64_public_key):
return key_from_base64(base64_public_key)
def key_from_base64(base64_key):
return base64.b64decode(base64_key).hex().upper()
def public_key_to_base64(ed25519_public_key):
return key_to_base64(ed25519_public_key)
def key_to_base64(ed25519_key):
ed25519_key = bytes.fromhex(ed25519_key)
return base64.b64encode(ed25519_key).decode('utf-8')
def amino_encoded_public_key(ed25519_public_key):
return bytes.fromhex('1624DE6220{}'.format(ed25519_public_key))

7
bigchaindb/upsert_validator/__init__.py Normal file
View File

@ -0,0 +1,7 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
from bigchaindb.upsert_validator.validator_election import ValidatorElection # noqa

68
bigchaindb/upsert_validator/validator_election.py Normal file
View File

@ -0,0 +1,68 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
from bigchaindb.common.exceptions import InvalidPowerChange
from bigchaindb.elections.election import Election
from bigchaindb.common.schema import TX_SCHEMA_VALIDATOR_ELECTION
from .validator_utils import (new_validator_set, encode_validator, validate_asset_public_key)
class ValidatorElection(Election):
OPERATION = 'VALIDATOR_ELECTION'
# NOTE: this transaction class extends create so the operation inheritence is achieved
# by renaming CREATE to VALIDATOR_ELECTION
CREATE = OPERATION
ALLOWED_OPERATIONS = (OPERATION,)
TX_SCHEMA_CUSTOM = TX_SCHEMA_VALIDATOR_ELECTION
def validate(self, bigchain, current_transactions=[]):
"""For more details refer BEP-21: https://github.com/bigchaindb/BEPs/tree/master/21
"""
current_validators = self.get_validators(bigchain)
super(ValidatorElection, self).validate(bigchain, current_transactions=current_transactions)
# NOTE: change more than 1/3 of the current power is not allowed
if self.asset['data']['power'] >= (1/3)*sum(current_validators.values()):
raise InvalidPowerChange('`power` change must be less than 1/3 of total power')
return self
@classmethod
def validate_schema(cls, tx):
super(ValidatorElection, cls).validate_schema(tx)
validate_asset_public_key(tx['asset']['data']['public_key'])
def has_concluded(self, bigchain, *args, **kwargs):
latest_block = bigchain.get_latest_block()
if latest_block is not None:
latest_block_height = latest_block['height']
latest_validator_change = bigchain.get_validator_change()['height']
# TODO change to `latest_block_height + 3` when upgrading to Tendermint 0.24.0.
if latest_validator_change == latest_block_height + 2:
# do not conclude the election if there is a change assigned already
return False
return super().has_concluded(bigchain, *args, **kwargs)
def on_approval(self, bigchain, new_height):
validator_updates = [self.asset['data']]
curr_validator_set = bigchain.get_validators(new_height)
updated_validator_set = new_validator_set(curr_validator_set,
validator_updates)
updated_validator_set = [v for v in updated_validator_set
if v['voting_power'] > 0]
# TODO change to `new_height + 2` when upgrading to Tendermint 0.24.0.
bigchain.store_validator_set(new_height + 1, updated_validator_set)
return encode_validator(self.asset['data'])
def on_rollback(self, bigchaindb, new_height):
# TODO change to `new_height + 2` when upgrading to Tendermint 0.24.0.
bigchaindb.delete_validator_set(new_height + 1)

85
bigchaindb/upsert_validator/validator_utils.py Normal file
View File

@ -0,0 +1,85 @@
import base64
import binascii
import codecs
import bigchaindb
from abci import types_v0_22_8, types_v0_31_5, TmVersion
from bigchaindb.common.exceptions import InvalidPublicKey, BigchainDBError
def encode_validator(v):
ed25519_public_key = v['public_key']['value']
# NOTE: tendermint expects public to be encoded in go-amino format
try:
version = TmVersion(bigchaindb.config["tendermint"]["version"])
except ValueError:
raise BigchainDBError('Invalid tendermint version, '
'check Planetmint configuration file')
validator_update_t, pubkey_t = {
TmVersion.v0_22_8: (types_v0_22_8.Validator, types_v0_22_8.PubKey),
TmVersion.v0_31_5: (types_v0_31_5.ValidatorUpdate, types_v0_31_5.PubKey)
}[version]
pub_key = pubkey_t(type='ed25519', data=bytes.fromhex(ed25519_public_key))
return validator_update_t(pub_key=pub_key, power=v['power'])
def decode_validator(v):
return {'public_key': {'type': 'ed25519-base64',
'value': codecs.encode(v.pub_key.data, 'base64').decode().rstrip('\n')},
'voting_power': v.power}
def new_validator_set(validators, updates):
validators_dict = {}
for v in validators:
validators_dict[v['public_key']['value']] = v
updates_dict = {}
for u in updates:
decoder = get_public_key_decoder(u['public_key'])
public_key64 = base64.b64encode(decoder(u['public_key']['value'])).decode('utf-8')
updates_dict[public_key64] = {'public_key': {'type': 'ed25519-base64',
'value': public_key64},
'voting_power': u['power']}
new_validators_dict = {**validators_dict, **updates_dict}
return list(new_validators_dict.values())
def encode_pk_to_base16(validator):
pk = validator['public_key']
decoder = get_public_key_decoder(pk)
public_key16 = base64.b16encode(decoder(pk['value'])).decode('utf-8')
validator['public_key']['value'] = public_key16
return validator
def validate_asset_public_key(pk):
pk_binary = pk['value'].encode('utf-8')
decoder = get_public_key_decoder(pk)
try:
pk_decoded = decoder(pk_binary)
if len(pk_decoded) != 32:
raise InvalidPublicKey('Public key should be of size 32 bytes')
except binascii.Error:
raise InvalidPublicKey('Invalid `type` specified for public key `value`')
def get_public_key_decoder(pk):
encoding = pk['type']
decoder = base64.b64decode
if encoding == 'ed25519-base16':
decoder = base64.b16decode
elif encoding == 'ed25519-base32':
decoder = base64.b32decode
elif encoding == 'ed25519-base64':
decoder = base64.b64decode
else:
raise InvalidPublicKey('Invalid `type` specified for public key `value`')
return decoder

209
bigchaindb/utils.py Normal file
View File

@ -0,0 +1,209 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
import contextlib
import threading
import queue
import multiprocessing as mp
import json
import setproctitle
from packaging import version
from bigchaindb.version import __tm_supported_versions__
from bigchaindb.tendermint_utils import key_from_base64
from bigchaindb.common.crypto import key_pair_from_ed25519_key
class ProcessGroup(object):
def __init__(self, concurrency=None, group=None, target=None, name=None,
args=None, kwargs=None, daemon=None):
self.concurrency = concurrency or mp.cpu_count()
self.group = group
self.target = target
self.name = name
self.args = args or ()
self.kwargs = kwargs or {}
self.daemon = daemon
self.processes = []
def start(self):
for i in range(self.concurrency):
proc = mp.Process(group=self.group, target=self.target,
name=self.name, args=self.args,
kwargs=self.kwargs, daemon=self.daemon)
proc.start()
self.processes.append(proc)
class Process(mp.Process):
"""Wrapper around multiprocessing.Process that uses
setproctitle to set the name of the process when running
the target task.
"""
def run(self):
setproctitle.setproctitle(self.name)
super().run()
# Inspired by:
# - http://stackoverflow.com/a/24741694/597097
def pool(builder, size, timeout=None):
"""Create a pool that imposes a limit on the number of stored
instances.
Args:
builder: a function to build an instance.
size: the size of the pool.
timeout(Optional[float]): the seconds to wait before raising
a ``queue.Empty`` exception if no instances are available
within that time.
Raises:
If ``timeout`` is defined but the request is taking longer
than the specified time, the context manager will raise
a ``queue.Empty`` exception.
Returns:
A context manager that can be used with the ``with``
statement.
"""
lock = threading.Lock()
local_pool = queue.Queue()
current_size = 0
@contextlib.contextmanager
def pooled():
nonlocal current_size
instance = None
# If we still have free slots, then we have room to create new
# instances.
if current_size < size:
with lock:
# We need to check again if we have slots available, since
# the situation might be different after acquiring the lock
if current_size < size:
current_size += 1
instance = builder()
# Watchout: current_size can be equal to size if the previous part of
# the function has been executed, that's why we need to check if the
# instance is None.
if instance is None:
instance = local_pool.get(timeout=timeout)
yield instance
local_pool.put(instance)
return pooled
# TODO: Rename this function, it's handling fulfillments not conditions
def condition_details_has_owner(condition_details, owner):
"""Check if the public_key of owner is in the condition details
as an Ed25519Fulfillment.public_key
Args:
condition_details (dict): dict with condition details
owner (str): base58 public key of owner
Returns:
bool: True if the public key is found in the condition details, False otherwise
"""
if 'subconditions' in condition_details:
result = condition_details_has_owner(condition_details['subconditions'], owner)
if result:
return True
elif isinstance(condition_details, list):
for subcondition in condition_details:
result = condition_details_has_owner(subcondition, owner)
if result:
return True
else:
if 'public_key' in condition_details \
and owner == condition_details['public_key']:
return True
return False
class Lazy:
"""Lazy objects are useful to create chains of methods to
execute later.
A lazy object records the methods that has been called, and
replay them when the :py:meth:`run` method is called. Note that
:py:meth:`run` needs an object `instance` to replay all the
methods that have been recorded.
"""
def __init__(self):
"""Instantiate a new Lazy object."""
self.stack = []
def __getattr__(self, name):
self.stack.append(name)
return self
def __call__(self, *args, **kwargs):
self.stack.append((args, kwargs))
return self
def __getitem__(self, key):
self.stack.append('__getitem__')
self.stack.append(([key], {}))
return self
def run(self, instance):
"""Run the recorded chain of methods on `instance`.
Args:
instance: an object.
"""
last = instance
for item in self.stack:
if isinstance(item, str):
last = getattr(last, item)
else:
last = last(*item[0], **item[1])
self.stack = []
return last
# Load Tendermint's public and private key from the file path
def load_node_key(path):
with open(path) as json_data:
priv_validator = json.load(json_data)
priv_key = priv_validator['priv_key']['value']
hex_private_key = key_from_base64(priv_key)
return key_pair_from_ed25519_key(hex_private_key)
def tendermint_version_is_compatible(running_tm_ver):
"""
Check Tendermint compatability with Planetmint server
:param running_tm_ver: Version number of the connected Tendermint instance
:type running_tm_ver: str
:return: True/False depending on the compatability with Planetmint server
:rtype: bool
"""
# Splitting because version can look like this e.g. 0.22.8-40d6dc2e
tm_ver = running_tm_ver.split('-')
if not tm_ver:
return False
for ver in __tm_supported_versions__:
if version.parse(ver) == version.parse(tm_ver[0]):
return True
return False

25
bigchaindb/validation.py Normal file
View File

@ -0,0 +1,25 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
class BaseValidationRules():
"""Base validation rules for Planetmint.
A validation plugin must expose a class inheriting from this one via an entry_point.
All methods listed below must be implemented.
"""
@staticmethod
def validate_transaction(bigchaindb, transaction):
"""See :meth:`bigchaindb.models.Transaction.validate`
for documentation.
"""
return transaction.validate(bigchaindb)
@staticmethod
def validate_block(bigchaindb, block):
"""See :meth:`bigchaindb.models.Block.validate` for documentation."""
return block.validate(bigchaindb)

10
bigchaindb/version.py Normal file
View File

@ -0,0 +1,10 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
__version__ = '2.2.2'
__short_version__ = '2.2'
# Supported Tendermint versions
__tm_supported_versions__ = ["0.31.5", "0.22.8"]

0
bigchaindb/web/__init__.py Normal file
View File

48
bigchaindb/web/routes.py Normal file
View File

@ -0,0 +1,48 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
"""API routes definition"""
from flask_restful import Api
from bigchaindb.web.views import (
assets,
metadata,
blocks,
info,
transactions as tx,
outputs,
validators,
)
def add_routes(app):
"""Add the routes to an app"""
for (prefix, routes) in API_SECTIONS:
api = Api(app, prefix=prefix)
for ((pattern, resource, *args), kwargs) in routes:
kwargs.setdefault('strict_slashes', False)
api.add_resource(resource, pattern, *args, **kwargs)
def r(*args, **kwargs):
return (args, kwargs)
ROUTES_API_V1 = [
r('/', info.ApiV1Index),
r('assets/', assets.AssetListApi),
r('metadata/', metadata.MetadataApi),
r('blocks/<int:block_id>', blocks.BlockApi),
r('blocks/', blocks.BlockListApi),
r('transactions/<string:tx_id>', tx.TransactionApi),
r('transactions', tx.TransactionListApi),
r('outputs/', outputs.OutputListApi),
r('validators/', validators.ValidatorsApi),
]
API_SECTIONS = [
(None, [r('/', info.RootIndex)]),
('/api/v1/', ROUTES_API_V1),
]

118
bigchaindb/web/server.py Normal file
View File

@ -0,0 +1,118 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
"""This module contains basic functions to instantiate the Planetmint API.
The application is implemented in Flask and runs using Gunicorn.
"""
import copy
import multiprocessing
from flask import Flask
from flask_cors import CORS
import gunicorn.app.base
from bigchaindb import utils
from bigchaindb import BigchainDB
from bigchaindb.web.routes import add_routes
from bigchaindb.web.strip_content_type_middleware import StripContentTypeMiddleware
# TODO: Figure out if we do we need all this boilerplate.
class StandaloneApplication(gunicorn.app.base.BaseApplication):
"""Run a **wsgi** app wrapping it in a Gunicorn Base Application.
Adapted from:
- http://docs.gunicorn.org/en/latest/custom.html
"""
def __init__(self, app, *, options=None):
"""Initialize a new standalone application.
Args:
app: A wsgi Python application.
options (dict): the configuration.
"""
self.options = options or {}
self.application = app
super().__init__()
def load_config(self):
# find a better way to pass this such that
# the custom logger class can access it.
custom_log_config = self.options.get('custom_log_config')
self.cfg.env_orig['custom_log_config'] = custom_log_config
config = dict((key, value) for key, value in self.options.items()
if key in self.cfg.settings and value is not None)
config['default_proc_name'] = 'bigchaindb_gunicorn'
for key, value in config.items():
# not sure if we need the `key.lower` here, will just keep
# keep it for now.
self.cfg.set(key.lower(), value)
def load(self):
return self.application
def create_app(*, debug=False, threads=1, bigchaindb_factory=None):
"""Return an instance of the Flask application.
Args:
debug (bool): a flag to activate the debug mode for the app
(default: False).
threads (int): number of threads to use
Return:
an instance of the Flask application.
"""
if not bigchaindb_factory:
bigchaindb_factory = Planetmint
app = Flask(__name__)
app.wsgi_app = StripContentTypeMiddleware(app.wsgi_app)
CORS(app)
app.debug = debug
app.config['bigchain_pool'] = utils.pool(bigchaindb_factory, size=threads)
add_routes(app)
return app
def create_server(settings, log_config=None, bigchaindb_factory=None):
"""Wrap and return an application ready to be run.
Args:
settings (dict): a dictionary containing the settings, more info
here http://docs.gunicorn.org/en/latest/settings.html
Return:
an initialized instance of the application.
"""
settings = copy.deepcopy(settings)
if not settings.get('workers'):
settings['workers'] = (multiprocessing.cpu_count() * 2) + 1
if not settings.get('threads'):
# Note: Threading is not recommended currently, as the frontend workload
# is largely CPU bound and parallisation across Python threads makes it
# slower.
settings['threads'] = 1
settings['custom_log_config'] = log_config
app = create_app(debug=settings.get('debug', False),
threads=settings['threads'],
bigchaindb_factory=bigchaindb_factory)
standalone = StandaloneApplication(app, options=settings)
return standalone

32
bigchaindb/web/strip_content_type_middleware.py Normal file
View File

@ -0,0 +1,32 @@
# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
import logging
logger = logging.getLogger(__name__)
class StripContentTypeMiddleware:
"""WSGI middleware to strip Content-Type header for GETs."""
def __init__(self, app):
"""Create the new middleware.
Args:
app: a flask application
"""
self.app = app
def __call__(self, environ, start_response):
"""Run the middleware and then call the original WSGI application."""
if environ['REQUEST_METHOD'] == 'GET':
try:
del environ['CONTENT_TYPE']
except KeyError:
pass
else:
logger.debug('Remove header "Content-Type" from GET request')
return self.app(environ, start_response)

Some files were not shown because too many files have changed in this diff Show More