Merge remote-tracking branch 'remotes/bigchaindb/develop' into feat-multisig

Conflicts:
	bigchaindb/core.py

CHANGELOG.md

@@ -0,0 +1,99 @@
+# Change Log (Release Notes)
+All _notable_ changes to this project will be documented in this file (`CHANGELOG.md`).
+This project adheres to [Semantic Versioning](http://semver.org/) (or at least we try).
+Contributors to this file, please follow the guidelines on [keepachangelog.com](http://keepachangelog.com/).
+Note that each version (or "release") is the name of a [Git _tag_](https://git-scm.com/book/en/v2/Git-Basics-Tagging) of a particular commit, so the associated date and time are the date and time of that commit (as reported by GitHub), _not_ the "Uploaded on" date listed on PyPI (which may differ).
+For reference, the possible headings are:
+
+* **Added** for new features.
+* **Changed** for changes in existing functionality.
+* **Deprecated** for once-stable features removed in upcoming releases.
+* **Removed** for deprecated features removed in this release.
+* **Fixed** for any bug fixes.
+* **Security** to invite users to upgrade in case of vulnerabilities.
+
+
+## [Unreleased] - YYYY-MM-DD
+Tag name: TBD
+= commit: TBD
+committed: TBD
+
+### Added
+- `CHANGELOG.md` (this file)
+- Multisig support: [Pull Request #107](https://github.com/bigchaindb/bigchaindb/pull/107)
+- API/Wire protocol (RESTful HTTP API): [Pull Request #102](https://github.com/bigchaindb/bigchaindb/pull/102)
+- Python driver/SDK/API: [Pull Request #102](https://github.com/bigchaindb/bigchaindb/pull/102)
+- Python Style Guide: [Pull Request #89](https://github.com/bigchaindb/bigchaindb/pull/89)
+- Monitoring & dashboard tools: [Pull Request #72](https://github.com/bigchaindb/bigchaindb/pull/72)
+
+### Changed
+- Rewrote [`README.md`](https://github.com/bigchaindb/bigchaindb/blob/develop/README.md) into four sets of links: Pull Requests [#80](https://github.com/bigchaindb/bigchaindb/pull/80) and [#115](https://github.com/bigchaindb/bigchaindb/pull/115)
+
+### Fixed
+- Bug related to config overwrite: [Pull Request #97](https://github.com/bigchaindb/bigchaindb/pull/97)
+- [Issue #71](https://github.com/bigchaindb/bigchaindb/issues/71) (Voter is not validating blocks correctly when checking for double spends) in [Pull Request #76](https://github.com/bigchaindb/bigchaindb/pull/76)
+
+
+## [0.1.4] - 2016-02-22
+Tag name: v0.1.4 
+= commit: c4c850f480bc9ae72df2a54f81c0825b6fb4ed62 
+committed: Feb 22, 2016, 11:51 AM GMT+1
+
+### Added
+- Added to `classifiers` to setup.py
+
+### Changed
+- Allow running pytest tests in parallel (using [xdist](http://pytest.org/latest/xdist.html)): [Pull Request #65](https://github.com/bigchaindb/bigchaindb/pull/65)
+- Allow non-interactive first start: [Pull Request #64](https://github.com/bigchaindb/bigchaindb/pull/64) to resolve [Issue #58](https://github.com/bigchaindb/bigchaindb/issues/58)
+
+
+## [0.1.3] - 2016-02-16
+Tag name: v0.1.3
+= commit 8926e3216c1ee39b9bc332e5ef1df2a8901262dd
+committed Feb 16, 2016, 11:37 AM GMT+1
+
+### Changed
+- Changed from using Git Flow to GitHub flow (but with `develop` as the default branch).
+
+
+## [0.1.2] - 2016-02-15
+Tag name: v0.1.2 
+= commit d2ff24166d69dda68dd7b4a24a88279b1d37e222 
+committed Feb 15, 2016, 2:23 PM GMT+1
+
+### Added
+- Various tests
+
+### Fixed
+- Fix exception when running `start`: [Pull Request #32](https://github.com/bigchaindb/bigchaindb/pull/32) resolved [Issue #35]
+
+## [0.1.1] - 2016-02-15
+Tag name: v0.1.1 
+= commit 2a025448b29fe7056760de1039c73bbcfe992461 
+committed Feb 15, 2016, 10:48 AM GMT+1
+
+### Added
+- "release 0.1.1": [Pull Request #37](https://github.com/bigchaindb/bigchaindb/pull/37)
+
+### Removed
+- `tox.ini` [Pull Request #18](https://github.com/bigchaindb/bigchaindb/pull/18)
+- `requirements.txt` in the root directory, and the entire `requirements/` directory: [Pull Request #14](https://github.com/bigchaindb/bigchaindb/pull/14)
+
+### Fixed
+- Hotfix for AttributeError, fixed [Issue #27](https://github.com/bigchaindb/bigchaindb/issues/27)
+
+
+## [0.1.0] - 2016-02-10
+Tag name: v0.1.0
+= commit 8539e8dc2d036a4e0a866a3fb9e55889503254d5
+committed Feb 10, 2016, 10:04 PM GMT+1
+
+The first public release of BigchainDB, including:
+
+- Initial BigchainDB Server code, including many tests and some code for benchmarking.
+- Initial documentation (in `bigchaindb/docs`).
+- Initial `README.md`, `ROADMAP.md`, `CODE_OF_CONDUCT.md`, and `CONTRIBUTING.md`.
+- Packaging for PyPI, including `setup.py` and `setup.cfg`.
+- Initial `Dockerfile` and `docker-compose.yml` (for deployment using Docker and Docker Compose).
+- Initial `.gitignore` (list of things for git to ignore).
+- Initial `.travis.yml` (used by Travis CI).

CONTRIBUTING.md

@@ -84,6 +84,8 @@ git merge upstream/develop
 
 Once you're done commiting a set of new things and you're ready to submit them for inclusion, please be sure to run all the tests (as per the instructions at the end of our [Python Style Guide](PYTHON_STYLE_GUIDE.md)).
 
+If your addition or change is substantial, then please add a line or two to the [CHANGELOG.md file](https://github.com/bigchaindb/bigchaindb/blob/develop/CHANGELOG.md), following the guidelines given at the top of that file.
+
 (When you submit your pull request [following the instructions below], we run all the tests automatically, so we will see if some are failing. If you don't know why some tests are failing, you can still submit your pull request, but be sure to note the failing tests and to ask for help with resolving them.)
 
 ### Step 6 - Push Your New Branch to origin

LICENSES.md

@@ -2,7 +2,7 @@
 
 All officially-supported BigchainDB _driver code_ is licensed under the Apache License, Version 2.0, the full text of which can be found at [http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0).
 
-All short code snippets embedded in the official BigchainDB _documentation_ is licensed under the Apache License, Version 2.0, the full text of which can be found at [http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0).
+All short code snippets embedded in the official BigchainDB _documentation_ are licensed under the Apache License, Version 2.0, the full text of which can be found at [http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0).
 
 All _other_ officially-supported BigchainDB code is licensed under the GNU Affero General Public License version 3 (AGPLv3), the full text of which can be found at [http://www.gnu.org/licenses/agpl.html](http://www.gnu.org/licenses/agpl.html).
 

ROADMAP.md

@@ -2,20 +2,20 @@
 
 ## BigchainDB Protocols
 * Validation of other nodes
-* Byzantine fault tolerance
+* Fault tolerance
 * Permissions framework
-* Benchmarks (e.g. on transactions/second and latency)
-* API/Wire protocol exposed by the BigchainDB dameon (HTTP or other). Eventually, the _only_ way for a client to communicate with a BigchainDB database will be via this API.
 * Protocol audits including security audits
 
 ## Implementation/Code
 * Node validation framework (inspect and agree or not with what the other nodes are doing)
-* Federation management and monitoring/dashboard
+* Federation management tools
-* Packaging, dockerization, AWS image, etc. (i.e. easy deployment options)
+* More tools for benchmarking a cluster
-* Drivers/SDKs for common client-side languages (e.g. Python, Ruby, JavaScript, Java)
+* Descriptions and results of more benchmarking tests
+* AWS image and other easy deployment options
+* Drivers/SDKs for more client-side languages (e.g. JavaScript, Ruby, Java)
 * ORM to better-decouple BigchainDB from its data store (will make it easy to try other databases)
 * Code audits including security audits
 
 ## Other/Future
-* Multisig
+* Byzantine fault tolerance
 * Better support for smart contract frameworks

bigchaindb/__init__.py

@@ -39,7 +39,8 @@ config = {
         'host': e('BIGCHAIN_STATSD_HOST', default='localhost'),
         'port': e('BIGCHAIN_STATSD_PORT', default=8125),
         'rate': e('BIGCHAIN_STATSD_SAMPLERATE', default=0.01)
-    }
+    },
+    'api_endpoint': 'http://localhost:8008/api/v1'
 }
 
 # We need to maintain a backup copy of the original config dict in case

bigchaindb/client.py

@@ -0,0 +1,97 @@
+import requests
+
+import bigchaindb
+from bigchaindb import util
+from bigchaindb import config_utils
+from bigchaindb import exceptions
+from bigchaindb import crypto
+
+
+class Client:
+    """Client for BigchainDB.
+
+    A Client is initialized with a keypair and is able to create, sign, and submit transactions to a Node
+    in the Federation. At the moment, a Client instance is bounded to a specific ``host`` in the Federation.
+    In the future, a Client might connect to >1 hosts.
+    """
+
+    def __init__(self, public_key=None, private_key=None, api_endpoint=None):
+        """Initialize the Client instance
+
+        There are three ways in which the Client instance can get its parameters.
+        The order by which the parameters are chosen are:
+
+            1. Setting them by passing them to the `__init__` method itself.
+            2. Setting them as environment variables
+            3. Reading them from the `config.json` file.
+
+        Args:
+            public_key (str): the base58 encoded public key for the ECDSA secp256k1 curve.
+            private_key (str): the base58 encoded private key for the ECDSA secp256k1 curve.
+            host (str): hostname where the rethinkdb is running.
+            port (int): port in which rethinkb is running (usually 28015).
+        """
+
+        config_utils.autoconfigure()
+
+        self.public_key = public_key or bigchaindb.config['keypair']['public']
+        self.private_key = private_key or bigchaindb.config['keypair']['private']
+        self.api_endpoint = api_endpoint or bigchaindb.config['api_endpoint']
+
+        if not self.public_key or not self.private_key:
+            raise exceptions.KeypairNotFoundException()
+
+    def create(self, payload=None):
+        """Issue a transaction to create an asset.
+
+        Args:
+            payload (dict): the payload for the transaction.
+
+        Return:
+            The transaction pushed to the Federation.
+        """
+
+        tx = util.create_tx(self.public_key, self.public_key, None, operation='CREATE', payload=payload)
+        signed_tx = util.sign_tx(tx, self.private_key)
+        return self._push(signed_tx)
+
+    def transfer(self, new_owner, tx_input, payload=None):
+        """Issue a transaction to transfer an asset.
+
+        Args:
+            new_owner (str): the public key of the new owner
+            tx_input (str): the id of the transaction to use as input
+            payload (dict, optional): the payload for the transaction.
+
+        Return:
+            The transaction pushed to the Federation.
+        """
+
+        tx = util.create_tx(self.public_key, new_owner, tx_input, operation='TRANSFER', payload=payload)
+        signed_tx = util.sign_tx(tx, self.private_key)
+        return self._push(signed_tx)
+
+    def _push(self, tx):
+        """Submit a transaction to the Federation.
+
+        Args:
+            tx (dict): the transaction to be pushed to the Federation.
+
+        Return:
+            The transaction pushed to the Federation.
+        """
+
+        res = requests.post(self.api_endpoint + '/transactions/', json=tx)
+        return res.json()
+
+
+def temp_client():
+    """Create a new temporary client.
+
+    Return:
+        A client initialized with a keypair generated on the fly.
+    """
+
+    private_key, public_key = crypto.generate_key_pair()
+    return Client(private_key=private_key, public_key=public_key, api_endpoint='http://localhost:5000/api/v1')
+

bigchaindb/core.py

@@ -1,17 +1,17 @@
 import rethinkdb as r
-import time
 import random
 import json
 import rapidjson
 
-from datetime import datetime
 
 import bigchaindb
+from bigchaindb import util
 from bigchaindb import config_utils
 from bigchaindb import exceptions
-from bigchaindb.crypto import hash_data, PublicKey, PrivateKey, generate_key_pair
+from bigchaindb import crypto
 from bigchaindb.monitor import Monitor
 
+
 monitor = Monitor()
 
 
@@ -19,10 +19,6 @@ class GenesisBlockAlreadyExistsError(Exception):
     pass
 
 
-class KeypairNotFoundException(Exception):
-    pass
-
-
 class Bigchain(object):
     """Bigchain API
 
@@ -47,8 +43,8 @@ class Bigchain(object):
             public_key (str): the base58 encoded public key for the ECDSA secp256k1 curve.
             private_key (str): the base58 encoded private key for the ECDSA secp256k1 curve.
             keyring (list[str]): list of base58 encoded public keys of the federation nodes.
-
         """
+
         config_utils.autoconfigure()
         self.host = host or bigchaindb.config['database']['host']
         self.port = port or bigchaindb.config['database']['port']
@@ -58,7 +54,7 @@ class Bigchain(object):
         self.federation_nodes = keyring or bigchaindb.config['keyring']
 
         if not self.me or not self.me_private:
-            raise KeypairNotFoundException()
+            raise exceptions.KeypairNotFoundException()
 
         self._conn = None
 
@@ -75,6 +71,7 @@ class Bigchain(object):
     def create_transaction(self, current_owners, new_owners, tx_input, operation, payload=None):
         """Create a new transaction
 
+        Refer to the documentation of ``bigchaindb.util.create_tx``
         A transaction in the bigchain is a transfer of a digital asset between two entities represented
         by public keys.
 
@@ -138,10 +135,12 @@ class Bigchain(object):
         }
 
         return transaction
+        return util.create_tx(current_owner, new_owner, tx_input, operation, payload)
 
     def sign_transaction(self, transaction, private_key, public_key=None):
         """Sign a transaction
 
+        Refer to the documentation of ``bigchaindb.util.sign_tx``
         A transaction signed with the `current_owner` corresponding private key.
 
         Args:
@@ -153,6 +152,8 @@ class Bigchain(object):
             dict: transaction with the `signature` field included.
 
         """
+
+        # return util.sign_tx(transaction, private_key)
         private_key = PrivateKey(private_key)
         if len(transaction['transaction']['current_owners']) == 1:
             signatures_updated = private_key.sign(self.serialize(transaction))
@@ -173,24 +174,16 @@ class Bigchain(object):
         return signed_transaction
 
     def verify_signature(self, signed_transaction):
-        """Verify the signature of a transaction
+        """Verify the signature of a transaction.
-
-        A valid transaction should have been signed `current_owner` corresponding private key.
-
-        Args:
-            signed_transaction (dict): a transaction with the `signature` included.
-
-        Returns:
-            bool: True if the signature is correct, False otherwise.
 
+        Refer to the documentation of ``bigchaindb.crypto.verify_signature``
         """
+
         data = signed_transaction.copy()
 
         # if assignee field in the transaction, remove it
         if 'assignee' in data:
             data.pop('assignee')
-        if 'signatures' not in data:
-            return False
 
         signatures = data.pop('signatures')
         for public_key_base58 in signed_transaction['transaction']['current_owners']:
@@ -211,7 +204,7 @@ class Bigchain(object):
         return True
 
     @monitor.timer('write_transaction', rate=bigchaindb.config['statsd']['rate'])
-    def write_transaction(self, signed_transaction):
+    def write_transaction(self, signed_transaction, durability='soft'):
         """Write the transaction to bigchain.
 
         When first writing a transaction to the bigchain the transaction will be kept in a backlog until
@@ -237,7 +230,7 @@ class Bigchain(object):
         signed_transaction.update({'assignee': assignee})
 
         # write to the backlog
-        response = r.table('backlog').insert(signed_transaction, durability='soft').run(self.conn)
+        response = r.table('backlog').insert(signed_transaction, durability=durability).run(self.conn)
         return response
 
     # TODO: the same `txid` can be in two different blocks
@@ -253,8 +246,8 @@ class Bigchain(object):
             A dict with the transaction details if the transaction was found.
 
             If no transaction with that `txid` was found it returns `None`
-
         """
+
         response = r.table('bigchain').concat_map(lambda doc: doc['block']['transactions'])\
             .filter(lambda transaction: transaction['id'] == txid).run(self.conn)
 
@@ -284,8 +277,8 @@ class Bigchain(object):
         Returns:
             A list of transactions containing that payload. If no transaction exists with that payload it
             returns `None`
-
         """
+
         cursor = r.table('bigchain')\
             .get_all(payload_hash, index='payload_hash')\
             .run(self.conn)
@@ -304,7 +297,6 @@ class Bigchain(object):
 
         Returns:
             The transaction that used the `txid` as an input if it exists else it returns `None`
-
         """
         # checks if an input was already spent
         # checks if the bigchain has any transaction with input `transaction_id`
@@ -330,8 +322,8 @@ class Bigchain(object):
 
         Returns:
             list: list of `txids` currently owned by `owner`
-
         """
+
         # TODO: fix for multisig. new_owners is a list!
         response = r.table('bigchain')\
                     .concat_map(lambda doc: doc['block']['transactions'])\
@@ -366,6 +358,7 @@ class Bigchain(object):
             InvalidHash: if the hash of the transaction is wrong
             InvalidSignature: if the signature of the transaction is wrong
         """
+
         # If the operation is CREATE the transaction should have no inputs and should be signed by a
         # federation node
         if transaction['transaction']['operation'] == 'CREATE':
@@ -395,15 +388,7 @@ class Bigchain(object):
                     raise exceptions.DoubleSpend('input `{}` was already spent'.format(
                         transaction['transaction']['input']))
 
-        # Check hash of the transaction
+        util.check_hash_and_signature(transaction)
-        calculated_hash = hash_data(self.serialize(transaction['transaction']))
-        if calculated_hash != transaction['id']:
-            raise exceptions.InvalidHash()
-
-        # Check signature
-        if not self.verify_signature(transaction):
-            raise exceptions.InvalidSignature()
-
         return transaction
 
     def is_valid_transaction(self, transaction):
@@ -416,8 +401,8 @@ class Bigchain(object):
 
         Returns:
             bool: `True` if the transaction is valid, `False` otherwise
-
         """
+
         try:
             self.validate_transaction(transaction)
             return transaction
@@ -437,20 +422,20 @@ class Bigchain(object):
 
         Returns:
             dict: created block.
-
         """
+
         # Create the new block
         block = {
-            'timestamp': self.timestamp(),
+            'timestamp': util.timestamp(),
             'transactions': validated_transactions,
             'node_pubkey': self.me,
             'voters': self.federation_nodes + [self.me]
         }
 
         # Calculate the hash of the new block
-        block_data = self.serialize(block)
+        block_data = util.serialize(block)
-        block_hash = hash_data(block_data)
+        block_hash = crypto.hash_data(block_data)
-        block_signature = PrivateKey(self.me_private).sign(block_data)
+        block_signature = crypto.PrivateKey(self.me_private).sign(block_data)
 
         block = {
             'id': block_hash,
@@ -472,11 +457,10 @@ class Bigchain(object):
         Returns:
             The block if the block is valid else it raises and exception
             describing the reason why the block is invalid.
-
         """
 
         # 1. Check if current hash is correct
-        calculated_hash = hash_data(self.serialize(block['block']))
+        calculated_hash = crypto.hash_data(util.serialize(block['block']))
         if calculated_hash != block['id']:
             raise exceptions.InvalidHash()
 
@@ -498,8 +482,8 @@ class Bigchain(object):
 
         Returns:
             bool: `True` if the block is valid, `False` otherwise.
-
         """
+
         try:
             self.validate_block(block)
             return True
@@ -512,8 +496,8 @@ class Bigchain(object):
 
         Args:
             block (dict): block to write to bigchain.
-
         """
+
         block_serialized = rapidjson.dumps(block)
         r.table('bigchain').insert(r.json(block_serialized), durability=durability).run(self.conn)
 
@@ -560,18 +544,18 @@ class Bigchain(object):
             previous_block_id (str): The id of the previous block.
             decision (bool): Whether the block is valid or invalid.
             invalid_reason (Optional[str]): Reason the block is invalid
-
         """
+
         vote = {
             'voting_for_block': block['id'],
             'previous_block': previous_block_id,
             'is_block_valid': decision,
             'invalid_reason': invalid_reason,
-            'timestamp': self.timestamp()
+            'timestamp': util.timestamp()
         }
 
-        vote_data = self.serialize(vote)
+        vote_data = util.serialize(vote)
-        signature = PrivateKey(self.me_private).sign(vote_data)
+        signature = crypto.PrivateKey(self.me_private).sign(vote_data)
 
         vote_signed = {
             'node_pubkey': self.me,
@@ -582,9 +566,8 @@ class Bigchain(object):
         return vote_signed
 
     def write_vote(self, block, vote, block_number):
-        """
+        """Write the vote to the database."""
-        Write the vote to the database
+
-        """
         update = {'votes': r.row['votes'].append(vote)}
 
         # We need to *not* override the existing block_number, if any
@@ -598,9 +581,8 @@ class Bigchain(object):
          .run(self.conn)
 
     def get_last_voted_block(self):
-        """
+        """Returns the last block that this node voted on."""
-        Returns the last block that this node voted on
+
-        """
         # query bigchain for all blocks this node is a voter but didn't voted on
         last_voted = r.table('bigchain')\
             .filter(r.row['block']['voters'].contains(self.me))\
@@ -619,9 +601,7 @@ class Bigchain(object):
         return last_voted[0]
 
     def get_unvoted_blocks(self):
-        """
+        """Return all the blocks that has not been voted by this node."""
-        Return all the blocks that has not been voted by this node.
-        """
 
         unvoted = r.table('bigchain')\
             .filter(lambda doc: doc['votes'].contains(lambda vote: vote['node_pubkey'] == self.me).not_())\
@@ -633,59 +613,3 @@ class Bigchain(object):
 
         return unvoted
 
-    @staticmethod
-    def serialize(data):
-        """Static method used to serialize a dict into a JSON formatted string.
-
-        This method enforces rules like the separator and order of keys. This ensures that all dicts
-        are serialized in the same way.
-
-        This is specially important for hashing data. We need to make sure that everyone serializes their data
-        in the same way so that we do not have hash mismatches for the same structure due to serialization
-        differences.
-
-        Args:
-            data (dict): dict to serialize
-
-        Returns:
-            str: JSON formatted string
-
-        """
-        return json.dumps(data, skipkeys=False, ensure_ascii=False,
-                          separators=(',', ':'), sort_keys=True)
-
-    @staticmethod
-    def deserialize(data):
-        """Static method used to deserialize a JSON formatted string into a dict.
-
-        Args:
-            data (str): JSON formatted string.
-
-        Returns:
-            dict: dict resulting from the serialization of a JSON formatted string.
-
-        """
-        return json.loads(data, encoding="utf-8")
-
-    @staticmethod
-    def timestamp():
-        """Static method to calculate a UTC timestamp with microsecond precision.
-
-        Returns:
-            str: UTC timestamp.
-
-        """
-        dt = datetime.utcnow()
-        return "{0:.6f}".format(time.mktime(dt.timetuple()) + dt.microsecond / 1e6)
-
-    @staticmethod
-    def generate_keys():
-        """Generates a key pair.
-
-        Returns:
-            tuple: `(private_key, public_key)`. ECDSA key pair using the secp256k1 curve encoded
-            in base58.
-
-        """
-        # generates and returns the keys serialized in hex
-        return generate_key_pair()

bigchaindb/crypto.py

@@ -1,9 +1,9 @@
 # Separate all crypto code so that we can easily test several implementations
 
-import hashlib
-import sha3
 import binascii
 import base58
+
+import sha3
 import bitcoin
 
 from cryptography.hazmat.backends import default_backend
@@ -148,4 +148,8 @@ def generate_key_pair():
 
 
 def hash_data(data):
-    return hashlib.sha3_256(data.encode()).hexdigest()
+    """Hash the provided data using SHA3-256"""
+
+    return sha3.sha3_256(data.encode()).hexdigest()
+
+

bigchaindb/exceptions.py

@@ -25,3 +25,7 @@ class DatabaseAlreadyExists(Exception):
 class DatabaseDoesNotExist(Exception):
     """Raised when trying to delete the database but the db is not there"""
 
+class KeypairNotFoundException(Exception):
+    """Raised if operation cannot proceed because the keypair was not given"""
+
+

bigchaindb/processes.py

@@ -6,6 +6,7 @@ import rethinkdb as r
 from bigchaindb import Bigchain
 from bigchaindb.voter import Voter
 from bigchaindb.block import Block
+from bigchaindb.web import server
 
 
 logger = logging.getLogger(__name__)
@@ -80,3 +81,9 @@ class Processes(object):
 
         logger.info('starting voter')
         p_voter.start()
+
+        # start the web api
+        webapi = server.create_app()
+        p_webapi = mp.Process(name='webapi', target=webapi.run, kwargs={'host': 'localhost'})
+        p_webapi.start()
+

bigchaindb/util.py

@@ -1,4 +1,12 @@
+
+import json
+import time
 import multiprocessing as mp
+from datetime import datetime
+
+import bigchaindb
+from bigchaindb import exceptions
+from bigchaindb.crypto import PrivateKey, PublicKey, hash_data
 
 
 class ProcessGroup(object):
@@ -22,3 +30,191 @@ class ProcessGroup(object):
             proc.start()
             self.processes.append(proc)
 
+
+def serialize(data):
+    """Serialize a dict into a JSON formatted string.
+
+    This function enforces rules like the separator and order of keys. This ensures that all dicts
+    are serialized in the same way.
+
+    This is specially important for hashing data. We need to make sure that everyone serializes their data
+    in the same way so that we do not have hash mismatches for the same structure due to serialization
+    differences.
+
+    Args:
+        data (dict): dict to serialize
+
+    Returns:
+        str: JSON formatted string
+
+    """
+    return json.dumps(data, skipkeys=False, ensure_ascii=False,
+                      separators=(',', ':'), sort_keys=True)
+
+
+def deserialize(data):
+    """Deserialize a JSON formatted string into a dict.
+
+    Args:
+        data (str): JSON formatted string.
+
+    Returns:
+        dict: dict resulting from the serialization of a JSON formatted string.
+    """
+
+    return json.loads(data, encoding="utf-8")
+
+
+def timestamp():
+    """Calculate a UTC timestamp with microsecond precision.
+
+    Returns:
+        str: UTC timestamp.
+
+    """
+    dt = datetime.utcnow()
+    return "{0:.6f}".format(time.mktime(dt.timetuple()) + dt.microsecond / 1e6)
+
+
+def create_tx(current_owner, new_owner, tx_input, operation, payload=None):
+    """Create a new transaction
+
+    A transaction in the bigchain is a transfer of a digital asset between two entities represented
+    by public keys.
+
+    Currently the bigchain supports two types of operations:
+
+        `CREATE` - Only federation nodes are allowed to use this operation. In a create operation
+        a federation node creates a digital asset in the bigchain and assigns that asset to a public
+        key. The owner of the private key can then decided to transfer this digital asset by using the
+        `transaction id` of the transaction as an input in a `TRANSFER` transaction.
+
+        `TRANSFER` - A transfer operation allows for a transfer of the digital assets between entities.
+
+    Args:
+        current_owner (str): base58 encoded public key of the current owner of the asset.
+        new_owner (str): base58 encoded public key of the new owner of the digital asset.
+        tx_input (str): id of the transaction to use as input.
+        operation (str): Either `CREATE` or `TRANSFER` operation.
+        payload (Optional[dict]): dictionary with information about asset.
+
+    Returns:
+        dict: unsigned transaction.
+
+
+    Raises:
+        TypeError: if the optional ``payload`` argument is not a ``dict``.
+    """
+
+    data = None
+    if payload is not None:
+        if isinstance(payload, dict):
+            hash_payload = hash_data(serialize(payload))
+            data = {
+                'hash': hash_payload,
+                'payload': payload
+            }
+        else:
+            raise TypeError('`payload` must be an dict instance')
+
+    hash_payload = hash_data(serialize(payload))
+    data = {
+        'hash': hash_payload,
+        'payload': payload
+    }
+
+    tx = {
+        'current_owner': current_owner,
+        'new_owner': new_owner,
+        'input': tx_input,
+        'operation': operation,
+        'timestamp': timestamp(),
+        'data': data
+    }
+
+    # serialize and convert to bytes
+    tx_serialized = serialize(tx)
+    tx_hash = hash_data(tx_serialized)
+
+    # create the transaction
+    transaction = {
+        'id': tx_hash,
+        'transaction': tx
+    }
+
+    return transaction
+
+
+def sign_tx(transaction, private_key):
+    """Sign a transaction
+
+    A transaction signed with the `current_owner` corresponding private key.
+
+    Args:
+        transaction (dict): transaction to sign.
+        private_key (str): base58 encoded private key to create a signature of the transaction.
+
+    Returns:
+        dict: transaction with the `signature` field included.
+
+    """
+    private_key = PrivateKey(private_key)
+    signature = private_key.sign(serialize(transaction))
+    signed_transaction = transaction.copy()
+    signed_transaction.update({'signature': signature})
+    return signed_transaction
+
+
+def create_and_sign_tx(private_key, current_owner, new_owner, tx_input, operation='TRANSFER', payload=None):
+    tx = create_tx(current_owner, new_owner, tx_input, operation, payload)
+    return sign_tx(tx, private_key)
+
+
+def check_hash_and_signature(transaction):
+    # Check hash of the transaction
+    calculated_hash = hash_data(serialize(transaction['transaction']))
+    if calculated_hash != transaction['id']:
+        raise exceptions.InvalidHash()
+
+    # Check signature
+    if not verify_signature(transaction):
+        raise exceptions.InvalidSignature()
+
+
+def verify_signature(signed_transaction):
+    """Verify the signature of a transaction
+
+    A valid transaction should have been signed `current_owner` corresponding private key.
+
+    Args:
+        signed_transaction (dict): a transaction with the `signature` included.
+
+    Returns:
+        bool: True if the signature is correct, False otherwise.
+    """
+
+    data = signed_transaction.copy()
+
+    # if assignee field in the transaction, remove it
+    if 'assignee' in data:
+        data.pop('assignee')
+
+    signature = data.pop('signature')
+    public_key_base58 = signed_transaction['transaction']['current_owner']
+    public_key = PublicKey(public_key_base58)
+    return public_key.verify(serialize(data), signature)
+
+
+def transform_create(tx):
+    """Change the owner and signature for a ``CREATE`` transaction created by a node"""
+
+    # XXX: the next instruction opens a new connection to the DB, consider using a singleton or a global
+    #      if you need a Bigchain instance.
+    b = bigchaindb.Bigchain()
+    transaction = tx['transaction']
+    payload = None
+    if transaction['data'] and 'payload' in transaction['data']:
+        payload = transaction['data']['payload']
+    new_tx = create_tx(b.me, transaction['current_owner'], None, 'CREATE', payload=payload)
+    return new_tx
+

bigchaindb/web/server.py

@@ -0,0 +1,21 @@
+"""This module contains basic functions to instantiate the BigchainDB API. """
+
+from flask import Flask
+
+from bigchaindb import Bigchain
+from bigchaindb.web import views
+
+
+def create_app(debug=False):
+    """Return an instance of the Flask application.
+
+    Args:
+        debug (bool): a flag to activate the debug mode for the app (default: False).
+    """
+
+    app = Flask(__name__)
+    app.debug = debug
+    app.config['bigchain'] = Bigchain()
+    app.register_blueprint(views.basic_views, url_prefix='/api/v1')
+    return app
+

bigchaindb/web/views.py

@@ -0,0 +1,68 @@
+"""This module provides the blueprint for some basic API endpoints.
+
+For more information please refer to the documentation in Apiary:
+ - http://docs.bigchaindb.apiary.io/
+"""
+
+import flask
+from flask import current_app, request, Blueprint
+
+from bigchaindb import util
+
+
+basic_views = Blueprint('basic_views', __name__)
+
+
+@basic_views.record
+def get_bigchain(state):
+    bigchain = state.app.config.get('bigchain')
+
+    if bigchain is None:
+        raise Exception('This blueprint expects you to provide '
+                        'database access through `bigchain`')
+
+
+
+@basic_views.route('/transactions/<tx_id>')
+def get_transaction(tx_id):
+    """API endpoint to get details about a transaction.
+
+    Args:
+        tx_id (str): the id of the transaction.
+
+    Return:
+        A JSON string containing the data about the transaction.
+    """
+
+    bigchain = current_app.config['bigchain']
+
+    tx = bigchain.get_transaction(tx_id)
+    return flask.jsonify(**tx)
+
+
+@basic_views.route('/transactions/', methods=['POST'])
+def create_transaction():
+    """API endpoint to push transactions to the Federation.
+
+    Return:
+        A JSON string containing the data about the transaction.
+    """
+    bigchain = current_app.config['bigchain']
+
+    val = {}
+
+    # `force` will try to format the body of the POST request even if the `content-type` header is not
+    # set to `application/json`
+    tx = request.get_json(force=True)
+
+    if tx['transaction']['operation'] == 'CREATE':
+        tx = util.transform_create(tx)
+        tx = util.sign_tx(tx, bigchain.me_private)
+
+    if not util.verify_signature(tx):
+        val['error'] = 'Invalid transaction signature'
+
+    val = bigchain.write_transaction(tx)
+
+    return flask.jsonify(**tx)
+

docs/source/_templates/license-template.html

@@ -1,4 +0,0 @@
-<h3>License</h3>
-<p>
-This documentation is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.
-</p>

docs/source/_templates/sidebar-links-template.html

@@ -1,6 +0,0 @@
-<h3>Quick links</h3>
-<p>
-<a href="https://github.com/bigchaindb/bigchaindb">GitHub repository</a>
-<br>
-<a href="https://www.bigchaindb.com/">BigchainDB website</a>
-</p>

docs/source/_templates/sidebar-title-template.html

@@ -1 +0,0 @@
-<h1>BigchainDB Documentation</h1>

docs/source/conf.py

@@ -168,13 +168,10 @@ html_static_path = ['_static']
 #html_use_smartypants = True
 
 # Custom sidebar templates, maps document names to template names.
-html_sidebars = {
+#html_sidebars = {
-    '**': ['sidebar-title-template.html',
+#    '**': ['globaltoc.html',
-           'globaltoc.html',
+#           'searchbox.html'],
-           'sidebar-links-template.html',
+#}
-           'searchbox.html',
-           'license-template.html'],
-}
 
 # Additional templates that should be rendered to pages, maps page names to
 # template names.

docs/source/index.rst

@@ -16,12 +16,14 @@ Table of Contents
    installing
    getting-started
    bigchaindb-cli
+   python-api-tutorial
    admin
    cryptography
    models
    json-serialization
    developer-interface
    monitoring
+   licenses
    contributing
    faq
    release-notes

docs/source/licenses.md

@@ -0,0 +1,3 @@
+# Licenses
+
+Information about how the BigchainDB code and documentation are licensed can be found in [the LICENSES.md file](https://github.com/bigchaindb/bigchaindb/blob/develop/LICENSES.md) (in the root directory of the repository).

docs/source/python-api-tutorial.md

@@ -0,0 +1,53 @@
+# Getting started with the HTTP API
+
+The preferred way to communicate with a Node in the BigchainDB Federation is via HTTP requests.
+Each Node exposes a simple HTTP API that provides, right now, two endpoints, one to get information about a specific
+transaction id, one to push transactions to the BigchainDB network.
+
+The endpoints are documented in [Apiary](http://docs.bigchaindb.apiary.io/).
+
+
+## Usage example using the Python client
+
+```python
+In [1]: from bigchaindb.client import temp_client
+In [2]: c1 = temp_client()
+In [3]: c2 = temp_client()
+In [4]: tx1 = c1.create()
+In [5]: tx1
+Out[5]:
+{'assignee': '2Bi5NUv1UL7h3ZGs5AsE6Gr3oPQhE2vGsYCapNYrAU4pr',
+'id': '26f21d8b5f9731cef631733b8cd1da05f87aa59eb2f939277a2fefeb774ae133',
+'signature': '304402201b904f22e9f5a502070244b64822adf28...',
+'transaction': {'current_owner': '2Bi5NUv1UL7h3ZGs5AsE6Gr3oPQhE2vGsYCapNYrAU4pr',
+ 'data': {'hash': 'efbde2c3aee204a69b7696d4b10ff31137fe78e3946306284f806e2dfc68b805',
+  'payload': None},
+ 'input': None,
+ 'new_owner': '247epGEcoX9m6yvR6sEZvYGb1XCpUUWtCNUVKgJGrFWCr',
+ 'operation': 'CREATE',
+ 'timestamp': '1456763521.824126'}}
+In [7]: c1.transfer(c2.public_key, tx1['id'])
+Out[7]:
+{'assignee': '2Bi5NUv1UL7h3ZGs5AsE6Gr3oPQhE2vGsYCapNYrAU4pr',
+'id': '34b62c9fdfd93f5907f35e2495239ae1cb62e9519ff64a8710f3f77a9f040857',
+'signature': '3046022100b2b2432c20310dfcda6a2bab3c893b0cd17e70fe...',
+'transaction': {'current_owner': '247epGEcoX9m6yvR6sEZvYGb1XCpUUWtCNUVKgJGrFWCr',
+ 'data': {'hash': 'efbde2c3aee204a69b7696d4b10ff31137fe78e3946306284f806e2dfc68b805',
+  'payload': None},
+ 'input': '26f21d8b5f9731cef631733b8cd1da05f87aa59eb2f939277a2fefeb774ae133',
+ 'new_owner': 'p5Ci1KJkPHvRBnxqyq36m8GXwkWSuhMiZSg8aB1ZrZgJ',
+ 'operation': 'TRANSFER',
+ 'timestamp': '1456763549.446138'}}
+```
+
+
+# Roadmap
+
+The development of the API is still at the beginning and you can follow it on
+[GitHub](https://github.com/bigchaindb/bigchaindb/issues?q=is%3Aissue+is%3Aopen+label%3Arest-api)
+
+There are several key features still missing like:
+ - validating the structure of the transaction
+ - returns the correct error codes if something goes wrong
+ - add an endpoint to query unspents for a given public key
+

docs/source/release-notes.md

@@ -4,4 +4,6 @@ You can find a list of all BigchainDB releases and release notes on GitHub at:
 
 [https://github.com/bigchaindb/bigchaindb/releases](https://github.com/bigchaindb/bigchaindb/releases)
 
-We also have [a roadmap document in bigchaindb/ROADMAP.md](https://github.com/bigchaindb/bigchaindb/blob/develop/ROADMAP.md).
+The [CHANGELOG.md file](https://github.com/bigchaindb/bigchaindb/blob/develop/CHANGELOG.md) contains much the same information, but it also has notes about what to expect in the _next_ release.
+
+We also have [a roadmap document in ROADMAP.md](https://github.com/bigchaindb/bigchaindb/blob/develop/ROADMAP.md).

setup.py

@@ -15,6 +15,7 @@ tests_require = [
     'pytest',
     'pytest-cov',
     'pytest-xdist',
+    'pytest-flask',
 ]
 
 dev_require = [
@@ -55,7 +56,7 @@ setup(
         'Operating System :: POSIX :: Linux',
     ],
 
-    packages=['bigchaindb', 'bigchaindb.commands', 'bigchaindb.db'],
+    packages=['bigchaindb', 'bigchaindb.commands', 'bigchaindb.db', 'bigchaindb.web'],
 
     entry_points={
         'console_scripts': [
@@ -73,6 +74,8 @@ setup(
         'logstats==0.2.1',
         'base58==0.2.2',
         'bitcoin==1.1.42',
+        'flask==0.10.1',
+        'requests==2.9',
     ],
     setup_requires=['pytest-runner'],
     tests_require=tests_require,

tests/conftest.py

@@ -7,13 +7,14 @@ Tasks:
 """
 
 import os
+import copy
 
 import pytest
 
 
 DB_NAME = 'bigchain_test_{}'.format(os.getpid())
 
-config = {
+CONFIG = {
     'database': {
         'name': DB_NAME
     },
@@ -36,7 +37,7 @@ def restore_config(request, node_config):
 
 @pytest.fixture(scope='module')
 def node_config():
-    return config
+    return copy.deepcopy(CONFIG)
 
 
 @pytest.fixture
@@ -47,3 +48,11 @@ def user_private_key():
 @pytest.fixture
 def user_public_key():
     return USER_PUBLIC_KEY
+
+
+@pytest.fixture
+def b(request, node_config):
+    restore_config(request, node_config)
+    from bigchaindb import Bigchain
+    return Bigchain()
+

tests/db/conftest.py

@@ -9,6 +9,7 @@ Tasks:
 import pytest
 import rethinkdb as r
 
+import bigchaindb
 from bigchaindb import Bigchain
 from bigchaindb.db import get_conn
 
@@ -80,6 +81,23 @@ def cleanup_tables(request, node_config):
 
 
 @pytest.fixture
-def b():
+def inputs(user_public_key, amount=1, b=None):
-    return Bigchain()
+    # 1. create the genesis block
+    b = b or Bigchain()
+    try:
+        b.create_genesis_block()
+    except bigchaindb.core.GenesisBlockAlreadyExistsError:
+        pass
+
+    # 2. create block with transactions for `USER` to spend
+    transactions = []
+    for i in range(amount):
+        tx = b.create_transaction(b.me, user_public_key, None, 'CREATE')
+        tx_signed = b.sign_transaction(tx, b.me_private)
+        transactions.append(tx_signed)
+        b.write_transaction(tx_signed)
+
+    block = b.create_block(transactions)
+    b.write_block(block, durability='hard')
+    return block
 

tests/db/test_bigchain_api.py

@@ -6,38 +6,13 @@ import pytest
 import rethinkdb as r
 
 import bigchaindb
+from bigchaindb import util
 from bigchaindb import exceptions
-from bigchaindb import Bigchain
+from bigchaindb.crypto import PrivateKey, PublicKey, generate_key_pair, hash_data
-from bigchaindb.crypto import hash_data, PrivateKey, PublicKey, generate_key_pair
 from bigchaindb.voter import Voter
 from bigchaindb.block import Block
 
 
-def create_inputs(user_public_key, amount=1, b=None):
-    # 1. create the genesis block
-    b = b or Bigchain()
-    try:
-        b.create_genesis_block()
-    except bigchaindb.core.GenesisBlockAlreadyExistsError:
-        pass
-
-    # 2. create block with transactions for `USER` to spend
-    transactions = []
-    for i in range(amount):
-        tx = b.create_transaction(b.me, user_public_key, None, 'CREATE')
-        tx_signed = b.sign_transaction(tx, b.me_private)
-        transactions.append(tx_signed)
-        b.write_transaction(tx_signed)
-
-    block = b.create_block(transactions)
-    b.write_block(block, durability='hard')
-    return block
-
-
-@pytest.fixture
-def inputs(user_public_key):
-    return create_inputs(user_public_key)
-
 
 @pytest.mark.skipif(reason='Some tests throw a ResourceWarning that might result in some weird '
                            'exceptions while running the tests. The problem seems to *not* '
@@ -81,7 +56,7 @@ class TestBigchainApi(object):
             'operation': 'd',
             'timestamp': tx['transaction']['timestamp'],
             'data': {
-                'hash': hash_data(b.serialize(payload)),
+                'hash': hash_data(util.serialize(payload)),
                 'payload': payload
             }
         }
@@ -89,7 +64,7 @@ class TestBigchainApi(object):
         # assert tx_hash == tx_calculated_hash
 
     def test_transaction_signature(self, b):
-        sk, vk = b.generate_keys()
+        sk, vk = generate_key_pair()
         tx = b.create_transaction(vk, 'b', 'c', 'd')
 
         assert b.verify_signature(tx) is False
@@ -119,7 +94,7 @@ class TestBigchainApi(object):
 
     def test_serializer(self, b):
         tx = b.create_transaction('a', 'b', 'c', 'd')
-        assert b.deserialize(b.serialize(tx)) == tx
+        assert util.deserialize(util.serialize(tx)) == tx
 
     @pytest.mark.usefixtures('inputs')
     def test_write_transaction(self, b, user_public_key, user_private_key):
@@ -147,7 +122,7 @@ class TestBigchainApi(object):
         b.write_block(block, durability='hard')
 
         response = b.get_transaction(tx_signed["id"])
-        assert b.serialize(tx_signed) == b.serialize(response)
+        assert util.serialize(tx_signed) == util.serialize(response)
 
     @pytest.mark.usefixtures('inputs')
     def test_assign_transaction_one_node(self, b, user_public_key, user_private_key):
@@ -162,11 +137,11 @@ class TestBigchainApi(object):
         # check if the assignee is the current node
         assert response['assignee'] == b.me
 
+    @pytest.mark.usefixtures('inputs')
     def test_assign_transaction_multiple_nodes(self, b, user_public_key, user_private_key):
         # create 5 federation nodes
         for _ in range(5):
-            b.federation_nodes.append(b.generate_keys()[1])
+            b.federation_nodes.append(generate_key_pair()[1])
-        create_inputs(user_public_key, amount=20, b=b)
 
         # test assignee for several transactions
         for _ in range(20):
@@ -243,11 +218,11 @@ class TestBigchainApi(object):
 
     def test_create_new_block(self, b):
         new_block = b.create_block([])
-        block_hash = hash_data(b.serialize(new_block['block']))
+        block_hash = hash_data(util.serialize(new_block['block']))
 
         assert new_block['block']['voters'] == [b.me]
         assert new_block['block']['node_pubkey'] == b.me
-        assert PublicKey(b.me).verify(b.serialize(new_block['block']), new_block['signature']) is True
+        assert PublicKey(b.me).verify(util.serialize(new_block['block']), new_block['signature']) is True
         assert new_block['id'] == block_hash
         assert new_block['votes'] == []
 
@@ -422,13 +397,13 @@ class TestBlockValidation(object):
 
         # create a block with invalid transactions
         block = {
-            'timestamp': b.timestamp(),
+            'timestamp': util.timestamp(),
             'transactions': [tx_invalid],
             'node_pubkey': b.me,
             'voters': b.federation_nodes
         }
 
-        block_data = b.serialize(block)
+        block_data = util.serialize(block)
         block_hash = hash_data(block_data)
         block_signature = PrivateKey(b.me_private).sign(block_data)
 
@@ -541,7 +516,7 @@ class TestBigchainVoter(object):
         assert vote['vote']['is_block_valid'] is True
         assert vote['vote']['invalid_reason'] is None
         assert vote['node_pubkey'] == b.me
-        assert PublicKey(b.me).verify(b.serialize(vote['vote']), vote['signature']) is True
+        assert PublicKey(b.me).verify(util.serialize(vote['vote']), vote['signature']) is True
 
     def test_invalid_block_voting(self, b, user_public_key):
         # create queue and voter
@@ -582,7 +557,7 @@ class TestBigchainVoter(object):
         assert vote['vote']['is_block_valid'] is False
         assert vote['vote']['invalid_reason'] is None
         assert vote['node_pubkey'] == b.me
-        assert PublicKey(b.me).verify(b.serialize(vote['vote']), vote['signature']) is True
+        assert PublicKey(b.me).verify(util.serialize(vote['vote']), vote['signature']) is True
 
     def test_vote_creation_valid(self, b):
         # create valid block
@@ -596,7 +571,7 @@ class TestBigchainVoter(object):
         assert vote['vote']['is_block_valid'] is True
         assert vote['vote']['invalid_reason'] is None
         assert vote['node_pubkey'] == b.me
-        assert PublicKey(b.me).verify(b.serialize(vote['vote']), vote['signature']) is True
+        assert PublicKey(b.me).verify(util.serialize(vote['vote']), vote['signature']) is True
 
     def test_vote_creation_invalid(self, b):
         # create valid block
@@ -610,7 +585,7 @@ class TestBigchainVoter(object):
         assert vote['vote']['is_block_valid'] is False
         assert vote['vote']['invalid_reason'] is None
         assert vote['node_pubkey'] == b.me
-        assert PublicKey(b.me).verify(b.serialize(vote['vote']), vote['signature']) is True
+        assert PublicKey(b.me).verify(util.serialize(vote['vote']), vote['signature']) is True
 
 
 class TestBigchainBlock(object):

tests/db/test_utils.py

@@ -4,6 +4,7 @@ import pytest
 import rethinkdb as r
 
 import bigchaindb
+from bigchaindb import util
 from bigchaindb.db import utils
 from .conftest import setup_database as _setup_database
 

tests/db/test_voter.py

@@ -3,8 +3,10 @@ import time
 import rethinkdb as r
 import multiprocessing as mp
 
+from bigchaindb import util
+
 from bigchaindb.voter import Voter, BlockStream
-from bigchaindb.crypto import PublicKey
+from bigchaindb.crypto import PublicKey, generate_key_pair
 
 
 class TestBigchainVoter(object):
@@ -43,7 +45,7 @@ class TestBigchainVoter(object):
         assert vote['vote']['is_block_valid'] is True
         assert vote['vote']['invalid_reason'] is None
         assert vote['node_pubkey'] == b.me
-        assert PublicKey(b.me).verify(b.serialize(vote['vote']), vote['signature']) is True
+        assert PublicKey(b.me).verify(util.serialize(vote['vote']), vote['signature']) is True
 
     def test_valid_block_voting_with_create_transaction(self, b):
         q_new_block = mp.Queue()
@@ -51,7 +53,7 @@ class TestBigchainVoter(object):
         genesis = b.create_genesis_block()
 
         # create a `CREATE` transaction
-        test_user_priv, test_user_pub = b.generate_keys()
+        test_user_priv, test_user_pub = generate_key_pair()
         tx = b.create_transaction(b.me, test_user_pub, None, 'CREATE')
         tx_signed = b.sign_transaction(tx, b.me_private)
         assert b.is_valid_transaction(tx_signed)
@@ -85,7 +87,7 @@ class TestBigchainVoter(object):
         assert vote['vote']['is_block_valid'] is True
         assert vote['vote']['invalid_reason'] is None
         assert vote['node_pubkey'] == b.me
-        assert PublicKey(b.me).verify(b.serialize(vote['vote']), vote['signature']) is True
+        assert PublicKey(b.me).verify(util.serialize(vote['vote']), vote['signature']) is True
 
     def test_valid_block_voting_with_transfer_transactions(self, b):
         q_new_block = mp.Queue()
@@ -93,7 +95,7 @@ class TestBigchainVoter(object):
         b.create_genesis_block()
 
         # create a `CREATE` transaction
-        test_user_priv, test_user_pub = b.generate_keys()
+        test_user_priv, test_user_pub = generate_key_pair()
         tx = b.create_transaction(b.me, test_user_pub, None, 'CREATE')
         tx_signed = b.sign_transaction(tx, b.me_private)
         assert b.is_valid_transaction(tx_signed)
@@ -122,7 +124,7 @@ class TestBigchainVoter(object):
         assert len(blocks[1]['votes']) == 1
 
         # create a `TRANSFER` transaction
-        test_user2_priv, test_user2_pub = b.generate_keys()
+        test_user2_priv, test_user2_pub = generate_key_pair()
         tx2 = b.create_transaction(test_user_pub, test_user2_pub, tx['id'], 'TRANSFER')
         tx2_signed = b.sign_transaction(tx2, test_user_priv)
         assert b.is_valid_transaction(tx2_signed)
@@ -156,7 +158,7 @@ class TestBigchainVoter(object):
         assert vote['vote']['is_block_valid'] is True
         assert vote['vote']['invalid_reason'] is None
         assert vote['node_pubkey'] == b.me
-        assert PublicKey(b.me).verify(b.serialize(vote['vote']), vote['signature']) is True
+        assert PublicKey(b.me).verify(util.serialize(vote['vote']), vote['signature']) is True
 
     def test_invalid_block_voting(self, b, user_public_key):
         # create queue and voter
@@ -195,7 +197,7 @@ class TestBigchainVoter(object):
         assert vote['vote']['is_block_valid'] is False
         assert vote['vote']['invalid_reason'] is None
         assert vote['node_pubkey'] == b.me
-        assert PublicKey(b.me).verify(b.serialize(vote['vote']), vote['signature']) is True
+        assert PublicKey(b.me).verify(util.serialize(vote['vote']), vote['signature']) is True
 
     def test_vote_creation_valid(self, b):
         # create valid block
@@ -209,7 +211,7 @@ class TestBigchainVoter(object):
         assert vote['vote']['is_block_valid'] is True
         assert vote['vote']['invalid_reason'] is None
         assert vote['node_pubkey'] == b.me
-        assert PublicKey(b.me).verify(b.serialize(vote['vote']), vote['signature']) is True
+        assert PublicKey(b.me).verify(util.serialize(vote['vote']), vote['signature']) is True
 
     def test_vote_creation_invalid(self, b):
         # create valid block
@@ -223,7 +225,7 @@ class TestBigchainVoter(object):
         assert vote['vote']['is_block_valid'] is False
         assert vote['vote']['invalid_reason'] is None
         assert vote['node_pubkey'] == b.me
-        assert PublicKey(b.me).verify(b.serialize(vote['vote']), vote['signature']) is True
+        assert PublicKey(b.me).verify(util.serialize(vote['vote']), vote['signature']) is True
 
     def test_voter_considers_unvoted_blocks_when_single_node(self, b):
         # simulate a voter going donw in a single node environment
@@ -299,7 +301,7 @@ class TestBlockStream(object):
 
     def test_if_federation_size_is_greater_than_one_ignore_past_blocks(self, b):
         for _ in range(5):
-            b.federation_nodes.append(b.generate_keys()[1])
+            b.federation_nodes.append(generate_key_pair()[1])
         new_blocks = mp.Queue()
         bs = BlockStream(new_blocks)
         block_1 = b.create_block([])

tests/test_client.py

@@ -0,0 +1,59 @@
+import pytest
+
+
+@pytest.fixture
+def client():
+    from bigchaindb.client import temp_client
+    return temp_client()
+
+
+@pytest.fixture
+def mock_requests_post(monkeypatch):
+    class MockResponse:
+        def __init__(self, json):
+            self._json = json
+
+        def json(self):
+            return self._json
+
+    def mockreturn(*args, **kwargs):
+        return MockResponse(kwargs.get('json'))
+
+    monkeypatch.setattr('requests.post', mockreturn)
+
+
+def test_temp_client_returns_a_temp_client():
+    from bigchaindb.client import temp_client
+    client = temp_client()
+    assert client.public_key
+    assert client.private_key
+
+
+def test_client_can_create_assets(mock_requests_post, client):
+    from bigchaindb import util
+
+    tx = client.create()
+
+    # XXX: `CREATE` operations require the node that receives the transaction to modify the data in
+    #      the transaction itself.
+    #      `current_owner` will be overwritten with the public key of the node in the federation
+    #      that will create the real transaction. `signature` will be overwritten with the new signature.
+    #      Note that this scenario is ignored by this test.
+    assert tx['transaction']['current_owner'] == client.public_key
+    assert tx['transaction']['new_owner'] == client.public_key
+    assert tx['transaction']['input'] == None
+
+    assert util.verify_signature(tx)
+
+
+def test_client_can_transfer_assets(mock_requests_post, client):
+    from bigchaindb import util
+
+    tx = client.transfer('a', 123)
+
+    assert tx['transaction']['current_owner'] == client.public_key
+    assert tx['transaction']['new_owner'] == 'a'
+    assert tx['transaction']['input'] == 123
+
+    assert util.verify_signature(tx)
+

tests/test_commands.py

@@ -87,7 +87,7 @@ def test_bigchain_run_start_assume_yes_create_default_config(monkeypatch, mock_p
         value['return'] = newconfig
 
     monkeypatch.setattr(config_utils, 'write_config', mock_write_config)
-    monkeypatch.setattr(config_utils, 'file_config', lambda x: config_utils.dict_config(value['return']))
+    monkeypatch.setattr(config_utils, 'file_config', lambda x: config_utils.dict_config(expected_config))
     monkeypatch.setattr('os.path.exists', lambda path: False)
 
     args = Namespace(config=None, yes=True)

tests/test_core.py

@@ -1,10 +1,8 @@
-import copy
 import pytest
 
 
 @pytest.fixture
-def config(request):
+def config(request, monkeypatch):
-    import bigchaindb
     config = {
         'database': {
             'host': 'host',
@@ -18,13 +16,10 @@ def config(request):
         'keyring': [],
         'CONFIGURED': True,
     }
-    bigchaindb.config.update(config)
 
-    def fin():
+    monkeypatch.setattr('bigchaindb.config', config)
-        bigchaindb.config = bigchaindb._config
+
-        bigchaindb._config = copy.deepcopy(bigchaindb._config)
+    return config
-    request.addfinalizer(fin)
-    return bigchaindb.config
 
 
 def test_bigchain_class_default_initialization(config):

tests/test_util.py

@@ -0,0 +1,12 @@
+from bigchaindb import util
+
+
+def test_transform_create(b, user_private_key, user_public_key):
+    tx = util.create_tx(user_public_key, user_public_key, None, 'CREATE')
+    tx = util.transform_create(tx)
+    tx = util.sign_tx(tx, b.me_private)
+
+    assert tx['transaction']['current_owner'] == b.me
+    assert tx['transaction']['new_owner'] == user_public_key
+    assert util.verify_signature(tx)
+

tests/utils/test_config_utils.py

@@ -3,14 +3,15 @@ import copy
 import pytest
 
 import bigchaindb
+from bigchaindb import exceptions
 
 
 ORIGINAL_CONFIG = copy.deepcopy(bigchaindb._config)
 
 
 @pytest.fixture(scope='function', autouse=True)
-def clean_config():
+def clean_config(monkeypatch):
-    bigchaindb.config = copy.deepcopy(ORIGINAL_CONFIG)
+    monkeypatch.setattr('bigchaindb.config', copy.deepcopy(ORIGINAL_CONFIG))
 
 
 def test_bigchain_instance_is_initialized_when_conf_provided():
@@ -34,5 +35,5 @@ def test_bigchain_instance_raises_when_not_configured(monkeypatch):
     # from existing configurations
     monkeypatch.setattr(config_utils, 'autoconfigure', lambda: 0)
 
-    with pytest.raises(bigchaindb.core.KeypairNotFoundException):
+    with pytest.raises(exceptions.KeypairNotFoundException):
         bigchaindb.Bigchain()

tests/web/conftest.py

@@ -0,0 +1,35 @@
+import pytest
+from ..db import conftest
+
+
+@pytest.fixture(autouse=True)
+def restore_config(request, node_config):
+    from bigchaindb import config_utils
+    config_utils.dict_config(node_config)
+
+
+@pytest.fixture(scope='module', autouse=True)
+def setup_database(request, node_config):
+    conftest.setup_database(request, node_config)
+
+
+@pytest.fixture(scope='function', autouse=True)
+def cleanup_tables(request, node_config):
+    conftest.cleanup_tables(request, node_config)
+
+
+@pytest.fixture
+def app(request, node_config):
+    # XXX: For whatever reason this fixture runs before `restore_config`,
+    #      so we need to manually call it.
+    restore_config(request, node_config)
+
+    from bigchaindb.web import server
+    app = server.create_app(debug=True)
+    return app
+
+
+@pytest.fixture
+def inputs(user_public_key):
+    conftest.inputs(user_public_key)
+

tests/web/test_basic_views.py

@@ -0,0 +1,42 @@
+import json
+
+import pytest
+from bigchaindb import crypto
+from bigchaindb import util
+
+
+TX_ENDPOINT = '/api/v1/transactions/'
+
+
+@pytest.mark.usefixtures('inputs')
+def test_get_transaction_endpoint(b, client, user_public_key):
+    input_tx = b.get_owned_ids(user_public_key).pop()
+    tx = b.get_transaction(input_tx)
+    res = client.get(TX_ENDPOINT + input_tx)
+    assert tx == res.json
+
+
+def test_post_create_transaction_endpoint(b, client):
+    keypair = crypto.generate_key_pair()
+
+    tx = util.create_and_sign_tx(keypair[0], keypair[1], keypair[1], None, 'CREATE')
+
+    res = client.post(TX_ENDPOINT, data=json.dumps(tx))
+    assert res.json['transaction']['current_owner'] == b.me
+    assert res.json['transaction']['new_owner'] == keypair[1]
+
+
+def test_post_transfer_transaction_endpoint(b, client):
+    from_keypair = crypto.generate_key_pair()
+    to_keypair = crypto.generate_key_pair()
+
+    tx = util.create_and_sign_tx(from_keypair[0], from_keypair[1], from_keypair[1], None, 'CREATE')
+    res = client.post(TX_ENDPOINT, data=json.dumps(tx))
+    tx_id = res.json['id']
+
+    transfer = util.create_and_sign_tx(from_keypair[0], from_keypair[1], to_keypair[1], tx_id)
+    res = client.post(TX_ENDPOINT, data=json.dumps(transfer))
+
+    assert res.json['transaction']['current_owner'] == from_keypair[1]
+    assert res.json['transaction']['new_owner'] == to_keypair[1]
+