1342 Commits

Author SHA1 Message Date
z-bowen
2eb32d93f6 Problem: Election class variable to VOTE_TYPE unnecessary
Solution: Removed the variable and hardcoded everything to use the `Vote` class
2018-09-06 17:16:12 +02:00
z-bowen
fdacca7976 Problem: Multiple small issues with style etc.
Solution: Addressed comments from PR
2018-09-05 17:24:01 +02:00
z-bowen
b6a5150da9 Merge branch 'master' into abstract_election_class 2018-09-05 10:52:23 +02:00
z-bowen
0c64fbfb5d Problem: Election class variable ELECTION_TYPE is overly specific
Solution: Renamed `ELECTION_TYPE` to `OPERATION`
2018-09-05 10:41:05 +02:00
z-bowen
331b52cc4c Problem: Schema still refers to ValidatorElectionVote instead of Vote
Solution:  Renamed `TX_SCHEMA_VALIDATOR_ELECTION_VOTE` as `TX_SCHEMA_VOTE`
2018-09-05 10:36:44 +02:00
z-bowen
7d756bf1a9 Problem: Hardcoded reference to the VOTE_TYPE in Election base class
Solution: Pointed the reference to the class variable
2018-09-04 17:23:24 +02:00
z-bowen
30a5b9374d Problem: Error in docstring return signature
Solution: Fixed the docstring
2018-09-04 17:21:48 +02:00
z-bowen
1e3caf8ccd Problem: UpsertValidatorVote can be generalized to just be Vote
Solution: Renamed, refactored and moved the `Vote` class to tie in with the more general `Election` base class
2018-09-04 17:13:34 +02:00
z-bowen
84a7cf7657 Problem: Election class needs to be updated to store concluded elections in the elections table
Solution: Updated the class to use the new table
2018-09-04 15:58:35 +02:00
Vanshdeep Singh
cb22557771 Problem: Validation code not optimized (#2490)
Solution: memoize data
2018-09-04 15:30:52 +02:00
z-bowen
fe8eb84cf7 Merge branch 'master' into abstract_election_class 2018-09-04 15:06:03 +02:00
z-bowen
e7dae7db72 Problem: Need a table to store data for all elections
Solution: Created the `elections` table with secondary_index `election_id`
2018-09-04 12:08:16 +02:00
z-bowen
b468504bfc Problem: Missed a method in Bigchain class when updating the naming for an election method
Solution: Finished the refactoring
2018-09-04 11:12:51 +02:00
z-bowen
bd76b31e40 Revert "Problem: No need to store different types of elections in their own tables"
This reverts commit db45374d3c690429d18a25bcc319f8056c016500.
2018-09-04 10:55:23 +02:00
Lev Berman
230a5b2d69 ABCI chain migration conclusion (#2488)
* Problem: No good way to check for val set absence.

Solution: Make get_validator_set/get_validators return None/[] when there are no validators yet.

* Problem: Incompatible ABCI chain upgrades.

Solution: Record known chains and sync through InitChain. Triggering the migration and adjusting other ABCI endpoints will follow.
2018-09-03 15:41:49 +02:00
Lev Berman
fe0a4c494b
Problem: No MongoDB support for election ID index. (#2507)
Solution: Election ID has to be unique but not every validator set record has it. MongoDB does not support partial indexes, does not even allow for multiple Nones. This is a temporary fix since we are introducing an `election` collection to store election IDs in #2498.
2018-09-03 15:09:16 +02:00
Muawia Khan
8e55b11da2 Problem: cyrptoconditions dependency updated because of vulnerability (#2494)
* Problem: cyrptoconditions dependency updated because of vulnerability CVE-2018-10903

* update cc to ~=0.7.2

* Fix test using b58encode

* Fixing some more tests failing because of base58 update
2018-09-03 13:25:58 +02:00
z-bowen
db45374d3c Problem: No need to store different types of elections in their own tables
Solution: Remove `DB_TABLE` property from `Election` class
2018-09-03 12:45:37 +02:00
z-bowen
560a8154b7 Problem: Responding to a suggestion for improved method naming
Solution: Refactored `get_result_by_election_id` to `get_election_result_by_id`
2018-09-03 12:00:06 +02:00
z-bowen
3d67565bda Problem: ValidatorElection and MigrationElection need to inherit from a common Election class
Solution: Factored the common logic out of `ValidatorElection` and moved it to `Election` parent class
2018-08-31 13:56:55 +02:00
Shahbaz Nazir
2d1f670eec
Problem: BigchainDB has un-necessary code to initialize a replica set and check if MongoDB was started with replicaSet (#2491)
Solution: Remove un-necessary code. Deployment of MongoDB with or without replicaSet should be the responsibility of MongoDB admin which can and cannot be a BigchainDB node operator. As far as BigchainDB is concerned replicaset, if provided in bigchaindb configs, should be used to establish connection with MongoDB.
2018-08-31 13:54:32 +02:00
Zachary Bowen
cfc2c5900b Rework upsert-validator show status (#2496)
* Problem: We need to store the `election_id` as part of the `validator_update` so we can efficiently check which election was resposible for the change

Solution: Added the parameter to `store_validator_set` and aligned the tests

* Problem: Logic for `upsert-validator show` is convoluted

Solution: Rewrote the function to be much simpler

* Problem: Need a uniqueness constraint for election_id wrt validator changes

Solution: Added a new key to the db schema
2018-08-31 09:47:47 +02:00
Zachary Bowen
7a0b474d11 Problem: Users want to know upsert-validator election status.
Solution: Introduce the `upsert-validator show` command. Soon to be re-implemented via storing and querying identifiers of concluded elections.
2018-08-30 10:47:37 +02:00
Vanshdeep Singh
3cf368aab7 Problem: No unique indexes on transaction id and block height (#2492)
Solution: Created unique indexes when setting up the collections
2018-08-29 17:45:08 +02:00
vrde
407b771185 Problem: validating create+transfer might crash the system (#2487)
Solution: if a TRANSFER transaction is validated after a CREATE
transaction, the system crashes with `AttributeError: 'NoneType' object
has no attribute 'txid'`.

This happens because querying `get_spent` checks the attributes `txid`
and `output` of `input.fulfills` for every transaction in the current
buffer (`current_transactions`). For a CREATE, `input.fulfills` is None,
so the check would fail.

The solution is to check if `input.fulfills` is defined. If not, then
the current transaction cannot spend any output, so we can safely skip
it.
2018-08-29 14:13:22 +02:00
Vanshdeep Singh
8e97c753eb Problem: upsert-validator 'approve' command does not transfer vote to election public key (#2480)
Solution: Transfer vote to the election public key; generalize implemenation to handle election txn type

* Problem: Upsert valdiator 'new' doesn't accept public in base64 format

Solution: Tendermint stores all keys in base64 format so it would suitable to
abandon base58 encoding in favour of base64 encoding

* Problem: Not test for invalid execution of upsert-validator 'new'

Solution: Write tests to when invalid power or private key path has been supplied

* Problem: Exceptions are not informational when executing upsert-validator

Solution: generate error logs or print statement indicating success
2018-08-28 14:21:59 +02:00
vrde
41a2687b9b Problem: websocket fails with ELECTION transaction (#2482)
Solution: have a more general approach to process transaction types. If
a transaction does *not* contain `asset.id`, then the `id` of the
`asset` is the `id` of the transaction.
2018-08-27 15:00:02 +02:00
codegeschrei
d31ab9fb40 Problem: There is no way to vote for an election (#2458)
* Problem: `run_upsert_validator_approve` was getting the voting power in a convoluted way

Solution: Changed it to get the voting power from the election outputs

* Problem: `run_upsert_validator_approve` casts votes for *all* voters, not just the user calling the command

Solution: Filter vote txs by the users public key

* Problem: Docs needed a more specific description of how to input the path to the private-key file

Solution: Changed the wording a bit
2018-08-24 09:52:00 +02:00
codegeschrei
90f2fdfc23 Problem: several tests are skipped (#2452)
* Problem: several tests are skipped
Solution: activate or remove tests

* Problem: store_transaction is deprecated
Solution: replace it with store_bulk_transaction

* Problem: we don't test the transaction split
Solution: undelete `test_store_transaction`

* Problem: failing tests
Solution: merge master and change deprecated store_transaction
2018-08-22 10:48:43 +02:00
codegeschrei
55a9151c14 Problem: tests in test_bigchain_api are not activated (#2456)
* Problem: tests in test_bigchain_api are not activated
Solution: remove or active tests

* Problem: TransactionNotInValidBlock was removed
Solution: remove TransactionNotInValidBlock from tests
2018-08-22 09:56:44 +02:00
Troy McConaghy
acc60bcce1 Problem: 'backlog' still referenced by some code & comments (#2474)
Solution: Remove all references to 'backlog'.
Also make the tuple of TABLES common to all backend databases.
2018-08-21 10:27:25 +02:00
Vanshdeep Singh
01dba7e883 Problem: Cannot conclude validator election (#2445)
Solution: Gather votes and conclude election when supermajority is achieved
2018-08-20 16:57:32 +02:00
vrde
0142e98dba Problem: Rapid JSON is outdated and slow (#2470)
Solution: Use the last version of Rapid JSON.
2018-08-17 14:08:49 +02:00
Troy McConaghy
abdd23f5a6 Problem: Source files contain no license info (#2455)
* Problem: Source files contain no license info

Solution: Add comments with SPDX license info to source files

* Python 3 files don't need # -*- coding: utf-8 -*-
2018-08-16 12:31:32 +02:00
Zachary Bowen
3092beb995 Reenable test bigchain api - part 1 (#2415)
* Problem: core.py contains an unused class, `Bigchain`

Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.

* Fixed flake8 complaint about too many blank lines

* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...

Sorry in advance!

* Updating from master changed BigchainDB.process_post_response to a private method, so I had to align with that.

* Fixed a couple stale references to bigchaindb.Bigchain in docstrings

* Missed a reference to `Bigchain` in a patch call...

* Problem: Tests in class `TestBigChainAPI` from `tests/db/test_bigchain_api.py` were disabled

Solution: Fixed the tests that still made sense, and deleted the rest.

* Problem: Flake8 style complaints

Solution: Made things prettier

* Problem: Debugging comments were slightly NSFW

Solution: Deleted the debugging comments

* Problem: Stale use of `pytest.mark.genesis` decorator on these tests

Solution: Removed the decorator

* Problem: CriticalDoubleInclusion exception was no longer used anywhere in the code

Solution: Deleted it

* Problem: Unused exception `GenesisBlockAlreadyExistsError`

Solution: Deleted it

* Problem: I removed a test that covered writing to the DB and checking that everything is stored faithfully

Solution: Replaced it and got it working again
2018-08-13 09:54:45 +02:00
Zachary Bowen
3760824261 Create dynamic upsert validator commands (#2446)
* Problem: Need a method to initiate a new upsert-validator election

Solution: Added subcommand `new` to `upsert_validator`

* Problem: Changes to upsert-validator needed to be reflected in the docs

Solution: Wrote a section for `upsert-validator new`
2018-08-09 17:29:21 +02:00
codegeschrei
44be8f572f Problem: Block parameters are not required anymore (#2374)
* Problem: all blocks are valid
Solution: remove the status from transactions

* Problem: changed the return type of get_transaction_filtered
Solution: return tx class instead of dict
2018-08-08 15:08:34 +02:00
Vanshdeep Singh
54b81d3ae8 Problem: Random failure for event handler test (#2423)
Solution: Random failure of test
`tests/test_events.py::test_event_handler_raises_when_called_after_start` seem
to be because of `get_nowait` which returns way to quickly even when the queue
has data. Increasing timeout seems to fix the issue.
2018-08-08 14:23:07 +02:00
Vanshdeep Singh
2e9a9b1121 Problem: Validator set not tracked by BigchainDB (#2436)
* Problem: Validator set not tracked by BigchainDB

Solution: BigchainDB depends on tendermint's RPC API to get the validator set
which is not avaiable during replay so the validators set should be tracked
inside BigchainDB

* Problem: Unclear code and documentation

Solution: Fix decode_validator and docs strings

* Problem: Doc strings missing

Solution: Add doc string for store_validato_set
2018-08-06 11:37:43 +02:00
Vanshdeep Singh
1a74afa9cd Problem: Node operator cannot vote on a ValidatorElection (#2428)
* Problem: Node operator cannot vote on a ValidatorElection

Solution: Implement validator election voting spec

* Problem: Incorrent code comments

Solution: Update comments with correct context

* Problem: Delegated vote not casted back to election

Solution: Update test to cast votes back to election id and assert their validity
2018-08-02 11:49:59 +02:00
Muawia Khan
49bc495cc4 Last pr before 2.0 beta 5 (#2430)
* Last PR before the release of BigchainDB 2.0 Beta 5

* Fix editing error

* Change development status back to 4 - beta

* Rephrase some fixes I

* Rephrase some fixes II
2018-08-01 15:40:05 +02:00
Vanshdeep Singh
f13c9a9d57 Problem: Stateful validation doesn't raise double spend exception (#2422)
* Problem: Stateful validation doesn't raise double spend exception

Solution: Transaction.validate should raise exception DoubleSpend if the given
transaction is already a part of the database

* Problem: Double spend exception message not accurate

Solution: The exception message should state that the double spend is because of
spending the same input more than once in the transaction
2018-07-31 14:06:21 +02:00
Lev Berman
4795a78d49
Last PR before the release of BigchainDB 2.0 Beta 4 (#2420) 2018-07-30 16:23:00 +02:00
Vanshdeep Singh
5adc1cac3e Problem: ReadTheDocs build fails. (#2419)
Solution: Move the schema spec for validator election to common directory so
that the yaml schema files can be found
2018-07-30 15:19:51 +02:00
Vanshdeep Singh
7dcdefc58b Problem: There is no way to add new validators (BEP-21) (#2392).
Solution: Start by integrating the new election spec specifed in BEP-21.
2018-07-27 17:38:24 +02:00
Muawia Khan
c3f5e2a654 Problem: It is cumbersome to manage BigchainDB and Tendermint processes (#2410).
* Introduce a Monit-based setup for managing processes. Add the corresponding section to the network setup guide.
2018-07-27 17:33:55 +02:00
vrde
af2bf61dc8 Problem: websocket might be unresponsive (#2413)
Solution: It might be related on how aiohttp 2.x handles `send_str`.
AFAIK the call `send_str` was not "fully async". In aiohttp 3.x they
[changed it][1]:

> send_str(), send_bytes(), send_json(), ping() and pong() are genuine
async functions now. (#2475)

So this patch adds support for aiohttp 3.x, and uses the `send_str`
coroutine.

[1]: https://github.com/aio-libs/aiohttp/blob/master/HISTORY.rst#deprecations-and-removals-1
2018-07-26 15:30:10 +02:00
vrde
2087f7090f Problem: abruptly closed sockets still in memory (#2408)
Solution: Catch the proper exception and remove socket from memory.
2018-07-26 15:29:53 +02:00
Muawia Khan
0c33b4c211 Problem: Signals do not propagate to child BigchainDB processes (#2395).
Solution: Run child processes as daemons to make the signals propagate. Add a short shutdown description to the setup guide.
2018-07-26 14:42:32 +02:00
Zachary Bowen
2386ca9d71 Refactor tendermint directory to project root (#2401)
* Problem: core.py contains an unused class, `Bigchain`

Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.

* Problem: core.py contains an unused class, `Bigchain`

Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.

* Fixed flake8 complaint about too many blank lines

* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...

Sorry in advance!

* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...

Sorry in advance!

* Updating from master changed BigchainDB.process_post_response to a private method, so I had to align with that.

* Fixed a couple stale references to bigchaindb.Bigchain in docstrings

* Missed a reference to `Bigchain` in a patch call...

* Problem: BigchainDB class should be part of project root

Solution: Removed the /tendermint directory and moved its contents to project root

* Problem: Flake8 complained that imports were not at the top of the file

Solution: Had to play around with the order of imports to avoid cyclic dependencies, but its working and style compliant now

* Problem: Stale reference to /tendermint directory in the index

Solution: Removed the references to /tendermint

* Problem: Flake8 complaining of unused import in __init__.py

Solution: The import is there so I can import App directly from bigchaindb, rather than from bigchaindb.core (which I think improves code readability. I added a # noqa to silence Flake8.

* Problem: Stale references to `bigchaindb.tendermint.BigchainDB` in the rst files cause Sphinx to fail

Solution: Updated the autodoc files to use `bigchaindb.BigchainDB` instead

* Problem: Stale reference to the `tendermint` directory in an @patch in a disabled test

Solution: Updated the @patch for completeness

* Problem: BigchainDB class should be part of project root

Solution: Removed the /tendermint directory and moved its contents to project root

* Problem: Flake8 complained that imports were not at the top of the file

Solution: Had to play around with the order of imports to avoid cyclic dependencies, but its working and style compliant now

* Problem: Stale reference to /tendermint directory in the index

Solution: Removed the references to /tendermint

* Problem: Flake8 complaining of unused import in __init__.py

Solution: The import is there so I can import App directly from bigchaindb, rather than from bigchaindb.core (which I think improves code readability. I added a # noqa to silence Flake8.

* Problem: Stale references to `bigchaindb.tendermint.BigchainDB` in the rst files cause Sphinx to fail

Solution: Updated the autodoc files to use `bigchaindb.BigchainDB` instead

* Problem: Stale reference to the `tendermint` directory in an @patch in a disabled test

Solution: Updated the @patch for completeness
2018-07-25 16:59:25 +02:00