1038 Commits

Author SHA1 Message Date
vrde
4b54e702f8 Problem: Events API is unreliable (#2529)
Solution: Stop getting events from the Tendermint events API. Get the
event right before returning from the COMMIT phase of Tendermint, and
publish it to the events queue.
2018-09-12 16:55:23 +02:00
Lev Berman
35e35ecd57 Problem: New collections are not created in existing DB. (#2520)
Solution: Do not abort the initialisation if a collection exists. Unify the index creation.
2018-09-12 14:39:07 +02:00
Zachary Bowen
8a7650c13a Generalize election management commands (#2515)
* Problem: `ValidatorElection` and `MigrationElection` need to inherit from a common `Election` class

Solution: Factored the common logic out of `ValidatorElection` and moved it to `Election` parent class

* Problem: Adding base58 as a requirement seems to break the build...

Solution: Reverting the changes

* Problem: Responding to a suggestion for improved method naming

Solution: Refactored `get_result_by_election_id` to `get_election_result_by_id`

* Problem: No need to store different types of elections in their own tables

Solution: Remove `DB_TABLE` property from `Election` class

* Revert "Problem: No need to store different types of elections in their own tables"

This reverts commit db45374d3c690429d18a25bcc319f8056c016500.

* Problem: Missed a method in `Bigchain` class when updating the naming for an election method

Solution: Finished the refactoring

* Problem: Need a table to store data for all elections

Solution: Created the `elections` table with secondary_index `election_id`

* Problem: `Election` class needs to be updated to store concluded elections in the `elections` table

Solution: Updated the class to use the new table

* Problem: `UpsertValidatorVote` can be generalized to just be `Vote`

Solution: Renamed, refactored and moved the `Vote` class to tie in with the more general `Election` base class

* Problem: Error in docstring return signature

Solution: Fixed the docstring

* Problem: Hardcoded reference to the `VOTE_TYPE` in `Election` base class

Solution: Pointed the reference to the class variable

* Problem: Schema still refers to `ValidatorElectionVote` instead of `Vote`

Solution:  Renamed `TX_SCHEMA_VALIDATOR_ELECTION_VOTE` as `TX_SCHEMA_VOTE`

* Problem: `Election` class variable `ELECTION_TYPE` is overly specific

Solution: Renamed `ELECTION_TYPE` to `OPERATION`

* Problem: Command line options for `upsert-validator` can be generalized to manage any type of election

Solution: Refactored the CLI to manage generalized elections

* Problem: Default for `show_election` not implemented for `Election` class

Solution: Create a default method that work if all fields in the 'asset[data]' can be displayed without additional formatting

* Problem: Multiple small issues with style etc.

Solution: Addressed comments from PR

* Problem: `Election` class variable to `VOTE_TYPE` unnecessary

Solution: Removed the variable and hardcoded everything to use the `Vote` class

* Problem: Minor style issues with PR

Solution: Addressing comments

* Problem: Changes to format for validator keys broke some tests

Solution: Aligned the tests to reflect the changed key format

* Problem: `election show` command displaying the base56 public key

Solution: Cast any public key to base64

* Problem: `election_parser` help message still refers to upsert-validator

Solution: Updated the help message
2018-09-12 13:41:16 +02:00
Lev Berman
b33e3808a6 Problem: Test fails if Tendermint is not up. (#2530)
Solution: Mark the test with the abci mark since it requires running BigchainDB and Tendermint. Fails since the mark cleanup in #2522.
2018-09-11 16:41:27 +02:00
Zachary Bowen
0fe749d830 Create abstract election class (#2498).
* Problem: `ValidatorElection` and `MigrationElection` need to inherit from a common `Election` class.

Solution: Factored the common logic out of `ValidatorElection` and moved it to `Election` parent class.

* Problem: No need to store different types of elections in their own tables

Solution: Remove `DB_TABLE` property from `Election` class.

Solution: Created the `elections` table with secondary_index `election_id`.

* Problem: `UpsertValidatorVote` can be generalized to just be `Vote`

Solution: Renamed, refactored and moved the `Vote` class to tie in with the more general `Election` base class.

* Problem: `election_id` is not unique if two elections have the same properties.

Solution: Added a random `uuid4` seed to enforce uniqueness.
2018-09-11 15:39:46 +02:00
codegeschrei
dd84d4eb6f Problem: There are unnecessary markers (#2522)
* Problem: we have unused and outdated fixtures
Solution: clean up fixtures and tests accordingly

* Problem: there are still unused fixtures
Solution: remove Merlin keys

* Problem: There are unnecessary markers
Solution: remove the tendermint marker for tests
2018-09-10 09:53:18 +02:00
Vanshdeep Singh
cbfbfa8fc4 Problem: Public key format not standardized (#2508)
Solution: Add 'type' field to public key so that it can be decoded properly
2018-09-06 16:06:36 +02:00
Lev Berman
699494613f Problem: The commit mode behaves incorrectly. (#2510)
* Problem: The commit mode behaves incorrectly.

Solution: Parse the Tendermint response properly. The functionality was disabled in https://github.com/bigchaindb/bigchaindb/pull/2235/files#diff-c6511560546a7dc577e7e647b5bfdaceL68 and was not fixed since then.

* Add a test case for the sync mode.

* Do not strictly expect deliver_tx in the response.

* Fix post_mock in web/test_transactions.py.

* Check for the error field first.
2018-09-06 15:49:00 +02:00
Vanshdeep Singh
2656302c60 Problem: Bigchaindb crashes when restarted after init chain (#2519)
Solution: Return appropirate height so that Tendermint doesn't execute
init_chain more than once
2018-09-06 11:27:52 +02:00
Zachary Bowen
bee9468bd0 Problem: test_make_sure_we_dont_remove_any_command does not check for upsert-validator show (#2499).
Solution: Extend the test.
2018-09-05 10:51:32 +02:00
Vanshdeep Singh
cb22557771 Problem: Validation code not optimized (#2490)
Solution: memoize data
2018-09-04 15:30:52 +02:00
Troy McConaghy
c72c7a4626 Problem: tests/README.md either wrong or redundant (#2479)
The information on that page was either wrong or redundant, so I either deleted it or linked to where to find the same information in the docs about contributing to BigchainDB.
2018-09-04 13:31:00 +02:00
Lev Berman
230a5b2d69 ABCI chain migration conclusion (#2488)
* Problem: No good way to check for val set absence.

Solution: Make get_validator_set/get_validators return None/[] when there are no validators yet.

* Problem: Incompatible ABCI chain upgrades.

Solution: Record known chains and sync through InitChain. Triggering the migration and adjusting other ABCI endpoints will follow.
2018-09-03 15:41:49 +02:00
Lev Berman
fe0a4c494b
Problem: No MongoDB support for election ID index. (#2507)
Solution: Election ID has to be unique but not every validator set record has it. MongoDB does not support partial indexes, does not even allow for multiple Nones. This is a temporary fix since we are introducing an `election` collection to store election IDs in #2498.
2018-09-03 15:09:16 +02:00
Muawia Khan
8e55b11da2 Problem: cyrptoconditions dependency updated because of vulnerability (#2494)
* Problem: cyrptoconditions dependency updated because of vulnerability CVE-2018-10903

* update cc to ~=0.7.2

* Fix test using b58encode

* Fixing some more tests failing because of base58 update
2018-09-03 13:25:58 +02:00
Shahbaz Nazir
2d1f670eec
Problem: BigchainDB has un-necessary code to initialize a replica set and check if MongoDB was started with replicaSet (#2491)
Solution: Remove un-necessary code. Deployment of MongoDB with or without replicaSet should be the responsibility of MongoDB admin which can and cannot be a BigchainDB node operator. As far as BigchainDB is concerned replicaset, if provided in bigchaindb configs, should be used to establish connection with MongoDB.
2018-08-31 13:54:32 +02:00
Zachary Bowen
cfc2c5900b Rework upsert-validator show status (#2496)
* Problem: We need to store the `election_id` as part of the `validator_update` so we can efficiently check which election was resposible for the change

Solution: Added the parameter to `store_validator_set` and aligned the tests

* Problem: Logic for `upsert-validator show` is convoluted

Solution: Rewrote the function to be much simpler

* Problem: Need a uniqueness constraint for election_id wrt validator changes

Solution: Added a new key to the db schema
2018-08-31 09:47:47 +02:00
Zachary Bowen
7a0b474d11 Problem: Users want to know upsert-validator election status.
Solution: Introduce the `upsert-validator show` command. Soon to be re-implemented via storing and querying identifiers of concluded elections.
2018-08-30 10:47:37 +02:00
Vanshdeep Singh
3cf368aab7 Problem: No unique indexes on transaction id and block height (#2492)
Solution: Created unique indexes when setting up the collections
2018-08-29 17:45:08 +02:00
vrde
407b771185 Problem: validating create+transfer might crash the system (#2487)
Solution: if a TRANSFER transaction is validated after a CREATE
transaction, the system crashes with `AttributeError: 'NoneType' object
has no attribute 'txid'`.

This happens because querying `get_spent` checks the attributes `txid`
and `output` of `input.fulfills` for every transaction in the current
buffer (`current_transactions`). For a CREATE, `input.fulfills` is None,
so the check would fail.

The solution is to check if `input.fulfills` is defined. If not, then
the current transaction cannot spend any output, so we can safely skip
it.
2018-08-29 14:13:22 +02:00
Vanshdeep Singh
8e97c753eb Problem: upsert-validator 'approve' command does not transfer vote to election public key (#2480)
Solution: Transfer vote to the election public key; generalize implemenation to handle election txn type

* Problem: Upsert valdiator 'new' doesn't accept public in base64 format

Solution: Tendermint stores all keys in base64 format so it would suitable to
abandon base58 encoding in favour of base64 encoding

* Problem: Not test for invalid execution of upsert-validator 'new'

Solution: Write tests to when invalid power or private key path has been supplied

* Problem: Exceptions are not informational when executing upsert-validator

Solution: generate error logs or print statement indicating success
2018-08-28 14:21:59 +02:00
vrde
41a2687b9b Problem: websocket fails with ELECTION transaction (#2482)
Solution: have a more general approach to process transaction types. If
a transaction does *not* contain `asset.id`, then the `id` of the
`asset` is the `id` of the transaction.
2018-08-27 15:00:02 +02:00
codegeschrei
6fdcaf44a7 Problem: we have unused and outdated fixtures (#2476)
* Problem: we have unused and outdated fixtures
Solution: clean up fixtures and tests accordingly
2018-08-27 09:48:20 +02:00
codegeschrei
d31ab9fb40 Problem: There is no way to vote for an election (#2458)
* Problem: `run_upsert_validator_approve` was getting the voting power in a convoluted way

Solution: Changed it to get the voting power from the election outputs

* Problem: `run_upsert_validator_approve` casts votes for *all* voters, not just the user calling the command

Solution: Filter vote txs by the users public key

* Problem: Docs needed a more specific description of how to input the path to the private-key file

Solution: Changed the wording a bit
2018-08-24 09:52:00 +02:00
codegeschrei
90f2fdfc23 Problem: several tests are skipped (#2452)
* Problem: several tests are skipped
Solution: activate or remove tests

* Problem: store_transaction is deprecated
Solution: replace it with store_bulk_transaction

* Problem: we don't test the transaction split
Solution: undelete `test_store_transaction`

* Problem: failing tests
Solution: merge master and change deprecated store_transaction
2018-08-22 10:48:43 +02:00
codegeschrei
55a9151c14 Problem: tests in test_bigchain_api are not activated (#2456)
* Problem: tests in test_bigchain_api are not activated
Solution: remove or active tests

* Problem: TransactionNotInValidBlock was removed
Solution: remove TransactionNotInValidBlock from tests
2018-08-22 09:56:44 +02:00
Vanshdeep Singh
01dba7e883 Problem: Cannot conclude validator election (#2445)
Solution: Gather votes and conclude election when supermajority is achieved
2018-08-20 16:57:32 +02:00
Troy McConaghy
abdd23f5a6 Problem: Source files contain no license info (#2455)
* Problem: Source files contain no license info

Solution: Add comments with SPDX license info to source files

* Python 3 files don't need # -*- coding: utf-8 -*-
2018-08-16 12:31:32 +02:00
Zachary Bowen
3092beb995 Reenable test bigchain api - part 1 (#2415)
* Problem: core.py contains an unused class, `Bigchain`

Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.

* Fixed flake8 complaint about too many blank lines

* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...

Sorry in advance!

* Updating from master changed BigchainDB.process_post_response to a private method, so I had to align with that.

* Fixed a couple stale references to bigchaindb.Bigchain in docstrings

* Missed a reference to `Bigchain` in a patch call...

* Problem: Tests in class `TestBigChainAPI` from `tests/db/test_bigchain_api.py` were disabled

Solution: Fixed the tests that still made sense, and deleted the rest.

* Problem: Flake8 style complaints

Solution: Made things prettier

* Problem: Debugging comments were slightly NSFW

Solution: Deleted the debugging comments

* Problem: Stale use of `pytest.mark.genesis` decorator on these tests

Solution: Removed the decorator

* Problem: CriticalDoubleInclusion exception was no longer used anywhere in the code

Solution: Deleted it

* Problem: Unused exception `GenesisBlockAlreadyExistsError`

Solution: Deleted it

* Problem: I removed a test that covered writing to the DB and checking that everything is stored faithfully

Solution: Replaced it and got it working again
2018-08-13 09:54:45 +02:00
Zachary Bowen
3760824261 Create dynamic upsert validator commands (#2446)
* Problem: Need a method to initiate a new upsert-validator election

Solution: Added subcommand `new` to `upsert_validator`

* Problem: Changes to upsert-validator needed to be reflected in the docs

Solution: Wrote a section for `upsert-validator new`
2018-08-09 17:29:21 +02:00
codegeschrei
44be8f572f Problem: Block parameters are not required anymore (#2374)
* Problem: all blocks are valid
Solution: remove the status from transactions

* Problem: changed the return type of get_transaction_filtered
Solution: return tx class instead of dict
2018-08-08 15:08:34 +02:00
codegeschrei
bdfa059046 Problem: some tests are not activated (#2390)
* Problem: core.py contains an unused class, `Bigchain`

Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.

* Problem: core.py contains an unused class, `Bigchain`

Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.

* Fixed flake8 complaint about too many blank lines

* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...

Sorry in advance!

* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...

Sorry in advance!

* Updating from master changed BigchainDB.process_post_response to a private method, so I had to align with that.

* Fixed a couple stale references to bigchaindb.Bigchain in docstrings

* Missed a reference to `Bigchain` in a patch call...

* Problem: some tests are not activated
Solution: activate and remove or fix tests

* Problem: accidentally un-skipped test_get_blocks_status_containing_tx during merge

Solution: Replaced the skip
2018-08-08 12:17:44 +02:00
Vanshdeep Singh
2e9a9b1121 Problem: Validator set not tracked by BigchainDB (#2436)
* Problem: Validator set not tracked by BigchainDB

Solution: BigchainDB depends on tendermint's RPC API to get the validator set
which is not avaiable during replay so the validators set should be tracked
inside BigchainDB

* Problem: Unclear code and documentation

Solution: Fix decode_validator and docs strings

* Problem: Doc strings missing

Solution: Add doc string for store_validato_set
2018-08-06 11:37:43 +02:00
Vanshdeep Singh
1a74afa9cd Problem: Node operator cannot vote on a ValidatorElection (#2428)
* Problem: Node operator cannot vote on a ValidatorElection

Solution: Implement validator election voting spec

* Problem: Incorrent code comments

Solution: Update comments with correct context

* Problem: Delegated vote not casted back to election

Solution: Update test to cast votes back to election id and assert their validity
2018-08-02 11:49:59 +02:00
Vanshdeep Singh
533030cdae Problem: No test to validate duplicate inputs (#2425)
Solution: Update tests
2018-07-31 14:33:28 +02:00
Vanshdeep Singh
f13c9a9d57 Problem: Stateful validation doesn't raise double spend exception (#2422)
* Problem: Stateful validation doesn't raise double spend exception

Solution: Transaction.validate should raise exception DoubleSpend if the given
transaction is already a part of the database

* Problem: Double spend exception message not accurate

Solution: The exception message should state that the double spend is because of
spending the same input more than once in the transaction
2018-07-31 14:06:21 +02:00
Vanshdeep Singh
7dcdefc58b Problem: There is no way to add new validators (BEP-21) (#2392).
Solution: Start by integrating the new election spec specifed in BEP-21.
2018-07-27 17:38:24 +02:00
Lev Berman
a0670b6d06
Problem: config utils tests are disabled. (#2402)
Solution: enable them back, cleanup no longer relevant parts.
2018-07-27 17:35:44 +02:00
Lev Berman
2f6413fcd0 Problem: Transaction schema tests were disabled. (#2404)
Solution: Re-enable the tests. Remove some no longer relevant parts.
2018-07-26 16:44:56 +02:00
Zachary Bowen
2386ca9d71 Refactor tendermint directory to project root (#2401)
* Problem: core.py contains an unused class, `Bigchain`

Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.

* Problem: core.py contains an unused class, `Bigchain`

Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.

* Fixed flake8 complaint about too many blank lines

* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...

Sorry in advance!

* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...

Sorry in advance!

* Updating from master changed BigchainDB.process_post_response to a private method, so I had to align with that.

* Fixed a couple stale references to bigchaindb.Bigchain in docstrings

* Missed a reference to `Bigchain` in a patch call...

* Problem: BigchainDB class should be part of project root

Solution: Removed the /tendermint directory and moved its contents to project root

* Problem: Flake8 complained that imports were not at the top of the file

Solution: Had to play around with the order of imports to avoid cyclic dependencies, but its working and style compliant now

* Problem: Stale reference to /tendermint directory in the index

Solution: Removed the references to /tendermint

* Problem: Flake8 complaining of unused import in __init__.py

Solution: The import is there so I can import App directly from bigchaindb, rather than from bigchaindb.core (which I think improves code readability. I added a # noqa to silence Flake8.

* Problem: Stale references to `bigchaindb.tendermint.BigchainDB` in the rst files cause Sphinx to fail

Solution: Updated the autodoc files to use `bigchaindb.BigchainDB` instead

* Problem: Stale reference to the `tendermint` directory in an @patch in a disabled test

Solution: Updated the @patch for completeness

* Problem: BigchainDB class should be part of project root

Solution: Removed the /tendermint directory and moved its contents to project root

* Problem: Flake8 complained that imports were not at the top of the file

Solution: Had to play around with the order of imports to avoid cyclic dependencies, but its working and style compliant now

* Problem: Stale reference to /tendermint directory in the index

Solution: Removed the references to /tendermint

* Problem: Flake8 complaining of unused import in __init__.py

Solution: The import is there so I can import App directly from bigchaindb, rather than from bigchaindb.core (which I think improves code readability. I added a # noqa to silence Flake8.

* Problem: Stale references to `bigchaindb.tendermint.BigchainDB` in the rst files cause Sphinx to fail

Solution: Updated the autodoc files to use `bigchaindb.BigchainDB` instead

* Problem: Stale reference to the `tendermint` directory in an @patch in a disabled test

Solution: Updated the @patch for completeness
2018-07-25 16:59:25 +02:00
codegeschrei
d521a00925 Problem: a valid transaction is detected as double spend (#2389)
Solution: query the wanted data per input
2018-07-18 10:52:33 +02:00
codegeschrei
fdf0283508 Problem: consensus and blocks are deprecated (#2384)
Solution: remove test files for consensus and blocks
2018-07-16 19:21:27 +02:00
Zachary Bowen
4d2e58416c Problem: Bigchain class is unused and redundant (#2366)
* Problem: core.py contains an unused class, `Bigchain`

Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.

* Problem: core.py contains an unused class, `Bigchain`

Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.

* Fixed flake8 complaint about too many blank lines

* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...

Sorry in advance!

* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...

Sorry in advance!

* Updating from master changed BigchainDB.process_post_response to a private method, so I had to align with that.

* Fixed a couple stale references to bigchaindb.Bigchain in docstrings

* Missed a reference to `Bigchain` in a patch call...

* Merge with master and re-do some changes
2018-07-10 17:34:51 +02:00
Vanshdeep Singh
c7503f5689 Problem: Support for latest Tendermint missing (#2375)
Solution: Upgrade to py-abci 0.5.1 to use latest Tendermint
2018-07-10 14:16:02 +02:00
Lev Berman
744ab3d5ef Problem: 2 implementations of fastquery exist. (#2365)
* Problem: 2 implementations of fastquery exist.

Solution: Remove the old deprecated implementation. Update the tests.

* Problem: There are still 3 outdated fastquery tests.

Solution: Fix the tests.
2018-06-29 15:48:58 +02:00
Muawia Khan
1bad851e07
Problem: Tendermint configuration not present in BigchainDB config (#2342)
* Problem: Tendermint configuration not present in BigchainDB config

* Handle tendermint configurations properly

- Update docs

* Nitpick

* Missed some conflicts

* Make changes in tendermint/lib.py instead of deprecated core.py

- Also remove redundant info from docs about default values

* Fix docsserver automethod
2018-06-29 18:45:22 +05:00
codegeschrei
af996650c3 Problem: The Block class is deprecated (#2368)
* Problem: docs are not building anymore
Solution: add wget to the requirements for docs

* Problem: The Block class is deprecated
Solution: remove the block class and all occurrences
2018-06-29 15:32:16 +02:00
codegeschrei
7449b026fa Problem: voting code is not used anymore (#2357)
* Problem: voting code is not used anymore
Solution: remove all voting related code

* Problem: Some voting functionality is still present.

Solution: Remove it. Update some of the related tests.

* Problem: some skipped tests are now running
Solution: remove pytest mark to not run them

* Problem: fastquery is not related to votes
Solution: remove it in another PR
2018-06-28 11:30:13 +02:00
Shahbaz Nazir
2da720e542 Problem: There is no logging for benchmark stats (#2349)
* add log level for benchmarking

Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>

* Bug fixes and code re-factor

Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>

* remove un-necessary imports

Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>

* fix failing tests

Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>

* update docs

Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>

* fix typo

Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>

* fix flake8

Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>

* fix flake8 issues

Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>

* nitpic fixes

Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>
2018-06-14 15:19:26 +02:00
Shahbaz Nazir
bec8d13e36
Problem: Current logging server keeps running in some tear down scenarios (#2304)
Solution: Remove the current pub/sub based logging infra as this model is no longer necessary. This can now be replaced by a simple system level logging config.
2018-06-11 13:08:03 +02:00