110 Commits

Author SHA1 Message Date
Troy McConaghy
20e6c6a686
Update acceptance Dockerfile to use latest Python driver (#2594)
* Update acceptance Dockerfile to use latest Python driver

* Fixed four Flake8 errors
2018-11-03 17:34:36 +01:00
Lev Berman
24ca0b32a9 Separate pending and effective validator updates. (#2556)
* Separate pending and effective validator updates.

- Pending validator updates do not prevent elections from concluding.
- ValidatorElection overrides has_conclude to not conclude when there is a pending update for the matching height.
- Effective validator updates deem past elections inconclusive.

* Problem: Looking for election block is inefficient.

Solution: Record placed elections, update the records upon election conclusion.

* Clarify the conclusion order in Election.process_blocks.

* Insert election records in bulk.

Otherwise, one can significantly slow nodes down by posting a whole bunch of unique elections.

* Change get_election to use find_one.

* Calculate total votes without making extra query.

* Fix the pending valset check.

* Fix election test setup.
2018-09-21 10:51:57 +02:00
Muawia Khan
eb139fba00 Remove benchmark logging (#2565).
* Problem: Benchmark logging is not needed.

* Solution: Clean it up.
2018-09-20 14:30:30 +02:00
Lev Berman
126e90e732 Adjust multiple elections conclusion. (#2553)
- Do not conclude migration election if there is a migration in progress.
- Rewrite election tests to not use mocks and assert many different things.
- Record concluded elections in the `election` collection.
2018-09-18 12:32:16 +02:00
Zachary Bowen
e6893632dc Create migration election class (#2535).
* Problem: We need a way to synchronize a halt to block production to allow for upgrades across breaking changes.

* Solution: Created `MigrationElection`.

* Problem: Need documentation for `migration` elections.

* Solution: Updated the docs.

* Problem: `MigrationElection` needs 'new' CLI method.

* Solution: Updated the definition of `election` to include the new `migration` type.

* Problem: The way `end_block` checks for concluded elections assumes there is only one type of election (so we can't conclude an `upsert-validator` and a `chain-migration` at the same height).

* Solution: Re-engineered the code in `Elections` to conclude multiple elections in the same block. If more than one election change the validator set, only one of them is applied.

* Problem: Tendermint change to store validator changes at height h+2 will break `Election.get_status`.

* Solution: Reworked `get_validator_change` to look at only the latest block height or less.
2018-09-17 13:59:57 +02:00
Muawia Khan
bd39076522 Problem: Users trying to use an incompatible version of Tendermint (#2541)
-  Hard-wire the supported Tendermint version(s) right in the code of BigchainDB Server. Check the version of Tendermint and disconnect if Tendermint version is an unsupported one.
- Expose a CLI command bigchaindb tendermint-version to display the supported Tendermint versions.
- PR also takes care the long list of warnings we get when we run tests.
    - Updated deprecated pymongo methods
    - Do not call pytest fixtures directly.
    - Also added the doc for the new cli command
2018-09-14 15:37:18 +02:00
Lev Berman
35e35ecd57 Problem: New collections are not created in existing DB. (#2520)
Solution: Do not abort the initialisation if a collection exists. Unify the index creation.
2018-09-12 14:39:07 +02:00
Vanshdeep Singh
cbfbfa8fc4 Problem: Public key format not standardized (#2508)
Solution: Add 'type' field to public key so that it can be decoded properly
2018-09-06 16:06:36 +02:00
Vanshdeep Singh
cb22557771 Problem: Validation code not optimized (#2490)
Solution: memoize data
2018-09-04 15:30:52 +02:00
Zachary Bowen
cfc2c5900b Rework upsert-validator show status (#2496)
* Problem: We need to store the `election_id` as part of the `validator_update` so we can efficiently check which election was resposible for the change

Solution: Added the parameter to `store_validator_set` and aligned the tests

* Problem: Logic for `upsert-validator show` is convoluted

Solution: Rewrote the function to be much simpler

* Problem: Need a uniqueness constraint for election_id wrt validator changes

Solution: Added a new key to the db schema
2018-08-31 09:47:47 +02:00
codegeschrei
6fdcaf44a7 Problem: we have unused and outdated fixtures (#2476)
* Problem: we have unused and outdated fixtures
Solution: clean up fixtures and tests accordingly
2018-08-27 09:48:20 +02:00
codegeschrei
d31ab9fb40 Problem: There is no way to vote for an election (#2458)
* Problem: `run_upsert_validator_approve` was getting the voting power in a convoluted way

Solution: Changed it to get the voting power from the election outputs

* Problem: `run_upsert_validator_approve` casts votes for *all* voters, not just the user calling the command

Solution: Filter vote txs by the users public key

* Problem: Docs needed a more specific description of how to input the path to the private-key file

Solution: Changed the wording a bit
2018-08-24 09:52:00 +02:00
Troy McConaghy
abdd23f5a6 Problem: Source files contain no license info (#2455)
* Problem: Source files contain no license info

Solution: Add comments with SPDX license info to source files

* Python 3 files don't need # -*- coding: utf-8 -*-
2018-08-16 12:31:32 +02:00
Zachary Bowen
3092beb995 Reenable test bigchain api - part 1 (#2415)
* Problem: core.py contains an unused class, `Bigchain`

Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.

* Fixed flake8 complaint about too many blank lines

* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...

Sorry in advance!

* Updating from master changed BigchainDB.process_post_response to a private method, so I had to align with that.

* Fixed a couple stale references to bigchaindb.Bigchain in docstrings

* Missed a reference to `Bigchain` in a patch call...

* Problem: Tests in class `TestBigChainAPI` from `tests/db/test_bigchain_api.py` were disabled

Solution: Fixed the tests that still made sense, and deleted the rest.

* Problem: Flake8 style complaints

Solution: Made things prettier

* Problem: Debugging comments were slightly NSFW

Solution: Deleted the debugging comments

* Problem: Stale use of `pytest.mark.genesis` decorator on these tests

Solution: Removed the decorator

* Problem: CriticalDoubleInclusion exception was no longer used anywhere in the code

Solution: Deleted it

* Problem: Unused exception `GenesisBlockAlreadyExistsError`

Solution: Deleted it

* Problem: I removed a test that covered writing to the DB and checking that everything is stored faithfully

Solution: Replaced it and got it working again
2018-08-13 09:54:45 +02:00
Zachary Bowen
3760824261 Create dynamic upsert validator commands (#2446)
* Problem: Need a method to initiate a new upsert-validator election

Solution: Added subcommand `new` to `upsert_validator`

* Problem: Changes to upsert-validator needed to be reflected in the docs

Solution: Wrote a section for `upsert-validator new`
2018-08-09 17:29:21 +02:00
Vanshdeep Singh
1a74afa9cd Problem: Node operator cannot vote on a ValidatorElection (#2428)
* Problem: Node operator cannot vote on a ValidatorElection

Solution: Implement validator election voting spec

* Problem: Incorrent code comments

Solution: Update comments with correct context

* Problem: Delegated vote not casted back to election

Solution: Update test to cast votes back to election id and assert their validity
2018-08-02 11:49:59 +02:00
Vanshdeep Singh
7dcdefc58b Problem: There is no way to add new validators (BEP-21) (#2392).
Solution: Start by integrating the new election spec specifed in BEP-21.
2018-07-27 17:38:24 +02:00
Zachary Bowen
2386ca9d71 Refactor tendermint directory to project root (#2401)
* Problem: core.py contains an unused class, `Bigchain`

Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.

* Problem: core.py contains an unused class, `Bigchain`

Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.

* Fixed flake8 complaint about too many blank lines

* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...

Sorry in advance!

* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...

Sorry in advance!

* Updating from master changed BigchainDB.process_post_response to a private method, so I had to align with that.

* Fixed a couple stale references to bigchaindb.Bigchain in docstrings

* Missed a reference to `Bigchain` in a patch call...

* Problem: BigchainDB class should be part of project root

Solution: Removed the /tendermint directory and moved its contents to project root

* Problem: Flake8 complained that imports were not at the top of the file

Solution: Had to play around with the order of imports to avoid cyclic dependencies, but its working and style compliant now

* Problem: Stale reference to /tendermint directory in the index

Solution: Removed the references to /tendermint

* Problem: Flake8 complaining of unused import in __init__.py

Solution: The import is there so I can import App directly from bigchaindb, rather than from bigchaindb.core (which I think improves code readability. I added a # noqa to silence Flake8.

* Problem: Stale references to `bigchaindb.tendermint.BigchainDB` in the rst files cause Sphinx to fail

Solution: Updated the autodoc files to use `bigchaindb.BigchainDB` instead

* Problem: Stale reference to the `tendermint` directory in an @patch in a disabled test

Solution: Updated the @patch for completeness

* Problem: BigchainDB class should be part of project root

Solution: Removed the /tendermint directory and moved its contents to project root

* Problem: Flake8 complained that imports were not at the top of the file

Solution: Had to play around with the order of imports to avoid cyclic dependencies, but its working and style compliant now

* Problem: Stale reference to /tendermint directory in the index

Solution: Removed the references to /tendermint

* Problem: Flake8 complaining of unused import in __init__.py

Solution: The import is there so I can import App directly from bigchaindb, rather than from bigchaindb.core (which I think improves code readability. I added a # noqa to silence Flake8.

* Problem: Stale references to `bigchaindb.tendermint.BigchainDB` in the rst files cause Sphinx to fail

Solution: Updated the autodoc files to use `bigchaindb.BigchainDB` instead

* Problem: Stale reference to the `tendermint` directory in an @patch in a disabled test

Solution: Updated the @patch for completeness
2018-07-25 16:59:25 +02:00
Vanshdeep Singh
c7503f5689 Problem: Support for latest Tendermint missing (#2375)
Solution: Upgrade to py-abci 0.5.1 to use latest Tendermint
2018-07-10 14:16:02 +02:00
Muawia Khan
1bad851e07
Problem: Tendermint configuration not present in BigchainDB config (#2342)
* Problem: Tendermint configuration not present in BigchainDB config

* Handle tendermint configurations properly

- Update docs

* Nitpick

* Missed some conflicts

* Make changes in tendermint/lib.py instead of deprecated core.py

- Also remove redundant info from docs about default values

* Fix docsserver automethod
2018-06-29 18:45:22 +05:00
codegeschrei
7449b026fa Problem: voting code is not used anymore (#2357)
* Problem: voting code is not used anymore
Solution: remove all voting related code

* Problem: Some voting functionality is still present.

Solution: Remove it. Update some of the related tests.

* Problem: some skipped tests are now running
Solution: remove pytest mark to not run them

* Problem: fastquery is not related to votes
Solution: remove it in another PR
2018-06-28 11:30:13 +02:00
Shahbaz Nazir
2da720e542 Problem: There is no logging for benchmark stats (#2349)
* add log level for benchmarking

Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>

* Bug fixes and code re-factor

Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>

* remove un-necessary imports

Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>

* fix failing tests

Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>

* update docs

Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>

* fix typo

Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>

* fix flake8

Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>

* fix flake8 issues

Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>

* nitpic fixes

Signed-off-by: Shahbaz Nazir <shahbaz@bigchaindb.com>
2018-06-14 15:19:26 +02:00
Muawia Khan
2b39566a4b Problem: Some configurations are defunct (#2338)
* Problem: BigchainDB config has stale/deprecated parameters

- Remove `keyring` and `keypair` from config
- Update tests
- Add `tendermint` config

* Fix flake8

* Update some naming conventions and redundant changes

* Remove redundant routine `fast_query`

* Remove deprecated parameters and descriptions

* remove some more unwanted code

* Problem: Two flake8 errors made Travis fail

Solution: Fix the two flake8 errors

* Address comments

- Remove reference of nodes_except_me and me_private and me
  as attributes of BigchainDB instances
- Update and re-add test(s)
- Do not introduce `tendermint` in configuration instead handle that
  in a separate PR along with docs

* Address comments

- Remove tests that are already covered with 2.0
- Remove tests that are no longer relevant
- Add TODO for more cleanup

* Remove tendermint config from configure command
2018-06-08 15:50:50 +02:00
Lev Berman
89b28b8471 Remove unsupported backends (#2289)
* Problem: RethinkDB, change feed, old mongo, admin interface are not supported any longer.

Solution: Remove unsupported functionality. Bring the MongoDB backend implementation completely to the localmongodb package. Fix the test setup.

* Problem: Nothing depends on multipipes any longer.

Solution: Remove multipipes from setup.py.

* Problem: The how-to-run-tests doc uses --database-backend.

Solution: Do not include the --database-backend option into the documented pytest usage.

* Problem: The backends docs are outdated.

Solution: Document MongoDB as the default and only backend for BigchainDB.

* Problem: The inputs fixtures uses old blocks API.

Solution: Change the inputs fixtures to use the new blocks API.

* Problem: rethinkdb package is not used anymore.

Solution: Remove the rethinkdb dependency from setup.py.

* Problem: The abci-marked tests use outdated Mongo conn.

Solution: Replace MongoDBConnection with LocalMongoDBConnection for them.
2018-05-23 11:34:00 +02:00
Lev Berman
2e9c20491c Problem: web transaction tests were broken or disabled. (#2278)
Solution: Fix or reenable the tests after the migration to BigchainDB 2.*.
2018-05-16 15:17:33 +02:00
Lev Berman
89d665c01e Problem: Asset tests are disabled. (#2277)
Solution: Update the tests to work with BigchainDB 2.* and enable them back.
2018-05-15 11:19:47 +02:00
Vanshdeep Singh
63635f12fc Problem: Event stream test failing (#2262)
Solution: Extend the wait time for Tendermint to start
2018-05-03 16:42:43 +02:00
Vanshdeep Singh
65bb6e5c91 Problem: No integration test for upsert-validator (#2212)
Solution: Write necessary test
2018-04-24 16:17:49 +02:00
Vanshdeep Singh
7f6782e31d Problem: ABCI server not run for tests (#2138)
Solution: Create seperate test mark for which the ABCI server will be running
2018-04-10 17:29:25 +02:00
codegeschrei
6f69f39ff4 Problem: there are no f-strings in python 3.5 (#2185)
Solution: change f-strings to a compatible format for Python 3.5
2018-04-09 17:02:59 +02:00
Sylvain Bellemare
802cc73d33 Problem: import order does not follow pep8
Solution: follow pep8
2018-03-01 17:50:20 +01:00
Sylvain Bellemare
c85c664215 Problem: fixture with no asset fails
Solution: instantiate the transaction with a dummy asset
2018-03-01 15:11:58 +01:00
Sylvain Bellemare
51d4f0f2d9 Problem: utxoset needs to be updated
Solution: update utxoset via store_transaction
2018-03-01 15:11:58 +01:00
Vanshdeep Singh
5bfa8e29d8 Crash recovery mechanism (#2045)
* Crash recovery mechanism

* Propogate exception

* Added docs and crash receovery during block write

* Fix flake8 issue

* Remove approach 1 for crash recovery, recover db on 'bigchiandb start'

* Fix CI build issues

* Remove documentation
2018-02-21 10:50:12 +01:00
Sylvain Bellemare
1f537b1557 Problem: to get tx utxoset & inputs is cumbersome (#2055)
Solution: add properties to more easily get the utxoset and inputs of a
transaction
2018-02-15 18:23:29 +01:00
vrde
cd6095e854 Problem: test suite doesn't pick up env vars
Solution: call `env_config` from the configuration fixture
2018-02-14 16:43:09 +01:00
Sylvain Bellemare
161ccdda5d Implement UTXO set backend (#2033) 2018-02-14 15:37:19 +01:00
vrde
55073366ec Problem: new env vars are not managed by the conf
Solution: add `BIGCHAINDB_` to new tendermint vars. Reference:
https://github.com/bigchaindb/bigchaindb/pull/2039#discussion_r167880795
2018-02-14 12:18:59 +01:00
Sylvain Bellemare
6e3f9e8f54 Docker compose for travis (#2039)
* Replace double quotes with single quotes (flake8)

* Test event subscriber to tendermint via ws

* Problem: Tendermint is not part of stack for CI

Solution: Add Tendermint to stack for CI. For simplicity's sake
docker-compose is being used.
2018-02-14 10:19:04 +01:00
Sylvain Bellemare
cc069fa640 Add db_context fixture 2018-02-08 15:26:50 +01:00
Sylvain Bellemare
52d0c7024a
Merge branch 'master' into tendermint 2017-12-23 04:58:18 +01:00
Sylvain Bellemare
8bd8dc02a4 Fixes #1969 2017-12-22 18:03:57 +01:00
Sylvain Bellemare
cb268a3832
Merge branch 'master' into tendermint 2017-12-06 16:59:26 +01:00
Sylvain Bellemare
9dd2e026b0 Fixes #1891 2017-12-05 19:38:52 +01:00
vrde
1e104ad2c4
Merge branch 'master' into tendermint 2017-11-30 18:17:36 +01:00
Katha
e377fb57af
Pre commit styling (#1914)
* run pre-commit and fix errors

* mention pre-commit in our guides
2017-11-30 15:04:14 +01:00
kansi
fcfe577813 Added localmongodb marker 2017-11-24 12:30:57 +05:30
kansi
6c3f176122 Integrate asset search 2017-11-23 21:28:51 +05:30
Vanshdeep Singh
5ce2eb1f97 Bypass CI issues for rethinkdb (#1821)
Add custom marker to skip some travis/rdb tests
2017-11-08 19:41:17 +01:00
Krish
3ec4153375 Support for advertised host, port, scheme for Websockets (#1703)
* Support for advertised host, port, scheme for Websockets

* Update docs

* Add a docstring

* Set env vars in Dockerfile-dev too
2017-08-08 12:04:58 +02:00