Problem statement:
BigchainDB v2.0.0b9 has been around for quite a while. Recently we have updated
Tendermint supported version to v0.31.5 which has incompatible blockchain.
Despite the fact that we have defined instructions on chain migration, no one
expected to migrate to incompatible chain within patch version range. So there
is a demand for Tendermint v0.22.8 compatibility among BigchainDB users.
Work has been done:
bigchaindb-abci package was upgraded to support multiple API versions.
New configuration field stating tendermint version was added.
Signed-off-by: David Dashyan <mail@davie.li>
* Separate pending and effective validator updates.
- Pending validator updates do not prevent elections from concluding.
- ValidatorElection overrides has_conclude to not conclude when there is a pending update for the matching height.
- Effective validator updates deem past elections inconclusive.
* Problem: Looking for election block is inefficient.
Solution: Record placed elections, update the records upon election conclusion.
* Clarify the conclusion order in Election.process_blocks.
* Insert election records in bulk.
Otherwise, one can significantly slow nodes down by posting a whole bunch of unique elections.
* Change get_election to use find_one.
* Calculate total votes without making extra query.
* Fix the pending valset check.
* Fix election test setup.
- Do not conclude migration election if there is a migration in progress.
- Rewrite election tests to not use mocks and assert many different things.
- Record concluded elections in the `election` collection.
* Problem: We need a way to synchronize a halt to block production to allow for upgrades across breaking changes.
* Solution: Created `MigrationElection`.
* Problem: Need documentation for `migration` elections.
* Solution: Updated the docs.
* Problem: `MigrationElection` needs 'new' CLI method.
* Solution: Updated the definition of `election` to include the new `migration` type.
* Problem: The way `end_block` checks for concluded elections assumes there is only one type of election (so we can't conclude an `upsert-validator` and a `chain-migration` at the same height).
* Solution: Re-engineered the code in `Elections` to conclude multiple elections in the same block. If more than one election change the validator set, only one of them is applied.
* Problem: Tendermint change to store validator changes at height h+2 will break `Election.get_status`.
* Solution: Reworked `get_validator_change` to look at only the latest block height or less.
- Hard-wire the supported Tendermint version(s) right in the code of BigchainDB Server. Check the version of Tendermint and disconnect if Tendermint version is an unsupported one.
- Expose a CLI command bigchaindb tendermint-version to display the supported Tendermint versions.
- PR also takes care the long list of warnings we get when we run tests.
- Updated deprecated pymongo methods
- Do not call pytest fixtures directly.
- Also added the doc for the new cli command
* Problem: We need to store the `election_id` as part of the `validator_update` so we can efficiently check which election was resposible for the change
Solution: Added the parameter to `store_validator_set` and aligned the tests
* Problem: Logic for `upsert-validator show` is convoluted
Solution: Rewrote the function to be much simpler
* Problem: Need a uniqueness constraint for election_id wrt validator changes
Solution: Added a new key to the db schema
* Problem: `run_upsert_validator_approve` was getting the voting power in a convoluted way
Solution: Changed it to get the voting power from the election outputs
* Problem: `run_upsert_validator_approve` casts votes for *all* voters, not just the user calling the command
Solution: Filter vote txs by the users public key
* Problem: Docs needed a more specific description of how to input the path to the private-key file
Solution: Changed the wording a bit
* Problem: Source files contain no license info
Solution: Add comments with SPDX license info to source files
* Python 3 files don't need # -*- coding: utf-8 -*-
* Problem: core.py contains an unused class, `Bigchain`
Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.
* Fixed flake8 complaint about too many blank lines
* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...
Sorry in advance!
* Updating from master changed BigchainDB.process_post_response to a private method, so I had to align with that.
* Fixed a couple stale references to bigchaindb.Bigchain in docstrings
* Missed a reference to `Bigchain` in a patch call...
* Problem: Tests in class `TestBigChainAPI` from `tests/db/test_bigchain_api.py` were disabled
Solution: Fixed the tests that still made sense, and deleted the rest.
* Problem: Flake8 style complaints
Solution: Made things prettier
* Problem: Debugging comments were slightly NSFW
Solution: Deleted the debugging comments
* Problem: Stale use of `pytest.mark.genesis` decorator on these tests
Solution: Removed the decorator
* Problem: CriticalDoubleInclusion exception was no longer used anywhere in the code
Solution: Deleted it
* Problem: Unused exception `GenesisBlockAlreadyExistsError`
Solution: Deleted it
* Problem: I removed a test that covered writing to the DB and checking that everything is stored faithfully
Solution: Replaced it and got it working again
* Problem: Need a method to initiate a new upsert-validator election
Solution: Added subcommand `new` to `upsert_validator`
* Problem: Changes to upsert-validator needed to be reflected in the docs
Solution: Wrote a section for `upsert-validator new`
* Problem: Node operator cannot vote on a ValidatorElection
Solution: Implement validator election voting spec
* Problem: Incorrent code comments
Solution: Update comments with correct context
* Problem: Delegated vote not casted back to election
Solution: Update test to cast votes back to election id and assert their validity
* Problem: core.py contains an unused class, `Bigchain`
Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.
* Problem: core.py contains an unused class, `Bigchain`
Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.
* Fixed flake8 complaint about too many blank lines
* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...
Sorry in advance!
* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...
Sorry in advance!
* Updating from master changed BigchainDB.process_post_response to a private method, so I had to align with that.
* Fixed a couple stale references to bigchaindb.Bigchain in docstrings
* Missed a reference to `Bigchain` in a patch call...
* Problem: BigchainDB class should be part of project root
Solution: Removed the /tendermint directory and moved its contents to project root
* Problem: Flake8 complained that imports were not at the top of the file
Solution: Had to play around with the order of imports to avoid cyclic dependencies, but its working and style compliant now
* Problem: Stale reference to /tendermint directory in the index
Solution: Removed the references to /tendermint
* Problem: Flake8 complaining of unused import in __init__.py
Solution: The import is there so I can import App directly from bigchaindb, rather than from bigchaindb.core (which I think improves code readability. I added a # noqa to silence Flake8.
* Problem: Stale references to `bigchaindb.tendermint.BigchainDB` in the rst files cause Sphinx to fail
Solution: Updated the autodoc files to use `bigchaindb.BigchainDB` instead
* Problem: Stale reference to the `tendermint` directory in an @patch in a disabled test
Solution: Updated the @patch for completeness
* Problem: BigchainDB class should be part of project root
Solution: Removed the /tendermint directory and moved its contents to project root
* Problem: Flake8 complained that imports were not at the top of the file
Solution: Had to play around with the order of imports to avoid cyclic dependencies, but its working and style compliant now
* Problem: Stale reference to /tendermint directory in the index
Solution: Removed the references to /tendermint
* Problem: Flake8 complaining of unused import in __init__.py
Solution: The import is there so I can import App directly from bigchaindb, rather than from bigchaindb.core (which I think improves code readability. I added a # noqa to silence Flake8.
* Problem: Stale references to `bigchaindb.tendermint.BigchainDB` in the rst files cause Sphinx to fail
Solution: Updated the autodoc files to use `bigchaindb.BigchainDB` instead
* Problem: Stale reference to the `tendermint` directory in an @patch in a disabled test
Solution: Updated the @patch for completeness
* Problem: Tendermint configuration not present in BigchainDB config
* Handle tendermint configurations properly
- Update docs
* Nitpick
* Missed some conflicts
* Make changes in tendermint/lib.py instead of deprecated core.py
- Also remove redundant info from docs about default values
* Fix docsserver automethod
* Problem: voting code is not used anymore
Solution: remove all voting related code
* Problem: Some voting functionality is still present.
Solution: Remove it. Update some of the related tests.
* Problem: some skipped tests are now running
Solution: remove pytest mark to not run them
* Problem: fastquery is not related to votes
Solution: remove it in another PR
* Problem: BigchainDB config has stale/deprecated parameters
- Remove `keyring` and `keypair` from config
- Update tests
- Add `tendermint` config
* Fix flake8
* Update some naming conventions and redundant changes
* Remove redundant routine `fast_query`
* Remove deprecated parameters and descriptions
* remove some more unwanted code
* Problem: Two flake8 errors made Travis fail
Solution: Fix the two flake8 errors
* Address comments
- Remove reference of nodes_except_me and me_private and me
as attributes of BigchainDB instances
- Update and re-add test(s)
- Do not introduce `tendermint` in configuration instead handle that
in a separate PR along with docs
* Address comments
- Remove tests that are already covered with 2.0
- Remove tests that are no longer relevant
- Add TODO for more cleanup
* Remove tendermint config from configure command
* Problem: RethinkDB, change feed, old mongo, admin interface are not supported any longer.
Solution: Remove unsupported functionality. Bring the MongoDB backend implementation completely to the localmongodb package. Fix the test setup.
* Problem: Nothing depends on multipipes any longer.
Solution: Remove multipipes from setup.py.
* Problem: The how-to-run-tests doc uses --database-backend.
Solution: Do not include the --database-backend option into the documented pytest usage.
* Problem: The backends docs are outdated.
Solution: Document MongoDB as the default and only backend for BigchainDB.
* Problem: The inputs fixtures uses old blocks API.
Solution: Change the inputs fixtures to use the new blocks API.
* Problem: rethinkdb package is not used anymore.
Solution: Remove the rethinkdb dependency from setup.py.
* Problem: The abci-marked tests use outdated Mongo conn.
Solution: Replace MongoDBConnection with LocalMongoDBConnection for them.
* Replace double quotes with single quotes (flake8)
* Test event subscriber to tendermint via ws
* Problem: Tendermint is not part of stack for CI
Solution: Add Tendermint to stack for CI. For simplicity's sake
docker-compose is being used.
