38 Commits

Author SHA1 Message Date
lana-shanghai
4a008e51e3
[WIP] Documentation re-org (#2694)
* Reorganized docs

* Fixed internal links in basic usage

* fixed the docker-compose command and volume for docs

* fixed docs tests

* fix travis docs test

* tox ini file

* fixed readme localhost links

* edited tox and test docs to previous state

* Fix tests errors related to docs reorganization

Signed-off-by: David Dashyan <mail@davie.li>

* Added ansible script installation option

Signed-off-by: Lana Ivina <lana@ipdb.io>

* Added ansible script to network setup guide

Signed-off-by: Lana Ivina <lana@ipdb.io>

* Hid the non-working button for now. 

Signed-off: Lana Ivina <lana@ipdb.io>

* Try now button

Co-authored-by: David Dashyan <mail@davie.li>
2020-05-18 20:22:26 +07:00
David Dashyan
9e99c024d3
Replace headers (#2683)
Signed-off-by: David Dashyan <mail@davie.li>
2020-04-06 11:52:18 +02:00
Lev Berman
0c4d6ec906 Implement crash recovery for elections. (#2566)
1. Save pre-commit state in the beginning of end block. 
2. Provide an interface for custom elections to rollback on crash recovery. 
3. Simplify pre-commit management. 
4. Add crash recovery for updert-validator and chain migration elecitons.
2018-09-28 12:32:45 +02:00
Vanshdeep Singh
78dafce146 Problem: Outputs API responds with incorrect outputs (#2567).
* Problem: Outputs API doesn't respond with correct unspent/spent outputs

Solution: Fix fastquery such that embedded document is queried properly

* Problem: key order agnostic queries not implemented

Solution: get_spent queries embedded documents which respect key order. This is
not expected by the application hence the query should be altered to query any
kind of key order

* Problem: Mongo query for get_spent too complicated

Solution: Simplify query using $elemMatch

* Problem: No test for checking mixed spent outputs

Solution: Add test for `get_spending_transactions` to check that correct
matching is done when querying documents with multiple inputs

* Problem: tranasction ids not assert when getting spent outputs

Solution: assert tranasction ids
2018-09-26 10:17:39 +02:00
Lev Berman
24ca0b32a9 Separate pending and effective validator updates. (#2556)
* Separate pending and effective validator updates.

- Pending validator updates do not prevent elections from concluding.
- ValidatorElection overrides has_conclude to not conclude when there is a pending update for the matching height.
- Effective validator updates deem past elections inconclusive.

* Problem: Looking for election block is inefficient.

Solution: Record placed elections, update the records upon election conclusion.

* Clarify the conclusion order in Election.process_blocks.

* Insert election records in bulk.

Otherwise, one can significantly slow nodes down by posting a whole bunch of unique elections.

* Change get_election to use find_one.

* Calculate total votes without making extra query.

* Fix the pending valset check.

* Fix election test setup.
2018-09-21 10:51:57 +02:00
Muawia Khan
bd39076522 Problem: Users trying to use an incompatible version of Tendermint (#2541)
-  Hard-wire the supported Tendermint version(s) right in the code of BigchainDB Server. Check the version of Tendermint and disconnect if Tendermint version is an unsupported one.
- Expose a CLI command bigchaindb tendermint-version to display the supported Tendermint versions.
- PR also takes care the long list of warnings we get when we run tests.
    - Updated deprecated pymongo methods
    - Do not call pytest fixtures directly.
    - Also added the doc for the new cli command
2018-09-14 15:37:18 +02:00
Lev Berman
35e35ecd57 Problem: New collections are not created in existing DB. (#2520)
Solution: Do not abort the initialisation if a collection exists. Unify the index creation.
2018-09-12 14:39:07 +02:00
Zachary Bowen
0fe749d830 Create abstract election class (#2498).
* Problem: `ValidatorElection` and `MigrationElection` need to inherit from a common `Election` class.

Solution: Factored the common logic out of `ValidatorElection` and moved it to `Election` parent class.

* Problem: No need to store different types of elections in their own tables

Solution: Remove `DB_TABLE` property from `Election` class.

Solution: Created the `elections` table with secondary_index `election_id`.

* Problem: `UpsertValidatorVote` can be generalized to just be `Vote`

Solution: Renamed, refactored and moved the `Vote` class to tie in with the more general `Election` base class.

* Problem: `election_id` is not unique if two elections have the same properties.

Solution: Added a random `uuid4` seed to enforce uniqueness.
2018-09-11 15:39:46 +02:00
codegeschrei
dd84d4eb6f Problem: There are unnecessary markers (#2522)
* Problem: we have unused and outdated fixtures
Solution: clean up fixtures and tests accordingly

* Problem: there are still unused fixtures
Solution: remove Merlin keys

* Problem: There are unnecessary markers
Solution: remove the tendermint marker for tests
2018-09-10 09:53:18 +02:00
Vanshdeep Singh
cb22557771 Problem: Validation code not optimized (#2490)
Solution: memoize data
2018-09-04 15:30:52 +02:00
Lev Berman
230a5b2d69 ABCI chain migration conclusion (#2488)
* Problem: No good way to check for val set absence.

Solution: Make get_validator_set/get_validators return None/[] when there are no validators yet.

* Problem: Incompatible ABCI chain upgrades.

Solution: Record known chains and sync through InitChain. Triggering the migration and adjusting other ABCI endpoints will follow.
2018-09-03 15:41:49 +02:00
Lev Berman
fe0a4c494b
Problem: No MongoDB support for election ID index. (#2507)
Solution: Election ID has to be unique but not every validator set record has it. MongoDB does not support partial indexes, does not even allow for multiple Nones. This is a temporary fix since we are introducing an `election` collection to store election IDs in #2498.
2018-09-03 15:09:16 +02:00
Shahbaz Nazir
2d1f670eec
Problem: BigchainDB has un-necessary code to initialize a replica set and check if MongoDB was started with replicaSet (#2491)
Solution: Remove un-necessary code. Deployment of MongoDB with or without replicaSet should be the responsibility of MongoDB admin which can and cannot be a BigchainDB node operator. As far as BigchainDB is concerned replicaset, if provided in bigchaindb configs, should be used to establish connection with MongoDB.
2018-08-31 13:54:32 +02:00
Zachary Bowen
cfc2c5900b Rework upsert-validator show status (#2496)
* Problem: We need to store the `election_id` as part of the `validator_update` so we can efficiently check which election was resposible for the change

Solution: Added the parameter to `store_validator_set` and aligned the tests

* Problem: Logic for `upsert-validator show` is convoluted

Solution: Rewrote the function to be much simpler

* Problem: Need a uniqueness constraint for election_id wrt validator changes

Solution: Added a new key to the db schema
2018-08-31 09:47:47 +02:00
Vanshdeep Singh
3cf368aab7 Problem: No unique indexes on transaction id and block height (#2492)
Solution: Created unique indexes when setting up the collections
2018-08-29 17:45:08 +02:00
codegeschrei
6fdcaf44a7 Problem: we have unused and outdated fixtures (#2476)
* Problem: we have unused and outdated fixtures
Solution: clean up fixtures and tests accordingly
2018-08-27 09:48:20 +02:00
Troy McConaghy
abdd23f5a6 Problem: Source files contain no license info (#2455)
* Problem: Source files contain no license info

Solution: Add comments with SPDX license info to source files

* Python 3 files don't need # -*- coding: utf-8 -*-
2018-08-16 12:31:32 +02:00
Vanshdeep Singh
2e9a9b1121 Problem: Validator set not tracked by BigchainDB (#2436)
* Problem: Validator set not tracked by BigchainDB

Solution: BigchainDB depends on tendermint's RPC API to get the validator set
which is not avaiable during replay so the validators set should be tracked
inside BigchainDB

* Problem: Unclear code and documentation

Solution: Fix decode_validator and docs strings

* Problem: Doc strings missing

Solution: Add doc string for store_validato_set
2018-08-06 11:37:43 +02:00
Zachary Bowen
2386ca9d71 Refactor tendermint directory to project root (#2401)
* Problem: core.py contains an unused class, `Bigchain`

Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.

* Problem: core.py contains an unused class, `Bigchain`

Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.

* Fixed flake8 complaint about too many blank lines

* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...

Sorry in advance!

* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...

Sorry in advance!

* Updating from master changed BigchainDB.process_post_response to a private method, so I had to align with that.

* Fixed a couple stale references to bigchaindb.Bigchain in docstrings

* Missed a reference to `Bigchain` in a patch call...

* Problem: BigchainDB class should be part of project root

Solution: Removed the /tendermint directory and moved its contents to project root

* Problem: Flake8 complained that imports were not at the top of the file

Solution: Had to play around with the order of imports to avoid cyclic dependencies, but its working and style compliant now

* Problem: Stale reference to /tendermint directory in the index

Solution: Removed the references to /tendermint

* Problem: Flake8 complaining of unused import in __init__.py

Solution: The import is there so I can import App directly from bigchaindb, rather than from bigchaindb.core (which I think improves code readability. I added a # noqa to silence Flake8.

* Problem: Stale references to `bigchaindb.tendermint.BigchainDB` in the rst files cause Sphinx to fail

Solution: Updated the autodoc files to use `bigchaindb.BigchainDB` instead

* Problem: Stale reference to the `tendermint` directory in an @patch in a disabled test

Solution: Updated the @patch for completeness

* Problem: BigchainDB class should be part of project root

Solution: Removed the /tendermint directory and moved its contents to project root

* Problem: Flake8 complained that imports were not at the top of the file

Solution: Had to play around with the order of imports to avoid cyclic dependencies, but its working and style compliant now

* Problem: Stale reference to /tendermint directory in the index

Solution: Removed the references to /tendermint

* Problem: Flake8 complaining of unused import in __init__.py

Solution: The import is there so I can import App directly from bigchaindb, rather than from bigchaindb.core (which I think improves code readability. I added a # noqa to silence Flake8.

* Problem: Stale references to `bigchaindb.tendermint.BigchainDB` in the rst files cause Sphinx to fail

Solution: Updated the autodoc files to use `bigchaindb.BigchainDB` instead

* Problem: Stale reference to the `tendermint` directory in an @patch in a disabled test

Solution: Updated the @patch for completeness
2018-07-25 16:59:25 +02:00
Lev Berman
89b28b8471 Remove unsupported backends (#2289)
* Problem: RethinkDB, change feed, old mongo, admin interface are not supported any longer.

Solution: Remove unsupported functionality. Bring the MongoDB backend implementation completely to the localmongodb package. Fix the test setup.

* Problem: Nothing depends on multipipes any longer.

Solution: Remove multipipes from setup.py.

* Problem: The how-to-run-tests doc uses --database-backend.

Solution: Do not include the --database-backend option into the documented pytest usage.

* Problem: The backends docs are outdated.

Solution: Document MongoDB as the default and only backend for BigchainDB.

* Problem: The inputs fixtures uses old blocks API.

Solution: Change the inputs fixtures to use the new blocks API.

* Problem: rethinkdb package is not used anymore.

Solution: Remove the rethinkdb dependency from setup.py.

* Problem: The abci-marked tests use outdated Mongo conn.

Solution: Replace MongoDBConnection with LocalMongoDBConnection for them.
2018-05-23 11:34:00 +02:00
Vanshdeep Singh
127ee08053 Problem: No crash recovery mechanism (#2207)
Solution: Add crash recovery logic based on BEP#8
2018-04-18 10:46:16 +02:00
Vanshdeep Singh
a4986b7e71 Problem: No queries to store pre-commit state (#2135)
Solution: Add backend logic to store and retireve pre-commit state
2018-04-12 13:36:21 +02:00
Vanshdeep Singh
e4e528e5c4 Problem: Changing validators requires a network restart (#2070)
Solution: Allow nodes to add, update, or remove validators at runtime using a new command. Implements BEP3.
2018-03-29 18:25:26 +02:00
Sylvain Bellemare
2d2182dd19 Problem: unhandled error when deleting zero utxo
Solution: only execute the query if *unspent_outputs is not empty
2018-03-01 15:11:58 +01:00
Sylvain Bellemare
aaec67724a Problem: unhandled error when storing zero utxo
Solution: only execute the query if *unspent_outputs is not empty
2018-03-01 15:11:58 +01:00
Sylvain Bellemare
51d4f0f2d9 Problem: utxoset needs to be updated
Solution: update utxoset via store_transaction
2018-03-01 15:11:58 +01:00
Vanshdeep Singh
5bfa8e29d8 Crash recovery mechanism (#2045)
* Crash recovery mechanism

* Propogate exception

* Added docs and crash receovery during block write

* Fix flake8 issue

* Remove approach 1 for crash recovery, recover db on 'bigchiandb start'

* Fix CI build issues

* Remove documentation
2018-02-21 10:50:12 +01:00
Sylvain Bellemare
161ccdda5d Implement UTXO set backend (#2033) 2018-02-14 15:37:19 +01:00
Vanshdeep Singh
a7ed8cf4cb Write transactions on commit (#2017)
* Write transacitons on commit

* Fix docs strings, lazy check for duplicates
2018-02-02 14:40:24 +01:00
Vanshdeep Singh
e25d365828 Integrate blocks API (#1970)
* Integrate blocks api

* Update docs

* Fix docs

* Fixed mismatch between code and documentation

* Fixed docs
2018-01-31 14:42:41 +01:00
vrde
ab4328fa1f
Merge pull request #1939 from bigchaindb/feat/integrate-metadata-api
integrate metadata api
2017-12-20 16:01:19 +01:00
Troy McConaghy
7a491bb781 Removed pytest-catchlog from setup.py & fixed one failing test 2017-12-07 13:33:57 +01:00
kansi
b3f2eeb931 Integrate metadata api 2017-12-06 10:25:52 +05:30
kansi
2b3a9fa6f6 Integrate output search api 2017-11-30 23:59:42 +05:30
vrde
044a052644
Merge branch 'master' into tendermint 2017-11-29 17:18:00 +01:00
kansi
fcfe577813 Added localmongodb marker 2017-11-24 12:30:57 +05:30
kansi
6c3f176122 Integrate asset search 2017-11-23 21:28:51 +05:30
kansi
00dd16840f Added backend tests, configure app fixture 2017-11-23 16:27:35 +01:00