planetmint/tests/backend/tarantool/test_queries.py
Lorenz Herzberger 4472a1a3ee
refactor tarantool backend (#292)
* added initial interfaces for backend, refactored Asset and MetaData logic

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* adjusted input dataclass, added queries, removed convert

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* created backend models folder, replaced token_hex with uuid

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* Add cleanup and add constants

Signed-off-by: cybnon <stefan.weber93@googlemail.com>

* added to and from static methods to asset, input model and removed logic from tools

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* simplified store_bulk_transaction and corresponding query, adjusted test cases

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* changed script queries

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* Add Output model

Signed-off-by: cybnon <stefan.weber93@googlemail.com>

* Adapt Output class

Signed-off-by: cybnon <stefan.weber93@googlemail.com>

* Further fixes

Signed-off-by: cybnon <stefan.weber93@googlemail.com>

* Further fixes

* Get rid of decompose

Signed-off-by: cybnon <stefan.weber93@googlemail.com>

* refactored init.lua

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* refactored drop.lua

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* Add transaction data class

Signed-off-by: cybnon <stefan.weber93@googlemail.com>

* refactored init.lua

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* fix tests

Signed-off-by: cybnon <stefan.weber93@googlemail.com>

* Fix more tests

Signed-off-by: cybnon <stefan.weber93@googlemail.com>

* Format file

* Fix recursion error

* More fixes

Signed-off-by: cybnon <stefan.weber93@googlemail.com>

* Further fixes

Signed-off-by: cybnon <stefan.weber93@googlemail.com>

* using init.lua for db setup

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* fixed flush_db for new tarantool implementation

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* changed unique constraints

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* used new indexes on block related db operations

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* Adapt models

Signed-off-by: cybnon <stefan.weber93@googlemail.com>

* Check if blocks is empty

* adjusted get_txids_filtered for new indexes

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* Adaptions due to schema change

Signed-off-by: cybnon <stefan.weber93@googlemail.com>

* fixed get block test case

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* Fix subcondition serialization

Signed-off-by: cybnon <stefan.weber93@googlemail.com>

* Remove unnecessary method

Signed-off-by: cybnon <stefan.weber93@googlemail.com>

* More fixes

* renamed group_txs and used data models in fastquery

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* adjusted query test cases, removed unused code

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* replaced asset search with get_asset_by_cid

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* added limit to asset queries

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* replaced metadata search with cid lookup

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* fixed most of the test_lib test cases

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* fixed election test cases

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* fixed some more test cases

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* fixed 'is' vs '==' issue

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* - blackified & fixed recovery / delete transactions issues becaues of data model transitions
- reintegrated get_transaction() call in query -> delegating this to get_complete_transactions_by_ids

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* show election status uses the governance table from now on
show election status maps the asset["data"] object properly

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* fixed input object differences between old / new version and lookup of transaction in the governance pool

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* fixed TX lookup issues due to different pools

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* fixed wrong index name issue:  transaction_by_asset vs transaction_by_asset_id

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* fixed asset class key mixup

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* moved field removal methods to DbTransaction
redefined strcuture of DbTransction.to_dict() to be equal to the one of Transactions.to_dict()

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* added proper input conversion of the test cases and a proper input validation and object converion

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* simplified imports
fixed transfer input issues of the tests

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* fixed comparision issue : dict vs. object

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* fixed schema validation errors

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* added verification of ConditionDetails to the owner verification to avoid mixup between ConditionDetails and SubCondition
fixed Object comparision issues due to object changes

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* fixed object handling issue and complicated stuff

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* added missing import

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* added proper corner case handling in case a requested block is not found

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* fixed object comparision issue

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* fixed output handling for validate_transfer_inputs

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* fixed wrong search pool usage

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* fixed zenroom testcase

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* fixed last abci issues and blackified the code

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* added tarantool exception catching and raising as well as logging

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* fixed obj comparision issue in test_get_spent_issue_1271

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* added raiing CriticialDoubleSpend Exception for governance and transactions
fixed search space issue with election / voting commit lookup

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* * made returned outputs unique (get_owned_ids)
* added delete_output method to init.lua
* fixd output deletion issue by relaying the deletion to lua instead of the python code

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* fixed rollback after crash

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* adjusted assets=None to assets=[{"data":None}] to avoid exeptions in the background service

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* removed unused code

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* removed unused code, reverted transaction fetching, added return types to queries

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* removed duplicate code

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* removed depricated code

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

* store transactions of various versions (backwardcompatibility)
added _bdb variable to init/drop DBs for the single use cases (started failing as TXs are looked up in DB - compared to before)

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* added support for v2.0 transaction to DB writing/reading

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* fixed merge errors (arguments ... )

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* blackified

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* Simplified unit tests (#294)

* adjusted make test
* 1st improvments to ease testing
* simplified gh actions
* adjusted gh action file
* removed deps
* added sudo to apt calls
* removed predefined pytest module definitions
* added installing planetmint into the unit test container
* give time to the db container
* added environment variables to unit-test.yml
* removed acceptances tests from test executions

Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>

* removed unused code, updated version number

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>

Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com>
Signed-off-by: cybnon <stefan.weber93@googlemail.com>
Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com>
Co-authored-by: cybnon <stefan.weber93@googlemail.com>
Co-authored-by: Jürgen Eckel <juergen@riddleandcode.com>
Co-authored-by: Jürgen Eckel <eckelj@users.noreply.github.com>
2023-01-16 15:21:56 +01:00

177 lines
6.1 KiB
Python

# Copyright © 2020 Interplanetary Database Association e.V.,
# Planetmint and IPDB software contributors.
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
# Code is Apache-2.0 and docs are CC-BY-4.0
from copy import deepcopy
import pytest
import json
from transactions.common.transaction import Transaction
from transactions.types.assets.create import Create
from transactions.types.assets.transfer import Transfer
from planetmint.backend.interfaces import Asset, MetaData
from planetmint.backend.models import DbTransaction
pytestmark = pytest.mark.bdb
def test_get_txids_filtered(signed_create_tx, signed_transfer_tx, db_conn):
from planetmint.backend.tarantool import query
# create and insert two blocks, one for the create and one for the
# transfer transaction
create_tx_dict = signed_create_tx.to_dict()
transfer_tx_dict = signed_transfer_tx.to_dict()
query.store_transactions(signed_transactions=[create_tx_dict], connection=db_conn)
query.store_transactions(signed_transactions=[transfer_tx_dict], connection=db_conn)
asset_id = Transaction.get_asset_id([signed_create_tx, signed_transfer_tx])
# Test get by just asset id
txids = set(query.get_txids_filtered(connection=db_conn, asset_ids=[asset_id]))
assert txids == {signed_create_tx.id, signed_transfer_tx.id}
# Test get by asset and CREATE
txids = set(query.get_txids_filtered(connection=db_conn, asset_ids=[asset_id], operation=Transaction.CREATE))
assert txids == {signed_create_tx.id}
# Test get by asset and TRANSFER
txids = set(query.get_txids_filtered(connection=db_conn, asset_ids=[asset_id], operation=Transaction.TRANSFER))
assert txids == {signed_transfer_tx.id}
def test_get_owned_ids(signed_create_tx, user_pk, db_conn):
from planetmint.backend.tarantool import query
# insert a transaction
query.store_transactions(connection=db_conn, signed_transactions=[signed_create_tx.to_dict()])
txns = query.get_owned_ids(connection=db_conn, owner=user_pk)
tx_dict = signed_create_tx.to_dict()
owned_tx = txns[0].to_dict()
assert owned_tx == tx_dict
def test_store_block(db_conn):
from planetmint.lib import Block
from planetmint.backend.tarantool import query
block = Block(app_hash="random_utxo", height=3, transactions=[])
query.store_block(connection=db_conn, block=block._asdict())
# block = query.get_block(connection=db_conn)
blocks = db_conn.run(db_conn.space("blocks").select([]))
assert len(blocks) == 1
def test_get_block(db_conn):
from planetmint.lib import Block
from planetmint.backend.tarantool import query
block = Block(app_hash="random_utxo", height=3, transactions=[])
query.store_block(connection=db_conn, block=block._asdict())
block = dict(query.get_block(connection=db_conn, block_id=3))
assert block["height"] == 3
def test_store_pre_commit_state(db_conn):
from planetmint.backend.tarantool import query
state = dict(height=3, transactions=[])
query.store_pre_commit_state(connection=db_conn, state=state)
commit = query.get_pre_commit_state(connection=db_conn)
assert len([commit]) == 1
# cursor = db_context.conn.db.pre_commit.find({'commit_id': 'test'},
# projection={'_id': False})
def test_get_pre_commit_state(db_conn):
from planetmint.backend.tarantool import query
all_pre = db_conn.run(db_conn.space("pre_commits").select([]))
for pre in all_pre:
db_conn.run(db_conn.space("pre_commits").delete(pre[0]), only_data=False)
# TODO First IN, First OUT
state = dict(height=3, transactions=[])
# db_context.conn.db.pre_commit.insert_one
query.store_pre_commit_state(state=state, connection=db_conn)
resp = query.get_pre_commit_state(connection=db_conn)
assert resp == state
def test_validator_update(db_conn):
from planetmint.backend.tarantool import query
def gen_validator_update(height):
return {"validators": [], "height": height, "election_id": f"election_id_at_height_{height}"}
# return {'data': 'somedata', 'height': height, 'election_id': f'election_id_at_height_{height}'}
for i in range(1, 100, 10):
value = gen_validator_update(i)
query.store_validator_set(conn=db_conn, validators_update=value)
v1 = query.get_validator_set(connection=db_conn, height=8)
assert v1["height"] == 1
v41 = query.get_validator_set(connection=db_conn, height=50)
assert v41["height"] == 41
v91 = query.get_validator_set(connection=db_conn)
assert v91["height"] == 91
@pytest.mark.parametrize(
"description,stores,expected",
[
(
"Query empty database.",
[],
None,
),
(
"Store one chain with the default value for `is_synced`.",
[
{"height": 0, "chain_id": "some-id"},
],
{"height": 0, "chain_id": "some-id", "is_synced": True},
),
(
"Store one chain with a custom value for `is_synced`.",
[
{"height": 0, "chain_id": "some-id", "is_synced": False},
],
{"height": 0, "chain_id": "some-id", "is_synced": False},
),
(
"Store one chain, then update it.",
[
{"height": 0, "chain_id": "some-id", "is_synced": True},
{"height": 0, "chain_id": "new-id", "is_synced": False},
],
{"height": 0, "chain_id": "new-id", "is_synced": False},
),
(
"Store a chain, update it, store another chain.",
[
{"height": 0, "chain_id": "some-id", "is_synced": True},
{"height": 0, "chain_id": "some-id", "is_synced": False},
{"height": 10, "chain_id": "another-id", "is_synced": True},
],
{"height": 10, "chain_id": "another-id", "is_synced": True},
),
],
)
def test_store_abci_chain(description, stores, expected, db_conn):
from planetmint.backend.tarantool import query
for store in stores:
query.store_abci_chain(db_conn, **store)
actual = query.get_latest_abci_chain(db_conn)
assert expected == actual, description