mirror of
https://github.com/planetmint/planetmint.git
synced 2026-02-20 02:23:50 +00:00
refactor tarantool backend (#292)
* added initial interfaces for backend, refactored Asset and MetaData logic Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * adjusted input dataclass, added queries, removed convert Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * created backend models folder, replaced token_hex with uuid Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * Add cleanup and add constants Signed-off-by: cybnon <stefan.weber93@googlemail.com> * added to and from static methods to asset, input model and removed logic from tools Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * simplified store_bulk_transaction and corresponding query, adjusted test cases Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * changed script queries Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * Add Output model Signed-off-by: cybnon <stefan.weber93@googlemail.com> * Adapt Output class Signed-off-by: cybnon <stefan.weber93@googlemail.com> * Further fixes Signed-off-by: cybnon <stefan.weber93@googlemail.com> * Further fixes * Get rid of decompose Signed-off-by: cybnon <stefan.weber93@googlemail.com> * refactored init.lua Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * refactored drop.lua Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * Add transaction data class Signed-off-by: cybnon <stefan.weber93@googlemail.com> * refactored init.lua Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * fix tests Signed-off-by: cybnon <stefan.weber93@googlemail.com> * Fix more tests Signed-off-by: cybnon <stefan.weber93@googlemail.com> * Format file * Fix recursion error * More fixes Signed-off-by: cybnon <stefan.weber93@googlemail.com> * Further fixes Signed-off-by: cybnon <stefan.weber93@googlemail.com> * using init.lua for db setup Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * fixed flush_db for new tarantool implementation Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * changed unique constraints Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * used new indexes on block related db operations Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * Adapt models Signed-off-by: cybnon <stefan.weber93@googlemail.com> * Check if blocks is empty * adjusted get_txids_filtered for new indexes Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * Adaptions due to schema change Signed-off-by: cybnon <stefan.weber93@googlemail.com> * fixed get block test case Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * Fix subcondition serialization Signed-off-by: cybnon <stefan.weber93@googlemail.com> * Remove unnecessary method Signed-off-by: cybnon <stefan.weber93@googlemail.com> * More fixes * renamed group_txs and used data models in fastquery Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * adjusted query test cases, removed unused code Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * replaced asset search with get_asset_by_cid Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * added limit to asset queries Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * replaced metadata search with cid lookup Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * fixed most of the test_lib test cases Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * fixed election test cases Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * fixed some more test cases Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * fixed 'is' vs '==' issue Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * - blackified & fixed recovery / delete transactions issues becaues of data model transitions - reintegrated get_transaction() call in query -> delegating this to get_complete_transactions_by_ids Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * show election status uses the governance table from now on show election status maps the asset["data"] object properly Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * fixed input object differences between old / new version and lookup of transaction in the governance pool Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * fixed TX lookup issues due to different pools Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * fixed wrong index name issue: transaction_by_asset vs transaction_by_asset_id Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * fixed asset class key mixup Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * moved field removal methods to DbTransaction redefined strcuture of DbTransction.to_dict() to be equal to the one of Transactions.to_dict() Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * added proper input conversion of the test cases and a proper input validation and object converion Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * simplified imports fixed transfer input issues of the tests Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * fixed comparision issue : dict vs. object Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * fixed schema validation errors Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * added verification of ConditionDetails to the owner verification to avoid mixup between ConditionDetails and SubCondition fixed Object comparision issues due to object changes Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * fixed object handling issue and complicated stuff Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * added missing import Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * added proper corner case handling in case a requested block is not found Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * fixed object comparision issue Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * fixed output handling for validate_transfer_inputs Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * fixed wrong search pool usage Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * fixed zenroom testcase Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * fixed last abci issues and blackified the code Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * added tarantool exception catching and raising as well as logging Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * fixed obj comparision issue in test_get_spent_issue_1271 Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * added raiing CriticialDoubleSpend Exception for governance and transactions fixed search space issue with election / voting commit lookup Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * * made returned outputs unique (get_owned_ids) * added delete_output method to init.lua * fixd output deletion issue by relaying the deletion to lua instead of the python code Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * fixed rollback after crash Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * adjusted assets=None to assets=[{"data":None}] to avoid exeptions in the background service Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * removed unused code Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * removed unused code, reverted transaction fetching, added return types to queries Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * removed duplicate code Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * removed depricated code Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> * store transactions of various versions (backwardcompatibility) added _bdb variable to init/drop DBs for the single use cases (started failing as TXs are looked up in DB - compared to before) Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * added support for v2.0 transaction to DB writing/reading Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * fixed merge errors (arguments ... ) Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * blackified Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * Simplified unit tests (#294) * adjusted make test * 1st improvments to ease testing * simplified gh actions * adjusted gh action file * removed deps * added sudo to apt calls * removed predefined pytest module definitions * added installing planetmint into the unit test container * give time to the db container * added environment variables to unit-test.yml * removed acceptances tests from test executions Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> * removed unused code, updated version number Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> Signed-off-by: Lorenz Herzberger <lorenzherzberger@gmail.com> Signed-off-by: cybnon <stefan.weber93@googlemail.com> Signed-off-by: Jürgen Eckel <juergen@riddleandcode.com> Co-authored-by: cybnon <stefan.weber93@googlemail.com> Co-authored-by: Jürgen Eckel <juergen@riddleandcode.com> Co-authored-by: Jürgen Eckel <eckelj@users.noreply.github.com>
This commit is contained in:
Lorenz Herzberger
committed by
GitHub
GitHub
parent
8730d516a3
commit
4472a1a3ee
59
.github/workflows/unit-test-abci.yml
vendored
59
.github/workflows/unit-test-abci.yml
vendored
@@ -1,59 +0,0 @@
|
||||
# Copyright © 2020 Interplanetary Database Association e.V.,
|
||||
# Planetmint and IPDB software contributors.
|
||||
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
|
||||
# Code is Apache-2.0 and docs are CC-BY-4.0
|
||||
|
||||
name: Unit tests - with direct ABCI
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Check out repository code
|
||||
uses: actions/checkout@v3
|
||||
|
||||
- name: Build container
|
||||
run: |
|
||||
docker-compose -f docker-compose.yml build --no-cache --build-arg abci_status=enable planetmint
|
||||
|
||||
- name: Save image
|
||||
run: docker save -o planetmint.tar planetmint_planetmint
|
||||
|
||||
- name: Upload image
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: planetmint-abci
|
||||
path: planetmint.tar
|
||||
retention-days: 5
|
||||
|
||||
|
||||
test-with-abci:
|
||||
runs-on: ubuntu-latest
|
||||
needs: build
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- db: "Tarantool with ABCI"
|
||||
host: "tarantool"
|
||||
port: 3303
|
||||
abci: "enabled"
|
||||
|
||||
steps:
|
||||
- name: Check out repository code
|
||||
uses: actions/checkout@v3
|
||||
|
||||
- name: Download planetmint
|
||||
uses: actions/download-artifact@v3
|
||||
with:
|
||||
name: planetmint-abci
|
||||
|
||||
- name: Load planetmint
|
||||
run: docker load -i planetmint.tar
|
||||
|
||||
- name: Start containers
|
||||
run: docker-compose -f docker-compose.yml up -d planetmint
|
||||
|
||||
- name: Run tests
|
||||
run: docker exec planetmint_planetmint_1 pytest -v -m abci
|
||||
60
.github/workflows/unit-test-no-abci.yml
vendored
60
.github/workflows/unit-test-no-abci.yml
vendored
@@ -1,60 +0,0 @@
|
||||
# Copyright © 2020 Interplanetary Database Association e.V.,
|
||||
# Planetmint and IPDB software contributors.
|
||||
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
|
||||
# Code is Apache-2.0 and docs are CC-BY-4.0
|
||||
|
||||
name: Unit tests - with Planemint
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Check out repository code
|
||||
uses: actions/checkout@v3
|
||||
|
||||
- name: Build container
|
||||
run: |
|
||||
docker-compose -f docker-compose.yml build --no-cache planetmint
|
||||
|
||||
- name: Save image
|
||||
run: docker save -o planetmint.tar planetmint_planetmint
|
||||
|
||||
- name: Upload image
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: planetmint
|
||||
path: planetmint.tar
|
||||
retention-days: 5
|
||||
|
||||
|
||||
test-without-abci:
|
||||
runs-on: ubuntu-latest
|
||||
needs: build
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- db: "Tarantool without ABCI"
|
||||
host: "tarantool"
|
||||
port: 3303
|
||||
|
||||
steps:
|
||||
- name: Check out repository code
|
||||
uses: actions/checkout@v3
|
||||
|
||||
- name: Download planetmint
|
||||
uses: actions/download-artifact@v3
|
||||
with:
|
||||
name: planetmint
|
||||
|
||||
- name: Load planetmint
|
||||
run: docker load -i planetmint.tar
|
||||
|
||||
- name: Start containers
|
||||
run: docker-compose -f docker-compose.yml up -d bdb
|
||||
|
||||
- name: Run tests
|
||||
run: docker exec planetmint_planetmint_1 pytest -v --cov=planetmint --cov-report xml:htmlcov/coverage.xml
|
||||
|
||||
- name: Upload Coverage to Codecov
|
||||
uses: codecov/codecov-action@v3
|
||||
42
.github/workflows/unit-tests.yml
vendored
Normal file
42
.github/workflows/unit-tests.yml
vendored
Normal file
@@ -0,0 +1,42 @@
|
||||
# Copyright © 2020 Interplanetary Database Association e.V.,
|
||||
# Planetmint and IPDB software contributors.
|
||||
# SPDX-License-Identifier: (Apache-2.0 AND CC-BY-4.0)
|
||||
# Code is Apache-2.0 and docs are CC-BY-4.0
|
||||
|
||||
name: Unit tests
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
unified-unit-tests:
|
||||
runs-on: ubuntu-latest
|
||||
env:
|
||||
PLANETMINT_DATABASE_BACKEND: tarantool_db
|
||||
PLANETMINT_DATABASE_HOST: localhost
|
||||
PLANETMINT_DATABASE_PORT: 3303
|
||||
PLANETMINT_SERVER_BIND: 0.0.0.0:9984
|
||||
PLANETMINT_WSSERVER_HOST: 0.0.0.0
|
||||
PLANETMINT_WSSERVER_ADVERTISED_HOST: localhost
|
||||
PLANETMINT_TENDERMINT_HOST: localhost
|
||||
PLANETMINT_TENDERMINT_PORT: 26657
|
||||
|
||||
steps:
|
||||
- name: Check out repository code
|
||||
uses: actions/checkout@v3
|
||||
|
||||
- name: Setup python
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: 3.9
|
||||
|
||||
- name: Prepare OS
|
||||
run: sudo apt-get update && sudo apt-get install -y git zsh curl tarantool-common vim build-essential cmake
|
||||
|
||||
- name: Get Tendermint
|
||||
run: wget https://github.com/tendermint/tendermint/releases/download/v0.34.15/tendermint_0.34.15_linux_amd64.tar.gz && tar zxf tendermint_0.34.15_linux_amd64.tar.gz
|
||||
|
||||
- name: Install Planetmint
|
||||
run: pip install -e '.[dev]'
|
||||
|
||||
|
||||
- name: Execute Tests
|
||||
run: make test
|
||||
Reference in New Issue
Block a user