65 Commits

Author SHA1 Message Date
Mike Zak
ab3c81c552 [NOD-1583] Move all TestXXX interfaces to testapi 2020-12-02 13:18:50 +02:00
Svarog
f7fa823f17
[NOD-1551] Requirements for performance tests (#1154)
* [NOD-1551] Add NewTestConsensusWithDataDir to factory

* [NOD-1551] Cache transaction ID

* [NOD-1551] Should return err if err != nil

* [NOD-1551] BuildBlockWithParents returns the blocks pastUTXOData

* [NOD-1551] Set BlockCoinbaseMaturity to 0 in TestDoubleSpends

* [NOD-1551] Fix comments

* --amend

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2020-11-26 12:12:01 +02:00
Svarog
546ea83123
[NOD-1570] Fix the way UTXO iterators work (#1153)
* [NOD-1570] Implement utxo.IteratorWithDiff

* [NOD-1570] Utilize utxo.ITeratorWithDiff in RestorePastUTXOSetIterator and VirtualUTXOSetIterator

* [NOD-1570] Fix comment
2020-11-25 18:28:42 +02:00
Ori Newman
5b2fae0457
[NOD-1568] Add staticcheck checks (#1150) 2020-11-25 11:43:51 +02:00
Ori Newman
45d9b63572
[NOD-1567] Add clone methods to data stores types (#1149)
* [NOD-1567] Add clone methods to data stores types

* [NOD-1567] Fix comments

* [NOD-1567] Fix test
2020-11-24 17:56:18 +02:00
Ori Newman
c1505b4748
[NOD-1555] Use stageDiff to update virtualDiffParents (#1139)
* [NOD-1555] Filter ancestors in updateVirtualDiffParents

* [NOD-1555] Use stageDiff to update virtualDiffParents

* [NOD-1555] Don't add existing blocks in addToVirtualDiffParents

* [NOD-1555] Remove redundant check

* [NOD-1555] Fix log and rename removeAncestorsFromVirtualDiffParents->removeAncestorsFromVirtualDiffParentsAndAssignDiffChild

* [NOD-1555] Add logs

* [NOD-1555] Fix comment

* [NOD-1555] Fix logs
2020-11-23 15:09:39 +02:00
stasatdaglabs
dec9ef5f75
[NOD-1555] Implement TestResolveBlockStatusSanity (#1138)
* [NOD-1555] Implement TestResolveBlockStatusSanity.

* [NOD-1555] Fix the test name string.
2020-11-23 14:29:41 +02:00
Mike Zak
5211727206 [NOD-1557] Cover the consensusStateManager package in trace logs (#1135)
* [NOD-1557] Add trace logs in add_block_to_virtual.go.

* [NOD-1557] Add trace logs in resolve_block_status.go.

* [NOD-1557] Add trace logs in calculate_past_utxo.go.

* [NOD-1557] Add trace logs in finality.go.

* [NOD-1557] Add trace logs in multisets.go.

* [NOD-1557] Fix compilation errors.

* [NOD-1557] Add trace logs to verify_and_build_utxo.go.

* [NOD-1557] Add trace logs to update_virtual.go.

* [NOD-1557] Add trace logs to set_pruning_utxo_set.go.

* [NOD-1557] Add trace logs to populate_tx_with_utxo_entries.go.

* [NOD-1557] Add trace logs to pick_virtual_parents.go.

* [NOD-1557] Make go vet happy.

* [NOD-1557] Clarify that some logic in AddBlockToVirtual is there for the sake of logging alone.

* [NOD-1557] Call blockStatusStore directly in AddBlockToVirtual when refetching the block status.
2020-11-23 13:08:10 +02:00
Elichai Turkel
b3a3121725
Add TestFinality back (#1129)
* Add VirtualFinalityPoint to TestConsensusStateManager

* Add TestFinality back
2020-11-22 12:30:27 +02:00
Svarog
950dd0cc8d
[NOD-1556] Add some logs (#1110)
* [NOD-1556] Add logs regarding block status and virtual blue score

* [NOD-1556] UTXODiffAlgebra: add the offending outpoint to the text of errors

* [NOD-1556] Make checkIntersectionWithRule return ok as well
2020-11-19 11:17:05 +02:00
Ori Newman
60c24d8dea Fix TestBlueBlockWindow (#1098)
* Fix TestBlueBlockWindow

* Add comments
2020-11-17 16:00:16 +02:00
stasatdaglabs
213be67c47 [NOD-1538] Fix isBlockInHeaderPruningPointFuture. (#1094) 2020-11-17 16:00:16 +02:00
Mike Zak
7224d58940 [NOD-1532] Add comments 2020-11-17 16:00:16 +02:00
Mike Zak
b2188f5993 [NOD-1532] AlwaysCallResolveBlockStatus in BuildBlock + Fixes to make it work if there's nothing to resolve 2020-11-17 16:00:16 +02:00
Mike Zak
dbd15aecf5 [NOD-1532] Invert condition in isViolatingFinality 2020-11-17 16:00:16 +02:00
Mike Zak
66f5a5bd7d [NOD-1532] Genesis is not violating finality by definition 2020-11-17 16:00:16 +02:00
Mike Zak
c994200878 [NOD-1532] ResolveBlockStatus should return the blockStatus 2020-11-17 16:00:16 +02:00
Svarog
f2df48139f [NOD-1532] Remove virtual from ParentChildren when checking for virtualSelectedParent candidates (#1089)
* [NOD-1532] Remove virtual from ParentChildren when checking for virtualSelectedParent candidates

* [NOD-1532] Fix minor issues
2020-11-17 16:00:16 +02:00
Mike Zak
d8f72e2b27 [NOD-1532] Update VirtualUTXODiffParents diffs even if list didn't change 2020-11-17 16:00:16 +02:00
Mike Zak
83a88d9989 [NOD-1532] newUTXOSetIterator should start with -1 index 2020-11-17 16:00:16 +02:00
Mike Zak
151910c27a [NOD-1532] Check UTXOCommitment for all blocks 2020-11-17 16:00:16 +02:00
Mike Zak
fca8ed57bd [NOD-1532] Add another block in TestUTXOCommitment 2020-11-17 16:00:16 +02:00
Ori Newman
56679818be [NOD-1535] Add non coinbase transactions to diff (#1084) 2020-11-17 16:00:16 +02:00
Mike Zak
2dcfe90850 [NOD-1532] Shouldn't update parent diff if the parent is not UTXO-verified 2020-11-17 16:00:16 +02:00
Mike Zak
34be898491 [NOD-1532] utxoSetIterator should be a pointer receiver 2020-11-17 16:00:16 +02:00
Mike Zak
f4a2fbf64f [NOD-1532] Fixes in updateVirtualDiffParents 2020-11-17 16:00:16 +02:00
Ori Newman
a0c6076ccc [NOD-1535] Add new block to virtual diff parents only if it's valid (#1077) 2020-11-17 16:00:16 +02:00
Mike Zak
fddce00d08 [NOD-1532] Fixes in updateVirtualDiffParent 2020-11-17 16:00:16 +02:00
Mike Zak
ae682d59f7 [NOD-1532] Fixes in updateVirtualDiffParent 2020-11-17 16:00:16 +02:00
Ori Newman
347f3de15c [NOD-1535] fix reachability test (#1075)
* [NOD-1535] Don't compare pointers

* [NOD-1535] Fix condition on updateVirtualDiffParents
2020-11-17 16:00:16 +02:00
Mike Zak
efe1986a56 [NOD-1532] Don't validate coinbase transaction in normal flow 2020-11-17 16:00:16 +02:00
Mike Zak
3ab507b66f [NOD-1532] Use correct coinbase transaction in buildBlockWith Parents 2020-11-17 16:00:16 +02:00
Mike Zak
7d14f24b84 [NOD-1532] Fix some error messages 2020-11-17 16:00:16 +02:00
Ori Newman
fc5e39f6cc [NOD-1535] fix reachability test (#1061)
* Revert "[NOD-1500] Delete integration tests"

This reverts commit fcb57a206690a884fa6afb69d5d493282954a8bf.

* [NOD-1518] hashserialization -> consenusserialization

* [NOD-1518] Fix add genesis to virtual

* [NOD-1518] Fix a bug in SerializeCoinbasePayload.

* [NOD-1518] Fix a loop error and make pastMedianTime behave correctly everywhere on genesis.

* [NOD-1518] Fix another bug and an infinite loop.

* [NOD-1518] Fix uninitialized slice.

* [NOD-1518] Fix bad should-commit checks and another infinite loop.

* [NOD-1518] Fix nil serialization.

* [NOD-1518] Rename blockHash to currentBlockHash.

* [NOD-1518] Move the check whether stagedVirtualUTXOSet != nil to the top of commitVirtualUTXODiff.

* [NOD-1518] Simplify utxoDiffStore.Commit.

* [NOD-1518] Unextract resolveBlockStatusAndCheckFinality.

* [NOD-1518] Move no-transactions logic into CalculateIDMerkleRoot.

* [NOD-1518] Remove redundant is-staged check.

* [NOD-1518] Fix merge errors.

* [NOD-1518] Don't write anything if utxoDiffChild is nil.

* [NOD-1518] Stage virtualAcceptanceData and virtualMultiset.

* [NOD-1518] Fix bugs in getBlockTemplate and submitBlock.

* [NOD-1518] Fix bad validation order in validateHeaderInContext.

* [NOD-1518] Fix bug in Next().

* [NOD-1518] Fix nil dereference of subnetworks in AddressCache.

* [NOD-1518] Fix multisetStore.Get returning a pointer to a multiset that is changed in place.

* [NOD-1518] Break on genesis in countSubtrees.

* [NOD-1518] Fix createBlockLocator.

* [NOD-1518] Fix MsgTxToDomainTransaction.

* [NOD-1518] Set MaxTxVersion to 1.

* [NOD-1518] Fix missing error handling, bug in MsgTxToDomainTransaction, and bad subnetwork equality check.

* [NOD-1518] Fix bug in hasUTXOByOutpointFromStagedVirtualUTXODiff.

* [NOD-1518] Remove irrelevant comments.

* [NOD-1518] Generate transactions with sufficient fee in tx_relay_test.

* [NOD-1518] Fix broken RPC handlers.

* [NOD-1518] Fix merge errors.

* [NOD-1518] Fix bad exists check in restorePastUTXO and missing genesis check in CalculatePastUTXOAndAcceptanceData.

* [NOD-1518] Add a comment.

* [NOD-1518] Use a regular mutex instead of a read-write mutex in consensus to avoid dealing with sneaky not-actually-read functions.

* [NOD-1518] Fix a deadlock in GetVirtualSelectedParent.

* [NOD-1518] Fix missing handler registration for CmdHeader.

* [NOD-1518] Fix processHeader calling OnNewBlock and LogBlock. Also fix conversion errors in IBDRootUTXOSetAndBlock.

* [NOD-1518] Fix bad Command() in MsgIBDRootUTXOSetAndBlock.

* [NOD-1518] Fix bad SyncStateMissingUTXOSet logic in resolveSyncState.

* [NOD-1518] Rename mode to syncState.

* [NOD-1518] Fix headers-only blocks coming in after the consensus thinks it's synced.

* [NOD-1518] Fix selectedChildIterator.Next not ignoring virtual, infinite loop in HashSet.Length().

* [NOD-1518] Fix not-properly wrapped IBD blocks.

* [NOD-1518] Fix bad conversion in RequestIBDBlocks.

* [NOD-1518] Fix bad string for CmdRequestHeaders.

* [NOD-1518] Fix bad string for CmdDoneHeaders.

* [NOD-1518] Fix bad Command() for MsgIBDRootNotFound.

* [NOD-1518] Fix bad areHeaderTipsSyncedMaxTimeDifference value.

* [NOD-1518] Add missing string for CmdRequestIBDBlocks.

* [NOD-1518] Fix bad check for SyncStateMissingBlockBodies.

* [NOD-1518] Fix bad timeout durations in tests.

* [NOD-1518] Fix IBD blocks not calling OnNewBlock.

* [NOD-1518] Change when IBD finishes.

* [NOD-1518] Properly clone utxoDiffChild.

* [NOD-1535] Fix reachability tests

* [NOD-1518] Fix merge errors.

* [NOD-1518] Move call to LogBlock to into OnNewBlock.

* [NOD-1518] Return "not implemented" in unimplemented RPC handlers.

* [NOD-1518] Extract cloning of hashes to a method over DomainHash.

* [NOD-1518] Use isHeaderOnlyBlock.

* [NOD-1518] Use constants.TransactionVersion.

* [NOD-1518] Break immediately if we reached the virtual in SelectedChildIterator.

* [NOD-1518] Don't stage nil utxoDiffChild.

* [NOD-1518] Properly check the genesis hash in CalculatePastUTXOAndAcceptanceData.

* [NOD-1518] Explain why we break on current == nil in countSubtrees.

* [NOD-1518] Add a comment explaining why we check against StatusValid in resolveSyncState.

* [NOD-1535] Add external reachability tests

* [NOD-1535] Fix reachability tests and fix related bugs

* [NOD-1535] Add setters fox reindex slack and window

* [NOD-1535] Remove redundant line

* [NOD-1535] Add comment

* [NOD-1535] Fix comments

* [NOD-1535] Rename DBReader->DatabaseContext

* [NOD-1535] Check that reindex root is changed

* [NOD-1535] Fix calculateNewTips

Co-authored-by: Mike Zak <feanorr@gmail.com>
Co-authored-by: stasatdaglabs <stas@daglabs.com>
2020-11-17 16:00:16 +02:00
Svarog
8ccf381fc7 [NOD-1532] csm unit tests (#1059)
* Revert "[NOD-1500] Delete integration tests"

This reverts commit fcb57a206690a884fa6afb69d5d493282954a8bf.

* [NOD-1518] hashserialization -> consenusserialization

* [NOD-1518] Fix add genesis to virtual

* [NOD-1518] Fix a bug in SerializeCoinbasePayload.

* [NOD-1518] Fix a loop error and make pastMedianTime behave correctly everywhere on genesis.

* [NOD-1518] Fix another bug and an infinite loop.

* [NOD-1518] Fix uninitialized slice.

* [NOD-1518] Fix bad should-commit checks and another infinite loop.

* [NOD-1518] Fix nil serialization.

* [NOD-1518] Rename blockHash to currentBlockHash.

* [NOD-1518] Move the check whether stagedVirtualUTXOSet != nil to the top of commitVirtualUTXODiff.

* [NOD-1518] Simplify utxoDiffStore.Commit.

* [NOD-1518] Unextract resolveBlockStatusAndCheckFinality.

* [NOD-1518] Move no-transactions logic into CalculateIDMerkleRoot.

* [NOD-1518] Remove redundant is-staged check.

* [NOD-1518] Fix merge errors.

* [NOD-1518] Don't write anything if utxoDiffChild is nil.

* [NOD-1518] Stage virtualAcceptanceData and virtualMultiset.

* [NOD-1518] Fix bugs in getBlockTemplate and submitBlock.

* [NOD-1518] Fix bad validation order in validateHeaderInContext.

* [NOD-1518] Fix bug in Next().

* [NOD-1518] Fix nil dereference of subnetworks in AddressCache.

* [NOD-1518] Fix multisetStore.Get returning a pointer to a multiset that is changed in place.

* [NOD-1518] Break on genesis in countSubtrees.

* [NOD-1518] Fix createBlockLocator.

* [NOD-1518] Fix MsgTxToDomainTransaction.

* [NOD-1518] Set MaxTxVersion to 1.

* [NOD-1518] Fix missing error handling, bug in MsgTxToDomainTransaction, and bad subnetwork equality check.

* [NOD-1518] Fix bug in hasUTXOByOutpointFromStagedVirtualUTXODiff.

* [NOD-1518] Remove irrelevant comments.

* [NOD-1518] Generate transactions with sufficient fee in tx_relay_test.

* [NOD-1518] Fix broken RPC handlers.

* [NOD-1518] Fix merge errors.

* [NOD-1518] Fix bad exists check in restorePastUTXO and missing genesis check in CalculatePastUTXOAndAcceptanceData.

* [NOD-1518] Add a comment.

* [NOD-1518] Use a regular mutex instead of a read-write mutex in consensus to avoid dealing with sneaky not-actually-read functions.

* [NOD-1518] Fix a deadlock in GetVirtualSelectedParent.

* [NOD-1518] Fix missing handler registration for CmdHeader.

* [NOD-1518] Fix processHeader calling OnNewBlock and LogBlock. Also fix conversion errors in IBDRootUTXOSetAndBlock.

* [NOD-1518] Fix bad Command() in MsgIBDRootUTXOSetAndBlock.

* [NOD-1518] Fix bad SyncStateMissingUTXOSet logic in resolveSyncState.

* [NOD-1518] Rename mode to syncState.

* [NOD-1518] Fix headers-only blocks coming in after the consensus thinks it's synced.

* [NOD-1518] Fix selectedChildIterator.Next not ignoring virtual, infinite loop in HashSet.Length().

* [NOD-1518] Fix not-properly wrapped IBD blocks.

* [NOD-1532] Add TestMultiset

* [NOD-1518] Fix bad conversion in RequestIBDBlocks.

* [NOD-1518] Fix bad string for CmdRequestHeaders.

* [NOD-1518] Fix bad string for CmdDoneHeaders.

* [NOD-1518] Fix bad Command() for MsgIBDRootNotFound.

* [NOD-1532] Add TestPastUTXOMultiset

* [NOD-1518] Fix bad areHeaderTipsSyncedMaxTimeDifference value.

* [NOD-1532] Added TestDoubleSpends

* [NOD-1518] Add missing string for CmdRequestIBDBlocks.

* [NOD-1518] Fix bad check for SyncStateMissingBlockBodies.

* [NOD-1518] Fix bad timeout durations in tests.

* [NOD-1518] Fix IBD blocks not calling OnNewBlock.

* [NOD-1518] Change when IBD finishes.

* [NOD-1518] Properly clone utxoDiffChild.

* [NOD-1532] Update hashes of blocks

* [NOD-1532] Fix genesis blocks and a few more bugs

* [NOD-1532] Bugfix: incorrect key passed to dbTx.Put

* [NOD-1532] Make sure there's no nil payloads

* [NOD-1532] Fix AddBlockToVirtual

* [NOD-1532] Update tips and virtualDiffParents properly

* [NOD-1532] Allow nil payload

* [NOD-1532] Check for actual error and not just some RuleError

* [NOD-1532] Get rid of SimpleCoinbaseData and make OpTrueScript P2SH

* [NOD-1532] If coinbaseData is nil - fill in with generic coinbaseData

Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: stasatdaglabs <stas@daglabs.com>
2020-11-17 16:00:16 +02:00
stasatdaglabs
eef5e3768c
[NOD-1518] Fix genesis block insertion and integration tests (#1013)
* Revert "[NOD-1500] Delete integration tests"

This reverts commit fcb57a206690a884fa6afb69d5d493282954a8bf.

* [NOD-1518] hashserialization -> consenusserialization

* [NOD-1518] Fix add genesis to virtual

* [NOD-1518] Fix a bug in SerializeCoinbasePayload.

* [NOD-1518] Fix a loop error and make pastMedianTime behave correctly everywhere on genesis.

* [NOD-1518] Fix another bug and an infinite loop.

* [NOD-1518] Fix uninitialized slice.

* [NOD-1518] Fix bad should-commit checks and another infinite loop.

* [NOD-1518] Fix nil serialization.

* [NOD-1518] Rename blockHash to currentBlockHash.

* [NOD-1518] Move the check whether stagedVirtualUTXOSet != nil to the top of commitVirtualUTXODiff.

* [NOD-1518] Simplify utxoDiffStore.Commit.

* [NOD-1518] Unextract resolveBlockStatusAndCheckFinality.

* [NOD-1518] Move no-transactions logic into CalculateIDMerkleRoot.

* [NOD-1518] Remove redundant is-staged check.

* [NOD-1518] Fix merge errors.

* [NOD-1518] Don't write anything if utxoDiffChild is nil.

* [NOD-1518] Stage virtualAcceptanceData and virtualMultiset.

* [NOD-1518] Fix bugs in getBlockTemplate and submitBlock.

* [NOD-1518] Fix bad validation order in validateHeaderInContext.

* [NOD-1518] Fix bug in Next().

* [NOD-1518] Fix nil dereference of subnetworks in AddressCache.

* [NOD-1518] Fix multisetStore.Get returning a pointer to a multiset that is changed in place.

* [NOD-1518] Break on genesis in countSubtrees.

* [NOD-1518] Fix createBlockLocator.

* [NOD-1518] Fix MsgTxToDomainTransaction.

* [NOD-1518] Set MaxTxVersion to 1.

* [NOD-1518] Fix missing error handling, bug in MsgTxToDomainTransaction, and bad subnetwork equality check.

* [NOD-1518] Fix bug in hasUTXOByOutpointFromStagedVirtualUTXODiff.

* [NOD-1518] Remove irrelevant comments.

* [NOD-1518] Generate transactions with sufficient fee in tx_relay_test.

* [NOD-1518] Fix broken RPC handlers.

* [NOD-1518] Fix merge errors.

* [NOD-1518] Fix bad exists check in restorePastUTXO and missing genesis check in CalculatePastUTXOAndAcceptanceData.

* [NOD-1518] Add a comment.

* [NOD-1518] Use a regular mutex instead of a read-write mutex in consensus to avoid dealing with sneaky not-actually-read functions.

* [NOD-1518] Fix a deadlock in GetVirtualSelectedParent.

* [NOD-1518] Fix missing handler registration for CmdHeader.

* [NOD-1518] Fix processHeader calling OnNewBlock and LogBlock. Also fix conversion errors in IBDRootUTXOSetAndBlock.

* [NOD-1518] Fix bad Command() in MsgIBDRootUTXOSetAndBlock.

* [NOD-1518] Fix bad SyncStateMissingUTXOSet logic in resolveSyncState.

* [NOD-1518] Rename mode to syncState.

* [NOD-1518] Fix headers-only blocks coming in after the consensus thinks it's synced.

* [NOD-1518] Fix selectedChildIterator.Next not ignoring virtual, infinite loop in HashSet.Length().

* [NOD-1518] Fix not-properly wrapped IBD blocks.

* [NOD-1518] Fix bad conversion in RequestIBDBlocks.

* [NOD-1518] Fix bad string for CmdRequestHeaders.

* [NOD-1518] Fix bad string for CmdDoneHeaders.

* [NOD-1518] Fix bad Command() for MsgIBDRootNotFound.

* [NOD-1518] Fix bad areHeaderTipsSyncedMaxTimeDifference value.

* [NOD-1518] Add missing string for CmdRequestIBDBlocks.

* [NOD-1518] Fix bad check for SyncStateMissingBlockBodies.

* [NOD-1518] Fix bad timeout durations in tests.

* [NOD-1518] Fix IBD blocks not calling OnNewBlock.

* [NOD-1518] Change when IBD finishes.

* [NOD-1518] Properly clone utxoDiffChild.

* [NOD-1518] Fix merge errors.

* [NOD-1518] Move call to LogBlock to into OnNewBlock.

* [NOD-1518] Return "not implemented" in unimplemented RPC handlers.

* [NOD-1518] Extract cloning of hashes to a method over DomainHash.

* [NOD-1518] Use isHeaderOnlyBlock.

* [NOD-1518] Use constants.TransactionVersion.

* [NOD-1518] Break immediately if we reached the virtual in SelectedChildIterator.

* [NOD-1518] Don't stage nil utxoDiffChild.

* [NOD-1518] Properly check the genesis hash in CalculatePastUTXOAndAcceptanceData.

* [NOD-1518] Explain why we break on current == nil in countSubtrees.

* [NOD-1518] Add a comment explaining why we check against StatusValid in resolveSyncState.

Co-authored-by: Mike Zak <feanorr@gmail.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2020-11-12 15:19:39 +02:00
Ori Newman
4736213ba4
[NOD-1528] Make data stores copy data on stage (#1020)
* [NOD-1528] Make data stores copy data on stage

* [NOD-1528] Add proto objects to serialize consensus state objects

* [NOD-1528] Fix receiver names

* [NOD-1528] Add copy to block store and utxo diff staging

* [NOD-1528] Return errors where needed
2020-11-10 18:32:42 +02:00
Ori Newman
23c1ea6c31
[NOD-1525] Implement headers first ibd (#1017)
* [NOD-1525] Implement headers first IBD

* [NOD-1525] Fix proto translators

* [NOD-1525] Register missing flows

* [NOD-1525] Rename SyncStateNormal->SyncStateRelay, simplifiy IBD peer selection code and get rid of panic in FinishIBD

* [NOD-1525] Remove redundant methods from interface
2020-11-10 16:14:51 +02:00
Svarog
6db337c8c5
[NOD-1519] Add TestAPI for BuildBlockWithParents (#1011)
* [NOD-1519] Separate BlockBuilder and BlockProcessor

* [NOD-1519] Wire blockBuilder properly + implement buildBlockWithParents

* [NOD-1519] Added testapi package, TestConsensus interface, and TestConsensus factory

* [NOD-1519] Add comments

* [NOD-1519] Separate TestBlockBuilder out of BlockBuilder

* [NOD-1519] TestBlockBuilder should also implement BlockBuilder

* [NOD-1519] Add NewTestConsensus to factory interface
2020-11-09 12:02:42 +02:00
Svarog
281944762d
[NOD-1500] Glue between domain and application (#1007)
* [NOD-1500] Added Domain type and Constructor

* [NOD-1500] Replaced dag+txpool with domain in flowContext

* [NOD-1500] Replaced dag+txpool with domain in flowContext

* [NOD-1500] Converters: domain objects from/to appmessage

* [NOD-1500] Convert hashes to DomainHashes in appmessages

* [NOD-1500] Remove references to daghash in dagconfig

* [NOD-1500] Fixed all appmessage usages of hashes

* [NOD-1500] Update all RPC to use domain

* [NOD-1500] Big chunk of protocol flows re-wired to domain

* [NOD-1500] Finished re-wiring all protocol flows to new Domain

* [NOD-1500] Fix some mempool and kaspaminer compilation errors

* [NOD-1500] Deleted util/{block,tx,daghash} and dbaccess

* [NOD-1500] util.CoinbaseTransactionIndex -> transactionhelper.CoinbaseTransactionIndex

* [NOD-1500] Fix txsigner

* [NOD-1500] Removed all references to util/subnetworkid

* [NOD-1500] Update RpcGetBlock related messages

* [NOD-1500] Many more compilation fixes

* [NOD-1500] Return full list of missing blocks for orphan resolution

* [NOD-1500] Fixed handshake

* [NOD-1500] Fixed flowcontext compilation

* [NOD-1500] Update users of StartIBDIfRequired to handle error

* [NOD-1500] Removed some more fields from RPC

* [NOD-1500] Fix the getBlockTemplate flow

* [NOD-1500] Fix HandleGetCurrentNetwork

* [NOD-1500] Remove redundant code

* [NOD-1500] Remove obsolete notifications

* [NOD-1500] Split MiningManager and Consensus to separate fields in Domain

* [NOD-1500] Update two wrong references to location of txscript

* [NOD-1500] Added comments

* [NOD-1500] Fix some tests

* [NOD-1500] Removed serialization logic from appmessage

* [NOD-1500] Rename database/serialization/messages.proto to dbobjects.proto

* [NOD-1500] Delete integration tests

* [NOD-1500] Remove txsort

* [NOD-1500] Fix tiny bug

* [NOD-1500] Remove rogue dependancy on bchd

* [NOD-1500] Some stylistic fixes
2020-11-08 11:55:54 +02:00
Ori Newman
215ab512cd
[NOD-1515] Allow to process pruning point while in SyncStateMissingUTXOSet (#1005) 2020-11-05 12:39:39 +02:00
Ori Newman
5566aaf95a
[NOD-1512] Implement utxo deserialization (#1003)
* [NOD-1512] Implement UTXO set deserialization

* [NOD-1512] Remove redundant file

* [NOD-1512] Don't use big endian for serialization

* [NOD-1512] Use Read/Write element

* [NOD-1512] Unexport ReadElement

* [NOD-1512] Fix StageVirtualUTXOSet

* [NOD-1512] Get rid of dagParams in consensusStateManager

* [NOD-1512] Get rid of dagParams in consensusStateManager
2020-11-05 10:59:49 +02:00
Ori Newman
52c73d3a08
[NOD-1511] Implement missingBlockBodyHashes (#1000)
* [NOD-1511] Implement missingBlockBodyHashes

* [NOD-1511] Rename selectedparentiterator.go to blockiterator.go

* [NOD-1511] Fix condition

* [NOD-1511] Simplify missingBlocks logic
2020-11-05 10:11:42 +02:00
Ori Newman
baf8d25656
[NOD-1510] Update reachability reindex root based on header tips selected tip (#1001) 2020-11-04 16:55:59 +02:00
Elichai Turkel
6a46cb2be6
[NOD-1503] Pruning Manager (#994)
* Update pruningmanager interface

* Add a ProtoUTXOSet to hashserialization

* Update miningmanager with all the necessary stores and managers

* Implement mining manager

* Prune P.AC not in V.Past

* PruningManager fix all review comments
2020-11-04 10:29:45 +02:00
Ori Newman
7f2ef708a6
[NOD-1506] Implement SetPruningPointUTXOSet (#996)
* [NOD-1506] Implement SetPruningPointUTXOSet

* [NOD-1506] Rename ErrHeaderlessBlockInIBD->ErrMissingBlockHeaderInIBD

* [NOD-1506] Change virtualHeaderHash
2020-11-03 18:24:45 +02:00
Svarog
8dc246a2a7
[NOD-1498] Consensus State Store (#992)
* [NOD-1420] Start working on ConsensusStateManager. Might be redundant due to recent changes

* [NOD-1420] Convert model to externalapi in utxo_algerbra helpers

* [NOD-1420] Add UTXO-diff algebra

* [NOD-1420] Prepare skeleton of calculateAcceptanceDataAndMultiset

* [NOD-1420] Added skeleton for AddBlockToVirtual

* [NOD-1420] Implement PopulateTransactionWithUTXOEntries

* [NOD-1420] Implement restorePastUTXO

* [NOD-1420] Implement finality check

* [NOD-1420] Move handling of tips to consensusStateManager

* [NOD-1420] Implement calculateAcceptanceDataAndMultiset

* [NOD-1420] Start implementing resolveBlockStatus

* [NOD-1420] Implement resolveBlockStatus

* [NOD-1420] Update related fields in end of resolveSingleBlockStatus

* [NOD-1420] Start working on selectVirtualParents

* [NOD-1420] Implemented BlockHeap

* [NOD-1420] Implement selectVirtualParents

* [NOD-1420] Implement updateVirtual

* [NOD-1420] Added comments where they were missing

* [NOD-1420] Place all consensusStateManager functions in correct files

* [NOD-1420] Return the missing outpoints from populateTransactionWithUTXOEntriesFromVirtualOrDiff

* [NOD-1420] Outpoint.ID -> TransactionID

* [NOD-1420] Fix Stringer tests

* [NOD-1420] Copy hash.FromString into utils

* [NOD-1420] SetParents should return an error

* [NOD-1420] Remove all reachabilityManager references from consensusStateManager

* [NOD-1420] Remove VirtualData. Get the info from the stores where needed

* [NOD-1420] Invert parameters to IsAncestorOf

* [NOD-1420] Use model.AcceptanceData

* [NOD-1420] Don't return accumulatedMassBefore in error cases

* [NOD-1420] Don't expect store functions to return nil when the requested data was found - instead add HasXXX functions

* [NOD-1420] addTransactionToMultiset sets isCoinbase properly

* [NOD-1420] expected hash string length is externalapi.DomainHashSize * 2

* [NOD-1420] Rename reachabilityTree -> reachabilityManager + updateReindexRoot if isNextVirtualSelectedParent

* [NOD-1420] ValidateCoinbaseTransaction in csm.verifyAndBuildUTXO

* [NOD-1420] Re-write HAsUTXODiffChild

* [NOD-1420] delete past_utxo.go.bak

* [NOD-1420] Implement validateCoinbaseTransaction in CSM

* [NOD-1420] Imlemented missing functionality in ValidateTransactionAndPopulateWithConsensusData

* [NOD-1420] Moved merge depth logic to MergeDepthManager

* [NOD-1420] Add logs

* [NOD-1498] Implement tips-related methods of consensusStateStore

* [NOD-1498] Implement consensusStateStore virtualDiffParents functionality

* [NOD-1498] Implement ConsensusStateStore UTXO-Set part

* [NOD-1498] Implement rest of consensusStateStore methods

* [NOD-1498] Use io.ReadFull instead of r.Read

* [NOD-1498] Added comments

* [NOD-1498] Move utxo serialization to protobufs

* [NOD-1498] Add comments

* [NOD-1498] Minor fixes in ConsensusStateStore

* [NOD-1498] Use empty bucket key + simplify serializeUTXOEntry
2020-11-03 14:42:26 +02:00
Ori Newman
c7f2de73df
[NOD-1502] Implement RestorePastUTXOSetIterator (#993)
* [NOD-1502] Implement RestorePastUTXOSetIterator

* [NOD-1502] Rename newUtxoSetIterator->newUTXOSetIterator
2020-11-02 18:36:55 +02:00
Svarog
c5707f64dc
[NOD-1420] Implement consensusStateManager (#985)
* [NOD-1420] Start working on ConsensusStateManager. Might be redundant due to recent changes

* [NOD-1420] Convert model to externalapi in utxo_algerbra helpers

* [NOD-1420] Add UTXO-diff algebra

* [NOD-1420] Prepare skeleton of calculateAcceptanceDataAndMultiset

* [NOD-1420] Added skeleton for AddBlockToVirtual

* [NOD-1420] Implement PopulateTransactionWithUTXOEntries

* [NOD-1420] Implement restorePastUTXO

* [NOD-1420] Implement finality check

* [NOD-1420] Move handling of tips to consensusStateManager

* [NOD-1420] Implement calculateAcceptanceDataAndMultiset

* [NOD-1420] Start implementing resolveBlockStatus

* [NOD-1420] Implement resolveBlockStatus

* [NOD-1420] Update related fields in end of resolveSingleBlockStatus

* [NOD-1420] Start working on selectVirtualParents

* [NOD-1420] Implemented BlockHeap

* [NOD-1420] Implement selectVirtualParents

* [NOD-1420] Implement updateVirtual

* [NOD-1420] Added comments where they were missing

* [NOD-1420] Place all consensusStateManager functions in correct files

* [NOD-1420] Return the missing outpoints from populateTransactionWithUTXOEntriesFromVirtualOrDiff

* [NOD-1420] Outpoint.ID -> TransactionID

* [NOD-1420] Fix Stringer tests

* [NOD-1420] Copy hash.FromString into utils

* [NOD-1420] SetParents should return an error

* [NOD-1420] Remove all reachabilityManager references from consensusStateManager

* [NOD-1420] Remove VirtualData. Get the info from the stores where needed

* [NOD-1420] Invert parameters to IsAncestorOf

* [NOD-1420] Use model.AcceptanceData

* [NOD-1420] Don't return accumulatedMassBefore in error cases

* [NOD-1420] Don't expect store functions to return nil when the requested data was found - instead add HasXXX functions

* [NOD-1420] addTransactionToMultiset sets isCoinbase properly

* [NOD-1420] expected hash string length is externalapi.DomainHashSize * 2

* [NOD-1420] Rename reachabilityTree -> reachabilityManager + updateReindexRoot if isNextVirtualSelectedParent

* [NOD-1420] ValidateCoinbaseTransaction in csm.verifyAndBuildUTXO

* [NOD-1420] Re-write HAsUTXODiffChild

* [NOD-1420] delete past_utxo.go.bak

* [NOD-1420] Implement validateCoinbaseTransaction in CSM

* [NOD-1420] Imlemented missing functionality in ValidateTransactionAndPopulateWithConsensusData

* [NOD-1420] Moved merge depth logic to MergeDepthManager

* [NOD-1420] Add logs
2020-11-02 16:18:53 +02:00
stasatdaglabs
62bb841e89
[NOD-1497] Add additional methods to consensus' external API (#991)
* [NOD-1497] Add missing APIs.

* [NOD-1497] Rename some new APIs.

* [NOD-1497] Implement getBlock and getBlockHeader.

* [NOD-1497] Implement getPruningPointUTXOSet.

* [NOD-1497] Implement getSelectedParent.

* [NOD-1497] Implement getBlockInfo.

* [NOD-1497] Fix merge errors.

* [NOD-1497] Implement syncManager-related functions in consensus.

* [NOD-1497] Implement SetPruningPointUTXOSet in consensus.

* [NOD-1497] Add dependency from syncManager to dagTraversalManager.

* [NOD-1497] Move IsBlockHeaderInPruningPointFutureAndVirtualPast to syncManager.

* [NOD-1497] Rename lowHigh to lowHash.
2020-11-02 13:24:03 +02:00