12 Commits

Author SHA1 Message Date
Svarog
c13a4d90ed
Mempool redesign (#1752)
* Added model and stubs for all main methods

* Add constructors to all main objects

* Implement BlockCandidateTransactions

* implement expireOldTransactions and expireOrphanTransactions

* Rename isHighPriority to neverExpires

* Add stub for checkDoubleSpends

* Revert "Rename isHighPriority to neverExpires"

This reverts commit b2da9a4a00c02fb380d2518cf54fa16257bd8423.

* Imeplement transactionsOrderedByFeeRate

* Orphan maps should be idToOrphan

* Add error.go to mempool

* Invert the condition for banning when mempool rejects a transaction

* Move all model objects to model package

* Implement getParentsInPool

* Implemented mempoolUTXOSet.addTransaction

* Implement removeTransaction, remove sanity checks

* Implemented mempoolUTXOSet.checkDoubleSpends

* Implemented removeOrphan

* Implement removeOrphan

* Implement maybeAddOrphan and AddOrphan

* Implemented processOrphansAfterAcceptedTransaction

* Implement transactionsPool.addTransaction

* Implement RemoveTransaction

* If a transaction was removed from the mempool - update it's redeemers in orphan pool as well

* Use maximumOrphanTransactionCount

* Add allowOrphans to ValidateAndInsertTransaction stub

* Implement validateTransaction functions

* Implement fillInputs

* Implement ValidateAndInsertTransaction

* Implement HandleNewBlockTransactions

* Implement missing mempool interface methods

* Add comments to exported functions

* Call ValidateTransactionInIsolation where needed

* Implement RevalidateHighPriorityTransactions

* Rewire kaspad to use new mempool, and fix compilation errors

* Update rebroadcast logic to use new structure

* Handle non-standard transaction errors properly

* Add mutex to mempool

* bugfix: GetTransaction panics when ok is false

* properly calculate targetBlocksPerSecond in config.go

* Fix various lint errors and tests

* Fix expected text in test for duplicate transactions

* Skip the coinbase transaction in HandleNewBlockTransactions

* Unorphan the correct transactions

* Call ValidateTransactionAndPopulateWithConsensusData on unorphanTransaction

* Re-apply policy_test as check_transactions_standard_test

* removeTransaction: Remove redeemers in orphan pool as well

* Remove redundant check for uint64 < 0

* Export and rename isDust -> IsTransactionOutputDust to allow usage by rothschild

* Add allowOrphan to SubmitTransaction RPC request

* Remove all implementation from mempool.go

* tidy go mod

* Don't pass acceptedOrphans to handleNewBlockTransactions

* Use t.Errorf in stead of t.Fatalf

* Remove minimum relay fee from TestDust, as it's no longer configurable

* Add separate VirtualDAASCore method for faster retrieval where it's repeated multiple times

* Broadcast all transactions that were accepted

* Don't re-use GetVirtualDAAScore in GetVirtualInfo - this causes a deadlock

* Use real transaction count, and not Orphan

* Get mempool config from outside, incorporating values received from cli

* Use MinRelayFee and MaxOrphanTxs from global kaspad config

* Add explanation for the seemingly redundant check for transaction version in checkTransactionStandard

* Update some comment

* Convert creation of acceptedTransactions to a single line

* Move mempoolUTXOSet out of checkDoubleSpends

* Add test for attempt to insert double spend into mempool

* fillInputs: Skip check for coinbase - it's always false in mempool

* Clarify comment about removeRedeemers when removing random orphan

* Don't remove high-priority transactions in limitTransactionCount

* Use mempool.removeTransaction in limitTransactionCount

* Add mutex comment to handleNewBlockTransactions

* Return error from limitTransactionCount

* Pluralize the map types

* mempoolUTXOSet.removeTransaction: Don't restore utxo if it was not created in mempool

* Don't evacuate from orphanPool high-priority transactions

* Disallow double-spends in orphan pool

* Don't use exported (and locking) methods from inside mempool

* Check for double spends in mempool during revalidateTransaction

* Add checkOrphanDuplicate

* Add orphan to acceptedOrphans, not current

* Add TestHighPriorityTransactions

* Fix off-by-one error in limitTransactionCount

* Add TestRevalidateHighPriorityTransactions

* Remove checkDoubleSpends from revalidateTransaction

* Fix TestRevalidateHighPriorityTransactions

* Move check for MaximumOrphanCount to beggining of maybeAddOrphan

* Rename all map type to singulateToSingularMap

* limitOrphanPool only after the orphan was added

* TestDoubleSpendInMempool: use createChildTxWhenParentTxWasAddedByConsensus instead of createTransactionWithUTXOEntry

* Fix some comments

* Have separate min/max transaction versions for mempool

* Add comment on defaultMaximumOrphanTransactionCount to keep it small as long as we have recursion

* Fix comment

* Rename: createChildTxWhenParentTxWasAddedByConsensus -> createChildTxWhereParentTxWasAddedByConsensus

* Handle error from createChildTxWhereParentTxWasAddedByConsensus

* Rename createChildTxWhereParentTxWasAddedByConsensus -> createChildAndParentTxsAndAddParentToConsensus

* Convert all MaximumXXX constants to uint64

* Add comment

* remove mutex comments
2021-06-23 15:49:20 +03:00
Ori Newman
b8413fcecb
Add the mempool size to getInfo RPC command (#1584)
* Add the mempool size to getInfo RPC command

* Add mempool.Len()

* Rename mempool.Len() to mempool.TransactionCount()

Co-authored-by: Svarog <feanorr@gmail.com>
2021-03-09 12:48:33 +02:00
Svarog
3f08bf87a8
Fix mempool.RemoveTransactions (#1551)
* Fix mempool.RemoveTransactions

* Don't crash if mempool.RemoveTransactions returns an error. log.Critical instead
2021-02-24 12:47:21 +02:00
stasatdaglabs
d9b97afb92
Don't swallow errors in HandleNewBlockTransactions. (#1390)
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-01-11 17:16:15 +02:00
Ori Newman
4ad89056c2
Implement getMempoolEntries (#1369)
* Implement getMempoolEntries

* Fix comment

* Fix comment
2021-01-06 17:26:21 +02:00
Ori Newman
5b2fae0457
[NOD-1568] Add staticcheck checks (#1150) 2020-11-25 11:43:51 +02:00
stasatdaglabs
eef5e3768c
[NOD-1518] Fix genesis block insertion and integration tests (#1013)
* Revert "[NOD-1500] Delete integration tests"

This reverts commit fcb57a206690a884fa6afb69d5d493282954a8bf.

* [NOD-1518] hashserialization -> consenusserialization

* [NOD-1518] Fix add genesis to virtual

* [NOD-1518] Fix a bug in SerializeCoinbasePayload.

* [NOD-1518] Fix a loop error and make pastMedianTime behave correctly everywhere on genesis.

* [NOD-1518] Fix another bug and an infinite loop.

* [NOD-1518] Fix uninitialized slice.

* [NOD-1518] Fix bad should-commit checks and another infinite loop.

* [NOD-1518] Fix nil serialization.

* [NOD-1518] Rename blockHash to currentBlockHash.

* [NOD-1518] Move the check whether stagedVirtualUTXOSet != nil to the top of commitVirtualUTXODiff.

* [NOD-1518] Simplify utxoDiffStore.Commit.

* [NOD-1518] Unextract resolveBlockStatusAndCheckFinality.

* [NOD-1518] Move no-transactions logic into CalculateIDMerkleRoot.

* [NOD-1518] Remove redundant is-staged check.

* [NOD-1518] Fix merge errors.

* [NOD-1518] Don't write anything if utxoDiffChild is nil.

* [NOD-1518] Stage virtualAcceptanceData and virtualMultiset.

* [NOD-1518] Fix bugs in getBlockTemplate and submitBlock.

* [NOD-1518] Fix bad validation order in validateHeaderInContext.

* [NOD-1518] Fix bug in Next().

* [NOD-1518] Fix nil dereference of subnetworks in AddressCache.

* [NOD-1518] Fix multisetStore.Get returning a pointer to a multiset that is changed in place.

* [NOD-1518] Break on genesis in countSubtrees.

* [NOD-1518] Fix createBlockLocator.

* [NOD-1518] Fix MsgTxToDomainTransaction.

* [NOD-1518] Set MaxTxVersion to 1.

* [NOD-1518] Fix missing error handling, bug in MsgTxToDomainTransaction, and bad subnetwork equality check.

* [NOD-1518] Fix bug in hasUTXOByOutpointFromStagedVirtualUTXODiff.

* [NOD-1518] Remove irrelevant comments.

* [NOD-1518] Generate transactions with sufficient fee in tx_relay_test.

* [NOD-1518] Fix broken RPC handlers.

* [NOD-1518] Fix merge errors.

* [NOD-1518] Fix bad exists check in restorePastUTXO and missing genesis check in CalculatePastUTXOAndAcceptanceData.

* [NOD-1518] Add a comment.

* [NOD-1518] Use a regular mutex instead of a read-write mutex in consensus to avoid dealing with sneaky not-actually-read functions.

* [NOD-1518] Fix a deadlock in GetVirtualSelectedParent.

* [NOD-1518] Fix missing handler registration for CmdHeader.

* [NOD-1518] Fix processHeader calling OnNewBlock and LogBlock. Also fix conversion errors in IBDRootUTXOSetAndBlock.

* [NOD-1518] Fix bad Command() in MsgIBDRootUTXOSetAndBlock.

* [NOD-1518] Fix bad SyncStateMissingUTXOSet logic in resolveSyncState.

* [NOD-1518] Rename mode to syncState.

* [NOD-1518] Fix headers-only blocks coming in after the consensus thinks it's synced.

* [NOD-1518] Fix selectedChildIterator.Next not ignoring virtual, infinite loop in HashSet.Length().

* [NOD-1518] Fix not-properly wrapped IBD blocks.

* [NOD-1518] Fix bad conversion in RequestIBDBlocks.

* [NOD-1518] Fix bad string for CmdRequestHeaders.

* [NOD-1518] Fix bad string for CmdDoneHeaders.

* [NOD-1518] Fix bad Command() for MsgIBDRootNotFound.

* [NOD-1518] Fix bad areHeaderTipsSyncedMaxTimeDifference value.

* [NOD-1518] Add missing string for CmdRequestIBDBlocks.

* [NOD-1518] Fix bad check for SyncStateMissingBlockBodies.

* [NOD-1518] Fix bad timeout durations in tests.

* [NOD-1518] Fix IBD blocks not calling OnNewBlock.

* [NOD-1518] Change when IBD finishes.

* [NOD-1518] Properly clone utxoDiffChild.

* [NOD-1518] Fix merge errors.

* [NOD-1518] Move call to LogBlock to into OnNewBlock.

* [NOD-1518] Return "not implemented" in unimplemented RPC handlers.

* [NOD-1518] Extract cloning of hashes to a method over DomainHash.

* [NOD-1518] Use isHeaderOnlyBlock.

* [NOD-1518] Use constants.TransactionVersion.

* [NOD-1518] Break immediately if we reached the virtual in SelectedChildIterator.

* [NOD-1518] Don't stage nil utxoDiffChild.

* [NOD-1518] Properly check the genesis hash in CalculatePastUTXOAndAcceptanceData.

* [NOD-1518] Explain why we break on current == nil in countSubtrees.

* [NOD-1518] Add a comment explaining why we check against StatusValid in resolveSyncState.

Co-authored-by: Mike Zak <feanorr@gmail.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2020-11-12 15:19:39 +02:00
Svarog
281944762d
[NOD-1500] Glue between domain and application (#1007)
* [NOD-1500] Added Domain type and Constructor

* [NOD-1500] Replaced dag+txpool with domain in flowContext

* [NOD-1500] Replaced dag+txpool with domain in flowContext

* [NOD-1500] Converters: domain objects from/to appmessage

* [NOD-1500] Convert hashes to DomainHashes in appmessages

* [NOD-1500] Remove references to daghash in dagconfig

* [NOD-1500] Fixed all appmessage usages of hashes

* [NOD-1500] Update all RPC to use domain

* [NOD-1500] Big chunk of protocol flows re-wired to domain

* [NOD-1500] Finished re-wiring all protocol flows to new Domain

* [NOD-1500] Fix some mempool and kaspaminer compilation errors

* [NOD-1500] Deleted util/{block,tx,daghash} and dbaccess

* [NOD-1500] util.CoinbaseTransactionIndex -> transactionhelper.CoinbaseTransactionIndex

* [NOD-1500] Fix txsigner

* [NOD-1500] Removed all references to util/subnetworkid

* [NOD-1500] Update RpcGetBlock related messages

* [NOD-1500] Many more compilation fixes

* [NOD-1500] Return full list of missing blocks for orphan resolution

* [NOD-1500] Fixed handshake

* [NOD-1500] Fixed flowcontext compilation

* [NOD-1500] Update users of StartIBDIfRequired to handle error

* [NOD-1500] Removed some more fields from RPC

* [NOD-1500] Fix the getBlockTemplate flow

* [NOD-1500] Fix HandleGetCurrentNetwork

* [NOD-1500] Remove redundant code

* [NOD-1500] Remove obsolete notifications

* [NOD-1500] Split MiningManager and Consensus to separate fields in Domain

* [NOD-1500] Update two wrong references to location of txscript

* [NOD-1500] Added comments

* [NOD-1500] Fix some tests

* [NOD-1500] Removed serialization logic from appmessage

* [NOD-1500] Rename database/serialization/messages.proto to dbobjects.proto

* [NOD-1500] Delete integration tests

* [NOD-1500] Remove txsort

* [NOD-1500] Fix tiny bug

* [NOD-1500] Remove rogue dependancy on bchd

* [NOD-1500] Some stylistic fixes
2020-11-08 11:55:54 +02:00
Elichai Turkel
87ad9dfc59
[NOD-1423] Refactor the miner and mempool (#981)
* Make TransactionOutputEstimatedSerializedSize public

* Update the mempool interface

* Refactor the mempool to the new design

* refactor txselection and blocktemplatebuilder to the new design

* Update the mining manager

* Update the MiningManager factory

* mempool fix requested changed
2020-11-01 18:27:49 +02:00
stasatdaglabs
8c63835971
[NOD-1461] Make further design changes (#959)
* [NOD-1461] Split blockValidator and TransactionValidator.

* [NOD-1461] Remove feeDataStore.

* [NOD-1461] Move tips out of ConsensusStateManager and into DAGTopologyManager.

* [NOD-1461] Add UTXODiffManager.

* [NOD-1461] Add RestoreDiffFromVirtual.

* [NOD-1461] Add AcceptanceManager.

* [NOD-1461] Replace SetTips with AddTip.

* [NOD-1461] Fix merge errors.

* [NOD-1461] Rename CoinbaseData to DomainCoinbaseData.
2020-10-20 09:35:58 +03:00
stasatdaglabs
a96a5fd2ef
[NOD-1462] Simplify consensus external API (#958)
* [NOD-1461] Change the external api interface to not having anything besides DomainTransactions and DomainBlocks.

* [NOD-1462] Move external api types to a separate package.

* [NOD-1462] Clarify which model we're using in miningmanager.

* [NOD-1462] Extract coinbase data to its own struct.

* [NOD-1462] Add a comment above CoinbaseData.

* [NOD-1462] Fix the comment above CoinbaseData.
2020-10-19 17:59:04 +03:00
stasatdaglabs
db475bd511
[NOD-1460] Make the miningmanager package structure similar to consensus package's (#957)
* [NOD-1460] Move the miningmanager interfaces into its model package.

* [NOD-1460] Decouple miningmanager model from appmessage.

* [NOD-1460] Decouple miningmanager model from util.

* [NOD-1460] Make miningmanager implementation structs unexported.
2020-10-18 10:52:41 +03:00