* protobuf for new block template notification structs
* appmessage and wire for new block template notification structs
* Set up the entire handler/call-chain for managing the new-block-template event
* Fix a bug in the case where syncer chain is fully known to syncee
* Extract chain negotiation to a separated method
* Bump version and update the changelog
* Add zoom-in progress validation and some debug logs
* Improved error explanation
* go fmt
* Validate zoom-in progress through a total count
* a patch for fixing p2p v4 IBD issues for all side-chains
* Perform side-chain check earlier to avoid IBD start
* A few comments explaining the IBD patch
* Improve output of non-critical protocol errors to avoid user panic
* Add log messages at the end of IBD with headers proof
* Found a case where this was falsely triggered due to the wrong equality test
* The first message is expected to contain headers and not a "done" message (+comment and error text fixes)
* Dequeue w/o timeout during pp anticone batch processing
* Add a verification step for catching possible new IBD errors
* Fetch missing bodies for both, syncer selected tip past and relay block past
* Make sure the syncer is behaving correctly to avoid out of index errors
* Make sure progress reporter does not exceed 100%
* No orphan roots, so no need to queue the empty list
* Add a log to report utxo fetch failure with err message
* A duplicate blocks should not appear as a warning
* typo
* add p2p v5 which is currently identical to v4
* set all internal imports to v5
* set default version to 5
* Send pruning point and its anticone in batches
* go lint
* Fix jsom format
* Use DequeueWithTimeout
* Assert that batch size < route capacity
* oops, this is a flow handler, by definition it needs to be w/o a timeout
* here however, a timeout is required
* Keep IDs of prev messages unmodified
* previous merge operation accidentally erased an important part of this pr
* Extend timeout of simple sync
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* add p2p v5 which is currently identical to v4
* set all internal imports to v5
* wip
* set default version to 5
* protobuf gen for new ibd chain locator
* wire for new ibd chain locator types
* new ibd shared block algo -- only basic test passing
* address the case where pruning points disagree, now both IBD tests pass
* protobuf gen for new past diff request message
* wire for new request past diff message
* handle and flow for new request past diff message - logic unimplemented yet
* implement ibd sync past diff of relay and selected tip
* go fmt
* remove unused methods
* missed one err check
* addressing simple comments
* apply the traversal limit logic and sort headers
* rename pastdiff -> anticone
* apply Don't relay blocks in virtual anticone #1970 to v5
* go fmt
* Fixed minor comments
* Limit the number of chain negotiation restarts
* Report progress percentage when downloading headers in IBD.
* Extract reporting logic to a separate type.
* Report progress for IBD missing block bodies.
* Separate IBD to a new flow (so now invs are handled concurrently and no route capacity errors)
* Invs messages should be queued while waiting for BlockLocator msg
* Close IBD channel so that HandleIBDFlow exits too
* Apply flow separation to p2p protocol v4
* Manage the IBDRequestChannel through the Peer struct
* Some IBDs take a little longer
* Copy blockrelay flows to v4
* Remove duplicate sending of the same DAA blocks
* Advance testnet version
* Renames and add comments
* Add IBD test between v3 and v4
* Fix syncee v4 p2p version
* Check if netsync finished with selected tip
* Implement upgrade mechanism for p2p
* Remove dependencies from flowcontext to v3
* Add p2p v4
* Add Ready flow
* Remove copy paste code of v3
* Register SendAddresses flow at the top level
* Add option to set protocol version from CLI and add TestAddressExchangeV3V4
* Send ready message on minimal net adapter
* Rename defaultMaxProtocolVersion->maxAcceptableProtocolVersion
* Update to version v0.11.8
* Call OnPruningPointUTXOSetOverride after committing the staging consensus
* use domain.Consensus() in utxo index so it'll always check the right consensus
* Get rid of genesis's UTXO dump
* Allow known orphans when AllowSubmitBlockWhenNotSynced=true
* gofmt
* Avoid IBD without changing the pruning point when only genesis is available
* Add DisallowDirectBlocksOnTopOfGenesis=true for mainnet
* Remove any mention to nobanning to let stability tests run
* Rename ifGenesisSetUtxoSet to loadUTXODataForGenesis
Co-authored-by: Ori Newman <>
* Modify DefaultTimeout to 120 seconds
A temporary workaround for nodes having trouble to sync (currently the download of pruning point related data during IBD takes more than 30 seconds)
* Cache existence in reachability store
* Cache block level in the header
* Fix IBD indication on submit block
* Add hardForkOmitGenesisFromParentsDAAScore logic
* Fix NumThreads bug in the wallet
* Get rid of ParentsAtLevel header method
* Fix a bug in BuildPruningPointProof
* Increase race detector timeout
* Add cache to BuildPruningPointProof
* Add comments and temp comment out go vet
* Fix ParentsAtLevel
* Dont fill empty parents
* Change HardForkOmitGenesisFromParentsDAAScore in fast netsync test
* Add --allow-submit-block-when-not-synced in stability tests
* Fix TestPruning
* Return fast tests
* Fix off by one error on kaspawallet
* Fetch only one block with trusted data at a time
* Update fork DAA score
* Don't ban for unexpected message type
* Fix tests
Co-authored-by: Michael Sutton <mikisiton2@gmail.com>
Co-authored-by: Ori Newman <>
* Send pruning point anticone in topological order
Fix a UTXO pagination bug
Lengthen the stabilization time for the last DAA test
* Extend "sudden hash rate drop" test length to 45 minutes
Co-authored-by: Kaspa Profiler <>
* Check blue score before requesting a pruning proof
* BuildPruningPointProof should return empty proof if the pruning point is genesis
* Don't fail many-tips if kaspad exits ungracefully
* Calculate GHOSTDAG, reachability etc for each level
* Don't preallocate cache for dag stores except level 0 and reduce the number of connections in the integration test to 32
* Reduce the number of connections in the integration test to 16
* Increase page file
* BuildPruningPointProof
* BuildPruningPointProof
* Add PruningProofManager
* Implement ApplyPruningPointProof
* Add prefix and fix blockAtDepth and fill headersByLevel
* Some bug fixes
* Include all relevant blocks for each level in the proof
* Fix syncAndValidatePruningPointProof to return the right block hash
* Fix block window
* Fix isAncestorOfPruningPoint
* Ban for rule errors on pruning proof
* Find common ancestor for blockAtDepthMAtNextLevel
* Use pruning proof in TestValidateAndInsertImportedPruningPoint
* stage status and finality point for proof blocks
* Uncomment golint
* Change test timeouts
* Calculate merge set for ApplyPruningPointProof
* Increase test timeout
* Add better caching for daa window store
* Return to default timeout
* Add ErrPruningProofMissesBlocksBelowPruningPoint
* Add errDAAWindowBlockNotFound
* Force connection loop next iteration on connection manager stop
* Revert to Test64IncomingConnections
* Remove BlockAtDepth from DAGTraversalManager
* numBullies->16
* Set page file size to 8gb
* Increase p2p max message size
* Test64IncomingConnections->Test16IncomingConnections
* Add comment for PruningProofM
* Add comment in `func (c *ConnectionManager) Stop()`
* Rename isAncestorOfPruningPoint->isAncestorOfSelectedTip
* Revert page file to 16gb
* Improve ExpectedHeaderPruningPoint perf
* Fix comment
* Revert "Improve ExpectedHeaderPruningPoint perf"
This reverts commit bca1080e7140c78d510f51bbea858ae280c2f38e.
* Don't test windows
* Add PruningPointProof to externalapi.
* Add BuildPruningPointProof and ValidatePruningPointProof to Consensus.
* Add the pruning point proof to the protocol.
* Add the pruning point proof to the wire package.
* Add PruningPointBlueWork.
* Make go vet happy.
* Properly initialize PruningPointProof in consensus.go.
* Validate pruning point blue work.
* Populate PruningPointBlueWork with the actual blue work of the pruning point.
* Revert "Populate PruningPointBlueWork with the actual blue work of the pruning point."
This reverts commit f2a9829998a8161ca24c259f971bd144084f0488.
* Revert "Validate pruning point blue work."
This reverts commit c6a90c5d2c5137657484d49ae8f33609aeb50235.
* Revert "Properly initialize PruningPointProof in consensus.go."
This reverts commit 9391574bbf711ab14eaf0c7700ae4d4b2e782e9d.
* Revert "Add PruningPointBlueWork."
This reverts commit 48182f652a5ff5048017d2c26640a1ffec79bc60.
* Fix PruningPointProof and MsgPruningPointProof to be two-dimensional.
* Fix wire PruningPointProof to be two-dimensional.
* Replace header finality point with pruning point
* Fix TestTransactionAcceptance
* Fix pruning candidate
* Store all past pruning points
* Pass pruning points on IBD
* Add blue score to block header
* Simplify ArePruningPointsInValidChain
* Fix static check errors
* Fix genesis
* Renames and text fixing
* Use ExpectedHeaderPruningPoint in block builder
* Fix TestCheckPruningPointViolation
* Pruning headers p2p basic structure
* Remove headers-first
* Fix consensus tests except TestValidateAndInsertPruningPointWithSideBlocks and TestValidateAndInsertImportedPruningPoint
* Add virtual genesis
* Implement PruningPointAndItsAnticoneWithMetaData
* Start fixing TestValidateAndInsertImportedPruningPoint
* Fix TestValidateAndInsertImportedPruningPoint
* Fix BlockWindow
* Update p2p and gRPC
* Fix all tests except TestHandleRelayInvs
* Delete TestHandleRelayInvs parts that cover the old IBD flow
* Fix lint errors
* Add p2p_request_ibd_blocks.go
* Clean code
* Make MsgBlockWithMetaData implement its own representation
* Remove redundant check if highest share block is below the pruning point
* Fix TestCheckLockTimeVerifyConditionedByAbsoluteTimeWithWrongLockTime
* Fix comments, errors ane names
* Fix window size to the real value
* Check reindex root after each block at TestUpdateReindexRoot
* Remove irrelevant check
* Renames and comments
* Remove redundant argument from sendGetBlockLocator
* Don't delete staging on non-recoverable errors
* Renames and comments
* Remove redundant code
* Commit changes inside ResolveVirtual
* Add comment to IsRecoverableError
* Remove blocksWithMetaDataGHOSTDAGDataStore
* Increase windows pagefile
* Move DeleteStagingConsensus outside of defer
* Get rid of mustAccepted in receiveBlockWithMetaData
* Ban on invalid pruning point
* Rename interface_datastructures_daawindowstore.go to interface_datastructures_blocks_with_meta_data_daa_window_store.go
* * Change GetVirtualSelectedParentChainFromBlockResponseMessage and VirtualSelectedParentChainChangedNotificationMessage to show only added block hashes
* Remove ResolveVirtual
* Use externalapi.ConsensusWrapper inside MiningManager
* Fix pruningmanager.blockwithmetadata
* Set pruning point selected child when importing the pruning point UTXO set
* Change virtual genesis hash
* replace the selected parent with virtual genesis on removePrunedBlocksFromGHOSTDAGData
* Get rid of low hash in block locators
* Remove +1 from everywhere we use difficultyAdjustmentWindowSize and increase the default value by one
* Add comments about consensus wrapper
* Don't use separate staging area when resolving resolveBlockStatus
* Fix netsync stability test
* Fix checkResolveVirtual
* Rename ConsensusWrapper->ConsensusReference
* Get rid of blockHeapNode
* Add comment to defaultDifficultyAdjustmentWindowSize
* Add SelectedChild to DAGTraversalManager
* Remove redundant copy
* Rename blockWindowHeap->calculateBlockWindowHeap
* Move isVirtualGenesisOnlyParent to utils
* Change BlockWithMetaData->BlockWithTrustedData
* Get rid of maxReasonLength
* Split IBD to 100 blocks each time
* Fix a bug in calculateBlockWindowHeap
* Switch to trusted data when encountering virtual genesis in blockWithTrustedData
* Move ConsensusReference to domain
* Update ConsensusReference comment
* Add comment
* Rename shouldNotAddGenesis->skipAddingGenesis
* Update constants
* Add to transaction SigOpCount
* Update mass calculation, and move it from InContext to InIsolation
* Update block validation accordingly
* Add SigOpCount validation during TransactionInContext
* Remove checking of mass vs maxMassAcceptedByBlock from consensusStateManager
* Update mining manager with latest changes
* Add SigOpCount to MsgTx.Copy()
* Fix initTestTransactionAcceptanceDataForClone
* Fix all tests in transaction_equal_clone_test.go
* Fix TestBlockMass
* Fix tests in transactionvalidator package
* Add SigOpCount to sighash
* Fix TestPruningDepth
* Fix problems in libkaspawalelt
* Fix integration tests
* Fix CalculateSignatureHash tests
* Remove remaining places talking about block size
* Add sanity check to checkBlockMass to make sure all transactions have their mass filled
* always add own sigOpCount to sigHash
* Update protowire/rpc.md
* Start working on removing any remaining reference to block/tx size
* Update rpc transaction verbose data to include mass rather then size
* Convert verboseData and block size check to mass
* Remove remaining usages of tx size in mempool
* Move transactionEstimatedSerializedSize to transactionvalidator
* Add PopulateMass to fakeRelayInvsContext
* Move PopulateMass to beggining of ValidateAndInsertTransaction + fix in it
* Assign mass a new number for backward-compatibility
* Adds name to route, and writes it in every error message
* Update all calls with route name
* Fixed a few missed points
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
* Make use of maxBlocks instead of maxBlueScoreDifference in antiPastHashesBetween
* Make use of maxBlocks instead of maxBlueScoreDifference in antiPastHashesBetween
* Make use of maxBlocks instead of maxBlueScoreDifference in antiPastHashesBetween
* Make use of maxBlocks instead of maxBlueScoreDifference in antiPastHashesBetween
* Make use of maxBlocks instead of maxBlueScoreDifference in antiPastHashesBetween
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Added model and stubs for all main methods
* Add constructors to all main objects
* Implement BlockCandidateTransactions
* implement expireOldTransactions and expireOrphanTransactions
* Rename isHighPriority to neverExpires
* Add stub for checkDoubleSpends
* Revert "Rename isHighPriority to neverExpires"
This reverts commit b2da9a4a00c02fb380d2518cf54fa16257bd8423.
* Imeplement transactionsOrderedByFeeRate
* Orphan maps should be idToOrphan
* Add error.go to mempool
* Invert the condition for banning when mempool rejects a transaction
* Move all model objects to model package
* Implement getParentsInPool
* Implemented mempoolUTXOSet.addTransaction
* Implement removeTransaction, remove sanity checks
* Implemented mempoolUTXOSet.checkDoubleSpends
* Implemented removeOrphan
* Implement removeOrphan
* Implement maybeAddOrphan and AddOrphan
* Implemented processOrphansAfterAcceptedTransaction
* Implement transactionsPool.addTransaction
* Implement RemoveTransaction
* If a transaction was removed from the mempool - update it's redeemers in orphan pool as well
* Use maximumOrphanTransactionCount
* Add allowOrphans to ValidateAndInsertTransaction stub
* Implement validateTransaction functions
* Implement fillInputs
* Implement ValidateAndInsertTransaction
* Implement HandleNewBlockTransactions
* Implement missing mempool interface methods
* Add comments to exported functions
* Call ValidateTransactionInIsolation where needed
* Implement RevalidateHighPriorityTransactions
* Rewire kaspad to use new mempool, and fix compilation errors
* Update rebroadcast logic to use new structure
* Handle non-standard transaction errors properly
* Add mutex to mempool
* bugfix: GetTransaction panics when ok is false
* properly calculate targetBlocksPerSecond in config.go
* Fix various lint errors and tests
* Fix expected text in test for duplicate transactions
* Skip the coinbase transaction in HandleNewBlockTransactions
* Unorphan the correct transactions
* Call ValidateTransactionAndPopulateWithConsensusData on unorphanTransaction
* Re-apply policy_test as check_transactions_standard_test
* removeTransaction: Remove redeemers in orphan pool as well
* Remove redundant check for uint64 < 0
* Export and rename isDust -> IsTransactionOutputDust to allow usage by rothschild
* Add allowOrphan to SubmitTransaction RPC request
* Remove all implementation from mempool.go
* tidy go mod
* Don't pass acceptedOrphans to handleNewBlockTransactions
* Use t.Errorf in stead of t.Fatalf
* Remove minimum relay fee from TestDust, as it's no longer configurable
* Add separate VirtualDAASCore method for faster retrieval where it's repeated multiple times
* Broadcast all transactions that were accepted
* Don't re-use GetVirtualDAAScore in GetVirtualInfo - this causes a deadlock
* Use real transaction count, and not Orphan
* Get mempool config from outside, incorporating values received from cli
* Use MinRelayFee and MaxOrphanTxs from global kaspad config
* Add explanation for the seemingly redundant check for transaction version in checkTransactionStandard
* Update some comment
* Convert creation of acceptedTransactions to a single line
* Move mempoolUTXOSet out of checkDoubleSpends
* Add test for attempt to insert double spend into mempool
* fillInputs: Skip check for coinbase - it's always false in mempool
* Clarify comment about removeRedeemers when removing random orphan
* Don't remove high-priority transactions in limitTransactionCount
* Use mempool.removeTransaction in limitTransactionCount
* Add mutex comment to handleNewBlockTransactions
* Return error from limitTransactionCount
* Pluralize the map types
* mempoolUTXOSet.removeTransaction: Don't restore utxo if it was not created in mempool
* Don't evacuate from orphanPool high-priority transactions
* Disallow double-spends in orphan pool
* Don't use exported (and locking) methods from inside mempool
* Check for double spends in mempool during revalidateTransaction
* Add checkOrphanDuplicate
* Add orphan to acceptedOrphans, not current
* Add TestHighPriorityTransactions
* Fix off-by-one error in limitTransactionCount
* Add TestRevalidateHighPriorityTransactions
* Remove checkDoubleSpends from revalidateTransaction
* Fix TestRevalidateHighPriorityTransactions
* Move check for MaximumOrphanCount to beggining of maybeAddOrphan
* Rename all map type to singulateToSingularMap
* limitOrphanPool only after the orphan was added
* TestDoubleSpendInMempool: use createChildTxWhenParentTxWasAddedByConsensus instead of createTransactionWithUTXOEntry
* Fix some comments
* Have separate min/max transaction versions for mempool
* Add comment on defaultMaximumOrphanTransactionCount to keep it small as long as we have recursion
* Fix comment
* Rename: createChildTxWhenParentTxWasAddedByConsensus -> createChildTxWhereParentTxWasAddedByConsensus
* Handle error from createChildTxWhereParentTxWasAddedByConsensus
* Rename createChildTxWhereParentTxWasAddedByConsensus -> createChildAndParentTxsAndAddParentToConsensus
* Convert all MaximumXXX constants to uint64
* Add comment
* remove mutex comments
* Add prefix to stores
* Add prefix to forgotten stores
* Add a special type for prefix
* Rename transaction->dbTx
* Change error message
* Use countKeyName
* Rename Temporary Consesnsus to Staging
* Add DeleteStagingConsensus to Domain interface
* Add lock to staging consensus
* Make prefix type-safer
* Use ioutil.TempDir instead of t.TempDir
* Add BlockHash optional parameter to EstimateNetworkBlockHashesPerSecond
* Allow to pass '-' for optional values in kaspactl
* Solve a division-by-zero in estimateNetworkHashesPerSecond
* Add BlockHash to toAppMessage/fromAppMessage functions
* Rename: topHash -> StartHash
* Return proper error message if provided startHash doesn't deserialize into a hash
* Fix getBlock RPC command to return transactions
* Fix getBlocks RPC command to return transactions and blocks
* Add GetBlockEvenIfHeaderOnly and use it for getBlock and getBlocks
* Implement GetBlockEvenIfHeaderOnly for fakeRelayInvsContext
* Use less nested code
(cherry picked from commit 50fd86e2878a6dcf1399610d8edc146466a5b492)
* [NOD-1344] relaytransactions: simple unit tests
* [NOD-1344] Add mid-complexity unit tests for relaytransactions
* Improve TestHandleRelayedTransactionssub tests
* Improve TestHandleRequestedTransactions sub tests
* [NOD-1344] Fix Simple call test
* [NOD-1344] Fix tests after redesign
* Divide transactionrelay_test.go to 2 separated tests and updates the tests.
* Changes due to review:change the test files name and the test function names, adds new comments and fix typo.
* Delete an unnecessary comparison to True in the if statement condition.
* Update the branch to v0.11.0-dev.
Co-authored-by: karim1king <karimkaspersky@yahoo.com>
Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: Svarog <feanorr@gmail.com>
* Replace BlockMessage with RpcBlock in rpc.proto.
* Convert everything in kaspad to use RPCBlocks and fix tests.
* Fix compilation errors in stability tests and the miner.
* Update TransactionVerboseData in rpc.proto.
* Update TransactionVerboseData in the rest of kaspad.
* Make golint happy.
* Include RpcTransactionVerboseData in RpcTransaction instead of the other way around.
* Regenerate rpc.pb.go after merge.
* Update appmessage types.
* Update appmessage request and response types.
* Reimplement conversion functions between appmessage.RPCTransaction and protowire.RpcTransaction.
* Extract RpcBlockHeader toAppMessage/fromAppMessage out of RpcBlock.
* Fix compilation errors in getBlock, getBlocks, and submitBlock.
* Fix compilation errors in getMempoolEntry.
* Fix compilation errors in notifyBlockAdded.
* Update verbosedata.go.
* Fix compilation errors in getBlock and getBlocks.
* Fix compilation errors in getBlocks tests.
* Fix conversions between getBlocks message types.
* Fix integration tests.
* Fix a comment.
* Add selectedParent to the verbose block response.
* Wait for flows to finish before shutting down
* Use CompareAndSwap
* Add comment
* Fix error message
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
