* Unite all reachability stores
* Upgrade script
* Fix tests
* Add UpdateReindexRoot to RebuildReachability
* Use dbTx when deleting reachability stores
* Use ghostdagDataWithoutPrunedBlocks when rebuilding reachability
* Use next tree ancestor wherever possible and avoid finality point search if the block is too close to pruning point
* Address the boundary case where the pruning point becomes the finality point
* some minor fixes
* Remove RebuildReachability and use manual syncing between old and new consensus for migration
* Remove sanity test (it failed when tips where not in the same order)
Co-authored-by: msutton <mikisiton2@gmail.com>
* Calculate GHOSTDAG, reachability etc for each level
* Don't preallocate cache for dag stores except level 0 and reduce the number of connections in the integration test to 32
* Reduce the number of connections in the integration test to 16
* Increase page file
* BuildPruningPointProof
* BuildPruningPointProof
* Add PruningProofManager
* Implement ApplyPruningPointProof
* Add prefix and fix blockAtDepth and fill headersByLevel
* Some bug fixes
* Include all relevant blocks for each level in the proof
* Fix syncAndValidatePruningPointProof to return the right block hash
* Fix block window
* Fix isAncestorOfPruningPoint
* Ban for rule errors on pruning proof
* Find common ancestor for blockAtDepthMAtNextLevel
* Use pruning proof in TestValidateAndInsertImportedPruningPoint
* stage status and finality point for proof blocks
* Uncomment golint
* Change test timeouts
* Calculate merge set for ApplyPruningPointProof
* Increase test timeout
* Add better caching for daa window store
* Return to default timeout
* Add ErrPruningProofMissesBlocksBelowPruningPoint
* Add errDAAWindowBlockNotFound
* Force connection loop next iteration on connection manager stop
* Revert to Test64IncomingConnections
* Remove BlockAtDepth from DAGTraversalManager
* numBullies->16
* Set page file size to 8gb
* Increase p2p max message size
* Test64IncomingConnections->Test16IncomingConnections
* Add comment for PruningProofM
* Add comment in `func (c *ConnectionManager) Stop()`
* Rename isAncestorOfPruningPoint->isAncestorOfSelectedTip
* Revert page file to 16gb
* Improve ExpectedHeaderPruningPoint perf
* Fix comment
* Revert "Improve ExpectedHeaderPruningPoint perf"
This reverts commit bca1080e7140c78d510f51bbea858ae280c2f38e.
* Don't test windows
* Pruning headers p2p basic structure
* Remove headers-first
* Fix consensus tests except TestValidateAndInsertPruningPointWithSideBlocks and TestValidateAndInsertImportedPruningPoint
* Add virtual genesis
* Implement PruningPointAndItsAnticoneWithMetaData
* Start fixing TestValidateAndInsertImportedPruningPoint
* Fix TestValidateAndInsertImportedPruningPoint
* Fix BlockWindow
* Update p2p and gRPC
* Fix all tests except TestHandleRelayInvs
* Delete TestHandleRelayInvs parts that cover the old IBD flow
* Fix lint errors
* Add p2p_request_ibd_blocks.go
* Clean code
* Make MsgBlockWithMetaData implement its own representation
* Remove redundant check if highest share block is below the pruning point
* Fix TestCheckLockTimeVerifyConditionedByAbsoluteTimeWithWrongLockTime
* Fix comments, errors ane names
* Fix window size to the real value
* Check reindex root after each block at TestUpdateReindexRoot
* Remove irrelevant check
* Renames and comments
* Remove redundant argument from sendGetBlockLocator
* Don't delete staging on non-recoverable errors
* Renames and comments
* Remove redundant code
* Commit changes inside ResolveVirtual
* Add comment to IsRecoverableError
* Remove blocksWithMetaDataGHOSTDAGDataStore
* Increase windows pagefile
* Move DeleteStagingConsensus outside of defer
* Get rid of mustAccepted in receiveBlockWithMetaData
* Ban on invalid pruning point
* Rename interface_datastructures_daawindowstore.go to interface_datastructures_blocks_with_meta_data_daa_window_store.go
* * Change GetVirtualSelectedParentChainFromBlockResponseMessage and VirtualSelectedParentChainChangedNotificationMessage to show only added block hashes
* Remove ResolveVirtual
* Use externalapi.ConsensusWrapper inside MiningManager
* Fix pruningmanager.blockwithmetadata
* Set pruning point selected child when importing the pruning point UTXO set
* Change virtual genesis hash
* replace the selected parent with virtual genesis on removePrunedBlocksFromGHOSTDAGData
* Get rid of low hash in block locators
* Remove +1 from everywhere we use difficultyAdjustmentWindowSize and increase the default value by one
* Add comments about consensus wrapper
* Don't use separate staging area when resolving resolveBlockStatus
* Fix netsync stability test
* Fix checkResolveVirtual
* Rename ConsensusWrapper->ConsensusReference
* Get rid of blockHeapNode
* Add comment to defaultDifficultyAdjustmentWindowSize
* Add SelectedChild to DAGTraversalManager
* Remove redundant copy
* Rename blockWindowHeap->calculateBlockWindowHeap
* Move isVirtualGenesisOnlyParent to utils
* Change BlockWithMetaData->BlockWithTrustedData
* Get rid of maxReasonLength
* Split IBD to 100 blocks each time
* Fix a bug in calculateBlockWindowHeap
* Switch to trusted data when encountering virtual genesis in blockWithTrustedData
* Move ConsensusReference to domain
* Update ConsensusReference comment
* Add comment
* Rename shouldNotAddGenesis->skipAddingGenesis
* Save DAA score and DAA added blocks for each block
* Add test
* Add pruning support
* Replace 8 with uint64Length
* Separate DAABlocksStore cache size to DAA score and daaAddedBlocks
* Add selected chain store and optimize block locator with it
* Fix build error
* Fix comments
* Fix IsStaged
* Rename CalculateSelectedParentChainChanges to CalculateChainPath and SelectedParentChainChanges->SelectedChainPath
* Use binary.LittleEndian directly to allow compiler optimizations
* Remove boolean from HeadersSelectedChainStore interface
* Prevent endless loop in block locator
* Get rid of insertMode
* Rename AddBlockToVirtual->AddBlock
* When F is not in the future of P, enforce finality with P and not with F.
* Don't allow blocks with invalid parents or with missing block body
* Check finality violation before checking block status
* Implement CalculateIndependentPruningPoint
* Move checkBlockStatus to validateBlock
* Add ValidateBlock to block processor interface
* Adjust SetPruningPoint to the new IBD flow
* Add pruning store to CSM's constructor
* Flip wrong condition on AddHeaderTip
* Fix func (hts *headerSelectedTipStore) Has
* Fix block stage order
* Call to ValidateBodyInContext from validatePostProofOfWork
* Enable overrideDAGParams
* Update log
* Rename SetPruningPoint to ValidateAndInsertPruningPoint and move most of its logic inside block processor
* Rename hasValidatedHeader->hasValidatedOnlyHeader
* Fix typo
* Name return values for fetchMissingUTXOSet
* Add comment
* Return ErrMissingParents when block body is missing
* Add logs and comments
* Fix merge error
* Fix pruning point calculation to be by virtual selected parent
* Replace CalculateIndependentPruningPoint to CalculatePruningPointByHeaderSelectedTip
* Fix isAwaitingUTXOSet to check pruning point by headers
* Change isAwaitingUTXOSet indication
* Remove IsBlockInHeaderPruningPointFuture from BlockInfo
* Fix LowestChainBlockAboveOrEqualToBlueScore
* Add validateNewPruningPointTransactions
* Add validateNewPruningAgainstPastUTXO
* Rename set_pruning_utxo_set.go to update_pruning_utxo_set.go
* Check missing block body hashes by missing block instead of status
* Validate pruning point against past UTXO with the pruning point as block hash
* Remove virtualHeaderHash
* Fix comment
* Fix imports
* Revert "[NOD-1500] Delete integration tests"
This reverts commit fcb57a206690a884fa6afb69d5d493282954a8bf.
* [NOD-1518] hashserialization -> consenusserialization
* [NOD-1518] Fix add genesis to virtual
* [NOD-1518] Fix a bug in SerializeCoinbasePayload.
* [NOD-1518] Fix a loop error and make pastMedianTime behave correctly everywhere on genesis.
* [NOD-1518] Fix another bug and an infinite loop.
* [NOD-1518] Fix uninitialized slice.
* [NOD-1518] Fix bad should-commit checks and another infinite loop.
* [NOD-1518] Fix nil serialization.
* [NOD-1518] Rename blockHash to currentBlockHash.
* [NOD-1518] Move the check whether stagedVirtualUTXOSet != nil to the top of commitVirtualUTXODiff.
* [NOD-1518] Simplify utxoDiffStore.Commit.
* [NOD-1518] Unextract resolveBlockStatusAndCheckFinality.
* [NOD-1518] Move no-transactions logic into CalculateIDMerkleRoot.
* [NOD-1518] Remove redundant is-staged check.
* [NOD-1518] Fix merge errors.
* [NOD-1518] Don't write anything if utxoDiffChild is nil.
* [NOD-1518] Stage virtualAcceptanceData and virtualMultiset.
* [NOD-1518] Fix bugs in getBlockTemplate and submitBlock.
* [NOD-1518] Fix bad validation order in validateHeaderInContext.
* [NOD-1518] Fix bug in Next().
* [NOD-1518] Fix nil dereference of subnetworks in AddressCache.
* [NOD-1518] Fix multisetStore.Get returning a pointer to a multiset that is changed in place.
* [NOD-1518] Break on genesis in countSubtrees.
* [NOD-1518] Fix createBlockLocator.
* [NOD-1518] Fix MsgTxToDomainTransaction.
* [NOD-1518] Set MaxTxVersion to 1.
* [NOD-1518] Fix missing error handling, bug in MsgTxToDomainTransaction, and bad subnetwork equality check.
* [NOD-1518] Fix bug in hasUTXOByOutpointFromStagedVirtualUTXODiff.
* [NOD-1518] Remove irrelevant comments.
* [NOD-1518] Generate transactions with sufficient fee in tx_relay_test.
* [NOD-1518] Fix broken RPC handlers.
* [NOD-1518] Fix merge errors.
* [NOD-1518] Fix bad exists check in restorePastUTXO and missing genesis check in CalculatePastUTXOAndAcceptanceData.
* [NOD-1518] Add a comment.
* [NOD-1518] Use a regular mutex instead of a read-write mutex in consensus to avoid dealing with sneaky not-actually-read functions.
* [NOD-1518] Fix a deadlock in GetVirtualSelectedParent.
* [NOD-1518] Fix missing handler registration for CmdHeader.
* [NOD-1518] Fix processHeader calling OnNewBlock and LogBlock. Also fix conversion errors in IBDRootUTXOSetAndBlock.
* [NOD-1518] Fix bad Command() in MsgIBDRootUTXOSetAndBlock.
* [NOD-1518] Fix bad SyncStateMissingUTXOSet logic in resolveSyncState.
* [NOD-1518] Rename mode to syncState.
* [NOD-1518] Fix headers-only blocks coming in after the consensus thinks it's synced.
* [NOD-1518] Fix selectedChildIterator.Next not ignoring virtual, infinite loop in HashSet.Length().
* [NOD-1518] Fix not-properly wrapped IBD blocks.
* [NOD-1518] Fix bad conversion in RequestIBDBlocks.
* [NOD-1518] Fix bad string for CmdRequestHeaders.
* [NOD-1518] Fix bad string for CmdDoneHeaders.
* [NOD-1518] Fix bad Command() for MsgIBDRootNotFound.
* [NOD-1518] Fix bad areHeaderTipsSyncedMaxTimeDifference value.
* [NOD-1518] Add missing string for CmdRequestIBDBlocks.
* [NOD-1518] Fix bad check for SyncStateMissingBlockBodies.
* [NOD-1518] Fix bad timeout durations in tests.
* [NOD-1518] Fix IBD blocks not calling OnNewBlock.
* [NOD-1518] Change when IBD finishes.
* [NOD-1518] Properly clone utxoDiffChild.
* [NOD-1535] Fix reachability tests
* [NOD-1518] Fix merge errors.
* [NOD-1518] Move call to LogBlock to into OnNewBlock.
* [NOD-1518] Return "not implemented" in unimplemented RPC handlers.
* [NOD-1518] Extract cloning of hashes to a method over DomainHash.
* [NOD-1518] Use isHeaderOnlyBlock.
* [NOD-1518] Use constants.TransactionVersion.
* [NOD-1518] Break immediately if we reached the virtual in SelectedChildIterator.
* [NOD-1518] Don't stage nil utxoDiffChild.
* [NOD-1518] Properly check the genesis hash in CalculatePastUTXOAndAcceptanceData.
* [NOD-1518] Explain why we break on current == nil in countSubtrees.
* [NOD-1518] Add a comment explaining why we check against StatusValid in resolveSyncState.
* [NOD-1535] Add external reachability tests
* [NOD-1535] Fix reachability tests and fix related bugs
* [NOD-1535] Add setters fox reindex slack and window
* [NOD-1535] Remove redundant line
* [NOD-1535] Add comment
* [NOD-1535] Fix comments
* [NOD-1535] Rename DBReader->DatabaseContext
* [NOD-1535] Check that reindex root is changed
* [NOD-1535] Fix calculateNewTips
Co-authored-by: Mike Zak <feanorr@gmail.com>
Co-authored-by: stasatdaglabs <stas@daglabs.com>
* Revert "[NOD-1500] Delete integration tests"
This reverts commit fcb57a206690a884fa6afb69d5d493282954a8bf.
* [NOD-1518] hashserialization -> consenusserialization
* [NOD-1518] Fix add genesis to virtual
* [NOD-1518] Fix a bug in SerializeCoinbasePayload.
* [NOD-1518] Fix a loop error and make pastMedianTime behave correctly everywhere on genesis.
* [NOD-1518] Fix another bug and an infinite loop.
* [NOD-1518] Fix uninitialized slice.
* [NOD-1518] Fix bad should-commit checks and another infinite loop.
* [NOD-1518] Fix nil serialization.
* [NOD-1518] Rename blockHash to currentBlockHash.
* [NOD-1518] Move the check whether stagedVirtualUTXOSet != nil to the top of commitVirtualUTXODiff.
* [NOD-1518] Simplify utxoDiffStore.Commit.
* [NOD-1518] Unextract resolveBlockStatusAndCheckFinality.
* [NOD-1518] Move no-transactions logic into CalculateIDMerkleRoot.
* [NOD-1518] Remove redundant is-staged check.
* [NOD-1518] Fix merge errors.
* [NOD-1518] Don't write anything if utxoDiffChild is nil.
* [NOD-1518] Stage virtualAcceptanceData and virtualMultiset.
* [NOD-1518] Fix bugs in getBlockTemplate and submitBlock.
* [NOD-1518] Fix bad validation order in validateHeaderInContext.
* [NOD-1518] Fix bug in Next().
* [NOD-1518] Fix nil dereference of subnetworks in AddressCache.
* [NOD-1518] Fix multisetStore.Get returning a pointer to a multiset that is changed in place.
* [NOD-1518] Break on genesis in countSubtrees.
* [NOD-1518] Fix createBlockLocator.
* [NOD-1518] Fix MsgTxToDomainTransaction.
* [NOD-1518] Set MaxTxVersion to 1.
* [NOD-1518] Fix missing error handling, bug in MsgTxToDomainTransaction, and bad subnetwork equality check.
* [NOD-1518] Fix bug in hasUTXOByOutpointFromStagedVirtualUTXODiff.
* [NOD-1518] Remove irrelevant comments.
* [NOD-1518] Generate transactions with sufficient fee in tx_relay_test.
* [NOD-1518] Fix broken RPC handlers.
* [NOD-1518] Fix merge errors.
* [NOD-1518] Fix bad exists check in restorePastUTXO and missing genesis check in CalculatePastUTXOAndAcceptanceData.
* [NOD-1518] Add a comment.
* [NOD-1518] Use a regular mutex instead of a read-write mutex in consensus to avoid dealing with sneaky not-actually-read functions.
* [NOD-1518] Fix a deadlock in GetVirtualSelectedParent.
* [NOD-1518] Fix missing handler registration for CmdHeader.
* [NOD-1518] Fix processHeader calling OnNewBlock and LogBlock. Also fix conversion errors in IBDRootUTXOSetAndBlock.
* [NOD-1518] Fix bad Command() in MsgIBDRootUTXOSetAndBlock.
* [NOD-1518] Fix bad SyncStateMissingUTXOSet logic in resolveSyncState.
* [NOD-1518] Rename mode to syncState.
* [NOD-1518] Fix headers-only blocks coming in after the consensus thinks it's synced.
* [NOD-1518] Fix selectedChildIterator.Next not ignoring virtual, infinite loop in HashSet.Length().
* [NOD-1518] Fix not-properly wrapped IBD blocks.
* [NOD-1532] Add TestMultiset
* [NOD-1518] Fix bad conversion in RequestIBDBlocks.
* [NOD-1518] Fix bad string for CmdRequestHeaders.
* [NOD-1518] Fix bad string for CmdDoneHeaders.
* [NOD-1518] Fix bad Command() for MsgIBDRootNotFound.
* [NOD-1532] Add TestPastUTXOMultiset
* [NOD-1518] Fix bad areHeaderTipsSyncedMaxTimeDifference value.
* [NOD-1532] Added TestDoubleSpends
* [NOD-1518] Add missing string for CmdRequestIBDBlocks.
* [NOD-1518] Fix bad check for SyncStateMissingBlockBodies.
* [NOD-1518] Fix bad timeout durations in tests.
* [NOD-1518] Fix IBD blocks not calling OnNewBlock.
* [NOD-1518] Change when IBD finishes.
* [NOD-1518] Properly clone utxoDiffChild.
* [NOD-1532] Update hashes of blocks
* [NOD-1532] Fix genesis blocks and a few more bugs
* [NOD-1532] Bugfix: incorrect key passed to dbTx.Put
* [NOD-1532] Make sure there's no nil payloads
* [NOD-1532] Fix AddBlockToVirtual
* [NOD-1532] Update tips and virtualDiffParents properly
* [NOD-1532] Allow nil payload
* [NOD-1532] Check for actual error and not just some RuleError
* [NOD-1532] Get rid of SimpleCoinbaseData and make OpTrueScript P2SH
* [NOD-1532] If coinbaseData is nil - fill in with generic coinbaseData
Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: stasatdaglabs <stas@daglabs.com>
* [NOD-1529] Add all stores and processes to consensus, and add access to TestConsensus
* [NOD-1529] Move the getters of TestConsensus to separate file
* [NOD-1529] Add AddBlock to TestConsensus
* [NOD-1529] Update NewTestConsensus to be more all-encompassing
* [NOD-1529] Remove test directory in teardown
* [NOD-1529] Add ForAllNets function
* [NOD-1529] Add comment
