* Remove pruningPointUTXOSetStaging and implement UpdatePruningPointUTXOSet.
* Implement StageStartSavingNewPruningPointUTXOSet, HadStartedSavingNewPruningPointUTXOSet, and FinishSavingNewPruningPointUTXOSet.
* Fix a bad return.
* Implement UpdatePruningPointUTXOSetIfRequired.
* Call UpdatePruningPointUTXOSetIfRequired on consensus creation.
* Rename savingNewPruningPointUTXOSetKey to updatingPruningPointUTXOSet.
* Add a log.
* Add calls to runtime.GC() at its start and end.
* Rename a variable.
* Replace calls to runtime.GC to calls to LogMemoryStats.
* Wrap the contents of LogMemoryStats in a log closure.
* Replaced the content of MsgIBDRootUTXOSetChunk with pairs of outpoint-utxo entry pairs.
* Rename utxoIter to utxoIterator.
* Add a big stinky TODO on an assert.
* Replace pruningStore staging with a UTXO set iterator.
* Reimplement receiveAndInsertIBDRootUTXOSet.
* Extract OutpointAndUTXOEntryPairsToDomainOutpointAndUTXOEntryPairs into domainconverters.go.
* Pass the outpoint and utxy entry pairs to the pruning store.
* Implement InsertCandidatePruningPointUTXOs.
* Implement ClearCandidatePruningPointUTXOs.
* Implement UpdateCandidatePruningPointMultiset.
* Use the candidate pruning point multiset in updatePruningPoint.
* Implement CandidatePruningPointUTXOIterator.
* Use the pruning point utxo set iterator for StageVirtualUTXOSet.
* Defer ClearCandidatePruningPointUTXOs.
* Implement OverwriteVirtualUTXOSet.
* Implement CommitCandidatePruningPointUTXOSet.
* Implement BeginOverwritingVirtualUTXOSet and FinishOverwritingVirtualUTXOSet.
* Implement overwriteVirtualUTXOSetAndCommitPruningPointUTXOSet.
* Rename ClearCandidatePruningPointUTXOs to ClearCandidatePruningPointData.
* Add missing methods to dbManager.
* Implement PruningPointUTXOs.
* Implement RecoverUTXOIfRequired.
* Delete the utxoserialization package.
* Fix compilation errors in TestValidateAndInsertPruningPoint.
* Switch order of operations in the if statements in PruningPointUTXOs so that Next() wouldn't be unnecessarily called.
* Fix missing pruning point utxo set staging and bad slice length.
* Fix no default multiset in InsertCandidatePruningPointUTXOs.
* Make go vet happy.
* Rename candidateXXX to importedXXX.
* Do some more renaming.
* Rename some more.
* Fix bad MsgIBDRootNotFound logic.
* Fix an error message.
* Simplify receiveIBDRootBlock.
* Fix error message in receiveAndInsertIBDRootUTXOSet.
* Do some more renaming.
* Fix merge errors.
* Fix a bug caused by calling iterator.First() unnecessarily.
* Remove databaseContext from stores and don't use a transaction in ClearXXX functions.
* Simplify receiveAndInsertIBDRootUTXOSet.
* Fix offset count in PruningPointUTXOs().
* Fix readOnlyUTXOIteratorWithDiff.First().
* Split handleRequestIBDRootUTXOSetAndBlockFlow into smaller methods.
* Rename IbdRootNotFound to UnexpectedPruningPoint.
* Rename requestIBDRootHash to requestPruningPointHash.
* Rename IBDRootHash to PruningPointHash.
* Rename RequestIBDRootUTXOSetAndBlock to RequestPruningPointUTXOSetAndBlock.
* Rename IBDRootUTXOSetChunk to PruningPointUTXOSetChunk.
* Rename RequestNextIBDRootUTXOSetChunk to RequestNextPruningPointUTXOSetChunk.
* Rename DoneIBDRootUTXOSetChunks to DonePruningPointUTXOSetChunks.
* Rename remaining references to IBD root.
* Fix an error message.
* Add a check for HadStartedImportingPruningPointUTXOSet in commitVirtualUTXODiff.
* Add a check for HadStartedImportingPruningPointUTXOSet in ImportPruningPointUTXOSetIntoVirtualUTXOSet.
* Move FinishImportingPruningPointUTXOSet closer to HadStartedImportingPruningPointUTXOSet.
* Remove reference to pruningStore in utxoSetIterator.
* Pointerify utxoSetIterator receivers.
* Fix bad insert in CommitImportedPruningPointUTXOSet.
* Rename commitImportedPruningPointUTXOSetAll to applyImportedPruningPointUTXOSet.
* Simplify PruningPointUTXOs.
* Add populateTransactionWithUTXOEntriesFromUTXOSet.
* Fix a TODO comment.
* Rename InsertImportedPruningPointUTXOs to AppendImportedPruningPointUTXOs.
* Extract handleRequestPruningPointUTXOSetAndBlockMessage to a separate method.
* Rename stuff in readOnlyUTXOIteratorWithDiff.First().
* Address toAddIterator in readOnlyUTXOIteratorWithDiff.First().
* Call First() before any full iteration on ReadOnlyUTXOSetIterator.
* Call First() before any full iteration on a database Cursor.
* Put StartImportingPruningPointUTXOSet inside the pruning point transaction.
* Make serializeOutpoint and serializeUTXOEntry free functions in pruningStore.
* Fix readOnlyUTXOIteratorWithDiff.First().
* Fix bad validations in importPruningPoint.
* Remove superfluous call to validateBlockTransactionsAgainstPastUTXO.
* Increase default log sizes, and increase kaspad log sizes
* Add an option to not print logs to stdout
* Allow logs to be printed in the current working directory
* Add more pruning related logs
* Add comment and increase log rotations to save last 64 logs
* Add archival cli flag
* If --archival was activated - don't delete anything
* Fix tests
* Still change block status to StatusHeaderOnly even in archival nodes
* Remove the limit on the amount of hashes returned from antiPastHashesBetween.
* Guard against requests with a non-existing block hash.
* Move missing-block-hash guards to consensus.go.
* Ban a peer that doesn't send us all the requested headers during IBD.
* Extract blockHeap.ToSlice.
* Re-request headers in requestHeaders if we didn't receive the highHash.
* 1) Calculate pruning point incrementally
2) Add IsValidPruningPoint to pruning manager and consensus
3) Use reachability children for selected child iterator
* Add request IBD root hash flow
* Fix UpdatePruningPointByVirtual and IsValidPruningPoint
* Regenerate messages.pb.go
* Make the pruning point the earliest chain block with finality interval higher than the previous pruning point
* Fix merge errors
* [NOD-1575] Implement Clone and Equal for all model types
* [NOD-1575] Add assertion for transaction ID equality
* [NOD-1575] Use DomainTransaction.Equal to compare to expected coinbase transaction
* [NOD-1575] Add TestDomainBlockHeader_Clone
* [NOD-1575] Don't clone nil values
* [NOD-1575] Add type assertions
* [NOD-1575] Don't clone nil values
* [NOD-1575] Add missing Equals
* [NOD-1575] Add length checks
* [NOD-1575] Update comment
* [NOD-1575] Check length for TransactionAcceptanceData
* [NOD-1575] Explicitly clone nils where needed
* [NOD-1575] Clone tx id
* [NOD-1575] Flip condition
* Nod 1576 make coverage tests for equal clone inside model externalapi (#1177)
* [NOD-1576] Make coverage tests for equal and clone inside model and externalapi
* Some formatting and naming fixes
* Made transactionToCompare type exported
* Added some tests and made some changes to the tests code
* No changes made
* Some formatting and naming changes made
* Made better test coverage for externalapi clone and equal functions
* Changed expected result for two cases
* Added equal and clone functions tests for ghostdag and utxodiff
* Added tests
* [NOD-1576] Implement reachabilitydata equal/clone unit tests
* [NOD-1576] Full coverage of reachabilitydata equal/clone unit tests
* Made changes and handling panic to transaction_equal_clone_test.go and formating of utxodiff_equal_clone_test.go
* Added recoverForEqual2 for handling panic to transaction_equal_clone_test.go
* [NOD-1576] Full coverage of transaction equal unit test
* [NOD-1576] Add expects panic
* [NOD-1576] Allow composites in go vet
* [NOD-1576] Code review fixes (#1223)
* [NOD-1576] Code review fixes
* [NOD-1576] Code review fixes part 2
* [NOD-1576] Fix wrong name
Co-authored-by: karim1king <karimkaspersky@yahoo.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: Karim <karim1king@users.noreply.github.com>
* Fix merge errors
* Use Equal where possible
* Use Equal where possible
* Use Equal where possible
Co-authored-by: andrey-hash <74914043+andrey-hash@users.noreply.github.com>
Co-authored-by: karim1king <karimkaspersky@yahoo.com>
Co-authored-by: Karim <karim1king@users.noreply.github.com>
* Get rid of insertMode
* Rename AddBlockToVirtual->AddBlock
* When F is not in the future of P, enforce finality with P and not with F.
* Don't allow blocks with invalid parents or with missing block body
* Check finality violation before checking block status
* Implement CalculateIndependentPruningPoint
* Move checkBlockStatus to validateBlock
* Add ValidateBlock to block processor interface
* Adjust SetPruningPoint to the new IBD flow
* Add pruning store to CSM's constructor
* Flip wrong condition on AddHeaderTip
* Fix func (hts *headerSelectedTipStore) Has
* Fix block stage order
* Call to ValidateBodyInContext from validatePostProofOfWork
* Enable overrideDAGParams
* Update log
* Rename SetPruningPoint to ValidateAndInsertPruningPoint and move most of its logic inside block processor
* Rename hasValidatedHeader->hasValidatedOnlyHeader
* Fix typo
* Name return values for fetchMissingUTXOSet
* Add comment
* Return ErrMissingParents when block body is missing
* Add logs and comments
* Fix merge error
* Fix pruning point calculation to be by virtual selected parent
* Replace CalculateIndependentPruningPoint to CalculatePruningPointByHeaderSelectedTip
* Fix isAwaitingUTXOSet to check pruning point by headers
* Change isAwaitingUTXOSet indication
* Remove IsBlockInHeaderPruningPointFuture from BlockInfo
* Fix LowestChainBlockAboveOrEqualToBlueScore
* Add validateNewPruningPointTransactions
* Add validateNewPruningAgainstPastUTXO
* Rename set_pruning_utxo_set.go to update_pruning_utxo_set.go
* Check missing block body hashes by missing block instead of status
* Validate pruning point against past UTXO with the pruning point as block hash
* Remove virtualHeaderHash
* Fix comment
* Fix imports
* [NOD-1126]
1. Change function name in BlockValidator interface from: "ValidateProofOfWorkAndDifficulty" to "ValidatePruningPointViolationAndProofOfWorkAndDifficulty".
2. Add to the blockValidator struct the pruningManager (also added to the function "New" Respectively).
3. Added new function "checkPruningPointViolation" of blockValidator type.
4. Add new internal check - "checkPruningPointViolation", on the function "ValidateProofOfWorkAndDifficulty".(The third check).
5. Add new error rule - "ErrPruningPointViolation".
* [Issue-1126]
1. Remove the function "PruningPoint" from PruningManager interface.
2. Changes in blockValidator struct - remove pruningManager, and adding pruningStore.
3. Reads for "pruningPoint" function from pruningStore instead of pruningManager (because of note 1 above) in the functions: * "checkPruningPointViolation" of type blockValidator.
* "FindNextPruningPoint" of type pruningManager.
* [Issue-1126]
1. Add missing error handling.
* [Issue-1126] Changes in function "checkPruningPointViolation": If header = genesis, stop checking and return nil.
* [Issue-1126] In function "checkPruningPointViolation" - change from a for loop to the "IsAncestorOfAny" function.
* [#1126] "FindNextPruningPoint" - save the pruning point in case the point is the genesis and change code internal order.
* [#1126] "FindNextPruningPoint" - cosmetics change.
* [#1126] "FindNextPruningPoint" - remove "return nil" when there is no pruning point on the if expression.
Co-authored-by: tal <tal@daglabs.com>
* [NOD-1512] Implement UTXO set deserialization
* [NOD-1512] Remove redundant file
* [NOD-1512] Don't use big endian for serialization
* [NOD-1512] Use Read/Write element
* [NOD-1512] Unexport ReadElement
* [NOD-1512] Fix StageVirtualUTXOSet
* [NOD-1512] Get rid of dagParams in consensusStateManager
* [NOD-1512] Get rid of dagParams in consensusStateManager
* Update pruningmanager interface
* Add a ProtoUTXOSet to hashserialization
* Update miningmanager with all the necessary stores and managers
* Implement mining manager
* Prune P.AC not in V.Past
* PruningManager fix all review comments
* [NOD-1476] Add dependency to BlockRelationStore in BlockProcessor.
* [NOD-1476] Add dependency to BlockStatusStore in BlockValidator.
* [NOD-1476] Add dependency to GHOSTDAGManager in BlockValidator.
* [NOD-1476] Rename CalculateConsensusStateChanges to AddBlockToVirtual.
* [NOD-1476] Remove RestoreDiffFromVirtual.
* [NOD-1476] Remove RestorePastUTXOSet.
* [NOD-1476] Add dependency to GHOSTDAGDataStore in ConsensusStateManager.
* [NOD-1476] Rename CalculateAcceptanceDataAndUTXOMultiset to just CalculateAcceptanceData.
* [NOD-1476] Remove UTXODiffManager and add dependencies to AcceptanceManager.
* [NOD-1476] Rename CalculateAcceptanceData to CalculateAcceptanceDataAndMultiset.
* [NOD-1476] Add dependency to DAGTopologyManager from ConsensusStateManager.
* [NOD-1476] Add dependency to BlockStore from ConsensusStateManager.
* [NOD-1476] Add dependency to PruningManager from ConsensusStateManager.
* [NOD-1476] Remove unnecessary stuff from ConsensusStateChanges.
* [NOD-1476] Add dependency to UTXODiffStore from ConsensusStateManager.
* [NOD-1476] Add tips to BlockRelationsStore.
* [NOD-1476] Add dependency to BlockRelationsStore from ConsensusStateManager.
* [NOD-1476] Remove Tips() from ConsensusStateStore.
* [NOD-1476] Remove acceptanceManager.
* [NOD-1476] Remove irrelevant functions out of ConsensusStateManager.
* [NOD-1461] Change the external api interface to not having anything besides DomainTransactions and DomainBlocks.
* [NOD-1462] Move external api types to a separate package.
* [NOD-1462] Clarify which model we're using in miningmanager.
* [NOD-1462] Extract coinbase data to its own struct.
* [NOD-1462] Add a comment above CoinbaseData.
* [NOD-1462] Fix the comment above CoinbaseData.
* [NOD-1458] Rename RestoreUTXOSet to RestorePastUTXOSet.
* [NOD-1458] Make CalculateAcceptanceDataAndMultiset take BlockGHOSTDAGData and nothing else.
* [NOD-1458] Make ConsensusStateStore's Update take ConsensusStateChanges instead of just UTXODiff.
* [NOD-1458] Add Tips() to ConsensusStateStore.
* [NOD-1458] Make all implementation structs private.
* [NOD-1458] Remove BlockAtDepth and add highHash to ChainBlockAtBlueScore.
* [NOD-1458] Rename CalculateAcceptanceDataAndMultiset to CalculateAcceptanceDataAndUTXOMultiset.
* [NOD-1458] Add a dependency to GHOSTDAGManager from ConsensusStateManager.
* [NOD-1458] Add ChooseSelectedParent to GHOSTDAGManager.
* [NOD-1458] Add DifficultyManager.
* [NOD-1458] Add PastMedianTimeManager.
* [NOD-1458] Add Hash() to Multiset.
* [NOD-1458] Add a dependency to ghostdagManager from blockProcessor.
* [NOD-1458] Add errors to all interfaces that need them.
* [NOD-1458] Uppercasify types in comments.
* [NOD-1458] Fix a bad comment.
* [NOD-1458] Fix a comment.
* [NOD-1458] Rename ChainBlockAtBlueScore to HighestChainBlockBelowBlueScore.
* [NOD-1458] Replace BlockAndTransactionValidator with an anonymous interface.
* [NOD-1413] Remove /cmd/addblock
* [NOD-1413] Define and implement TransactionValidator.
* [NOD-1413] Make changes to ConsensusStateManager's interface.
* [NOD-1413] Make changes to PruningManager's interface.
* [NOD-1413] Make changes to DAGTraversalManager's interface.
* [NOD-1413] Make changes to MultisetStore's interface.
* [NOD-1413] Make changes to UTXODiffStore's interface.
* [NOD-1413] Make changes to UTXODiffStore's interface harder.
* [NOD-1413] Make changes to AcceptanceDataStore's interface harder.
* [NOD-1413] Make changes to PruningStore's interface.
* [NOD-1413] Delete BlockIndex.
* [NOD-1413] Add FeeDataStore.
* [NOD-1413] Update BlockMessageStore's interface.
* [NOD-1413] Fix interface violations.
* [NOD-1413] Add FeeDataStore to BlockProcessor.
* [NOD-1413] Make go vet happy.
* [NOD-1413] Add missing fields to ConsensusStateChanges.
* [NOD-1413] Add another missing field to ConsensusStateChanges.
* [NOD-1413] Add a reference to blockStore in consensusStateManager.
* [NOD-1413] Add missing methods to UTXODiffStore.
* [NOD-1413] Rename pruningPointStore to pruningStore everywhere.
* [NOD-1413] Remove superfluous parameters from CalculateConsensusStateChanges.
* [NOD-1413] Add missing dependencies to PruningManager.
* [NOD-1413] Remove implementation-y functions from TransactionValidator's interface.
* [NOD-1413] Make go vet happy.
* [NOD-1413] Add a couple of methods to DAGTopologyManager.
* [NOD-1413] Fix a typo in a file name.
* [NOD-1413] Remove non-interface functions from Validator.
* [NOD-1416] Move processes/datastructures interfaces into the model package.
* [NOD-1416] Decouple the model from dbaccess.
* [NOD-1413] Implement DomainBlock and DomainTransaction.
* [NOD-1413] Decouple model from appmessage.
* [NOD-1413] Decouple model from util.
* [NOD-1413] Decouple model from subnetworkid.
* [NOD-1413] Remove an unused const.
* [NOD-1413] Add DomainHash and DomainTransactionID.
* [NOD-1413] Decouple model from daghash.
* [NOD-1413] Decouple model from mstime.
* [NOD-1413] Decouple model from go-secp256k1.
* [NOD-1413] Add a proxy over dbaccess.
* [NOD-1413] Add comments over all added types.
* [NOD-1413] Fix a comment.
* [NOD-1413] Get rid of DomainTime.
* [NOD-1413] Simplify BlockGHOSTDAGData.
* [NOD-1414] Add interfaces for Factory and State.
* [NOD-1414] Create interfaces for algorithms and data stores.
* [NOD-1414] Create empty implementations for algorithms and data stores.
* [NOD-1414] Add new functions for all the implementations.
* [NOD-1414] Begin filling in the interfaces.
* [NOD-1414] Fill in the interfaces for the data structures.
* [NOD-1414] Fill in the interfaces for the algorithms.
* [NOD-1414] Fix a bug in package names.
* [NOD-1414] Connect up the various interfaces.
* [NOD-1414] Add stubs to all the implementations.
* [NOD-1414] Create MiningManager and its Factory.
* [NOD-1414] Add interfaces for mempool and blockTemplateBuilder.
* [NOD-1414] Add implementation structs for miningManager.
* [NOD-1414] Add stub implementations for mempool and blockTemplateBuilder.
* [NOD-1414] Rename state to kaspadState.
* [NOD-1414] Restructure where interfaces sit.
* [NOD-1414] Restructure where interfaces sit in the algorithms package as well.
* [NOD-1414] Move remaining models out of models.go.
* [NOD-1414] Modified some interfaces.
* [NOD-1414] Make go vet happy.
* [NOD-1414] Move SerializedUTXOSet into PruningManager.
* [NOD-1414] Modify FindNextPruningPoint to return found and nextPruningPointUTXOSet.
* [NOD-1414] Add IsDAGAncestorOf.
* [NOD-1414] Add PruningPoint().
* [NOD-1414] Add Entry() to ReadOnlyUTXOSet.
* [NOD-1414] Add MergeSet() to BlockGHOSTDAGData.
* [NOD-1414] Write comments for all the exported types and functions in miningmanager.
* [NOD-1414] Add comments to the upper levels of KaspadState.
* [NOD-1414] Replace AddNode with ReachabilityChangeset.
* [NOD-1414] Add payAddress and extraData to GetBlockTemplate.
* [NOD-1414] Add scriptPublicKey and extraData to BuildBlock.
* [NOD-1414] Rename algorithms to processes.
* [NOD-1414] Rename kaspadState to consensus.
* [NOD-1414] Add ValidateAgainstPastUTXO and ValidateFinality.
* [NOD-1414] Add BlockGHOSTDAGData to ReachabilityChangeset.
* [NOD-1414] Fix the comment over Mempool.
* [NOD-1414] Fix the comment over ValidateTransaction.
* [NOD-1414] Fill up the data structures.
* [NOD-1414] Add comments to remaining uncommented items in miningmanager.
* [NOD-1414] Add comments to structs and constructors.
* [NOD-1414] Rename Set to Insert.
* [NOD-1414] Add comments to everything inside datastructures.
* [NOD-1414] Add comments to everything inside models.
* [NOD-1414] Add comments to the interfaces in processes.
* [NOD-1414] Add comments to everything in processes.
* [NOD-1414] Make go vet happy.
* [NOD-1414] Rename scriptPublicKey to coinbaseScriptPublicKey.
* [NOD-1414] Add handlers to the consensus.
* [NOD-1414] Add highHash to blockAtDepth.
* [NOD-1414] Add resolveFinalityConflict.
* [NOD-1414] Reorg BlockValidator.
* [NOD-1414] In ResolveFinalityConflicts, rename blockHash to newFinalityBlockHash.
* [NOD-1414] Fix a comment.
* [NOD-1414] Make reachability structs public.
* [NOD-1414] Make UTXO structs public.
