* Create BlockParentBuilder.
* Implement BuildParents.
* Explictly set level 0 blocks to be the same as direct parents.
* Add checkIndirectParents to validateBlockHeaderInContext.
* Fix test_block_builder.go and BlockLevelParents::Equal.
* Don't check indirect parents for blocks with trusted data.
* Handle pruned blocks when building block level parents.
* Fix bad deletions from unprocessedXxxParents.
* Fix merge errors.
* Fix bad pruning point parent replaces.
* Fix duplicates in newBlockLevelParents.
* Skip checkIndirectParents
* Get rid of staging constant IDs
* Fix BuildParents
* Fix tests
* Add comments
* Change order of directParentHashes
* Get rid of maybeAddDirectParentParents
* Add comments
* Add blockToReferences type
* Use ParentsAtLevel
Co-authored-by: stasatdaglabs <stas@daglabs.com>
* Replace header finality point with pruning point
* Fix TestTransactionAcceptance
* Fix pruning candidate
* Store all past pruning points
* Pass pruning points on IBD
* Add blue score to block header
* Simplify ArePruningPointsInValidChain
* Fix static check errors
* Fix genesis
* Renames and text fixing
* Use ExpectedHeaderPruningPoint in block builder
* Fix TestCheckPruningPointViolation
* Add prefix to stores
* Add prefix to forgotten stores
* Add a special type for prefix
* Rename transaction->dbTx
* Change error message
* Use countKeyName
* Rename Temporary Consesnsus to Staging
* Add DeleteStagingConsensus to Domain interface
* Add lock to staging consensus
* Make prefix type-safer
* Use ioutil.TempDir instead of t.TempDir
* Add oldPruningPoint to pruningStore
* Make the pruning store work with utxo diff and return an iterator over pruning point utxoset
* Redesign pruning point utxo storage by creating a diff and modifying the old pruning utxo set
* Fix review comments
* Rename updatePruningPointUTXOSet
* Add StagingArea struct
* Implemented staging areas in blockStore
* Move blockStagingShard to separate folder
* Apply staging shard to acceptanceDataStore
* Update blockHeaderStore with StagingArea
* Add StagingArea to BlockRelationStore
* Add StagingArea to blockStatusStore
* Add StagingArea to consensusStateStore
* Add StagingArea to daaBlocksStore
* Add StagingArea to finalityStore
* Add StagingArea to ghostdagDataStore
* Add StagingArea to headersSelectedChainStore and headersSelectedTipStore
* Add StagingArea to multisetStore
* Add StagingArea to pruningStore
* Add StagingArea to reachabilityDataStore
* Add StagingArea to utxoDiffStore
* Fix forgotten compilation error
* Update reachability manager and some more things with StagingArea
* Add StagingArea to dagTopologyManager, and some more
* Add StagingArea to GHOSTDAGManager, and some more
* Add StagingArea to difficultyManager, and some more
* Add StagingArea to dagTraversalManager, and some more
* Add StagingArea to headerTipsManager, and some more
* Add StagingArea to constnsusStateManager, pastMedianTimeManager
* Add StagingArea to transactionValidator
* Add StagingArea to finalityManager
* Add StagingArea to mergeDepthManager
* Add StagingArea to pruningManager
* Add StagingArea to rest of ValidateAndInsertBlock
* Add StagingArea to blockValidator
* Add StagingArea to coinbaseManager
* Add StagingArea to syncManager
* Add StagingArea to blockBuilder
* Update consensus with StagingArea
* Add StagingArea to ghostdag2
* Fix remaining compilation errors
* Update names of stagingShards
* Fix forgotten stagingArea passing
* Mark stagingShard.isCommited = true once commited
* Move isStaged to stagingShard, so that it's available without going through store
* Make blockHeaderStore count be avilable from stagingShard
* Fix remaining forgotten stagingArea passing
* commitAllChanges should call dbTx.Commit in the end
* Fix all tests tests in blockValidator
* Fix all tests in consensusStateManager and some more
* Fix all tests in pruningManager
* Add many missing stagingAreas in tests
* Fix many tests
* Fix most of all other tests
* Fix ghostdag_test.go
* Add comment to StagingArea
* Make list of StagingShards an array
* Add comment to StagingShardID
* Make sure all staging shards are pointer-receiver
* Undo bucket rename in block_store
* Typo: isCommited -> isCommitted
* Add comment explaining why stagingArea.shards is an array
* Add Close() function to all the iterators
* Add defer iterator.Close() whenever we open an iterator
* Add isClosed to all iterators and panic/return error if used after closing
Co-authored-by: Svarog <feanorr@gmail.com>
* Implement TestGetPruningPointUTXOs.
* Fix a bad error message.
* Fix TestGetPruningPointUTXOs for testnet.
* Make sure all the UTXOs are returned in TestGetPruningPointUTXOs.
* Implement BenchmarkGetPruningPointUTXOs.
* Pass fromOutpoint into GetPruningPointUTXOs instead of offset, so it could Seek over the cursor.
* Fix weird benchmark timer calls.
* Remove unnecessary collection of outpointAndUTXOEntryPairs from BenchmarkGetPruningPointUTXOs.
* Fix a comment.
* Remove pruningPointUTXOSetStaging and implement UpdatePruningPointUTXOSet.
* Implement StageStartSavingNewPruningPointUTXOSet, HadStartedSavingNewPruningPointUTXOSet, and FinishSavingNewPruningPointUTXOSet.
* Fix a bad return.
* Implement UpdatePruningPointUTXOSetIfRequired.
* Call UpdatePruningPointUTXOSetIfRequired on consensus creation.
* Rename savingNewPruningPointUTXOSetKey to updatingPruningPointUTXOSet.
* Add a log.
* Add calls to runtime.GC() at its start and end.
* Rename a variable.
* Replace calls to runtime.GC to calls to LogMemoryStats.
* Wrap the contents of LogMemoryStats in a log closure.
* Replaced the content of MsgIBDRootUTXOSetChunk with pairs of outpoint-utxo entry pairs.
* Rename utxoIter to utxoIterator.
* Add a big stinky TODO on an assert.
* Replace pruningStore staging with a UTXO set iterator.
* Reimplement receiveAndInsertIBDRootUTXOSet.
* Extract OutpointAndUTXOEntryPairsToDomainOutpointAndUTXOEntryPairs into domainconverters.go.
* Pass the outpoint and utxy entry pairs to the pruning store.
* Implement InsertCandidatePruningPointUTXOs.
* Implement ClearCandidatePruningPointUTXOs.
* Implement UpdateCandidatePruningPointMultiset.
* Use the candidate pruning point multiset in updatePruningPoint.
* Implement CandidatePruningPointUTXOIterator.
* Use the pruning point utxo set iterator for StageVirtualUTXOSet.
* Defer ClearCandidatePruningPointUTXOs.
* Implement OverwriteVirtualUTXOSet.
* Implement CommitCandidatePruningPointUTXOSet.
* Implement BeginOverwritingVirtualUTXOSet and FinishOverwritingVirtualUTXOSet.
* Implement overwriteVirtualUTXOSetAndCommitPruningPointUTXOSet.
* Rename ClearCandidatePruningPointUTXOs to ClearCandidatePruningPointData.
* Add missing methods to dbManager.
* Implement PruningPointUTXOs.
* Implement RecoverUTXOIfRequired.
* Delete the utxoserialization package.
* Fix compilation errors in TestValidateAndInsertPruningPoint.
* Switch order of operations in the if statements in PruningPointUTXOs so that Next() wouldn't be unnecessarily called.
* Fix missing pruning point utxo set staging and bad slice length.
* Fix no default multiset in InsertCandidatePruningPointUTXOs.
* Make go vet happy.
* Rename candidateXXX to importedXXX.
* Do some more renaming.
* Rename some more.
* Fix bad MsgIBDRootNotFound logic.
* Fix an error message.
* Simplify receiveIBDRootBlock.
* Fix error message in receiveAndInsertIBDRootUTXOSet.
* Do some more renaming.
* Fix merge errors.
* Fix a bug caused by calling iterator.First() unnecessarily.
* Remove databaseContext from stores and don't use a transaction in ClearXXX functions.
* Simplify receiveAndInsertIBDRootUTXOSet.
* Fix offset count in PruningPointUTXOs().
* Fix readOnlyUTXOIteratorWithDiff.First().
* Split handleRequestIBDRootUTXOSetAndBlockFlow into smaller methods.
* Rename IbdRootNotFound to UnexpectedPruningPoint.
* Rename requestIBDRootHash to requestPruningPointHash.
* Rename IBDRootHash to PruningPointHash.
* Rename RequestIBDRootUTXOSetAndBlock to RequestPruningPointUTXOSetAndBlock.
* Rename IBDRootUTXOSetChunk to PruningPointUTXOSetChunk.
* Rename RequestNextIBDRootUTXOSetChunk to RequestNextPruningPointUTXOSetChunk.
* Rename DoneIBDRootUTXOSetChunks to DonePruningPointUTXOSetChunks.
* Rename remaining references to IBD root.
* Fix an error message.
* Add a check for HadStartedImportingPruningPointUTXOSet in commitVirtualUTXODiff.
* Add a check for HadStartedImportingPruningPointUTXOSet in ImportPruningPointUTXOSetIntoVirtualUTXOSet.
* Move FinishImportingPruningPointUTXOSet closer to HadStartedImportingPruningPointUTXOSet.
* Remove reference to pruningStore in utxoSetIterator.
* Pointerify utxoSetIterator receivers.
* Fix bad insert in CommitImportedPruningPointUTXOSet.
* Rename commitImportedPruningPointUTXOSetAll to applyImportedPruningPointUTXOSet.
* Simplify PruningPointUTXOs.
* Add populateTransactionWithUTXOEntriesFromUTXOSet.
* Fix a TODO comment.
* Rename InsertImportedPruningPointUTXOs to AppendImportedPruningPointUTXOs.
* Extract handleRequestPruningPointUTXOSetAndBlockMessage to a separate method.
* Rename stuff in readOnlyUTXOIteratorWithDiff.First().
* Address toAddIterator in readOnlyUTXOIteratorWithDiff.First().
* Call First() before any full iteration on ReadOnlyUTXOSetIterator.
* Call First() before any full iteration on a database Cursor.
* Put StartImportingPruningPointUTXOSet inside the pruning point transaction.
* Make serializeOutpoint and serializeUTXOEntry free functions in pruningStore.
* Fix readOnlyUTXOIteratorWithDiff.First().
* Fix bad validations in importPruningPoint.
* Remove superfluous call to validateBlockTransactionsAgainstPastUTXO.
* Increase size of reachability cache
* Change DomainHash to struct with unexported hashArray
* Fixing compilation errors stemming from new DomainHash structure
* Remove obsolete Read/WriteElement methods in appmessage
* Fix all tests
* Fix all tests
* Add comments
* A few renamings
* go mod tidy
* 1) Calculate pruning point incrementally
2) Add IsValidPruningPoint to pruning manager and consensus
3) Use reachability children for selected child iterator
* Add request IBD root hash flow
* Fix UpdatePruningPointByVirtual and IsValidPruningPoint
* Regenerate messages.pb.go
* Make the pruning point the earliest chain block with finality interval higher than the previous pruning point
* Fix merge errors
* [NOD-1566] Add a dependency to golang-lru.
* [NOD-1566] Add caching to blockstore.go.
* [NOD-1566] Add LRUCache to all store objects and initialize them.
* [NOD-1566] Add caching to acceptanceDataStore.
* [NOD-1566] Add caching to blockHeaderStore.
* [NOD-1566] Implement a simpler LRU cache.
* [NOD-1566] Use the simpler cache implementation everywhere.
* [NOD-1566] Remove dependency in golang-lru.
* [NOD-1566] Fix object reuse issues in store Get functions.
* [NOD-1566] Add caching to blockRelationStore.
* [NOD-1566] Add caching to blockStatusStore.
* [NOD-1566] Add caching to ghostdagDataStore.
* [NOD-1566] Add caching to multisetStore.
* [NOD-1566] Add caching to reachabilityDataStore.
* [NOD-1566] Add caching to utxoDiffStore.
* [NOD-1566] Add caching to reachabilityReindexRoot.
* [NOD-1566] Add caching to pruningStore.
* [NOD-1566] Add caching to headerTipsStore.
* [NOD-1566] Add caching to consensusStateStore.
* [NOD-1566] Add comments explaining why we don't discard staging at the normal location in consensusStateStore.
* [NOD-1566] Make go vet happy.
* [NOD-1566] Fix merge errors.
* [NOD-1566] Add a missing break statement.
* [NOD-1566] Run go mod tidy.
* [NOD-1566] Remove serializedUTXOSetCache.
* [NOD-1528] Make data stores copy data on stage
* [NOD-1528] Add proto objects to serialize consensus state objects
* [NOD-1528] Fix receiver names
* [NOD-1528] Add copy to block store and utxo diff staging
* [NOD-1528] Return errors where needed
* Update pruningmanager interface
* Add a ProtoUTXOSet to hashserialization
* Update miningmanager with all the necessary stores and managers
* Implement mining manager
* Prune P.AC not in V.Past
* PruningManager fix all review comments
* [NOD-1496] Implement headers only verification
* [NOD-1496] Add checkParentsExist
* [NOD-1496] Stage block statuses in block processor
* [NOD-1496] Rename AddBlock->AddHeaderTip
* [NOD-1496] Return early from validateAndInsertBlock on header only and put ValidateProofOfWorkAndDifficulty inside validateBlock
* [NOD-1461] Change the external api interface to not having anything besides DomainTransactions and DomainBlocks.
* [NOD-1462] Move external api types to a separate package.
* [NOD-1462] Clarify which model we're using in miningmanager.
* [NOD-1462] Extract coinbase data to its own struct.
* [NOD-1462] Add a comment above CoinbaseData.
* [NOD-1462] Fix the comment above CoinbaseData.
* [NOD-1458] Rename RestoreUTXOSet to RestorePastUTXOSet.
* [NOD-1458] Make CalculateAcceptanceDataAndMultiset take BlockGHOSTDAGData and nothing else.
* [NOD-1458] Make ConsensusStateStore's Update take ConsensusStateChanges instead of just UTXODiff.
* [NOD-1458] Add Tips() to ConsensusStateStore.
* [NOD-1458] Make all implementation structs private.
* [NOD-1458] Remove BlockAtDepth and add highHash to ChainBlockAtBlueScore.
* [NOD-1458] Rename CalculateAcceptanceDataAndMultiset to CalculateAcceptanceDataAndUTXOMultiset.
* [NOD-1458] Add a dependency to GHOSTDAGManager from ConsensusStateManager.
* [NOD-1458] Add ChooseSelectedParent to GHOSTDAGManager.
* [NOD-1458] Add DifficultyManager.
* [NOD-1458] Add PastMedianTimeManager.
* [NOD-1458] Add Hash() to Multiset.
* [NOD-1458] Add a dependency to ghostdagManager from blockProcessor.
* [NOD-1458] Add errors to all interfaces that need them.
* [NOD-1458] Uppercasify types in comments.
* [NOD-1458] Fix a bad comment.
* [NOD-1458] Fix a comment.
* [NOD-1458] Rename ChainBlockAtBlueScore to HighestChainBlockBelowBlueScore.
* [NOD-1458] Replace BlockAndTransactionValidator with an anonymous interface.
* [NOD-1413] Remove /cmd/addblock
* [NOD-1413] Define and implement TransactionValidator.
* [NOD-1413] Make changes to ConsensusStateManager's interface.
* [NOD-1413] Make changes to PruningManager's interface.
* [NOD-1413] Make changes to DAGTraversalManager's interface.
* [NOD-1413] Make changes to MultisetStore's interface.
* [NOD-1413] Make changes to UTXODiffStore's interface.
* [NOD-1413] Make changes to UTXODiffStore's interface harder.
* [NOD-1413] Make changes to AcceptanceDataStore's interface harder.
* [NOD-1413] Make changes to PruningStore's interface.
* [NOD-1413] Delete BlockIndex.
* [NOD-1413] Add FeeDataStore.
* [NOD-1413] Update BlockMessageStore's interface.
* [NOD-1413] Fix interface violations.
* [NOD-1413] Add FeeDataStore to BlockProcessor.
* [NOD-1413] Make go vet happy.
* [NOD-1413] Add missing fields to ConsensusStateChanges.
* [NOD-1413] Add another missing field to ConsensusStateChanges.
* [NOD-1413] Add a reference to blockStore in consensusStateManager.
* [NOD-1413] Add missing methods to UTXODiffStore.
* [NOD-1413] Rename pruningPointStore to pruningStore everywhere.
* [NOD-1413] Remove superfluous parameters from CalculateConsensusStateChanges.
* [NOD-1413] Add missing dependencies to PruningManager.
* [NOD-1413] Remove implementation-y functions from TransactionValidator's interface.
* [NOD-1413] Make go vet happy.
* [NOD-1413] Add a couple of methods to DAGTopologyManager.
* [NOD-1413] Fix a typo in a file name.
* [NOD-1413] Remove non-interface functions from Validator.
