* [NOD-1105] Don't use a database transaction when recovering acceptance index.
* Revert "[NOD-1105] Don't use a database transaction when recovering acceptance index."
This reverts commit da550f8e
* [NOD-1105] When recovering acceptance index, use a database transaction per block instead of for the entire recovery.
commit e303efef4209b8d62f1aac2cb57ac79829411556
Author: stasatdaglabs <stas@daglabs.com>
Date: Mon Jun 29 11:59:36 2020 +0300
[NOD-1063] Rename a test.
commit bfecd57470ec8aeb0a1b0ef82c051dde364c536e
Author: stasatdaglabs <stas@daglabs.com>
Date: Mon Jun 29 11:57:36 2020 +0300
[NOD-1063] Fix a comment.
commit b969e5922da16a3734806c03075fe2e45e64958b
Author: stasatdaglabs <stas@daglabs.com>
Date: Sun Jun 28 18:14:44 2020 +0300
[NOD-1063] Convert modifiedTreeNode to an out param.
commit 170f9872f432b2f7177cb099149c9959031e4f1e
Author: stasatdaglabs <stas@daglabs.com>
Date: Sun Jun 28 17:05:01 2020 +0300
[NOD-1063] Fix a bug in which a block is added to the selected parent chain below the reindex root.
* [NOD-1055] Give higher priority for requesting missing ancestors when sending a getdata message (#767)
* [NOD-1063] Remove the remainingInterval field.
* [NOD-1063] Add helper functions to reachabilityTreeNode.
* [NOD-1063] Add reachabilityReindexRoot.
* [NOD-1063] Start implementing findNextReachabilityReindexRoot.
* [NOD-1063] Implement findCommonAncestor.
* [NOD-1063] Implement findReachabilityTreeAncestorInChildren.
* [NOD-1063] Add reachabilityReindexWindow.
* [NOD-1063] Fix findReachabilityTreeAncestorInChildren.
* [NOD-1063] Remove BlockDAG reference in findReachabilityTreeAncestorInChildren.
* [NOD-1063] Extract updateReachabilityReindexRoot to a separate function.
* [NOD-1063] Add reachabilityReindexSlack.
* [NOD-1063] Implement splitReindexRootChildrenAroundChosen.
* [NOD-1063] Implement calcReachabilityTreeNodeSizes.
* [NOD-1063] Implement propagateChildIntervals.
* [NOD-1063] Extract tightenReachabilityTreeIntervalsBeforeChosenReindexRootChild and tightenReachabilityTreeIntervalsAfterChosenReindexRootChild to separate functions.
* [NOD-1063] Implement expandReachabilityTreeIntervalInChosenReindexRootChild.
* [NOD-1063] Finished implementing concentrateReachabilityTreeIntervalAroundReindexRootChild.
* [NOD-1063] Begin implementing reindexIntervalsBeforeReindexRoot.
* [NOD-1063] Implement top-level logic of reindexIntervalsBeforeReindexRoot.
* [NOD-1063] Implement reclaimIntervalBeforeChosenChild.
* [NOD-1063] Add a debug log for reindexIntervalsBeforeReindexRoot.
* [NOD-1063] Rename reindexIntervalsBeforeReindexRoot to reindexIntervalsEarlierThanReindexRoot.
* [NOD-1063] Implement reclaimIntervalAfterChosenChild.
* [NOD-1063] Add a debug log for updateReachabilityReindexRoot.
* [NOD-1063] Convert modifiedTreeNodes from slices to sets.
* [NOD-1063] Fix findCommonAncestor.
* [NOD-1063] Fix reindexIntervalsEarlierThanReindexRoot.`
* [NOD-1063] Remove redundant nil conditions.
* [NOD-1063] Make map[*reachabilityTreeNode]struct{} into a type alias with a copyAllFrom method.
* [NOD-1063] Remove setInterval.
* [NOD-1063] Create a new struct to hold reachability stuff called reachabilityTree.
* [NOD-1063] Rename functions under reachabilityTree.
* [NOD-1063] Move reachabilityStore into reachabilityTree.
* [NOD-1063] Move the rest of the functions in reachability.go into the reachabilityTree struct.
* [NOD-1063] Update newReachabilityTree to take an instance of reachabilityStore.
* [NOD-1063] Fix merge errors.
* [NOD-1063] Fix merge errors.
* [NOD-1063] Pass a reference to the dag into reachabilityTree.
* [NOD-1063] Use Wrapf instead of Errorf.
* [NOD-1063] Merge assignments.
* [NOD-1063] Disambiguate a varaible name.
* [NOD-1063] Add a test case for intervalBefore.
* [NOD-1063] Simplify splitChildrenAroundChosenChild.
* [NOD-1063] Fold temporary variables into newReachabilityInterval.
* [NOD-1063] Fold more temporary variables into newReachabilityInterval.
* [NOD-1063] Fix a bug in expandIntervalInReindexRootChosenChild.
* [NOD-1063] Remove blockNode from futureCoveringBlock.
* [NOD-1063] Get rid of futureCoveringBlock.
* [NOD-1063] Use findIndex directly in findAncestorAmongChildren.
* [NOD-1063] Make findIndex a bit nicer to use. Also rename it to findAncestorIndexOfNode.
* [NOD-1063] Rename childIntervalAllocationRange to intervalRangeForChildAllocation.
* [NOD-1063] Optimize findCommonAncestor.
* [NOD-1063] In reindexIntervalsBeforeChosenChild, use chosenChild.interval.start - 1 instead of childrenBeforeChosen[len(childrenBeforeChosen)-1].interval.end + 1.
* [NOD-1063] Rename reindexIntervalsBeforeChosenChild to reindexIntervalsBeforeNode.
* [NOD-1063] Add a comment explain what "the chosen child" is.
* [NOD-1063] In concentrateIntervalAroundReindexRootChosenChild, rename modifiedTreeNodes to allModifiedTreeNodes.
* [NOD-1063] Extract propagateIntervals to a function.
* [NOD-1063] Extract interval "contains" logic to a separate function.
* [NOD-1063] Simplify "looping up" logic in reclaimIntervalXXXChosenChild.
* [NOD-1063] Add comments to reclaimIntervalXXXChosenChild.
* [NOD-1063] Rename copyAllFrom to addAll.
* [NOD-1063] Rename reachabilityStore (the variable) to just store.
* [NOD-1063] Fix an error message.
* [NOD-1063] Reword a comment.
* [NOD-1063] Don't return -1 from findAncestorIndexOfNode.
* [NOD-1063] Extract slackReachabilityIntervalForReclaiming to a constant.
* [NOD-1063] Add a missing condition.
* [NOD-1063] Call isAncestorOf directly in insertNode.
* [NOD-1063] Rename chosenReindexRootChild to reindexRootChosenChild.
* [NOD-1063] Rename treeNodeSet to orderedTreeNodeSet.
* [NOD-1063] Add a disclaimer to orderedTreeNodeSet.
* [NOD-1063] Implement StoreReachabilityReindexRoot and FetchReachabilityReindexRoot.
* [NOD-1063] Move storing the reindex root to within reachabilityTree.
* [NOD-1063] Remove isAncestorOf from reachabilityInterval.
* [NOD-1063] Add a comment about graph theory conventions.
* [NOD-1063] Fix tests.
* [NOD-1063] Change inclusion in isAncestorOf functions.
* [NOD-1063] Rename a test.
* [NOD-1063] Implement TestIsInFuture.
* [NOD-1063] Fix error messages in TestIsInFuture.
* [NOD-1063] Fix error messages in TestIsInFuture.
* [NOD-1063] Rename isInSelectedParentChain to isInSelectedParentChainOf.
* [NOD-1063] Rename isInFuture to isInPast.
* [NOD-1063] Expand on a comment.
* [NOD-1063] Rename modifiedTreeNodes.
* [NOD-1063] Implement test: TestReindexIntervalsEarlierThanReindexRoot.
* [NOD-1063] Implement test: TestUpdateReindexRoot.
* [NOD-1063] Explain a check.
* [NOD-1063] Use a method instead of calling reachabilityStore.loaded directly.
* [NOD-1063] Lowercasified an error message.
* [NOD-1063] Fix failing test.
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* [NOD-1034] Use reachability to check finality
* [NOD-1034] Add comments and rename variables
* [NOD-1034] Fix comments
* [NOD-1034] Rename checkFinalityRules->checkFinalityViolation
* [NOD-1034] Change isAncestorOf to be exclusive
* [NOD-1034] Make isAncestorOf exclusive and also more explicit, and add TestReachabilityTreeNodeIsAncestorOf
* [NOD-614] Copy bitcoin-core ban score policy
* [NOD-614] Add ban score to disconnects
* [NOD-614] Fix wrong branch of AddBanScore
* [NOD-614] Add ban score on sending too many addresses
* [NOD-614] Add comments
* [NOD-614] Remove redundant reject messages
* [NOD-614] Fix log message
* [NOD-614] Ban every node that sends invalid invs
* [NOD-614] Make constants for ban scores
* [NOD-1007] Split checkBlockSanity subroutines.
* [NOD-1007] Put back the comments about performance.
* [NOD-1007] Make all the functions in checkBlockSanity take a *util.Block.
* [NOD-1007] Rename checkBlockTransactionsOrderedBySubnetwork to checkBlockTransactionOrder.
* [NOD-1007] Move a comment up a scope level.
* [NOD-1048] Make leveldb compaction much less frequent. Also, allocate an entire gigabyte for leveldb's blockCache and writeBuffer.
* [NOD-1048] Implement changing the options for testing purposes.
* [NOD-1048] Rename originalOptions to originalLDBOptions.
* [NOD-1048] Add a comment.
* [NOD-1040] Don't remove DAG tips from the diffStore's loaded set
* [NOD-1040] Fix TestClearOldEntries.
* Revert "[NOD-1040] Fix TestClearOldEntries."
This reverts commit e0705814
* Revert "[NOD-1040] Don't remove DAG tips from the diffStore's loaded set"
This reverts commit d3eba1c1
* [NOD-1040] Increase maxBlueScoreDifferenceToKeepLoaded to 1500.
* [NOD-1012] Disallow non-native/coinbase transactions.
* [NOD-1012] Fix logic error.
* [NOD-1012] Fix/skip tests and remove --subnetwork.
* [NOD-1012] Disconnect from non-native peers.
* [NOD-1012] Don't skip subnetwork tests.
* [NOD-1012] Use EnableNonNativeSubnetworks in peer.go.
* [NOD-1012] Set EnableNonNativeSubnetworks = true in the tests that need them rather than by default in Simnet.
* [NOD-965] Make dag.index.LookupNode return false if node is not found
* [NOD-965] Rename blockDAG->dag
* [NOD-965] Remove irrelevant test
* [NOD-965] Use bi.index's ok in LookupNode
* [NOD-975] Don't include block transactions inside its UTXO commitment (#711)
* [NOD-975] Don't include block transactions inside its UTXO commitment.
* Revert "[NOD-975] Don't include block transactions inside its UTXO commitment."
This reverts commit b1a2ae66
* [NOD-975] Implement a (currently failing) TestUTXOCommitment.
* [NOD-975] Remove the block's own transactions from calcMultiset.
* [NOD-975] Simplify calcMultiset.
* [NOD-975] Add a comment on top of selectedParentMultiset.
* [NOD-975] Use pastUTXO instead of selectedParentUTXO in calcMultiset.
* [NOD-975] Use selected parent's pastUTXO instead of this block's pastUTXO in calcMultiset.
* [NOD-975] Extract selectedParentPastUTXO to a separate function.
* [NOD-975] Remove selectedParentUTXO from pastUTXO's return values.
* [NOD-975] Add txs to TestUTXOCommitment.
* [NOD-975] Remove debug code.
* [NOD-975] In pastUTXOMultiSet, copy the multiset to avoid modifying the original.
* [NOD-975] Add a test: TestPastUTXOMultiSet.
* [NOD-975] Improve TestPastUTXOMultiSet.
* [NOD-976] Implement tests for UTXO commitments (#715)
* [NOD-975] Don't include block transactions inside its UTXO commitment.
* Revert "[NOD-975] Don't include block transactions inside its UTXO commitment."
This reverts commit b1a2ae66
* [NOD-975] Implement a (currently failing) TestUTXOCommitment.
* [NOD-975] Remove the block's own transactions from calcMultiset.
* [NOD-975] Simplify calcMultiset.
* [NOD-975] Add a comment on top of selectedParentMultiset.
* [NOD-975] Use pastUTXO instead of selectedParentUTXO in calcMultiset.
* [NOD-975] Use selected parent's pastUTXO instead of this block's pastUTXO in calcMultiset.
* [NOD-975] Extract selectedParentPastUTXO to a separate function.
* [NOD-975] Remove selectedParentUTXO from pastUTXO's return values.
* [NOD-975] Add txs to TestUTXOCommitment.
* [NOD-976] Generate new blockDB blocks for tests.
* [NOD-976] Fix TestBlueBlockWindow.
* [NOD-976] Fix TestIsKnownBlock.
* [NOD-976] Fix TestGHOSTDAG.
* [NOD-976] Fix TestUTXOCommitment.
* [NOD-976] Remove kaka.
* [NOD-990] Save utxo diffs of past UTXO (#724)
* [NOD-990] Save UTXO diffs of past UTXO
* [NOD-990] Check for block double spends with its past instead of building its UTXO
* [NOD-990] Call resetExtraNonceForTest in TestUTXOCommitment
* [NOD-990] Remove redundant functions diffFromTx and diffFromAcceptedTx
* [NOD-990] Rename i->j to avoid confusion
* [NOD-990] Break long lines
* [NOD-990] Rename ErrDoubleSpendsWithBlockTransaction -> ErrDoubleSpendInSameBlock
* [NOD-990] Make ErrDoubleSpendInSameBlock more detailed
* [NOD-990] Add testProcessBlockRuleError
* [NOD-990] Fix comment
* [NOD-990] Add test for duplicate transactions on the same block
* [NOD-990] Use pkg/errors on panic
* [NOD-990] Make cloneWithoutBase method
* [NOD-990] Break long lines
* [NOD-990] Fix comment
* [NOD-990] Fix wrong variable names
* [NOD-990] Fix comment
* [NOD-974] Generate new test blocks.
* [NOD-974] Fix TestIsKnownBlock and TestGHOSTDAG.
* [NOD-974] Fix TestUTXOCommitment.
* [NOD-974] Fix comments
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
Co-authored-by: stasatdaglabs <stas@daglabs.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* [NOD-877] In UTXOEntry serialization, extract packedFlags out to a separate Uint8.
* [NOD-877] Generate new test blocks.
* [NOD-877] Fix TestIsKnownBlock.
* [NOD-877] Fix TestBlueBlockWindow.
* [NOD-877] Fix TestUTXOSerialization and TestGHOSTDAG.
* [NOD-877] Fix TestVirtualBlock.
* [NOD-1006] Make CompactToBig take an out param so that we can reuse the same big.Int in averageTarget.
* [NOD-1006] Fix merge errors.
* [NOD-1006] Use CompactToBigWithDestination only in averageTarget.
* [NOD-1006] Fix refactor errors.
* [NOD-1006] Fix refactor errors.
* [NOD-1006] Optimize averageTarget with a big.Int pool.
* [NOD-1006] Defer releasing bigInts.
* [NOD-1006] Use a pool for requiredDifficulty as well.
* [NOD-1006] Move the big int pool to utils.
* [NOD-1006] Remove unnecessary line.
* [NOD-993] Use %+v when printing errors
* [NOD-993] Get rid of AssertError
* [NOD-993] Made ruleError use github.com/pkg/errors
* [NOD-993] remove redundant TODO
* [NOD-993] remove redundant Comment
* [NOD-993] Removed DeploymentError
* [NOD-909] Add tests for double spends
* [NOD-909] Add prepareAndProcessBlock that gets parent hashes and transactions as argument
* [NOD-909] Use PrepareAndProcessBlockForTest where possible
* [NOD-909] Use more meaningful names
* [NOD-909] Change a comment
* [NOD-909] Fix comment
* [NOD-909] Fix comment
* [NOD-828] Reimplement FFLDB (#663)
* [NOD-828] Create the database2 package that will some day replace the database package.
* [NOD-828] Implement a "bucket" key mechanism.
* [NOD-828] Move bucket.go into the ffldb2 package.
* [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway.
* [NOD-828] Copy over + fixup flat file structs from the old ffldb.
* [NOD-828] Implement flatFilePath.
* [NOD-828] Implement flatFileStore.write().
* [NOD-828] Implement flatFileStore.read().
* [NOD-828] Implement flatFileStore.rollback().
* [NOD-828] Sync the file to disk at the end of write().
* [NOD-828] Extract crc32ByteOrder to a separate variable.
* [NOD-828] Add a sanity test.
* [NOD-828] Remove context-unrelated methods from the Database interface.
* [NOD-828] Create an ffldb object. Simply work against a context.
* [NOD-828] Open the new database on start.
* [NOD-828] Create the leveldb package.
* [NOD-828] Implement opening/closing leveldb.
* [NOD-828] Implement get/put out of/into leveldb.
* [NOD-828] Implement transactions and make them implement a generic database interface.
* [NOD-828] Write sanity tests for leveldb with and without transactions.
* [NOD-828] Add another case to the transaction sanity test.
* [NOD-828] Implement AppendBlock/RetrieveBlock.
* [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package.
* [NOD-828] Add RollbackFlatData to the database interface.
* [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket.
* [NOD-828] Implement StoreBlock.
* [NOD-828] Implement FetchBlock.
* [NOD-828] Implement HasBlock.
* [NOD-828] Write a sanity test for block insertion.
* [NOD-828] Implement CurrentFlatDataLocation.
* [NOD-828] Implement storing the current block location.
* [NOD-828] Implement initializing/syncing the flat file block store and the "metadata".
* [NOD-828] Add InitBlockStore to TestBlockStoreSanity.
* [NOD-828] Fix rename errors.
* [NOD-828] Fix lint errors in the root database package.
* [NOD-828] Fix lint errors in the ffldb.go.
* [NOD-828] Fix lint errors in the flatfile/db.go.
* [NOD-828] Rename packages in such a way to make the linter happy.
* [NOD-828] Finish satisfying the linter.
* [NOD-828] Fix doc.go.
* [NOD-828] Fix comments in block.go.
* [NOD-828] Move dbaccess out of the database package.
* [NOD-828] Move opening/closing the database to dbaccess.
* [NOD-828] Move the Database interface to the root database package, since it's meant to be generic.
* [NOD-828] Make ffldb generic to later support additional databases.
* [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported.
* [NOD-828] Fix comments.
* [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore.
* [NOD-828] Make buckets nicer to use.
* [NOD-828] Implement cursors that iterate over some bucket.
* [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb.
* [NOD-828] Write a test making sure that flat file repair works.
* [NOD-828] Properly close the database in TestRepairFlatFiles.
* [NOD-828] Add a comment warning against putting and getting the same data within the same transaction.
* [NOD-828] Fix the flatFilesBucket description.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Fix the ffldb description.
* [NOD-828] Rename Database to DataAccessor and rename Handle to Database.
* [NOD-828] Make Get return nil if the value doesn't exist.
* [NOD-828] Attempt to close leveldb even if closing ffdb failed.
* [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket.
* [NOD-828] Fix not updating the store location in ffldb transactions.
* [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found.
* [NOD-828] Update the README and doc.go.
* [NOD-828] Simplify Bucket.Path().
* [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly.
* [NOD-828] Combine two lines into one.
* [NOD-828] Combine another two lines into one.
* [NOD-828] Move a misplaced comment.
* [NOD-828] Use Wrapf instead of Errorf where appropriate.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Fix Wrapf calls.
* [NOD-828] Fix comments.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Add a comment explaining the use of batches and snapshots.
* [NOD-828] Implement RollbackUnlessClosed().
* [NOD-828] Return both errors in StoreBlock rollback.
* [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor.
* [NOD-828] Make bucket a type alias instead of a struct.
* [NOD-828] Fix a typo.
* [NOD-828] Use copy instead of append in Bucket.
* [NOD-828] Extract flatFileLocationSerializedSize to a const.
* [NOD-828] Debugf -> Warnf in rollback.go.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Remove data length from flat file data format.
* [NOD-828] Rearrange TestLevelDBTransactionSanity a bit.
* [NOD-828] Add stack traces to all errors that come out of library functions.
* [NOD-828] Return errors from rollback().
* [NOD-828] Remove an irrelevant comment.
* [NOD-828] Remove redundant whitespace.
* [NOD-828] Handle nil in FetchBlock.
* [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct.
* Revert "[NOD-828] Make bucket a type alias instead of a struct."
This reverts commit 1fd39652
* [NOD-828] Fix revert errors.
* Revert "[NOD-828] Remove data length from flat file data format."
This reverts commit ef408e32
* [NOD-862] Move Cursor() into the DataAccessor interface.
* [NOD-828] Add Delete to DataAccessor.
* [NOD-828] Fix a couple of places that erroneously referenced blocks.
* [NOD-828] Add a comment on top of flatFileLocationSerializedSize.
* [NOD-828] Add Seek to Cursor.
* [NOD-828] Add First to Cursor.
* [NOD-828] Rename db to accessor in Context.
* [NOD-828] Make Get/Fetch calls return a boolean to indicate whether the requested item was found.
* [NOD-828] Name the output parameters of all Get functions.
* [NOD-828] Make RetrieveFromStore return whether the data was found.
* [NOD-887] Add a couple of QoL features to Cursor (#674)
* [NOD-887] Changed First to not return an error.
* [NOD-887] Fix merge error.
* [NOD-887] Make Cursor.Key not return the entire key path.
* [NOD-888] Add RollbackUnlessClosed to Context (#676)
* [NOD-888] Add RollbackUnlessClosed to Context.
* [NOD-888] Fix copy+paste error.
* [NOD-889] Instead of returning a boolean for not-found, return an error (#677)
* [NOD-889] Instead of returning a boolean for not-found, return an error.
* [NOD-889] Wrapped ErrNotFound for Get calls with nicer error messages.
* [NOD-889] Fix format.
* [NOD-889] Fix double space in a comment.
* [NOD-889] Add IsNotFoundError to dbaccess.
* [NOD-862] Replace calls to Tx.StoreBlock, Tx.HasBlock, Tx.FetchBlock with appropriate calls in dbaccess (#672)
* [NOD-828] Create the database2 package that will some day replace the database package.
* [NOD-828] Implement a "bucket" key mechanism.
* [NOD-828] Move bucket.go into the ffldb2 package.
* [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway.
* [NOD-828] Copy over + fixup flat file structs from the old ffldb.
* [NOD-828] Implement flatFilePath.
* [NOD-828] Implement flatFileStore.write().
* [NOD-828] Implement flatFileStore.read().
* [NOD-828] Implement flatFileStore.rollback().
* [NOD-828] Sync the file to disk at the end of write().
* [NOD-828] Extract crc32ByteOrder to a separate variable.
* [NOD-828] Add a sanity test.
* [NOD-828] Remove context-unrelated methods from the Database interface.
* [NOD-828] Create an ffldb object. Simply work against a context.
* [NOD-828] Open the new database on start.
* [NOD-828] Create the leveldb package.
* [NOD-828] Implement opening/closing leveldb.
* [NOD-828] Implement get/put out of/into leveldb.
* [NOD-828] Implement transactions and make them implement a generic database interface.
* [NOD-828] Write sanity tests for leveldb with and without transactions.
* [NOD-828] Add another case to the transaction sanity test.
* [NOD-828] Implement AppendBlock/RetrieveBlock.
* [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package.
* [NOD-828] Add RollbackFlatData to the database interface.
* [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket.
* [NOD-828] Implement StoreBlock.
* [NOD-828] Implement FetchBlock.
* [NOD-828] Implement HasBlock.
* [NOD-828] Write a sanity test for block insertion.
* [NOD-828] Implement CurrentFlatDataLocation.
* [NOD-828] Implement storing the current block location.
* [NOD-828] Implement initializing/syncing the flat file block store and the "metadata".
* [NOD-828] Add InitBlockStore to TestBlockStoreSanity.
* [NOD-828] Fix rename errors.
* [NOD-828] Fix lint errors in the root database package.
* [NOD-828] Fix lint errors in the ffldb.go.
* [NOD-828] Fix lint errors in the flatfile/db.go.
* [NOD-828] Rename packages in such a way to make the linter happy.
* [NOD-828] Finish satisfying the linter.
* [NOD-828] Fix doc.go.
* [NOD-828] Fix comments in block.go.
* [NOD-828] Move dbaccess out of the database package.
* [NOD-828] Move opening/closing the database to dbaccess.
* [NOD-828] Move the Database interface to the root database package, since it's meant to be generic.
* [NOD-828] Make ffldb generic to later support additional databases.
* [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported.
* [NOD-828] Fix comments.
* [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore.
* [NOD-828] Make buckets nicer to use.
* [NOD-828] Implement cursors that iterate over some bucket.
* [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb.
* [NOD-828] Write a test making sure that flat file repair works.
* [NOD-828] Properly close the database in TestRepairFlatFiles.
* [NOD-828] Add a comment warning against putting and getting the same data within the same transaction.
* [NOD-862] Use dbaccess.HasBlock instead of Tx.HasBlock in initDAGState.
* [NOD-862] Use dbaccess.StoreBlock instead of dbStoreBlock.
* [NOD-862] Use dbaccess.FetchBlock instead of various block fetching mechanisms.
* [NOD-828] Fix the flatFilesBucket description.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Fix the ffldb description.
* [NOD-828] Rename Database to DataAccessor and rename Handle to Database.
* [NOD-828] Make Get return nil if the value doesn't exist.
* [NOD-828] Attempt to close leveldb even if closing ffdb failed.
* [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket.
* [NOD-828] Fix not updating the store location in ffldb transactions.
* [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found.
* [NOD-828] Update the README and doc.go.
* [NOD-828] Simplify Bucket.Path().
* [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly.
* [NOD-828] Combine two lines into one.
* [NOD-828] Combine another two lines into one.
* [NOD-828] Move a misplaced comment.
* [NOD-828] Use Wrapf instead of Errorf where appropriate.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Fix Wrapf calls.
* [NOD-828] Fix comments.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Add a comment explaining the use of batches and snapshots.
* [NOD-828] Implement RollbackUnlessClosed().
* [NOD-828] Return both errors in StoreBlock rollback.
* [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor.
* [NOD-828] Make bucket a type alias instead of a struct.
* [NOD-828] Fix a typo.
* [NOD-828] Use copy instead of append in Bucket.
* [NOD-828] Extract flatFileLocationSerializedSize to a const.
* [NOD-828] Debugf -> Warnf in rollback.go.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Remove data length from flat file data format.
* [NOD-828] Rearrange TestLevelDBTransactionSanity a bit.
* [NOD-828] Add stack traces to all errors that come out of library functions.
* [NOD-828] Return errors from rollback().
* [NOD-828] Remove an irrelevant comment.
* [NOD-828] Remove redundant whitespace.
* [NOD-828] Handle nil in FetchBlock.
* [NOD-828] Implement a dbaccess.BlockNode struct.
* [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct.
* [NOD-828] Implement toDBBlockNode and fromDBBlockNode.
* Revert "[NOD-828] Make bucket a type alias instead of a struct."
This reverts commit 1fd39652
* [NOD-828] Fix revert errors.
* Revert "[NOD-828] Remove data length from flat file data format."
This reverts commit ef408e32
* [NOD-862] Implement storing index blocks.
* [NOD-862] Use database transactions where appropriate.
* [NOD-862] Fix tests failing on DAGSetup.
* [NOD-862] Fix bad make call.
* [NOD-862] Fix remaining database opening problems in tests.
* [NOD-862] Move Cursor() into the DataAccessor interface.
* [NOD-862] Move Cursor() into the DataAccessor interface.
* [NOD-862] Iterate over the new block index in dagio.
* [NOD-862] Fix block index key.
* [NOD-828] Add Delete to DataAccessor.
* [NOD-862] Pass byte slices to dbaccess instead of objects.
* [NOD-862] Fix errors.
* [NOD-862] Fix maybeAcceptBlock not checking block existence.
* [NOD-862] Fix TestAcceptanceIndexRecover.
* [NOD-862] Add comments to StoreBlockIndex and BlockIndexCursor.
* [NOD-828] Fix a couple of places that erroneously referenced blocks.
* [NOD-828] Add a comment on top of flatFileLocationSerializedSize.
* [NOD-828] Add Seek to Cursor.
* [NOD-828] Add First to Cursor.
* [NOD-828] Rename db to accessor in Context.
* [NOD-828] Make Get/Fetch calls return a boolean to indicate whether the requested item was found.
* [NOD-828] Name the output parameters of all Get functions.
* [NOD-828] Make RetrieveFromStore return whether the data was found.
* [NOD-862] Fix merge errors.
* [NOD-862] Fix DAGSetup using bad temp directories.
* [NOD-862] Fix TestProcessDelayedBlocks not closing the database properly.
* [NOD-862] Fix merge errors.
* [NOD-862] Merge flushToDBWithContext and flushToDB.
* [NOD-862] Remove TODO.
* [NOD-862] Add prefix to the temp dir in DAGSetup.
* [NOD-862] Bring back dbFetchBlockByHash.
* [NOD-862] Use BlockDAG.BlockByHash in p2p and rpc.
* [NOD-862] Use daghash.Hash in dbaccess.
* [NOD-862] Add defer to RollbackUnlessClosed after NewTx().
* [NOD-862] Extract dbStoreBlock to a separate function.
* [NOD-862] Fix grammar in comment.
* [NOD-862] Fix merge errors.
* [NOD-867] Migrate database logic in blockdag/dagio.go to dbaccess (#675)
* [NOD-828] Create the database2 package that will some day replace the database package.
* [NOD-828] Implement a "bucket" key mechanism.
* [NOD-828] Move bucket.go into the ffldb2 package.
* [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway.
* [NOD-828] Copy over + fixup flat file structs from the old ffldb.
* [NOD-828] Implement flatFilePath.
* [NOD-828] Implement flatFileStore.write().
* [NOD-828] Implement flatFileStore.read().
* [NOD-828] Implement flatFileStore.rollback().
* [NOD-828] Sync the file to disk at the end of write().
* [NOD-828] Extract crc32ByteOrder to a separate variable.
* [NOD-828] Add a sanity test.
* [NOD-828] Remove context-unrelated methods from the Database interface.
* [NOD-828] Create an ffldb object. Simply work against a context.
* [NOD-828] Open the new database on start.
* [NOD-828] Create the leveldb package.
* [NOD-828] Implement opening/closing leveldb.
* [NOD-828] Implement get/put out of/into leveldb.
* [NOD-828] Implement transactions and make them implement a generic database interface.
* [NOD-828] Write sanity tests for leveldb with and without transactions.
* [NOD-828] Add another case to the transaction sanity test.
* [NOD-828] Implement AppendBlock/RetrieveBlock.
* [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package.
* [NOD-828] Add RollbackFlatData to the database interface.
* [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket.
* [NOD-828] Implement StoreBlock.
* [NOD-828] Implement FetchBlock.
* [NOD-828] Implement HasBlock.
* [NOD-828] Write a sanity test for block insertion.
* [NOD-828] Implement CurrentFlatDataLocation.
* [NOD-828] Implement storing the current block location.
* [NOD-828] Implement initializing/syncing the flat file block store and the "metadata".
* [NOD-828] Add InitBlockStore to TestBlockStoreSanity.
* [NOD-828] Fix rename errors.
* [NOD-828] Fix lint errors in the root database package.
* [NOD-828] Fix lint errors in the ffldb.go.
* [NOD-828] Fix lint errors in the flatfile/db.go.
* [NOD-828] Rename packages in such a way to make the linter happy.
* [NOD-828] Finish satisfying the linter.
* [NOD-828] Fix doc.go.
* [NOD-828] Fix comments in block.go.
* [NOD-828] Move dbaccess out of the database package.
* [NOD-828] Move opening/closing the database to dbaccess.
* [NOD-828] Move the Database interface to the root database package, since it's meant to be generic.
* [NOD-828] Make ffldb generic to later support additional databases.
* [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported.
* [NOD-828] Fix comments.
* [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore.
* [NOD-828] Make buckets nicer to use.
* [NOD-828] Implement cursors that iterate over some bucket.
* [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb.
* [NOD-828] Write a test making sure that flat file repair works.
* [NOD-828] Properly close the database in TestRepairFlatFiles.
* [NOD-828] Add a comment warning against putting and getting the same data within the same transaction.
* [NOD-862] Use dbaccess.HasBlock instead of Tx.HasBlock in initDAGState.
* [NOD-862] Use dbaccess.StoreBlock instead of dbStoreBlock.
* [NOD-862] Use dbaccess.FetchBlock instead of various block fetching mechanisms.
* [NOD-828] Fix the flatFilesBucket description.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Fix the ffldb description.
* [NOD-828] Rename Database to DataAccessor and rename Handle to Database.
* [NOD-828] Make Get return nil if the value doesn't exist.
* [NOD-828] Attempt to close leveldb even if closing ffdb failed.
* [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket.
* [NOD-828] Fix not updating the store location in ffldb transactions.
* [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found.
* [NOD-828] Update the README and doc.go.
* [NOD-828] Simplify Bucket.Path().
* [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly.
* [NOD-828] Combine two lines into one.
* [NOD-828] Combine another two lines into one.
* [NOD-828] Move a misplaced comment.
* [NOD-828] Use Wrapf instead of Errorf where appropriate.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Fix Wrapf calls.
* [NOD-828] Fix comments.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Add a comment explaining the use of batches and snapshots.
* [NOD-828] Implement RollbackUnlessClosed().
* [NOD-828] Return both errors in StoreBlock rollback.
* [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor.
* [NOD-828] Make bucket a type alias instead of a struct.
* [NOD-828] Fix a typo.
* [NOD-828] Use copy instead of append in Bucket.
* [NOD-828] Extract flatFileLocationSerializedSize to a const.
* [NOD-828] Debugf -> Warnf in rollback.go.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Remove data length from flat file data format.
* [NOD-828] Rearrange TestLevelDBTransactionSanity a bit.
* [NOD-828] Add stack traces to all errors that come out of library functions.
* [NOD-828] Return errors from rollback().
* [NOD-828] Remove an irrelevant comment.
* [NOD-828] Remove redundant whitespace.
* [NOD-828] Handle nil in FetchBlock.
* [NOD-828] Implement a dbaccess.BlockNode struct.
* [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct.
* [NOD-828] Implement toDBBlockNode and fromDBBlockNode.
* Revert "[NOD-828] Make bucket a type alias instead of a struct."
This reverts commit 1fd39652
* [NOD-828] Fix revert errors.
* Revert "[NOD-828] Remove data length from flat file data format."
This reverts commit ef408e32
* [NOD-862] Implement storing index blocks.
* [NOD-862] Use database transactions where appropriate.
* [NOD-862] Fix tests failing on DAGSetup.
* [NOD-862] Fix bad make call.
* [NOD-862] Fix remaining database opening problems in tests.
* [NOD-862] Move Cursor() into the DataAccessor interface.
* [NOD-862] Move Cursor() into the DataAccessor interface.
* [NOD-862] Iterate over the new block index in dagio.
* [NOD-862] Fix block index key.
* [NOD-828] Add Delete to DataAccessor.
* [NOD-862] Pass byte slices to dbaccess instead of objects.
* [NOD-862] Fix errors.
* [NOD-862] Fix maybeAcceptBlock not checking block existence.
* [NOD-862] Fix TestAcceptanceIndexRecover.
* [NOD-862] Add comments to StoreBlockIndex and BlockIndexCursor.
* [NOD-828] Fix a couple of places that erroneously referenced blocks.
* [NOD-828] Add a comment on top of flatFileLocationSerializedSize.
* [NOD-828] Add Seek to Cursor.
* [NOD-828] Add First to Cursor.
* [NOD-828] Rename db to accessor in Context.
* [NOD-828] Make Get/Fetch calls return a boolean to indicate whether the requested item was found.
* [NOD-828] Name the output parameters of all Get functions.
* [NOD-828] Make RetrieveFromStore return whether the data was found.
* [NOD-862] Fix merge errors.
* [NOD-862] Fix DAGSetup using bad temp directories.
* [NOD-862] Fix TestProcessDelayedBlocks not closing the database properly.
* [NOD-867] Remove blockIndexBucket from dagio.
* [NOD-867] Fix wrong key in StoreIndexBucket.
* [NOD-867] Migrate DAG state to dbaccess.
* [NOD-867] Remove utxoSetVersionKeyName.
* [NOD-862] Fix merge errors.
* [NOD-867] Move localSubnetworkID into dagState.
* [NOD-867] Fix a comment.
* [NOD-867] Remove an unused function.
* [NOD-867] Migrate the database's UTXO set to dbaccess.
* [NOD-867] Add missing error check.
* [NOD-867] Changed First to not return an error.
* [NOD-867] Make Cursor.Key not return the entire key path.
* [NOD-887] Fix the comment above BlockIndexCursorFrom.
* [NOD-862] Merge flushToDBWithContext and flushToDB.
* [NOD-862] Remove TODO.
* [NOD-862] Add prefix to the temp dir in DAGSetup.
* [NOD-862] Bring back dbFetchBlockByHash.
* [NOD-862] Use BlockDAG.BlockByHash in p2p and rpc.
* [NOD-862] Use daghash.Hash in dbaccess.
* [NOD-862] Add defer to RollbackUnlessClosed after NewTx().
* [NOD-862] Extract dbStoreBlock to a separate function.
* [NOD-867] Remove TODOs.
* [NOD-867] Fix merge errors.
* [NOD-867] Fix comments and errors.
* [NOD-867] Unexport blockIndexKey.
* [NOD-867] Fix merge errors.
* [NOD-867] Move a misplaced comment.
* [NOD-867] Fix an error message.
* [NOD-867] Remove preallocation in initDAGState.
* [NOD-866] Migrate database logic in blockdag/indexers package to dbaccess (#682)
* [NOD-865] Delete blockidhash.go.
* [NOD-865] Remove a lot of no-longer relevant logic from indexers.
* [NOD-865] Pass TxContext to ConnectBlock.
* [NOD-865] Migrate the acceptance index to dbaccess.
* [NOD-865] Fix a block not being sent to ConnectBlock.
* [NOD-865] Pass the block's hash instead of the whole block.
* [NOD-865] Add forgotten Commit call.
* [NOD-865] Add comments.
* [NOD-866] Fix a comment.
* [NOD-866] Fix a comment.
* [NOD-866] Remove pointless indirection in acceptanceindex.
* [NOD-866] Fix comment over ForEachHash.
* [NOD-866] Rename ClearAcceptanceIndex to DropAcceptanceIndex.
* [NOD-866] Explain collecting keys before deleting them.
* [NOD-865] Move misc db logic to db access (#681)
* [NOD-828] Create the database2 package that will some day replace the database package.
* [NOD-828] Implement a "bucket" key mechanism.
* [NOD-828] Move bucket.go into the ffldb2 package.
* [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway.
* [NOD-828] Copy over + fixup flat file structs from the old ffldb.
* [NOD-828] Implement flatFilePath.
* [NOD-828] Implement flatFileStore.write().
* [NOD-828] Implement flatFileStore.read().
* [NOD-828] Implement flatFileStore.rollback().
* [NOD-828] Sync the file to disk at the end of write().
* [NOD-828] Extract crc32ByteOrder to a separate variable.
* [NOD-828] Add a sanity test.
* [NOD-828] Remove context-unrelated methods from the Database interface.
* [NOD-828] Create an ffldb object. Simply work against a context.
* [NOD-828] Open the new database on start.
* [NOD-828] Create the leveldb package.
* [NOD-828] Implement opening/closing leveldb.
* [NOD-828] Implement get/put out of/into leveldb.
* [NOD-828] Implement transactions and make them implement a generic database interface.
* [NOD-828] Write sanity tests for leveldb with and without transactions.
* [NOD-828] Add another case to the transaction sanity test.
* [NOD-828] Implement AppendBlock/RetrieveBlock.
* [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package.
* [NOD-828] Add RollbackFlatData to the database interface.
* [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket.
* [NOD-828] Implement StoreBlock.
* [NOD-828] Implement FetchBlock.
* [NOD-828] Implement HasBlock.
* [NOD-828] Write a sanity test for block insertion.
* [NOD-828] Implement CurrentFlatDataLocation.
* [NOD-828] Implement storing the current block location.
* [NOD-828] Implement initializing/syncing the flat file block store and the "metadata".
* [NOD-828] Add InitBlockStore to TestBlockStoreSanity.
* [NOD-828] Fix rename errors.
* [NOD-828] Fix lint errors in the root database package.
* [NOD-828] Fix lint errors in the ffldb.go.
* [NOD-828] Fix lint errors in the flatfile/db.go.
* [NOD-828] Rename packages in such a way to make the linter happy.
* [NOD-828] Finish satisfying the linter.
* [NOD-828] Fix doc.go.
* [NOD-828] Fix comments in block.go.
* [NOD-828] Move dbaccess out of the database package.
* [NOD-828] Move opening/closing the database to dbaccess.
* [NOD-828] Move the Database interface to the root database package, since it's meant to be generic.
* [NOD-828] Make ffldb generic to later support additional databases.
* [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported.
* [NOD-828] Fix comments.
* [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore.
* [NOD-828] Make buckets nicer to use.
* [NOD-828] Implement cursors that iterate over some bucket.
* [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb.
* [NOD-828] Write a test making sure that flat file repair works.
* [NOD-828] Properly close the database in TestRepairFlatFiles.
* [NOD-828] Add a comment warning against putting and getting the same data within the same transaction.
* [NOD-828] Fix the flatFilesBucket description.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Fix the ffldb description.
* [NOD-828] Rename Database to DataAccessor and rename Handle to Database.
* [NOD-828] Make Get return nil if the value doesn't exist.
* [NOD-828] Attempt to close leveldb even if closing ffdb failed.
* [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket.
* [NOD-828] Fix not updating the store location in ffldb transactions.
* [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found.
* [NOD-828] Update the README and doc.go.
* [NOD-828] Simplify Bucket.Path().
* [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly.
* [NOD-828] Combine two lines into one.
* [NOD-828] Combine another two lines into one.
* [NOD-828] Move a misplaced comment.
* [NOD-828] Use Wrapf instead of Errorf where appropriate.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Fix Wrapf calls.
* [NOD-828] Fix comments.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Add a comment explaining the use of batches and snapshots.
* [NOD-828] Implement RollbackUnlessClosed().
* [NOD-828] Return both errors in StoreBlock rollback.
* [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor.
* [NOD-828] Make bucket a type alias instead of a struct.
* [NOD-828] Fix a typo.
* [NOD-828] Use copy instead of append in Bucket.
* [NOD-828] Extract flatFileLocationSerializedSize to a const.
* [NOD-828] Debugf -> Warnf in rollback.go.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Remove data length from flat file data format.
* [NOD-828] Rearrange TestLevelDBTransactionSanity a bit.
* [NOD-828] Add stack traces to all errors that come out of library functions.
* [NOD-828] Return errors from rollback().
* [NOD-828] Remove an irrelevant comment.
* [NOD-828] Remove redundant whitespace.
* [NOD-828] Handle nil in FetchBlock.
* [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct.
* Revert "[NOD-828] Make bucket a type alias instead of a struct."
This reverts commit 1fd39652
* [NOD-828] Fix revert errors.
* Revert "[NOD-828] Remove data length from flat file data format."
This reverts commit ef408e32
* [NOD-862] Move Cursor() into the DataAccessor interface.
* [NOD-828] Add Delete to DataAccessor.
* [NOD-865] Move fee data db operations to dbaccess
* [NOD-865] Move reachability data db operations to dbaccess
* [NOD-865] Move UTXO diff data db operations to dbaccess
* [NOD-865] Move subnetwork data db operations to dbaccess
* [NOD-865] Fix createDAGState
* [NOD-865] Remove old Get signature with "exists"
* [NOD-865] Move multiset db operations to dbaccess
* [NOD-865] Use dbaccess transactions where possible
* [NOD-865] Remove old Get signature with "exists"
* [NOD-881] Recover TestGHOSTDAGErrors
* [NOD-865] Create function for db keys
* [NOD-865] Change Exists to Has, and use accessor.Has where possible
* [NOD-865] Make ClearReachabilityData transactive
* [NOD-865] Don't iterate cursors while changing db data
* [NOD-865] Rename RegisterSubnetwork -> StoreSubnetwork
* [NOD-865] Change bucket from utxodiffs to utxo-diffs
* [NOD-865] Rename SubnetworkExists->HasSubnetwork
* [NOD-865] Change a comment
* [NOD-865] Fix tests
* [NOD-865] Fix comment
* [NOD-865] Remove the prefix "db" from some functions
* [NOD-865] Remove redundant comments
* [NOD-865] Make clearBucket function
* [NOD-865] Make clear functions get a dbTx as an arg
* [NOD-865] Remove erroneous tx commit
Co-authored-by: stasatdaglabs <stas@daglabs.com>
* [NOD-868] Delete the old database package (#683)
* [NOD-828] Create the database2 package that will some day replace the database package.
* [NOD-828] Implement a "bucket" key mechanism.
* [NOD-828] Move bucket.go into the ffldb2 package.
* [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway.
* [NOD-828] Copy over + fixup flat file structs from the old ffldb.
* [NOD-828] Implement flatFilePath.
* [NOD-828] Implement flatFileStore.write().
* [NOD-828] Implement flatFileStore.read().
* [NOD-828] Implement flatFileStore.rollback().
* [NOD-828] Sync the file to disk at the end of write().
* [NOD-828] Extract crc32ByteOrder to a separate variable.
* [NOD-828] Add a sanity test.
* [NOD-828] Remove context-unrelated methods from the Database interface.
* [NOD-828] Create an ffldb object. Simply work against a context.
* [NOD-828] Open the new database on start.
* [NOD-828] Create the leveldb package.
* [NOD-828] Implement opening/closing leveldb.
* [NOD-828] Implement get/put out of/into leveldb.
* [NOD-828] Implement transactions and make them implement a generic database interface.
* [NOD-828] Write sanity tests for leveldb with and without transactions.
* [NOD-828] Add another case to the transaction sanity test.
* [NOD-828] Implement AppendBlock/RetrieveBlock.
* [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package.
* [NOD-828] Add RollbackFlatData to the database interface.
* [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket.
* [NOD-828] Implement StoreBlock.
* [NOD-828] Implement FetchBlock.
* [NOD-828] Implement HasBlock.
* [NOD-828] Write a sanity test for block insertion.
* [NOD-828] Implement CurrentFlatDataLocation.
* [NOD-828] Implement storing the current block location.
* [NOD-828] Implement initializing/syncing the flat file block store and the "metadata".
* [NOD-828] Add InitBlockStore to TestBlockStoreSanity.
* [NOD-828] Fix rename errors.
* [NOD-828] Fix lint errors in the root database package.
* [NOD-828] Fix lint errors in the ffldb.go.
* [NOD-828] Fix lint errors in the flatfile/db.go.
* [NOD-828] Rename packages in such a way to make the linter happy.
* [NOD-828] Finish satisfying the linter.
* [NOD-828] Fix doc.go.
* [NOD-828] Fix comments in block.go.
* [NOD-828] Move dbaccess out of the database package.
* [NOD-828] Move opening/closing the database to dbaccess.
* [NOD-828] Move the Database interface to the root database package, since it's meant to be generic.
* [NOD-828] Make ffldb generic to later support additional databases.
* [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported.
* [NOD-828] Fix comments.
* [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore.
* [NOD-828] Make buckets nicer to use.
* [NOD-828] Implement cursors that iterate over some bucket.
* [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb.
* [NOD-828] Write a test making sure that flat file repair works.
* [NOD-828] Properly close the database in TestRepairFlatFiles.
* [NOD-828] Add a comment warning against putting and getting the same data within the same transaction.
* [NOD-828] Fix the flatFilesBucket description.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Fix the ffldb description.
* [NOD-828] Rename Database to DataAccessor and rename Handle to Database.
* [NOD-828] Make Get return nil if the value doesn't exist.
* [NOD-828] Attempt to close leveldb even if closing ffdb failed.
* [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket.
* [NOD-828] Fix not updating the store location in ffldb transactions.
* [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found.
* [NOD-828] Update the README and doc.go.
* [NOD-828] Simplify Bucket.Path().
* [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly.
* [NOD-828] Combine two lines into one.
* [NOD-828] Combine another two lines into one.
* [NOD-828] Move a misplaced comment.
* [NOD-828] Use Wrapf instead of Errorf where appropriate.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Fix Wrapf calls.
* [NOD-828] Fix comments.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Add a comment explaining the use of batches and snapshots.
* [NOD-828] Implement RollbackUnlessClosed().
* [NOD-828] Return both errors in StoreBlock rollback.
* [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor.
* [NOD-828] Make bucket a type alias instead of a struct.
* [NOD-828] Fix a typo.
* [NOD-828] Use copy instead of append in Bucket.
* [NOD-828] Extract flatFileLocationSerializedSize to a const.
* [NOD-828] Debugf -> Warnf in rollback.go.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Remove data length from flat file data format.
* [NOD-828] Rearrange TestLevelDBTransactionSanity a bit.
* [NOD-828] Add stack traces to all errors that come out of library functions.
* [NOD-828] Return errors from rollback().
* [NOD-828] Remove an irrelevant comment.
* [NOD-828] Remove redundant whitespace.
* [NOD-828] Handle nil in FetchBlock.
* [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct.
* Revert "[NOD-828] Make bucket a type alias instead of a struct."
This reverts commit 1fd39652
* [NOD-828] Fix revert errors.
* Revert "[NOD-828] Remove data length from flat file data format."
This reverts commit ef408e32
* [NOD-862] Move Cursor() into the DataAccessor interface.
* [NOD-828] Add Delete to DataAccessor.
* [NOD-865] Move fee data db operations to dbaccess
* [NOD-865] Move reachability data db operations to dbaccess
* [NOD-865] Move UTXO diff data db operations to dbaccess
* [NOD-865] Move subnetwork data db operations to dbaccess
* [NOD-865] Fix createDAGState
* [NOD-865] Remove old Get signature with "exists"
* [NOD-865] Move multiset db operations to dbaccess
* [NOD-865] Use dbaccess transactions where possible
* [NOD-865] Remove old Get signature with "exists"
* [NOD-881] Recover TestGHOSTDAGErrors
* [NOD-865] Create function for db keys
* [NOD-865] Change Exists to Has, and use accessor.Has where possible
* [NOD-865] Make ClearReachabilityData transactive
* [NOD-865] Don't iterate cursors while changing db data
* [NOD-865] Rename RegisterSubnetwork -> StoreSubnetwork
* [NOD-865] Change bucket from utxodiffs to utxo-diffs
* [NOD-865] Rename SubnetworkExists->HasSubnetwork
* [NOD-865] Change a comment
* [NOD-868] Remove all tests from old database.
* [NOD-868] Remove all unused methods from the old database's interfaces.
* [NOD-865] Fix tests
* [NOD-868] Remove references to DB.
* [NOD-865] Fix comment
* [NOD-868] Remove the old ffldb besides the interface and errors.go.
* [NOD-868] Remove errors.go.
* [NOD-868] Remove the old database package.
* [NOD-868] Add openDB to DAGSetup to emulate the old dbpath in dag.config.
* [NOD-868] Rename database2 to database.
* [NOD-868] Use NewTx instead of NoTx where required.
* [NOD-868] Fix merge errors.
* [NOD-868] Rename dbXXX functions to just xxx.
* [NOD-868] Rename putDAGState to saveDAGState.
* [NOD-868] Replace comments in initDAGState with logs.
* [NOD-868] Explain the openDB parameter in DAGSetup.
* [NOD-868] Fixup doc.go and README.md.
* [NOD-868] Remove pointless transactions.
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* [NOD-805] Fix merge errors.
* [NOD-805] Fix a comment.
* [NOD-805] Don't return virtualTxsAcceptanceData from applyDAGChanges.
* [NOD-805] Add missing error handling in TestAcceptanceDataIndexRecover.
* [NOD-805] Rename blockDAG to dag in indexers/manager.go.
* [NOD-805] Defer cursor.Close() everywhere.
* [NOD-805] Rename scanFlatFiles to findCurrentLocation.
* [NOD-805] Extract crc32ChecksumLength and dataLengthLength to constants.
* [NOD-805] Handle open files properly in rollback.go.
* [NOD-805] Remove unnecessary func wrapper.
* [NOD-805] Remove unnecessary trimming in initialize.
* [NOD-805] Made StoreBlock accept only TxContext.
* [NOD-805] Changed the log level of an error message to Error.
* [NOD-805] Add a note about holding mutexes over deleteFile.
* [NOD-805] Remove a false comment.
* [NOD-805] Fix a comment.
* [NOD-805] Rename blk to block.
* [NOD-805] Extract utxoKey to a separate function.
* [NOD-805] Move dbaccess.xxxKey functions to the tops of their respective files.
* [NOD-805] Fix grammar in dbaccess/db.go.
* [NOD-805] Wrap a failed database corruption recovery error.
* [NOD-805] Split lines with WithStack in them.
* [NOD-805] Fix the comment over initialize.
* [NOD-805] Rename ffdb to flatFileDB and ldb to levelDB.
* [NOD-805] Fix a comment.
* [NOD-805] Fix a comment.
* [NOD-805] Use s.writeCursor instead of cursor.
* [NOD-805] Embed file in lockableFile.
* [NOD-805] the the -> the
* [NOD-805] openDB -> db
* [NOD-805] Use TxContext in all flushToDB functions.
* [NOD-805] Rename context -> dbContext.
* [NOD-805] Reword the comment at the beginning on initDAGState.
* [NOD-805] Explain cursor key trimming.
* [NOD-805] Remove Error from Cursor.
* [NOD-805] Return ErrNotFound from done Cursor Key and Value.
* [NOD-805] Add missing error handling.
* [NOD-805] Fix a comment.
* [NOD-805] Fix a variable name.
* [NOD-805] Remove pointless underscore.
* [NOD-805] Fix a comment.
* [NOD-805] Fix a variable name.
Co-authored-by: Mike Zak <feanorr@gmail.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* [NOD-881] Don't recalculate subtreesize for children
* [NOD-881] Make BenchmarkReindexInterval clearer
* [NOD-881] Use b.ResetTimer
* [NOD-881] Fix BenchmarkReindexInterval to use b.N
* [NOD-855] Save ECMH for each block UTXO
* [NOD-855] Remove UpdateExtraNonce method
* [NOD-855] Remove multiset data from UTXO diffs
* [NOD-855] Fix to fetch multiset of selected parent
* [NOD-855] Don't remove coinbase inputs from multiset
* [NOD-855] Create multisetBucketName on startup
* [NOD-855] Remove multiset from UTXO diff tests
* [NOD-855] clear new entries from multisetstore on saveChangesFromBlock
* [NOD-855] Fix tests
* [NOD-855] Use UnacceptedBlueScore when adding current block transactions to multiset
* [NOD-855] Hash utxo before adding it to multiset
* [NOD-855] Pass isCoinbase to NewUTXOEntry
* [NOD-855] Do not use hash when adding entries to multiset
* [NOD-855] When calculating multiset, replace the unaccepted blue score of selected parent transaction with the block blue score
* [NOD-855] Manually add a chained transaction to a block in TestChainedTransactions
* [NOD-855] Change name and comments
* [NOD-855] Use FindAcceptanceData to find a specific block acceptance data
* [NOD-855] Remove redundant copy of txIn.PreviousOutpoint
* [NOD-855] Use fmt.Sprintf when creating internalRPCError
* [NOD-848] Optimize allocations when serializing UTXO diffs
* [NOD-848] Use same UTXO serialization everywhere, and use compression as well
* [NOD-848] Fix usage of wrong buffer
* [NOD-848] Fix tests
* [NOD-848] Fix wire tests
* [NOD-848] Fix tests
* [NOD-848] Remove VLQ
* [NOD-848] Fix comments
* [NOD-848] Add varint for big endian encoding
* [NOD-848] In TestVarIntWire, assume the expected decoded value is the same as the serialization input
* [NOD-848] Serialize outpoint index with big endian varint
* [NOD-848] Remove p2pk from compression support
* [NOD-848] Fix comments
* [NOD-848] Remove p2pk from decompression support
* [NOD-848] Make entry compression optional
* [NOD-848] Fix tests
* [NOD-848] Fix comments and var names
* [NOD-848] Remove UTXO compression
* [NOD-848] Fix tests
* [NOD-848] Remove big endian varint
* [NOD-848] Fix comments
* [NOD-848] Rename ReadVarIntLittleEndian->ReadVarInt and fix WriteVarInt comment
* [NOD-848] Add outpointIndexByteOrder variable
* [NOD-848] Remove redundant comment
* [NOD-848] Fix outpointMaxSerializeSize to the correct value
* [NOD-848] Move subBuffer to utils
* [NOD-778] Add WithDiffInPlace
* [NOD-778] Fix bug in WithDiffInPlace
* [NOD-778] Add comment to WithDiffInPlace
* [NOD-778] Add double dag.restoreUTXO to benchmark, to remove time for hard-disk loading
* [NOD-778] Also test WithDiffInPlace in TestUTXODiffRules
* [NOD-778] Add tests for all cases possible in TestUTXODiffRules
* [NOD-778] Fix test-case 'first in toAdd in this, second in toRemove in this and toAdd in other'
* [NOD-778] Fixed in WithDiffInPlace
* [NOD-778] Update error messages when diffFrom(withDiffResult) fails in TestUTXODiffRules
* [NOD-778] diffFrom: disallow utxos both in d.toAdd, other.toAdd, and only one of d.toRemove and other.toRemove
* [NOD-778] Fix expected value in 'first in toRemove in this, second in toRemove in other'
* [NOD-778] diffFrom: Disallow situations where utxo both in d.toRemove and other.toRemove with different blue scores and no corresponding utxo in d.toAdd
* [NOD-778] WithDiff: Fix faulty logic that allows updates to blue scores
* [NOD-778] Fix WithDiffInPlace to pass all tests
* [NOD-778] Deleted temporary prints
* [NOD-778] Sorted TestUTXODiffRules tests according to spreadsheet
* [NOD-778] Delete deeputxo_test.go
* [NOD-778] Updated comments
* [NOD-778] Re-order
* [NOD-778] Re-order test-cases to be according to spreadsheet
* [NOD-778] Simplified case when both d.toRemove and other.toRemove have the same outpoint in diffFrom
* [NOD-778] Change a few error messages that say 'transaction' instead of 'outpoint'
* [NOD-778] Rename: utxoToAdd/Remove -> entryToAdd/Remove
* [NOD-788] Remove redundant else
* [NOD-778] Rename: existingUTXO -> existingEntry + remove redundant else
* [NOD-778] Correct test name
* [NOD-719] Added defers to unlocks (#618)
* [NOD-719] Added defers to unlocks
* [NOD-719] Added another defer to another Unlock
* [NOD-719] Added yet another defer to yet another Unlock
* [NOD-747] Change FinalityInterval to be 24 hours, isCurrent to be true if the DAG's time is less than 12 hours than the present, and change MaxInvPerMsg to be 1 << 17 (#625)
* [NOD-769] Add a log for when a reachability reindex occurs.
Co-authored-by: Svarog <feanorr@gmail.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* [NOD-719] Added defers to unlocks (#618)
* [NOD-719] Added defers to unlocks
* [NOD-719] Added another defer to another Unlock
* [NOD-719] Added yet another defer to yet another Unlock
* [NOD-747] Change FinalityInterval to be 24 hours, isCurrent to be true if the DAG's time is less than 12 hours than the present, and change MaxInvPerMsg to be 1 << 17 (#625)
