mirror of
https://github.com/kaspanet/kaspad.git
synced 2025-08-01 08:13:19 +00:00
85 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Svarog
|
6b5b4bfb2a
|
[NOD-1164] Remove the singleton from dbaccess, to enable multiple db connections in same run (#806)
* [NOD-1164] Defined DatabaseContext as the basic object of dbaccess * [NOD-1164] Update everything to use databaseContext * [NOD-1164] Fix tests * [NOD-1164] Add comments * [NOD-1164] Removed databaseContext from blockNode * [NOD-1164] Enforce DatabaseContext != nil * [NOD-1164] Remove redundant and wrong comment line |
||
|
Ori Newman
|
433cdb6006
|
[NOD-1126] implement block relay flow (#786)
* [NOD-1126] Implement block relay flow * [NOD-1126] Add StartGetRelayBlocksListener * [NOD-1126] Implement block relay flow * [NOD-1126] Integrate with new interface * [NOD-1126] Fix comments * [NOD-1126] Refactor protocol.go * [NOD-1126] Split long lines * [NOD-1126] Fix comment * [NOD-1126] move sharedRequestedBlocks to a separate file * [NOD-1126] Fix error message * [NOD-1126] Move handleInv to StartBlockRelay * [NOD-1126] Create hashesQueueSet type * [NOD-1126] Make deleteFromRequestedBlocks a method * [NOD-1126] Fix comment * [NOD-1126] Add block logger * [NOD-1126] Rename advertisedProtoVer->advertisedProtocolVer * [NOD-1126] Fix comment and an error message * [NOD-1126] Remove redundant loop * [NOD-1126] Move requestBlocks upper * [NOD-1126] Remove exiting blocks in requestedBlocks from hashesToRequest * [NOD-1126] Change comment * [NOD-1126] Rename stallResponseTimeout->timeout * [NOD-1126] Use switch inside readMsgBlock * [NOD-1126] Fix error message and remove redundant log * [NOD-1126] Rename pacakge names * [NOD-1126] Fix comment * [NOD-1126] Change file names * [NOD-1126] Convert block to partial if needed * [NOD-1126] Remove function redeclaration * [NOD-1126] continue instead of return * [NOD-1126] Rename LogBlockBlueScore->LogBlock * [NOD-1126] Add minimum functions to utils * [NOD-1126] Flip condition on readInv * [NOD-1126] Rename utilMath->mathUtil * [NOD-1126] Fix comment |
||
|
Svarog
|
1a43cabfb9
|
[NOD-1119] Refactor main, and remove p2p layer from it (#785)
* [NOD-1119] Removed all p2p server from all the initialization of server * [NOD-1119] Removed any calling for p2p server in main * [NOD-1119] Simplified some functions to not take both dag and dagParams * [NOD-1119] Simplify creation of mempool and rpc server * [NOD-1119] Setup indexes in separate function * [NOD-1119] Some cleanup in NewServer * [NOD-1119] Fix mempool test * [NOD-1119] Fix go format * [NOD-1119] Unexport dag.timeSource * [NOD-1119] Removed server package + renamed the Server object to Kaspad, and made it minimal * [NOD-1119] Delete redundant functions * Unexported kaspad and related methods * [NOD-1119] Unexported newKaspad * [NOD-1119] Revise comments and remove redundant function * [NOD-1119] Make comments of unexported methods lower-case * [NOD-1119] Some more refactoring in newKaspad |
||
|
Ori Newman
|
749775c7ea
|
[NOD-1098] Change timestamp precision to one millisecond (#778)
* [NOD-1098] Change timestamps to be millisecond precision * [NOD-1098] Change lock times to use milliseconds * [NOD-1098] Use milliseconds precision everywhere * [NOD-1098] Implement type mstime.Time * [NOD-1098] Fix block 100000 timestamp * [NOD-1098] Change orphan child to be one millisecond delay after its parent * [NOD-1098] Remove test that checks if header timestamps have the right precision, and instead add tests for mstime, and fix genesis for testnet and devnet * [NOD-1098] Fix comment * [NOD-1098] Fix comment * [NOD-1098] Fix testnet genesis * [NOD-1098] Rename UnixMilli->UnixMilliseconds |
||
|
stasatdaglabs
|
57b1653383
|
[NOD-1063] Optimize deep reachability tree insertions (#773)
* [NOD-1055] Give higher priority for requesting missing ancestors when sending a getdata message (#767) * [NOD-1063] Remove the remainingInterval field. * [NOD-1063] Add helper functions to reachabilityTreeNode. * [NOD-1063] Add reachabilityReindexRoot. * [NOD-1063] Start implementing findNextReachabilityReindexRoot. * [NOD-1063] Implement findCommonAncestor. * [NOD-1063] Implement findReachabilityTreeAncestorInChildren. * [NOD-1063] Add reachabilityReindexWindow. * [NOD-1063] Fix findReachabilityTreeAncestorInChildren. * [NOD-1063] Remove BlockDAG reference in findReachabilityTreeAncestorInChildren. * [NOD-1063] Extract updateReachabilityReindexRoot to a separate function. * [NOD-1063] Add reachabilityReindexSlack. * [NOD-1063] Implement splitReindexRootChildrenAroundChosen. * [NOD-1063] Implement calcReachabilityTreeNodeSizes. * [NOD-1063] Implement propagateChildIntervals. * [NOD-1063] Extract tightenReachabilityTreeIntervalsBeforeChosenReindexRootChild and tightenReachabilityTreeIntervalsAfterChosenReindexRootChild to separate functions. * [NOD-1063] Implement expandReachabilityTreeIntervalInChosenReindexRootChild. * [NOD-1063] Finished implementing concentrateReachabilityTreeIntervalAroundReindexRootChild. * [NOD-1063] Begin implementing reindexIntervalsBeforeReindexRoot. * [NOD-1063] Implement top-level logic of reindexIntervalsBeforeReindexRoot. * [NOD-1063] Implement reclaimIntervalBeforeChosenChild. * [NOD-1063] Add a debug log for reindexIntervalsBeforeReindexRoot. * [NOD-1063] Rename reindexIntervalsBeforeReindexRoot to reindexIntervalsEarlierThanReindexRoot. * [NOD-1063] Implement reclaimIntervalAfterChosenChild. * [NOD-1063] Add a debug log for updateReachabilityReindexRoot. * [NOD-1063] Convert modifiedTreeNodes from slices to sets. * [NOD-1063] Fix findCommonAncestor. * [NOD-1063] Fix reindexIntervalsEarlierThanReindexRoot.` * [NOD-1063] Remove redundant nil conditions. * [NOD-1063] Make map[*reachabilityTreeNode]struct{} into a type alias with a copyAllFrom method. * [NOD-1063] Remove setInterval. * [NOD-1063] Create a new struct to hold reachability stuff called reachabilityTree. * [NOD-1063] Rename functions under reachabilityTree. * [NOD-1063] Move reachabilityStore into reachabilityTree. * [NOD-1063] Move the rest of the functions in reachability.go into the reachabilityTree struct. * [NOD-1063] Update newReachabilityTree to take an instance of reachabilityStore. * [NOD-1063] Fix merge errors. * [NOD-1063] Fix merge errors. * [NOD-1063] Pass a reference to the dag into reachabilityTree. * [NOD-1063] Use Wrapf instead of Errorf. * [NOD-1063] Merge assignments. * [NOD-1063] Disambiguate a varaible name. * [NOD-1063] Add a test case for intervalBefore. * [NOD-1063] Simplify splitChildrenAroundChosenChild. * [NOD-1063] Fold temporary variables into newReachabilityInterval. * [NOD-1063] Fold more temporary variables into newReachabilityInterval. * [NOD-1063] Fix a bug in expandIntervalInReindexRootChosenChild. * [NOD-1063] Remove blockNode from futureCoveringBlock. * [NOD-1063] Get rid of futureCoveringBlock. * [NOD-1063] Use findIndex directly in findAncestorAmongChildren. * [NOD-1063] Make findIndex a bit nicer to use. Also rename it to findAncestorIndexOfNode. * [NOD-1063] Rename childIntervalAllocationRange to intervalRangeForChildAllocation. * [NOD-1063] Optimize findCommonAncestor. * [NOD-1063] In reindexIntervalsBeforeChosenChild, use chosenChild.interval.start - 1 instead of childrenBeforeChosen[len(childrenBeforeChosen)-1].interval.end + 1. * [NOD-1063] Rename reindexIntervalsBeforeChosenChild to reindexIntervalsBeforeNode. * [NOD-1063] Add a comment explain what "the chosen child" is. * [NOD-1063] In concentrateIntervalAroundReindexRootChosenChild, rename modifiedTreeNodes to allModifiedTreeNodes. * [NOD-1063] Extract propagateIntervals to a function. * [NOD-1063] Extract interval "contains" logic to a separate function. * [NOD-1063] Simplify "looping up" logic in reclaimIntervalXXXChosenChild. * [NOD-1063] Add comments to reclaimIntervalXXXChosenChild. * [NOD-1063] Rename copyAllFrom to addAll. * [NOD-1063] Rename reachabilityStore (the variable) to just store. * [NOD-1063] Fix an error message. * [NOD-1063] Reword a comment. * [NOD-1063] Don't return -1 from findAncestorIndexOfNode. * [NOD-1063] Extract slackReachabilityIntervalForReclaiming to a constant. * [NOD-1063] Add a missing condition. * [NOD-1063] Call isAncestorOf directly in insertNode. * [NOD-1063] Rename chosenReindexRootChild to reindexRootChosenChild. * [NOD-1063] Rename treeNodeSet to orderedTreeNodeSet. * [NOD-1063] Add a disclaimer to orderedTreeNodeSet. * [NOD-1063] Implement StoreReachabilityReindexRoot and FetchReachabilityReindexRoot. * [NOD-1063] Move storing the reindex root to within reachabilityTree. * [NOD-1063] Remove isAncestorOf from reachabilityInterval. * [NOD-1063] Add a comment about graph theory conventions. * [NOD-1063] Fix tests. * [NOD-1063] Change inclusion in isAncestorOf functions. * [NOD-1063] Rename a test. * [NOD-1063] Implement TestIsInFuture. * [NOD-1063] Fix error messages in TestIsInFuture. * [NOD-1063] Fix error messages in TestIsInFuture. * [NOD-1063] Rename isInSelectedParentChain to isInSelectedParentChainOf. * [NOD-1063] Rename isInFuture to isInPast. * [NOD-1063] Expand on a comment. * [NOD-1063] Rename modifiedTreeNodes. * [NOD-1063] Implement test: TestReindexIntervalsEarlierThanReindexRoot. * [NOD-1063] Implement test: TestUpdateReindexRoot. * [NOD-1063] Explain a check. * [NOD-1063] Use a method instead of calling reachabilityStore.loaded directly. * [NOD-1063] Lowercasified an error message. * [NOD-1063] Fix failing test. Co-authored-by: Ori Newman <orinewman1@gmail.com> |
||
|
Ori Newman
|
3a4571d671
|
[NOD-965] Make LookupNode return boolean (#729)
* [NOD-965] Make dag.index.LookupNode return false if node is not found * [NOD-965] Rename blockDAG->dag * [NOD-965] Remove irrelevant test * [NOD-965] Use bi.index's ok in LookupNode |
||
|
Svarog
|
378f0b659a
|
[NOD-993] Get rid of redundant error types + Use %+v when printing startup errors (#719)
* [NOD-993] Use %+v when printing errors * [NOD-993] Get rid of AssertError * [NOD-993] Made ruleError use github.com/pkg/errors * [NOD-993] remove redundant TODO * [NOD-993] remove redundant Comment * [NOD-993] Removed DeploymentError |
||
|
Ori Newman
|
7609c50641
|
[NOD-885] Use database.Key and database.Bucket instead of byte slices (#692)
* [NOD-885] Create database.Key type * [NOD-885] Rename FullKey()->FullKeyBytes() and Key()->KeyBytes() * [NOD-885] Make Key.String return a hex string * [NOD-885] Rename key parts * [NOD-885] Rename separator->bucketSeparator * [NOD-885] Rename SuffixBytes->Suffix and PrefixBytes->Prefix * [NOD-885] Change comments * [NOD-885] Change key prefix to bucket * [NOD-885] Don't use database.NewKey inside dbaccess * [NOD-885] Fix nil bug in Bucket.Path() * [NOD-885] Rename helpers.go -> keys.go * [NOD-885] Unexport database.NewKey * [NOD-885] Remove redundant code in Bucket.Path() |
||
|
stasatdaglabs
|
a31139d4a5
|
[NOD-895] Break down initDAGState to sub-routines (#690) | ||
|
Ori Newman
|
bfbc72724d
|
[NOD-873] Reuse allocated space when updating the UTXO set in database (#688) | ||
|
stasatdaglabs
|
f8b18e09d6
|
[NOD-805] Redesign the database (#685)
* [NOD-828] Reimplement FFLDB (#663) * [NOD-828] Create the database2 package that will some day replace the database package. * [NOD-828] Implement a "bucket" key mechanism. * [NOD-828] Move bucket.go into the ffldb2 package. * [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway. * [NOD-828] Copy over + fixup flat file structs from the old ffldb. * [NOD-828] Implement flatFilePath. * [NOD-828] Implement flatFileStore.write(). * [NOD-828] Implement flatFileStore.read(). * [NOD-828] Implement flatFileStore.rollback(). * [NOD-828] Sync the file to disk at the end of write(). * [NOD-828] Extract crc32ByteOrder to a separate variable. * [NOD-828] Add a sanity test. * [NOD-828] Remove context-unrelated methods from the Database interface. * [NOD-828] Create an ffldb object. Simply work against a context. * [NOD-828] Open the new database on start. * [NOD-828] Create the leveldb package. * [NOD-828] Implement opening/closing leveldb. * [NOD-828] Implement get/put out of/into leveldb. * [NOD-828] Implement transactions and make them implement a generic database interface. * [NOD-828] Write sanity tests for leveldb with and without transactions. * [NOD-828] Add another case to the transaction sanity test. * [NOD-828] Implement AppendBlock/RetrieveBlock. * [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package. * [NOD-828] Add RollbackFlatData to the database interface. * [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket. * [NOD-828] Implement StoreBlock. * [NOD-828] Implement FetchBlock. * [NOD-828] Implement HasBlock. * [NOD-828] Write a sanity test for block insertion. * [NOD-828] Implement CurrentFlatDataLocation. * [NOD-828] Implement storing the current block location. * [NOD-828] Implement initializing/syncing the flat file block store and the "metadata". * [NOD-828] Add InitBlockStore to TestBlockStoreSanity. * [NOD-828] Fix rename errors. * [NOD-828] Fix lint errors in the root database package. * [NOD-828] Fix lint errors in the ffldb.go. * [NOD-828] Fix lint errors in the flatfile/db.go. * [NOD-828] Rename packages in such a way to make the linter happy. * [NOD-828] Finish satisfying the linter. * [NOD-828] Fix doc.go. * [NOD-828] Fix comments in block.go. * [NOD-828] Move dbaccess out of the database package. * [NOD-828] Move opening/closing the database to dbaccess. * [NOD-828] Move the Database interface to the root database package, since it's meant to be generic. * [NOD-828] Make ffldb generic to later support additional databases. * [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported. * [NOD-828] Fix comments. * [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore. * [NOD-828] Make buckets nicer to use. * [NOD-828] Implement cursors that iterate over some bucket. * [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb. * [NOD-828] Write a test making sure that flat file repair works. * [NOD-828] Properly close the database in TestRepairFlatFiles. * [NOD-828] Add a comment warning against putting and getting the same data within the same transaction. * [NOD-828] Fix the flatFilesBucket description. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Fix the ffldb description. * [NOD-828] Rename Database to DataAccessor and rename Handle to Database. * [NOD-828] Make Get return nil if the value doesn't exist. * [NOD-828] Attempt to close leveldb even if closing ffdb failed. * [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket. * [NOD-828] Fix not updating the store location in ffldb transactions. * [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found. * [NOD-828] Update the README and doc.go. * [NOD-828] Simplify Bucket.Path(). * [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly. * [NOD-828] Combine two lines into one. * [NOD-828] Combine another two lines into one. * [NOD-828] Move a misplaced comment. * [NOD-828] Use Wrapf instead of Errorf where appropriate. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Fix Wrapf calls. * [NOD-828] Fix comments. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Add a comment explaining the use of batches and snapshots. * [NOD-828] Implement RollbackUnlessClosed(). * [NOD-828] Return both errors in StoreBlock rollback. * [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor. * [NOD-828] Make bucket a type alias instead of a struct. * [NOD-828] Fix a typo. * [NOD-828] Use copy instead of append in Bucket. * [NOD-828] Extract flatFileLocationSerializedSize to a const. * [NOD-828] Debugf -> Warnf in rollback.go. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Remove data length from flat file data format. * [NOD-828] Rearrange TestLevelDBTransactionSanity a bit. * [NOD-828] Add stack traces to all errors that come out of library functions. * [NOD-828] Return errors from rollback(). * [NOD-828] Remove an irrelevant comment. * [NOD-828] Remove redundant whitespace. * [NOD-828] Handle nil in FetchBlock. * [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct. * Revert "[NOD-828] Make bucket a type alias instead of a struct." This reverts commit 1fd39652 * [NOD-828] Fix revert errors. * Revert "[NOD-828] Remove data length from flat file data format." This reverts commit ef408e32 * [NOD-862] Move Cursor() into the DataAccessor interface. * [NOD-828] Add Delete to DataAccessor. * [NOD-828] Fix a couple of places that erroneously referenced blocks. * [NOD-828] Add a comment on top of flatFileLocationSerializedSize. * [NOD-828] Add Seek to Cursor. * [NOD-828] Add First to Cursor. * [NOD-828] Rename db to accessor in Context. * [NOD-828] Make Get/Fetch calls return a boolean to indicate whether the requested item was found. * [NOD-828] Name the output parameters of all Get functions. * [NOD-828] Make RetrieveFromStore return whether the data was found. * [NOD-887] Add a couple of QoL features to Cursor (#674) * [NOD-887] Changed First to not return an error. * [NOD-887] Fix merge error. * [NOD-887] Make Cursor.Key not return the entire key path. * [NOD-888] Add RollbackUnlessClosed to Context (#676) * [NOD-888] Add RollbackUnlessClosed to Context. * [NOD-888] Fix copy+paste error. * [NOD-889] Instead of returning a boolean for not-found, return an error (#677) * [NOD-889] Instead of returning a boolean for not-found, return an error. * [NOD-889] Wrapped ErrNotFound for Get calls with nicer error messages. * [NOD-889] Fix format. * [NOD-889] Fix double space in a comment. * [NOD-889] Add IsNotFoundError to dbaccess. * [NOD-862] Replace calls to Tx.StoreBlock, Tx.HasBlock, Tx.FetchBlock with appropriate calls in dbaccess (#672) * [NOD-828] Create the database2 package that will some day replace the database package. * [NOD-828] Implement a "bucket" key mechanism. * [NOD-828] Move bucket.go into the ffldb2 package. * [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway. * [NOD-828] Copy over + fixup flat file structs from the old ffldb. * [NOD-828] Implement flatFilePath. * [NOD-828] Implement flatFileStore.write(). * [NOD-828] Implement flatFileStore.read(). * [NOD-828] Implement flatFileStore.rollback(). * [NOD-828] Sync the file to disk at the end of write(). * [NOD-828] Extract crc32ByteOrder to a separate variable. * [NOD-828] Add a sanity test. * [NOD-828] Remove context-unrelated methods from the Database interface. * [NOD-828] Create an ffldb object. Simply work against a context. * [NOD-828] Open the new database on start. * [NOD-828] Create the leveldb package. * [NOD-828] Implement opening/closing leveldb. * [NOD-828] Implement get/put out of/into leveldb. * [NOD-828] Implement transactions and make them implement a generic database interface. * [NOD-828] Write sanity tests for leveldb with and without transactions. * [NOD-828] Add another case to the transaction sanity test. * [NOD-828] Implement AppendBlock/RetrieveBlock. * [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package. * [NOD-828] Add RollbackFlatData to the database interface. * [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket. * [NOD-828] Implement StoreBlock. * [NOD-828] Implement FetchBlock. * [NOD-828] Implement HasBlock. * [NOD-828] Write a sanity test for block insertion. * [NOD-828] Implement CurrentFlatDataLocation. * [NOD-828] Implement storing the current block location. * [NOD-828] Implement initializing/syncing the flat file block store and the "metadata". * [NOD-828] Add InitBlockStore to TestBlockStoreSanity. * [NOD-828] Fix rename errors. * [NOD-828] Fix lint errors in the root database package. * [NOD-828] Fix lint errors in the ffldb.go. * [NOD-828] Fix lint errors in the flatfile/db.go. * [NOD-828] Rename packages in such a way to make the linter happy. * [NOD-828] Finish satisfying the linter. * [NOD-828] Fix doc.go. * [NOD-828] Fix comments in block.go. * [NOD-828] Move dbaccess out of the database package. * [NOD-828] Move opening/closing the database to dbaccess. * [NOD-828] Move the Database interface to the root database package, since it's meant to be generic. * [NOD-828] Make ffldb generic to later support additional databases. * [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported. * [NOD-828] Fix comments. * [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore. * [NOD-828] Make buckets nicer to use. * [NOD-828] Implement cursors that iterate over some bucket. * [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb. * [NOD-828] Write a test making sure that flat file repair works. * [NOD-828] Properly close the database in TestRepairFlatFiles. * [NOD-828] Add a comment warning against putting and getting the same data within the same transaction. * [NOD-862] Use dbaccess.HasBlock instead of Tx.HasBlock in initDAGState. * [NOD-862] Use dbaccess.StoreBlock instead of dbStoreBlock. * [NOD-862] Use dbaccess.FetchBlock instead of various block fetching mechanisms. * [NOD-828] Fix the flatFilesBucket description. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Fix the ffldb description. * [NOD-828] Rename Database to DataAccessor and rename Handle to Database. * [NOD-828] Make Get return nil if the value doesn't exist. * [NOD-828] Attempt to close leveldb even if closing ffdb failed. * [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket. * [NOD-828] Fix not updating the store location in ffldb transactions. * [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found. * [NOD-828] Update the README and doc.go. * [NOD-828] Simplify Bucket.Path(). * [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly. * [NOD-828] Combine two lines into one. * [NOD-828] Combine another two lines into one. * [NOD-828] Move a misplaced comment. * [NOD-828] Use Wrapf instead of Errorf where appropriate. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Fix Wrapf calls. * [NOD-828] Fix comments. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Add a comment explaining the use of batches and snapshots. * [NOD-828] Implement RollbackUnlessClosed(). * [NOD-828] Return both errors in StoreBlock rollback. * [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor. * [NOD-828] Make bucket a type alias instead of a struct. * [NOD-828] Fix a typo. * [NOD-828] Use copy instead of append in Bucket. * [NOD-828] Extract flatFileLocationSerializedSize to a const. * [NOD-828] Debugf -> Warnf in rollback.go. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Remove data length from flat file data format. * [NOD-828] Rearrange TestLevelDBTransactionSanity a bit. * [NOD-828] Add stack traces to all errors that come out of library functions. * [NOD-828] Return errors from rollback(). * [NOD-828] Remove an irrelevant comment. * [NOD-828] Remove redundant whitespace. * [NOD-828] Handle nil in FetchBlock. * [NOD-828] Implement a dbaccess.BlockNode struct. * [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct. * [NOD-828] Implement toDBBlockNode and fromDBBlockNode. * Revert "[NOD-828] Make bucket a type alias instead of a struct." This reverts commit 1fd39652 * [NOD-828] Fix revert errors. * Revert "[NOD-828] Remove data length from flat file data format." This reverts commit ef408e32 * [NOD-862] Implement storing index blocks. * [NOD-862] Use database transactions where appropriate. * [NOD-862] Fix tests failing on DAGSetup. * [NOD-862] Fix bad make call. * [NOD-862] Fix remaining database opening problems in tests. * [NOD-862] Move Cursor() into the DataAccessor interface. * [NOD-862] Move Cursor() into the DataAccessor interface. * [NOD-862] Iterate over the new block index in dagio. * [NOD-862] Fix block index key. * [NOD-828] Add Delete to DataAccessor. * [NOD-862] Pass byte slices to dbaccess instead of objects. * [NOD-862] Fix errors. * [NOD-862] Fix maybeAcceptBlock not checking block existence. * [NOD-862] Fix TestAcceptanceIndexRecover. * [NOD-862] Add comments to StoreBlockIndex and BlockIndexCursor. * [NOD-828] Fix a couple of places that erroneously referenced blocks. * [NOD-828] Add a comment on top of flatFileLocationSerializedSize. * [NOD-828] Add Seek to Cursor. * [NOD-828] Add First to Cursor. * [NOD-828] Rename db to accessor in Context. * [NOD-828] Make Get/Fetch calls return a boolean to indicate whether the requested item was found. * [NOD-828] Name the output parameters of all Get functions. * [NOD-828] Make RetrieveFromStore return whether the data was found. * [NOD-862] Fix merge errors. * [NOD-862] Fix DAGSetup using bad temp directories. * [NOD-862] Fix TestProcessDelayedBlocks not closing the database properly. * [NOD-862] Fix merge errors. * [NOD-862] Merge flushToDBWithContext and flushToDB. * [NOD-862] Remove TODO. * [NOD-862] Add prefix to the temp dir in DAGSetup. * [NOD-862] Bring back dbFetchBlockByHash. * [NOD-862] Use BlockDAG.BlockByHash in p2p and rpc. * [NOD-862] Use daghash.Hash in dbaccess. * [NOD-862] Add defer to RollbackUnlessClosed after NewTx(). * [NOD-862] Extract dbStoreBlock to a separate function. * [NOD-862] Fix grammar in comment. * [NOD-862] Fix merge errors. * [NOD-867] Migrate database logic in blockdag/dagio.go to dbaccess (#675) * [NOD-828] Create the database2 package that will some day replace the database package. * [NOD-828] Implement a "bucket" key mechanism. * [NOD-828] Move bucket.go into the ffldb2 package. * [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway. * [NOD-828] Copy over + fixup flat file structs from the old ffldb. * [NOD-828] Implement flatFilePath. * [NOD-828] Implement flatFileStore.write(). * [NOD-828] Implement flatFileStore.read(). * [NOD-828] Implement flatFileStore.rollback(). * [NOD-828] Sync the file to disk at the end of write(). * [NOD-828] Extract crc32ByteOrder to a separate variable. * [NOD-828] Add a sanity test. * [NOD-828] Remove context-unrelated methods from the Database interface. * [NOD-828] Create an ffldb object. Simply work against a context. * [NOD-828] Open the new database on start. * [NOD-828] Create the leveldb package. * [NOD-828] Implement opening/closing leveldb. * [NOD-828] Implement get/put out of/into leveldb. * [NOD-828] Implement transactions and make them implement a generic database interface. * [NOD-828] Write sanity tests for leveldb with and without transactions. * [NOD-828] Add another case to the transaction sanity test. * [NOD-828] Implement AppendBlock/RetrieveBlock. * [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package. * [NOD-828] Add RollbackFlatData to the database interface. * [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket. * [NOD-828] Implement StoreBlock. * [NOD-828] Implement FetchBlock. * [NOD-828] Implement HasBlock. * [NOD-828] Write a sanity test for block insertion. * [NOD-828] Implement CurrentFlatDataLocation. * [NOD-828] Implement storing the current block location. * [NOD-828] Implement initializing/syncing the flat file block store and the "metadata". * [NOD-828] Add InitBlockStore to TestBlockStoreSanity. * [NOD-828] Fix rename errors. * [NOD-828] Fix lint errors in the root database package. * [NOD-828] Fix lint errors in the ffldb.go. * [NOD-828] Fix lint errors in the flatfile/db.go. * [NOD-828] Rename packages in such a way to make the linter happy. * [NOD-828] Finish satisfying the linter. * [NOD-828] Fix doc.go. * [NOD-828] Fix comments in block.go. * [NOD-828] Move dbaccess out of the database package. * [NOD-828] Move opening/closing the database to dbaccess. * [NOD-828] Move the Database interface to the root database package, since it's meant to be generic. * [NOD-828] Make ffldb generic to later support additional databases. * [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported. * [NOD-828] Fix comments. * [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore. * [NOD-828] Make buckets nicer to use. * [NOD-828] Implement cursors that iterate over some bucket. * [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb. * [NOD-828] Write a test making sure that flat file repair works. * [NOD-828] Properly close the database in TestRepairFlatFiles. * [NOD-828] Add a comment warning against putting and getting the same data within the same transaction. * [NOD-862] Use dbaccess.HasBlock instead of Tx.HasBlock in initDAGState. * [NOD-862] Use dbaccess.StoreBlock instead of dbStoreBlock. * [NOD-862] Use dbaccess.FetchBlock instead of various block fetching mechanisms. * [NOD-828] Fix the flatFilesBucket description. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Fix the ffldb description. * [NOD-828] Rename Database to DataAccessor and rename Handle to Database. * [NOD-828] Make Get return nil if the value doesn't exist. * [NOD-828] Attempt to close leveldb even if closing ffdb failed. * [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket. * [NOD-828] Fix not updating the store location in ffldb transactions. * [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found. * [NOD-828] Update the README and doc.go. * [NOD-828] Simplify Bucket.Path(). * [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly. * [NOD-828] Combine two lines into one. * [NOD-828] Combine another two lines into one. * [NOD-828] Move a misplaced comment. * [NOD-828] Use Wrapf instead of Errorf where appropriate. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Fix Wrapf calls. * [NOD-828] Fix comments. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Add a comment explaining the use of batches and snapshots. * [NOD-828] Implement RollbackUnlessClosed(). * [NOD-828] Return both errors in StoreBlock rollback. * [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor. * [NOD-828] Make bucket a type alias instead of a struct. * [NOD-828] Fix a typo. * [NOD-828] Use copy instead of append in Bucket. * [NOD-828] Extract flatFileLocationSerializedSize to a const. * [NOD-828] Debugf -> Warnf in rollback.go. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Remove data length from flat file data format. * [NOD-828] Rearrange TestLevelDBTransactionSanity a bit. * [NOD-828] Add stack traces to all errors that come out of library functions. * [NOD-828] Return errors from rollback(). * [NOD-828] Remove an irrelevant comment. * [NOD-828] Remove redundant whitespace. * [NOD-828] Handle nil in FetchBlock. * [NOD-828] Implement a dbaccess.BlockNode struct. * [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct. * [NOD-828] Implement toDBBlockNode and fromDBBlockNode. * Revert "[NOD-828] Make bucket a type alias instead of a struct." This reverts commit 1fd39652 * [NOD-828] Fix revert errors. * Revert "[NOD-828] Remove data length from flat file data format." This reverts commit ef408e32 * [NOD-862] Implement storing index blocks. * [NOD-862] Use database transactions where appropriate. * [NOD-862] Fix tests failing on DAGSetup. * [NOD-862] Fix bad make call. * [NOD-862] Fix remaining database opening problems in tests. * [NOD-862] Move Cursor() into the DataAccessor interface. * [NOD-862] Move Cursor() into the DataAccessor interface. * [NOD-862] Iterate over the new block index in dagio. * [NOD-862] Fix block index key. * [NOD-828] Add Delete to DataAccessor. * [NOD-862] Pass byte slices to dbaccess instead of objects. * [NOD-862] Fix errors. * [NOD-862] Fix maybeAcceptBlock not checking block existence. * [NOD-862] Fix TestAcceptanceIndexRecover. * [NOD-862] Add comments to StoreBlockIndex and BlockIndexCursor. * [NOD-828] Fix a couple of places that erroneously referenced blocks. * [NOD-828] Add a comment on top of flatFileLocationSerializedSize. * [NOD-828] Add Seek to Cursor. * [NOD-828] Add First to Cursor. * [NOD-828] Rename db to accessor in Context. * [NOD-828] Make Get/Fetch calls return a boolean to indicate whether the requested item was found. * [NOD-828] Name the output parameters of all Get functions. * [NOD-828] Make RetrieveFromStore return whether the data was found. * [NOD-862] Fix merge errors. * [NOD-862] Fix DAGSetup using bad temp directories. * [NOD-862] Fix TestProcessDelayedBlocks not closing the database properly. * [NOD-867] Remove blockIndexBucket from dagio. * [NOD-867] Fix wrong key in StoreIndexBucket. * [NOD-867] Migrate DAG state to dbaccess. * [NOD-867] Remove utxoSetVersionKeyName. * [NOD-862] Fix merge errors. * [NOD-867] Move localSubnetworkID into dagState. * [NOD-867] Fix a comment. * [NOD-867] Remove an unused function. * [NOD-867] Migrate the database's UTXO set to dbaccess. * [NOD-867] Add missing error check. * [NOD-867] Changed First to not return an error. * [NOD-867] Make Cursor.Key not return the entire key path. * [NOD-887] Fix the comment above BlockIndexCursorFrom. * [NOD-862] Merge flushToDBWithContext and flushToDB. * [NOD-862] Remove TODO. * [NOD-862] Add prefix to the temp dir in DAGSetup. * [NOD-862] Bring back dbFetchBlockByHash. * [NOD-862] Use BlockDAG.BlockByHash in p2p and rpc. * [NOD-862] Use daghash.Hash in dbaccess. * [NOD-862] Add defer to RollbackUnlessClosed after NewTx(). * [NOD-862] Extract dbStoreBlock to a separate function. * [NOD-867] Remove TODOs. * [NOD-867] Fix merge errors. * [NOD-867] Fix comments and errors. * [NOD-867] Unexport blockIndexKey. * [NOD-867] Fix merge errors. * [NOD-867] Move a misplaced comment. * [NOD-867] Fix an error message. * [NOD-867] Remove preallocation in initDAGState. * [NOD-866] Migrate database logic in blockdag/indexers package to dbaccess (#682) * [NOD-865] Delete blockidhash.go. * [NOD-865] Remove a lot of no-longer relevant logic from indexers. * [NOD-865] Pass TxContext to ConnectBlock. * [NOD-865] Migrate the acceptance index to dbaccess. * [NOD-865] Fix a block not being sent to ConnectBlock. * [NOD-865] Pass the block's hash instead of the whole block. * [NOD-865] Add forgotten Commit call. * [NOD-865] Add comments. * [NOD-866] Fix a comment. * [NOD-866] Fix a comment. * [NOD-866] Remove pointless indirection in acceptanceindex. * [NOD-866] Fix comment over ForEachHash. * [NOD-866] Rename ClearAcceptanceIndex to DropAcceptanceIndex. * [NOD-866] Explain collecting keys before deleting them. * [NOD-865] Move misc db logic to db access (#681) * [NOD-828] Create the database2 package that will some day replace the database package. * [NOD-828] Implement a "bucket" key mechanism. * [NOD-828] Move bucket.go into the ffldb2 package. * [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway. * [NOD-828] Copy over + fixup flat file structs from the old ffldb. * [NOD-828] Implement flatFilePath. * [NOD-828] Implement flatFileStore.write(). * [NOD-828] Implement flatFileStore.read(). * [NOD-828] Implement flatFileStore.rollback(). * [NOD-828] Sync the file to disk at the end of write(). * [NOD-828] Extract crc32ByteOrder to a separate variable. * [NOD-828] Add a sanity test. * [NOD-828] Remove context-unrelated methods from the Database interface. * [NOD-828] Create an ffldb object. Simply work against a context. * [NOD-828] Open the new database on start. * [NOD-828] Create the leveldb package. * [NOD-828] Implement opening/closing leveldb. * [NOD-828] Implement get/put out of/into leveldb. * [NOD-828] Implement transactions and make them implement a generic database interface. * [NOD-828] Write sanity tests for leveldb with and without transactions. * [NOD-828] Add another case to the transaction sanity test. * [NOD-828] Implement AppendBlock/RetrieveBlock. * [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package. * [NOD-828] Add RollbackFlatData to the database interface. * [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket. * [NOD-828] Implement StoreBlock. * [NOD-828] Implement FetchBlock. * [NOD-828] Implement HasBlock. * [NOD-828] Write a sanity test for block insertion. * [NOD-828] Implement CurrentFlatDataLocation. * [NOD-828] Implement storing the current block location. * [NOD-828] Implement initializing/syncing the flat file block store and the "metadata". * [NOD-828] Add InitBlockStore to TestBlockStoreSanity. * [NOD-828] Fix rename errors. * [NOD-828] Fix lint errors in the root database package. * [NOD-828] Fix lint errors in the ffldb.go. * [NOD-828] Fix lint errors in the flatfile/db.go. * [NOD-828] Rename packages in such a way to make the linter happy. * [NOD-828] Finish satisfying the linter. * [NOD-828] Fix doc.go. * [NOD-828] Fix comments in block.go. * [NOD-828] Move dbaccess out of the database package. * [NOD-828] Move opening/closing the database to dbaccess. * [NOD-828] Move the Database interface to the root database package, since it's meant to be generic. * [NOD-828] Make ffldb generic to later support additional databases. * [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported. * [NOD-828] Fix comments. * [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore. * [NOD-828] Make buckets nicer to use. * [NOD-828] Implement cursors that iterate over some bucket. * [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb. * [NOD-828] Write a test making sure that flat file repair works. * [NOD-828] Properly close the database in TestRepairFlatFiles. * [NOD-828] Add a comment warning against putting and getting the same data within the same transaction. * [NOD-828] Fix the flatFilesBucket description. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Fix the ffldb description. * [NOD-828] Rename Database to DataAccessor and rename Handle to Database. * [NOD-828] Make Get return nil if the value doesn't exist. * [NOD-828] Attempt to close leveldb even if closing ffdb failed. * [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket. * [NOD-828] Fix not updating the store location in ffldb transactions. * [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found. * [NOD-828] Update the README and doc.go. * [NOD-828] Simplify Bucket.Path(). * [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly. * [NOD-828] Combine two lines into one. * [NOD-828] Combine another two lines into one. * [NOD-828] Move a misplaced comment. * [NOD-828] Use Wrapf instead of Errorf where appropriate. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Fix Wrapf calls. * [NOD-828] Fix comments. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Add a comment explaining the use of batches and snapshots. * [NOD-828] Implement RollbackUnlessClosed(). * [NOD-828] Return both errors in StoreBlock rollback. * [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor. * [NOD-828] Make bucket a type alias instead of a struct. * [NOD-828] Fix a typo. * [NOD-828] Use copy instead of append in Bucket. * [NOD-828] Extract flatFileLocationSerializedSize to a const. * [NOD-828] Debugf -> Warnf in rollback.go. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Remove data length from flat file data format. * [NOD-828] Rearrange TestLevelDBTransactionSanity a bit. * [NOD-828] Add stack traces to all errors that come out of library functions. * [NOD-828] Return errors from rollback(). * [NOD-828] Remove an irrelevant comment. * [NOD-828] Remove redundant whitespace. * [NOD-828] Handle nil in FetchBlock. * [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct. * Revert "[NOD-828] Make bucket a type alias instead of a struct." This reverts commit 1fd39652 * [NOD-828] Fix revert errors. * Revert "[NOD-828] Remove data length from flat file data format." This reverts commit ef408e32 * [NOD-862] Move Cursor() into the DataAccessor interface. * [NOD-828] Add Delete to DataAccessor. * [NOD-865] Move fee data db operations to dbaccess * [NOD-865] Move reachability data db operations to dbaccess * [NOD-865] Move UTXO diff data db operations to dbaccess * [NOD-865] Move subnetwork data db operations to dbaccess * [NOD-865] Fix createDAGState * [NOD-865] Remove old Get signature with "exists" * [NOD-865] Move multiset db operations to dbaccess * [NOD-865] Use dbaccess transactions where possible * [NOD-865] Remove old Get signature with "exists" * [NOD-881] Recover TestGHOSTDAGErrors * [NOD-865] Create function for db keys * [NOD-865] Change Exists to Has, and use accessor.Has where possible * [NOD-865] Make ClearReachabilityData transactive * [NOD-865] Don't iterate cursors while changing db data * [NOD-865] Rename RegisterSubnetwork -> StoreSubnetwork * [NOD-865] Change bucket from utxodiffs to utxo-diffs * [NOD-865] Rename SubnetworkExists->HasSubnetwork * [NOD-865] Change a comment * [NOD-865] Fix tests * [NOD-865] Fix comment * [NOD-865] Remove the prefix "db" from some functions * [NOD-865] Remove redundant comments * [NOD-865] Make clearBucket function * [NOD-865] Make clear functions get a dbTx as an arg * [NOD-865] Remove erroneous tx commit Co-authored-by: stasatdaglabs <stas@daglabs.com> * [NOD-868] Delete the old database package (#683) * [NOD-828] Create the database2 package that will some day replace the database package. * [NOD-828] Implement a "bucket" key mechanism. * [NOD-828] Move bucket.go into the ffldb2 package. * [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway. * [NOD-828] Copy over + fixup flat file structs from the old ffldb. * [NOD-828] Implement flatFilePath. * [NOD-828] Implement flatFileStore.write(). * [NOD-828] Implement flatFileStore.read(). * [NOD-828] Implement flatFileStore.rollback(). * [NOD-828] Sync the file to disk at the end of write(). * [NOD-828] Extract crc32ByteOrder to a separate variable. * [NOD-828] Add a sanity test. * [NOD-828] Remove context-unrelated methods from the Database interface. * [NOD-828] Create an ffldb object. Simply work against a context. * [NOD-828] Open the new database on start. * [NOD-828] Create the leveldb package. * [NOD-828] Implement opening/closing leveldb. * [NOD-828] Implement get/put out of/into leveldb. * [NOD-828] Implement transactions and make them implement a generic database interface. * [NOD-828] Write sanity tests for leveldb with and without transactions. * [NOD-828] Add another case to the transaction sanity test. * [NOD-828] Implement AppendBlock/RetrieveBlock. * [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package. * [NOD-828] Add RollbackFlatData to the database interface. * [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket. * [NOD-828] Implement StoreBlock. * [NOD-828] Implement FetchBlock. * [NOD-828] Implement HasBlock. * [NOD-828] Write a sanity test for block insertion. * [NOD-828] Implement CurrentFlatDataLocation. * [NOD-828] Implement storing the current block location. * [NOD-828] Implement initializing/syncing the flat file block store and the "metadata". * [NOD-828] Add InitBlockStore to TestBlockStoreSanity. * [NOD-828] Fix rename errors. * [NOD-828] Fix lint errors in the root database package. * [NOD-828] Fix lint errors in the ffldb.go. * [NOD-828] Fix lint errors in the flatfile/db.go. * [NOD-828] Rename packages in such a way to make the linter happy. * [NOD-828] Finish satisfying the linter. * [NOD-828] Fix doc.go. * [NOD-828] Fix comments in block.go. * [NOD-828] Move dbaccess out of the database package. * [NOD-828] Move opening/closing the database to dbaccess. * [NOD-828] Move the Database interface to the root database package, since it's meant to be generic. * [NOD-828] Make ffldb generic to later support additional databases. * [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported. * [NOD-828] Fix comments. * [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore. * [NOD-828] Make buckets nicer to use. * [NOD-828] Implement cursors that iterate over some bucket. * [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb. * [NOD-828] Write a test making sure that flat file repair works. * [NOD-828] Properly close the database in TestRepairFlatFiles. * [NOD-828] Add a comment warning against putting and getting the same data within the same transaction. * [NOD-828] Fix the flatFilesBucket description. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Fix the ffldb description. * [NOD-828] Rename Database to DataAccessor and rename Handle to Database. * [NOD-828] Make Get return nil if the value doesn't exist. * [NOD-828] Attempt to close leveldb even if closing ffdb failed. * [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket. * [NOD-828] Fix not updating the store location in ffldb transactions. * [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found. * [NOD-828] Update the README and doc.go. * [NOD-828] Simplify Bucket.Path(). * [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly. * [NOD-828] Combine two lines into one. * [NOD-828] Combine another two lines into one. * [NOD-828] Move a misplaced comment. * [NOD-828] Use Wrapf instead of Errorf where appropriate. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Fix Wrapf calls. * [NOD-828] Fix comments. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Add a comment explaining the use of batches and snapshots. * [NOD-828] Implement RollbackUnlessClosed(). * [NOD-828] Return both errors in StoreBlock rollback. * [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor. * [NOD-828] Make bucket a type alias instead of a struct. * [NOD-828] Fix a typo. * [NOD-828] Use copy instead of append in Bucket. * [NOD-828] Extract flatFileLocationSerializedSize to a const. * [NOD-828] Debugf -> Warnf in rollback.go. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Remove data length from flat file data format. * [NOD-828] Rearrange TestLevelDBTransactionSanity a bit. * [NOD-828] Add stack traces to all errors that come out of library functions. * [NOD-828] Return errors from rollback(). * [NOD-828] Remove an irrelevant comment. * [NOD-828] Remove redundant whitespace. * [NOD-828] Handle nil in FetchBlock. * [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct. * Revert "[NOD-828] Make bucket a type alias instead of a struct." This reverts commit 1fd39652 * [NOD-828] Fix revert errors. * Revert "[NOD-828] Remove data length from flat file data format." This reverts commit ef408e32 * [NOD-862] Move Cursor() into the DataAccessor interface. * [NOD-828] Add Delete to DataAccessor. * [NOD-865] Move fee data db operations to dbaccess * [NOD-865] Move reachability data db operations to dbaccess * [NOD-865] Move UTXO diff data db operations to dbaccess * [NOD-865] Move subnetwork data db operations to dbaccess * [NOD-865] Fix createDAGState * [NOD-865] Remove old Get signature with "exists" * [NOD-865] Move multiset db operations to dbaccess * [NOD-865] Use dbaccess transactions where possible * [NOD-865] Remove old Get signature with "exists" * [NOD-881] Recover TestGHOSTDAGErrors * [NOD-865] Create function for db keys * [NOD-865] Change Exists to Has, and use accessor.Has where possible * [NOD-865] Make ClearReachabilityData transactive * [NOD-865] Don't iterate cursors while changing db data * [NOD-865] Rename RegisterSubnetwork -> StoreSubnetwork * [NOD-865] Change bucket from utxodiffs to utxo-diffs * [NOD-865] Rename SubnetworkExists->HasSubnetwork * [NOD-865] Change a comment * [NOD-868] Remove all tests from old database. * [NOD-868] Remove all unused methods from the old database's interfaces. * [NOD-865] Fix tests * [NOD-868] Remove references to DB. * [NOD-865] Fix comment * [NOD-868] Remove the old ffldb besides the interface and errors.go. * [NOD-868] Remove errors.go. * [NOD-868] Remove the old database package. * [NOD-868] Add openDB to DAGSetup to emulate the old dbpath in dag.config. * [NOD-868] Rename database2 to database. * [NOD-868] Use NewTx instead of NoTx where required. * [NOD-868] Fix merge errors. * [NOD-868] Rename dbXXX functions to just xxx. * [NOD-868] Rename putDAGState to saveDAGState. * [NOD-868] Replace comments in initDAGState with logs. * [NOD-868] Explain the openDB parameter in DAGSetup. * [NOD-868] Fixup doc.go and README.md. * [NOD-868] Remove pointless transactions. Co-authored-by: Ori Newman <orinewman1@gmail.com> * [NOD-805] Fix merge errors. * [NOD-805] Fix a comment. * [NOD-805] Don't return virtualTxsAcceptanceData from applyDAGChanges. * [NOD-805] Add missing error handling in TestAcceptanceDataIndexRecover. * [NOD-805] Rename blockDAG to dag in indexers/manager.go. * [NOD-805] Defer cursor.Close() everywhere. * [NOD-805] Rename scanFlatFiles to findCurrentLocation. * [NOD-805] Extract crc32ChecksumLength and dataLengthLength to constants. * [NOD-805] Handle open files properly in rollback.go. * [NOD-805] Remove unnecessary func wrapper. * [NOD-805] Remove unnecessary trimming in initialize. * [NOD-805] Made StoreBlock accept only TxContext. * [NOD-805] Changed the log level of an error message to Error. * [NOD-805] Add a note about holding mutexes over deleteFile. * [NOD-805] Remove a false comment. * [NOD-805] Fix a comment. * [NOD-805] Rename blk to block. * [NOD-805] Extract utxoKey to a separate function. * [NOD-805] Move dbaccess.xxxKey functions to the tops of their respective files. * [NOD-805] Fix grammar in dbaccess/db.go. * [NOD-805] Wrap a failed database corruption recovery error. * [NOD-805] Split lines with WithStack in them. * [NOD-805] Fix the comment over initialize. * [NOD-805] Rename ffdb to flatFileDB and ldb to levelDB. * [NOD-805] Fix a comment. * [NOD-805] Fix a comment. * [NOD-805] Use s.writeCursor instead of cursor. * [NOD-805] Embed file in lockableFile. * [NOD-805] the the -> the * [NOD-805] openDB -> db * [NOD-805] Use TxContext in all flushToDB functions. * [NOD-805] Rename context -> dbContext. * [NOD-805] Reword the comment at the beginning on initDAGState. * [NOD-805] Explain cursor key trimming. * [NOD-805] Remove Error from Cursor. * [NOD-805] Return ErrNotFound from done Cursor Key and Value. * [NOD-805] Add missing error handling. * [NOD-805] Fix a comment. * [NOD-805] Fix a variable name. * [NOD-805] Remove pointless underscore. * [NOD-805] Fix a comment. * [NOD-805] Fix a variable name. Co-authored-by: Mike Zak <feanorr@gmail.com> Co-authored-by: Ori Newman <orinewman1@gmail.com> |
||
|
Ori Newman
|
d83862f36c
|
[NOD-855] Save ECMH for block utxo and not diff utxo (#669)
* [NOD-855] Save ECMH for each block UTXO * [NOD-855] Remove UpdateExtraNonce method * [NOD-855] Remove multiset data from UTXO diffs * [NOD-855] Fix to fetch multiset of selected parent * [NOD-855] Don't remove coinbase inputs from multiset * [NOD-855] Create multisetBucketName on startup * [NOD-855] Remove multiset from UTXO diff tests * [NOD-855] clear new entries from multisetstore on saveChangesFromBlock * [NOD-855] Fix tests * [NOD-855] Use UnacceptedBlueScore when adding current block transactions to multiset * [NOD-855] Hash utxo before adding it to multiset * [NOD-855] Pass isCoinbase to NewUTXOEntry * [NOD-855] Do not use hash when adding entries to multiset * [NOD-855] When calculating multiset, replace the unaccepted blue score of selected parent transaction with the block blue score * [NOD-855] Manually add a chained transaction to a block in TestChainedTransactions * [NOD-855] Change name and comments * [NOD-855] Use FindAcceptanceData to find a specific block acceptance data * [NOD-855] Remove redundant copy of txIn.PreviousOutpoint * [NOD-855] Use fmt.Sprintf when creating internalRPCError |
||
|
Ori Newman
|
d3b1953deb
|
[NOD-848] optimize utxo diffs serialize allocations (#666)
* [NOD-848] Optimize allocations when serializing UTXO diffs * [NOD-848] Use same UTXO serialization everywhere, and use compression as well * [NOD-848] Fix usage of wrong buffer * [NOD-848] Fix tests * [NOD-848] Fix wire tests * [NOD-848] Fix tests * [NOD-848] Remove VLQ * [NOD-848] Fix comments * [NOD-848] Add varint for big endian encoding * [NOD-848] In TestVarIntWire, assume the expected decoded value is the same as the serialization input * [NOD-848] Serialize outpoint index with big endian varint * [NOD-848] Remove p2pk from compression support * [NOD-848] Fix comments * [NOD-848] Remove p2pk from decompression support * [NOD-848] Make entry compression optional * [NOD-848] Fix tests * [NOD-848] Fix comments and var names * [NOD-848] Remove UTXO compression * [NOD-848] Fix tests * [NOD-848] Remove big endian varint * [NOD-848] Fix comments * [NOD-848] Rename ReadVarIntLittleEndian->ReadVarInt and fix WriteVarInt comment * [NOD-848] Add outpointIndexByteOrder variable * [NOD-848] Remove redundant comment * [NOD-848] Fix outpointMaxSerializeSize to the correct value * [NOD-848] Move subBuffer to utils |
||
|
Ori Newman
|
ee08531a52
|
[NOD-610] Rename newSet->newBlockSet and setFromSlice->blockSetFromSlice (#635) | ||
|
Ori Newman
|
a4bb070722
|
[NOD-754] Fix staticcheck errors (#627)
* [NOD-754] Fix staticcheck errors * [NOD-754] Remove some unused exported functions * [NOD-754] Fix staticcheck errors * [NOD-754] Don't panic if out/in close fails * [NOD-754] Wrap outside errors with custom message |
||
|
Ori Newman
|
d8954f1339
|
[NOD-615] Make bluesAnticoneSizes a map with *blockNode as a key (#619) | ||
|
Ori Newman
|
eb953286ec
|
[NOD-641] Upgrade to github.com/pkg/errors v0.9.1 and use errors.As where needed (#614)
* [NOD-641] Upgrade to github.com/pkg/errors v0.9.1 and use errors.As where needed * [NOD-641] Fix find and replace error * [NOD-641] Use errors.As for error type checking * [NOD-641] Fix errors.As for pointer types * [NOD-641] Use errors.As where needed * [NOD-641] Rename rErr->ruleErr * [NOD-641] Rename derr->dbErr * [NOD-641] e->flagsErr where necessary * [NOD-641] change jerr to more appropriate name * [NOD-641] Rename cerr->bdRuleErr * [NOD-641] Rename serr->scriptErr * [NOD-641] Use errors.Is instead of testutil.AreErrorsEqual in TestNewHashFromStr * [NOD-641] Rename bdRuleErr->dagRuleErr * [NOD-641] Rename mErr->msgErr * [NOD-641] Rename dErr->deserializeErr |
||
|
Ori Newman
|
41c8178ad3
|
[NOD-648] Add TestProcessDelayedBlocks (#612)
* [NOD-648] Add TestProcessDelayedBlocks * [NOD-648] Add one second to secondsUntilDelayedBlockIsValid to make sure the delayedBlock timestamp will be valid, and add comments * [NOD-648] Remove redundant import * [NOD-648] Use fakeTimeSource instead of time.Sleep * [NOD-648] Rename dag.HaveBlock->dag.IsKnownBlock, dag.BlockExists->dag.IsInDAG * [NOD-648] Add comment * [NOD-641] Rename HaveBlock->IsKnownBlock, BlockExists->IsInDAG |
||
|
Dan Aharoni
|
52e0a0967d |
[NOD-629] change GHOSTDAG k to uint8 (#594)
* [NOD-629] Change GHOSTDAG K to a a single byte using type * [NOD-629] Rename variable * [NOD-629] Rename K to KSize * [NOD-629] Remove redundant casting * [NOD-629] Add test for KSize * [NOD-629] Seperate block serialization and db store * [NOD-629] Make sure K is serialized as uint8 * [NOD-629] Rename KSize to KType * [NOD-629] Comment for test * [NOD-629] Change fail message * [NOD-629] Remove newlines * [NOD-629] Fix test * [NOD-629] Do not use maxuint8, but !0 instead * [NOD-629] Fix test * [NOD-629] Merge conflict * [NOD-629] Fix test; Update comment |
||
|
stasatdaglabs
|
49418f4222 |
[NOD-669] Rename start/endHash -> low/highHash (#591)
* [NOD-669] Remove the "get" from getBlueBlocksBetween. * [NOD-669] Remove the "Get" from GetBlueBlocksHeadersBetween. * [NOD-669] In blueBlocksBetween, rename startHash to lowHash and stopHash to highHash. * [NOD-669] Rename startHash to lowHash and stopHash to highHash in blockLocator logic. * [NOD-669] Remove zeroHash logic in blockLocator. * [NOD-669] Finish renaming startHash and stopHash in blockdag. * [NOD-669] Rename startHash and stopHash in blockdag where I previously missed it. * [NOD-669] Rename startHash and stopHash in blockdag where I previously missed it some more. * [NOD-669] Rename startHash and stopHash in blockdag where I previously missed it some more some more. * [NOD-669] Fix bad grammar in method names. * [NOD-669] Rename lowHash to blockHash in SelectedParentChain. * [NOD-669] Fix a comment. |
||
|
stasatdaglabs
|
a7d3a40465 | [NOD-646] Fix initDAGState treating invalid blocks as genesis blocks. (#588) | ||
|
stasatdaglabs
|
359b16fca9 |
[NOD-616] Remove blockNode.chainHeight (#586)
* [NOD-616] Remove unused methods from BlockDAG. * [NOD-616] Remove Height from GetRawMempoolVerboseResult and TxDesc. * [NOD-616] Replaced BlockDAG.ChainHeight with SelectedTipBlueScore. * [NOD-616] Remove the unused BlockChainHeightByHash. * [NOD-616] Remove the unused blockChainHeight from checkBlockHeaderContext. * [NOD-616] Remove chainHeight from util.Block. * [NOD-616] Remove TestChainHeight. * [NOD-616] Update unknown rule activation warning to use blueScore. * [NOD-616] Update thresholdState to use blueScore instead of chainHeight. * [NOD-616] Update blockLocator to use blueScore instead of chainHeight. * [NOD-616] Remove blockNode.chainHeight. * [NOD-616] Fix comments and variable names. * [NOD-616] Replace a weird for loop with a while loop. * [NOD-616] Fix a comment. * [NOD-616] Remove pre-allocation in blockLocator. * [NOD-616] Coalesce checks that startHash and stopHash are not the same into the same condition. * [NOD-616] Fix a comment. * [NOD-616] Remove weird blueScore logic around childHashStrings. * [NOD-616] Fix hash pointer comparison. * [NOD-616] Fix a comment. * [NOD-616] Add ban score to peers misusing GetBlockLocator. * [NOD-616] Replace adding ban score with disconnecting. * [NOD-616] Add blueScore to FilteredBlockAddedNtfn. |
||
|
Ori Newman
|
2174a0a7f2 |
[NOD-497] Implement GHOSTDAG (#575)
* [NOD-540] Implement reachability (#545) * [NOD-540] Begin implementing reachability. * [NOD-540] Finish implementing reachability. * [NOD-540] Implement TestIsFutureBlock. * [NOD-540] Implement TestInsertFutureBlock. * [NOD-540] Add comments. * [NOD-540] Add comment for interval in blockNode. * [NOD-540] Updated comments over insertFutureBlock and isFutureBlock. * [NOD-540] Implement interval splitting methods. * [NOD-540] Begin implementing tree manipulation in blockNode. * [NOD-540] Implement countSubtreesUp. * [NOD-540] Add a comment explaining an impossible condition. * [NOD-540] Implement applyIntervalDown. * [NOD-540] Moved the reachability tree stuff into reachability.go. * [NOD-540] Add some comments. * [NOD-540] Add more comments, implement isInPast. * [NOD-540] Fix comments. * [NOD-540] Implement TestSplitFraction. * [NOD-540] Implement TestSplitExact. * [NOD-540] Implement TestSplit. * [NOD-540] Add comments to structs. * [NOD-540] Implement TestAddTreeChild. * [NOD-540] Fix a comment. * [NOD-540] Rename isInPast to isAncestorOf. * [NOD-540] Rename futureBlocks to futureCoveringSet. * [NOD-540] Rename isFutureBlock to isInFuture. * [NOD-540] move reachabilityInterval to the top of reachability.go. * [NOD-540] Change "s.t." to "such that" in a comment. * [NOD-540] Fix indentation. * [NOD-540] Fix a potential bug involving float inaccuracy. * [NOD-540] Wrote a more descriptive error message. * [NOD-540] Fix error messsage. * [NOD-540] Fix the recursive countSubtreesUp. * [NOD-540] Rename countSubtreesUp to countSubtrees and applyIntervalDown to propagateInterval. * [NOD-540] Implement updating reachability for a valid new block. * [NOD-540] Implement a disk storage for reachability data. * [NOD-540] Fix not all tree nodes being written to the database. * [NOD-540] Implement serialization for reachabilityData. * [NOD-540] Implement some deserialization for reachabilityData. * [NOD-540] Implement restoring the reachabilityStore on node restart. * [NOD-540] Made interval and remainingInterval pointers. * [NOD-540] Rename setTreeInterval to setInterval. * [NOD-540] Rename reindexTreeIntervals to reindexIntervals and fixed the comment above it. * [NOD-540] Expand the comment above reindexIntervals. * [NOD-540] Fix comment above countSubtrees. * [NOD-540] Fix comment above countSubtrees some more. * [NOD-540] Fix comment above split. * [NOD-540] Fix comment above isAncestorOf. * [NOD-540] Fix comment above reachabilityTreeNode. * [NOD-540] Fix weird condition in addTreeChild. * [NOD-540] Rename addTreeChild to addChild. * [NOD-540] Fix weird condition in splitFraction. * [NOD-540] Reverse the lines in reachabilityTreeNode.String(). * [NOD-540] Renamed f to fraction and x to size. * [NOD-540] Fix comment above bisect. * [NOD-540] Implement rtn.isAncestorOf(). * [NOD-540] Use treeNode isAncestorOf instead of treeInterval isAncestorOf. * [NOD-540] Use newReachabilityInterval instead of struct initialization. * [NOD-540] Make reachabilityTreeNode.String() use strings.Join. * [NOD-540] Use sync.RWMutex instead of locks.PriorityMutex. * [NOD-540] Rename thisTreeNode to newTreeNode. * [NOD-540] Rename setTreeNode to addTreeNode. * [NOD-540] Extracted selectedParentAnticone to a separate function. * [NOD-540] Rename node to this. * [NOD-540] Move updateReachability and isAncestorOf from dag.go to reachability.go. * [NOD-540] Add whitespace after multiline function signatures in reachability.go. * [NOD-540] Make splitFraction return an error on empty interval. * [NOD-540] Add a comment about rounding to splitFraction. * [NOD-540] Replace sneaky tabs with spaces. * [NOD-540] Rename split to splitExponential. * [NOD-540] Extract exponentialFractions to a separate function. * [NOD-540] Rename bisect to findIndex. * [NOD-540] Add call to reachabilityStore.clearDirtyEntries at the end of saveChangesFromBlock. * [NOD-540] Explain the dirty hack in reachabilityStore.init(). * [NOD-540] Split the function signature for deserializeReachabilityData to two lines. * [NOD-540] Add a comment about float precision loss to exponentialFractions. * [NOD-540] Corrected a comment about float precision loss to exponentialFractions. * [NOD-540] Fixed a comment about float precision loss to exponentialFractions some more. * [NOD-540] Added further comments above futureCoveringBlockSet. * [NOD-540] Rename addTreeNode to setTreeNode. * [NOD-540] Rename splitExponential to splitWithExponentialBias. * [NOD-540] Fix object references in reachabilityData deserialization (#563) * [NOD-540] Fix broken references in deserialization. * [NOD-540] Fix broken references in futureCoveringSet deserialization. Also add comments. * [NOD-540] Don't deserialize on the first pass in reachabilityStore.init(). * [NOD-540] Remove redundant assignment to loaded[hash]. * [NOD-540] Use NewHash instead of SetBytes. Rename data to destination. * [NOD-540] Preallocate futureCoveringSet. * [NOD-541] Implement GHOSTDAG (#560) * [NOD-541] Implement GHOSTDAG * [NOD-541] Replace the old PHANTOM variant with GHOSTDAG * [NOD-541] Move dag.updateReachability to the top of dag.applyDAGChanges to update reachability before the virtual block is updated * [NOD-541] Fix blueAnticoneSize * [NOD-541] Initialize node.bluesAnticoneSizes * [NOD-541] Fix pastUTXO and applyBlueBlocks blues order * [NOD-541] Add serialization logic to node.bluesAnticoneSizes * [NOD-541] Fix GHOSTDAG to not count the new block and the blue candidates anticone, add selected parent to blues, and save to node.bluesAnticoneSizes properly * [NOD-541] Fix test names in inner strings * [NOD-541] Writing TestGHOSTDAG * [NOD-541] In blueAnticoneSize change node->current * [NOD-541] name ghostdag return values * [NOD-541] fix ghostdag to return slice * [NOD-541] Split k-cluster violation rules * [NOD-541] Add missing space * [NOD-541] Add comment to ghostdag * [NOD-541] In selectedParentAnticone rename past->selectedParentPast * [NOD-541] Fix misrefernces to TestChainUpdates * [NOD-541] Fix ghostdag comment * [NOD-541] Make PrepareBlockForTest in blockdag package * [NOD-541] Make PrepareBlockForTest in blockdag package * [NOD-541] Assign to selectedParentAnticone[i] instead of appending * [NOD-541] Remove redundant forceTransactions arguments from PrepareBlockForTEST * [NOD-541] Add non-selected parents to anticoneHeap * [NOD-541] add test for ghostdag * [NOD-541] Add comments * [NOD-541] Use adjusted time for initializing blockNode * [NOD-541] Rename isAncestorOf -> isAncestorOfBlueCandidate * [NOD-541] Remove params from PrepareBlockForTest * [NOD-541] Fix TestChainHeight * [NOD-541] Remove recursive lock * [NOD-541] Fix TestTxIndexConnectBlock * [NOD-541] Fix TestBlueBlockWindow * [NOD-541] Put prepareAndProcessBlock in common_test.go * [NOD-541] Fix TestConfirmations * [NOD-541] Fix TestAcceptingBlock * [NOD-541] Fix TestDifficulty * [NOD-541] Fix TestVirtualBlock * [NOD-541] Fix TestSelectedPath * [NOD-541] Fix TestChainUpdates * [NOD-541] Shorten TestDifficulty test time * [NOD-541] Make PrepareBlockForTest use minimal valid block time * [NOD-541] Remove TODO comment * [NOD-541] Move blockdag related mining functions to mining.go * [NOD-541] Use NextBlockCoinbaseTransaction instead of NextBlockCoinbaseTransactionNoLock in NextCoinbaseFromAddress * [NOD-541] Remove useMinimalTime from BlockForMining * [NOD-541] Make MedianAdjustedTime a *BlockDAG method * [NOD-541] Fix ghostdag to use anticone slice instead of heap * [NOD-541] Fix NewBlockTemplate locks * [NOD-541] Fix ghostdag comments * [NOD-541] Convert MedianAdjustedTime to NextBlockTime * [NOD-541] Fix ghostdag comment * [NOD-541] Fix TestGHOSTDAG comment * [NOD-541] Add comment before sanity check * [NOD-541] Explicitly initialize .blues in ghostdag * [NOD-541] Rename *blockNode.lessThan to *blockNode.less * [NOD-541] Remove redundant check if block != chainBlock * [NOD-541] Fix comment * [NOD-541] Fix comment * [NOD-497] Add comment; General refactoring * [NOD-497] General refactoring. * [NOD-497] Use isAncestor of the tree rather than the node * [NOD-497] Remove reachability mutex lock as it is redundant (dag lock is held so no need); General refactoring. * [NOD-497] Update comment * [NOD-497] Undo test blocktimestamp * [NOD-497] Update comments; Use BlockNode.less for blockset; * [NOD-497] Change processBlock to return boolean and not the delay duration (merge conflict) * [NOD-497] Undo change for bluest to use less; Change blocknode less to use daghash.Less Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com> Co-authored-by: Dan Aharoni <dereeno@protonmail.com> |
||
|
Dan Aharoni
|
ea6f7a28c2 |
[NOD-420] Process delayed blocks (#529)
* [NOD-420] Delay blocks with valid timestamp (non-delayed) that point to a delayed block. * [NOD-420] Mark block as requested when setting as delayed. * [NOD-420] Merge master; Use dag.timeSource.AdjustedTime() instead of time.Now; * [NOD-420] Return nil when not expecting an error * [NOD-420] Initialise delyaed blocks mapping * [NOD-420] Trigger delayed blocks processing every time we process a block. * [NOD-420] Hold the read lock in processDelayedBlocks * [NOD-420] Add delayed blocks heap sorted by their process time so we could process them in order. * [NOD-420] Update debug log * [NOD-420] Fix process blocks loop * [NOD-420] Add comment * [NOD-420] Log error message * [NOD-420] Implement peek method for delayed block heap. extract delayed block processing to another function. * [NOD-420] Trigger process delayed blocks only in process block * [NOD-420] Move delayed block addition to process block * [NOD-420] Use process block to make sure we fully process the delayed block and deal with orphans. * [NOD-420] Unexport functions when not needed; Return isDelayed boolean from ProcessBlock instead of the delay duration * [NOd-420] Remove redundant delayedBlocksLock * [NOD-420] Resolve merge conflict; Return delay 0 instead of boolean * [NOD-420] Do not treat delayed block as orphan * [NOD-420] Make sure block is not processed if we have already sa delayed. * [NOD-420] Process delayed block if parent is delayed to make sure it would not be treated as orphan. * [NOD-420] Rename variable * [NOD-420] Rename function. Move maxDelayOfParents to process.go * [NOD-420] Fix typo * [NOD-420] Handle errors from processDelayedBlocks properly * [NOD-420] Return default values if err != nil from dag.addDelayedBlock * [NOD-420] Return default values if err != nil from dag.addDelayedBlock in another place Co-authored-by: Svarog <feanorr@gmail.com> |
||
|
stasatdaglabs
|
03b7af9a13 |
[NOD-532] Replace "chain" with "DAG" where appropriate (#537)
* [NOD-532] Change chain to DAG in the root package. * [NOD-532] Change chain to DAG in checkpoints.go. * [NOD-532] Change chain to DAG in blockdag. * [NOD-532] Change chain to DAG in cmd. * [NOD-532] Change chain to DAG in dagconfig. * [NOD-532] Change chain to DAG in database. * [NOD-532] Change chain to DAG in mempool. * [NOD-532] Change chain to DAG in mempool. * [NOD-532] Change chain to DAG in netsync. * [NOD-532] Change chain to DAG in rpcclient. * [NOD-532] Change chain to DAG in server. * [NOD-532] Change chain to DAG in txscript. * [NOD-532] Change chain to DAG in util. * [NOD-532] Change chain to DAG in wire. * [NOD-532] Remove block heights in dagio.go examples. * [NOD-532] Rename fakeChain to fakeDAG. * [NOD-532] Fix comments, remove unused EnableBCInfoHacks flag. * [NOD-532] Fix comments and variable names. * [NOD-532] Fix comments. * [NOD-532] Fix merge errors. * [NOD-532] Formatted project. |
||
|
stasatdaglabs
|
f46dec449d |
[NOD-510] Change all references to Bitcoin to Kaspa (#531)
* [NOD-510] Change coinbase flags to kaspad. * [NOD-510] Removed superfluous spaces after periods in comments. * [NOD-510] Rename btcd -> kaspad in the root folder. * [NOD-510] Rename BtcEncode -> KaspaEncode and BtcDecode -> KaspaDecode. * [NOD-510] Rename BtcEncode -> KaspaEncode and BtcDecode -> KaspaDecode. * [NOD-510] Continue renaming btcd -> kaspad. * [NOD-510] Rename btcjson -> kaspajson. * [NOD-510] Rename file names inside kaspajson. * [NOD-510] Rename kaspajson -> jsonrpc. * [NOD-510] Finish renaming in addrmgr. * [NOD-510] Rename package btcec to ecc. * [NOD-510] Finish renaming stuff in blockdag. * [NOD-510] Rename stuff in cmd. * [NOD-510] Rename stuff in config. * [NOD-510] Rename stuff in connmgr. * [NOD-510] Rename stuff in dagconfig. * [NOD-510] Rename stuff in database. * [NOD-510] Rename stuff in docker. * [NOD-510] Rename stuff in integration. * [NOD-510] Rename jsonrpc to rpcmodel. * [NOD-510] Rename stuff in limits. * [NOD-510] Rename stuff in logger. * [NOD-510] Rename stuff in mempool. * [NOD-510] Rename stuff in mining. * [NOD-510] Rename stuff in netsync. * [NOD-510] Rename stuff in peer. * [NOD-510] Rename stuff in release. * [NOD-510] Rename stuff in rpcclient. * [NOD-510] Rename stuff in server. * [NOD-510] Rename stuff in signal. * [NOD-510] Rename stuff in txscript. * [NOD-510] Rename stuff in util. * [NOD-510] Rename stuff in wire. * [NOD-510] Fix failing tests. * [NOD-510] Fix merge errors. * [NOD-510] Fix go vet errors. * [NOD-510] Remove merged file that's no longer relevant. * [NOD-510] Add a comment above Op0. * [NOD-510] Fix some comments referencing Bitcoin Core. * [NOD-510] Fix some more comments referencing Bitcoin Core. * [NOD-510] Fix bitcoin -> kaspa. * [NOD-510] Fix more bitcoin -> kaspa. * [NOD-510] Fix comments, remove DisconnectBlock in addrindex. * [NOD-510] Rename KSPD to KASD. * [NOD-510] Fix comments and user agent. |
||
|
stasatdaglabs
|
30f0e95969 |
[NOD-506] Remove blockNode.height and any references to it (#522)
* [NOD-506] Remove blockNode.height. * [NOD-506] Use blueScore instead of chainHeight in validateParents. |
||
|
Svarog
|
369ec449a8 |
[NOD-509] Change organization name to kaspanet (#524)
* [NOD-509] Change organization name to kaspanet * [NOD-509] Reorganize imports |
||
|
Svarog
|
f4c6859e51 | [NOD-509] Updated repository and imports to github.com/daglabs/kaspad (#521) | ||
|
Ori Newman
|
c88fa1492e |
[NOD-375] Move to pkg/errors (#447)
* [NOD-375] Move to pkg/errors * [NOD-375] Fix tests * [NOD-375] Make AreErrorsEqual a shared function |
||
|
Ori Newman
|
e6a4ed04f3 |
[NOD-338] recover indexer if didnt work for a while (#422)
* [NOD-338] Recover indexer if it didn't work for a while * [NOD-338] Recover indexer if it didn't work for a while * [NOD-338] Recover indexer if it didn't work for a while * [NOD-338] Add tests and move blockidhash.go to blockdag package * [NOD-338] Delete index current block id when dropping index, and do some refactoring * [NOD-338] Change comments * [NOD-338] Change recover error messages * [NOD-338] Fix comments * [NOD-338] Fix comments and fix test name |
||
|
stasatdaglabs
|
adf4b4380e
|
[NOD-289] Implement API-Server bootstrapping and booting after downtime (#408)
* [NOD-289] Implemented database isCurrent checking and connection. * [NOD-289] Added GetChainFromBlock to RPCClient. * [NOD-289] Limited the amount of blocks in GetChainFromBlockResponse. * [NOD-289] Fixed various issues that were keeping GetChainFromBlocks from working properly. * [NOD-289] Created blockloop.go. * [NOD-289] Updated go.mod after merge. * [NOD-289] Implemented collection of current selected parent chain. * [NOD-289] Fixed test. Reverted not deleting utxoDiffData from the DB. * [NOD-289] Implemented GetBlocks. * [NOD-289] Added comment to BlockHashesFrom. * [NOD-289] Added GetBlocks to rpcclient. * [NOD-289] Added verboseBlocks to GetBlocks. * [NOD-289] Implemented block insertion. * [NOD-289] Added AUTO_INCREMENT to tables that were missing it. * [NOD-289] Made gasLimit in subnetwork nullable. * [NOD-289] Renamed transactions_outputs to transaction_outputs. * [NOD-289] Fixed weird coinbase behavior in vin. * [NOD-289] Made collectCurrentBlocks start from the most recent startHash. * [NOD-289] Added IsChainBlock to GetBlockVerboseResult. * [NOD-289] Implemented adding a block from onBlockAdded. * [NOD-289] Added removedParentChainHashes to getChainFromBlock. * [NOD-289] Implemented updating the selected parent chain from onChainChanged. * [NOD-289] Implemented some initial logic for updating the UTXO. * [NOD-289] Fixed merge errors. * [NOD-326] Fixed some more merge errors. * [NOD-289] Added error handling for missing required records. * [NOD-289] Implemented handling removedChainHashes. * [NOD-289] Implemented handling addedChainBlocks. * [NOD-289] Fixed incorrect coinbase check. * [NOD-289] Implemented inserting the transaction output address. * [NOD-289] Added updating block.IsChainBlock. * [NOD-289] Split insertBlock into many small functions. * [NOD-289] Split updateSelectedParentChain into smaller functions. * [NOD-289] Fixed pointer errors. * [NOD-289] Fixed a bad exists check. * [NOD-289] Fixed a couple of small bugs. * [NOD-289] Fixed a TxID/Hash mixup. * [NOD-289] Added block/tx mass to getBlockVerboseResponse. * [NOD-289] Renamed blockLoop.go to sync.go. Added comments. * [NOD-289] Deleted apiserver README. * [NOD-289] Fixed golint errors. * [NOD-289] Renamed findMostRecentBlockHash to findHashOfBluestBlock. * [NOD-289] Fixed style in syncBlocks and fixed a comment. * [NOD-289] Copied NewErrorFromDBErrors over from NOD-324. * [NOD-289] Created a couple of utils to make error handling with gorm slightly less painful. * [NOD-289] Added error handling for database calls. * [NOD-289] Fixed some more style/comments. * [NOD-289] Fixed comments. * [NOD-289] Renamed TransactionInput.TransactionOutput to TransactionInput.PreviousTransactionOutput. * [NOD-289] Added a commends about pagination in getBlocks and getChainFromBlock. * [NOD-289] Removed the coinbase field from Vin. * [NOD-289] Deferred handling chainChangedMsgs until we have the appropriate data. * [NOD-289] Optimized queries in updateRemovedChainHashes and updateAddedChainBlocks. * [NOD-289] Optimized queries in insertBlockParents. * [NOD-289] Optimized queries in insertTransactionInput. * [NOD-289] Split Where calls to separate lines. * [NOD-289] Fixed merge errors. * [NOD-289] Exited early from insertBlockParents if we're the genesis block. * [NOD-289] Improved nextChainChangedChan mechanism. * [NOD-289] Fixed the above sync mechanism a bit. * [NOD-289] Renamed IsDBRecordNotFoundError to HasDBRecordNotFoundError and IsDBError to HasDBError. * [NOD-289] Replaced old error handling for db errors with the lovely new stuff. * [NOD-289] Exited early if we already inserted a block. This saves us checking if a record already exists for some record types. * [NOD-289] Decoupled syncBlocks from syncSelectedParentChain. * [NOD-289] Made a comment more explicit. * [NOD-289] Extracted net resolution to a separate function. * [NOD-289] Extracted syncing to a separate function. * [NOD-289] Fixed a comment. * [NOD-289] Fixed merge erros. * [NOD-289] Fixed a couple of bugs. * [NOD-289] Fixed another bug. * [NOD-289] Extracted ChainChangedMsg conversion to a separate function. * [NOD-289] Optimized queries in canHandleChainChangedMsg. * [NOD-289] Moved the sync function closer to its call site. * [NOD-289] Renamed HasDBRecordNotFoundError to IsDBRecordNotFoundError. * [NOD-289] Used count instead of first. * [NOD-289] Renamed address to hexAddress. |
||
|
stasatdaglabs
|
b07a118431 |
[NOD-292] When writing block to database - also create record in block index (#376)
* [NOD-292] In accept.go, made dbStoreBlock and flushToDB occur within the same transaction. * [NOD-292] Implemented processing blocks that were not validated on BTCD start. * [NOD-292] Fixed processing logic on init. Added a test for it. * [NOD-292] Fixed some comments. * [NOD-292] Made unlocks deferred in a couple of places. * [NOD-292] Made unprocessed block reprocess via ProcessBlock rather than maybeAcceptBlock. * [NOD-292] Fixed grammar in comment. Added an explanation to TestAcceptingInInit. * [NOD-292] Split flushToDB into two versions. * [NOD-292] Fixed a bad assignment. * [NOD-292] Fixed bad spacing. |
||
|
Svarog
|
96842353de | [NOD-250] Calculate UTXOCommitment when loading DAG + add UTXOCommitment to getBlockDagInfo (#347) | ||
|
stasatdaglabs
|
7400eabc6d | [NOD-233] Fixed iteration order when iterating over the blockIndex bucket. (#338) | ||
|
Ori Newman
|
d6297a3192 |
[NOD-225] Finalize nodes below finality point (#335)
* [NOD-225] Finalize nodes below finality point * [NOD-225] finalizeNodesBelowFinalityPoint only if dag.lastFinalityPoint is changed * [NOD-225] change comment in validateParents * [NOD-225] add string to ErrInvalidParentsRelation error * [NOD-225] Change comment in validateParents * [NOD-225] Change comment in validateParents * [NOD-225] change comment in validateParents * [NOD-225] Delete diff data from db directly from finalizeNodesBelowFinalityPoint * [NOD-225] Refactor updateFinalityPoint |
||
|
Ori Newman
|
263737b3fb |
[NOD-196] move coinbase scriptpukey to payload (#330)
* [NOD-196] move coinbase scriptpukey to payload (no tests) (#311) * [NOD-196] Move coinbase scriptPubKey to payload * [NOD-196] Rename SubnetworkID.IsFull to SubnetworkID.IsBuiltIn * [NOD-196] Fix comments * [NOD-196] Add block subsidy to fee transaction * [NOD-196] Fix comments * [NOD-217] Merge coinbase and fee transaction (#328) * [NOD-196] Fix tests * [NOD-196] Fix tests * [NOD-217] Add error to getBluesFeeData * [NOD-217] Merge Coinbase and fee transaction * [NOD-217] Format project * [NOD-217] Remove OpTrue default for mining.NewBlockTemplate * [NOD-196] Format project * [NOD-217] Add missing space before comment * [NOD-196] Change MaxCoinbasePayloadLen to 150 |
||
|
stasatdaglabs
|
ffd886498a |
[NOD-208] Make block reward maturity use the same mechanism as confirmations (#327)
* [NOD-208] Added blockBlueScore to UTXOEntry. * [NOD-208] Added blueBlockScore to NewUTXOEntry. * [NOD-208] Fixed compilation errors in policy, utxoset, and dag tests. * [NOD-208] Changed validateBlockRewardMaturity and CheckTransactionInputsAndCalulateFee to use blueScore. * [NOD-208] Changed CalcBlockSubsidy to use blueScore. * [NOD-208] Changed SequenceLockActive to use blueScore. * [NOD-208] Removed ExtractCoinbaseHeight. * [NOD-208] Removed reference to block height in ensureNoDuplicateTx. * [NOD-208] Changed IsFinalizedTransaction to use blueScore. * [NOD-208] Fixed merge errors. * [NOD-208] Made UTXOEntry serialization use blueScore. * [NOD-208] Changed CalcPriority and calcInputValueAge to use blueScore. * [NOD-208] Changed calcSequenceLock to use blueScore. * [NOD-208] Removed blockChainHeight from UTXOEntry. * [NOD-208] Fixed compilation errors in feeEstimator. Fixed a bug in the test pool hardness. * [NOD-208] Fixed oldestChainBlockWithBlueScoreGreaterThan not handling an extreme case. * [NOD-208] Fixed TestDiffFromTx. * [NOD-208] Got rid of priority and support of free transactions. * [NOD-208] Fixed TestProcessTransaction. * [NOD-208] Fixed TestTxFeePrioHeap. * [NOD-208] Fixed TestAddrIndex and TestFeeEstimatorCfg. * [NOD-208] Removed unused rateLimit parameter from ProcessTransaction. * [NOD-208] Fixed tests that rely on CreateTxChain. * [NOD-208] Fixed tests that rely on CreateSignedTxForSubnetwork. * [NOD-208] Fixed TestFetchTransaction. * [NOD-208] Fixed TestHandleNewBlock. Fixed HandleNewBlock erroneously processing fee transactions. * [NOD-208] Fixed TestTxIndexConnectBlock. * [NOD-208] Removed the use of Height() from the fee estimator. * [NOD-208] Removed unused methods from rpcwebsocket.go. * [NOD-208] Removed Height from util.Block. * [NOD-208] Removed ErrForkTooOld. It doesn't make sense in a DAG. * [NOD-208] Made blockHeap use blueScore instead of height. * [NOD-208] Removed fee estimator. * [NOD-208] Removed DAG.Height. * [NOD-208] Made TestAncestorErrors test chainHeight instead of height. * [NOD-208] Fixed a couple of comments that were still speaking about block height. * [NOD-208] Replaced all uses of HighestTipHash with SelectedTipHash. * [NOD-208] Remove blockNode highest and some remaining erroneous uses of height. * [NOD-208] Fixed a couple of comments. Fixed outPoint -> outpoint merge error. * [NOD-208] Fixed a couple more comments. * [NOD-208] Used calcMinRequiredTxRelayFee instead of DefaultMinRelayTxFee for mempool tests. * [NOD-208] Renamed mempool Config BestHeight to DAGChainHeight. * [NOD-208] Fixed a bug in oldestChainBlockWithBlueScoreGreaterThan. Made calcSequenceLock use the node's selected parent chain rather than the virtual block's. * [NOD-208] Removed chainHeight from blockNode String(). Renamed checkpointsByHeight to checkpointsByChainHeight and prevCheckpointHeight to prevCheckpointChainHeight. Removed reference to chainHeight in blockIndexKey. Fixed comments in dagio.go. * [NOD-208] Removed indexers/blocklogger.go, as no one was using it. * [NOD-208] Made blocklogger.go log blueScore instead of height. * [NOD-208] Fixed typo. * [NOD-208] Fixed comments, did minor renaming. * [NOD-208] Made a "common sense" wrapper around sort.Search. * [NOD-208] Fixed comment in SearchSlice. |
||
|
Svarog
|
396842ae40 |
[NOD-207] Rename any place that says 'OutPoint' to 'Outpoint' (#323)
* [NOD-207] Rename any place that says 'OutPoint' to 'Outpoint' * [NOD-207] Fix any place that says output point |
||
|
Svarog
|
072c753323 |
[NOD-216] Revert implicit fee transaction (#322)
* Revert "[NOD-214] Remove Fee transaction from addrindex (#321)" This reverts commit e4b2d869d49471021c56cb27cbc3001134b94dfd. * Revert "[NOD-195] Make fee tx implicit (#315)" This reverts commit ccca580a4b3ff8a44138041c0ce3bc0ce5847a4d. |
||
|
Svarog
|
ccca580a4b |
[NOD-195] Make fee tx implicit (#315)
* [NOD-195] Made fee tx implicit * [NOD-195] Removed redundant checks for fee transactions * [NOD-195] Add fee tx data into acceptence data and fee data * [NOD-195] Fix some tests * [NOD-195] Update Block100000 with new data * [NOD-195] Fixed remaining tests * [NOD-195] Save and load feeTx to/from database * [NOD-195] Remove DisconnectBlock methods from indexers, since they are not used anywhere * [NOD-195] Add fee tx to addrindex * [NOD-195] Don't populate inputs for fee transactions * [NOD-195] Delete feeTxBucket in removeDAGState * [NOD-195] Got rid of util.FeeTRansactionIndex |
||
|
Ori Newman
|
901bde1fd4 |
[NOD-202] undo createDAGState if blockdag new fails (#318)
* [NOD-204] Add UTXOCommitment to GetBlockTemplateResult * [NOD-204] Add UTXOCommitment to GetBlockTemplateResult * [NOD-206] Avoid leaking blocks from previous miner when switching miners * [NOD-202] Undo createDAGState if blockdag.New fails * [NOD-202] Fix gofmt errors |
||
|
Ori Newman
|
aa51b5f071 |
[NOD-179] Added ECMH-Multiset to all UTXO structs (#304)
* [NOD-172] Port EMCH from bchd * [NOD-172] Fix hdkeychain.TestErrors and add btcec.TestRecoverCompact * [NOD-172] Make ECMH immutable * [NOD-172] Fix gofmt errors * [NOD-172] Add TestMultiset_NewMultisetFromDataSlice and fix Point to be immutable * [NOD-172] Fix gofmt errors * [NOD-172] Add test for checking that the Union of a multiset and its inverse is zero * [NOD-179] Add ECMH Point to all UTXO-structs * [NOD-179] Fix utxo set tests * [NOD-179] Fix mempool tests * [NOD-179] Remove RemoveTxOuts * [NOD-179] Move serializeBlockUTXODiffData to the top of the file * [NOD-179] Fix serializeBlockUTXODiffData comment format * [NOD-179] Fix AddTx comment and name return values |
||
|
stasatdaglabs
|
b1d3ca0206 |
[NOD-177] Remove idMerkleRoot (#299)
* [NOD-177] Removed references to idMerkleRoot. * [NOD-177] Generated new genesis hashes. * [NOD-177] Generated new blk_ blocks. * [NOD-177] Fixed TestHaveBlock. * [NOD-177] Fixed The rest of the tests. * [NOD-177] Fixed a couple of comments and a duplicate test. * [NOD-177] Fixed blocks1-256.bz2. |
||
|
Svarog
|
ca0619bbcf |
[NOD-176] Moved daghash from dagconfig to util (#298)
* [NOD-176] Moved daghash from dagconfig to util * [NOD-176] Fixed order of includes with gofmt |
||
|
Ori Newman
|
d7a2ab52a1 | [NOD-173] Add UTXO commitment to block header (#297) | ||
|
Evgeny Khirin
|
08d94c7a47 | [NOD-163] Added BlockHeader.AcceptedIDMerkleRoot (#293) | ||
|
Evgeny Khirin
|
a73f218402 |
[NOD-150] Removed blockNode.workSum (#286)
* [NOD-150] Removed blockNode.workSum * [NOD-150] Fixed comment |
||
|
Svarog
|
33036278ac |
[NOD-144] Use chainHeight in SelectedAncestor, and update all logic that uses it (#281)
* [NOD-144] Use chainHeight in SelectedAncestor, and update all logic that uses it * [NOD-144] Moved UnminedHeight to blockdag, and updated all references |
||
|
Ori Newman
|
1362fc45e0 | [NOD-148] Delete block index (#275) |