mirror of
https://github.com/kaspanet/kaspad.git
synced 2025-09-13 21:10:12 +00:00
31 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
stasatdaglabs
|
e0f587f599
|
[NOD-877] Separate UTXO header code to two fields in serialization: blue score and packed flags (#725)
* [NOD-877] In UTXOEntry serialization, extract packedFlags out to a separate Uint8. * [NOD-877] Generate new test blocks. * [NOD-877] Fix TestIsKnownBlock. * [NOD-877] Fix TestBlueBlockWindow. * [NOD-877] Fix TestUTXOSerialization and TestGHOSTDAG. * [NOD-877] Fix TestVirtualBlock. |
||
|
stasatdaglabs
|
f8b18e09d6
|
[NOD-805] Redesign the database (#685)
* [NOD-828] Reimplement FFLDB (#663) * [NOD-828] Create the database2 package that will some day replace the database package. * [NOD-828] Implement a "bucket" key mechanism. * [NOD-828] Move bucket.go into the ffldb2 package. * [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway. * [NOD-828] Copy over + fixup flat file structs from the old ffldb. * [NOD-828] Implement flatFilePath. * [NOD-828] Implement flatFileStore.write(). * [NOD-828] Implement flatFileStore.read(). * [NOD-828] Implement flatFileStore.rollback(). * [NOD-828] Sync the file to disk at the end of write(). * [NOD-828] Extract crc32ByteOrder to a separate variable. * [NOD-828] Add a sanity test. * [NOD-828] Remove context-unrelated methods from the Database interface. * [NOD-828] Create an ffldb object. Simply work against a context. * [NOD-828] Open the new database on start. * [NOD-828] Create the leveldb package. * [NOD-828] Implement opening/closing leveldb. * [NOD-828] Implement get/put out of/into leveldb. * [NOD-828] Implement transactions and make them implement a generic database interface. * [NOD-828] Write sanity tests for leveldb with and without transactions. * [NOD-828] Add another case to the transaction sanity test. * [NOD-828] Implement AppendBlock/RetrieveBlock. * [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package. * [NOD-828] Add RollbackFlatData to the database interface. * [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket. * [NOD-828] Implement StoreBlock. * [NOD-828] Implement FetchBlock. * [NOD-828] Implement HasBlock. * [NOD-828] Write a sanity test for block insertion. * [NOD-828] Implement CurrentFlatDataLocation. * [NOD-828] Implement storing the current block location. * [NOD-828] Implement initializing/syncing the flat file block store and the "metadata". * [NOD-828] Add InitBlockStore to TestBlockStoreSanity. * [NOD-828] Fix rename errors. * [NOD-828] Fix lint errors in the root database package. * [NOD-828] Fix lint errors in the ffldb.go. * [NOD-828] Fix lint errors in the flatfile/db.go. * [NOD-828] Rename packages in such a way to make the linter happy. * [NOD-828] Finish satisfying the linter. * [NOD-828] Fix doc.go. * [NOD-828] Fix comments in block.go. * [NOD-828] Move dbaccess out of the database package. * [NOD-828] Move opening/closing the database to dbaccess. * [NOD-828] Move the Database interface to the root database package, since it's meant to be generic. * [NOD-828] Make ffldb generic to later support additional databases. * [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported. * [NOD-828] Fix comments. * [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore. * [NOD-828] Make buckets nicer to use. * [NOD-828] Implement cursors that iterate over some bucket. * [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb. * [NOD-828] Write a test making sure that flat file repair works. * [NOD-828] Properly close the database in TestRepairFlatFiles. * [NOD-828] Add a comment warning against putting and getting the same data within the same transaction. * [NOD-828] Fix the flatFilesBucket description. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Fix the ffldb description. * [NOD-828] Rename Database to DataAccessor and rename Handle to Database. * [NOD-828] Make Get return nil if the value doesn't exist. * [NOD-828] Attempt to close leveldb even if closing ffdb failed. * [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket. * [NOD-828] Fix not updating the store location in ffldb transactions. * [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found. * [NOD-828] Update the README and doc.go. * [NOD-828] Simplify Bucket.Path(). * [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly. * [NOD-828] Combine two lines into one. * [NOD-828] Combine another two lines into one. * [NOD-828] Move a misplaced comment. * [NOD-828] Use Wrapf instead of Errorf where appropriate. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Fix Wrapf calls. * [NOD-828] Fix comments. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Add a comment explaining the use of batches and snapshots. * [NOD-828] Implement RollbackUnlessClosed(). * [NOD-828] Return both errors in StoreBlock rollback. * [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor. * [NOD-828] Make bucket a type alias instead of a struct. * [NOD-828] Fix a typo. * [NOD-828] Use copy instead of append in Bucket. * [NOD-828] Extract flatFileLocationSerializedSize to a const. * [NOD-828] Debugf -> Warnf in rollback.go. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Remove data length from flat file data format. * [NOD-828] Rearrange TestLevelDBTransactionSanity a bit. * [NOD-828] Add stack traces to all errors that come out of library functions. * [NOD-828] Return errors from rollback(). * [NOD-828] Remove an irrelevant comment. * [NOD-828] Remove redundant whitespace. * [NOD-828] Handle nil in FetchBlock. * [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct. * Revert "[NOD-828] Make bucket a type alias instead of a struct." This reverts commit 1fd39652 * [NOD-828] Fix revert errors. * Revert "[NOD-828] Remove data length from flat file data format." This reverts commit ef408e32 * [NOD-862] Move Cursor() into the DataAccessor interface. * [NOD-828] Add Delete to DataAccessor. * [NOD-828] Fix a couple of places that erroneously referenced blocks. * [NOD-828] Add a comment on top of flatFileLocationSerializedSize. * [NOD-828] Add Seek to Cursor. * [NOD-828] Add First to Cursor. * [NOD-828] Rename db to accessor in Context. * [NOD-828] Make Get/Fetch calls return a boolean to indicate whether the requested item was found. * [NOD-828] Name the output parameters of all Get functions. * [NOD-828] Make RetrieveFromStore return whether the data was found. * [NOD-887] Add a couple of QoL features to Cursor (#674) * [NOD-887] Changed First to not return an error. * [NOD-887] Fix merge error. * [NOD-887] Make Cursor.Key not return the entire key path. * [NOD-888] Add RollbackUnlessClosed to Context (#676) * [NOD-888] Add RollbackUnlessClosed to Context. * [NOD-888] Fix copy+paste error. * [NOD-889] Instead of returning a boolean for not-found, return an error (#677) * [NOD-889] Instead of returning a boolean for not-found, return an error. * [NOD-889] Wrapped ErrNotFound for Get calls with nicer error messages. * [NOD-889] Fix format. * [NOD-889] Fix double space in a comment. * [NOD-889] Add IsNotFoundError to dbaccess. * [NOD-862] Replace calls to Tx.StoreBlock, Tx.HasBlock, Tx.FetchBlock with appropriate calls in dbaccess (#672) * [NOD-828] Create the database2 package that will some day replace the database package. * [NOD-828] Implement a "bucket" key mechanism. * [NOD-828] Move bucket.go into the ffldb2 package. * [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway. * [NOD-828] Copy over + fixup flat file structs from the old ffldb. * [NOD-828] Implement flatFilePath. * [NOD-828] Implement flatFileStore.write(). * [NOD-828] Implement flatFileStore.read(). * [NOD-828] Implement flatFileStore.rollback(). * [NOD-828] Sync the file to disk at the end of write(). * [NOD-828] Extract crc32ByteOrder to a separate variable. * [NOD-828] Add a sanity test. * [NOD-828] Remove context-unrelated methods from the Database interface. * [NOD-828] Create an ffldb object. Simply work against a context. * [NOD-828] Open the new database on start. * [NOD-828] Create the leveldb package. * [NOD-828] Implement opening/closing leveldb. * [NOD-828] Implement get/put out of/into leveldb. * [NOD-828] Implement transactions and make them implement a generic database interface. * [NOD-828] Write sanity tests for leveldb with and without transactions. * [NOD-828] Add another case to the transaction sanity test. * [NOD-828] Implement AppendBlock/RetrieveBlock. * [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package. * [NOD-828] Add RollbackFlatData to the database interface. * [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket. * [NOD-828] Implement StoreBlock. * [NOD-828] Implement FetchBlock. * [NOD-828] Implement HasBlock. * [NOD-828] Write a sanity test for block insertion. * [NOD-828] Implement CurrentFlatDataLocation. * [NOD-828] Implement storing the current block location. * [NOD-828] Implement initializing/syncing the flat file block store and the "metadata". * [NOD-828] Add InitBlockStore to TestBlockStoreSanity. * [NOD-828] Fix rename errors. * [NOD-828] Fix lint errors in the root database package. * [NOD-828] Fix lint errors in the ffldb.go. * [NOD-828] Fix lint errors in the flatfile/db.go. * [NOD-828] Rename packages in such a way to make the linter happy. * [NOD-828] Finish satisfying the linter. * [NOD-828] Fix doc.go. * [NOD-828] Fix comments in block.go. * [NOD-828] Move dbaccess out of the database package. * [NOD-828] Move opening/closing the database to dbaccess. * [NOD-828] Move the Database interface to the root database package, since it's meant to be generic. * [NOD-828] Make ffldb generic to later support additional databases. * [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported. * [NOD-828] Fix comments. * [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore. * [NOD-828] Make buckets nicer to use. * [NOD-828] Implement cursors that iterate over some bucket. * [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb. * [NOD-828] Write a test making sure that flat file repair works. * [NOD-828] Properly close the database in TestRepairFlatFiles. * [NOD-828] Add a comment warning against putting and getting the same data within the same transaction. * [NOD-862] Use dbaccess.HasBlock instead of Tx.HasBlock in initDAGState. * [NOD-862] Use dbaccess.StoreBlock instead of dbStoreBlock. * [NOD-862] Use dbaccess.FetchBlock instead of various block fetching mechanisms. * [NOD-828] Fix the flatFilesBucket description. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Fix the ffldb description. * [NOD-828] Rename Database to DataAccessor and rename Handle to Database. * [NOD-828] Make Get return nil if the value doesn't exist. * [NOD-828] Attempt to close leveldb even if closing ffdb failed. * [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket. * [NOD-828] Fix not updating the store location in ffldb transactions. * [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found. * [NOD-828] Update the README and doc.go. * [NOD-828] Simplify Bucket.Path(). * [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly. * [NOD-828] Combine two lines into one. * [NOD-828] Combine another two lines into one. * [NOD-828] Move a misplaced comment. * [NOD-828] Use Wrapf instead of Errorf where appropriate. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Fix Wrapf calls. * [NOD-828] Fix comments. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Add a comment explaining the use of batches and snapshots. * [NOD-828] Implement RollbackUnlessClosed(). * [NOD-828] Return both errors in StoreBlock rollback. * [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor. * [NOD-828] Make bucket a type alias instead of a struct. * [NOD-828] Fix a typo. * [NOD-828] Use copy instead of append in Bucket. * [NOD-828] Extract flatFileLocationSerializedSize to a const. * [NOD-828] Debugf -> Warnf in rollback.go. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Remove data length from flat file data format. * [NOD-828] Rearrange TestLevelDBTransactionSanity a bit. * [NOD-828] Add stack traces to all errors that come out of library functions. * [NOD-828] Return errors from rollback(). * [NOD-828] Remove an irrelevant comment. * [NOD-828] Remove redundant whitespace. * [NOD-828] Handle nil in FetchBlock. * [NOD-828] Implement a dbaccess.BlockNode struct. * [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct. * [NOD-828] Implement toDBBlockNode and fromDBBlockNode. * Revert "[NOD-828] Make bucket a type alias instead of a struct." This reverts commit 1fd39652 * [NOD-828] Fix revert errors. * Revert "[NOD-828] Remove data length from flat file data format." This reverts commit ef408e32 * [NOD-862] Implement storing index blocks. * [NOD-862] Use database transactions where appropriate. * [NOD-862] Fix tests failing on DAGSetup. * [NOD-862] Fix bad make call. * [NOD-862] Fix remaining database opening problems in tests. * [NOD-862] Move Cursor() into the DataAccessor interface. * [NOD-862] Move Cursor() into the DataAccessor interface. * [NOD-862] Iterate over the new block index in dagio. * [NOD-862] Fix block index key. * [NOD-828] Add Delete to DataAccessor. * [NOD-862] Pass byte slices to dbaccess instead of objects. * [NOD-862] Fix errors. * [NOD-862] Fix maybeAcceptBlock not checking block existence. * [NOD-862] Fix TestAcceptanceIndexRecover. * [NOD-862] Add comments to StoreBlockIndex and BlockIndexCursor. * [NOD-828] Fix a couple of places that erroneously referenced blocks. * [NOD-828] Add a comment on top of flatFileLocationSerializedSize. * [NOD-828] Add Seek to Cursor. * [NOD-828] Add First to Cursor. * [NOD-828] Rename db to accessor in Context. * [NOD-828] Make Get/Fetch calls return a boolean to indicate whether the requested item was found. * [NOD-828] Name the output parameters of all Get functions. * [NOD-828] Make RetrieveFromStore return whether the data was found. * [NOD-862] Fix merge errors. * [NOD-862] Fix DAGSetup using bad temp directories. * [NOD-862] Fix TestProcessDelayedBlocks not closing the database properly. * [NOD-862] Fix merge errors. * [NOD-862] Merge flushToDBWithContext and flushToDB. * [NOD-862] Remove TODO. * [NOD-862] Add prefix to the temp dir in DAGSetup. * [NOD-862] Bring back dbFetchBlockByHash. * [NOD-862] Use BlockDAG.BlockByHash in p2p and rpc. * [NOD-862] Use daghash.Hash in dbaccess. * [NOD-862] Add defer to RollbackUnlessClosed after NewTx(). * [NOD-862] Extract dbStoreBlock to a separate function. * [NOD-862] Fix grammar in comment. * [NOD-862] Fix merge errors. * [NOD-867] Migrate database logic in blockdag/dagio.go to dbaccess (#675) * [NOD-828] Create the database2 package that will some day replace the database package. * [NOD-828] Implement a "bucket" key mechanism. * [NOD-828] Move bucket.go into the ffldb2 package. * [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway. * [NOD-828] Copy over + fixup flat file structs from the old ffldb. * [NOD-828] Implement flatFilePath. * [NOD-828] Implement flatFileStore.write(). * [NOD-828] Implement flatFileStore.read(). * [NOD-828] Implement flatFileStore.rollback(). * [NOD-828] Sync the file to disk at the end of write(). * [NOD-828] Extract crc32ByteOrder to a separate variable. * [NOD-828] Add a sanity test. * [NOD-828] Remove context-unrelated methods from the Database interface. * [NOD-828] Create an ffldb object. Simply work against a context. * [NOD-828] Open the new database on start. * [NOD-828] Create the leveldb package. * [NOD-828] Implement opening/closing leveldb. * [NOD-828] Implement get/put out of/into leveldb. * [NOD-828] Implement transactions and make them implement a generic database interface. * [NOD-828] Write sanity tests for leveldb with and without transactions. * [NOD-828] Add another case to the transaction sanity test. * [NOD-828] Implement AppendBlock/RetrieveBlock. * [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package. * [NOD-828] Add RollbackFlatData to the database interface. * [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket. * [NOD-828] Implement StoreBlock. * [NOD-828] Implement FetchBlock. * [NOD-828] Implement HasBlock. * [NOD-828] Write a sanity test for block insertion. * [NOD-828] Implement CurrentFlatDataLocation. * [NOD-828] Implement storing the current block location. * [NOD-828] Implement initializing/syncing the flat file block store and the "metadata". * [NOD-828] Add InitBlockStore to TestBlockStoreSanity. * [NOD-828] Fix rename errors. * [NOD-828] Fix lint errors in the root database package. * [NOD-828] Fix lint errors in the ffldb.go. * [NOD-828] Fix lint errors in the flatfile/db.go. * [NOD-828] Rename packages in such a way to make the linter happy. * [NOD-828] Finish satisfying the linter. * [NOD-828] Fix doc.go. * [NOD-828] Fix comments in block.go. * [NOD-828] Move dbaccess out of the database package. * [NOD-828] Move opening/closing the database to dbaccess. * [NOD-828] Move the Database interface to the root database package, since it's meant to be generic. * [NOD-828] Make ffldb generic to later support additional databases. * [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported. * [NOD-828] Fix comments. * [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore. * [NOD-828] Make buckets nicer to use. * [NOD-828] Implement cursors that iterate over some bucket. * [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb. * [NOD-828] Write a test making sure that flat file repair works. * [NOD-828] Properly close the database in TestRepairFlatFiles. * [NOD-828] Add a comment warning against putting and getting the same data within the same transaction. * [NOD-862] Use dbaccess.HasBlock instead of Tx.HasBlock in initDAGState. * [NOD-862] Use dbaccess.StoreBlock instead of dbStoreBlock. * [NOD-862] Use dbaccess.FetchBlock instead of various block fetching mechanisms. * [NOD-828] Fix the flatFilesBucket description. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Fix the ffldb description. * [NOD-828] Rename Database to DataAccessor and rename Handle to Database. * [NOD-828] Make Get return nil if the value doesn't exist. * [NOD-828] Attempt to close leveldb even if closing ffdb failed. * [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket. * [NOD-828] Fix not updating the store location in ffldb transactions. * [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found. * [NOD-828] Update the README and doc.go. * [NOD-828] Simplify Bucket.Path(). * [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly. * [NOD-828] Combine two lines into one. * [NOD-828] Combine another two lines into one. * [NOD-828] Move a misplaced comment. * [NOD-828] Use Wrapf instead of Errorf where appropriate. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Fix Wrapf calls. * [NOD-828] Fix comments. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Add a comment explaining the use of batches and snapshots. * [NOD-828] Implement RollbackUnlessClosed(). * [NOD-828] Return both errors in StoreBlock rollback. * [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor. * [NOD-828] Make bucket a type alias instead of a struct. * [NOD-828] Fix a typo. * [NOD-828] Use copy instead of append in Bucket. * [NOD-828] Extract flatFileLocationSerializedSize to a const. * [NOD-828] Debugf -> Warnf in rollback.go. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Remove data length from flat file data format. * [NOD-828] Rearrange TestLevelDBTransactionSanity a bit. * [NOD-828] Add stack traces to all errors that come out of library functions. * [NOD-828] Return errors from rollback(). * [NOD-828] Remove an irrelevant comment. * [NOD-828] Remove redundant whitespace. * [NOD-828] Handle nil in FetchBlock. * [NOD-828] Implement a dbaccess.BlockNode struct. * [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct. * [NOD-828] Implement toDBBlockNode and fromDBBlockNode. * Revert "[NOD-828] Make bucket a type alias instead of a struct." This reverts commit 1fd39652 * [NOD-828] Fix revert errors. * Revert "[NOD-828] Remove data length from flat file data format." This reverts commit ef408e32 * [NOD-862] Implement storing index blocks. * [NOD-862] Use database transactions where appropriate. * [NOD-862] Fix tests failing on DAGSetup. * [NOD-862] Fix bad make call. * [NOD-862] Fix remaining database opening problems in tests. * [NOD-862] Move Cursor() into the DataAccessor interface. * [NOD-862] Move Cursor() into the DataAccessor interface. * [NOD-862] Iterate over the new block index in dagio. * [NOD-862] Fix block index key. * [NOD-828] Add Delete to DataAccessor. * [NOD-862] Pass byte slices to dbaccess instead of objects. * [NOD-862] Fix errors. * [NOD-862] Fix maybeAcceptBlock not checking block existence. * [NOD-862] Fix TestAcceptanceIndexRecover. * [NOD-862] Add comments to StoreBlockIndex and BlockIndexCursor. * [NOD-828] Fix a couple of places that erroneously referenced blocks. * [NOD-828] Add a comment on top of flatFileLocationSerializedSize. * [NOD-828] Add Seek to Cursor. * [NOD-828] Add First to Cursor. * [NOD-828] Rename db to accessor in Context. * [NOD-828] Make Get/Fetch calls return a boolean to indicate whether the requested item was found. * [NOD-828] Name the output parameters of all Get functions. * [NOD-828] Make RetrieveFromStore return whether the data was found. * [NOD-862] Fix merge errors. * [NOD-862] Fix DAGSetup using bad temp directories. * [NOD-862] Fix TestProcessDelayedBlocks not closing the database properly. * [NOD-867] Remove blockIndexBucket from dagio. * [NOD-867] Fix wrong key in StoreIndexBucket. * [NOD-867] Migrate DAG state to dbaccess. * [NOD-867] Remove utxoSetVersionKeyName. * [NOD-862] Fix merge errors. * [NOD-867] Move localSubnetworkID into dagState. * [NOD-867] Fix a comment. * [NOD-867] Remove an unused function. * [NOD-867] Migrate the database's UTXO set to dbaccess. * [NOD-867] Add missing error check. * [NOD-867] Changed First to not return an error. * [NOD-867] Make Cursor.Key not return the entire key path. * [NOD-887] Fix the comment above BlockIndexCursorFrom. * [NOD-862] Merge flushToDBWithContext and flushToDB. * [NOD-862] Remove TODO. * [NOD-862] Add prefix to the temp dir in DAGSetup. * [NOD-862] Bring back dbFetchBlockByHash. * [NOD-862] Use BlockDAG.BlockByHash in p2p and rpc. * [NOD-862] Use daghash.Hash in dbaccess. * [NOD-862] Add defer to RollbackUnlessClosed after NewTx(). * [NOD-862] Extract dbStoreBlock to a separate function. * [NOD-867] Remove TODOs. * [NOD-867] Fix merge errors. * [NOD-867] Fix comments and errors. * [NOD-867] Unexport blockIndexKey. * [NOD-867] Fix merge errors. * [NOD-867] Move a misplaced comment. * [NOD-867] Fix an error message. * [NOD-867] Remove preallocation in initDAGState. * [NOD-866] Migrate database logic in blockdag/indexers package to dbaccess (#682) * [NOD-865] Delete blockidhash.go. * [NOD-865] Remove a lot of no-longer relevant logic from indexers. * [NOD-865] Pass TxContext to ConnectBlock. * [NOD-865] Migrate the acceptance index to dbaccess. * [NOD-865] Fix a block not being sent to ConnectBlock. * [NOD-865] Pass the block's hash instead of the whole block. * [NOD-865] Add forgotten Commit call. * [NOD-865] Add comments. * [NOD-866] Fix a comment. * [NOD-866] Fix a comment. * [NOD-866] Remove pointless indirection in acceptanceindex. * [NOD-866] Fix comment over ForEachHash. * [NOD-866] Rename ClearAcceptanceIndex to DropAcceptanceIndex. * [NOD-866] Explain collecting keys before deleting them. * [NOD-865] Move misc db logic to db access (#681) * [NOD-828] Create the database2 package that will some day replace the database package. * [NOD-828] Implement a "bucket" key mechanism. * [NOD-828] Move bucket.go into the ffldb2 package. * [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway. * [NOD-828] Copy over + fixup flat file structs from the old ffldb. * [NOD-828] Implement flatFilePath. * [NOD-828] Implement flatFileStore.write(). * [NOD-828] Implement flatFileStore.read(). * [NOD-828] Implement flatFileStore.rollback(). * [NOD-828] Sync the file to disk at the end of write(). * [NOD-828] Extract crc32ByteOrder to a separate variable. * [NOD-828] Add a sanity test. * [NOD-828] Remove context-unrelated methods from the Database interface. * [NOD-828] Create an ffldb object. Simply work against a context. * [NOD-828] Open the new database on start. * [NOD-828] Create the leveldb package. * [NOD-828] Implement opening/closing leveldb. * [NOD-828] Implement get/put out of/into leveldb. * [NOD-828] Implement transactions and make them implement a generic database interface. * [NOD-828] Write sanity tests for leveldb with and without transactions. * [NOD-828] Add another case to the transaction sanity test. * [NOD-828] Implement AppendBlock/RetrieveBlock. * [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package. * [NOD-828] Add RollbackFlatData to the database interface. * [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket. * [NOD-828] Implement StoreBlock. * [NOD-828] Implement FetchBlock. * [NOD-828] Implement HasBlock. * [NOD-828] Write a sanity test for block insertion. * [NOD-828] Implement CurrentFlatDataLocation. * [NOD-828] Implement storing the current block location. * [NOD-828] Implement initializing/syncing the flat file block store and the "metadata". * [NOD-828] Add InitBlockStore to TestBlockStoreSanity. * [NOD-828] Fix rename errors. * [NOD-828] Fix lint errors in the root database package. * [NOD-828] Fix lint errors in the ffldb.go. * [NOD-828] Fix lint errors in the flatfile/db.go. * [NOD-828] Rename packages in such a way to make the linter happy. * [NOD-828] Finish satisfying the linter. * [NOD-828] Fix doc.go. * [NOD-828] Fix comments in block.go. * [NOD-828] Move dbaccess out of the database package. * [NOD-828] Move opening/closing the database to dbaccess. * [NOD-828] Move the Database interface to the root database package, since it's meant to be generic. * [NOD-828] Make ffldb generic to later support additional databases. * [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported. * [NOD-828] Fix comments. * [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore. * [NOD-828] Make buckets nicer to use. * [NOD-828] Implement cursors that iterate over some bucket. * [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb. * [NOD-828] Write a test making sure that flat file repair works. * [NOD-828] Properly close the database in TestRepairFlatFiles. * [NOD-828] Add a comment warning against putting and getting the same data within the same transaction. * [NOD-828] Fix the flatFilesBucket description. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Fix the ffldb description. * [NOD-828] Rename Database to DataAccessor and rename Handle to Database. * [NOD-828] Make Get return nil if the value doesn't exist. * [NOD-828] Attempt to close leveldb even if closing ffdb failed. * [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket. * [NOD-828] Fix not updating the store location in ffldb transactions. * [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found. * [NOD-828] Update the README and doc.go. * [NOD-828] Simplify Bucket.Path(). * [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly. * [NOD-828] Combine two lines into one. * [NOD-828] Combine another two lines into one. * [NOD-828] Move a misplaced comment. * [NOD-828] Use Wrapf instead of Errorf where appropriate. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Fix Wrapf calls. * [NOD-828] Fix comments. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Add a comment explaining the use of batches and snapshots. * [NOD-828] Implement RollbackUnlessClosed(). * [NOD-828] Return both errors in StoreBlock rollback. * [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor. * [NOD-828] Make bucket a type alias instead of a struct. * [NOD-828] Fix a typo. * [NOD-828] Use copy instead of append in Bucket. * [NOD-828] Extract flatFileLocationSerializedSize to a const. * [NOD-828] Debugf -> Warnf in rollback.go. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Remove data length from flat file data format. * [NOD-828] Rearrange TestLevelDBTransactionSanity a bit. * [NOD-828] Add stack traces to all errors that come out of library functions. * [NOD-828] Return errors from rollback(). * [NOD-828] Remove an irrelevant comment. * [NOD-828] Remove redundant whitespace. * [NOD-828] Handle nil in FetchBlock. * [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct. * Revert "[NOD-828] Make bucket a type alias instead of a struct." This reverts commit 1fd39652 * [NOD-828] Fix revert errors. * Revert "[NOD-828] Remove data length from flat file data format." This reverts commit ef408e32 * [NOD-862] Move Cursor() into the DataAccessor interface. * [NOD-828] Add Delete to DataAccessor. * [NOD-865] Move fee data db operations to dbaccess * [NOD-865] Move reachability data db operations to dbaccess * [NOD-865] Move UTXO diff data db operations to dbaccess * [NOD-865] Move subnetwork data db operations to dbaccess * [NOD-865] Fix createDAGState * [NOD-865] Remove old Get signature with "exists" * [NOD-865] Move multiset db operations to dbaccess * [NOD-865] Use dbaccess transactions where possible * [NOD-865] Remove old Get signature with "exists" * [NOD-881] Recover TestGHOSTDAGErrors * [NOD-865] Create function for db keys * [NOD-865] Change Exists to Has, and use accessor.Has where possible * [NOD-865] Make ClearReachabilityData transactive * [NOD-865] Don't iterate cursors while changing db data * [NOD-865] Rename RegisterSubnetwork -> StoreSubnetwork * [NOD-865] Change bucket from utxodiffs to utxo-diffs * [NOD-865] Rename SubnetworkExists->HasSubnetwork * [NOD-865] Change a comment * [NOD-865] Fix tests * [NOD-865] Fix comment * [NOD-865] Remove the prefix "db" from some functions * [NOD-865] Remove redundant comments * [NOD-865] Make clearBucket function * [NOD-865] Make clear functions get a dbTx as an arg * [NOD-865] Remove erroneous tx commit Co-authored-by: stasatdaglabs <stas@daglabs.com> * [NOD-868] Delete the old database package (#683) * [NOD-828] Create the database2 package that will some day replace the database package. * [NOD-828] Implement a "bucket" key mechanism. * [NOD-828] Move bucket.go into the ffldb2 package. * [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway. * [NOD-828] Copy over + fixup flat file structs from the old ffldb. * [NOD-828] Implement flatFilePath. * [NOD-828] Implement flatFileStore.write(). * [NOD-828] Implement flatFileStore.read(). * [NOD-828] Implement flatFileStore.rollback(). * [NOD-828] Sync the file to disk at the end of write(). * [NOD-828] Extract crc32ByteOrder to a separate variable. * [NOD-828] Add a sanity test. * [NOD-828] Remove context-unrelated methods from the Database interface. * [NOD-828] Create an ffldb object. Simply work against a context. * [NOD-828] Open the new database on start. * [NOD-828] Create the leveldb package. * [NOD-828] Implement opening/closing leveldb. * [NOD-828] Implement get/put out of/into leveldb. * [NOD-828] Implement transactions and make them implement a generic database interface. * [NOD-828] Write sanity tests for leveldb with and without transactions. * [NOD-828] Add another case to the transaction sanity test. * [NOD-828] Implement AppendBlock/RetrieveBlock. * [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package. * [NOD-828] Add RollbackFlatData to the database interface. * [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket. * [NOD-828] Implement StoreBlock. * [NOD-828] Implement FetchBlock. * [NOD-828] Implement HasBlock. * [NOD-828] Write a sanity test for block insertion. * [NOD-828] Implement CurrentFlatDataLocation. * [NOD-828] Implement storing the current block location. * [NOD-828] Implement initializing/syncing the flat file block store and the "metadata". * [NOD-828] Add InitBlockStore to TestBlockStoreSanity. * [NOD-828] Fix rename errors. * [NOD-828] Fix lint errors in the root database package. * [NOD-828] Fix lint errors in the ffldb.go. * [NOD-828] Fix lint errors in the flatfile/db.go. * [NOD-828] Rename packages in such a way to make the linter happy. * [NOD-828] Finish satisfying the linter. * [NOD-828] Fix doc.go. * [NOD-828] Fix comments in block.go. * [NOD-828] Move dbaccess out of the database package. * [NOD-828] Move opening/closing the database to dbaccess. * [NOD-828] Move the Database interface to the root database package, since it's meant to be generic. * [NOD-828] Make ffldb generic to later support additional databases. * [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported. * [NOD-828] Fix comments. * [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore. * [NOD-828] Make buckets nicer to use. * [NOD-828] Implement cursors that iterate over some bucket. * [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb. * [NOD-828] Write a test making sure that flat file repair works. * [NOD-828] Properly close the database in TestRepairFlatFiles. * [NOD-828] Add a comment warning against putting and getting the same data within the same transaction. * [NOD-828] Fix the flatFilesBucket description. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Fix the ffldb description. * [NOD-828] Rename Database to DataAccessor and rename Handle to Database. * [NOD-828] Make Get return nil if the value doesn't exist. * [NOD-828] Attempt to close leveldb even if closing ffdb failed. * [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket. * [NOD-828] Fix not updating the store location in ffldb transactions. * [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found. * [NOD-828] Update the README and doc.go. * [NOD-828] Simplify Bucket.Path(). * [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly. * [NOD-828] Combine two lines into one. * [NOD-828] Combine another two lines into one. * [NOD-828] Move a misplaced comment. * [NOD-828] Use Wrapf instead of Errorf where appropriate. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Fix Wrapf calls. * [NOD-828] Fix comments. * [NOD-828] Remove superfluous whitespace. * [NOD-828] Add a comment explaining the use of batches and snapshots. * [NOD-828] Implement RollbackUnlessClosed(). * [NOD-828] Return both errors in StoreBlock rollback. * [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor. * [NOD-828] Make bucket a type alias instead of a struct. * [NOD-828] Fix a typo. * [NOD-828] Use copy instead of append in Bucket. * [NOD-828] Extract flatFileLocationSerializedSize to a const. * [NOD-828] Debugf -> Warnf in rollback.go. * [NOD-828] Fix a comment. * [NOD-828] Fix a comment. * [NOD-828] Remove data length from flat file data format. * [NOD-828] Rearrange TestLevelDBTransactionSanity a bit. * [NOD-828] Add stack traces to all errors that come out of library functions. * [NOD-828] Return errors from rollback(). * [NOD-828] Remove an irrelevant comment. * [NOD-828] Remove redundant whitespace. * [NOD-828] Handle nil in FetchBlock. * [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct. * Revert "[NOD-828] Make bucket a type alias instead of a struct." This reverts commit 1fd39652 * [NOD-828] Fix revert errors. * Revert "[NOD-828] Remove data length from flat file data format." This reverts commit ef408e32 * [NOD-862] Move Cursor() into the DataAccessor interface. * [NOD-828] Add Delete to DataAccessor. * [NOD-865] Move fee data db operations to dbaccess * [NOD-865] Move reachability data db operations to dbaccess * [NOD-865] Move UTXO diff data db operations to dbaccess * [NOD-865] Move subnetwork data db operations to dbaccess * [NOD-865] Fix createDAGState * [NOD-865] Remove old Get signature with "exists" * [NOD-865] Move multiset db operations to dbaccess * [NOD-865] Use dbaccess transactions where possible * [NOD-865] Remove old Get signature with "exists" * [NOD-881] Recover TestGHOSTDAGErrors * [NOD-865] Create function for db keys * [NOD-865] Change Exists to Has, and use accessor.Has where possible * [NOD-865] Make ClearReachabilityData transactive * [NOD-865] Don't iterate cursors while changing db data * [NOD-865] Rename RegisterSubnetwork -> StoreSubnetwork * [NOD-865] Change bucket from utxodiffs to utxo-diffs * [NOD-865] Rename SubnetworkExists->HasSubnetwork * [NOD-865] Change a comment * [NOD-868] Remove all tests from old database. * [NOD-868] Remove all unused methods from the old database's interfaces. * [NOD-865] Fix tests * [NOD-868] Remove references to DB. * [NOD-865] Fix comment * [NOD-868] Remove the old ffldb besides the interface and errors.go. * [NOD-868] Remove errors.go. * [NOD-868] Remove the old database package. * [NOD-868] Add openDB to DAGSetup to emulate the old dbpath in dag.config. * [NOD-868] Rename database2 to database. * [NOD-868] Use NewTx instead of NoTx where required. * [NOD-868] Fix merge errors. * [NOD-868] Rename dbXXX functions to just xxx. * [NOD-868] Rename putDAGState to saveDAGState. * [NOD-868] Replace comments in initDAGState with logs. * [NOD-868] Explain the openDB parameter in DAGSetup. * [NOD-868] Fixup doc.go and README.md. * [NOD-868] Remove pointless transactions. Co-authored-by: Ori Newman <orinewman1@gmail.com> * [NOD-805] Fix merge errors. * [NOD-805] Fix a comment. * [NOD-805] Don't return virtualTxsAcceptanceData from applyDAGChanges. * [NOD-805] Add missing error handling in TestAcceptanceDataIndexRecover. * [NOD-805] Rename blockDAG to dag in indexers/manager.go. * [NOD-805] Defer cursor.Close() everywhere. * [NOD-805] Rename scanFlatFiles to findCurrentLocation. * [NOD-805] Extract crc32ChecksumLength and dataLengthLength to constants. * [NOD-805] Handle open files properly in rollback.go. * [NOD-805] Remove unnecessary func wrapper. * [NOD-805] Remove unnecessary trimming in initialize. * [NOD-805] Made StoreBlock accept only TxContext. * [NOD-805] Changed the log level of an error message to Error. * [NOD-805] Add a note about holding mutexes over deleteFile. * [NOD-805] Remove a false comment. * [NOD-805] Fix a comment. * [NOD-805] Rename blk to block. * [NOD-805] Extract utxoKey to a separate function. * [NOD-805] Move dbaccess.xxxKey functions to the tops of their respective files. * [NOD-805] Fix grammar in dbaccess/db.go. * [NOD-805] Wrap a failed database corruption recovery error. * [NOD-805] Split lines with WithStack in them. * [NOD-805] Fix the comment over initialize. * [NOD-805] Rename ffdb to flatFileDB and ldb to levelDB. * [NOD-805] Fix a comment. * [NOD-805] Fix a comment. * [NOD-805] Use s.writeCursor instead of cursor. * [NOD-805] Embed file in lockableFile. * [NOD-805] the the -> the * [NOD-805] openDB -> db * [NOD-805] Use TxContext in all flushToDB functions. * [NOD-805] Rename context -> dbContext. * [NOD-805] Reword the comment at the beginning on initDAGState. * [NOD-805] Explain cursor key trimming. * [NOD-805] Remove Error from Cursor. * [NOD-805] Return ErrNotFound from done Cursor Key and Value. * [NOD-805] Add missing error handling. * [NOD-805] Fix a comment. * [NOD-805] Fix a variable name. * [NOD-805] Remove pointless underscore. * [NOD-805] Fix a comment. * [NOD-805] Fix a variable name. Co-authored-by: Mike Zak <feanorr@gmail.com> Co-authored-by: Ori Newman <orinewman1@gmail.com> |
||
|
Ori Newman
|
d3b1953deb
|
[NOD-848] optimize utxo diffs serialize allocations (#666)
* [NOD-848] Optimize allocations when serializing UTXO diffs * [NOD-848] Use same UTXO serialization everywhere, and use compression as well * [NOD-848] Fix usage of wrong buffer * [NOD-848] Fix tests * [NOD-848] Fix wire tests * [NOD-848] Fix tests * [NOD-848] Remove VLQ * [NOD-848] Fix comments * [NOD-848] Add varint for big endian encoding * [NOD-848] In TestVarIntWire, assume the expected decoded value is the same as the serialization input * [NOD-848] Serialize outpoint index with big endian varint * [NOD-848] Remove p2pk from compression support * [NOD-848] Fix comments * [NOD-848] Remove p2pk from decompression support * [NOD-848] Make entry compression optional * [NOD-848] Fix tests * [NOD-848] Fix comments and var names * [NOD-848] Remove UTXO compression * [NOD-848] Fix tests * [NOD-848] Remove big endian varint * [NOD-848] Fix comments * [NOD-848] Rename ReadVarIntLittleEndian->ReadVarInt and fix WriteVarInt comment * [NOD-848] Add outpointIndexByteOrder variable * [NOD-848] Remove redundant comment * [NOD-848] Fix outpointMaxSerializeSize to the correct value * [NOD-848] Move subBuffer to utils |
||
|
Ori Newman
|
eb953286ec
|
[NOD-641] Upgrade to github.com/pkg/errors v0.9.1 and use errors.As where needed (#614)
* [NOD-641] Upgrade to github.com/pkg/errors v0.9.1 and use errors.As where needed * [NOD-641] Fix find and replace error * [NOD-641] Use errors.As for error type checking * [NOD-641] Fix errors.As for pointer types * [NOD-641] Use errors.As where needed * [NOD-641] Rename rErr->ruleErr * [NOD-641] Rename derr->dbErr * [NOD-641] e->flagsErr where necessary * [NOD-641] change jerr to more appropriate name * [NOD-641] Rename cerr->bdRuleErr * [NOD-641] Rename serr->scriptErr * [NOD-641] Use errors.Is instead of testutil.AreErrorsEqual in TestNewHashFromStr * [NOD-641] Rename bdRuleErr->dagRuleErr * [NOD-641] Rename mErr->msgErr * [NOD-641] Rename dErr->deserializeErr |
||
|
stasatdaglabs
|
03b7af9a13 |
[NOD-532] Replace "chain" with "DAG" where appropriate (#537)
* [NOD-532] Change chain to DAG in the root package. * [NOD-532] Change chain to DAG in checkpoints.go. * [NOD-532] Change chain to DAG in blockdag. * [NOD-532] Change chain to DAG in cmd. * [NOD-532] Change chain to DAG in dagconfig. * [NOD-532] Change chain to DAG in database. * [NOD-532] Change chain to DAG in mempool. * [NOD-532] Change chain to DAG in mempool. * [NOD-532] Change chain to DAG in netsync. * [NOD-532] Change chain to DAG in rpcclient. * [NOD-532] Change chain to DAG in server. * [NOD-532] Change chain to DAG in txscript. * [NOD-532] Change chain to DAG in util. * [NOD-532] Change chain to DAG in wire. * [NOD-532] Remove block heights in dagio.go examples. * [NOD-532] Rename fakeChain to fakeDAG. * [NOD-532] Fix comments, remove unused EnableBCInfoHacks flag. * [NOD-532] Fix comments and variable names. * [NOD-532] Fix comments. * [NOD-532] Fix merge errors. * [NOD-532] Formatted project. |
||
|
stasatdaglabs
|
a140327dd2 |
[NOD-500] Remove checkpoints (#541)
* [NOD-502] Remove checkpoints. * [NOD-502] Remove remaining references to checkpoints. * [NOD-500] Split RejectFinality to RejectDifficulty. * [NOD-500] Remove support for headers-first in p2p. * [NOD-500] Panic in newHashFromStr in case of an error. |
||
|
Svarog
|
369ec449a8 |
[NOD-509] Change organization name to kaspanet (#524)
* [NOD-509] Change organization name to kaspanet * [NOD-509] Reorganize imports |
||
|
Svarog
|
f4c6859e51 | [NOD-509] Updated repository and imports to github.com/daglabs/kaspad (#521) | ||
|
Ori Newman
|
c88fa1492e |
[NOD-375] Move to pkg/errors (#447)
* [NOD-375] Move to pkg/errors * [NOD-375] Fix tests * [NOD-375] Make AreErrorsEqual a shared function |
||
|
Ori Newman
|
a789680db1 |
[NOD-314] change pkscript to scriptpubkey (#400)
* [NOD-314] Change everywhere PkScript to ScriptPubKey * [NOD-314] Change everywhere PkScript to ScriptPubKey * [NOD-314] Rename pkPops -> scriptPubKeyPops |
||
|
Ori Newman
|
263737b3fb |
[NOD-196] move coinbase scriptpukey to payload (#330)
* [NOD-196] move coinbase scriptpukey to payload (no tests) (#311) * [NOD-196] Move coinbase scriptPubKey to payload * [NOD-196] Rename SubnetworkID.IsFull to SubnetworkID.IsBuiltIn * [NOD-196] Fix comments * [NOD-196] Add block subsidy to fee transaction * [NOD-196] Fix comments * [NOD-217] Merge coinbase and fee transaction (#328) * [NOD-196] Fix tests * [NOD-196] Fix tests * [NOD-217] Add error to getBluesFeeData * [NOD-217] Merge Coinbase and fee transaction * [NOD-217] Format project * [NOD-217] Remove OpTrue default for mining.NewBlockTemplate * [NOD-196] Format project * [NOD-217] Add missing space before comment * [NOD-196] Change MaxCoinbasePayloadLen to 150 |
||
|
stasatdaglabs
|
ffd886498a |
[NOD-208] Make block reward maturity use the same mechanism as confirmations (#327)
* [NOD-208] Added blockBlueScore to UTXOEntry. * [NOD-208] Added blueBlockScore to NewUTXOEntry. * [NOD-208] Fixed compilation errors in policy, utxoset, and dag tests. * [NOD-208] Changed validateBlockRewardMaturity and CheckTransactionInputsAndCalulateFee to use blueScore. * [NOD-208] Changed CalcBlockSubsidy to use blueScore. * [NOD-208] Changed SequenceLockActive to use blueScore. * [NOD-208] Removed ExtractCoinbaseHeight. * [NOD-208] Removed reference to block height in ensureNoDuplicateTx. * [NOD-208] Changed IsFinalizedTransaction to use blueScore. * [NOD-208] Fixed merge errors. * [NOD-208] Made UTXOEntry serialization use blueScore. * [NOD-208] Changed CalcPriority and calcInputValueAge to use blueScore. * [NOD-208] Changed calcSequenceLock to use blueScore. * [NOD-208] Removed blockChainHeight from UTXOEntry. * [NOD-208] Fixed compilation errors in feeEstimator. Fixed a bug in the test pool hardness. * [NOD-208] Fixed oldestChainBlockWithBlueScoreGreaterThan not handling an extreme case. * [NOD-208] Fixed TestDiffFromTx. * [NOD-208] Got rid of priority and support of free transactions. * [NOD-208] Fixed TestProcessTransaction. * [NOD-208] Fixed TestTxFeePrioHeap. * [NOD-208] Fixed TestAddrIndex and TestFeeEstimatorCfg. * [NOD-208] Removed unused rateLimit parameter from ProcessTransaction. * [NOD-208] Fixed tests that rely on CreateTxChain. * [NOD-208] Fixed tests that rely on CreateSignedTxForSubnetwork. * [NOD-208] Fixed TestFetchTransaction. * [NOD-208] Fixed TestHandleNewBlock. Fixed HandleNewBlock erroneously processing fee transactions. * [NOD-208] Fixed TestTxIndexConnectBlock. * [NOD-208] Removed the use of Height() from the fee estimator. * [NOD-208] Removed unused methods from rpcwebsocket.go. * [NOD-208] Removed Height from util.Block. * [NOD-208] Removed ErrForkTooOld. It doesn't make sense in a DAG. * [NOD-208] Made blockHeap use blueScore instead of height. * [NOD-208] Removed fee estimator. * [NOD-208] Removed DAG.Height. * [NOD-208] Made TestAncestorErrors test chainHeight instead of height. * [NOD-208] Fixed a couple of comments that were still speaking about block height. * [NOD-208] Replaced all uses of HighestTipHash with SelectedTipHash. * [NOD-208] Remove blockNode highest and some remaining erroneous uses of height. * [NOD-208] Fixed a couple of comments. Fixed outPoint -> outpoint merge error. * [NOD-208] Fixed a couple more comments. * [NOD-208] Used calcMinRequiredTxRelayFee instead of DefaultMinRelayTxFee for mempool tests. * [NOD-208] Renamed mempool Config BestHeight to DAGChainHeight. * [NOD-208] Fixed a bug in oldestChainBlockWithBlueScoreGreaterThan. Made calcSequenceLock use the node's selected parent chain rather than the virtual block's. * [NOD-208] Removed chainHeight from blockNode String(). Renamed checkpointsByHeight to checkpointsByChainHeight and prevCheckpointHeight to prevCheckpointChainHeight. Removed reference to chainHeight in blockIndexKey. Fixed comments in dagio.go. * [NOD-208] Removed indexers/blocklogger.go, as no one was using it. * [NOD-208] Made blocklogger.go log blueScore instead of height. * [NOD-208] Fixed typo. * [NOD-208] Fixed comments, did minor renaming. * [NOD-208] Made a "common sense" wrapper around sort.Search. * [NOD-208] Fixed comment in SearchSlice. |
||
|
Ori Newman
|
aa51b5f071 |
[NOD-179] Added ECMH-Multiset to all UTXO structs (#304)
* [NOD-172] Port EMCH from bchd * [NOD-172] Fix hdkeychain.TestErrors and add btcec.TestRecoverCompact * [NOD-172] Make ECMH immutable * [NOD-172] Fix gofmt errors * [NOD-172] Add TestMultiset_NewMultisetFromDataSlice and fix Point to be immutable * [NOD-172] Fix gofmt errors * [NOD-172] Add test for checking that the Union of a multiset and its inverse is zero * [NOD-179] Add ECMH Point to all UTXO-structs * [NOD-179] Fix utxo set tests * [NOD-179] Fix mempool tests * [NOD-179] Remove RemoveTxOuts * [NOD-179] Move serializeBlockUTXODiffData to the top of the file * [NOD-179] Fix serializeBlockUTXODiffData comment format * [NOD-179] Fix AddTx comment and name return values |
||
|
Svarog
|
ca0619bbcf |
[NOD-176] Moved daghash from dagconfig to util (#298)
* [NOD-176] Moved daghash from dagconfig to util * [NOD-176] Fixed order of includes with gofmt |
||
|
Svarog
|
33036278ac |
[NOD-144] Use chainHeight in SelectedAncestor, and update all logic that uses it (#281)
* [NOD-144] Use chainHeight in SelectedAncestor, and update all logic that uses it * [NOD-144] Moved UnminedHeight to blockdag, and updated all references |
||
|
Ori Newman
|
254eab96cd |
[NOD-55] Change daghash hash to pointer in most places (#239)
* [NOD-55] Change daghash.Hash to pointer in most places * [NOD-55] Fixed format error * [NOD-55] Fixed merge error * [NOD-55] Cancel copying hash in blockSet.hashes() |
||
|
Ori Newman
|
0aa7c430e0 |
Dev 364 implement the dag fee structure (#181)
* [DEV-364] Add fee transactions validation * [DEV-364] make NextBlockFeeTransactions for creating block templates * [DEV-364] apply coinbase rules to fee transaction is some cases * [DEV-364] Add comments * [DEV-364] put getTXO as separate function * [DEV-364] Make getParentsFeeData a separate function * [DEV-364] fix calculateFees * [DEV-364] force maximum sequence for fee transactions * [DEV-364] add TestValidateFeeTransactions * [DEV-364] change fee transaction to be one tx per block rather than one tx for each blue * [DEV-364] fix tests * [DEV-364] Use constants instead of inline numbers |
||
|
stasatdaglabs
|
a82e6ae24a |
[DEV-92] Package blockdag tests (#177)
* [DEV-92] Covered blocknode.go with tests. * [DEV-92] Added test for blockSet.highest. Fixed a bug in it. * [DEV-92] Added tests for blockSet.subtract and blockSet.addSet. * [DEV-92] Covered blockset.go with tests. * [DEV-92] Got rid of some old stuff related to STXOs. * [DEV-92] Covered error.go with tests. * [DEV-92] Covered utxoSet with tests. * [DEV-92] Fixed formatting. |
||
|
Evgeny Khirin
|
100fbbaaa4 |
[DEV-361] Create type TxID as alias to daghash.Hash. (#175)
* [DEV-361] Create type TxID as alias to daghash.Hash. Use it for transaction IDs * [DEV-361] Fixed missed renames * [DEV-361] Removed usage of zeroHash * [DEV-361] Fixed more missed renames |
||
|
Ori Newman
|
b963c0d364
|
Dev 334 make id merkle root and transaction id, fix tests, and add new tests (#166)
* [DEV-329] Add TxID * [DEV-329] Change Transaction inputs to reference by tx id instead of tx hash * [DEV-329] Fix tests * [DEV-329] change txhash to txid in mempool * [DEV-334] Make IDMerkleRoot * [DEV-329] Add txid that excludes payload, gas and ScriptSigs (#158) * [DEV-329] Add TxID * [DEV-329] Change Transaction inputs to reference by tx id instead of tx hash * [DEV-329] Fix tests * [DEV-329] change txhash to txid in mempool * [DEV-329] replace thinEncoding bool with txEncoding bitmask * [DEV-329] Change txencoding var names * [DEV-329] change txEncodingexcludeSignatureScript -> txEncodingExcludeSignatureScript * [DEV-334] Add IDMerkleRoot to blocknode and recalculate IDMerkleRoot when extraNonce is changed * [DEV-334] Fix tests * [DEV-334] Fix tests * [DEV-334] fix SubnetworkDAGCoin -> SubnetworkIDNative * [DEV-334] Add ID() function to Coin interface and rename hash to txID in a few places * [DEV-334] Add Root method for merkle root * [DEV-334] add comment to dag.SubnetworkID() * [DEV-334] fix serializeSize comment |
||
|
stasatdaglabs
|
f00651c4e3 |
[DEV-353] Rename subnetwork to subnetworkID everywhere in the code except for the command line flag (#165)
* [DEV-353] Renamed subnetwork -> subnetworkID. * [DEV-353] Renamed dagcoin -> native. |
||
|
Svarog
|
e3994cddac | [DEV-347] Normalize the way 'subnetwork' is capitalized (#161) | ||
|
stasatdaglabs
|
d33d633e77 |
[DEV-307] Create the sub-network registry (#147)
* [DEV-307] Implemented adding pending registry transactions. * [DEV-307] Implemented a skeleton for the sub-network registry algorithm. * [DEV-307] Implemented the serialization/deserialization logic for pending sub-network transactions. * [DEV-307] Implemented marking sub-network registry transactions as successfully registered. * [DEV-307] Implemented clearing pending sub-network entries and writing registered sub-networks. * [DEV-307] Added comments for all the dagio functions. * [DEV-307] Fixed a couple of bugs and added a test for checking serialization/deserialization of pending sub-network transactions. * [DEV-307] Implemented getting a registered sub-network's gas limit. * [DEV-307] Updated sub-network announcement transaction validation rules. * [DEV-307] Specified what is validated. * [DEV-307] Added initialization for lastSubNetworkID. * [DEV-307] Renamed extractValidSubNetworkRegistryTxs to validateAndExtractSubNetworkRegistryTxs to better reflect what it does. * Dev 303 implement block finality (#139) * [DEV-303] Implement block finality * [DEV-303] Add finality tests * [DEV-303] Make finality tests use maybeAcceptBlock * [DEV-303] Only check finality rules if we are fastAdd mode * [DEV-303] replace hasBiggerFinalityScoreThan checks with .finalityScore() * [DEV-307] Forgot to actually create the sub-network buckets. * [DEV-307] Wrote a full-flow test for sub-network registry and fixed a couple of bugs that it had uncovered. * [DEV-307] Took firstSubNetworkID out of dagconfig.Params. It's unnecessary there. * [DEV-307] Added persistance for dag.lastSubNetworkID. * [DEV-307] Moved sub-network stuff to their own files. Got rid of firstSubNetworkID. Replaced SubNetworkReservedLast with SubNetworkUnreservedFirst. Added ErrSubNetworkRegistry. * [DEV-307] Added a newline at the end of dag_test.go. * [DEV-307] Renamed previousFinalityPoint to initialFinalityPoint. * [DEV-307] Moved sub-network IO stuff to subnetworks.go and subnetworks_test.go. * [DEV-307] Merged dbPutRegisteredSubNetworkTx and dbRegisterSubNetwork. Fixed a too-big allocation and a block double-processing bug. * [DEV-307] Simplified the serialized representation of a slice of transactions. * [DEV-307] Fixed comments. |
||
|
Ori Newman
|
c0aafdf7e1 |
[DEV-303] Implement block finality (#139)
* [DEV-303] Implement block finality * [DEV-303] Add finality tests * [DEV-303] Make finality tests use maybeAcceptBlock * [DEV-303] Only check finality rules if we are fastAdd mode * [DEV-303] replace hasBiggerFinalityScoreThan checks with .finalityScore() |
||
|
Ori Newman
|
18f54d13a4 | [DEV-126] Get rid of tfSpent in txoFlags (#69) | ||
|
stasatdaglabs
|
37cd482db3 |
[DEV-75] Update UTXO model to work with Diffs (#53)
* [DEV-75] Removed fetchEntryByHash, which was no longer used. * [DEV-75] Removed a few functions in manager.go that weren't used by anything. * [DEV-75] checkConnectBlock will soon not accept a utxoViewpoint, so removed the call to view.Tip() so that it could be deleted. * [DEV-75] Got rid of tips in UtxoViewpoint. * [DEV-75] Added the full UTXO set to the virtual block. * [DEV-75] Implemented UTXO-wrangling when adding a new block. * [DEV-75] Added isCoinbase to utxoEntry creation. * [DEV-75] Added blockHeight to utxoEntry creation. * Implemented fetching the fullUTXOSet from the database. * [DEV-75] Simplified DAGState because almost all of the state in it was unnecessary. Also got rid of dbDAGState. * [DEV-75] Made the process around adding a new block, UTXO-wise, much safer. * [DEV-75] Implemented melding the virtual UTXO diff to the database. * [DEV-75] Fixed utxoSet loading from the wrong bucket. * [DEV-75] Began pruning utxoviewpoint.go. Replaced FetchUtxoEntry with a fullUTXOSet-based GetUTXOEntry. * [DEV-75] Removed fetchUtxos. * [DEV-75] Moved GetUTXOEntry into the virtual block. * [DEV-75] Updated IndexManager to not use utxoViewpoints. * [DEV-75] Fixed some bad login in restoreUTXO involving nodeDiffs. * [DEV-75] Got rid of the UTXO spend journal, which wasn't used anywhere. * [DEV-75] Moved some STXO-related validation logic out of connectToDAG and into checkConnectBlock. * [DEV-75] Renamed UtxoXxx to UTXOXxx. Removed a bunch of methods that were no longer used by anything. * [DEV-75] Another Utxo -> UTXO rename. * [DEV-75] Removed IsModified from UTXOView, which was not used anywhere. * [DEV-75] Renamed nodeDiff to provisionalNode. Added a bunch of comments. * [DEV-75] Removed the test for genesis in pastUTXO, since it can never happen. * [DEV-75] Wrote tests for errors in pastUTXO. * [DEV-75] Wrote tests for errors in restoreUTXO. * [DEV-75] Improved testErrorThroughPatching. * [DEV-75] Wrote tests for errors in verifyAndBuildUTXO. * [DEV-75] Used TipHashes instead of tips().hashes(), fixed comments in a few places, and brought back an error return that I erroneously removed. * [DEV-75] Removed UTXO set logs. * [DEV-75] Recreated add/remove/contains functions for utxoCollection. * [DEV-75] Changed newUTXOEntry to use an object initializer. * [DEV-75] Changed the UTXO bucket version to 1. * Added a comment that clarifies that the index is not initialized before initDAGState is called. * [DEV-75] Renamed GetVirtualBlock to VirtualBlock. * [DEV-75] Removed superfluous variables. * [DEV-75] Combined connectBlockToParents with updateParentsDiffs. * [DEV-75] Removed another superfluous variable. * [DEV-75] In pastUTXO, first fetch transactions from the database and only then add them. * [DEV-75] Reworded the comment for commit(). * [DEV-75] Made all the connectUTXO subfunctions not BlockDAG methods. * [DEV-75] Updated the comment for connectUTXO. * [DEV-75] Updated the comment explaining why we're creating provisionalNodes. * [DEV-75] Removed a couple of unnecessary calls to toProvisionalNode. * [DEV-75] Renamed connectUTXO to applyUTXOChanges. * [DEV-75] Replaced allProvisionalNodes with provisionalNodeSet, an object that holds provisionalNodes for this particular call to applyUTXOChanges. * [DEV-75] Changed createProvisionalNode to accept a boolean "withParents" instead of relying on the caller to supply the node's parents or an empty set. * [DEV-75] Made most applyUTXOChanges subfunctions be methods on provisionalNode. * [DEV-75] Fixed a couple of bad comments. * [DEV-75] Added descriptive error messages to callers of applyUTXOChanges. * [DEV-75] Fixed weird English. * [DEV-75] Replaced DAGState with a slice of DAG tip hashes. * [DEV-75] In createProvisionalNode, changed withParents to withRelatives to avoid certain kinds of attacks. * [DEV-75] Renamed createdProvisionalNode to newProvisionalNode. * [DEV-75] Added precalculating the amount of transactions inside a new block's blue set. * [DEV-75] Pruned unnecessary variable. |
||
|
Ori Newman
|
713b01c69d |
[DEV-86] Update BlockDAG.initDAGState to load the whole dag
* [DEV-86] remove selectedPrevBlock from initDAGState * [DEV-86] re-implement dagview to be dag compatible * [DEV-86] change comments |
||
|
stasatdaglabs
|
460216be65 |
[DEV-89] Convert DAGView into VirtualBlock (#41)
* [DEV-74] Implemented and written tests for utxoIterator. * [DEV-74] Improved utxoIterator tests. * [DEV-74] Implemented utxoCollection tests. * [DEV-74] Implemented utxoDiff and its tests. * [DEV-74] Implemented utxoSet. * [DEV -74] Added tests for fullUTXOSet. * [DEV-74] Added some tests for diffUTXOSet. * [DEV-74] Wrote tests for diffUTXOSet iterator. * [DEV-74] Added a negative test for diffUTXOSet.withDiff. * [DEV-74] Wrote tests for addTx. * [DEV-74] Wrote a toRemove test for addTx. * [DEV-74] Changed blockNode.utxoDiff to be of type utxoDiff. * [DEV-74] Removed superfluous whitespace. * [DEV-74] Renamed confusing "previousHash" to "hash". * [DEV-74] Fixed bad test and corrected failing test. * [DEV-74] Moved confusing "negatives" test to be part of the general utxoCollection test. * [DEV-74] Removed utxoDiff.inverted. * [DEV-74] Renamed blockNode.utxoDiff to blockNode.diff. * [DEV-74] Renamed diff to diffFrom for clarity's sake. * [DEV-74] Converted the type of utxoCollection from map[daghash.Hash]map[uint32]*wire.TxOut to map[wire.OutPoint]*UtxoEntry. * [DEV-74] Corrected test names in utxoCollection_test. * [DEV-74] Removed superfluous utxoCollection iterator and moved utxoIterator into utxoset.go. * [DEV-74] Renamed variables in utxoset.go. * [DEV-74] Renamed verifyTx to areInputsInUTXO and removed a superfulous test. * [DEV-74] Fixed bad test logic in TestDiffUTXOSet_addTx. * [DEV-74] Added a few comments. Added reference-equals checks to clone functions. * [DEV-74] Moved utxoCollection and utxoDiff into utxoset.go. * [DEV-74] Wrote explanations for utxoCollection and utxoDiff tests. * [DEV-74] Wrote explanations for all utxoSet tests besides addTx. * [DEV-74] Wrote explanations for TestDiffUTXOSet_addTx. * [DEV-74] Moved the documentation for utxoDiff into utxoset.go. * [DEV-74] Wrote an explanation on utxoSet. * [DEV-75] Found a typo. * [DEV-75] Renamed dag -> virtual, dagView -> virtualBlock. * [DEV-75] Renamed newDAGView to newVirtualBlock. * [DEV-75] Moved queries for the genesis block from virtualBlock to BlockDAG. * [DEV-75] Got rid of chainView height and findFork. * [DEV-75] Renamed receivers from c to v. * [DEV-75] Updated initBlockNode to allow for virtual (headerless) nodes, updated dbDAGState to contain multiple tip hashes, implemented virtualBlock.setTips. * [DEV-75] Got rid of virtualBlock.equals, which was not used anywhere. * [DEV-75] Got rid of virtualBlock.tip(). * [DEV-75] Got rid of SetTip everywhere except for tests. * [DEV-75] Got rid of Next(). * [DEV-75] Got rid of Contains(). * [DEV-75] Got rid of HeightRange(), as no one was using it. * [DEV-75] Made verifyDAG in rpcserver.go not use block height for iteration. * [DEV-75] Got rid of the part of Manager.Init() that handled "catching up" for side chains, which allowed me to get rid of BlockDAG.BlockByHeight(). * [DEV-75] Dropped support for the RPC command getblockhash since it was getting blocks by their height. * [DEV-75] Dropped getnetworkhashps since it was reliant on height, fixed another couple of RPC commands to return nextHashes instead of a nextHash, and got rid of nodeByHeight in virtualBlock. * [DEV-75] Got rid of setTip(). * [DEV-75] Moved blockLocator() out of virtualBlock and into BlockDAG. Also removed TestLocateInventory(). * [DEV-75] Implemented addTip(). * [DEV-75] Cleaned up virtualblock.go a bit. * [DEV-75] Erased irrelevant tests in virtualblock_test.go. Moved dag-related tests into dag_test.go. * [DEV-75] Removed unnecessary nil check. * [DEV-75] Wrote tests for virtualBlock. * [DEV-75] Fixed bad test, added explanations to tests. * [DEV-89] Fixed a comment. * [DEV-89] Fixed another comment. * [DEV-89] Removed the section in Manager::Init that handled rolling back indexes to the main chain if their tip is an orphaned fork. This could only happen during reorg, which no longer exists. Also removed BlockDAG::MainChainHasBlock, which was no longer used by anyone. * [DEV-89] Removed the nil check inside initBlockNode() and amended the one place that called it with nil. * [DEV-89] Renamed the receiver param for BlockDAG from b to dag. * [DEV-89] Moved fastLog2Floor from dag.go to btcutil/btcmath.go. * [DEV-89] Renamed tstTip to testTip. * [DEV-89] Renamed phanom_test.go to phantom_test.go. * [DEV-89] Fixed comments, renamed mainChainHeight to dagHeight. * [DEV-89] Rewrote virtualBlock.addTip(). * [DEV-89] Fixed a comment. (chain -> DAG) * [DEV-89] Fixed another chain -> DAG comment. |
||
|
Ori Newman
|
29f1b2baff | [DEV-31] Regenrate block databases and genesis blocks in order to be compatiable with the new structure (uint64 timestamps) | ||
|
stasatdaglabs
|
ead8a06219 |
[DEV-65] Convert BestState and GetSnapshot to State and GetCurrentState (#27)
* [DEV-65] Renamed BestState to State and GetSnapshot to GetCurrentState. * [DEV-65] Implemented State and SelectedTip, updated dagState (the serializable version of State), and updated State creations. * [DEV-65] Fixed references to GetCurrentState and fixed tests. * [DEV-65] Renamed State to DAGState, corrected comments, created a new thread-safe method setDAGState, fixed broken references I missed. * [DEV-65] Converted dagState (the object) to dbDAGState, and made it serialize/deserialize into/out of JSON. * [DEV-65] Renamed Txns to Txs and fixed a couple of comments. * [DEV-65] Renamed some more "Txns" and removed some dead code. |
||
|
stasatdaglabs
|
b07805cc12 |
[DEV-54] Rename "chainXXX", "bestXXX", etc. inside blockdag package (#25)
* [DEV-54] Renamed chainXXX to dagXXX. * [DEV-54] Additional chain -> dag renamings. * [DEV-54] Renamed a couple of more "chain"s I missed. |