Mirroristas/kaspad
2
0
Fork 0
mirror of https://github.com/kaspanet/kaspad.git synced 2025-09-13 21:10:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,724 Commits 83 Branches 299 Tags 172 MiB
Commit Graph

46 Commits

Author SHA1 Message Date
stasatdaglabs
f8b18e09d6
[NOD-805] Redesign the database (#685) 
* [NOD-828] Reimplement FFLDB (#663)

* [NOD-828] Create the database2 package that will some day replace the database package.

* [NOD-828] Implement a "bucket" key mechanism.

* [NOD-828] Move bucket.go into the ffldb2 package.

* [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway.

* [NOD-828] Copy over + fixup flat file structs from the old ffldb.

* [NOD-828] Implement flatFilePath.

* [NOD-828] Implement flatFileStore.write().

* [NOD-828] Implement flatFileStore.read().

* [NOD-828] Implement flatFileStore.rollback().

* [NOD-828] Sync the file to disk at the end of write().

* [NOD-828] Extract crc32ByteOrder to a separate variable.

* [NOD-828] Add a sanity test.

* [NOD-828] Remove context-unrelated methods from the Database interface.

* [NOD-828] Create an ffldb object. Simply work against a context.

* [NOD-828] Open the new database on start.

* [NOD-828] Create the leveldb package.

* [NOD-828] Implement opening/closing leveldb.

* [NOD-828] Implement get/put out of/into leveldb.

* [NOD-828] Implement transactions and make them implement a generic database interface.

* [NOD-828] Write sanity tests for leveldb with and without transactions.

* [NOD-828] Add another case to the transaction sanity test.

* [NOD-828] Implement AppendBlock/RetrieveBlock.

* [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package.

* [NOD-828] Add RollbackFlatData to the database interface.

* [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket.

* [NOD-828] Implement StoreBlock.

* [NOD-828] Implement FetchBlock.

* [NOD-828] Implement HasBlock.

* [NOD-828] Write a sanity test for block insertion.

* [NOD-828] Implement CurrentFlatDataLocation.

* [NOD-828] Implement storing the current block location.

* [NOD-828] Implement initializing/syncing the flat file block store and the "metadata".

* [NOD-828] Add InitBlockStore to TestBlockStoreSanity.

* [NOD-828] Fix rename errors.

* [NOD-828] Fix lint errors in the root database package.

* [NOD-828] Fix lint errors in the ffldb.go.

* [NOD-828] Fix lint errors in the flatfile/db.go.

* [NOD-828] Rename packages in such a way to make the linter happy.

* [NOD-828] Finish satisfying the linter.

* [NOD-828] Fix doc.go.

* [NOD-828] Fix comments in block.go.

* [NOD-828] Move dbaccess out of the database package.

* [NOD-828] Move opening/closing the database to dbaccess.

* [NOD-828] Move the Database interface to the root database package, since it's meant to be generic.

* [NOD-828] Make ffldb generic to later support additional databases.

* [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported.

* [NOD-828] Fix comments.

* [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore.

* [NOD-828] Make buckets nicer to use.

* [NOD-828] Implement cursors that iterate over some bucket.

* [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb.

* [NOD-828] Write a test making sure that flat file repair works.

* [NOD-828] Properly close the database in TestRepairFlatFiles.

* [NOD-828] Add a comment warning against putting and getting the same data within the same transaction.

* [NOD-828] Fix the flatFilesBucket description.

* [NOD-828] Remove superfluous whitespace.

* [NOD-828] Fix the ffldb description.

* [NOD-828] Rename Database to DataAccessor and rename Handle to Database.

* [NOD-828] Make Get return nil if the value doesn't exist.

* [NOD-828] Attempt to close leveldb even if closing ffdb failed.

* [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket.

* [NOD-828] Fix not updating the store location in ffldb transactions.

* [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found.

* [NOD-828] Update the README and doc.go.

* [NOD-828] Simplify Bucket.Path().

* [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly.

* [NOD-828] Combine two lines into one.

* [NOD-828] Combine another two lines into one.

* [NOD-828] Move a misplaced comment.

* [NOD-828] Use Wrapf instead of Errorf where appropriate.

* [NOD-828] Fix a comment.

* [NOD-828] Fix a comment.

* [NOD-828] Fix Wrapf calls.

* [NOD-828] Fix comments.

* [NOD-828] Remove superfluous whitespace.

* [NOD-828] Add a comment explaining the use of batches and snapshots.

* [NOD-828] Implement RollbackUnlessClosed().

* [NOD-828] Return both errors in StoreBlock rollback.

* [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor.

* [NOD-828] Make bucket a type alias instead of a struct.

* [NOD-828] Fix a typo.

* [NOD-828] Use copy instead of append in Bucket.

* [NOD-828] Extract flatFileLocationSerializedSize to a const.

* [NOD-828] Debugf -> Warnf in rollback.go.

* [NOD-828] Fix a comment.

* [NOD-828] Fix a comment.

* [NOD-828] Remove data length from flat file data format.

* [NOD-828] Rearrange TestLevelDBTransactionSanity a bit.

* [NOD-828] Add stack traces to all errors that come out of library functions.

* [NOD-828] Return errors from rollback().

* [NOD-828] Remove an irrelevant comment.

* [NOD-828] Remove redundant whitespace.

* [NOD-828] Handle nil in FetchBlock.

* [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct.

* Revert "[NOD-828] Make bucket a type alias instead of a struct."

This reverts commit 1fd39652

* [NOD-828] Fix revert errors.

* Revert "[NOD-828] Remove data length from flat file data format."

This reverts commit ef408e32

* [NOD-862] Move Cursor() into the DataAccessor interface.

* [NOD-828] Add Delete to DataAccessor.

* [NOD-828] Fix a couple of places that erroneously referenced blocks.

* [NOD-828] Add a comment on top of flatFileLocationSerializedSize.

* [NOD-828] Add Seek to Cursor.

* [NOD-828] Add First to Cursor.

* [NOD-828] Rename db to accessor in Context.

* [NOD-828] Make Get/Fetch calls return a boolean to indicate whether the requested item was found.

* [NOD-828] Name the output parameters of all Get functions.

* [NOD-828] Make RetrieveFromStore return whether the data was found.

* [NOD-887] Add a couple of QoL features to Cursor (#674)

* [NOD-887] Changed First to not return an error.

* [NOD-887] Fix merge error.

* [NOD-887] Make Cursor.Key not return the entire key path.

* [NOD-888] Add RollbackUnlessClosed to Context (#676)

* [NOD-888] Add RollbackUnlessClosed to Context.

* [NOD-888] Fix copy+paste error.

* [NOD-889] Instead of returning a boolean for not-found, return an error (#677)

* [NOD-889] Instead of returning a boolean for not-found, return an error.

* [NOD-889] Wrapped ErrNotFound for Get calls with nicer error messages.

* [NOD-889] Fix format.

* [NOD-889] Fix double space in a comment.

* [NOD-889] Add IsNotFoundError to dbaccess.

* [NOD-862] Replace calls to Tx.StoreBlock, Tx.HasBlock, Tx.FetchBlock with appropriate calls in dbaccess (#672)

* [NOD-828] Create the database2 package that will some day replace the database package.

* [NOD-828] Implement a "bucket" key mechanism.

* [NOD-828] Move bucket.go into the ffldb2 package.

* [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway.

* [NOD-828] Copy over + fixup flat file structs from the old ffldb.

* [NOD-828] Implement flatFilePath.

* [NOD-828] Implement flatFileStore.write().

* [NOD-828] Implement flatFileStore.read().

* [NOD-828] Implement flatFileStore.rollback().

* [NOD-828] Sync the file to disk at the end of write().

* [NOD-828] Extract crc32ByteOrder to a separate variable.

* [NOD-828] Add a sanity test.

* [NOD-828] Remove context-unrelated methods from the Database interface.

* [NOD-828] Create an ffldb object. Simply work against a context.

* [NOD-828] Open the new database on start.

* [NOD-828] Create the leveldb package.

* [NOD-828] Implement opening/closing leveldb.

* [NOD-828] Implement get/put out of/into leveldb.

* [NOD-828] Implement transactions and make them implement a generic database interface.

* [NOD-828] Write sanity tests for leveldb with and without transactions.

* [NOD-828] Add another case to the transaction sanity test.

* [NOD-828] Implement AppendBlock/RetrieveBlock.

* [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package.

* [NOD-828] Add RollbackFlatData to the database interface.

* [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket.

* [NOD-828] Implement StoreBlock.

* [NOD-828] Implement FetchBlock.

* [NOD-828] Implement HasBlock.

* [NOD-828] Write a sanity test for block insertion.

* [NOD-828] Implement CurrentFlatDataLocation.

* [NOD-828] Implement storing the current block location.

* [NOD-828] Implement initializing/syncing the flat file block store and the "metadata".

* [NOD-828] Add InitBlockStore to TestBlockStoreSanity.

* [NOD-828] Fix rename errors.

* [NOD-828] Fix lint errors in the root database package.

* [NOD-828] Fix lint errors in the ffldb.go.

* [NOD-828] Fix lint errors in the flatfile/db.go.

* [NOD-828] Rename packages in such a way to make the linter happy.

* [NOD-828] Finish satisfying the linter.

* [NOD-828] Fix doc.go.

* [NOD-828] Fix comments in block.go.

* [NOD-828] Move dbaccess out of the database package.

* [NOD-828] Move opening/closing the database to dbaccess.

* [NOD-828] Move the Database interface to the root database package, since it's meant to be generic.

* [NOD-828] Make ffldb generic to later support additional databases.

* [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported.

* [NOD-828] Fix comments.

* [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore.

* [NOD-828] Make buckets nicer to use.

* [NOD-828] Implement cursors that iterate over some bucket.

* [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb.

* [NOD-828] Write a test making sure that flat file repair works.

* [NOD-828] Properly close the database in TestRepairFlatFiles.

* [NOD-828] Add a comment warning against putting and getting the same data within the same transaction.

* [NOD-862] Use dbaccess.HasBlock instead of Tx.HasBlock in initDAGState.

* [NOD-862] Use dbaccess.StoreBlock instead of dbStoreBlock.

* [NOD-862] Use dbaccess.FetchBlock instead of various block fetching mechanisms.

* [NOD-828] Fix the flatFilesBucket description.

* [NOD-828] Remove superfluous whitespace.

* [NOD-828] Fix the ffldb description.

* [NOD-828] Rename Database to DataAccessor and rename Handle to Database.

* [NOD-828] Make Get return nil if the value doesn't exist.

* [NOD-828] Attempt to close leveldb even if closing ffdb failed.

* [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket.

* [NOD-828] Fix not updating the store location in ffldb transactions.

* [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found.

* [NOD-828] Update the README and doc.go.

* [NOD-828] Simplify Bucket.Path().

* [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly.

* [NOD-828] Combine two lines into one.

* [NOD-828] Combine another two lines into one.

* [NOD-828] Move a misplaced comment.

* [NOD-828] Use Wrapf instead of Errorf where appropriate.

* [NOD-828] Fix a comment.

* [NOD-828] Fix a comment.

* [NOD-828] Fix Wrapf calls.

* [NOD-828] Fix comments.

* [NOD-828] Remove superfluous whitespace.

* [NOD-828] Add a comment explaining the use of batches and snapshots.

* [NOD-828] Implement RollbackUnlessClosed().

* [NOD-828] Return both errors in StoreBlock rollback.

* [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor.

* [NOD-828] Make bucket a type alias instead of a struct.

* [NOD-828] Fix a typo.

* [NOD-828] Use copy instead of append in Bucket.

* [NOD-828] Extract flatFileLocationSerializedSize to a const.

* [NOD-828] Debugf -> Warnf in rollback.go.

* [NOD-828] Fix a comment.

* [NOD-828] Fix a comment.

* [NOD-828] Remove data length from flat file data format.

* [NOD-828] Rearrange TestLevelDBTransactionSanity a bit.

* [NOD-828] Add stack traces to all errors that come out of library functions.

* [NOD-828] Return errors from rollback().

* [NOD-828] Remove an irrelevant comment.

* [NOD-828] Remove redundant whitespace.

* [NOD-828] Handle nil in FetchBlock.

* [NOD-828] Implement a dbaccess.BlockNode struct.

* [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct.

* [NOD-828] Implement toDBBlockNode and fromDBBlockNode.

* Revert "[NOD-828] Make bucket a type alias instead of a struct."

This reverts commit 1fd39652

* [NOD-828] Fix revert errors.

* Revert "[NOD-828] Remove data length from flat file data format."

This reverts commit ef408e32

* [NOD-862] Implement storing index blocks.

* [NOD-862] Use database transactions where appropriate.

* [NOD-862] Fix tests failing on DAGSetup.

* [NOD-862] Fix bad make call.

* [NOD-862] Fix remaining database opening problems in tests.

* [NOD-862] Move Cursor() into the DataAccessor interface.

* [NOD-862] Move Cursor() into the DataAccessor interface.

* [NOD-862] Iterate over the new block index in dagio.

* [NOD-862] Fix block index key.

* [NOD-828] Add Delete to DataAccessor.

* [NOD-862] Pass byte slices to dbaccess instead of objects.

* [NOD-862] Fix errors.

* [NOD-862] Fix maybeAcceptBlock not checking block existence.

* [NOD-862] Fix TestAcceptanceIndexRecover.

* [NOD-862] Add comments to StoreBlockIndex and BlockIndexCursor.

* [NOD-828] Fix a couple of places that erroneously referenced blocks.

* [NOD-828] Add a comment on top of flatFileLocationSerializedSize.

* [NOD-828] Add Seek to Cursor.

* [NOD-828] Add First to Cursor.

* [NOD-828] Rename db to accessor in Context.

* [NOD-828] Make Get/Fetch calls return a boolean to indicate whether the requested item was found.

* [NOD-828] Name the output parameters of all Get functions.

* [NOD-828] Make RetrieveFromStore return whether the data was found.

* [NOD-862] Fix merge errors.

* [NOD-862] Fix DAGSetup using bad temp directories.

* [NOD-862] Fix TestProcessDelayedBlocks not closing the database properly.

* [NOD-862] Fix merge errors.

* [NOD-862] Merge flushToDBWithContext and flushToDB.

* [NOD-862] Remove TODO.

* [NOD-862] Add prefix to the temp dir in DAGSetup.

* [NOD-862] Bring back dbFetchBlockByHash.

* [NOD-862] Use BlockDAG.BlockByHash in p2p and rpc.

* [NOD-862] Use daghash.Hash in dbaccess.

* [NOD-862] Add defer to RollbackUnlessClosed after NewTx().

* [NOD-862] Extract dbStoreBlock to a separate function.

* [NOD-862] Fix grammar in comment.

* [NOD-862] Fix merge errors.

* [NOD-867] Migrate database logic in blockdag/dagio.go to dbaccess (#675)

* [NOD-828] Create the database2 package that will some day replace the database package.

* [NOD-828] Implement a "bucket" key mechanism.

* [NOD-828] Move bucket.go into the ffldb2 package.

* [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway.

* [NOD-828] Copy over + fixup flat file structs from the old ffldb.

* [NOD-828] Implement flatFilePath.

* [NOD-828] Implement flatFileStore.write().

* [NOD-828] Implement flatFileStore.read().

* [NOD-828] Implement flatFileStore.rollback().

* [NOD-828] Sync the file to disk at the end of write().

* [NOD-828] Extract crc32ByteOrder to a separate variable.

* [NOD-828] Add a sanity test.

* [NOD-828] Remove context-unrelated methods from the Database interface.

* [NOD-828] Create an ffldb object. Simply work against a context.

* [NOD-828] Open the new database on start.

* [NOD-828] Create the leveldb package.

* [NOD-828] Implement opening/closing leveldb.

* [NOD-828] Implement get/put out of/into leveldb.

* [NOD-828] Implement transactions and make them implement a generic database interface.

* [NOD-828] Write sanity tests for leveldb with and without transactions.

* [NOD-828] Add another case to the transaction sanity test.

* [NOD-828] Implement AppendBlock/RetrieveBlock.

* [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package.

* [NOD-828] Add RollbackFlatData to the database interface.

* [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket.

* [NOD-828] Implement StoreBlock.

* [NOD-828] Implement FetchBlock.

* [NOD-828] Implement HasBlock.

* [NOD-828] Write a sanity test for block insertion.

* [NOD-828] Implement CurrentFlatDataLocation.

* [NOD-828] Implement storing the current block location.

* [NOD-828] Implement initializing/syncing the flat file block store and the "metadata".

* [NOD-828] Add InitBlockStore to TestBlockStoreSanity.

* [NOD-828] Fix rename errors.

* [NOD-828] Fix lint errors in the root database package.

* [NOD-828] Fix lint errors in the ffldb.go.

* [NOD-828] Fix lint errors in the flatfile/db.go.

* [NOD-828] Rename packages in such a way to make the linter happy.

* [NOD-828] Finish satisfying the linter.

* [NOD-828] Fix doc.go.

* [NOD-828] Fix comments in block.go.

* [NOD-828] Move dbaccess out of the database package.

* [NOD-828] Move opening/closing the database to dbaccess.

* [NOD-828] Move the Database interface to the root database package, since it's meant to be generic.

* [NOD-828] Make ffldb generic to later support additional databases.

* [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported.

* [NOD-828] Fix comments.

* [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore.

* [NOD-828] Make buckets nicer to use.

* [NOD-828] Implement cursors that iterate over some bucket.

* [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb.

* [NOD-828] Write a test making sure that flat file repair works.

* [NOD-828] Properly close the database in TestRepairFlatFiles.

* [NOD-828] Add a comment warning against putting and getting the same data within the same transaction.

* [NOD-862] Use dbaccess.HasBlock instead of Tx.HasBlock in initDAGState.

* [NOD-862] Use dbaccess.StoreBlock instead of dbStoreBlock.

* [NOD-862] Use dbaccess.FetchBlock instead of various block fetching mechanisms.

* [NOD-828] Fix the flatFilesBucket description.

* [NOD-828] Remove superfluous whitespace.

* [NOD-828] Fix the ffldb description.

* [NOD-828] Rename Database to DataAccessor and rename Handle to Database.

* [NOD-828] Make Get return nil if the value doesn't exist.

* [NOD-828] Attempt to close leveldb even if closing ffdb failed.

* [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket.

* [NOD-828] Fix not updating the store location in ffldb transactions.

* [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found.

* [NOD-828] Update the README and doc.go.

* [NOD-828] Simplify Bucket.Path().

* [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly.

* [NOD-828] Combine two lines into one.

* [NOD-828] Combine another two lines into one.

* [NOD-828] Move a misplaced comment.

* [NOD-828] Use Wrapf instead of Errorf where appropriate.

* [NOD-828] Fix a comment.

* [NOD-828] Fix a comment.

* [NOD-828] Fix Wrapf calls.

* [NOD-828] Fix comments.

* [NOD-828] Remove superfluous whitespace.

* [NOD-828] Add a comment explaining the use of batches and snapshots.

* [NOD-828] Implement RollbackUnlessClosed().

* [NOD-828] Return both errors in StoreBlock rollback.

* [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor.

* [NOD-828] Make bucket a type alias instead of a struct.

* [NOD-828] Fix a typo.

* [NOD-828] Use copy instead of append in Bucket.

* [NOD-828] Extract flatFileLocationSerializedSize to a const.

* [NOD-828] Debugf -> Warnf in rollback.go.

* [NOD-828] Fix a comment.

* [NOD-828] Fix a comment.

* [NOD-828] Remove data length from flat file data format.

* [NOD-828] Rearrange TestLevelDBTransactionSanity a bit.

* [NOD-828] Add stack traces to all errors that come out of library functions.

* [NOD-828] Return errors from rollback().

* [NOD-828] Remove an irrelevant comment.

* [NOD-828] Remove redundant whitespace.

* [NOD-828] Handle nil in FetchBlock.

* [NOD-828] Implement a dbaccess.BlockNode struct.

* [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct.

* [NOD-828] Implement toDBBlockNode and fromDBBlockNode.

* Revert "[NOD-828] Make bucket a type alias instead of a struct."

This reverts commit 1fd39652

* [NOD-828] Fix revert errors.

* Revert "[NOD-828] Remove data length from flat file data format."

This reverts commit ef408e32

* [NOD-862] Implement storing index blocks.

* [NOD-862] Use database transactions where appropriate.

* [NOD-862] Fix tests failing on DAGSetup.

* [NOD-862] Fix bad make call.

* [NOD-862] Fix remaining database opening problems in tests.

* [NOD-862] Move Cursor() into the DataAccessor interface.

* [NOD-862] Move Cursor() into the DataAccessor interface.

* [NOD-862] Iterate over the new block index in dagio.

* [NOD-862] Fix block index key.

* [NOD-828] Add Delete to DataAccessor.

* [NOD-862] Pass byte slices to dbaccess instead of objects.

* [NOD-862] Fix errors.

* [NOD-862] Fix maybeAcceptBlock not checking block existence.

* [NOD-862] Fix TestAcceptanceIndexRecover.

* [NOD-862] Add comments to StoreBlockIndex and BlockIndexCursor.

* [NOD-828] Fix a couple of places that erroneously referenced blocks.

* [NOD-828] Add a comment on top of flatFileLocationSerializedSize.

* [NOD-828] Add Seek to Cursor.

* [NOD-828] Add First to Cursor.

* [NOD-828] Rename db to accessor in Context.

* [NOD-828] Make Get/Fetch calls return a boolean to indicate whether the requested item was found.

* [NOD-828] Name the output parameters of all Get functions.

* [NOD-828] Make RetrieveFromStore return whether the data was found.

* [NOD-862] Fix merge errors.

* [NOD-862] Fix DAGSetup using bad temp directories.

* [NOD-862] Fix TestProcessDelayedBlocks not closing the database properly.

* [NOD-867] Remove blockIndexBucket from dagio.

* [NOD-867] Fix wrong key in StoreIndexBucket.

* [NOD-867] Migrate DAG state to dbaccess.

* [NOD-867] Remove utxoSetVersionKeyName.

* [NOD-862] Fix merge errors.

* [NOD-867] Move localSubnetworkID into dagState.

* [NOD-867] Fix a comment.

* [NOD-867] Remove an unused function.

* [NOD-867] Migrate the database's UTXO set to dbaccess.

* [NOD-867] Add missing error check.

* [NOD-867] Changed First to not return an error.

* [NOD-867] Make Cursor.Key not return the entire key path.

* [NOD-887] Fix the comment above BlockIndexCursorFrom.

* [NOD-862] Merge flushToDBWithContext and flushToDB.

* [NOD-862] Remove TODO.

* [NOD-862] Add prefix to the temp dir in DAGSetup.

* [NOD-862] Bring back dbFetchBlockByHash.

* [NOD-862] Use BlockDAG.BlockByHash in p2p and rpc.

* [NOD-862] Use daghash.Hash in dbaccess.

* [NOD-862] Add defer to RollbackUnlessClosed after NewTx().

* [NOD-862] Extract dbStoreBlock to a separate function.

* [NOD-867] Remove TODOs.

* [NOD-867] Fix merge errors.

* [NOD-867] Fix comments and errors.

* [NOD-867] Unexport blockIndexKey.

* [NOD-867] Fix merge errors.

* [NOD-867] Move a misplaced comment.

* [NOD-867] Fix an error message.

* [NOD-867] Remove preallocation in initDAGState.

* [NOD-866] Migrate database logic in blockdag/indexers package to dbaccess (#682)

* [NOD-865] Delete blockidhash.go.

* [NOD-865] Remove a lot of no-longer relevant logic from indexers.

* [NOD-865] Pass TxContext to ConnectBlock.

* [NOD-865] Migrate the acceptance index to dbaccess.

* [NOD-865] Fix a block not being sent to ConnectBlock.

* [NOD-865] Pass the block's hash instead of the whole block.

* [NOD-865] Add forgotten Commit call.

* [NOD-865] Add comments.

* [NOD-866] Fix a comment.

* [NOD-866] Fix a comment.

* [NOD-866] Remove pointless indirection in acceptanceindex.

* [NOD-866] Fix comment over ForEachHash.

* [NOD-866] Rename ClearAcceptanceIndex to DropAcceptanceIndex.

* [NOD-866] Explain collecting keys before deleting them.

* [NOD-865] Move misc db logic to db access (#681)

* [NOD-828] Create the database2 package that will some day replace the database package.

* [NOD-828] Implement a "bucket" key mechanism.

* [NOD-828] Move bucket.go into the ffldb2 package.

* [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway.

* [NOD-828] Copy over + fixup flat file structs from the old ffldb.

* [NOD-828] Implement flatFilePath.

* [NOD-828] Implement flatFileStore.write().

* [NOD-828] Implement flatFileStore.read().

* [NOD-828] Implement flatFileStore.rollback().

* [NOD-828] Sync the file to disk at the end of write().

* [NOD-828] Extract crc32ByteOrder to a separate variable.

* [NOD-828] Add a sanity test.

* [NOD-828] Remove context-unrelated methods from the Database interface.

* [NOD-828] Create an ffldb object. Simply work against a context.

* [NOD-828] Open the new database on start.

* [NOD-828] Create the leveldb package.

* [NOD-828] Implement opening/closing leveldb.

* [NOD-828] Implement get/put out of/into leveldb.

* [NOD-828] Implement transactions and make them implement a generic database interface.

* [NOD-828] Write sanity tests for leveldb with and without transactions.

* [NOD-828] Add another case to the transaction sanity test.

* [NOD-828] Implement AppendBlock/RetrieveBlock.

* [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package.

* [NOD-828] Add RollbackFlatData to the database interface.

* [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket.

* [NOD-828] Implement StoreBlock.

* [NOD-828] Implement FetchBlock.

* [NOD-828] Implement HasBlock.

* [NOD-828] Write a sanity test for block insertion.

* [NOD-828] Implement CurrentFlatDataLocation.

* [NOD-828] Implement storing the current block location.

* [NOD-828] Implement initializing/syncing the flat file block store and the "metadata".

* [NOD-828] Add InitBlockStore to TestBlockStoreSanity.

* [NOD-828] Fix rename errors.

* [NOD-828] Fix lint errors in the root database package.

* [NOD-828] Fix lint errors in the ffldb.go.

* [NOD-828] Fix lint errors in the flatfile/db.go.

* [NOD-828] Rename packages in such a way to make the linter happy.

* [NOD-828] Finish satisfying the linter.

* [NOD-828] Fix doc.go.

* [NOD-828] Fix comments in block.go.

* [NOD-828] Move dbaccess out of the database package.

* [NOD-828] Move opening/closing the database to dbaccess.

* [NOD-828] Move the Database interface to the root database package, since it's meant to be generic.

* [NOD-828] Make ffldb generic to later support additional databases.

* [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported.

* [NOD-828] Fix comments.

* [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore.

* [NOD-828] Make buckets nicer to use.

* [NOD-828] Implement cursors that iterate over some bucket.

* [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb.

* [NOD-828] Write a test making sure that flat file repair works.

* [NOD-828] Properly close the database in TestRepairFlatFiles.

* [NOD-828] Add a comment warning against putting and getting the same data within the same transaction.

* [NOD-828] Fix the flatFilesBucket description.

* [NOD-828] Remove superfluous whitespace.

* [NOD-828] Fix the ffldb description.

* [NOD-828] Rename Database to DataAccessor and rename Handle to Database.

* [NOD-828] Make Get return nil if the value doesn't exist.

* [NOD-828] Attempt to close leveldb even if closing ffdb failed.

* [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket.

* [NOD-828] Fix not updating the store location in ffldb transactions.

* [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found.

* [NOD-828] Update the README and doc.go.

* [NOD-828] Simplify Bucket.Path().

* [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly.

* [NOD-828] Combine two lines into one.

* [NOD-828] Combine another two lines into one.

* [NOD-828] Move a misplaced comment.

* [NOD-828] Use Wrapf instead of Errorf where appropriate.

* [NOD-828] Fix a comment.

* [NOD-828] Fix a comment.

* [NOD-828] Fix Wrapf calls.

* [NOD-828] Fix comments.

* [NOD-828] Remove superfluous whitespace.

* [NOD-828] Add a comment explaining the use of batches and snapshots.

* [NOD-828] Implement RollbackUnlessClosed().

* [NOD-828] Return both errors in StoreBlock rollback.

* [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor.

* [NOD-828] Make bucket a type alias instead of a struct.

* [NOD-828] Fix a typo.

* [NOD-828] Use copy instead of append in Bucket.

* [NOD-828] Extract flatFileLocationSerializedSize to a const.

* [NOD-828] Debugf -> Warnf in rollback.go.

* [NOD-828] Fix a comment.

* [NOD-828] Fix a comment.

* [NOD-828] Remove data length from flat file data format.

* [NOD-828] Rearrange TestLevelDBTransactionSanity a bit.

* [NOD-828] Add stack traces to all errors that come out of library functions.

* [NOD-828] Return errors from rollback().

* [NOD-828] Remove an irrelevant comment.

* [NOD-828] Remove redundant whitespace.

* [NOD-828] Handle nil in FetchBlock.

* [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct.

* Revert "[NOD-828] Make bucket a type alias instead of a struct."

This reverts commit 1fd39652

* [NOD-828] Fix revert errors.

* Revert "[NOD-828] Remove data length from flat file data format."

This reverts commit ef408e32

* [NOD-862] Move Cursor() into the DataAccessor interface.

* [NOD-828] Add Delete to DataAccessor.

* [NOD-865] Move fee data db operations to dbaccess

* [NOD-865] Move reachability data db operations to dbaccess

* [NOD-865] Move UTXO diff data db operations to dbaccess

* [NOD-865] Move subnetwork data db operations to dbaccess

* [NOD-865] Fix createDAGState

* [NOD-865] Remove old Get signature with "exists"

* [NOD-865] Move multiset db operations to dbaccess

* [NOD-865] Use dbaccess transactions where possible

* [NOD-865] Remove old Get signature with "exists"

* [NOD-881] Recover TestGHOSTDAGErrors

* [NOD-865] Create function for db keys

* [NOD-865] Change Exists to Has, and use accessor.Has where possible

* [NOD-865] Make ClearReachabilityData transactive

* [NOD-865] Don't iterate cursors while changing db data

* [NOD-865] Rename RegisterSubnetwork -> StoreSubnetwork

* [NOD-865] Change bucket from utxodiffs to utxo-diffs

* [NOD-865] Rename SubnetworkExists->HasSubnetwork

* [NOD-865] Change a comment

* [NOD-865] Fix tests

* [NOD-865] Fix comment

* [NOD-865] Remove the prefix "db" from some functions

* [NOD-865] Remove redundant comments

* [NOD-865] Make clearBucket function

* [NOD-865] Make clear functions get a dbTx as an arg

* [NOD-865] Remove erroneous tx commit

Co-authored-by: stasatdaglabs <stas@daglabs.com>

* [NOD-868] Delete the old database package (#683)

* [NOD-828] Create the database2 package that will some day replace the database package.

* [NOD-828] Implement a "bucket" key mechanism.

* [NOD-828] Move bucket.go into the ffldb2 package.

* [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway.

* [NOD-828] Copy over + fixup flat file structs from the old ffldb.

* [NOD-828] Implement flatFilePath.

* [NOD-828] Implement flatFileStore.write().

* [NOD-828] Implement flatFileStore.read().

* [NOD-828] Implement flatFileStore.rollback().

* [NOD-828] Sync the file to disk at the end of write().

* [NOD-828] Extract crc32ByteOrder to a separate variable.

* [NOD-828] Add a sanity test.

* [NOD-828] Remove context-unrelated methods from the Database interface.

* [NOD-828] Create an ffldb object. Simply work against a context.

* [NOD-828] Open the new database on start.

* [NOD-828] Create the leveldb package.

* [NOD-828] Implement opening/closing leveldb.

* [NOD-828] Implement get/put out of/into leveldb.

* [NOD-828] Implement transactions and make them implement a generic database interface.

* [NOD-828] Write sanity tests for leveldb with and without transactions.

* [NOD-828] Add another case to the transaction sanity test.

* [NOD-828] Implement AppendBlock/RetrieveBlock.

* [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package.

* [NOD-828] Add RollbackFlatData to the database interface.

* [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket.

* [NOD-828] Implement StoreBlock.

* [NOD-828] Implement FetchBlock.

* [NOD-828] Implement HasBlock.

* [NOD-828] Write a sanity test for block insertion.

* [NOD-828] Implement CurrentFlatDataLocation.

* [NOD-828] Implement storing the current block location.

* [NOD-828] Implement initializing/syncing the flat file block store and the "metadata".

* [NOD-828] Add InitBlockStore to TestBlockStoreSanity.

* [NOD-828] Fix rename errors.

* [NOD-828] Fix lint errors in the root database package.

* [NOD-828] Fix lint errors in the ffldb.go.

* [NOD-828] Fix lint errors in the flatfile/db.go.

* [NOD-828] Rename packages in such a way to make the linter happy.

* [NOD-828] Finish satisfying the linter.

* [NOD-828] Fix doc.go.

* [NOD-828] Fix comments in block.go.

* [NOD-828] Move dbaccess out of the database package.

* [NOD-828] Move opening/closing the database to dbaccess.

* [NOD-828] Move the Database interface to the root database package, since it's meant to be generic.

* [NOD-828] Make ffldb generic to later support additional databases.

* [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported.

* [NOD-828] Fix comments.

* [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore.

* [NOD-828] Make buckets nicer to use.

* [NOD-828] Implement cursors that iterate over some bucket.

* [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb.

* [NOD-828] Write a test making sure that flat file repair works.

* [NOD-828] Properly close the database in TestRepairFlatFiles.

* [NOD-828] Add a comment warning against putting and getting the same data within the same transaction.

* [NOD-828] Fix the flatFilesBucket description.

* [NOD-828] Remove superfluous whitespace.

* [NOD-828] Fix the ffldb description.

* [NOD-828] Rename Database to DataAccessor and rename Handle to Database.

* [NOD-828] Make Get return nil if the value doesn't exist.

* [NOD-828] Attempt to close leveldb even if closing ffdb failed.

* [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket.

* [NOD-828] Fix not updating the store location in ffldb transactions.

* [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found.

* [NOD-828] Update the README and doc.go.

* [NOD-828] Simplify Bucket.Path().

* [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly.

* [NOD-828] Combine two lines into one.

* [NOD-828] Combine another two lines into one.

* [NOD-828] Move a misplaced comment.

* [NOD-828] Use Wrapf instead of Errorf where appropriate.

* [NOD-828] Fix a comment.

* [NOD-828] Fix a comment.

* [NOD-828] Fix Wrapf calls.

* [NOD-828] Fix comments.

* [NOD-828] Remove superfluous whitespace.

* [NOD-828] Add a comment explaining the use of batches and snapshots.

* [NOD-828] Implement RollbackUnlessClosed().

* [NOD-828] Return both errors in StoreBlock rollback.

* [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor.

* [NOD-828] Make bucket a type alias instead of a struct.

* [NOD-828] Fix a typo.

* [NOD-828] Use copy instead of append in Bucket.

* [NOD-828] Extract flatFileLocationSerializedSize to a const.

* [NOD-828] Debugf -> Warnf in rollback.go.

* [NOD-828] Fix a comment.

* [NOD-828] Fix a comment.

* [NOD-828] Remove data length from flat file data format.

* [NOD-828] Rearrange TestLevelDBTransactionSanity a bit.

* [NOD-828] Add stack traces to all errors that come out of library functions.

* [NOD-828] Return errors from rollback().

* [NOD-828] Remove an irrelevant comment.

* [NOD-828] Remove redundant whitespace.

* [NOD-828] Handle nil in FetchBlock.

* [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct.

* Revert "[NOD-828] Make bucket a type alias instead of a struct."

This reverts commit 1fd39652

* [NOD-828] Fix revert errors.

* Revert "[NOD-828] Remove data length from flat file data format."

This reverts commit ef408e32

* [NOD-862] Move Cursor() into the DataAccessor interface.

* [NOD-828] Add Delete to DataAccessor.

* [NOD-865] Move fee data db operations to dbaccess

* [NOD-865] Move reachability data db operations to dbaccess

* [NOD-865] Move UTXO diff data db operations to dbaccess

* [NOD-865] Move subnetwork data db operations to dbaccess

* [NOD-865] Fix createDAGState

* [NOD-865] Remove old Get signature with "exists"

* [NOD-865] Move multiset db operations to dbaccess

* [NOD-865] Use dbaccess transactions where possible

* [NOD-865] Remove old Get signature with "exists"

* [NOD-881] Recover TestGHOSTDAGErrors

* [NOD-865] Create function for db keys

* [NOD-865] Change Exists to Has, and use accessor.Has where possible

* [NOD-865] Make ClearReachabilityData transactive

* [NOD-865] Don't iterate cursors while changing db data

* [NOD-865] Rename RegisterSubnetwork -> StoreSubnetwork

* [NOD-865] Change bucket from utxodiffs to utxo-diffs

* [NOD-865] Rename SubnetworkExists->HasSubnetwork

* [NOD-865] Change a comment

* [NOD-868] Remove all tests from old database.

* [NOD-868] Remove all unused methods from the old database's interfaces.

* [NOD-865] Fix tests

* [NOD-868] Remove references to DB.

* [NOD-865] Fix comment

* [NOD-868] Remove the old ffldb besides the interface and errors.go.

* [NOD-868] Remove errors.go.

* [NOD-868] Remove the old database package.

* [NOD-868] Add openDB to DAGSetup to emulate the old dbpath in dag.config.

* [NOD-868] Rename database2 to database.

* [NOD-868] Use NewTx instead of NoTx where required.

* [NOD-868] Fix merge errors.

* [NOD-868] Rename dbXXX functions to just xxx.

* [NOD-868] Rename putDAGState to saveDAGState.

* [NOD-868] Replace comments in initDAGState with logs.

* [NOD-868] Explain the openDB parameter in DAGSetup.

* [NOD-868] Fixup doc.go and README.md.

* [NOD-868] Remove pointless transactions.

Co-authored-by: Ori Newman <orinewman1@gmail.com>

* [NOD-805] Fix merge errors.

* [NOD-805] Fix a comment.

* [NOD-805] Don't return virtualTxsAcceptanceData from applyDAGChanges.

* [NOD-805] Add missing error handling in TestAcceptanceDataIndexRecover.

* [NOD-805] Rename blockDAG to dag in indexers/manager.go.

* [NOD-805] Defer cursor.Close() everywhere.

* [NOD-805] Rename scanFlatFiles to findCurrentLocation.

* [NOD-805] Extract crc32ChecksumLength and dataLengthLength to constants.

* [NOD-805] Handle open files properly in rollback.go.

* [NOD-805] Remove unnecessary func wrapper.

* [NOD-805] Remove unnecessary trimming in initialize.

* [NOD-805] Made StoreBlock accept only TxContext.

* [NOD-805] Changed the log level of an error message to Error.

* [NOD-805] Add a note about holding mutexes over deleteFile.

* [NOD-805] Remove a false comment.

* [NOD-805] Fix a comment.

* [NOD-805] Rename blk to block.

* [NOD-805] Extract utxoKey to a separate function.

* [NOD-805] Move dbaccess.xxxKey functions to the tops of their respective files.

* [NOD-805] Fix grammar in dbaccess/db.go.

* [NOD-805] Wrap a failed database corruption recovery error.

* [NOD-805] Split lines with WithStack in them.

* [NOD-805] Fix the comment over initialize.

* [NOD-805] Rename ffdb to flatFileDB and ldb to levelDB.

* [NOD-805] Fix a comment.

* [NOD-805] Fix a comment.

* [NOD-805] Use s.writeCursor instead of cursor.

* [NOD-805] Embed file in lockableFile.

* [NOD-805] the the -> the

* [NOD-805] openDB -> db

* [NOD-805] Use TxContext in all flushToDB functions.

* [NOD-805] Rename context -> dbContext.

* [NOD-805] Reword the comment at the beginning on initDAGState.

* [NOD-805] Explain cursor key trimming.

* [NOD-805] Remove Error from Cursor.

* [NOD-805] Return ErrNotFound from done Cursor Key and Value.

* [NOD-805] Add missing error handling.

* [NOD-805] Fix a comment.

* [NOD-805] Fix a variable name.

* [NOD-805] Remove pointless underscore.

* [NOD-805] Fix a comment.

* [NOD-805] Fix a variable name.

Co-authored-by: Mike Zak <feanorr@gmail.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
 2020-04-02 13:56:32 +03:00
Ori Newman
d3b1953deb
[NOD-848] optimize utxo diffs serialize allocations (#666) 
* [NOD-848] Optimize allocations when serializing UTXO diffs

* [NOD-848] Use same UTXO serialization everywhere, and use compression as well

* [NOD-848] Fix usage of wrong buffer

* [NOD-848] Fix tests

* [NOD-848] Fix wire tests

* [NOD-848] Fix tests

* [NOD-848] Remove VLQ

* [NOD-848] Fix comments

* [NOD-848] Add varint for big endian encoding

* [NOD-848] In TestVarIntWire, assume the expected decoded value is the same as the serialization input

* [NOD-848] Serialize outpoint index with big endian varint

* [NOD-848] Remove p2pk from compression support

* [NOD-848] Fix comments

* [NOD-848] Remove p2pk from decompression support

* [NOD-848] Make entry compression optional

* [NOD-848] Fix tests

* [NOD-848] Fix comments and var names

* [NOD-848] Remove UTXO compression

* [NOD-848] Fix tests

* [NOD-848] Remove big endian varint

* [NOD-848] Fix comments

* [NOD-848] Rename ReadVarIntLittleEndian->ReadVarInt and fix WriteVarInt comment

* [NOD-848] Add outpointIndexByteOrder variable

* [NOD-848] Remove redundant comment

* [NOD-848] Fix outpointMaxSerializeSize to the correct value

* [NOD-848] Move subBuffer to utils
 2020-03-24 16:44:41 +02:00
Ori Newman
8909679f44
[NOD-818] Remove time adjustment (#658) 
* [NOD-818] Remove time adjustment

* [NOD-818] Remove interface ensuring and copyright message

* [NOD-818] Update comment
 2020-03-15 17:37:01 +02:00
Ori Newman
ee08531a52
[NOD-610] Rename newSet->newBlockSet and setFromSlice->blockSetFromSlice (#635) 2020-02-20 16:19:28 +02:00
Ori Newman
a4bb070722
[NOD-754] Fix staticcheck errors (#627) 
* [NOD-754] Fix staticcheck errors

* [NOD-754] Remove some unused exported functions

* [NOD-754] Fix staticcheck errors

* [NOD-754] Don't panic if out/in close fails

* [NOD-754] Wrap outside errors with custom message
 2020-02-18 16:56:38 +02:00
stasatdaglabs
11de12304e [NOD-700] Convert blockSet to map[*blockNode]struct{} (#604) 
* [NOD-700] Convert blockSet to map[*blockNode]struct{}.

* [NOD-700] Rename bluestNode to bluestBlock in bluest().

* [NOD-700] Make IsInSelectedParentChain not use the now-slower containsHash.

* [NOD-700] Rename block to node in blockset.go.

* [NOD-700] Remove containsHash and hashesEqual.

* [NOD-700] Add a comment to IsInSelectedParentChain about how it'll fail if the given blockHash is not within the block index.
 2020-01-27 11:48:58 +02:00
Ori Newman
2174a0a7f2 [NOD-497] Implement GHOSTDAG (#575) 
* [NOD-540] Implement reachability (#545)

* [NOD-540] Begin implementing reachability.

* [NOD-540] Finish implementing reachability.

* [NOD-540] Implement TestIsFutureBlock.

* [NOD-540] Implement TestInsertFutureBlock.

* [NOD-540] Add comments.

* [NOD-540] Add comment for interval in blockNode.

* [NOD-540] Updated comments over insertFutureBlock and isFutureBlock.

* [NOD-540] Implement interval splitting methods.

* [NOD-540] Begin implementing tree manipulation in blockNode.

* [NOD-540] Implement countSubtreesUp.

* [NOD-540] Add a comment explaining an impossible condition.

* [NOD-540] Implement applyIntervalDown.

* [NOD-540] Moved the reachability tree stuff into reachability.go.

* [NOD-540] Add some comments.

* [NOD-540] Add more comments, implement isInPast.

* [NOD-540] Fix comments.

* [NOD-540] Implement TestSplitFraction.

* [NOD-540] Implement TestSplitExact.

* [NOD-540] Implement TestSplit.

* [NOD-540] Add comments to structs.

* [NOD-540] Implement TestAddTreeChild.

* [NOD-540] Fix a comment.

* [NOD-540] Rename isInPast to isAncestorOf.

* [NOD-540] Rename futureBlocks to futureCoveringSet.

* [NOD-540] Rename isFutureBlock to isInFuture.

* [NOD-540] move reachabilityInterval to the top of reachability.go.

* [NOD-540] Change "s.t." to "such that" in a comment.

* [NOD-540] Fix indentation.

* [NOD-540] Fix a potential bug involving float inaccuracy.

* [NOD-540] Wrote a more descriptive error message.

* [NOD-540] Fix error messsage.

* [NOD-540] Fix the recursive countSubtreesUp.

* [NOD-540] Rename countSubtreesUp to countSubtrees and applyIntervalDown to propagateInterval.

* [NOD-540] Implement updating reachability for a valid new block.

* [NOD-540] Implement a disk storage for reachability data.

* [NOD-540] Fix not all tree nodes being written to the database.

* [NOD-540] Implement serialization for reachabilityData.

* [NOD-540] Implement some deserialization for reachabilityData.

* [NOD-540] Implement restoring the reachabilityStore on node restart.

* [NOD-540] Made interval and remainingInterval pointers.

* [NOD-540] Rename setTreeInterval to setInterval.

* [NOD-540] Rename reindexTreeIntervals to reindexIntervals and fixed the comment above it.

* [NOD-540] Expand the comment above reindexIntervals.

* [NOD-540] Fix comment above countSubtrees.

* [NOD-540] Fix comment above countSubtrees some more.

* [NOD-540] Fix comment above split.

* [NOD-540] Fix comment above isAncestorOf.

* [NOD-540] Fix comment above reachabilityTreeNode.

* [NOD-540] Fix weird condition in addTreeChild.

* [NOD-540] Rename addTreeChild to addChild.

* [NOD-540] Fix weird condition in splitFraction.

* [NOD-540] Reverse the lines in reachabilityTreeNode.String().

* [NOD-540] Renamed f to fraction and x to size.

* [NOD-540] Fix comment above bisect.

* [NOD-540] Implement rtn.isAncestorOf().

* [NOD-540] Use treeNode isAncestorOf instead of treeInterval isAncestorOf.

* [NOD-540] Use newReachabilityInterval instead of struct initialization.

* [NOD-540] Make reachabilityTreeNode.String() use strings.Join.

* [NOD-540] Use sync.RWMutex instead of locks.PriorityMutex.

* [NOD-540] Rename thisTreeNode to newTreeNode.

* [NOD-540] Rename setTreeNode to addTreeNode.

* [NOD-540] Extracted selectedParentAnticone to a separate function.

* [NOD-540] Rename node to this.

* [NOD-540] Move updateReachability and isAncestorOf from dag.go to reachability.go.

* [NOD-540] Add whitespace after multiline function signatures in reachability.go.

* [NOD-540] Make splitFraction return an error on empty interval.

* [NOD-540] Add a comment about rounding to splitFraction.

* [NOD-540] Replace sneaky tabs with spaces.

* [NOD-540] Rename split to splitExponential.

* [NOD-540] Extract exponentialFractions to a separate function.

* [NOD-540] Rename bisect to findIndex.

* [NOD-540] Add call to reachabilityStore.clearDirtyEntries at the end of saveChangesFromBlock.

* [NOD-540] Explain the dirty hack in reachabilityStore.init().

* [NOD-540] Split the function signature for deserializeReachabilityData to two lines.

* [NOD-540] Add a comment about float precision loss to exponentialFractions.

* [NOD-540] Corrected a comment about float precision loss to exponentialFractions.

* [NOD-540] Fixed a comment about float precision loss to exponentialFractions some more.

* [NOD-540] Added further comments above futureCoveringBlockSet.

* [NOD-540] Rename addTreeNode to setTreeNode.

* [NOD-540] Rename splitExponential to splitWithExponentialBias.

* [NOD-540] Fix object references in reachabilityData deserialization (#563)

* [NOD-540] Fix broken references in deserialization.

* [NOD-540] Fix broken references in futureCoveringSet deserialization. Also add comments.

* [NOD-540] Don't deserialize on the first pass in reachabilityStore.init().

* [NOD-540] Remove redundant assignment to loaded[hash].

* [NOD-540] Use NewHash instead of SetBytes. Rename data to destination.

* [NOD-540] Preallocate futureCoveringSet.

* [NOD-541] Implement GHOSTDAG (#560)

* [NOD-541] Implement GHOSTDAG

* [NOD-541] Replace the old PHANTOM variant with GHOSTDAG

* [NOD-541] Move dag.updateReachability to the top of dag.applyDAGChanges to update reachability before the virtual block is updated

* [NOD-541] Fix blueAnticoneSize

* [NOD-541] Initialize node.bluesAnticoneSizes

* [NOD-541] Fix pastUTXO and applyBlueBlocks blues order

* [NOD-541] Add serialization logic to node.bluesAnticoneSizes

* [NOD-541] Fix GHOSTDAG to not count the new block and the blue candidates anticone, add selected parent to blues, and save to node.bluesAnticoneSizes properly

* [NOD-541] Fix test names in inner strings

* [NOD-541] Writing TestGHOSTDAG

* [NOD-541] In blueAnticoneSize change node->current

* [NOD-541] name ghostdag return values

* [NOD-541] fix ghostdag to return slice

* [NOD-541] Split k-cluster violation rules

* [NOD-541] Add missing space

* [NOD-541] Add comment to ghostdag

* [NOD-541] In selectedParentAnticone rename past->selectedParentPast

* [NOD-541] Fix misrefernces to TestChainUpdates

* [NOD-541] Fix ghostdag comment

* [NOD-541] Make PrepareBlockForTest in blockdag package

* [NOD-541] Make PrepareBlockForTest in blockdag package

* [NOD-541] Assign to selectedParentAnticone[i] instead of appending

* [NOD-541] Remove redundant forceTransactions arguments from PrepareBlockForTEST

* [NOD-541] Add non-selected parents to anticoneHeap

* [NOD-541] add test for ghostdag

* [NOD-541] Add comments

* [NOD-541] Use adjusted time for initializing blockNode

* [NOD-541] Rename isAncestorOf -> isAncestorOfBlueCandidate

* [NOD-541] Remove params from PrepareBlockForTest

* [NOD-541] Fix TestChainHeight

* [NOD-541] Remove recursive lock

* [NOD-541] Fix TestTxIndexConnectBlock

* [NOD-541] Fix TestBlueBlockWindow

* [NOD-541] Put prepareAndProcessBlock in common_test.go

* [NOD-541] Fix TestConfirmations

* [NOD-541] Fix TestAcceptingBlock

* [NOD-541] Fix TestDifficulty

* [NOD-541] Fix TestVirtualBlock

* [NOD-541] Fix TestSelectedPath

* [NOD-541] Fix TestChainUpdates

* [NOD-541] Shorten TestDifficulty test time

* [NOD-541] Make PrepareBlockForTest use minimal valid block time

* [NOD-541] Remove TODO comment

* [NOD-541] Move blockdag related mining functions to mining.go

* [NOD-541] Use NextBlockCoinbaseTransaction instead of NextBlockCoinbaseTransactionNoLock in NextCoinbaseFromAddress

* [NOD-541] Remove useMinimalTime from BlockForMining

* [NOD-541] Make MedianAdjustedTime a *BlockDAG method

* [NOD-541] Fix ghostdag to use anticone slice instead of heap

* [NOD-541] Fix NewBlockTemplate locks

* [NOD-541] Fix ghostdag comments

* [NOD-541] Convert MedianAdjustedTime to NextBlockTime

* [NOD-541] Fix ghostdag comment

* [NOD-541] Fix TestGHOSTDAG comment

* [NOD-541] Add comment before sanity check

* [NOD-541] Explicitly initialize .blues in ghostdag

* [NOD-541] Rename *blockNode.lessThan to *blockNode.less

* [NOD-541] Remove redundant check if block != chainBlock

* [NOD-541] Fix comment

* [NOD-541] Fix comment

* [NOD-497] Add comment; General refactoring

* [NOD-497] General refactoring.

* [NOD-497] Use isAncestor of the tree rather than the node

* [NOD-497] Remove reachability mutex lock as it is redundant (dag lock is held so no need); General refactoring.

* [NOD-497] Update comment

* [NOD-497] Undo test blocktimestamp

* [NOD-497] Update comments; Use BlockNode.less for blockset;

* [NOD-497] Change processBlock to return boolean and not the delay duration (merge conflict)

* [NOD-497] Undo change for bluest to use less; Change blocknode less to use daghash.Less

Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
Co-authored-by: Dan Aharoni <dereeno@protonmail.com>
 2020-01-08 17:41:22 +02:00
stasatdaglabs
03b7af9a13 [NOD-532] Replace "chain" with "DAG" where appropriate (#537) 
* [NOD-532] Change chain to DAG in the root package.

* [NOD-532] Change chain to DAG in checkpoints.go.

* [NOD-532] Change chain to DAG in blockdag.

* [NOD-532] Change chain to DAG in cmd.

* [NOD-532] Change chain to DAG in dagconfig.

* [NOD-532] Change chain to DAG in database.

* [NOD-532] Change chain to DAG in mempool.

* [NOD-532] Change chain to DAG in mempool.

* [NOD-532] Change chain to DAG in netsync.

* [NOD-532] Change chain to DAG in rpcclient.

* [NOD-532] Change chain to DAG in server.

* [NOD-532] Change chain to DAG in txscript.

* [NOD-532] Change chain to DAG in util.

* [NOD-532] Change chain to DAG in wire.

* [NOD-532] Remove block heights in dagio.go examples.

* [NOD-532] Rename fakeChain to fakeDAG.

* [NOD-532] Fix comments, remove unused EnableBCInfoHacks flag.

* [NOD-532] Fix comments and variable names.

* [NOD-532] Fix comments.

* [NOD-532] Fix merge errors.

* [NOD-532] Formatted project.
 2019-12-17 13:40:03 +02:00
stasatdaglabs
f46dec449d [NOD-510] Change all references to Bitcoin to Kaspa (#531) 
* [NOD-510] Change coinbase flags to kaspad.

* [NOD-510] Removed superfluous spaces after periods in comments.

* [NOD-510] Rename btcd -> kaspad in the root folder.

* [NOD-510] Rename BtcEncode -> KaspaEncode and BtcDecode -> KaspaDecode.

* [NOD-510] Rename BtcEncode -> KaspaEncode and BtcDecode -> KaspaDecode.

* [NOD-510] Continue renaming btcd -> kaspad.

* [NOD-510] Rename btcjson -> kaspajson.

* [NOD-510] Rename file names inside kaspajson.

* [NOD-510] Rename kaspajson -> jsonrpc.

* [NOD-510] Finish renaming in addrmgr.

* [NOD-510] Rename package btcec to ecc.

* [NOD-510] Finish renaming stuff in blockdag.

* [NOD-510] Rename stuff in cmd.

* [NOD-510] Rename stuff in config.

* [NOD-510] Rename stuff in connmgr.

* [NOD-510] Rename stuff in dagconfig.

* [NOD-510] Rename stuff in database.

* [NOD-510] Rename stuff in docker.

* [NOD-510] Rename stuff in integration.

* [NOD-510] Rename jsonrpc to rpcmodel.

* [NOD-510] Rename stuff in limits.

* [NOD-510] Rename stuff in logger.

* [NOD-510] Rename stuff in mempool.

* [NOD-510] Rename stuff in mining.

* [NOD-510] Rename stuff in netsync.

* [NOD-510] Rename stuff in peer.

* [NOD-510] Rename stuff in release.

* [NOD-510] Rename stuff in rpcclient.

* [NOD-510] Rename stuff in server.

* [NOD-510] Rename stuff in signal.

* [NOD-510] Rename stuff in txscript.

* [NOD-510] Rename stuff in util.

* [NOD-510] Rename stuff in wire.

* [NOD-510] Fix failing tests.

* [NOD-510] Fix merge errors.

* [NOD-510] Fix go vet errors.

* [NOD-510] Remove merged file that's no longer relevant.

* [NOD-510] Add a comment above Op0.

* [NOD-510] Fix some comments referencing Bitcoin Core.

* [NOD-510] Fix some more comments referencing Bitcoin Core.

* [NOD-510] Fix bitcoin -> kaspa.

* [NOD-510] Fix more bitcoin -> kaspa.

* [NOD-510] Fix comments, remove DisconnectBlock in addrindex.

* [NOD-510] Rename KSPD to KASD.

* [NOD-510] Fix comments and user agent.
 2019-12-12 15:21:41 +02:00
Svarog
369ec449a8 [NOD-509] Change organization name to kaspanet (#524) 
* [NOD-509] Change organization name to kaspanet

* [NOD-509] Reorganize imports
 2019-12-08 17:33:42 +02:00
Svarog
f4c6859e51 [NOD-509] Updated repository and imports to github.com/daglabs/kaspad (#521) 2019-12-08 16:28:53 +02:00
Ori Newman
c88fa1492e [NOD-375] Move to pkg/errors (#447) 
* [NOD-375] Move to pkg/errors

* [NOD-375] Fix tests

* [NOD-375] Make AreErrorsEqual a shared function
 2019-11-04 11:24:12 +02:00
Ori Newman
e6a4ed04f3 [NOD-338] recover indexer if didnt work for a while (#422) 
* [NOD-338] Recover indexer if it didn't work for a while

* [NOD-338] Recover indexer if it didn't work for a while

* [NOD-338] Recover indexer if it didn't work for a while

* [NOD-338] Add tests and move blockidhash.go to blockdag package

* [NOD-338] Delete index current block id when dropping index, and do some refactoring

* [NOD-338] Change comments

* [NOD-338] Change recover error messages

* [NOD-338] Fix comments

* [NOD-338] Fix comments and fix test name
 2019-09-26 18:19:58 +03:00
Ori Newman
f7f44995d6 [NOD-215] implement difficulty adjustment algorithm (#331) 
* [NOD-215] Implement difficulty adjustment algorithm

* [NOD-215] Handle blocks with genesis parent, and fix adjustment factor calculation

* [NOD-215] Fix tests

* [NOD-215] fix calcNextRequiredDifficulty

* [NOD-215] Add TestDifficulty

* [NOD-215] Fix delay to be positive, and add tests for delayed blocks

* [NOD-215] Split calcBlockWindowMinMaxAndMedianTimestamps to two functions

* [NOD-215] Make explicit loop for padding blue block window with genesis

* [NOD-215] Name return values

* [NOD-215] Fix delay != 0 error messages

* [NOD-215] Fix comments

* [NOD-215] Fix blueBlockWindow

* [NOD-215] Add TestBlueBlockWindow

* [NOD-215] Rename PowLimit -> PowMax

* [NOD-215] Fix delay != 0 error messages

* [NOD-215] Move PowMaxBits to BlockDAG

* [NOD-215] Make blockWindow type

* [NOD-215] Make blueBlockWindow always pad with genesis

* [NOD-215] Remove redundant line in checkWindowIDs

* [NOD-215] Make medianTimestamp return error for empty window
 2019-06-26 15:47:39 +03:00
Ori Newman
263737b3fb [NOD-196] move coinbase scriptpukey to payload (#330) 
* [NOD-196] move coinbase scriptpukey to payload (no tests) (#311)

* [NOD-196] Move coinbase scriptPubKey to payload

* [NOD-196] Rename SubnetworkID.IsFull to SubnetworkID.IsBuiltIn

* [NOD-196] Fix comments

* [NOD-196] Add block subsidy to fee transaction

* [NOD-196] Fix comments

* [NOD-217] Merge coinbase and fee transaction (#328)

* [NOD-196] Fix tests

* [NOD-196] Fix tests

* [NOD-217] Add error to getBluesFeeData

* [NOD-217] Merge Coinbase and fee transaction

* [NOD-217] Format project

* [NOD-217] Remove OpTrue default for mining.NewBlockTemplate

* [NOD-196] Format project

* [NOD-217] Add missing space before comment

* [NOD-196] Change MaxCoinbasePayloadLen to 150
 2019-06-17 17:43:13 +03:00
stasatdaglabs
ffd886498a [NOD-208] Make block reward maturity use the same mechanism as confirmations (#327) 
* [NOD-208] Added blockBlueScore to UTXOEntry.

* [NOD-208] Added blueBlockScore to NewUTXOEntry.

* [NOD-208] Fixed compilation errors in policy, utxoset, and dag tests.

* [NOD-208] Changed validateBlockRewardMaturity and CheckTransactionInputsAndCalulateFee to use blueScore.

* [NOD-208] Changed CalcBlockSubsidy to use blueScore.

* [NOD-208] Changed SequenceLockActive to use blueScore.

* [NOD-208] Removed ExtractCoinbaseHeight.

* [NOD-208] Removed reference to block height in ensureNoDuplicateTx.

* [NOD-208] Changed IsFinalizedTransaction to use blueScore.

* [NOD-208] Fixed merge errors.

* [NOD-208] Made UTXOEntry serialization use blueScore.

* [NOD-208] Changed CalcPriority and calcInputValueAge to use blueScore.

* [NOD-208] Changed calcSequenceLock to use blueScore.

* [NOD-208] Removed blockChainHeight from UTXOEntry.

* [NOD-208] Fixed compilation errors in feeEstimator. Fixed a bug in the test pool hardness.

* [NOD-208] Fixed oldestChainBlockWithBlueScoreGreaterThan not handling an extreme case.

* [NOD-208] Fixed TestDiffFromTx.

* [NOD-208] Got rid of priority and support of free transactions.

* [NOD-208] Fixed TestProcessTransaction.

* [NOD-208] Fixed TestTxFeePrioHeap.

* [NOD-208] Fixed TestAddrIndex and TestFeeEstimatorCfg.

* [NOD-208] Removed unused rateLimit parameter from ProcessTransaction.

* [NOD-208] Fixed tests that rely on CreateTxChain.

* [NOD-208] Fixed tests that rely on CreateSignedTxForSubnetwork.

* [NOD-208] Fixed TestFetchTransaction.

* [NOD-208] Fixed TestHandleNewBlock. Fixed HandleNewBlock erroneously processing fee transactions.

* [NOD-208] Fixed TestTxIndexConnectBlock.

* [NOD-208] Removed the use of Height() from the fee estimator.

* [NOD-208] Removed unused methods from rpcwebsocket.go.

* [NOD-208] Removed Height from util.Block.

* [NOD-208] Removed ErrForkTooOld. It doesn't make sense in a DAG.

* [NOD-208] Made blockHeap use blueScore instead of height.

* [NOD-208] Removed fee estimator.

* [NOD-208] Removed DAG.Height.

* [NOD-208] Made TestAncestorErrors test chainHeight instead of height.

* [NOD-208] Fixed a couple of comments that were still speaking about block height.

* [NOD-208] Replaced all uses of HighestTipHash with SelectedTipHash.

* [NOD-208] Remove blockNode highest and some remaining erroneous uses of height.

* [NOD-208] Fixed a couple of comments. Fixed outPoint -> outpoint merge error.

* [NOD-208] Fixed a couple more comments.

* [NOD-208] Used calcMinRequiredTxRelayFee instead of DefaultMinRelayTxFee for mempool tests.

* [NOD-208] Renamed mempool Config BestHeight to DAGChainHeight.

* [NOD-208] Fixed a bug in oldestChainBlockWithBlueScoreGreaterThan. Made calcSequenceLock use the node's selected parent chain rather than the virtual block's.

* [NOD-208] Removed chainHeight from blockNode String().
Renamed checkpointsByHeight to checkpointsByChainHeight and prevCheckpointHeight to prevCheckpointChainHeight.
Removed reference to chainHeight in blockIndexKey.
Fixed comments in dagio.go.

* [NOD-208] Removed indexers/blocklogger.go, as no one was using it.

* [NOD-208] Made blocklogger.go log blueScore instead of height.

* [NOD-208] Fixed typo.

* [NOD-208] Fixed comments, did minor renaming.

* [NOD-208] Made a "common sense" wrapper around sort.Search.

* [NOD-208] Fixed comment in SearchSlice.
 2019-06-16 14:12:02 +03:00
Svarog
396842ae40 [NOD-207] Rename any place that says 'OutPoint' to 'Outpoint' (#323) 
* [NOD-207] Rename any place that says 'OutPoint' to 'Outpoint'

* [NOD-207] Fix any place that says output point
 2019-06-05 16:23:57 +03:00
stasatdaglabs
b1d3ca0206 [NOD-177] Remove idMerkleRoot (#299) 
* [NOD-177] Removed references to idMerkleRoot.

* [NOD-177] Generated new genesis hashes.

* [NOD-177] Generated new blk_ blocks.

* [NOD-177] Fixed TestHaveBlock.

* [NOD-177] Fixed The rest of the tests.

* [NOD-177] Fixed a couple of comments and a duplicate test.

* [NOD-177] Fixed blocks1-256.bz2.
 2019-05-15 16:16:57 +03:00
Svarog
ca0619bbcf [NOD-176] Moved daghash from dagconfig to util (#298) 
* [NOD-176] Moved daghash from dagconfig to util

* [NOD-176] Fixed order of includes with gofmt
 2019-05-14 14:05:36 +03:00
Ori Newman
d7a2ab52a1 [NOD-173] Add UTXO commitment to block header (#297) 2019-05-13 16:23:28 +03:00
Evgeny Khirin
08d94c7a47 [NOD-163] Added BlockHeader.AcceptedIDMerkleRoot (#293) 2019-05-12 11:53:47 +03:00
Svarog
e9ec8cd39c [NOD-142] Convert Height and ChainHeight to uint64 (#269) 
* [NOD-142] Updated util.FastLog2Floor to work on uint64

* [NOD-142] Convert height and chainHeight to uint64

* [NOD-142] A couple fixes in comments of TestFastLog2Floor

* [NOD-142] Make spendableOutOffset uint64 too
 2019-04-30 12:50:46 +03:00
Ori Newman
254eab96cd [NOD-55] Change daghash hash to pointer in most places (#239) 
* [NOD-55] Change daghash.Hash to pointer in most places

* [NOD-55] Fixed format error

* [NOD-55] Fixed merge error

* [NOD-55] Cancel copying hash in blockSet.hashes()
 2019-04-02 13:49:47 +03:00
Ori Newman
0aa7c430e0 Dev 364 implement the dag fee structure (#181) 
* [DEV-364] Add fee transactions validation

* [DEV-364] make NextBlockFeeTransactions for creating block templates

* [DEV-364] apply coinbase rules to fee transaction is some cases

* [DEV-364] Add comments

* [DEV-364] put getTXO as separate function

* [DEV-364] Make getParentsFeeData a separate function

* [DEV-364] fix calculateFees

* [DEV-364] force maximum sequence for fee transactions

* [DEV-364] add TestValidateFeeTransactions

* [DEV-364] change fee transaction to be one tx per block rather than one tx for each blue

* [DEV-364] fix tests

* [DEV-364] Use constants instead of inline numbers
 2019-02-11 17:33:55 +02:00
Evgeny Khirin
100fbbaaa4 [DEV-361] Create type TxID as alias to daghash.Hash. (#175) 
* [DEV-361] Create type TxID as alias to daghash.Hash. Use it for transaction IDs

* [DEV-361] Fixed missed renames

* [DEV-361] Removed usage of zeroHash

* [DEV-361] Fixed more missed renames
 2019-01-27 14:27:10 +02:00
Ori Newman
b963c0d364
Dev 334 make id merkle root and transaction id, fix tests, and add new tests (#166) 
* [DEV-329] Add TxID

* [DEV-329] Change Transaction inputs to reference by tx id instead of tx hash

* [DEV-329] Fix tests

* [DEV-329] change txhash to txid in mempool

* [DEV-334] Make IDMerkleRoot

* [DEV-329] Add txid that excludes payload, gas and ScriptSigs (#158)

* [DEV-329] Add TxID

* [DEV-329] Change Transaction inputs to reference by tx id instead of tx hash

* [DEV-329] Fix tests

* [DEV-329] change txhash to txid in mempool

* [DEV-329] replace thinEncoding bool with txEncoding bitmask

* [DEV-329] Change txencoding var names

* [DEV-329] change txEncodingexcludeSignatureScript -> txEncodingExcludeSignatureScript

* [DEV-334] Add IDMerkleRoot to blocknode and recalculate IDMerkleRoot when extraNonce is changed

* [DEV-334] Fix tests

* [DEV-334] Fix tests

* [DEV-334] fix SubnetworkDAGCoin -> SubnetworkIDNative

* [DEV-334] Add ID() function to Coin interface and rename hash to txID in a few places

* [DEV-334] Add Root method for merkle root

* [DEV-334] add comment to dag.SubnetworkID()

* [DEV-334] fix serializeSize comment
 2019-01-23 14:04:23 +02:00
Svarog
6b55950901 [DEV-345] Enforce requirement for transactions from 1(DAGCoin) or 2(sub network registry) sub network to have gas = 0 (#157) 
* [DEV-345] Validate that gas and payload are 0 when required by sub-network

* [DEV-345] Remove check for txOut.Value < 0, since txOut.Value is a uint64

* [DEV-345] Added tests for CheckTransactionSanity

* [DEV-345] Remove checks for Gas and Payload validity in wire.MsgTx.Decode

* [DEV-345] Verify that payload in Gas sub-network is always 8 bytes (uint64).

* [DEV-345] Renamed tstCheck{Script/Rule}Error to check{Script/Rule}Error

* [DEV-345] Improved formatting
 2019-01-13 11:01:43 +02:00
Ori Newman
c0aafdf7e1 [DEV-303] Implement block finality (#139) 
* [DEV-303] Implement block finality

* [DEV-303] Add finality tests

* [DEV-303] Make finality tests use maybeAcceptBlock

* [DEV-303] Only check finality rules if we are fastAdd mode

* [DEV-303] replace hasBiggerFinalityScoreThan checks with .finalityScore()
 2019-01-03 13:22:31 +02:00
Ori Newman
ef01f6410d [DEV-301] implement blockNode.chainHeight as the height of the selectedParent chain 
* [DEV-301] implement blockNode.chainHeight as the height of the selected parent chain

* [DEV-301] add description to TestChainHeight

* [DEV-301] Delete comment on TestChainHeight and expalain about the dag
 2018-12-23 16:54:45 +02:00
Ori Newman
9519b9f2a1 [DEV-211] Change fields of serialized blocks and transactions to match spec (#104) 
* [DEV-211] change block fields

* [DEV-211] change block fields

* [DEV-211] change comments to adhere to the new block field names
 2018-11-05 13:11:42 +02:00
Ori Newman
4951c0bee0 [DEV-260] Don't update parents on initBlockNode (#114) 
* [DEV-260] Don't update parents on initBlockNode

* [DEV-260] move addNodeAsChildToParents to blockdag/common_test.go

* [DEV-260] update newProvisionalNode comment
 2018-11-01 14:43:38 +02:00
Svarog
fae411706e [DEV-137] calculate height correctly in blockdag and mining packages (#72) 
* [DEV-137] calculate height correctly in blockdag and mining packages

* [DEV-137] Fix tests that didn't set block height
 2018-09-30 11:16:51 +03:00
Ori Newman
5fb220c38a Dev 105 get rid of utxoview (#58) 
* [DEV-105] use utxodiff in mempool instead of utxoview

* [DEV-105] get rid of utxoview

* [DEV-105] fix tests to use utxoset

* [DEV-105] remove utxoview type

* [DEV-105] move DagSetup to test_utils.go

* [DEV-105] add comments and add blockdag/test_utils_test.go

* [DEV-105] add restoreInputs arg to removeTransaction

* [DEV-105] give more descriptive names to vars

* [DEV-115] close txChan outside of HandleNewBlock

* [DEV-105] rename DagSetup -> DAGSetup
 2018-09-17 11:47:35 +03:00
stasatdaglabs
37cd482db3 [DEV-75] Update UTXO model to work with Diffs (#53) 
* [DEV-75] Removed fetchEntryByHash, which was no longer used.

* [DEV-75] Removed a few functions in manager.go that weren't used by anything.

* [DEV-75] checkConnectBlock will soon not accept a utxoViewpoint, so removed the call to view.Tip() so that it could be deleted.

* [DEV-75] Got rid of tips in UtxoViewpoint.

* [DEV-75] Added the full UTXO set to the virtual block.

* [DEV-75] Implemented UTXO-wrangling when adding a new block.

* [DEV-75] Added isCoinbase to utxoEntry creation.

* [DEV-75] Added blockHeight to utxoEntry creation.

* Implemented fetching the fullUTXOSet from the database.

* [DEV-75] Simplified DAGState because almost all of the state in it was unnecessary. Also got rid of dbDAGState.

* [DEV-75] Made the process around adding a new block, UTXO-wise, much safer.

* [DEV-75] Implemented melding the virtual UTXO diff to the database.

* [DEV-75] Fixed utxoSet loading from the wrong bucket.

* [DEV-75] Began pruning utxoviewpoint.go. Replaced FetchUtxoEntry with a fullUTXOSet-based GetUTXOEntry.

* [DEV-75] Removed fetchUtxos.

* [DEV-75] Moved GetUTXOEntry into the virtual block.

* [DEV-75] Updated IndexManager to not use utxoViewpoints.

* [DEV-75] Fixed some bad login in restoreUTXO involving nodeDiffs.

* [DEV-75] Got rid of the UTXO spend journal, which wasn't used anywhere.

* [DEV-75] Moved some STXO-related validation logic out of connectToDAG and into checkConnectBlock.

* [DEV-75] Renamed UtxoXxx to UTXOXxx. Removed a bunch of methods that were no longer used by anything.

* [DEV-75] Another Utxo -> UTXO rename.

* [DEV-75] Removed IsModified from UTXOView, which was not used anywhere.

* [DEV-75] Renamed nodeDiff to provisionalNode. Added a bunch of comments.

* [DEV-75] Removed the test for genesis in pastUTXO, since it can never happen.

* [DEV-75] Wrote tests for errors in pastUTXO.

* [DEV-75] Wrote tests for errors in restoreUTXO.

* [DEV-75] Improved testErrorThroughPatching.

* [DEV-75] Wrote tests for errors in verifyAndBuildUTXO.

* [DEV-75] Used TipHashes instead of tips().hashes(), fixed comments in a few places, and brought back an error return that I erroneously removed.

* [DEV-75] Removed UTXO set logs.

* [DEV-75] Recreated add/remove/contains functions for utxoCollection.

* [DEV-75] Changed newUTXOEntry to use an object initializer.

* [DEV-75] Changed the UTXO bucket version to 1.

* Added a comment that clarifies that the index is not initialized before initDAGState is called.

* [DEV-75] Renamed GetVirtualBlock to VirtualBlock.

* [DEV-75] Removed superfluous variables.

* [DEV-75] Combined connectBlockToParents with updateParentsDiffs.

* [DEV-75] Removed another superfluous variable.

* [DEV-75] In pastUTXO, first fetch transactions from the database and only then add them.

* [DEV-75] Reworded the comment for commit().

* [DEV-75] Made all the connectUTXO subfunctions not BlockDAG methods.

* [DEV-75] Updated the comment for connectUTXO.

* [DEV-75] Updated the comment explaining why we're creating provisionalNodes.

* [DEV-75] Removed a couple of unnecessary calls to toProvisionalNode.

* [DEV-75] Renamed connectUTXO to applyUTXOChanges.

* [DEV-75] Replaced allProvisionalNodes with provisionalNodeSet, an object that holds provisionalNodes for this particular call to applyUTXOChanges.

* [DEV-75] Changed createProvisionalNode to accept a boolean "withParents" instead of relying on the caller to supply the node's parents or an empty set.

* [DEV-75] Made most applyUTXOChanges subfunctions be methods on provisionalNode.

* [DEV-75] Fixed a couple of bad comments.

* [DEV-75] Added descriptive error messages to callers of applyUTXOChanges.

* [DEV-75] Fixed weird English.

* [DEV-75] Replaced DAGState with a slice of DAG tip hashes.

* [DEV-75] In createProvisionalNode, changed withParents to withRelatives to avoid certain kinds of attacks.

* [DEV-75] Renamed createdProvisionalNode to newProvisionalNode.

* [DEV-75] Added precalculating the amount of transactions inside a new block's blue set.

* [DEV-75] Pruned unnecessary variable.
 2018-08-30 16:00:03 +03:00
stasatdaglabs
c65d9aa168 [DEV-97] Combine btcd and btcutil repositories (#51) 
* [DEV-97] Moved github.com/daglabs/btcutil into github.com/daglabs/btcd/btcutil.

* [DEV-97] Updated Gopkg.toml to no longer refer to btcutil.

* [DEV-97] Renamed btcutil to util.
 2018-08-23 16:54:16 +03:00
Ori Newman
ce35aabb70 [DEV-88] Empty out btcd/blockdag/upgrade.go 2018-08-20 15:38:35 +03:00
stasatdaglabs
460216be65 [DEV-89] Convert DAGView into VirtualBlock (#41) 
* [DEV-74] Implemented and written tests for utxoIterator.

* [DEV-74] Improved utxoIterator tests.

* [DEV-74] Implemented utxoCollection tests.

* [DEV-74] Implemented utxoDiff and its tests.

* [DEV-74] Implemented utxoSet.

* [DEV -74] Added tests for fullUTXOSet.

* [DEV-74] Added some tests for diffUTXOSet.

* [DEV-74] Wrote tests for diffUTXOSet iterator.

* [DEV-74] Added a negative test for diffUTXOSet.withDiff.

* [DEV-74] Wrote tests for addTx.

* [DEV-74] Wrote a toRemove test for addTx.

* [DEV-74] Changed blockNode.utxoDiff to be of type utxoDiff.

* [DEV-74] Removed superfluous whitespace.

* [DEV-74] Renamed confusing "previousHash" to "hash".

* [DEV-74] Fixed bad test and corrected failing test.

* [DEV-74] Moved confusing "negatives" test to be part of the general utxoCollection test.

* [DEV-74] Removed utxoDiff.inverted.

* [DEV-74] Renamed blockNode.utxoDiff to blockNode.diff.

* [DEV-74] Renamed diff to diffFrom for clarity's sake.

* [DEV-74] Converted the type of utxoCollection from map[daghash.Hash]map[uint32]*wire.TxOut to map[wire.OutPoint]*UtxoEntry.

* [DEV-74] Corrected test names in utxoCollection_test.

* [DEV-74] Removed superfluous utxoCollection iterator and moved utxoIterator into utxoset.go.

* [DEV-74] Renamed variables in utxoset.go.

* [DEV-74] Renamed verifyTx to areInputsInUTXO and removed a superfulous test.

* [DEV-74] Fixed bad test logic in TestDiffUTXOSet_addTx.

* [DEV-74] Added a few comments. Added reference-equals checks to clone functions.

* [DEV-74] Moved utxoCollection and utxoDiff into utxoset.go.

* [DEV-74] Wrote explanations for utxoCollection and utxoDiff tests.

* [DEV-74] Wrote explanations for all utxoSet tests besides addTx.

* [DEV-74] Wrote explanations for TestDiffUTXOSet_addTx.

* [DEV-74] Moved the documentation for utxoDiff into utxoset.go.

* [DEV-74] Wrote an explanation on utxoSet.

* [DEV-75] Found a typo.

* [DEV-75] Renamed dag -> virtual, dagView -> virtualBlock.

* [DEV-75] Renamed newDAGView to newVirtualBlock.

* [DEV-75] Moved queries for the genesis block from virtualBlock to BlockDAG.

* [DEV-75] Got rid of chainView height and findFork.

* [DEV-75] Renamed receivers from c to v.

* [DEV-75] Updated initBlockNode to allow for virtual (headerless) nodes, updated dbDAGState to contain multiple tip hashes, implemented virtualBlock.setTips.

* [DEV-75] Got rid of virtualBlock.equals, which was not used anywhere.

* [DEV-75] Got rid of virtualBlock.tip().

* [DEV-75] Got rid of SetTip everywhere except for tests.

* [DEV-75] Got rid of Next().

* [DEV-75] Got rid of Contains().

* [DEV-75] Got rid of HeightRange(), as no one was using it.

* [DEV-75] Made verifyDAG in rpcserver.go not use block height for iteration.

* [DEV-75] Got rid of the part of Manager.Init() that handled "catching up" for side chains, which allowed me to get rid of BlockDAG.BlockByHeight().

* [DEV-75] Dropped support for the RPC command getblockhash since it was getting blocks by their height.

* [DEV-75] Dropped getnetworkhashps since it was reliant on height, fixed another couple of RPC commands to return nextHashes instead of a nextHash, and got rid of nodeByHeight in virtualBlock.

* [DEV-75] Got rid of setTip().

* [DEV-75] Moved blockLocator() out of virtualBlock and into BlockDAG. Also removed TestLocateInventory().

* [DEV-75] Implemented addTip().

* [DEV-75] Cleaned up virtualblock.go a bit.

* [DEV-75] Erased irrelevant tests in virtualblock_test.go. Moved dag-related tests into dag_test.go.

* [DEV-75] Removed unnecessary nil check.

* [DEV-75] Wrote tests for virtualBlock.

* [DEV-75] Fixed bad test, added explanations to tests.

* [DEV-89] Fixed a comment.

* [DEV-89] Fixed another comment.

* [DEV-89] Removed the section in Manager::Init that handled rolling back indexes to the main chain if their tip is an orphaned fork. This could only happen during reorg, which no longer exists. Also removed BlockDAG::MainChainHasBlock, which was no longer used by anyone.

* [DEV-89] Removed the nil check inside initBlockNode() and amended the one place that called it with nil.

* [DEV-89] Renamed the receiver param for BlockDAG from b to dag.

* [DEV-89] Moved fastLog2Floor from dag.go to btcutil/btcmath.go.

* [DEV-89] Renamed tstTip to testTip.

* [DEV-89] Renamed phanom_test.go to phantom_test.go.

* [DEV-89] Fixed comments, renamed mainChainHeight to dagHeight.

* [DEV-89] Rewrote virtualBlock.addTip().

* [DEV-89] Fixed a comment. (chain -> DAG)

* [DEV-89] Fixed another chain -> DAG comment.
 2018-08-09 18:21:03 +03:00
Ori Newman
904f2cf2e3 [DEV-72] Write Blues() 
* [DEV-62] add phantom constructs to blocknode

* [DEV-62] add phantom constructs to blocknode

* [DEV-72] write blues()

* [DEV-72] write blues()

* [DEV-72] write blues()

* [DEV-62] add comments to new phantom constructs in blocknode

* Fixed dbIndexConnectBlock. (#33)

* Fixed dbIndexConnectBlock.

* Removed redundant check in storeFilter.

* Created a new method to BlockHeader: IsGenesis.

* [DEV-71] Implement BlockHeap (#35)

* [DEV-71] Implemented BlockHeap.

* [DEV-71] Removed irrelevant comment.

* [DEV-71] Renamed variables in Pop() and split Less() to multiple lines.

* [DEV-72] write blues()

* [DEV-72] write blues()

* [DEV-72] write blues()

* [DEV-72] write blues tests

* [DEV-72] write blues tests

* [DEV-72] remove relevant past

* [DEV-72] write blues tests

* [DEV-72] write blues tests

* [DEV-72] write blues tests

* [DEV-72] write functions to order blockSet by hash and write blue tests

* [DEV-72] add secret mining and censorship attack tests

* [DEV-72] remove prints

* [DEV-72] remove K from dagconfig.Params

* [DEV-72] remove K from dagconfig.Params

* [DEV-72] change blueScore to uint64

* [DEV-72] block V was missing, so renamed w -> v, x -> w etc

* [DEV-72] use node.String instead of %v

* [DEV-72] block V was missing, so renamed w -> v, x -> w etc

* [DEV-72] add K to dagconfig.Params, and add expected reds to all phantom tests

* [DEV-72] set K=10 and add comments to phantom and phantom tests

* [DEV-72] fix formatting and add comments to TestPhantom

* [DEV-72] fix grammar
 2018-08-02 16:34:40 +03:00
Mike Zak
3b95ec953e Merge remote-tracking branch 'origin/master' into dev-20-primitive-blockdag 2018-07-22 12:43:04 +03:00
stasatdaglabs
b07805cc12 [DEV-54] Rename "chainXXX", "bestXXX", etc. inside blockdag package (#25) 
* [DEV-54] Renamed chainXXX to dagXXX.

* [DEV-54] Additional chain -> dag renamings.

* [DEV-54] Renamed a couple of more "chain"s I missed.
 2018-07-12 12:46:03 +03:00
stasatdaglabs
f4a0ec175d [DEV-45] Update blockdag package tests for primitive blockDAG (#24) 
* [DEV-45] Updated the BlockDB files to contain blocks that could be deserialized.

* [DEV-45] Fixed TestHaveBlock (finally)

* [DEV-45] Fixed TestLocateInventory and everything that's reliant on chainview_test::chainedNodes.

* [DEV-45] Fixed TestChainViewNil.

* [DEV-45] Fixed TestNotifications.

* [DEV-45] Fixed ExampleBlockChain_ProcessBlock.

* [DEV-45] Fixed TestCheckBlockScripts.

* [DEV-45] Fixed TestCheckConnectBlockTemplate.

* [DEV-45] Renamed the BlockDBs to their original names.

* [DEV-45] Skipping TestFullBlocks for until we have implemented Phantom.

* [DEV-45] Deleted a couple of methods that are no longer used. (They were previously used for reorganization)
 2018-07-11 17:09:35 +03:00
stasatdaglabs
79a0c1f124 [DEV-43] Change UTXOViewpoint.BestHash() to Tips() and update all related logic (#15) 
* [DEV-43] Changed BestHash to Tips, fixed logic inside utxoviewpoint.go.

* [DEV-43] Fixed broken references.

* [DEV-43] Replaced blockNode slices and hash slices with BlockSets.

* [DEV-43] Did some renaming, unexported blockSet, and rewrote AppendTip as AddBlock.

* [DEV-43] Removed explicit contains check from AddBlock.
 2018-07-01 15:52:56 +03:00
Stas Boutenko
57e7ad1287 [DEV-34] In blockNode, made parents a pointer slice, got rid of numPrevHashes, and extracted a lookupPreviousNodes function. Changed c-style loops to range loops. 2018-06-24 13:17:43 +03:00
Stas Boutenko
b0d766b7ab [DEV-34] Made a bunch of modifications just to satisfy the compiler. 2018-06-24 11:37:31 +03:00
Stas Boutenko
a26fd21ae7 [DEV-32] Renamed chaincfg to dagconfig and chainhash to daghash. 2018-06-19 18:19:54 +03:00
Stas Boutenko
6d64421965 [DEV-26] Renamed package blockchain to blockdag. 2018-06-17 18:46:47 +03:00