45 Commits

Author SHA1 Message Date
Svarog
fb11981da1
Make kaspactl usable by human beings (#1452)
* Add request_types and their help

* Added command parser

* Updated main to use command if it's specified

* Some progress in making everything work

* Fix command parser for pointers to structs

* Cleanup code

* Enhance usage text

* Fixed typo

* Some minor style fixing, and remove temporary code

* Correctly fallthrough in stringToValue unsupported types

Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2021-01-26 14:22:35 +02:00
stasatdaglabs
756f40c59a
Sync pruning point UTXO sets incrementally instead of all at once (#1431)
* Replaced the content of MsgIBDRootUTXOSetChunk with pairs of outpoint-utxo entry pairs.

* Rename utxoIter to utxoIterator.

* Add a big stinky TODO on an assert.

* Replace pruningStore staging with a UTXO set iterator.

* Reimplement receiveAndInsertIBDRootUTXOSet.

* Extract OutpointAndUTXOEntryPairsToDomainOutpointAndUTXOEntryPairs into domainconverters.go.

* Pass the outpoint and utxy entry pairs to the pruning store.

* Implement InsertCandidatePruningPointUTXOs.

* Implement ClearCandidatePruningPointUTXOs.

* Implement UpdateCandidatePruningPointMultiset.

* Use the candidate pruning point multiset in updatePruningPoint.

* Implement CandidatePruningPointUTXOIterator.

* Use the pruning point utxo set iterator for StageVirtualUTXOSet.

* Defer ClearCandidatePruningPointUTXOs.

* Implement OverwriteVirtualUTXOSet.

* Implement CommitCandidatePruningPointUTXOSet.

* Implement BeginOverwritingVirtualUTXOSet and FinishOverwritingVirtualUTXOSet.

* Implement overwriteVirtualUTXOSetAndCommitPruningPointUTXOSet.

* Rename ClearCandidatePruningPointUTXOs to ClearCandidatePruningPointData.

* Add missing methods to dbManager.

* Implement PruningPointUTXOs.

* Implement RecoverUTXOIfRequired.

* Delete the utxoserialization package.

* Fix compilation errors in TestValidateAndInsertPruningPoint.

* Switch order of operations in the if statements in PruningPointUTXOs so that Next() wouldn't be unnecessarily called.

* Fix missing pruning point utxo set staging and bad slice length.

* Fix no default multiset in InsertCandidatePruningPointUTXOs.

* Make go vet happy.

* Rename candidateXXX to importedXXX.

* Do some more renaming.

* Rename some more.

* Fix bad MsgIBDRootNotFound logic.

* Fix an error message.

* Simplify receiveIBDRootBlock.

* Fix error message in receiveAndInsertIBDRootUTXOSet.

* Do some more renaming.

* Fix merge errors.

* Fix a bug caused by calling iterator.First() unnecessarily.

* Remove databaseContext from stores and don't use a transaction in ClearXXX functions.

* Simplify receiveAndInsertIBDRootUTXOSet.

* Fix offset count in PruningPointUTXOs().

* Fix readOnlyUTXOIteratorWithDiff.First().

* Split handleRequestIBDRootUTXOSetAndBlockFlow into smaller methods.

* Rename IbdRootNotFound to UnexpectedPruningPoint.

* Rename requestIBDRootHash to requestPruningPointHash.

* Rename IBDRootHash to PruningPointHash.

* Rename RequestIBDRootUTXOSetAndBlock to RequestPruningPointUTXOSetAndBlock.

* Rename IBDRootUTXOSetChunk to PruningPointUTXOSetChunk.

* Rename RequestNextIBDRootUTXOSetChunk to RequestNextPruningPointUTXOSetChunk.

* Rename DoneIBDRootUTXOSetChunks to DonePruningPointUTXOSetChunks.

* Rename remaining references to IBD root.

* Fix an error message.

* Add a check for HadStartedImportingPruningPointUTXOSet in commitVirtualUTXODiff.

* Add a check for HadStartedImportingPruningPointUTXOSet in ImportPruningPointUTXOSetIntoVirtualUTXOSet.

* Move FinishImportingPruningPointUTXOSet closer to HadStartedImportingPruningPointUTXOSet.

* Remove reference to pruningStore in utxoSetIterator.

* Pointerify utxoSetIterator receivers.

* Fix bad insert in CommitImportedPruningPointUTXOSet.

* Rename commitImportedPruningPointUTXOSetAll to applyImportedPruningPointUTXOSet.

* Simplify PruningPointUTXOs.

* Add populateTransactionWithUTXOEntriesFromUTXOSet.

* Fix a TODO comment.

* Rename InsertImportedPruningPointUTXOs to AppendImportedPruningPointUTXOs.

* Extract handleRequestPruningPointUTXOSetAndBlockMessage to a separate method.

* Rename stuff in readOnlyUTXOIteratorWithDiff.First().

* Address toAddIterator in readOnlyUTXOIteratorWithDiff.First().

* Call First() before any full iteration on ReadOnlyUTXOSetIterator.

* Call First() before any full iteration on a database Cursor.

* Put StartImportingPruningPointUTXOSet inside the pruning point transaction.

* Make serializeOutpoint and serializeUTXOEntry free functions in pruningStore.

* Fix readOnlyUTXOIteratorWithDiff.First().

* Fix bad validations in importPruningPoint.

* Remove superfluous call to validateBlockTransactionsAgainstPastUTXO.
2021-01-21 17:24:52 +02:00
stasatdaglabs
15af6641fc
Send the IBD root UTXO set in chunks instead of a massive monolythic message (#1412)
* Extract syncPruningPointUTXOSet to a separate method.

* Implement logic to send pruning point utxo set chunks in a loop.

* Replace IBDRootUTXOSetAndBlockMessage with IbdRootUtxoSetChunkMessage.

* Add a new message: RequestNextIBDRootUTXOSetChunk.

* Add a new message: DoneIBDRootUTXOSetChunks.

* Protect HandleRequestIBDRootUTXOSetAndBlock from rogue messages.

* Reimplement receiveIBDRootUTXOSetAndBlock.

* Add CmdDoneIBDRootUTXOSetChunks to the HandleRelayInvs flow.

* Decrease the max message size to 10mb.

* Fix bad step.

* Fix confusion between outgoing/incoming routes.

* Measure how long it takes to send/receive the UTXO set.

* Use LogAndMeasure in handleRequestIBDRootUTXOSetAndBlockFlow.
2021-01-13 18:03:07 +02:00
Svarog
4988817da1
Reject SubmitBlock if the node is in IBD (#1409)
* Reject SubmitBlock if the node is in IBD

* Add comments

* Don't use iota for RejectReason constants, since in .proto those are hard-coded
2021-01-13 15:04:55 +02:00
stasatdaglabs
b195301a99
Add IsIBDPeer to GetConnectedPeerInfoResponse. (#1367)
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-01-06 17:33:51 +02:00
talelbaz
8a309a7d2a
Upgradability mechanisms script version (#1313)
* ''

* ''

* ''

* Changes genesis block version to 0.

* a

* a

* All tests are done.

* All tests passed for changed block version from int32 to uint16

* Adds validation of rejecting blocks with unknown versions.

* Changes txn version from int32 to uint16.

* .

* Adds comments to exported functions.

* Change functions name from ConvertFromRpcScriptPubKeyToRPCScriptPubKey to ConvertFromAppMsgRPCScriptPubKeyToRPCScriptPubKey and from ConvertFromRPCScriptPubKeyToRpcScriptPubKey to ConvertFromRPCScriptPubKeyToAppMsgRPCScriptPubKey

* change comment to "ScriptPublicKey represents a Kaspad ScriptPublicKey"

* delete part (tx.Version < 0) that cannot be exist on the if statement.

* Revert protobuf version.

* Fix a comment.

* Fix a comment.

* Rename a variable.

* Rename a variable.

* Remove a const.

* Rename a type.

* Rename a field.

* Rename a field.

* Remove commented-out code.

* Remove dangerous nil case in DomainTransactionOutput.Clone().

* Remove a constant.

* Fix a string.

* Fix wrong totalScriptPubKeySize in transactionMassStandalonePart.

* Remove a constant.

* Remove an unused error.

* Fix a serialization error.

* Specify version types to be uint16 explicitly.

* Use constants.ScriptPublicKeyVersion.

* Fix a bad test.

* Remove some whitespace.

* Add a case to utxoEntry.Equal().

* Rename scriptPubKey to scriptPublicKey.

* Remove a TODO.

* Rename constants.

* Rename a variable.

* Add version to parseShortForm.

Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: stasatdaglabs <stas@daglabs.com>
2021-01-05 17:50:09 +02:00
stasatdaglabs
acef311fb4
Improve the performance of downloading headers (#1340)
* Add a new message: BlockHeadersMessage.

* Add a new message: BlockHeadersMessage.

* Send a lot of headers as a single message instead of many small messages.

* Keep a short queue of blockHeadersMessages so that there's never a moment when the node is not validating and inserting headers

* Add a missing return statement.

* Remove MsgBlockHeader from payloads.
2021-01-03 17:57:14 +02:00
stasatdaglabs
7dd0188838
Move the heavy lifting in BlockLocator from the syncer to the syncee (#1324)
* Add a new message: BlockLocatorHighestHash.

* Add a new message: IBDBlockLocator.

* Implement HandleIBDBlockLocator.

* Reimplement findHighestSharedBlockHash.

* Make HandleIBDBlockLocator only return hashes that are in the selected parent chain of the target hash.

* Increase the cache sizes of blockRelationStore, reachabilityDataStore, and ghostdagDataStore.

* Fix wrong initial highHash in findHighestSharedBlockHash.

* Make go vet happy.

* Protect against receiving wrong messages when expecting MsgIBDBlockLocatorHighestHash.
2020-12-30 15:44:14 +02:00
Ori Newman
49b6cc6038
Add mutable and immutable header interfaces (#1305)
* Add mutable and immutable header interfaces

* Fix ShouldMine()

* Remove false comment

* Fix Equal signature

* Fix Equal implementation
2020-12-29 13:55:17 +02:00
Ori Newman
cfe013eca7
Add IsHeaderOnly field to BlockVerboseData (#1295) 2020-12-27 18:23:51 +02:00
stasatdaglabs
9d0f513e49
Implement a simple mechanism to stop a miner from mining while kaspad is not synced (#1284)
* Reintroduce isSynced into the GetBlockTemplate response.

* Add a warning for when kaspad is not synced.

* Rephrase a log.
2020-12-24 17:15:44 +02:00
Svarog
05941a76e7
Make DomainHash and TransactionID read-only structs (#1282)
* Increase size of reachability cache

* Change DomainHash to struct with unexported hashArray

* Fixing compilation errors stemming from new DomainHash structure

* Remove obsolete Read/WriteElement methods in appmessage

* Fix all tests

* Fix all tests

* Add comments

* A few renamings

* go mod tidy
2020-12-24 16:15:23 +02:00
Elichai Turkel
273c271771
Remove hash reversal (#1270)
* Remove hash reversing

* Move toBig to pow.go

* Update some tests
2020-12-23 12:42:37 +02:00
Ori Newman
729e3db145
Pruning calculation changes (#1250)
* 1) Calculate pruning point incrementally
2) Add IsValidPruningPoint to pruning manager and consensus
3) Use reachability children for selected child iterator

* Add request IBD root hash flow

* Fix UpdatePruningPointByVirtual and IsValidPruningPoint

* Regenerate messages.pb.go

* Make the pruning point the earliest chain block with finality interval higher than the previous pruning point

* Fix merge errors
2020-12-23 11:37:39 +02:00
Ori Newman
cb9d7e313d
Implement Clone and Equal for all model types (#1155)
* [NOD-1575] Implement Clone and Equal for all model types

* [NOD-1575] Add assertion for transaction ID equality

* [NOD-1575] Use DomainTransaction.Equal to compare to expected coinbase transaction

* [NOD-1575] Add TestDomainBlockHeader_Clone

* [NOD-1575] Don't clone nil values

* [NOD-1575] Add type assertions

* [NOD-1575] Don't clone nil values

* [NOD-1575] Add missing Equals

* [NOD-1575] Add length checks

* [NOD-1575] Update comment

* [NOD-1575] Check length for TransactionAcceptanceData

* [NOD-1575] Explicitly clone nils where needed

* [NOD-1575] Clone tx id

* [NOD-1575] Flip condition

* Nod 1576 make coverage tests for equal clone inside model externalapi (#1177)

* [NOD-1576] Make coverage tests for equal and clone inside model and externalapi

* Some formatting and naming fixes

* Made transactionToCompare type exported

* Added some tests and made some changes to the tests code

* No changes made

* Some formatting and naming changes made

* Made better test coverage for externalapi clone and equal functions

* Changed expected result for two cases

* Added equal and clone functions tests for ghostdag and utxodiff

* Added tests

* [NOD-1576] Implement reachabilitydata equal/clone unit tests

* [NOD-1576]  Full coverage of reachabilitydata equal/clone unit tests

* Made changes and handling panic to transaction_equal_clone_test.go and formating of utxodiff_equal_clone_test.go

* Added recoverForEqual2 for handling panic to transaction_equal_clone_test.go

* [NOD-1576]  Full coverage of transaction equal unit test

* [NOD-1576] Add expects panic

* [NOD-1576] Allow composites in go vet

* [NOD-1576] Code review fixes (#1223)

* [NOD-1576] Code review fixes

* [NOD-1576] Code review fixes part 2

* [NOD-1576] Fix wrong name

Co-authored-by: karim1king <karimkaspersky@yahoo.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: Karim <karim1king@users.noreply.github.com>

* Fix merge errors

* Use Equal where possible

* Use Equal where possible

* Use Equal where possible

Co-authored-by: andrey-hash <74914043+andrey-hash@users.noreply.github.com>
Co-authored-by: karim1king <karimkaspersky@yahoo.com>
Co-authored-by: Karim <karim1king@users.noreply.github.com>
2020-12-22 17:38:54 +02:00
stasatdaglabs
c2cec2f170
Fix the Sequence field in transaction inputs always getting deserialized to MaxUint64. (#1258) 2020-12-22 16:29:38 +02:00
stasatdaglabs
21a459c0f4
Implement virtual selected parent chain RPC methods (#1249)
* [NOD-1579] Rename AcceptedTxIDs to AcceptedTransactionIDs.

* [NOD-1579] Add InsertBlockResult to ValidateAndInsertBlock results.

* [NOD-1593] Rename InsertBlockResult to BlockInsertionResult.

* [NOD-1593] Add SelectedParentChainChanges to AddBlockToVirtual's result.

* [NOD-1593] Implement findSelectedParentChainChanges.

* [NOD-1593] Implement TestFindSelectedParentChainChanges.

* [NOD-1593] Fix a string.

* [NOD-1593] Finish implementing TestFindSelectedParentChainChanges.

* [NOD-1593] Fix merge errors.

* [NOD-1597] Begin implementing UTXOIndex.

* [NOD-1597] Connect UTXOIndex to RPC.

* [NOD-1597] Connect Consensus to UTXOIndex.

* [NOD-1597] Add AcceptanceData to BlockInfo.

* [NOD-1597] Implement UTXOIndex.Update().

* [NOD-1597] Implement add(), remove(), and discard() in utxoIndexStore.

* [NOD-1597] Add error cases to add() and remove().

* [NOD-1597] Add special cases to add() and remove().

* [NOD-1597] Implement commit.

* [NOD-1597] Add a mutex around UTXOIndex.Update().

* [NOD-1597] Return changes to the UTXO from Update().

* [NOD-1597] Add NotifyUTXOsChangedRequestMessage and related structs.

* [NOD-1597] Implement HandleNotifyUTXOsChanged.

* [NOD-1597] Begin implementing TestUTXOIndex.

* [NOD-1597] Implement RegisterForUTXOsChangedNotifications.

* [NOD-1597] Fix bad transaction.ID usage.

* [NOD-1597] Implement convertUTXOChangesToUTXOsChangedNotification.

* [NOD-1597] Make UTXOsChangedNotificationMessage.Removed UTXOsByAddressesEntry instead of just RPCOutpoint so that the client can discern which address was the UTXO removed for.

* [NOD-1597] Collect outpoints in TestUTXOIndex.

* [NOD-1597] Rename RPC stuff.

* [NOD-1597] Add messages for GetUTXOsByAddresses.

* [NOD-1597] Implement HandleGetUTXOsByAddresses.

* [NOD-1597] Implement GetUTXOsByAddresses.

* [NOD-1597] Implement UTXOs().

* [NOD-1597] Implement getUTXOOutpointEntryPairs().

* [NOD-1597] Expand TestUTXOIndex.

* [NOD-1597] Convert SubmitTransaction to use RPCTransaction instead of MsgTx.

* [NOD-1597] Finish implementing TestUTXOIndex.

* [NOD-1597] Add messages for GetVirtualSelectedParentBlueScore.

* [NOD-1597] Implement HandleGetVirtualSelectedParentBlueScore and GetVirtualSelectedParentBlueScore.

* [NOD-1597] Implement TestVirtualSelectedParentBlueScore.

* [NOD-1597] Implement NotifyVirtualSelectedParentBlueScoreChanged.

* [NOD-1597] Expand TestVirtualSelectedParentBlueScore.

* [NOD-1597] Implement notifyVirtualSelectedParentBlueScoreChanged.

* [NOD-1597] Make go lint happy.

* [NOD-1593] Fix merge errors.

* [NOD-1593] Rename findSelectedParentChainChanges to calculateSelectedParentChainChanges.

* [NOD-1593] Expand TestCalculateSelectedParentChainChanges.

* [NOD-1597] Add logs to utxoindex.go.

* [NOD-1597] Add logs to utxoindex/store.go.

* [NOD-1597] Add logs to RPCManager.NotifyXXX functions.

* Implement notifySelectedParentChainChanged.

* Implement TestSelectedParentChain.

* Rename NotifyChainChanged to NotifyVirtualSelectedParentChainChanged.

* Rename GetChainFromBlock to GetVirtualSelectedParentChainFromBlock.

* Remove AcceptanceIndex from the config.

* Implement HandleGetVirtualSelectedParentChainFromBlock.

* Expand TestVirtualSelectedParentChain.

* Fix merge errors.

* Add a comment.

* Move a comment.
2020-12-21 14:43:32 +02:00
Elichai Turkel
45edacfbfa
Replace Double-SHA256 with blake2b and implement domain seperation (#1245)
* Replace default hasher (Double-SHA256) with domain seperated blake2b

* Replace all hashes with domain seperated blake2b

* Update the genesis blocks

* Replace OP_HASH256 with OP_BLAKE2B

* Fix the merkle tree by appending zeros instead of duplicating the hash when there is 1 branch left

* Update tests

* Add a payloadHash function

* Update gitignore to ignore binaries

* Fix a bug in the blake2b opcode
2020-12-21 12:51:45 +02:00
stasatdaglabs
053bb351b5
[NOD-1597] Implement a UTXO index (#1221)
* [NOD-1579] Rename AcceptedTxIDs to AcceptedTransactionIDs.

* [NOD-1579] Add InsertBlockResult to ValidateAndInsertBlock results.

* [NOD-1593] Rename InsertBlockResult to BlockInsertionResult.

* [NOD-1593] Add SelectedParentChainChanges to AddBlockToVirtual's result.

* [NOD-1593] Implement findSelectedParentChainChanges.

* [NOD-1593] Implement TestFindSelectedParentChainChanges.

* [NOD-1593] Fix a string.

* [NOD-1593] Finish implementing TestFindSelectedParentChainChanges.

* [NOD-1593] Fix merge errors.

* [NOD-1597] Begin implementing UTXOIndex.

* [NOD-1597] Connect UTXOIndex to RPC.

* [NOD-1597] Connect Consensus to UTXOIndex.

* [NOD-1597] Add AcceptanceData to BlockInfo.

* [NOD-1597] Implement UTXOIndex.Update().

* [NOD-1597] Implement add(), remove(), and discard() in utxoIndexStore.

* [NOD-1597] Add error cases to add() and remove().

* [NOD-1597] Add special cases to add() and remove().

* [NOD-1597] Implement commit.

* [NOD-1597] Add a mutex around UTXOIndex.Update().

* [NOD-1597] Return changes to the UTXO from Update().

* [NOD-1597] Add NotifyUTXOsChangedRequestMessage and related structs.

* [NOD-1597] Implement HandleNotifyUTXOsChanged.

* [NOD-1597] Begin implementing TestUTXOIndex.

* [NOD-1597] Implement RegisterForUTXOsChangedNotifications.

* [NOD-1597] Fix bad transaction.ID usage.

* [NOD-1597] Implement convertUTXOChangesToUTXOsChangedNotification.

* [NOD-1597] Make UTXOsChangedNotificationMessage.Removed UTXOsByAddressesEntry instead of just RPCOutpoint so that the client can discern which address was the UTXO removed for.

* [NOD-1597] Collect outpoints in TestUTXOIndex.

* [NOD-1597] Rename RPC stuff.

* [NOD-1597] Add messages for GetUTXOsByAddresses.

* [NOD-1597] Implement HandleGetUTXOsByAddresses.

* [NOD-1597] Implement GetUTXOsByAddresses.

* [NOD-1597] Implement UTXOs().

* [NOD-1597] Implement getUTXOOutpointEntryPairs().

* [NOD-1597] Expand TestUTXOIndex.

* [NOD-1597] Convert SubmitTransaction to use RPCTransaction instead of MsgTx.

* [NOD-1597] Finish implementing TestUTXOIndex.

* [NOD-1597] Add messages for GetVirtualSelectedParentBlueScore.

* [NOD-1597] Implement HandleGetVirtualSelectedParentBlueScore and GetVirtualSelectedParentBlueScore.

* [NOD-1597] Implement TestVirtualSelectedParentBlueScore.

* [NOD-1597] Implement NotifyVirtualSelectedParentBlueScoreChanged.

* [NOD-1597] Expand TestVirtualSelectedParentBlueScore.

* [NOD-1597] Implement notifyVirtualSelectedParentBlueScoreChanged.

* [NOD-1597] Make go lint happy.

* [NOD-1593] Fix merge errors.

* [NOD-1593] Rename findSelectedParentChainChanges to calculateSelectedParentChainChanges.

* [NOD-1593] Expand TestCalculateSelectedParentChainChanges.

* [NOD-1597] Add logs to utxoindex.go.

* [NOD-1597] Add logs to utxoindex/store.go.

* [NOD-1597] Add logs to RPCManager.NotifyXXX functions.

* [NOD-1597] Ignore transactions that aren't accepted.

* [NOD-1597] Use GetBlockAcceptanceData instead of GetBlockInfo.

* [NOD-1597] Convert scriptPublicKey to string directly, instead of using hex.

* [NOD-1597] Add a comment.

* [NOD-1597] Guard against calling utxoindex methods when utxoindex is turned off.

* [NOD-1597] Add lock to UTXOs.

* [NOD-1597] Guard against calls to getUTXOOutpointEntryPairs when staging isn't empty.
2020-12-20 17:24:56 +02:00
stasatdaglabs
f90d7d796a
[NOD-1593] Return SelectedParentChainChanged from ValidateAndInsertBlock (#1202)
* [NOD-1579] Rename AcceptedTxIDs to AcceptedTransactionIDs.

* [NOD-1579] Add InsertBlockResult to ValidateAndInsertBlock results.

* [NOD-1593] Rename InsertBlockResult to BlockInsertionResult.

* [NOD-1593] Add SelectedParentChainChanges to AddBlockToVirtual's result.

* [NOD-1593] Implement findSelectedParentChainChanges.

* [NOD-1593] Implement TestFindSelectedParentChainChanges.

* [NOD-1593] Fix a string.

* [NOD-1593] Finish implementing TestFindSelectedParentChainChanges.

* [NOD-1593] Fix merge errors.

* [NOD-1593] Fix merge errors.

* [NOD-1593] Rename findSelectedParentChainChanges to calculateSelectedParentChainChanges.

* [NOD-1593] Expand TestCalculateSelectedParentChainChanges.
2020-12-15 11:37:52 +02:00
Svarog
0d8f7bba40
[NOD-1595] Implement all fields of GetBlockDAGInfo (#1200)
* [NOD-1595] Implement all fields of GetBlockDAGInfo

* [NOD-1595]

* [NOD-1595] Don't swallow errors in GetDifficultyRatio

* [NOD-1595] Change roundingPrecision in GetDifficultyRatio to 2 decimal places
2020-12-09 12:14:15 +02:00
Svarog
e04f76b800
[NOD-1594] Add HeaderCount to GetBlockCount rpc call (#1197) 2020-12-08 19:11:35 +02:00
stasatdaglabs
7f899b0d09
[NOD-1579] Improve the IBD mechanism (#1174)
* [NOD-1579] Remove selected tip hash messages.

* [NOD-1579] Start moving IBD stuff into blockrelay.

* [NOD-1579] Rename relaytransactions to transactionrelay.

* [NOD-1579] Move IBD files into blockrelay.

* [NOD-1579] Remove flow stuff from ibd.go.

* [NOD-1579] Bring back IsInIBD().

* [NOD-1579] Simplify block relay flow.

* [NOD-1579] Check orphan pool for missing parents to avoid unnecessary processing.

* [NOD-1579] Implement processOrphan.

* [NOD-1579] Implement addToOrphanSetAndRequestMissingParents.

* [NOD-1579] Fix TestIBD.

* [NOD-1579] Implement isBlockInOrphanResolutionRange.

* [NOD-1579] Implement limited block locators.

* [NOD-1579] Add some comments.

* [NOD-1579] Specifically check for StatusHeaderOnly in blockrelay.

* [NOD-1579] Simplify runIBDIfNotRunning.

* [NOD-1579] Don't run IBD if it is already running.

* [NOD-1579] Fix a comment.

* [NOD-1579] Rename mode to syncInfo.

* [NOD-1579] Simplify validateAndInsertBlock.

* [NOD-1579] Fix bad SyncStateSynced condition.

* [NOD-1579] Implement validateAgainstSyncStateAndResolveInsertMode.

* [NOD-1579] Use insertModeHeader.

* [NOD-1579] Add logs to TrySetIBDRunning and UnsetIBDRunning.

* [NOD-1579] Implement and use dequeueIncomingMessageAndSkipInvs.

* [NOD-1579] Fix a log.

* [NOD-1579] Fix a bug in createBlockLocator.

* [NOD-1579] Rename a variable.

* [NOD-1579] Fix a slew of bugs in missingBlockBodyHashes and selectedChildIterator.

* [NOD-1579] Fix bad chunk size in syncMissingBlockBodies.

* [NOD-1579] Remove maxOrphanBlueScoreDiff.

* [NOD-1579] Fix merge errors.

* [NOD-1579] Remove a debug log.

* [NOD-1579] Add logs.

* [NOD-1579] Make various go quality tools happy.

* [NOD-1579] Fix a typo in a variable name.

* [NOD-1579] Fix full blocks over header-only blocks not failing the missing-parents validation.

* [NOD-1579] Add an error log about a condition that should never happen.

* [NOD-1579] Check all antiPast hashes instead of just the lowHash's anticone to filter for header-only blocks.

* [NOD-1579] Remove the nil stuff from GetBlockLocator.

* [NOD-1579] Remove superfluous condition in handleRelayInvsFlow.start().

* [NOD-1579] Return a boolean from requestBlock instead of comparing to nil.

* [NOD-1579] Fix a bad log.Debugf.

* [NOD-1579] Remove a redundant check.

* [NOD-1579] Change an info log to a warning log.

* [NOD-1579] Move OnNewBlock out of relayBlock.

* [NOD-1579] Remove redundant exists check from runIBDIfNotRunning.

* [NOD-1579] Fix bad call to OnNewBlock.

* [NOD-1579] Remove an impossible check.

* [NOD-1579] Added a log.

* [NOD-1579] Rename insertModeBlockWithoutUpdatingVirtual to insertModeBlockBody.

* [NOD-1579] Add a check for duplicate headers.

* [NOD-1579] Added a comment.

* [NOD-1579] Tighten a stop condition.

* [NOD-1579] Simplify a log.

* [NOD-1579] Clarify a log.

* [NOD-1579] Move a log.
2020-12-06 16:23:56 +02:00
Mike Zak
9866abb75a [NOD-1583] Split consensusserialization to consensushashing and serialization 2020-12-02 13:18:50 +02:00
Svarog
80c445c78b
[NOD-1551] Optimize binary writes + remove redundant VarInt code (#1163)
* [NOD-1551] Optimize binary writes + remove redundant VarInt code

* [NOD-1551] Remove varInt tests

* [NOD-1551] Fix TestBech32 for Go1.15
2020-11-30 13:45:06 +02:00
Elichai Turkel
3f92ddd827
Add blueScore to RPC GetBlock, and add more INFO logs (#1103)
* Add BlueScore to RPC command GetBlock

* Add info logs when getting new blocks from the p2p
2020-11-18 11:19:12 +02:00
stasatdaglabs
a34091991a [NOD-1538] Fix MinimalNetAdapter and don't insert BlockRelations before making sure the block's parents exist (#1074)
* [NOD-1538] Fix minimal net adapter.

* [NOD-1538] Don't insert block relation until we've validated that the block's parents exist.

* [NOD-1538] Don't hold addressManager in MinimalNetAdapter.

* [NOD-1538] Fix a comment in messages.proto.
2020-11-17 16:00:16 +02:00
stasatdaglabs
eef5e3768c
[NOD-1518] Fix genesis block insertion and integration tests (#1013)
* Revert "[NOD-1500] Delete integration tests"

This reverts commit fcb57a206690a884fa6afb69d5d493282954a8bf.

* [NOD-1518] hashserialization -> consenusserialization

* [NOD-1518] Fix add genesis to virtual

* [NOD-1518] Fix a bug in SerializeCoinbasePayload.

* [NOD-1518] Fix a loop error and make pastMedianTime behave correctly everywhere on genesis.

* [NOD-1518] Fix another bug and an infinite loop.

* [NOD-1518] Fix uninitialized slice.

* [NOD-1518] Fix bad should-commit checks and another infinite loop.

* [NOD-1518] Fix nil serialization.

* [NOD-1518] Rename blockHash to currentBlockHash.

* [NOD-1518] Move the check whether stagedVirtualUTXOSet != nil to the top of commitVirtualUTXODiff.

* [NOD-1518] Simplify utxoDiffStore.Commit.

* [NOD-1518] Unextract resolveBlockStatusAndCheckFinality.

* [NOD-1518] Move no-transactions logic into CalculateIDMerkleRoot.

* [NOD-1518] Remove redundant is-staged check.

* [NOD-1518] Fix merge errors.

* [NOD-1518] Don't write anything if utxoDiffChild is nil.

* [NOD-1518] Stage virtualAcceptanceData and virtualMultiset.

* [NOD-1518] Fix bugs in getBlockTemplate and submitBlock.

* [NOD-1518] Fix bad validation order in validateHeaderInContext.

* [NOD-1518] Fix bug in Next().

* [NOD-1518] Fix nil dereference of subnetworks in AddressCache.

* [NOD-1518] Fix multisetStore.Get returning a pointer to a multiset that is changed in place.

* [NOD-1518] Break on genesis in countSubtrees.

* [NOD-1518] Fix createBlockLocator.

* [NOD-1518] Fix MsgTxToDomainTransaction.

* [NOD-1518] Set MaxTxVersion to 1.

* [NOD-1518] Fix missing error handling, bug in MsgTxToDomainTransaction, and bad subnetwork equality check.

* [NOD-1518] Fix bug in hasUTXOByOutpointFromStagedVirtualUTXODiff.

* [NOD-1518] Remove irrelevant comments.

* [NOD-1518] Generate transactions with sufficient fee in tx_relay_test.

* [NOD-1518] Fix broken RPC handlers.

* [NOD-1518] Fix merge errors.

* [NOD-1518] Fix bad exists check in restorePastUTXO and missing genesis check in CalculatePastUTXOAndAcceptanceData.

* [NOD-1518] Add a comment.

* [NOD-1518] Use a regular mutex instead of a read-write mutex in consensus to avoid dealing with sneaky not-actually-read functions.

* [NOD-1518] Fix a deadlock in GetVirtualSelectedParent.

* [NOD-1518] Fix missing handler registration for CmdHeader.

* [NOD-1518] Fix processHeader calling OnNewBlock and LogBlock. Also fix conversion errors in IBDRootUTXOSetAndBlock.

* [NOD-1518] Fix bad Command() in MsgIBDRootUTXOSetAndBlock.

* [NOD-1518] Fix bad SyncStateMissingUTXOSet logic in resolveSyncState.

* [NOD-1518] Rename mode to syncState.

* [NOD-1518] Fix headers-only blocks coming in after the consensus thinks it's synced.

* [NOD-1518] Fix selectedChildIterator.Next not ignoring virtual, infinite loop in HashSet.Length().

* [NOD-1518] Fix not-properly wrapped IBD blocks.

* [NOD-1518] Fix bad conversion in RequestIBDBlocks.

* [NOD-1518] Fix bad string for CmdRequestHeaders.

* [NOD-1518] Fix bad string for CmdDoneHeaders.

* [NOD-1518] Fix bad Command() for MsgIBDRootNotFound.

* [NOD-1518] Fix bad areHeaderTipsSyncedMaxTimeDifference value.

* [NOD-1518] Add missing string for CmdRequestIBDBlocks.

* [NOD-1518] Fix bad check for SyncStateMissingBlockBodies.

* [NOD-1518] Fix bad timeout durations in tests.

* [NOD-1518] Fix IBD blocks not calling OnNewBlock.

* [NOD-1518] Change when IBD finishes.

* [NOD-1518] Properly clone utxoDiffChild.

* [NOD-1518] Fix merge errors.

* [NOD-1518] Move call to LogBlock to into OnNewBlock.

* [NOD-1518] Return "not implemented" in unimplemented RPC handlers.

* [NOD-1518] Extract cloning of hashes to a method over DomainHash.

* [NOD-1518] Use isHeaderOnlyBlock.

* [NOD-1518] Use constants.TransactionVersion.

* [NOD-1518] Break immediately if we reached the virtual in SelectedChildIterator.

* [NOD-1518] Don't stage nil utxoDiffChild.

* [NOD-1518] Properly check the genesis hash in CalculatePastUTXOAndAcceptanceData.

* [NOD-1518] Explain why we break on current == nil in countSubtrees.

* [NOD-1518] Add a comment explaining why we check against StatusValid in resolveSyncState.

Co-authored-by: Mike Zak <feanorr@gmail.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2020-11-12 15:19:39 +02:00
Ori Newman
7a7821e1c8
[NOD-1313] Refactor AddressManager (#918) (#1049)
* [NOD-1313] Refactor AddressManager (#918)

* [NOD-1313] Refactor AddressManager.

* [NOD-1313]Remove old tests.Minor improvements,fixes.

* [NOD-1313] After merge fixes. Fix import cycle.

* [NOD-1313] Integration tests fixes.

* [NOD-1313] Allocate new slice for the returned key.

* [NOD-1313] AddressManager improvements and fixes.

* Move local and banned addresses to separate lists.
* Move AddressManager config to the separate file.
* Add LocalAddressManager.
* Remove redundant KnownAddress structure.
* Restore local addresses functionality.
* Call initListeners from the LocalAddressManager.
* AddressManager minor improvements and fixes.

* [NOD-1313] Minor fixes.

* [NOD-1313] Implement HandleGetPeerAddresses. Refactoring.

* [NOD-1313] After-merge fixes.

* [NOD-1313] Minor improvements.

* AddressManager: added BannedAddresses() method.
* AddressManager: HandleGetPeerAddresses() add banned addresses
  separately.
* AddressManager: remove addressEntry redundant struct.
* ConnectionManager: checkOutgoingConnections() minor improvements and
  fixes.
* Minor refactoring.
* Minor fixes.

* [NOD-1313] GetPeerAddresses RPC message update

* GetPeerAddresses RPC: add BannedAddresses in the separate field.
* Update protobuf.

* [NOD-1534] Update messages.pb.go

Co-authored-by: Kirill <gammerxpower@gmail.com>
2020-11-12 14:40:41 +02:00
Svarog
37fbdcb453
[NOD-1526] Restore txscript tests (#1019)
* [NOD-1526] Fix compilation errors

* [NOD-1526] Make MsgTx.PayloadHash non-pointer

* [NOD-1526] Fixed many tests

* [NOD-1526] Fix reference_test.go

* [NOD-1526] Removed last instances of appmessage in consensus

* [NOD-1526] No need to check for subnetwork
2020-11-12 10:22:17 +02:00
Ori Newman
23c1ea6c31
[NOD-1525] Implement headers first ibd (#1017)
* [NOD-1525] Implement headers first IBD

* [NOD-1525] Fix proto translators

* [NOD-1525] Register missing flows

* [NOD-1525] Rename SyncStateNormal->SyncStateRelay, simplifiy IBD peer selection code and get rid of panic in FinishIBD

* [NOD-1525] Remove redundant methods from interface
2020-11-10 16:14:51 +02:00
Svarog
281944762d
[NOD-1500] Glue between domain and application (#1007)
* [NOD-1500] Added Domain type and Constructor

* [NOD-1500] Replaced dag+txpool with domain in flowContext

* [NOD-1500] Replaced dag+txpool with domain in flowContext

* [NOD-1500] Converters: domain objects from/to appmessage

* [NOD-1500] Convert hashes to DomainHashes in appmessages

* [NOD-1500] Remove references to daghash in dagconfig

* [NOD-1500] Fixed all appmessage usages of hashes

* [NOD-1500] Update all RPC to use domain

* [NOD-1500] Big chunk of protocol flows re-wired to domain

* [NOD-1500] Finished re-wiring all protocol flows to new Domain

* [NOD-1500] Fix some mempool and kaspaminer compilation errors

* [NOD-1500] Deleted util/{block,tx,daghash} and dbaccess

* [NOD-1500] util.CoinbaseTransactionIndex -> transactionhelper.CoinbaseTransactionIndex

* [NOD-1500] Fix txsigner

* [NOD-1500] Removed all references to util/subnetworkid

* [NOD-1500] Update RpcGetBlock related messages

* [NOD-1500] Many more compilation fixes

* [NOD-1500] Return full list of missing blocks for orphan resolution

* [NOD-1500] Fixed handshake

* [NOD-1500] Fixed flowcontext compilation

* [NOD-1500] Update users of StartIBDIfRequired to handle error

* [NOD-1500] Removed some more fields from RPC

* [NOD-1500] Fix the getBlockTemplate flow

* [NOD-1500] Fix HandleGetCurrentNetwork

* [NOD-1500] Remove redundant code

* [NOD-1500] Remove obsolete notifications

* [NOD-1500] Split MiningManager and Consensus to separate fields in Domain

* [NOD-1500] Update two wrong references to location of txscript

* [NOD-1500] Added comments

* [NOD-1500] Fix some tests

* [NOD-1500] Removed serialization logic from appmessage

* [NOD-1500] Rename database/serialization/messages.proto to dbobjects.proto

* [NOD-1500] Delete integration tests

* [NOD-1500] Remove txsort

* [NOD-1500] Fix tiny bug

* [NOD-1500] Remove rogue dependancy on bchd

* [NOD-1500] Some stylistic fixes
2020-11-08 11:55:54 +02:00
Ori Newman
62ddd8fe1c [NOD-1444] Implement getHeaders RPC command (#944)
* [NOD-1444] Implement getHeaders RPC command

* [NOD-1444] Fix tests and comments

* [NOD-1444] Fix error message

* [NOD-1444] Make GetHeaders propagate header serialization errors

* [NOD-1444] RLock the dag on GetHeaders

* [NOD-1444] Change the error field number to 1000
2020-10-06 11:18:31 +03:00
Svarog
22237a4a8d
[NOD-1439] Added Stop command (#940)
* [NOD-1439] Added Stop command

* [NOD-1439] Added comment explaining why we wait before closing the StopChan

* [NOD-1439] Warnf -> Warn

* [NOD-1439] Rename Stop command to Shut Down

* [NOD-1439] Clean up pauseBeforeShutDown

* [NOD-1439] Add ShutDownRequestMessage case for toRPCPayload

* [NOD-1439] Minor stylistic changes
2020-09-29 15:59:47 +03:00
Ori Newman
f8d0f7f67a
[NOD-1405] Add getMempoolEntries RPC command (#937)
* [NOD-1405] Add getMempoolEntries RPC command

* [NOD-1405] Remove redundant fields from GetMempoolEntryResponseMessage
2020-09-23 15:51:02 +03:00
stasatdaglabs
75e539f4d2
[NOD-1357] Implement getMempoolEntry (#931)
* [NOD-1357] Implement getMempoolEntry.

* [NOD-1357] Fix a nil point reference.

* [NOD-1357] Add a comment above BuildTransactionVerboseData.
2020-09-16 16:53:36 +03:00
Ori Newman
86411a5ca5
[NOD-1389] Add panic on NumParentBlocks if number of parents is greater than 255 (#928) 2020-09-14 12:19:46 +03:00
Elichai Turkel
e66de86a82
[NOD-1387] Some small optimizations to block processing (#926)
* Preallocate when cloning utxoCollection/blockSet

* inline binary serialization to serializeUTXO

* Add a benchmark for serializeUTXO
2020-09-13 18:12:13 +03:00
Svarog
1e08bfca9c
[NOD-1032] Consensus updates pre-pruning (#917)
* [NOD-1249] Add pruning related constants (#869)

* [NOD-1249] Add pruning related constants

* [NOD-1249] Change status suspect to UTXONotVerified

* [NOD-1249] Add TestPruningDepth

* [NOD-1249] Add comment to pruningDepth

* [NOD-1249] Add pruning helper functions (#875)

* [NOD-1249] Added node.blockAtDepth

* [NOD-1249] Added node.finalityPoint()

* [NOD-1249] Add hasFinalityPointInOthersSelectedChain

* [NOD-1249] Add nonFinalityViolatingBlues

* [NOD-1249] Added isInPastOfAny

* [NOD-1249] Updated all calls to blockNode functions that require dag

* [NOD-1249] Add blockNode.reds field and persist it

* [NOD-1249] Add checkObjectiveFinality

* [NOD-1249] Add isViolatingSubjectiveFinality

* [NOD-1249] Added to TestGHOSTDAG check that reds are as expected

* [NOD-1249] Add checkMergeLimit and checkDAGRelations

* [NOD-1249] Invert condition in blockInDepth

* [NOD-1249] Make isInPastOfAny resemble isInPast

* [NOD-1249] Added comments to isInPast and isInPastOfAny

* [NOD-1252] Remove any references to legacy finality (#876)

* [NOD-1032] validateParents: check number of parents and that no parents were manually rejected (#877)

* [NOD-1254] Block verification changes (#882)

* [NOD-1254] Call checkDAGRelations and move it to correct place

* [NOD-1254] Use blockStatuses properly

* [NOD-1254] Add support for setting node's verification flag and set it to UTXONotVerified once block passes basic verification

* [NOD-1254] Check for subjctiveFinality, and for node not being in the selectedParentChain

* [NOD-1254] Make blockStatus an ordinary value - not bit flags

* [NOD-1254] Isolate all utxo-requiring validation into a single separate if branches

* [NOD-1254] Re-arrange connectBloc so that things that happen in UTXO-validated blocks only are all grouped together

* [NOD-1254] Resolve and check selectedParent's status before validatingUTXO

* [NOD-1254] Separate virtualUTXODiff from utxoVerificationOutput

* [NOD-1254] Stylistic fixes

* [NOD-1254] Use dag.index.(Set)BlockNodeStatus instead of accessing node.status

* [NOD-1288] Sub-routinize checkConnectToPastUTXO

* [NOD-1288] Re-write checkConnectToPastUTXO in a way that allows to filter-out invalid transactions

* [NOD-1288] Make checkTxSequenceLock use already calculated utxo outputs

* [NOD-1288] Make checkTxMass use already calculated utxo outputs

* [NOD-1288] Use dag.sigCache for ValidateTransactionScripts

* [NOD-1288] Use checkConnectTransactionToPastUTXO in applyBlueBlocks

* [NOD-1288] Clean-up old code-path from no longer used functions

* [NOD-1288] Skip any irrelevant parts of txo verification if block is genesis

* [NOD-1288] Set  where it should have been

* [NOD-1288] Fix reachability checks to never use the new node + make isInSelectedParentChainOf return true if node == other

* [NOD-1288] invert the condition for isNewSelectedTip

* [NOD-1288] Separate checkIsAccepted to own function, and properly handle coinbase

* [NOD-1288] Don't update utxo-diff for UTXONotVerified parents/tips + Make PrepareBlockForTest resolve the selectedParent's UTXOSet if needed

* [NOD-1288] Include mass off coinbase transactions

* [NOD-1288] Move comment to correct position

* [NOD-1288] If blockAtDepth should return genesis - do it immidiately

* [NOD-1288] Comment-out instead of removeing scriptval_test.go

* [NOD-1288] Rename: entry -> utxoEntry

* [NOD-1288] Remove special function for calcCoinbaseTxMass

* [NOD-1288] Remove redundant check from checkEntryAmounts

* [NOD-1288] Rename: MaxMassPerBlock -> MaxMassAcceptedByBlock

* [NOD-1255] Implement boundedMergeBreakingParents

* [NOD-1255] Implement selectAllowedTips

* [NOD-1255] Integrate virtual parent selection into block verification process

* [NOD-1255] Add node to tips all the time, remove it from candidates and add it's parents if it's disqualified

* [NOD-1255] remove tips from virtaulBlock

* [NOD-1255] Rename: didVirtualParentsChanged -> didVirtualParentsChange

* [NOD-1255] Remove redundant sanity check

* [NOD-1255] Handle a forgotten error

* [NOD-1255] Prettify selectVirtualParents

* [NOD-1255] UpdateTipsUTXO should run over all UTXO-Verified tips, even if they are not parents of virtual

* [NOD-1311] Make isInPast inclusive

* [NOD-1032] Handle finality conflicts (#904)

* [NOD-1312] AddTip should not include finalityViolating and manuallyRejected blocks

* [NOD-1312] Implement resolveFinalityConflict

* [NOD-1312] Implement dag notifications for finalityChanges + updateing DAG state

* [NOD-1312] Added finality conflict rpc boilerplate

* [NOD-1312] Implement handling of getFinalityConflicts + resolveFinalityConflict RPCs

* [NOD-1312] Implement finality conflict related notifications

* [NOD-1312] Move all time to millisecond time

* [NOD-1312] Add comments + unexport some methods

* [NOD-1312] Add clarification in comments

* [NOD-1312] Move updateFinalityConflictResolution to finality_conflicts.go

* [NOD-1312] Rename: currentSelectedTip -> selectedTip

* [NOD-1312] Add long comment to ResolveFinalityConflict

* [NOD-1312] Convert areAllInSelectedParentChainOf into isInSelectedParentChainOfAll

* [NOD-1312] Rename chainUpdates (type) -> selectedParentChainUpdates, to distinguish from the variable chainUpdates

* [NOD-1032] Make all blockdag tests compile

* [NOD-1278] Fix finality-related tests (#910)

* [NOD-1032] Don't return node.dag.genesis from blockAtDepth because it might still not exist

* [NOD-1032] Actually add a tip in dag.addTip

* [NOD-1278] Add transaction to utxo set if it's coinbase

* [NOD-1278] Use VirtualParentHashes instead of TipHashes where appropriate

* [NOD-1278] If no valid virtual parent candidates - return error, don't wait for panic

* [NOD-1278] Transition TestCalcSequenceLock from newTestDAG to DAGSetup

* [NOD-1278] Fix .bluest() tie-breaker

* [NOD-1278] Remove feeData structure, as it no longer works, store feeData in acceptanceData

* [NOD-1278] Remove dag parameter from blockNode methods

* [NOD-1278] Fix TestBlueBlockWindow

* [NOD-1278] Don't subject selectedParent to MaxMergeSet

* [NOD-1278] se PrepareAndProcessBlockForTest instead of .addTip in TestSelectedPath

* [NOD-1278] Fixed TestDAGStateSerialization

* [NOD-1278] Fix TestAcceptanceIndexRecover

* [NOD-1278] Fix TestCheckConnectBlockTemplate

* [NOD-1278] Fix TestChainUpdates

* [NOD-1278] Fix and rename TestVirtualBlock -> TestTips

* [NOD-1278] Rename checkIsAccepted -> maybeAcceptTx

* [NOD-1278] Re-activate TestDoubleSpends

* Revert "[NOD-1278] Fixed TestDAGStateSerialization"

This reverts commit 845095d6de7207b07cf819d05f3f38ad94da9cf6.

* [NOD-1278] Remove dag parameter from expectedCoinbaseTransaction

* [NOD-1348] Implemented simplified Finality Conflict Resolution scheme (#911)

* [NOD-1348] Rename functions according to Research spec

* [NOD-1348] Added blockSet.areAllIn

* [NOD-1348] Implemented simplified finality conflict resolution scheme

* [NOD-1348] Refactorings and fixes in selectVirtualParents

* [NOD-1278] Fix bugs discovered by unit-tests + Fix unit-tests (#916)

* Updated to version v0.3.1

* [NOD-858] Don't switch sync peer if the syncing process hasn't yet started with the current sync peer (#700)

* [NOD-858] Don't switch sync peer if the syncing process hasn't yet started with the current sync peer

* [NOD-858] SetShouldSendBlockLocator(false) on OnBlockLocator

* [NOD-858] Rename shouldSendBlockLocator->wasBlockLocatorRequested

* [NOD-858] Move panic to shouldReplaceSyncPeer

* [NOD-869] Add a print after os.Exit(1) to see if it is ever called (#701)

* [NOD-1238] Fix acceptance index never being initialized. (#859)

* [NOD-1278] Genesis never violates finality

* [NOD-1348] Refactorings and fixes in selectVirtualParents

* [NOD-1278] Don't call dag.selectVirtualParents for genesis

* [NOD-1278] Properly organize errors in maybeAcceptBlock

* [NOD-1278] updateTipsUTXO should only run on tips whose status is

* [NOD-1278] updateTipsUTXO should only run on tips whose status is `valid`

* [NOD-1278] Fix TestDoubleSpends

* [NOD-1278] Fix TestDAGIndexFailedStatus

* [NOD-1278] IsFinalizedTransaction should use uint64 everywhere

* [NOD-1278] If tx is coinbase and not selectedParent - don't update pastUTXO

* [NOD-1278] Store tips and validTips separately

* [NOD-1278] Store ValidTips and VirtualParents in dagState

* [NOD-1278] Fix TestProcessOrphans

* [NOD-1278] Fix TestProcessOrphans

* [NOD-1278] Fix TestOrderInDiffFromAcceptanceData

* [NOD-1278] Fix TestHelp

* [NOD-1278] Remove mining.PrepareBlockForTest; use blockdag.PrepareBlockForTest instead

* [NOD-1278] Explicitly disallow chained transactions

* [NOD-1278]

* [NOD-1278] Fix some comments

Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
Co-authored-by: Yuval Shaul <yuval.shaul@gmail.com>

* [NOD-1355] Add unit-test for finality + When resolving finalityConflict - make sure the block that will come out selectedTip is statusValid (#919)

* [NOD-1355] Added test for finality

* [NOD-1355] When resolving finalityConflict - make sure the block that will come out selectedTip is statusValid

* [NOD-1032] Renames: anything about inputsWithReferencedUTXOEntries -> remove 'Referenced'

* [NOD-1032] Don't ignore non-rule errors

* [NOD-1032] Fix comment

* [NOD-1032] Enhanced comments on TestChainUpdates

* [NOD-1032] Remove scriptval_test.go

* [NOD-1032] Extracted isNewSelectedTip to a method

* [NOD-1032] Use dag.Now() instead of mstime.Now()

* [NOD-1032] Print block status when accepting block

* [NOD-1032] Add comment explaining boundedMergeBreakingParents

* [NOD-1032] Enhanced test and imporved comments in TestFinality

* [NOD-1032] Rename: Objective finality -> bounded merge depth

* [NOD-1032] No need to check that validTips are valid

* [NOD-1032] Remove dag from arguments of updateDiffAndDiffChild

* [NOD-1032] Correct variable names in LookupNodes

[NOD-1032] Correct variable names in LookupNodes

* [NOD-1032] Fix some comments

* [NOD-1032] Some style changes

* [NOD-1032] Refactor isAnyInPastOf

* [NOD-1032] Enhance comment in updateVirtualParents

* [NOD-1032] Flip condition in updateVirtualParents

* [NOD-1032] Stylistic and grammatic fixes in dag.go and dag_test.go

* [NOD-1032] Explain why updateSelectedParentSet receives geneses on init

* [NOD-1032] Remove ErrParentManuallyRejected

* [NOD-1032] Added wrapper for resolveNodeStatus that creates a special transaction for it

* [NOD-1032] Rename: statusUTXONotVerified -> statusUTXOPendingVerification

* [NOD-1032] Use virtual parents in CurrentBits()

* [NOD-1032] rename: isViolatingSubjectiveFinality -> isViolatingFinality

* [NOD-1032] Move updateVirtualAndTips to applyDAGChanges

* [NOD-1032] Invert condition for isFinalityPointInPast

* [NOD-1032] Fix antiPastBetween isInPast became inclusive

* [NOD-1032] Remove redundant call for addTip

* [NOD-1032] Use calcCoinbaseTxMass where appropriate

* [NOD-1032] Remove time fields from conflict notifications

* [NOD-1032] Assign the correct thing to i

* [NOD-1032] unify checkOutputsAmounts and checkTxOutputAmounts

* [NOD-1032] Cleanup in CheckTransactionInputsAndCalulateFee

* [NOD-1032] Fixed some style and comments

* [NOD-1032] If selectedParent is disqualifiedFromChain - validateAndApplyUTXOSet should return this as a ruleError

* [NOD-1032] Set the status in resolveNodeStatus

* [NOD-1032] Correct comment on boundedMergeBreakingParents

* [NOD-1032] Fix a typo.

* [NOD-1032] Update a variable name.

* [NOD-1032] Fix a comment.

* [NOD-1032] Fix merge errors.

* [NOD-1032] Add VirtualParentHashes to getBlockDagInfo.

* [NOD-1032] Update handleGetBlockTemplate.

* [NOD-1032] Comment out all the old RPC stuff.

* [NOD-1032] Remove irrelevant type.

* [NOD-1032] Implement ResolveFinalityConflict.

* [NOD-1032] Remove irrelevant comments.

* [NOD-1032] Implement NotifyFinalityConflicts.

* [NOD-1032] Add FinalityConflictNotification and FinalityConflictResolvedNotification.

* [NOD-1032] Finish implementing finality conflict notifications.

* [NOD-1032] Remove old RPC stuff.

* [NOD-1032] Fix grammar in a comment.

Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
Co-authored-by: Yuval Shaul <yuval.shaul@gmail.com>
Co-authored-by: stasatdaglabs <stas@daglabs.com>
2020-09-13 17:49:59 +03:00
stasatdaglabs
26b3ce4eb7
[NOD-1367] Implement getBlockCount and getBlockDagInfo (#920)
* [NOD-1367] Implement GetBlockCount.

* [NOD-1367] Implement GetBlockDagInfo.
2020-09-09 14:10:19 +03:00
stasatdaglabs
4c915f12b7
[NOD-1319] Reimplement kaspad's RPC in gRPC (#914)
* [NOD-1319] Create the protorpc package.

* [NOD-1319] Use a general ClientStream in closeSend.

* [NOD-1319] Decouple p2pServer from gRPCServer.

* [NOD-1319] Begin implementing rpcServer.

* [NOD-1319] Move grpcStream to grpc_connection.go.

* [NOD-1319] Fold the rpc messages.proto into a common message.proto.

* [NOD-1319] Remove code duplication in MessageStream.

* [NOD-1319] Rename methods in netadapter.

* [NOD-1319] Rename message_xxx to p2p_xxx.

* [NOD-1319] Section off p2p messages from rpc messages in messages.proto.

* [NOD-1319] Split toPayload to a p2p part and and rpc part.

* [NOD-1319] Rename msgxxx.go to p2p_msgxx.go in the appmessage package.

* [NOD-1319] Implement GetCurrentVersionRequestMessage and GetCurrentVersionResponseMessage.

* [NOD-1319] Implement toAppMessage and fromAppMessage for getCurrentNetwork

* [NOD-1319] Make a temporary workaround so that tests pass.

* [NOD-1319] Begin implementing the rpc manager.

* [NOD-1319] Implement an initial routerInitializer for rpc.

* [NOD-1319] Rename the spawn in routerInitializer.

* [NOD-1319] Implement an RPC context.

* [NOD-1319] Move the actual handlers to a separate package.

* [NOD-1319] Use the correct value for the GetCurrentNetwork response.

* [NOD-1319] Fix some names.

* [NOD-1319] Begin implementing a cli rpc client.

* [NOD-1319] Implement connecting to the RPC server.

* [NOD-1319] Make sure that connecting to the server and sending/receiving messages works.

* [NOD-1319] Make kaspactl2 speak in json strings.

* [NOD-1319] Finish implementing kaspactl2.

* [NOD-1319] Remove debug messages.

* [NOD-1319] Properly handle errors in rpc.go.

* [NOD-1319] Move the grpc client to a separate package.

* [NOD-1319] Extract Post out of PostString.

* [NOD-1319] Implement PostAppMessage.

* [NOD-1319] Stub out submitBlock.

* [NOD-1319] Stub out getBlockTemplate.

* [NOD-1319] Combine request and reponse files.

* [NOD-1319] Implement submitBlock.

* [NOD-1319] Implement returning errors from RPC.

* [NOD-1319] Begin implementing getBlockTemplate.

* [NOD-1319] Add missing field in GetBlockTemplateRequestMessage.

* [NOD-1319] Implement a minimal getBlockTemplate.

* [NOD-1319] Add getBlockTemplate stuff to grpc.

* [NOD-1319] Implement the rest of getBlockTemplate.

* [NOD-1319] Add block/transaction added handlers to the protocol manager.

* [NOD-1319] Implement NotifyTransactionAddedToMempool.

* [NOD-1319] Implement NotifyBlockAddedToDAG.

* [NOD-1319] Connect block/transaction added handlers.

* [NOD-1319] Add notifyBlockAdded.

* [NOD-1319] Add a notification system.

* [NOD-1319] Improve the notification system.

* [NOD-1319] Add a block added listener stub.

* [NOD-1319] Add BlockAddedNotificationMessage.

* [NOD-1319] Finish implementing HandleNotifyBlockAdded.

* [NOD-1319] Println instead of Print in kaspactl2.

* [NOD-1319] Remove unused flags in kaspactl2.

* [NOD-1319] Make kaspaminer work with the new RPC.

* [NOD-1319] Fix a bad log.

* [NOD-1319] Make kaspaminer work.

* [NOD-1319] Disconnect the old RPC.

* [NOD-1319] Move grpcclient.go.

* [NOD-1319] Begin generalizing the rpcClient.

* [NOD-1319] Move errors to the side of the payload.

* [NOD-1319] Add errors to appmessage.

* [NOD-1319] Move AttachRouter to grpcclient.

* [NOD-1319] Fix kaspaminer not handling responses.

* [NOD-1319] Properly handle blockAddedNotifications.

* [NOD-1319] Move errors into individual response objects.

* [NOD-1319] Begin replacing the RPC client in the integration tests.

* [NOD-1319] Implement GetPeerAddresses.

* [NOD-1319] Implement GetPeerAddresses.

* [NOD-1319] Fix setOnBlockAddedHandler.

* [NOD-1319] Remove the old kaspactl.

* [NOD-1319] Move ConvertGetBlockTemplateResultToBlock to the mining package.

* [NOD-1319] Implement getSelectedTipHash.

* [NOD-1319] Simplify testRPCRouter.

* [NOD-1319] Write stubs for the required test RPC commands.

* [NOD-1319] Implement a minimal getMempoolEntry.

* [NOD-1319] Implement a minimal getMempoolEntry.

* [NOD-1319] Implement getConnectedPeerInfo.

* [NOD-1319] Delete the old RPC.

* [NOD-1319] Fix a fromAppMessage.

* [NOD-1319] Implement connectToPeer.

* [NOD-1319] Fix a bug in registerForBlockAddedNotifications.

* [NOD-1319] Fix a deadlock in closing notification listeners.

* [NOD-1319] Fix merge errors.

* [NOD-1319] Fix an import.

* [NOD-1319] Properly handle errors in grpcclient.

* [NOD-1319] Fix TestIBD.

* [NOD-1319] Prevent kaspaminer from running when not connected.

* [NOD-1319] Implement sendRawTransaction.

* [NOD-1319] Implement sendRawTransaction in the client.

* [NOD-1319] Extract a general RPC client from the integration test RPC client.

* [NOD-1319] Use the general RPC client for the miner.

* [NOD-1319] Move the rpcclient package out of netadapter.

* [NOD-1319] Normalize onBlockAdded.

* [NOD-1319] Begin implementing notifyChainChanged.

* [NOD-1319] Implement the model for notifyChainChanged.

* [NOD-1319] Implement conversions for notifyChainChanged.

* [NOD-1319] Implement HandleNotifyChainChanged.

* [NOD-1319] Normalize notifications.

* [NOD-1319] Implement RegisterForChainChangedNotifications.

* [NOD-1319] Begin connecting blockdag's chain-changed notification with the RPC.

* [NOD-1319] Finish implementing notifyChainChanged.

* [NOD-1319] Implement getBlockHex.

* [NOD-1319] Rename getBlockHex to getBlock.

* [NOD-1319] Implement the verbose model for getBlock.

* [NOD-1319] Implement buildBlockVerboseData.

* [NOD-1319] Implement buildTransactionVerboseData.

* [NOD-1319] Move verboseData stuff to verbosedata.go.

* [NOD-1319] Add includeTransactionVerboseData.

* [NOD-1319] Begin implementing getSubnetwork.

* [NOD-1319] Finish implementing getSubnetwork.

* [NOD-1319] Begin implementing getChainFromBlock.

* [NOD-1319] Finish implementing getChainFromBlock.

* [NOD-1319] Begin implementing getBlocks.

* [NOD-1319] Finish implementing getBlocks.

* [NOD-1319] Fix bad responses in HandleNotifyChainChanged.

* [NOD-1319] Fix bugs in verbosedata.go.

* [NOD-1319] Fix more bugs in verbosedata.go.

* [NOD-1319] Make go vet happy.

* [NOD-1319] Extract handleBlockDAGNotifications to a method.

* [NOD-1319] Add a newline.

* [NOD-1319] Use peers instead of connections to check if connected.

* [NOD-1319] Add a comment.

* [NOD-1319] Lock the dag lock in getBlock.

* [NOD-1319] Rename netAdapter.connections to p2pConnections.

* [NOD-1319] In protowire, rename wireXXX to protoXXX.

* [NOD-1319] Rename PostString to PostJSON.

* [NOD-1319] Disallow empty transactions in SendRawTransaction.

* [NOD-1319] Disallow empty blocks in SubmitBlocks.

* [NOD-1319] Add SetLogger.

* [NOD-1319] Fix an error message.

* [NOD-1319] Fix an error message.

* [NOD-1319] Rename testTimeout to rpcTimeout.

* [NOD-1319] Rename SendRawTransaction to SubmitTransaction.

* [NOD-1319] Rename ConnectToPeer to AddPeer.

* [NOD-1319] Add missing longPollID to request.

* [NOD-1319] Rename ChainChangedChainBlock to ChainBlock.

* [NOD-1319] Rename Vin and Vout.

* [NOD-1319] Implement RPCErrorf.

* [NOD-1319] Fix RPCErrorf's comment.

* [NOD-1319] Remove unused flags in kaspaminer.
2020-09-07 14:35:40 +03:00
stasatdaglabs
15b545ee2b
[NOD-592] Remove TODOs and XXXs from the codebase (#890)
* [NOD-592] Remove TODOs related to fake nonces.

* [NOD-592] Remove irrelevant TODOs from handleRescanBlocks and parseTxAcceptedVerboseNtfnParams.

* [NOD-592] Fix TODO in handleGetTxOut.

* [NOD-592] Remove irrelevant TODO from updateAddress.

* [NOD-592] Move StandardVerifyFlags to a separate file.

* [NOD-592] Remove TODOs in sign.go.

* [NOD-592] Remove TODO in scriptval_test.go.

* [NOD-592] Remove TODO in reachabilitystore.go.

* [NOD-592] Remove XXXs.

* [NOD-592] Fix a comment.

* [NOD-557] Move AddAddressByIP out of AddressManager since it's used only for tests..

* [NOD-557] Remove rescan blocks.

* [NOD-592] Fix handleGetTxOut.
2020-08-23 17:17:06 +03:00
stasatdaglabs
667b2d46e9
[NOD-557] Remove RegTest (#889)
* [NOD-557] Remove regTest network.

* [NOD-557] Remove remaining references to regTest.

* [NOD-557] Move newHashFromStr from params.go to params_test.go.

* [NOD-557] Rename test to network in register_test.go.

* [NOD-557] Replaced removed tests in TestDecodeAddressErrorConditions.
2020-08-23 15:38:27 +03:00
stasatdaglabs
5d20772f94
[NOD-1293] Fix kaspad sending 127.0.0.1 in its msgVersion (#886)
* [NOD-1293] Use addressManager's GetBestLocalAddress.

* [NOD-1293] Copy the initListeners function from the old p2p to the address manager.

* [NOD-1293] Remove debug logs.

* [NOD-1293] Remove unused import.

* [NOD-1293] Fix a comment.
2020-08-23 13:11:48 +03:00
stasatdaglabs
d14809694f
[NOD-1223] Reorganize directory structure (#874)
* [NOD-1223] Delete unused files/packages.

* [NOD-1223] Move signal and limits to the os package.

* [NOD-1223] Put database and dbaccess into the db package.

* [NOD-1223] Fold the logs package into the logger package.

* [NOD-1223] Rename domainmessage to appmessage.

* [NOD-1223] Rename to/from DomainMessage to AppMessage.

* [NOD-1223] Move appmessage to the app packge.

* [NOD-1223] Move protocol to the app packge.

* [NOD-1223] Move the network package to the infrastructure packge.

* [NOD-1223] Rename cmd to executables.

* [NOD-1223] Fix go.doc in the logger package.
2020-08-18 10:26:39 +03:00