* Pruning headers p2p basic structure
* Remove headers-first
* Fix consensus tests except TestValidateAndInsertPruningPointWithSideBlocks and TestValidateAndInsertImportedPruningPoint
* Add virtual genesis
* Implement PruningPointAndItsAnticoneWithMetaData
* Start fixing TestValidateAndInsertImportedPruningPoint
* Fix TestValidateAndInsertImportedPruningPoint
* Fix BlockWindow
* Update p2p and gRPC
* Fix all tests except TestHandleRelayInvs
* Delete TestHandleRelayInvs parts that cover the old IBD flow
* Fix lint errors
* Add p2p_request_ibd_blocks.go
* Clean code
* Make MsgBlockWithMetaData implement its own representation
* Remove redundant check if highest share block is below the pruning point
* Fix TestCheckLockTimeVerifyConditionedByAbsoluteTimeWithWrongLockTime
* Fix comments, errors ane names
* Fix window size to the real value
* Check reindex root after each block at TestUpdateReindexRoot
* Remove irrelevant check
* Renames and comments
* Remove redundant argument from sendGetBlockLocator
* Don't delete staging on non-recoverable errors
* Renames and comments
* Remove redundant code
* Commit changes inside ResolveVirtual
* Add comment to IsRecoverableError
* Remove blocksWithMetaDataGHOSTDAGDataStore
* Increase windows pagefile
* Move DeleteStagingConsensus outside of defer
* Get rid of mustAccepted in receiveBlockWithMetaData
* Ban on invalid pruning point
* Rename interface_datastructures_daawindowstore.go to interface_datastructures_blocks_with_meta_data_daa_window_store.go
* * Change GetVirtualSelectedParentChainFromBlockResponseMessage and VirtualSelectedParentChainChangedNotificationMessage to show only added block hashes
* Remove ResolveVirtual
* Use externalapi.ConsensusWrapper inside MiningManager
* Fix pruningmanager.blockwithmetadata
* Set pruning point selected child when importing the pruning point UTXO set
* Change virtual genesis hash
* replace the selected parent with virtual genesis on removePrunedBlocksFromGHOSTDAGData
* Get rid of low hash in block locators
* Remove +1 from everywhere we use difficultyAdjustmentWindowSize and increase the default value by one
* Add comments about consensus wrapper
* Don't use separate staging area when resolving resolveBlockStatus
* Fix netsync stability test
* Fix checkResolveVirtual
* Rename ConsensusWrapper->ConsensusReference
* Get rid of blockHeapNode
* Add comment to defaultDifficultyAdjustmentWindowSize
* Add SelectedChild to DAGTraversalManager
* Remove redundant copy
* Rename blockWindowHeap->calculateBlockWindowHeap
* Move isVirtualGenesisOnlyParent to utils
* Change BlockWithMetaData->BlockWithTrustedData
* Get rid of maxReasonLength
* Split IBD to 100 blocks each time
* Fix a bug in calculateBlockWindowHeap
* Switch to trusted data when encountering virtual genesis in blockWithTrustedData
* Move ConsensusReference to domain
* Update ConsensusReference comment
* Add comment
* Rename shouldNotAddGenesis->skipAddingGenesis
* Update constants
* Add to transaction SigOpCount
* Update mass calculation, and move it from InContext to InIsolation
* Update block validation accordingly
* Add SigOpCount validation during TransactionInContext
* Remove checking of mass vs maxMassAcceptedByBlock from consensusStateManager
* Update mining manager with latest changes
* Add SigOpCount to MsgTx.Copy()
* Fix initTestTransactionAcceptanceDataForClone
* Fix all tests in transaction_equal_clone_test.go
* Fix TestBlockMass
* Fix tests in transactionvalidator package
* Add SigOpCount to sighash
* Fix TestPruningDepth
* Fix problems in libkaspawalelt
* Fix integration tests
* Fix CalculateSignatureHash tests
* Remove remaining places talking about block size
* Add sanity check to checkBlockMass to make sure all transactions have their mass filled
* always add own sigOpCount to sigHash
* Update protowire/rpc.md
* Start working on removing any remaining reference to block/tx size
* Update rpc transaction verbose data to include mass rather then size
* Convert verboseData and block size check to mass
* Remove remaining usages of tx size in mempool
* Move transactionEstimatedSerializedSize to transactionvalidator
* Add PopulateMass to fakeRelayInvsContext
* Move PopulateMass to beggining of ValidateAndInsertTransaction + fix in it
* Assign mass a new number for backward-compatibility
* Adds name to route, and writes it in every error message
* Update all calls with route name
* Fixed a few missed points
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
* Added model and stubs for all main methods
* Add constructors to all main objects
* Implement BlockCandidateTransactions
* implement expireOldTransactions and expireOrphanTransactions
* Rename isHighPriority to neverExpires
* Add stub for checkDoubleSpends
* Revert "Rename isHighPriority to neverExpires"
This reverts commit b2da9a4a00c02fb380d2518cf54fa16257bd8423.
* Imeplement transactionsOrderedByFeeRate
* Orphan maps should be idToOrphan
* Add error.go to mempool
* Invert the condition for banning when mempool rejects a transaction
* Move all model objects to model package
* Implement getParentsInPool
* Implemented mempoolUTXOSet.addTransaction
* Implement removeTransaction, remove sanity checks
* Implemented mempoolUTXOSet.checkDoubleSpends
* Implemented removeOrphan
* Implement removeOrphan
* Implement maybeAddOrphan and AddOrphan
* Implemented processOrphansAfterAcceptedTransaction
* Implement transactionsPool.addTransaction
* Implement RemoveTransaction
* If a transaction was removed from the mempool - update it's redeemers in orphan pool as well
* Use maximumOrphanTransactionCount
* Add allowOrphans to ValidateAndInsertTransaction stub
* Implement validateTransaction functions
* Implement fillInputs
* Implement ValidateAndInsertTransaction
* Implement HandleNewBlockTransactions
* Implement missing mempool interface methods
* Add comments to exported functions
* Call ValidateTransactionInIsolation where needed
* Implement RevalidateHighPriorityTransactions
* Rewire kaspad to use new mempool, and fix compilation errors
* Update rebroadcast logic to use new structure
* Handle non-standard transaction errors properly
* Add mutex to mempool
* bugfix: GetTransaction panics when ok is false
* properly calculate targetBlocksPerSecond in config.go
* Fix various lint errors and tests
* Fix expected text in test for duplicate transactions
* Skip the coinbase transaction in HandleNewBlockTransactions
* Unorphan the correct transactions
* Call ValidateTransactionAndPopulateWithConsensusData on unorphanTransaction
* Re-apply policy_test as check_transactions_standard_test
* removeTransaction: Remove redeemers in orphan pool as well
* Remove redundant check for uint64 < 0
* Export and rename isDust -> IsTransactionOutputDust to allow usage by rothschild
* Add allowOrphan to SubmitTransaction RPC request
* Remove all implementation from mempool.go
* tidy go mod
* Don't pass acceptedOrphans to handleNewBlockTransactions
* Use t.Errorf in stead of t.Fatalf
* Remove minimum relay fee from TestDust, as it's no longer configurable
* Add separate VirtualDAASCore method for faster retrieval where it's repeated multiple times
* Broadcast all transactions that were accepted
* Don't re-use GetVirtualDAAScore in GetVirtualInfo - this causes a deadlock
* Use real transaction count, and not Orphan
* Get mempool config from outside, incorporating values received from cli
* Use MinRelayFee and MaxOrphanTxs from global kaspad config
* Add explanation for the seemingly redundant check for transaction version in checkTransactionStandard
* Update some comment
* Convert creation of acceptedTransactions to a single line
* Move mempoolUTXOSet out of checkDoubleSpends
* Add test for attempt to insert double spend into mempool
* fillInputs: Skip check for coinbase - it's always false in mempool
* Clarify comment about removeRedeemers when removing random orphan
* Don't remove high-priority transactions in limitTransactionCount
* Use mempool.removeTransaction in limitTransactionCount
* Add mutex comment to handleNewBlockTransactions
* Return error from limitTransactionCount
* Pluralize the map types
* mempoolUTXOSet.removeTransaction: Don't restore utxo if it was not created in mempool
* Don't evacuate from orphanPool high-priority transactions
* Disallow double-spends in orphan pool
* Don't use exported (and locking) methods from inside mempool
* Check for double spends in mempool during revalidateTransaction
* Add checkOrphanDuplicate
* Add orphan to acceptedOrphans, not current
* Add TestHighPriorityTransactions
* Fix off-by-one error in limitTransactionCount
* Add TestRevalidateHighPriorityTransactions
* Remove checkDoubleSpends from revalidateTransaction
* Fix TestRevalidateHighPriorityTransactions
* Move check for MaximumOrphanCount to beggining of maybeAddOrphan
* Rename all map type to singulateToSingularMap
* limitOrphanPool only after the orphan was added
* TestDoubleSpendInMempool: use createChildTxWhenParentTxWasAddedByConsensus instead of createTransactionWithUTXOEntry
* Fix some comments
* Have separate min/max transaction versions for mempool
* Add comment on defaultMaximumOrphanTransactionCount to keep it small as long as we have recursion
* Fix comment
* Rename: createChildTxWhenParentTxWasAddedByConsensus -> createChildTxWhereParentTxWasAddedByConsensus
* Handle error from createChildTxWhereParentTxWasAddedByConsensus
* Rename createChildTxWhereParentTxWasAddedByConsensus -> createChildAndParentTxsAndAddParentToConsensus
* Convert all MaximumXXX constants to uint64
* Add comment
* remove mutex comments
* Add notifyVirtualDaaScoreChanged to protowire.
* Add notifyVirtualDaaScoreChanged to the rest of kaspad.
* Add notifyVirtualDaaScoreChanged to the rest of kaspad.
* Test the DAA score notification in TestVirtualSelectedParentBlueScore.
* Rename TestVirtualSelectedParentBlueScore to TestVirtualSelectedParentBlueScoreAndVirtualDAAScore.
(cherry picked from commit 83e631548f206aaf2da5b2103681c715bc250c75)
* Add BlockHash optional parameter to EstimateNetworkBlockHashesPerSecond
* Allow to pass '-' for optional values in kaspactl
* Solve a division-by-zero in estimateNetworkHashesPerSecond
* Add BlockHash to toAppMessage/fromAppMessage functions
* Rename: topHash -> StartHash
* Return proper error message if provided startHash doesn't deserialize into a hash
* Add a flag for sanity check pruning point utxo set and do the sanity check only if it's enabled
* add description to EnableSanityCheckPruningUTXOSet
* review fix
Co-authored-by: Svarog <feanorr@gmail.com>
* Add windows to the CI
* Cast syscall.Stdin into an integer
* DataDir -> AppDir in service_windows.go
* Rename mempool-limits package to something non-main
* Close database after re-assigining to it
* Up rpcTimout to 10 seconds
* Copy some boilerplate from the other stability tests.
* Fix a copy+paste error in run.sh.
* Copy over some stability test boilerplate go code.
* Run kaspad in the background.
* Catch panics and initialize the RPC client.
* Mine enough blocks to fund filling up the mempool.
* Extract coinbase transactions out of the generated blocks.
* Tidy up a bit.
* Implement submitting transactions.
* Lower the amount of outputs in each transaction.
* Verify that the mempool size has the expected amount of transactions.
* Pregenerate enough funds before submitting the first transaction so that block creation doesn't interfere with the test.
* Empty mempool out by continuously adding blocks to the DAG.
* Handle orphan transactions when overfilling the mempool.
* Increase mempoolSizeLimit to 1m.
* Fix a comment.
* Fix a comment.
* Add mempool-limits to run-slow.sh.
* Rename generateTransactionsWithLotsOfOutputs to generateTransactionsWithMultipleOutputs.
* Rename generateCoinbaseTransaction to mineBlockAndGetCoinbaseTransaction.
* Make generateFundingCoinbaseTransactions return an object instead of store a global variable.
* Convert mempool-limits into a Go test.
* Convert panics to t.Fatalfs.
* Fix a comment.
* Increase mempoolSizeLimit to 1m.
* Run TestMempoolLimits only if RUN_STABILITY_TESTS is set.
* Move the run of mempool-limits in run-slow.sh.
* Add a comment above fundingCoinbaseTransactions.
* Make a couple of stylistic changes.
* Use transactionhelper.CoinbaseTransactionIndex instead of hardcoding 0.
* Make uninteresting errors print %+v instead of %s.
Co-authored-by: Svarog <feanorr@gmail.com>
* Remove a random address from the address manager if it's full.
* Implement TestOverfillAddressManager.
* Add connectionFailedCount to addresses.
* Mark connection failures.
* Mark connection successes.
* Implement removing by most connection failures.
* Expand TestOverfillAddressManager.
* Add comments.
* Use a better method for finding the address with the greatest connectionFailedCount.
* Fix a comment.
* Compare addresses by IP in TestOverfillAddressManager.
* Add a comment for updateNotBanned.
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Add a reconnect mechanism to RPCClient.
* Fix Reconnect().
* Connect the internal reconnection logic to the miner reconnection logic.
* Rename shouldReconnect to isClosed.
* Move safe reconnection logic from the miner to rpcclient.
* Remove sleep from HandleSubmitBlock.
* Properly handle client errors and only disconnect if we're already connected.
* Make go lint happy.
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Replace BlockMessage with RpcBlock in rpc.proto.
* Convert everything in kaspad to use RPCBlocks and fix tests.
* Fix compilation errors in stability tests and the miner.
* Update TransactionVerboseData in rpc.proto.
* Update TransactionVerboseData in the rest of kaspad.
* Make golint happy.
* Include RpcTransactionVerboseData in RpcTransaction instead of the other way around.
* Regenerate rpc.pb.go after merge.
* Update appmessage types.
* Update appmessage request and response types.
* Reimplement conversion functions between appmessage.RPCTransaction and protowire.RpcTransaction.
* Extract RpcBlockHeader toAppMessage/fromAppMessage out of RpcBlock.
* Fix compilation errors in getBlock, getBlocks, and submitBlock.
* Fix compilation errors in getMempoolEntry.
* Fix compilation errors in notifyBlockAdded.
* Update verbosedata.go.
* Fix compilation errors in getBlock and getBlocks.
* Fix compilation errors in getBlocks tests.
* Fix conversions between getBlocks message types.
* Fix integration tests.
* Fix a comment.
* Add selectedParent to the verbose block response.
* Use go:embed to embed sample-kaspad.conf (only on go1.16)
* Add a comment to justify the blank import.
* Change a variable name to sampleKaspad (instead configurationSampleKaspadString)
Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: Svarog <feanorr@gmail.com>
* Replace blue score with DAA score in UTXO entries
* Use DAA score for coinbase maturity
* Use DAA score for sequence lock
* Fix calcBlockSubsidy to use DAA score
* Don't pay to blocks that are not included in the DAA added blocks, and bestow red blocks reward to the merging block
* Fix TestGetPruningPointUTXOs
* Fix TestTransactionAcceptance
* Fix TestChainedTransactions
* Fix TestVirtualDiff
* Fix TestBlockWindow
* Fix TestPruning
* Use NewFromSlice instead of manually creating the hash set
* Add assert
* Add comment
* Remove redundant call to UpdateDAADataAndReturnDifficultyBits
* Add RequiredDifficulty, rename UpdateDAADataAndReturnDifficultyBits to StageDAADataAndReturnRequiredDifficulty and add comments
* Make buildUTXOInvalidHeader get bits as an argument
* Fix comments
* Handle errors in p2p handshake better
* Add a new errNil in protowire
* Add nil checks for all protowire.toAppMessage() functions for the p2p
* Add nil checks for all protowire.toAppMessage() functions for the RPC
* Add nil check for protwire KaspadMessage
Co-authored-by: Svarog <feanorr@gmail.com>
* Remove Subsystems map and replace with RegisterSubSystem
* Clean up the logger
* Fix LOGFLAGS and make LongFile work correctly
* Parallelize the logger backend
* More logger cleanup
* Initialize and close the logger backend wherever it's needed
* Move the location where the backend is closed, also print the log if it panics while writing
* Add TestMain to reachability manager tests to preserve the same log level
* Fix review comments
Co-authored-by: Svarog <feanorr@gmail.com>
* Add Close() function to all the iterators
* Add defer iterator.Close() whenever we open an iterator
* Add isClosed to all iterators and panic/return error if used after closing
Co-authored-by: Svarog <feanorr@gmail.com>
* Move existing address/bannedAddress functionality to a new addressStore object.
* Implement TestAddressManager.
* Implement serializeAddressKey and deserializeAddressKey.
* Implement serializeNetAddress and deserializeNetAddress.
* Store addresses and banned addresses to disk.
* Implement restoreNotBannedAddresses.
* Fix bannedDatabaseKey.
* Implement restoreBannedAddresses.
* Implement TestRestoreAddressManager.
* Defer closing the database in TestRestoreAddressManager.
* Defer closing the database in TestRestoreAddressManager.
* Add a log.
* Return errors where appropriate.
Co-authored-by: Elichai Turkel <elichai.turkel@gmail.com>
