* Add boilerplate for the `parse` sub command.
* Deserialize the given transaction hax.
* Implement the rest of the wallet parse command.
* Hide transaction inputs behind a `verbose` flag.
* Indicate that we aren't able to extract an address out of a nonstandard transaction.
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Add cache to block window
* Copy the window heap slice with the right capacity
* Use WindowHeapSliceStore
* Use the selected parent window as a basis (and some comments and variable renames)
* Clone slice on newSizedUpHeapFromSlice
* Rename isNotFoundError->currentIsNonTrustedBlock
* Increase windowHeapSliceStore cache size to 2000 and some cosmetic changes
* Make MaxBlockLevel a DAG params instead of a constant.
* Change the testnet network name to 9.
* Fix TestBlockWindow.
* Set MaxBlockLevels for non-mainnet networks to 250.
* Revert "Fix TestBlockWindow."
This reverts commit 30a7892f53.
* Fix TestPruning.
* Make kaspawallet store the utxos sorted by amount, so that the bigger utxos are spent first - making it less likely a compound will be required
* Start refactor addEntryToUTXOSet
* Add GetUTXOsByBalances command to rpc
* Store list of addresses, updated with the collectAddresses methods
(replacing collectUTXOs methods)
* Fix wrong commands in GetBalanceByAddress
* Rename: refreshExistingUTXOs -> refreshUTXOs
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Split ApplyPruningPointProof to multiple small database transactions.
* Increase the timeout duration in TestIBDWithPruning.
* Increase the timeout duration in simple-sync.
* Explain that if ApplyPruningPointProof fails, the database must be discarded.
* Report progress percentage when downloading headers in IBD.
* Extract reporting logic to a separate type.
* Report progress for IBD missing block bodies.
* Fix for rare consensus bug: daa window min-time-block was not deterministic when timestamps are equal
* Something is missing
* Extract compare logic to a function with better performance
* typo
* Add monitoring of heap and save heap profile if size is over some limit
* Exported function
* Extract dump logic to a function (for defer close)
* Change trackHeapSize ticker interval to 10 seconds
* Add timestamp to dump file name
Co-authored-by: Ori Newman <>
* Separate IBD to a new flow (so now invs are handled concurrently and no route capacity errors)
* Invs messages should be queued while waiting for BlockLocator msg
* Close IBD channel so that HandleIBDFlow exits too
* Apply flow separation to p2p protocol v4
* Manage the IBDRequestChannel through the Peer struct
* Some IBDs take a little longer
* Add --transaction-file to the sign wallet subcommand.
* Fix bad short sign config option.
* Trim whitespace around the hex file.
* Add --transaction-file to the broadcast subcommand.
* Copy blockrelay flows to v4
* Remove duplicate sending of the same DAA blocks
* Advance testnet version
* Renames and add comments
* Add IBD test between v3 and v4
* Fix syncee v4 p2p version
* Check if netsync finished with selected tip
* Implement upgrade mechanism for p2p
* Remove dependencies from flowcontext to v3
* Add p2p v4
* Add Ready flow
* Remove copy paste code of v3
* Register SendAddresses flow at the top level
* Add option to set protocol version from CLI and add TestAddressExchangeV3V4
* Send ready message on minimal net adapter
* Rename defaultMaxProtocolVersion->maxAcceptableProtocolVersion
* Address manager refactor stage 1
* Use a simpler weightedRand function which makes it easier parameterize to the process
* Switch back to connectionFailedCount
* Simplify selected entry deletion
* Fix function comment
Co-authored-by: Constantine Bitensky <cbitensky1@gmail.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Fix stability test mining (allow submission of non DAA blocks too)
* Comments for go fmt
* Use SubmitBlockAlsoIfNonDAA for all tests
Co-authored-by: Elichai Turkel <elichai.turkel@gmail.com>
* Don't assume pruning point is at specific level
* Reverse pruning points order in ArePruningPointsViolatingFinality
Co-authored-by: Michael Sutton <mikisiton2@gmail.com>
* Remove unused functions, constants, and variables.
* Rename maxSubsidy to baseSubsidy.
* Remove unused parameters from CalcBlockSubsidy.
* Remove link to old monetary policy.
* If a block's DAA score is smaller than half a year, it should have a base subsidy.
* Fix merge errors.
* Fix more merge errors.
* Add DeflationaryPhaseBaseSubsidy to the params.
* Implement TestCalcDeflationaryPeriodBlockSubsidy.
* Implement calcDeflationaryPeriodBlockSubsidy naively.
* Implement calcDeflationaryPeriodBlockSubsidy not naively.
* Adjust the subsidy based on target block rate.
* Fix deflationaryPhaseDaaScore in TestCalcDeflationaryPeriodBlockSubsidy.
* Explain how secondsPerMonth is calculated.
* Don't adjust the subsidy based on the target block rate.
* Update defaultDeflationaryPhaseDaaScore and add an explanation.
* Use a pre-calculated table for subsidy per month
* Make the generation function fail if base subsidy is changed
* go fmt
* Use test logger for printing + simplify print loop
Co-authored-by: msutton <mikisiton2@gmail.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Seed from DNS only if we ran out of addresses to connect to
* Remove address if it has failed 10 connections
* Change connectionFailedCountForRemove to 3
* Disallow by default RPC submission of out of date blocks (blocks which are out of virtual DAA window)
* go fmt
* Better condition test
* Make allowNonDAABlocks an RPC field and not a command line flag
* go fmt
* one more go fmt
* Update to version v0.11.8
* bugfix: addresses issue #1903, raised kaspawallet bruteforce core limit to 256
* fix contributing.md text
* added dns seeder kaspadns.kaspacalc.net
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Update to version v0.11.8
* Call OnPruningPointUTXOSetOverride after committing the staging consensus
* use domain.Consensus() in utxo index so it'll always check the right consensus
* Add NewFileFromMnemonics
* Export InternalKeychain and ExternalKeychain
* Rename NewFileFromMnemonics -> NewFileFromMnemonic
* NewFileFromMnemonic: change also argument name
* Use libkaspawallet.ExternalKeychain instead of externalKeychain
* Fix num threads using getAEAD instead of decryptMnemonic
* Use d.NumThread to init bruteforce for num threads in getAEAD
Co-authored-by: Ori Newman <>
* Get rid of genesis's UTXO dump
* Allow known orphans when AllowSubmitBlockWhenNotSynced=true
* gofmt
* Avoid IBD without changing the pruning point when only genesis is available
* Add DisallowDirectBlocksOnTopOfGenesis=true for mainnet
* Remove any mention to nobanning to let stability tests run
* Rename ifGenesisSetUtxoSet to loadUTXODataForGenesis
Co-authored-by: Ori Newman <>
* UTXO dump of block 0fca37ca667c2d550a6c4416dad9717e50927128c424fa4edbebc436ab13aeef
* Activate HF immediately and change reward to 1000
* Change protocol version and datadir location
* Delete comments
* Fix zero hash to muhash zero hash in genesis utxo dump check
* Don't omit genesis as direct parent
* Fix tests
* Change subsidy to 500
* Dont assume genesis multiset is empty
* Fix BlockReward test
* Fix TestValidateAndInsertImportedPruningPoint test
* Fix pruning point genesis utxo set
* Fix tests related to mainnet utxo set
* Dont change the difficulty before you have a full window
* Fix TestBlockWindow tests
* Remove global utxo set variable, and persist mainnetnet utxo deserialization between runs
* Fix last tests
* Make peer banning opt-in
* small fix for a test
* Fix go lint
* Fix Ori's review comments
* Change DAA score of genesis to checkpoint DAA score and fix all tests
* Fix the BlockLevel bits counting
* Fix some tests and make them run a little faster
* Change datadir name back to kaspa-mainnet and change db path from /data to /datadir
* Last changes for the release and change the version to 0.11.5
Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: Ori Newman <>
Co-authored-by: msutton <mikisiton2@gmail.com>
* Added a test which reproduces a bug where virtual's mergeset exceeds the mergeset limit -- this test currently fails
* Added a simple condition to fix the mergeset limit bug
* Format issue
* Modify DefaultTimeout to 120 seconds
A temporary workaround for nodes having trouble to sync (currently the download of pruning point related data during IBD takes more than 30 seconds)
* Cache existence in reachability store
* Cache block level in the header
* Fix IBD indication on submit block
* Add hardForkOmitGenesisFromParentsDAAScore logic
* Fix NumThreads bug in the wallet
* Get rid of ParentsAtLevel header method
* Fix a bug in BuildPruningPointProof
* Increase race detector timeout
* Add cache to BuildPruningPointProof
* Add comments and temp comment out go vet
* Fix ParentsAtLevel
* Dont fill empty parents
* Change HardForkOmitGenesisFromParentsDAAScore in fast netsync test
* Add --allow-submit-block-when-not-synced in stability tests
* Fix TestPruning
* Return fast tests
* Fix off by one error on kaspawallet
* Fetch only one block with trusted data at a time
* Update fork DAA score
* Don't ban for unexpected message type
* Fix tests
Co-authored-by: Michael Sutton <mikisiton2@gmail.com>
Co-authored-by: Ori Newman <>
* Decrease the dial timeout to 1 second
* Don't set blocks from the pruning point anticone as the header selected tip.
Co-authored-by: Kaspa Profiler <>
* Fix a bug in the matrix ranking algorithm
* Add tests and benchmarks for matrix generation and ranking
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Implement a MinerState to cache the matrix and friends
* Modify the miner and related code to use the new MinerCache
* Change MinerState to State
* Make go lint happy
Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: Kaspa Profiler <>
* Replace keccak with CSHAKE256 in oPoW
* Add benchmarks to hash writers to compare blake2b to the CSHAKE
* Update genesis blocks
* Update tests
* Define genesis's block level to be the maximal one
* Add message to genesis coinbase
* Add comments to genesis coinbase
* Fix tests
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Send pruning point anticone in topological order
Fix a UTXO pagination bug
Lengthen the stabilization time for the last DAA test
* Extend "sudden hash rate drop" test length to 45 minutes
Co-authored-by: Kaspa Profiler <>
* Implement isBlockRewardFixed.
* Fix factory.go.
* Call isBlockRewardFixed from calcBlockSubsidy.
* Fix bad call to ghostdagDataStore.Get.
* Extract blue score and blue work from the header instead of from the ghostdagDataStore.
* Fix coinbasemanager constructor arguments order
* Format consensus_defaults.go
* Check the mainnet switch from the block's point of view rather than the virtual's.
* Don't call newBlockPruningPoint twice in buildBlock.
* Properly handle new pruning point blocks in isBlockRewardFixed.
* Use the correct variable.
* Add a comment explaining what we do when the pruning point is not found in isBlockRewardFixed.
* Implement TestBlockRewardSwitch.
* Add missing error handling.
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Check blue score before requesting a pruning proof
* BuildPruningPointProof should return empty proof if the pruning point is genesis
* Don't fail many-tips if kaspad exits ungracefully
* Replace the old blockSubsidy parameters with the new ones.
* Return subsidyGenesisReward if blockHash is the genesis hash.
* Traverse a block's past for the subsidy calculation.
* Partially implement SubsidyStore.
* Refer to SubsidyStore from CoinbaseManager.
* Wrap calcBlockSubsidy in getBlockSubsidy, which first checks the database.
* Fix finalityStore not calling GenerateShardingID.
* Implement calculateAveragePastSubsidy.
* Implement calculateMergeSetSubsidySum.
* Implement calculateSubsidyRandomVariable.
* Implement calcBlockSubsidy.
* Add a TODO about floats.
* Update the calcBlockSubsidy TODO.
* Use binary.LittleEndian in calculateSubsidyRandomVariable.
* Fix bad range in calculateSubsidyRandomVariable.
* Replace float64 with big.Rat everywhere except for subsidyRandomVariable.
* Fix a nil dereference.
* Use a random walk to approximate the normal distribution.
* In order to avoid unsupported fractional results from powInt64, flip the numerator and the denominator manually.
* Set standardDeviation to 0.25, MaxSompi to 10_000_000_000 * SompiPerKaspa and defaultSubsidyGenesisReward to 1_000.
* Set the standard deviation to 0.2.
* Use a binomial distribution instead of trying to estimate the normal distribution.
* Change some values around.
* Clamp the block subsidy.
* Remove the fake duplicate constants in the util package.
* Reduce MaxSompi to only 100m Kaspa to avoid hitting the uint64 ceiling.
* Lower MaxSompi further to avoid new and exciting ways for the uint64 ceiling to be hit.
* Remove debug logs.
* Fix a couple of failing tests.
* Fix TestBlockWindow.
* Fix limitTransactionCount sometimes crashing on index-out-of-bounds.
* In TrustedDataDataDAABlock, replace BlockHeader with DomainBlock
* In calculateAveragePastSubsidy, use blockWindow instead of doing a BFS manually.
* Remove the reference to DAGTopologyManager in coinbaseManager.
* Add subsidy to the coinbase payload.
* Get rid of the subsidy store and extract subsidies out of coinbase transactions.
* Keep a blockWindow amount of blocks under the virtual for IBD purposes.
* Manually remove the virtual genesis from the merge set.
* Fix simnet genesis.
* Fix TestPruning.
* Fix TestCheckBlockIsNotPruned.
* Fix TestBlockWindow.
* Fix TestCalculateSignatureHashSchnorr.
* Fix TestCalculateSignatureHashECDSA.
* Fix serializing the wrong value into the coinbase payload.
* Rename coinbaseOutputForBlueBlock to coinbaseOutputAndSubsidyForBlueBlock.
* Add a TODO about optimizing trusted data DAA window blocks.
* Expand on a comment in TestCheckBlockIsNotPruned.
* In calcBlockSubsidy, divide the big.Int numerator by the big.Int denominator instead of converting to float64.
* Clarify a comment.
* Rename SubsidyMinGenesisReward to MinSubsidy.
* Properly handle trusted data blocks in calculateMergeSetSubsidySum.
* Use the first two bytes of the selected parent's hash for randomness instead of math/rand.
* Restore maxSompi to what it used to be.
* Fix TestPruning.
* Fix TestAmountCreation.
* Fix TestBlockWindow.
* Fix TestAmountUnitConversions.
* Increase the timeout in many-tips to 30 minutes.
* Check coinbase subsidy for every block
* Re-rename functions
* Use shift instead of powInt64 to determine subsidyRandom
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Calculate GHOSTDAG, reachability etc for each level
* Don't preallocate cache for dag stores except level 0 and reduce the number of connections in the integration test to 32
* Reduce the number of connections in the integration test to 16
* Increase page file
* BuildPruningPointProof
* BuildPruningPointProof
* Add PruningProofManager
* Implement ApplyPruningPointProof
* Add prefix and fix blockAtDepth and fill headersByLevel
* Some bug fixes
* Include all relevant blocks for each level in the proof
* Fix syncAndValidatePruningPointProof to return the right block hash
* Fix block window
* Fix isAncestorOfPruningPoint
* Ban for rule errors on pruning proof
* Find common ancestor for blockAtDepthMAtNextLevel
* Use pruning proof in TestValidateAndInsertImportedPruningPoint
* stage status and finality point for proof blocks
* Uncomment golint
* Change test timeouts
* Calculate merge set for ApplyPruningPointProof
* Increase test timeout
* Add better caching for daa window store
* Return to default timeout
* Add ErrPruningProofMissesBlocksBelowPruningPoint
* Add errDAAWindowBlockNotFound
* Force connection loop next iteration on connection manager stop
* Revert to Test64IncomingConnections
* Remove BlockAtDepth from DAGTraversalManager
* numBullies->16
* Set page file size to 8gb
* Increase p2p max message size
* Test64IncomingConnections->Test16IncomingConnections
* Add comment for PruningProofM
* Add comment in `func (c *ConnectionManager) Stop()`
* Rename isAncestorOfPruningPoint->isAncestorOfSelectedTip
* Revert page file to 16gb
* Improve ExpectedHeaderPruningPoint perf
* Fix comment
* Revert "Improve ExpectedHeaderPruningPoint perf"
This reverts commit bca1080e71.
* Don't test windows
* Create BlockParentBuilder.
* Implement BuildParents.
* Explictly set level 0 blocks to be the same as direct parents.
* Add checkIndirectParents to validateBlockHeaderInContext.
* Fix test_block_builder.go and BlockLevelParents::Equal.
* Don't check indirect parents for blocks with trusted data.
* Handle pruned blocks when building block level parents.
* Fix bad deletions from unprocessedXxxParents.
* Fix merge errors.
* Fix bad pruning point parent replaces.
* Fix duplicates in newBlockLevelParents.
* Skip checkIndirectParents
* Get rid of staging constant IDs
* Fix BuildParents
* Fix tests
* Add comments
* Change order of directParentHashes
* Get rid of maybeAddDirectParentParents
* Add comments
* Add blockToReferences type
* Use ParentsAtLevel
Co-authored-by: stasatdaglabs <stas@daglabs.com>
* Add PruningPointProof to externalapi.
* Add BuildPruningPointProof and ValidatePruningPointProof to Consensus.
* Add the pruning point proof to the protocol.
* Add the pruning point proof to the wire package.
* Add PruningPointBlueWork.
* Make go vet happy.
* Properly initialize PruningPointProof in consensus.go.
* Validate pruning point blue work.
* Populate PruningPointBlueWork with the actual blue work of the pruning point.
* Revert "Populate PruningPointBlueWork with the actual blue work of the pruning point."
This reverts commit f2a9829998.
* Revert "Validate pruning point blue work."
This reverts commit c6a90c5d2c.
* Revert "Properly initialize PruningPointProof in consensus.go."
This reverts commit 9391574bbf.
* Revert "Add PruningPointBlueWork."
This reverts commit 48182f652a.
* Fix PruningPointProof and MsgPruningPointProof to be two-dimensional.
* Fix wire PruningPointProof to be two-dimensional.
* Replace header finality point with pruning point
* Fix TestTransactionAcceptance
* Fix pruning candidate
* Store all past pruning points
* Pass pruning points on IBD
* Add blue score to block header
* Simplify ArePruningPointsInValidChain
* Fix static check errors
* Fix genesis
* Renames and text fixing
* Use ExpectedHeaderPruningPoint in block builder
* Fix TestCheckPruningPointViolation
* Replace the old parents in the block header with BlockLevelParents.
* Begin fixing compilation errors.
* Implement database serialization for block level parents.
* Implement p2p serialization for block level parents.
* Implement rpc serialization for block level parents.
* Add DirectParents() to the block header interface.
* Use DirectParents() instead of Parents() in some places.
* Revert test_block_builder.go.
* Add block level parents to hash serialization.
* Use the zero hash for genesis finality points.
* Fix failing tests.
* Fix a variable name.
* Update headerEstimatedSerializedSize.
* Add comments in blocklevelparents.go.
* Fix the rpc-stability stability test.
* Change the field number for `parents` fields in p2p.proto and rpc.proto.
* Remove MsgBlockHeader::NumParentBlocks.
* Implement GHOST.
* Implement TestGHOST.
* Make GHOST() take arbitrary subDAGs.
* Hold RootHashes in SubDAG rather than one GenesisHash.
* Select which root the GHOST chain starts with instead of passing a lowHash.
* If two child hashes have the same future size, decide which one is larger using the block hash.
* Extract blockHashWithLargestFutureSize to a separate function.
* Calculate future size for each block individually.
* Make TestGHOST deterministic.
* Increase the timeout for connecting 128 connections in TestRPCMaxInboundConnections.
* Implement BenchmarkGHOST.
* Fix an infinite loop.
* Use much larger benchmark data.
* Optimize `futureSizes` using reverse merge sets.
* Temporarily make the benchmark data smaller while GHOST is being optimized.
* Fix a bug in futureSizes.
* Fix a bug in populateReverseMergeSet.
* Choose a selectedChild at random instead of the one with the largest reverse merge set size.
* Rename populateReverseMergeSet to calculateReverseMergeSet.
* Use reachability to resolve isDescendantOf.
* Extract heightMaps to a separate object.
* Iterate using height maps in futureSizes.
* Don't store reverse merge sets in memory.
* Change calculateReverseMergeSet to calculateReverseMergeSetSize.
* Fix bad initial reverseMergeSetSize.
* Optimize calculateReverseMergeSetSize.
* Enlarge the benchmark data to 86k blocks.
* Limit the amount of inbound RPC connections.
* Increment/decrement the right variable.
* Implement TestRPCMaxInboundConnections.
* Make go vet happy.
* Increase RPCMaxInboundConnections to 128.
* Set NUM_CLIENTS=128 in the rpc-idle-clients stability test.
* Explain why the P2P server has unlimited inbound connections.
* Add DAAScore, BlueWork, and FinalityPoint to externalapi.BlockHeader.
* Add DAAScore, BlueWork, and FinalityPoint to NewImmutableBlockHeader and fix compilation errors.
* Add DAAScore, BlueWork, and FinalityPoint to protowire header types and fix failing tests.
* Check for header DAA score in validateDifficulty.
* Add DAA score to buildBlock.
* Fix failing tests.
* Add a blue work check in validateDifficultyDAAAndBlueWork.
* Add blue work to buildBlock and fix failing tests.
* Add finality point validation to ValidateHeaderInContext.
* Fix genesis blocks' finality points.
* Add finalityPoint to blockBuilder.
* Fix tests that failed due to missing reachability data.
* Make blockBuilder use VirtualFinalityPoint instead of directly calling FinalityPoint with the virtual hash.
* Don't validate the finality point for blocks with trusted data.
* Add debug logs.
* Skip finality point validation for block whose finality points are the virtual genesis.
* Revert "Add debug logs."
This reverts commit 3c18f519cc.
* Move checkDAAScore and checkBlueWork to validateBlockHeaderInContext.
* Add checkCoinbaseBlueScore to validateBodyInContext.
* Fix failing tests.
* Add DAAScore, blueWork, and finalityPoint to blocks' hashes.
* Generate new genesis blocks.
* Fix failing tests.
* In BuildUTXOInvalidBlock, get the bits from StageDAADataAndReturnRequiredDifficulty instead of calling RequiredDifficulty separately.
* Update the virtual after overriding the virtual utxo set
* Put the updateVirtual inside importVirtualUTXOSetAndPruningPointUTXOSet
* Add pruningPoint to importVirtualUTXOSetAndPruningPointUTXOSet
* Remove sanity check
* Pruning headers p2p basic structure
* Remove headers-first
* Fix consensus tests except TestValidateAndInsertPruningPointWithSideBlocks and TestValidateAndInsertImportedPruningPoint
* Add virtual genesis
* Implement PruningPointAndItsAnticoneWithMetaData
* Start fixing TestValidateAndInsertImportedPruningPoint
* Fix TestValidateAndInsertImportedPruningPoint
* Fix BlockWindow
* Update p2p and gRPC
* Fix all tests except TestHandleRelayInvs
* Delete TestHandleRelayInvs parts that cover the old IBD flow
* Fix lint errors
* Add p2p_request_ibd_blocks.go
* Clean code
* Make MsgBlockWithMetaData implement its own representation
* Remove redundant check if highest share block is below the pruning point
* Fix TestCheckLockTimeVerifyConditionedByAbsoluteTimeWithWrongLockTime
* Fix comments, errors ane names
* Fix window size to the real value
* Check reindex root after each block at TestUpdateReindexRoot
* Remove irrelevant check
* Renames and comments
* Remove redundant argument from sendGetBlockLocator
* Don't delete staging on non-recoverable errors
* Renames and comments
* Remove redundant code
* Commit changes inside ResolveVirtual
* Add comment to IsRecoverableError
* Remove blocksWithMetaDataGHOSTDAGDataStore
* Increase windows pagefile
* Move DeleteStagingConsensus outside of defer
* Get rid of mustAccepted in receiveBlockWithMetaData
* Ban on invalid pruning point
* Rename interface_datastructures_daawindowstore.go to interface_datastructures_blocks_with_meta_data_daa_window_store.go
* * Change GetVirtualSelectedParentChainFromBlockResponseMessage and VirtualSelectedParentChainChangedNotificationMessage to show only added block hashes
* Remove ResolveVirtual
* Use externalapi.ConsensusWrapper inside MiningManager
* Fix pruningmanager.blockwithmetadata
* Set pruning point selected child when importing the pruning point UTXO set
* Change virtual genesis hash
* replace the selected parent with virtual genesis on removePrunedBlocksFromGHOSTDAGData
* Get rid of low hash in block locators
* Remove +1 from everywhere we use difficultyAdjustmentWindowSize and increase the default value by one
* Add comments about consensus wrapper
* Don't use separate staging area when resolving resolveBlockStatus
* Fix netsync stability test
* Fix checkResolveVirtual
* Rename ConsensusWrapper->ConsensusReference
* Get rid of blockHeapNode
* Add comment to defaultDifficultyAdjustmentWindowSize
* Add SelectedChild to DAGTraversalManager
* Remove redundant copy
* Rename blockWindowHeap->calculateBlockWindowHeap
* Move isVirtualGenesisOnlyParent to utils
* Change BlockWithMetaData->BlockWithTrustedData
* Get rid of maxReasonLength
* Split IBD to 100 blocks each time
* Fix a bug in calculateBlockWindowHeap
* Switch to trusted data when encountering virtual genesis in blockWithTrustedData
* Move ConsensusReference to domain
* Update ConsensusReference comment
* Add comment
* Rename shouldNotAddGenesis->skipAddingGenesis
* ignore type flag
* Ignore type flag of relative time lock - interpret as DAA score
* Split verifyLockTime to functions with and without threshold.relative lockTimes dont need threshold check
* Change function name and order of the functions calls
Co-authored-by: tal <tal@daglabs.com>
* Update constants
* Add to transaction SigOpCount
* Update mass calculation, and move it from InContext to InIsolation
* Update block validation accordingly
* Add SigOpCount validation during TransactionInContext
* Remove checking of mass vs maxMassAcceptedByBlock from consensusStateManager
* Update mining manager with latest changes
* Add SigOpCount to MsgTx.Copy()
* Fix initTestTransactionAcceptanceDataForClone
* Fix all tests in transaction_equal_clone_test.go
* Fix TestBlockMass
* Fix tests in transactionvalidator package
* Add SigOpCount to sighash
* Fix TestPruningDepth
* Fix problems in libkaspawalelt
* Fix integration tests
* Fix CalculateSignatureHash tests
* Remove remaining places talking about block size
* Add sanity check to checkBlockMass to make sure all transactions have their mass filled
* always add own sigOpCount to sigHash
* Update protowire/rpc.md
* Start working on removing any remaining reference to block/tx size
* Update rpc transaction verbose data to include mass rather then size
* Convert verboseData and block size check to mass
* Remove remaining usages of tx size in mempool
* Move transactionEstimatedSerializedSize to transactionvalidator
* Add PopulateMass to fakeRelayInvsContext
* Move PopulateMass to beggining of ValidateAndInsertTransaction + fix in it
* Assign mass a new number for backward-compatibility
* Validate locktime when admitted into mempool and when build a block. Also, fix isFinalized to use DAAscore instead of blue score.
* Change the function name:ValidateTransactionInContextIgnoringUTXO
Co-authored-by: tal <tal@daglabs.com>
* Create a file
* Add tests for lockTime - CLTV scripts conditioned by time and block height
* Add a handle for an unhandled error.
* Renamed the test file
* Fix typo
* Add a counter for current block height.
* Change variable name
* Adds a test for wrong lock time, removed fundingTransaction variable
* Fix LockTimeThreshold constant, fix opcodeCheckLockTimeVerify and opcodeCheckSequenceVerify(padding in the end), add support for sequence and lock time number in the script builder, add more checks to the CLTV test.
* Call AddData instead of addData. Rename fixedSize to unpaddedSize
* Creating wrapper functions to lockTime&sequence numbers that call to a shared function in script builder.
Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Copy over boilerplate and begin implementing TestDAA.
* Implement a fairly reliable method of hashing at a certain hashrate.
* Convert the DAA test to an application.
* Start kaspad and make sure that hashrate throttling works with that as well.
* Finish implementing testConstantHashRate.
* Tidied up a bit.
* Convert TestDAA back into a go test.
* Reorganize TestDAA to be more like a traditional test.
* Add sudden hashrate drop/jump tests.
* Simplify targetHashNanosecondsFunction.
* Improve progress logs.
* Add more tests.
* Remove the no-longer relevant `hashes` part of targetHashNanosecondsFunction.
* Implement a constant hashrate increase test.
* Implement a constant hashrate decrease test.
* Give the correct run duration to the constant hashrate decrease test.
* Add cooldowns to exponential functions.
* Add run.sh to the DAA test.
* Add a README.
* Add `daa` to run-slow.sh.
* Make go lint happy.
* Fix the README's title.
* Keep running tests even if one of them failed on high block rate deviation.
* Fix hashrate peak/valley tests.
* Preallocate arrays for hash and mining durations.
* Add more statistics to the "mined block" log.
* Make sure runDAATest stops when it's suppposed to.
* Add a newline after "5 minute cooldown."
* Fix variable names.
* Rename totalElapsedTime to tatalElapsedDuration.
* In measureMachineHashNanoseconds, generate a random nonce only once.
* In runDAATest, add "DAA" to the start/finish log.
* Remove --logdir from kaspadRunCommand.
* In runDAATest, enlarge the nonce range to the entirety of uint64.
* Explain what targetHashNanosecondsFunction is.
* Move RunKaspadForTesting into common.
* Rename runForDuration to loopForDuration.
* Make go lint happy.
* Extract fetchBlockForMining to a separate function.
* Extract waitUntilTargetHashDurationHadElapsed to a separate function.
* Extract pushHashDuration and pushMiningDuration to separate functions.
* Extract logMinedBlockStatsAndUpdateStatFields to a separate function.
* Extract submitMinedBlock to a separate function.
* Extract tryNonceForMiningAndIncrementNonce to a separate function.
* Add comments.
* Use a rolling average instead of appending to an array for performance/accuracy.
* Change a word in a comment.
* Explain why we wait for five minutes at the end of the exponential increase/decrease tests.
Co-authored-by: Svarog <feanorr@gmail.com>
* Adds name to route, and writes it in every error message
* Update all calls with route name
* Fixed a few missed points
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
* Make use of maxBlocks instead of maxBlueScoreDifference in antiPastHashesBetween
* Make use of maxBlocks instead of maxBlueScoreDifference in antiPastHashesBetween
* Make use of maxBlocks instead of maxBlueScoreDifference in antiPastHashesBetween
* Make use of maxBlocks instead of maxBlueScoreDifference in antiPastHashesBetween
* Make use of maxBlocks instead of maxBlueScoreDifference in antiPastHashesBetween
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Change SequenceLockTimeDisabled to 1 << 63
* Move LockTimeThreshold to constants
* Update locktime constants according to new proposal
* Fix opcodeCheckSequenceVerify and failed tests
* Disallow numbers above 8 bytes in makeScriptNum
* Use littleEndian.Uint64 for sequence instead of ScriptNum
* Update comments on constants
* Update some more comments
* All orphans inputs should be added to op.orphansByPreviousOutpoint even if outpoint is not missing
* Remove redundant log
* processOrphansAfterAcceptedTransaction: wqCheck that UTXOEntry is empty before filling it
* Don't remove redeemers in expireOrphanTransactions
* Added model and stubs for all main methods
* Add constructors to all main objects
* Implement BlockCandidateTransactions
* implement expireOldTransactions and expireOrphanTransactions
* Rename isHighPriority to neverExpires
* Add stub for checkDoubleSpends
* Revert "Rename isHighPriority to neverExpires"
This reverts commit b2da9a4a00.
* Imeplement transactionsOrderedByFeeRate
* Orphan maps should be idToOrphan
* Add error.go to mempool
* Invert the condition for banning when mempool rejects a transaction
* Move all model objects to model package
* Implement getParentsInPool
* Implemented mempoolUTXOSet.addTransaction
* Implement removeTransaction, remove sanity checks
* Implemented mempoolUTXOSet.checkDoubleSpends
* Implemented removeOrphan
* Implement removeOrphan
* Implement maybeAddOrphan and AddOrphan
* Implemented processOrphansAfterAcceptedTransaction
* Implement transactionsPool.addTransaction
* Implement RemoveTransaction
* If a transaction was removed from the mempool - update it's redeemers in orphan pool as well
* Use maximumOrphanTransactionCount
* Add allowOrphans to ValidateAndInsertTransaction stub
* Implement validateTransaction functions
* Implement fillInputs
* Implement ValidateAndInsertTransaction
* Implement HandleNewBlockTransactions
* Implement missing mempool interface methods
* Add comments to exported functions
* Call ValidateTransactionInIsolation where needed
* Implement RevalidateHighPriorityTransactions
* Rewire kaspad to use new mempool, and fix compilation errors
* Update rebroadcast logic to use new structure
* Handle non-standard transaction errors properly
* Add mutex to mempool
* bugfix: GetTransaction panics when ok is false
* properly calculate targetBlocksPerSecond in config.go
* Fix various lint errors and tests
* Fix expected text in test for duplicate transactions
* Skip the coinbase transaction in HandleNewBlockTransactions
* Unorphan the correct transactions
* Call ValidateTransactionAndPopulateWithConsensusData on unorphanTransaction
* Re-apply policy_test as check_transactions_standard_test
* removeTransaction: Remove redeemers in orphan pool as well
* Remove redundant check for uint64 < 0
* Export and rename isDust -> IsTransactionOutputDust to allow usage by rothschild
* Add allowOrphan to SubmitTransaction RPC request
* Remove all implementation from mempool.go
* tidy go mod
* Don't pass acceptedOrphans to handleNewBlockTransactions
* Use t.Errorf in stead of t.Fatalf
* Remove minimum relay fee from TestDust, as it's no longer configurable
* Add separate VirtualDAASCore method for faster retrieval where it's repeated multiple times
* Broadcast all transactions that were accepted
* Don't re-use GetVirtualDAAScore in GetVirtualInfo - this causes a deadlock
* Use real transaction count, and not Orphan
* Get mempool config from outside, incorporating values received from cli
* Use MinRelayFee and MaxOrphanTxs from global kaspad config
* Add explanation for the seemingly redundant check for transaction version in checkTransactionStandard
* Update some comment
* Convert creation of acceptedTransactions to a single line
* Move mempoolUTXOSet out of checkDoubleSpends
* Add test for attempt to insert double spend into mempool
* fillInputs: Skip check for coinbase - it's always false in mempool
* Clarify comment about removeRedeemers when removing random orphan
* Don't remove high-priority transactions in limitTransactionCount
* Use mempool.removeTransaction in limitTransactionCount
* Add mutex comment to handleNewBlockTransactions
* Return error from limitTransactionCount
* Pluralize the map types
* mempoolUTXOSet.removeTransaction: Don't restore utxo if it was not created in mempool
* Don't evacuate from orphanPool high-priority transactions
* Disallow double-spends in orphan pool
* Don't use exported (and locking) methods from inside mempool
* Check for double spends in mempool during revalidateTransaction
* Add checkOrphanDuplicate
* Add orphan to acceptedOrphans, not current
* Add TestHighPriorityTransactions
* Fix off-by-one error in limitTransactionCount
* Add TestRevalidateHighPriorityTransactions
* Remove checkDoubleSpends from revalidateTransaction
* Fix TestRevalidateHighPriorityTransactions
* Move check for MaximumOrphanCount to beggining of maybeAddOrphan
* Rename all map type to singulateToSingularMap
* limitOrphanPool only after the orphan was added
* TestDoubleSpendInMempool: use createChildTxWhenParentTxWasAddedByConsensus instead of createTransactionWithUTXOEntry
* Fix some comments
* Have separate min/max transaction versions for mempool
* Add comment on defaultMaximumOrphanTransactionCount to keep it small as long as we have recursion
* Fix comment
* Rename: createChildTxWhenParentTxWasAddedByConsensus -> createChildTxWhereParentTxWasAddedByConsensus
* Handle error from createChildTxWhereParentTxWasAddedByConsensus
* Rename createChildTxWhereParentTxWasAddedByConsensus -> createChildAndParentTxsAndAddParentToConsensus
* Convert all MaximumXXX constants to uint64
* Add comment
* remove mutex comments
* Add prefix to stores
* Add prefix to forgotten stores
* Add a special type for prefix
* Rename transaction->dbTx
* Change error message
* Use countKeyName
* Rename Temporary Consesnsus to Staging
* Add DeleteStagingConsensus to Domain interface
* Add lock to staging consensus
* Make prefix type-safer
* Use ioutil.TempDir instead of t.TempDir
* Add notifyVirtualDaaScoreChanged to protowire.
* Add notifyVirtualDaaScoreChanged to the rest of kaspad.
* Add notifyVirtualDaaScoreChanged to the rest of kaspad.
* Test the DAA score notification in TestVirtualSelectedParentBlueScore.
* Rename TestVirtualSelectedParentBlueScore to TestVirtualSelectedParentBlueScoreAndVirtualDAAScore.
(cherry picked from commit 83e631548f)
* Add BlockHash optional parameter to EstimateNetworkBlockHashesPerSecond
* Allow to pass '-' for optional values in kaspactl
* Solve a division-by-zero in estimateNetworkHashesPerSecond
* Add BlockHash to toAppMessage/fromAppMessage functions
* Rename: topHash -> StartHash
* Return proper error message if provided startHash doesn't deserialize into a hash
* First limit the candidates size to 3*csm.maxBlockParents before taking the bottom csm.maxBlockParents/2
* Change log level of printing all tips to Tracef
* Add many-tips to run-fast.sh and run-slow.sh
* Fix preallocation size
* Assign intermediate variables
* Change mergeSet to be ordered topologically.
* Add special condition for genesis.
* Add check that the coinbase is validated.
* Change names of variables(old: chainHash, blueHash).
* Fix the DAG diagram in the comment above the function.
* Fix variables names.
Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Naive bip39 with address reuse
* Avoid address reuse in libkaspawallet
* Add wallet daemon
* Use daemon everywhere
* Add forceOverride
* Make CreateUnsignedTransaction endpoint receive amount in sompis
* Collect close UTXOs
* Filter out non-spendable UTXOs from selectUTXOs
* Use different paths for multisig and non multisig
* Fix tests to use non zero path
* Fix multisig cosigner index detection
* Add comments
* Fix dump_unencrypted_data.go according to bip39 and bip32
* Fix wrong derivation path for multisig on wallet creation
* Remove IsSynced endpoint and add validation if wallet is synced for the relevant endpoints
* Rename server address to daemon address
* Fix capacity for extendedPublicKeys
* Use ReadBytes instead of ReadLine
* Add validation when importing
* Increment before using index value, and use it as is
* Save keys file exactly where needed
* Use %+v printErrorAndExit
* Remove redundant consts
* Rnemae collectCloseUTXOs and collectFarUTXOs
* Move typedefs around
* Add comment to addressesToQuery
* Update collectUTXOsFromRecentAddresses comment about locks
* Split collectUTXOs to small functions
* Add sanity check
* Add addEntryToUTXOSet function
* Change validateIsSynced to isSynced
* Simplify createKeyPairsFromFunction logic
* Rename .Sync() to .Save()
* Fix typo
* Create bip39BitSize const
* Add consts to purposes
* Add multisig check for 'send'
* Rename updatedPSTxBytes to partiallySignedTransaction
* Change collectUTXOsFromFarAddresses's comment
* Use setters for last used indexes
* Don't use the pstx acronym
* Fix SetPath
* Remove spaces when reading lines
* Fix walletserver to daemonaddress
* Fix isUTXOSpendable to use DAA score
Co-authored-by: Svarog <feanorr@gmail.com>
* Fix overflow when checking coinbase maturity and don't ban peers that send transactions with immature spend
* Fix tests
Co-authored-by: Svarog <feanorr@gmail.com>
(cherry picked from commit a18f2f8802)
* Fix getBlock RPC command to return transactions
* Fix getBlocks RPC command to return transactions and blocks
* Add GetBlockEvenIfHeaderOnly and use it for getBlock and getBlocks
* Implement GetBlockEvenIfHeaderOnly for fakeRelayInvsContext
* Use less nested code
(cherry picked from commit 50fd86e287)
* Change removeTransactionAndItsChainedTransactions to be non-recursive
* Split the variables assigning.
* Change names of function and variables.
* Append the correct queue.
Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
* Unfinished code.
* Update the testnet version to testnet-5. (#1683)
* Generalize stability-tests/docker/Dockerfile. (#1685)
* Committed for rebasing.
* Adds stability-test many-tips, which tests kaspad handling with many tips in the DAG.
* Delete manytips_test.go.
* Add timeout to the test and create only one RPC client.
* Place the spawn before the for loop and remove a redundant condition.
Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
* [NOD-1344] relaytransactions: simple unit tests
* [NOD-1344] Add mid-complexity unit tests for relaytransactions
* Improve TestHandleRelayedTransactionssub tests
* Improve TestHandleRequestedTransactions sub tests
* [NOD-1344] Fix Simple call test
* [NOD-1344] Fix tests after redesign
* Divide transactionrelay_test.go to 2 separated tests and updates the tests.
* Changes due to review:change the test files name and the test function names, adds new comments and fix typo.
* Delete an unnecessary comparison to True in the if statement condition.
* Update the branch to v0.11.0-dev.
Co-authored-by: karim1king <karimkaspersky@yahoo.com>
Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: Svarog <feanorr@gmail.com>
* Add -v to the `go test` command.
* Generate a new keypair for mempool-limits.
* Set mempool-limits to time out only after 24 hours.
(cherry picked from commit eb1703b948)
* Use selectedParent instead of selectedTip for non-selectedTip blocks in restoreSingleBlockStatus
* Cache the selectedParent for re-use in a resolveSingleBlockStatus chain
* Implement and use reverseUTXOSet
* Reverse blocks in correct order
* Support resolveBlockStatus without separate stagingAreas for usage of testConsensus
* Handle the case where the tip of the resolved block is not the next selectedTip
* Unify isResolveTip
* Some minor fixes and cleanup
* Add full finality window re-org test to stability-slow
* rename: useSeparateStagingAreasPerBlock -> useSeparateStagingAreaPerBlock
* Better logs in resolveSingleBlockStatus
* A few retouches to reverseUTXODiffs
* TEMPORARY COMMIT: EXTRAT ALL DIFFFROMS TO SEPARATE METHODS
* TEMPORARY COMMIT: REMOVE DIFFICULTY CHECKS IN DEVNET
* Don't pre-allocate in utxo-algebra, since the numbers are not known ahead-of-time
* Add some logs to reverseUTXODiffs
* Revert "TEMPORARY COMMIT: REMOVE DIFFICULTY CHECKS IN DEVNET"
This reverts commit c0af9dc6ad.
* Revert "TEMPORARY COMMIT: EXTRAT ALL DIFFFROMS TO SEPARATE METHODS"
This reverts commit 4fcca1b48c.
* Remove redundant paranthesis
* Revise some logs messages
* Rename:oneBlockBeforeCurrentUTXOSet -> lastResolvedBlockUTXOSet
* Don't break if the block was resolved as invalid
* rename unverifiedBlocks to recentlyVerifiedBlcks in reverseUTXODiffs
* Add errors.New to the panic, for a stack trace
* Reverse the UTXODiffs after the main block has been commited
* Use the correct value for previousUTXODiff
* Add test for ReverseUTXODiff
* Fix some names and comments
* Update TestReverseUTXODiffs to use consensus.Config
* Fix comments mentioning 'oneBlockBeforeTip'
* Add a flag for sanity check pruning point utxo set and do the sanity check only if it's enabled
* add description to EnableSanityCheckPruningUTXOSet
* review fix
Co-authored-by: Svarog <feanorr@gmail.com>
* Add version file to database
* Remove redundant code
* Check for version before opening the database, create version file after
* Create version file before opening the database
* Add windows to the CI
* Cast syscall.Stdin into an integer
* DataDir -> AppDir in service_windows.go
* Rename mempool-limits package to something non-main
* Close database after re-assigining to it
* Up rpcTimout to 10 seconds
* Copy some boilerplate from the other stability tests.
* Fix a copy+paste error in run.sh.
* Copy over some stability test boilerplate go code.
* Run kaspad in the background.
* Catch panics and initialize the RPC client.
* Mine enough blocks to fund filling up the mempool.
* Extract coinbase transactions out of the generated blocks.
* Tidy up a bit.
* Implement submitting transactions.
* Lower the amount of outputs in each transaction.
* Verify that the mempool size has the expected amount of transactions.
* Pregenerate enough funds before submitting the first transaction so that block creation doesn't interfere with the test.
* Empty mempool out by continuously adding blocks to the DAG.
* Handle orphan transactions when overfilling the mempool.
* Increase mempoolSizeLimit to 1m.
* Fix a comment.
* Fix a comment.
* Add mempool-limits to run-slow.sh.
* Rename generateTransactionsWithLotsOfOutputs to generateTransactionsWithMultipleOutputs.
* Rename generateCoinbaseTransaction to mineBlockAndGetCoinbaseTransaction.
* Make generateFundingCoinbaseTransactions return an object instead of store a global variable.
* Convert mempool-limits into a Go test.
* Convert panics to t.Fatalfs.
* Fix a comment.
* Increase mempoolSizeLimit to 1m.
* Run TestMempoolLimits only if RUN_STABILITY_TESTS is set.
* Move the run of mempool-limits in run-slow.sh.
* Add a comment above fundingCoinbaseTransactions.
* Make a couple of stylistic changes.
* Use transactionhelper.CoinbaseTransactionIndex instead of hardcoding 0.
* Make uninteresting errors print %+v instead of %s.
Co-authored-by: Svarog <feanorr@gmail.com>
* Add oldPruningPoint to pruningStore
* Make the pruning store work with utxo diff and return an iterator over pruning point utxoset
* Redesign pruning point utxo storage by creating a diff and modifying the old pruning utxo set
* Fix review comments
* Rename updatePruningPointUTXOSet
* Add ECDSA support
* Add domain separation to ECDSA sighash
* Use InfallibleWrite instead of Write
* Rename funcs
* Fix wrong use if vm.sigCache
* Add TestCalculateSignatureHashECDSA
* Add consts
* Fix comment and test name
* Move consts to the top
* Fix comment
* Implement importing private keys into the wallet.
* Fix bad --import default.
* Fix typo in --import annotation.
* Make go lint happy.
* Make go lint happier.
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Remove a random address from the address manager if it's full.
* Implement TestOverfillAddressManager.
* Add connectionFailedCount to addresses.
* Mark connection failures.
* Mark connection successes.
* Implement removing by most connection failures.
* Expand TestOverfillAddressManager.
* Add comments.
* Use a better method for finding the address with the greatest connectionFailedCount.
* Fix a comment.
* Compare addresses by IP in TestOverfillAddressManager.
* Add a comment for updateNotBanned.
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Add a reconnect mechanism to RPCClient.
* Fix Reconnect().
* Connect the internal reconnection logic to the miner reconnection logic.
* Rename shouldReconnect to isClosed.
* Move safe reconnection logic from the miner to rpcclient.
* Remove sleep from HandleSubmitBlock.
* Properly handle client errors and only disconnect if we're already connected.
* Make go lint happy.
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Add missing call to broadcastTransactionsAfterBlockAdded.
* Fix a comment.
* Fix a comment some more.
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Replace BlockMessage with RpcBlock in rpc.proto.
* Convert everything in kaspad to use RPCBlocks and fix tests.
* Fix compilation errors in stability tests and the miner.
* Update TransactionVerboseData in rpc.proto.
* Update TransactionVerboseData in the rest of kaspad.
* Make golint happy.
* Include RpcTransactionVerboseData in RpcTransaction instead of the other way around.
* Regenerate rpc.pb.go after merge.
* Update appmessage types.
* Update appmessage request and response types.
* Reimplement conversion functions between appmessage.RPCTransaction and protowire.RpcTransaction.
* Extract RpcBlockHeader toAppMessage/fromAppMessage out of RpcBlock.
* Fix compilation errors in getBlock, getBlocks, and submitBlock.
* Fix compilation errors in getMempoolEntry.
* Fix compilation errors in notifyBlockAdded.
* Update verbosedata.go.
* Fix compilation errors in getBlock and getBlocks.
* Fix compilation errors in getBlocks tests.
* Fix conversions between getBlocks message types.
* Fix integration tests.
* Fix a comment.
* Add selectedParent to the verbose block response.
* Add mass limit to mempool
* Pass only params instead of multiple configuration options
* Remove acceptNonStd from mempool constructor
* Remove acceptNonStd from mempool constructor
* Fix test compilation
* [NOD-1429] add mining manager unit tests
* [NOD-1429] Add additional test
* found a bug, so stopped working on this test until the bug will be fix.
* Update miningmanager_test.go test.
* Delete payloadHash field - not used anymore in the current version.
* Change the condition for comparing slices instead of pointers.
* Fix due to review notes - change names, use testutils.CreateTransaction function and adds comments.
* Changes after fetch&merge to v0.10.0-dev
* Create a new function createChildTxWhenParentTxWasAddedByConsensus and add a comment
* Add an argument to create_transaction function and fix review notes
* Optimization
* Change to blockID(instead of the all transaction) in the error messages and fix review notes
* Change to blockID(instead of the all transaction) in the error messages and fix review notes
* Change format of error messages.
* Change name ofa variable
* Use go:embed to embed sample-kaspad.conf (only on go1.16)
* Revert "Use go:embed to embed sample-kaspad.conf (only on go1.16)"
This reverts commit bd28052b92.
Co-authored-by: karim1king <karimkaspersky@yahoo.com>
Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Add StagingArea struct
* Implemented staging areas in blockStore
* Move blockStagingShard to separate folder
* Apply staging shard to acceptanceDataStore
* Update blockHeaderStore with StagingArea
* Add StagingArea to BlockRelationStore
* Add StagingArea to blockStatusStore
* Add StagingArea to consensusStateStore
* Add StagingArea to daaBlocksStore
* Add StagingArea to finalityStore
* Add StagingArea to ghostdagDataStore
* Add StagingArea to headersSelectedChainStore and headersSelectedTipStore
* Add StagingArea to multisetStore
* Add StagingArea to pruningStore
* Add StagingArea to reachabilityDataStore
* Add StagingArea to utxoDiffStore
* Fix forgotten compilation error
* Update reachability manager and some more things with StagingArea
* Add StagingArea to dagTopologyManager, and some more
* Add StagingArea to GHOSTDAGManager, and some more
* Add StagingArea to difficultyManager, and some more
* Add StagingArea to dagTraversalManager, and some more
* Add StagingArea to headerTipsManager, and some more
* Add StagingArea to constnsusStateManager, pastMedianTimeManager
* Add StagingArea to transactionValidator
* Add StagingArea to finalityManager
* Add StagingArea to mergeDepthManager
* Add StagingArea to pruningManager
* Add StagingArea to rest of ValidateAndInsertBlock
* Add StagingArea to blockValidator
* Add StagingArea to coinbaseManager
* Add StagingArea to syncManager
* Add StagingArea to blockBuilder
* Update consensus with StagingArea
* Add StagingArea to ghostdag2
* Fix remaining compilation errors
* Update names of stagingShards
* Fix forgotten stagingArea passing
* Mark stagingShard.isCommited = true once commited
* Move isStaged to stagingShard, so that it's available without going through store
* Make blockHeaderStore count be avilable from stagingShard
* Fix remaining forgotten stagingArea passing
* commitAllChanges should call dbTx.Commit in the end
* Fix all tests tests in blockValidator
* Fix all tests in consensusStateManager and some more
* Fix all tests in pruningManager
* Add many missing stagingAreas in tests
* Fix many tests
* Fix most of all other tests
* Fix ghostdag_test.go
* Add comment to StagingArea
* Make list of StagingShards an array
* Add comment to StagingShardID
* Make sure all staging shards are pointer-receiver
* Undo bucket rename in block_store
* Typo: isCommited -> isCommitted
* Add comment explaining why stagingArea.shards is an array
* Use go:embed to embed sample-kaspad.conf (only on go1.16)
* Add a comment to justify the blank import.
* Change a variable name to sampleKaspad (instead configurationSampleKaspadString)
Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: Svarog <feanorr@gmail.com>
* Use BLAKE2B instead of HASH160, and get rid of any usage of RIPEMD160
* Change genesis coinbase payload script to OP_FALSE
* Fix tests after conflict
* Remove duplicate tests
* Change file name
* Change atomic swap to use proper hash size
* Move CalculateSignatureHash to consensushashing
* Added CalcSignatureHash_BIP143 with all parameters except the re-used hashes
* Add handling of outputHash
* Add sequencesHash to the mix
* Add previousOutputsHash to the mix
* Replace legacy CalculateSigHash with new one, and re-wire to all non-test code
* Add missing types to WriteElement
* Fix tests in txscript
* Fix tests in rest of code
* Add missing comments
* Add SubnetworkID and Gas to sigHash
* Add TestCalculateSignatureHash
* Invert condition in SigHashSingle getOutputsHash
* Explicitly define that payloadHash for native transactions is 0
* added benchmark to CalculateSignatureHash
* Reformat call for signAndCheck
* Change SigHashes to be true bit-fields
* Add check for transaction version
* Write length of byte array in WriteElement
* hashOutpoint should get outpoint, not txIn
* Use inputIndex instead of i to determine SigHashType
* Use correct transaction version + fix some typos
* Fix hashes in test
* Reformat an overly-long line
* Replace checkHashTypeEncoding with caalls to hashType.IsStandardSigHashType
* Convert hashType to uint8
* Add comment
* Replace blue score with DAA score in UTXO entries
* Use DAA score for coinbase maturity
* Use DAA score for sequence lock
* Fix calcBlockSubsidy to use DAA score
* Don't pay to blocks that are not included in the DAA added blocks, and bestow red blocks reward to the merging block
* Fix TestGetPruningPointUTXOs
* Fix TestTransactionAcceptance
* Fix TestChainedTransactions
* Fix TestVirtualDiff
* Fix TestBlockWindow
* Fix TestPruning
* Use NewFromSlice instead of manually creating the hash set
* Add assert
* Add comment
* Remove redundant call to UpdateDAADataAndReturnDifficultyBits
* Add RequiredDifficulty, rename UpdateDAADataAndReturnDifficultyBits to StageDAADataAndReturnRequiredDifficulty and add comments
* Make buildUTXOInvalidHeader get bits as an argument
* Fix comments
* [NOD-1424] Write general unit-tests for Reachability
* Update the tests of reachabilityManager.
* Add a diagram for the created DAG in the test.
* Change tabs to spaces in the diagram.
Co-authored-by: karim1king <karimkaspersky@yahoo.com>
Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
* Save DAA score and DAA added blocks for each block
* Add test
* Add pruning support
* Replace 8 with uint64Length
* Separate DAABlocksStore cache size to DAA score and daaAddedBlocks
* Move timesorter to its own package and remove unused functions
* Remove padding+genesis from BlockWindow
* Adjust the difficulty even when there's less than difficultyAdjustmentWindowSize blocks
* Remove unnecessary check from checkBlockTransactionsFinalized
* Update tests with new pastMedianTime and Difficulty
* Review nit
* Write better tests for red blocks and DAA
* Fix comments
* Fix blue chain size
* Remove high timestamps from blue chain
Co-authored-by: Svarog <feanorr@gmail.com>
* Refactor mergeSetIncrease to return the current BFS block to allow easier merging
* Remove unneeded Heap/HashSet usages
* Add new IsAnyAncestorOf to DagTopolyManager
* Check if the new candidate is in the future of any existing candidate
* Add comments and fix off-by-one in the mergeSetIncrease queue
* Fixed DAGToplogy test mock
* Fix review comments
* Handle errors in p2p handshake better
* Add a new errNil in protowire
* Add nil checks for all protowire.toAppMessage() functions for the p2p
* Add nil checks for all protowire.toAppMessage() functions for the RPC
* Add nil check for protwire KaspadMessage
Co-authored-by: Svarog <feanorr@gmail.com>
* Remove Subsystems map and replace with RegisterSubSystem
* Clean up the logger
* Fix LOGFLAGS and make LongFile work correctly
* Parallelize the logger backend
* More logger cleanup
* Initialize and close the logger backend wherever it's needed
* Move the location where the backend is closed, also print the log if it panics while writing
* Add TestMain to reachability manager tests to preserve the same log level
* Fix review comments
Co-authored-by: Svarog <feanorr@gmail.com>
* Add Close() function to all the iterators
* Add defer iterator.Close() whenever we open an iterator
* Add isClosed to all iterators and panic/return error if used after closing
Co-authored-by: Svarog <feanorr@gmail.com>
* resolveSingleBlockStatus: If the block being resolved is not going to be the next selectedTip - set it's diffParent to be the old selectedTip
* resolveSingleBlockStatus: If the block being resolved is going to be the next selectedTip - set it as old selectedTip's diffChild
* Remove any mentions of virtualDiffParents
* If block is genesis - don't do all the mumbo-jumbo with oldSelectedTip
* Check an unchecked error
* Write a better log message
* [NOD-1453] cover failing block validation
* [NOD-1453] Complete covering test for invalid block
* [NOD-1453] Fix validator tests after rebase
* [NOD-1453] Cover tests for valid blocks
* [NOD-1453] Implement unit tests for ValidateTransactionInIsolation
* [NOD-1453] Add tests for ValidateTransactionInContextAndPopulateMassAndFee
* [NOD-1453] Cover ValidateHeaderInContext test
* [NOD-1453] Fix after rebase
* not finish
* commited for update the branch.
* Adds new tests to block_body_in_isolation_test.go according to (and instead of ) blockvalisator_test.go
* Adds a comment to type MEDIAN.
* Fixes according to the review notes: add notes and change variables name.
* Fix comment.
* Remove an unused test( all the tests in this file were passed to other test files).
* Change a variable name(txWithAnEmptyInvalidScript to txWithInvalidSignature).
* adds missing '}'.
* Change spaces to tab
Co-authored-by: karim1king <karimkaspersky@yahoo.com>
Co-authored-by: Karim A <karim.a@it-dimension.com>
Co-authored-by: tal <tal@daglabs.com>
* Adds tests for validateDifficulty
* fixes according to the review notes: adding the test's goal and fix an unmatch test name on the NewTestConsensus.
* Fixes according to the review notes:delete the function genesisBits - No usages.
* Fix according to review - fix comments.
Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Make antiPastHashesBetween return blocks sorted in ghostdag-order
* Return sortedMergeSet instead of blueMergeSet
* Invert the order of parameters of IsAncestorOf
* Add RenderDAGToDot to TestConsensus
* Add HighHash explicitly, unless lowHash == highHash
* Use Equal instead of == when comparing hashes
* Fixed TestSyncManager_GetHashesBetween
* Fix tests
* findHighHashAccordingToMaxBlueScoreDifference: don't start looking if the whole thing fits
* Handle a missed error
* Remove redundant call to RenderToDot
* Fix bug in findHighHashAccordingToMaxBlueScoreDifference
* Move existing address/bannedAddress functionality to a new addressStore object.
* Implement TestAddressManager.
* Implement serializeAddressKey and deserializeAddressKey.
* Implement serializeNetAddress and deserializeNetAddress.
* Store addresses and banned addresses to disk.
* Implement restoreNotBannedAddresses.
* Fix bannedDatabaseKey.
* Implement restoreBannedAddresses.
* Implement TestRestoreAddressManager.
* Defer closing the database in TestRestoreAddressManager.
* Defer closing the database in TestRestoreAddressManager.
* Add a log.
* Return errors where appropriate.
Co-authored-by: Elichai Turkel <elichai.turkel@gmail.com>
* In PropagateUTXOsChangedNotifications, add the given addresses to the address list instead of replacing them.
* Add StopNotifyingUtxosChangedRequestMessage to rpc.proto.
* Implement StopNotifyingUTXOsChanged.
* Optimize convertUTXOChangesToUTXOsChangedNotification.
* Don't error out if antiPastHashesBetween have 2 blocks with the same blue score
* Prepend lowHash to RPC GetBlocks request
* Add a test for GetHashesBetween
* Add a test for GetBlocks RPC call
* Update antipast.go
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
* Remove BlockHexes from GetBlocks request and response
* Add GetBlocks RPC
* Include the selectedTip's anticone in GetBlocks
* Add Anticone to fakeRelayInvsContext
* Include verbose data only if it was requested + Add comments to HandleGetBlocks
* Allow antiPastHashesBetween to receive unrelated low and high hashes
* Convert to/from protowire GetBlocksResponse with no verbose data correctly
* Removed NextLowHash
* Update GetBlocks in rpc_client
* Validate in consensus.Anticone that blockHash exists
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
* Send peers the hash of the virtual selected parent once connection is established.
* Add a log to SendVirtualSelectedParentInv.
* Fix TestIBDWithPruning.
* Fix TestIBDWithPruning better and signal from the IBD syncer to the IBD syncee that the DAG is split amongst them.
* Fix TestVirtualSelectedParentChain.
* Add comments.
* Split deletePastBlocks into sub-routines
* Remove SelectedParentIterator, and refactor SelectedChildIterator to support First and Error
* Implement PruneAllBlocks
* Prune all blocks in the store
* Prune only blocks that are below the pruning point
* Defer call onEnd of LogAndMeasureExecutionTime
* Handle a forgotten error
* Minor style fixes
* Adds tests for the "blockAtDepth" function and verify old other tests.
* Optimization on create the Dag chain.
* Changes according to the review - more detailed error messages, added constants, changed to 3 independent graphs (instead of extending), and changes all the abbreviations.
* Changes according to the review - divide the test into three separate tests and change names to variables.
* Changes according to the review - the order of the function has changed.
* delete double lines
Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: Svarog <feanorr@gmail.com>
* Make leveldb cache configurable
* Fix leveldb tests
* Add a preallocate option to all caches and disable in tests
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
* Move testGHOSTDagSorter to testutils
* Allow overriding managers in consensus, starting with ghostdag
* Add test prefix to SetDataDir and SetGHOSTDAGManager
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
* Allow GetMissingBlockBodyHashes return an empty list if the missing blocks were requested before IBD start
* Add link to issue in comment about error to be fixed
* Limit mempool size to million transactions and remove the least profitable transactions
* Simplify insert
* Fix typo
* Improve findTxIndexInOrderedTransactionsByFeeRate readability
* Refactor miner and mine when waiting for block to validate
* Fix -n to work after the refactor.
Change foundBlockChan capacity.
Use lock instead of atomic in the template manager.
* Fix self assignment
* Fix lock
* Fix Dockerfile
* Add comment
* Add dummy go files for test only package, to mitigate golang/go#27333
* Stop ignoring errors when producing the coverage
* Add comments explaining the dummy go files
* Make the coverage output non-json
* Use hard-coded sample config instead of assumed path
* Fix bad path to sample-kaspad.conf in TestCreateDefaultConfigFile
Co-authored-by: Elichai Turkel <elichai.turkel@gmail.com>
* Reuse the ticker in ConnectionManager.waitTillNextIteration
* Fix a data race in ConnectionManager by locking the mutex
* Add a mutex to fakeRelayInvsContext in block relay flow test
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Prevent infinite tickers leaks in kaspaminer
* Reset ticker in ConnectionManager instead of allocating a new one
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* In kaspaminer, don't crash on submitBlock timeout.
* Make timeout messages have a log level of Warn.
* Wait for a second after receiving a reject for IBD.
Co-authored-by: Elichai Turkel <elichai.turkel@gmail.com>
* Mine JSON
* [Reindex tests] add test_params and validate_mining flag to test_consensus
* Rename file and extend tests
* Ignore local test datasets
* Use spaces over tabs
* Reindex algorithm - full algorithm, initial commit, some tests fail
* Reindex algorithm - a few critical fixes
* Reindex algorithm - move reindex struct and all related operations to new file
* Reindex algorithm - added a validateIntervals method and modified tests to use it (instead of exact comparisons)
* Reindex algorithm - modified reindexIntervals to receive the new child as argument and fixed an important related bug
* Reindex attack tests - move logic to helper function and add stretch test
* Reindex algorithm - variable names and some comments
* Reindex algorithm - minor changes
* Reindex algorithm - minor changes 2
* Reindex algorithm - extended stretch test
* Reindex algorithm - small fix to validate function
* Reindex tests - move tests and add DAG files
* go format fixes
* TestParams doc comment
* Reindex tests - exact comparisons are not needed
* Update to version 0.8.6
* Remove TestParams and use AddUTXOInvalidHeader instead
* Use gzipeed test files
* This unintended change somehow slipped in through branch merges
* Rename test
* Move interval increase/decrease methods to reachability interval file
* Addressing a bunch of minor review comments
* Addressed a few more minor review comments
* Make code of offsetSiblingsBefore and offsetSiblingsAfter symmetric
* Optimize reindex logic in cases where reorg occurs + reorg test
* Do not change reindex root too fast (on reorg)
* Some comments
* A few more comments
* Addressing review comments
* Remove TestNoAttackAlternateReorg and assert chain attack
* Minor
Co-authored-by: Elichai Turkel <elichai.turkel@gmail.com>
Co-authored-by: Mike Zak <feanorr@gmail.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Check for MsgUnexpectedPruningPoint on receivePruningPointBlock as well
* Add IBD test cases
* Revert "Check for MsgUnexpectedPruningPoint on receivePruningPointBlock as well"
This reverts commit 6a6d1ea180.
* Change log level for two logs
* Remove "testing a situation where the pruning point moved during IBD (before sending the pruning point block)"
* In kaspactl, prettify responses before printing them.
* Indent with four spaces instead of a tab.
* Unwrap the response.
* Simplify unwrapping the response.
* Don't unwrap responses.
* Use protojson.MarshalOptions for prettification.
* Add request_types and their help
* Added command parser
* Updated main to use command if it's specified
* Some progress in making everything work
* Fix command parser for pointers to structs
* Cleanup code
* Enhance usage text
* Fixed typo
* Some minor style fixing, and remove temporary code
* Correctly fallthrough in stringToValue unsupported types
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
* Ignore ErrDuplicateBlock for any blocks that were requested
* Don't check for DuplicateBlockErr in processHeaders + improve logs
* Return the check for ErrDuplicateBlock in processHeader
* Fix log message
* Implement TestGetPruningPointUTXOs.
* Fix a bad error message.
* Fix TestGetPruningPointUTXOs for testnet.
* Make sure all the UTXOs are returned in TestGetPruningPointUTXOs.
* Implement BenchmarkGetPruningPointUTXOs.
* Pass fromOutpoint into GetPruningPointUTXOs instead of offset, so it could Seek over the cursor.
* Fix weird benchmark timer calls.
* Remove unnecessary collection of outpointAndUTXOEntryPairs from BenchmarkGetPruningPointUTXOs.
* Fix a comment.
* Remove pruningPointUTXOSetStaging and implement UpdatePruningPointUTXOSet.
* Implement StageStartSavingNewPruningPointUTXOSet, HadStartedSavingNewPruningPointUTXOSet, and FinishSavingNewPruningPointUTXOSet.
* Fix a bad return.
* Implement UpdatePruningPointUTXOSetIfRequired.
* Call UpdatePruningPointUTXOSetIfRequired on consensus creation.
* Rename savingNewPruningPointUTXOSetKey to updatingPruningPointUTXOSet.
* Add a log.
* Add calls to runtime.GC() at its start and end.
* Rename a variable.
* Replace calls to runtime.GC to calls to LogMemoryStats.
* Wrap the contents of LogMemoryStats in a log closure.
* Replaced the content of MsgIBDRootUTXOSetChunk with pairs of outpoint-utxo entry pairs.
* Rename utxoIter to utxoIterator.
* Add a big stinky TODO on an assert.
* Replace pruningStore staging with a UTXO set iterator.
* Reimplement receiveAndInsertIBDRootUTXOSet.
* Extract OutpointAndUTXOEntryPairsToDomainOutpointAndUTXOEntryPairs into domainconverters.go.
* Pass the outpoint and utxy entry pairs to the pruning store.
* Implement InsertCandidatePruningPointUTXOs.
* Implement ClearCandidatePruningPointUTXOs.
* Implement UpdateCandidatePruningPointMultiset.
* Use the candidate pruning point multiset in updatePruningPoint.
* Implement CandidatePruningPointUTXOIterator.
* Use the pruning point utxo set iterator for StageVirtualUTXOSet.
* Defer ClearCandidatePruningPointUTXOs.
* Implement OverwriteVirtualUTXOSet.
* Implement CommitCandidatePruningPointUTXOSet.
* Implement BeginOverwritingVirtualUTXOSet and FinishOverwritingVirtualUTXOSet.
* Implement overwriteVirtualUTXOSetAndCommitPruningPointUTXOSet.
* Rename ClearCandidatePruningPointUTXOs to ClearCandidatePruningPointData.
* Add missing methods to dbManager.
* Implement PruningPointUTXOs.
* Implement RecoverUTXOIfRequired.
* Delete the utxoserialization package.
* Fix compilation errors in TestValidateAndInsertPruningPoint.
* Switch order of operations in the if statements in PruningPointUTXOs so that Next() wouldn't be unnecessarily called.
* Fix missing pruning point utxo set staging and bad slice length.
* Fix no default multiset in InsertCandidatePruningPointUTXOs.
* Make go vet happy.
* Rename candidateXXX to importedXXX.
* Do some more renaming.
* Rename some more.
* Fix bad MsgIBDRootNotFound logic.
* Fix an error message.
* Simplify receiveIBDRootBlock.
* Fix error message in receiveAndInsertIBDRootUTXOSet.
* Do some more renaming.
* Fix merge errors.
* Fix a bug caused by calling iterator.First() unnecessarily.
* Remove databaseContext from stores and don't use a transaction in ClearXXX functions.
* Simplify receiveAndInsertIBDRootUTXOSet.
* Fix offset count in PruningPointUTXOs().
* Fix readOnlyUTXOIteratorWithDiff.First().
* Split handleRequestIBDRootUTXOSetAndBlockFlow into smaller methods.
* Rename IbdRootNotFound to UnexpectedPruningPoint.
* Rename requestIBDRootHash to requestPruningPointHash.
* Rename IBDRootHash to PruningPointHash.
* Rename RequestIBDRootUTXOSetAndBlock to RequestPruningPointUTXOSetAndBlock.
* Rename IBDRootUTXOSetChunk to PruningPointUTXOSetChunk.
* Rename RequestNextIBDRootUTXOSetChunk to RequestNextPruningPointUTXOSetChunk.
* Rename DoneIBDRootUTXOSetChunks to DonePruningPointUTXOSetChunks.
* Rename remaining references to IBD root.
* Fix an error message.
* Add a check for HadStartedImportingPruningPointUTXOSet in commitVirtualUTXODiff.
* Add a check for HadStartedImportingPruningPointUTXOSet in ImportPruningPointUTXOSetIntoVirtualUTXOSet.
* Move FinishImportingPruningPointUTXOSet closer to HadStartedImportingPruningPointUTXOSet.
* Remove reference to pruningStore in utxoSetIterator.
* Pointerify utxoSetIterator receivers.
* Fix bad insert in CommitImportedPruningPointUTXOSet.
* Rename commitImportedPruningPointUTXOSetAll to applyImportedPruningPointUTXOSet.
* Simplify PruningPointUTXOs.
* Add populateTransactionWithUTXOEntriesFromUTXOSet.
* Fix a TODO comment.
* Rename InsertImportedPruningPointUTXOs to AppendImportedPruningPointUTXOs.
* Extract handleRequestPruningPointUTXOSetAndBlockMessage to a separate method.
* Rename stuff in readOnlyUTXOIteratorWithDiff.First().
* Address toAddIterator in readOnlyUTXOIteratorWithDiff.First().
* Call First() before any full iteration on ReadOnlyUTXOSetIterator.
* Call First() before any full iteration on a database Cursor.
* Put StartImportingPruningPointUTXOSet inside the pruning point transaction.
* Make serializeOutpoint and serializeUTXOEntry free functions in pruningStore.
* Fix readOnlyUTXOIteratorWithDiff.First().
* Fix bad validations in importPruningPoint.
* Remove superfluous call to validateBlockTransactionsAgainstPastUTXO.
* Add the Address of node to whom connected in log of send/receiveVersion
* Don't call functions before LogAndMeasureExecutionTime
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Fix checkParentHeadersExist and cover pruning_violation_proof_of_work_and_difficulty.go with tests
* Remove unused variable
* Change consensus violation
* Change condition order
* Get rid of irrelevant error codes in extractRejectCode
* Fix wrong test db names
* Fix checkParentHeadersExist
* Extract syncPruningPointUTXOSet to a separate method.
* Implement logic to send pruning point utxo set chunks in a loop.
* Replace IBDRootUTXOSetAndBlockMessage with IbdRootUtxoSetChunkMessage.
* Add a new message: RequestNextIBDRootUTXOSetChunk.
* Add a new message: DoneIBDRootUTXOSetChunks.
* Protect HandleRequestIBDRootUTXOSetAndBlock from rogue messages.
* Reimplement receiveIBDRootUTXOSetAndBlock.
* Add CmdDoneIBDRootUTXOSetChunks to the HandleRelayInvs flow.
* Decrease the max message size to 10mb.
* Fix bad step.
* Fix confusion between outgoing/incoming routes.
* Measure how long it takes to send/receive the UTXO set.
* Use LogAndMeasure in handleRequestIBDRootUTXOSetAndBlockFlow.
* Prevent a race condition in findHighestSharedBlockHash where we get headersSelectedTip and then pass it as highHash to GetBlockLocator, without locking consensus
* Restart findHighestSharedBlockHash if lowHash or highHash are no longer in selectedParentChain
* Test for specifically ErrBlockNotInSelectedParentChain instead of database NotFound error
* Fix TestCreateHeadersSelectedChainBlockLocator
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Log the hashrate of each block
* Add a test for GetHashrateString
* Move difficulty related functions to its own package
* Convert the validated log in validateAndInsertBlock to a log function
* Add tests for max/min int
* Adds test for error cases on the function checkBlockStatus.
* Fix review's comments.
* Move test to validateandinsertblock_test.go
Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Increase default log sizes, and increase kaspad log sizes
* Add an option to not print logs to stdout
* Allow logs to be printed in the current working directory
* Add more pruning related logs
* Add comment and increase log rotations to save last 64 logs
* Fix UTXO serialization, its test, and the static check that missed it
* Remove duplicate case
* Use one line for static check
Co-authored-by: Elichai Turkel <elichai.turkel@gmail.com>
* Add selected chain store and optimize block locator with it
* Fix build error
* Fix comments
* Fix IsStaged
* Rename CalculateSelectedParentChainChanges to CalculateChainPath and SelectedParentChainChanges->SelectedChainPath
* Use binary.LittleEndian directly to allow compiler optimizations
* Remove boolean from HeadersSelectedChainStore interface
* Prevent endless loop in block locator
* Initalize protoUTXOSetIterator with index = -1
* Handle error when failed to deserialize Script version
* Add support for (de)serialization of (u)int16
* Log the error when converting it into ErrMalformedUTXO
* commit for do fetch&merge
* Adds a new test to validate POW, and Fix Main-net and Testnet genesis block data.
* Fix window's test for testnet and change the expected pruning point for mainnet and testnet.
* Delete function "solveBlock" on proof_of_work_test.go and call the function mining.SolvaBlock instead. Also, remove using of random in "solveBlockWithWrongPOW" function.
* Replace 0xFFFFFFFFFFFFFFFF to math.MaxUint64 in "solveBlockWithWrongPOW" function and change the function's comment of "TestPOW"
* Replace 0xFFFFFFFFFFFFFFFF to math.MaxUint64 in "solveBlockWithWrongPOW" function and change the function's comment of "TestPOW"
* Change from <= to < in the for statement in "solveBlockWithWrongPOW" function
* Adds one arg to the function call "NewTestConsensus" (the function sig has changed).
Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Make sure block locator doesn't include a hash lower than the lowHash in
the block locator
* Use finalityStore to optimize LowestChainBlockAboveOrEqualToBlueScore
* Add archival cli flag
* If --archival was activated - don't delete anything
* Fix tests
* Still change block status to StatusHeaderOnly even in archival nodes
* Split messages.proto to p2p and rpc.
* Split messages.proto to p2p and rpc.
* Write a short intro to the RPC docs.
* Start documenting RPC calls.
* Use a custom protoc-gen-doc.
* Continue writing RPC documentation.
* Finish writing RPC documentation.
* Fix a formatting error.
* Fix merge errors.
* Fix formatting into protowire/README.md.
* Rerun go generate ..
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* ''
* ''
* ''
* Changes genesis block version to 0.
* a
* a
* All tests are done.
* All tests passed for changed block version from int32 to uint16
* Adds validation of rejecting blocks with unknown versions.
* Changes txn version from int32 to uint16.
* .
* Adds comments to exported functions.
* Change functions name from ConvertFromRpcScriptPubKeyToRPCScriptPubKey to ConvertFromAppMsgRPCScriptPubKeyToRPCScriptPubKey and from ConvertFromRPCScriptPubKeyToRpcScriptPubKey to ConvertFromRPCScriptPubKeyToAppMsgRPCScriptPubKey
* change comment to "ScriptPublicKey represents a Kaspad ScriptPublicKey"
* delete part (tx.Version < 0) that cannot be exist on the if statement.
* Revert protobuf version.
* Fix a comment.
* Fix a comment.
* Rename a variable.
* Rename a variable.
* Remove a const.
* Rename a type.
* Rename a field.
* Rename a field.
* Remove commented-out code.
* Remove dangerous nil case in DomainTransactionOutput.Clone().
* Remove a constant.
* Fix a string.
* Fix wrong totalScriptPubKeySize in transactionMassStandalonePart.
* Remove a constant.
* Remove an unused error.
* Fix a serialization error.
* Specify version types to be uint16 explicitly.
* Use constants.ScriptPublicKeyVersion.
* Fix a bad test.
* Remove some whitespace.
* Add a case to utxoEntry.Equal().
* Rename scriptPubKey to scriptPublicKey.
* Remove a TODO.
* Rename constants.
* Rename a variable.
* Add version to parseShortForm.
Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: stasatdaglabs <stas@daglabs.com>
* Convert BlockGHOSTDAGData from an interface to a public struct with getters
* Move hashes.Less to externalapi so it can access the hashes directly without copying
* Reduce calls to ghostdagstore.Get in blueWindow
* Simplify the logic in RequiredDifficulty and reuse big.Int
* Remove bigintpool as its no longer used
* Use ChooseSelectedParent in RequiredDifficulty instead of looping over the parents
* Remove comment
* bug invalidateAndInsertPruningPoint: if ValidateAndInsertBlock returned a non-RuleError error - the error was ignored
* Convert checkNoPrefilledFields into checkNoPrefilledInputs
* Add log line
* clone pruning point when passing to validateBlockTransactionsAgainstPastUTXO
* Add recoverability for UTXO index
* Add comment
* Rename UTXOOutpointPair->OutpointUTXOPair
* Get rid of the db transaction on resetStore and collect all keys before deleting
* Use VirtualSelectedParent instead of selected tip
* Fix error
* Add a new message: BlockHeadersMessage.
* Add a new message: BlockHeadersMessage.
* Send a lot of headers as a single message instead of many small messages.
* Keep a short queue of blockHeadersMessages so that there's never a moment when the node is not validating and inserting headers
* Add a missing return statement.
* Remove MsgBlockHeader from payloads.
* Send headers continuously without needing to run the BlockLocator protocol after ever ~maxBlueScoreDifference blocks
* Add logging.
* Make logs more descriptive.
* Change SyncRateWindow to 15 minutes + update sync times on block headers as well
* Rename result to isSyncRateTooLow
* Fix formula for expected blocks
* Add a field to TransactionAcceptanceData: TransactionInputUTXOEntries.
* Fix failing tests.
* Add transactionInputUtxoEntries to the database.
* Populate transactionInputUTXOEntries in applyMergeSetBlocks and use them in the UTXO index.
* Remove UTXOEntry.Clone().
* Add an additional equality test.
* Check that there are no prefilled fields when validating a block
* Use cleanBlockPrefilledFields in AddBlock
* Move cleanBlockPrefilledFields to BuildBlockWithParents
* Move cleanBlockPrefilledFields to func (bb *testBlockBuilder) BuildBlockWithParents
* Add a new message: BlockLocatorHighestHash.
* Add a new message: IBDBlockLocator.
* Implement HandleIBDBlockLocator.
* Reimplement findHighestSharedBlockHash.
* Make HandleIBDBlockLocator only return hashes that are in the selected parent chain of the target hash.
* Increase the cache sizes of blockRelationStore, reachabilityDataStore, and ghostdagDataStore.
* Fix wrong initial highHash in findHighestSharedBlockHash.
* Make go vet happy.
* Protect against receiving wrong messages when expecting MsgIBDBlockLocatorHighestHash.
* Rename reachabilityManager.data to dataForInsertion, and use it only during insertions
* Make reachabilityData an interface
* Fix db serialization of reachability data
* Fix reachabilityDataStore
* Fix all tests
* Cleanup debugging code
* Fix insertToFutureCoveringSet
* Add comments
* Rename to ReachabilityData and MutableReachabilityData
* if more candidates then max, choose half with highest blueWork and half with lowest
* Add a Test GhostDAG sorter
* Add a test for pick virtual parents
* Fix review nits
* Fix notifyVirtualSelectedParentBlueScoreChanged to show the selected tip blue score instead of the virtual's
* Fix ShouldMine() to fetch selected tip header
* Remove the limit on the amount of hashes returned from antiPastHashesBetween.
* Guard against requests with a non-existing block hash.
* Move missing-block-hash guards to consensus.go.
* Ban a peer that doesn't send us all the requested headers during IBD.
* Extract blockHeap.ToSlice.
* Re-request headers in requestHeaders if we didn't receive the highHash.
* Increase size of reachability cache
* Change DomainHash to struct with unexported hashArray
* Fixing compilation errors stemming from new DomainHash structure
* Remove obsolete Read/WriteElement methods in appmessage
* Fix all tests
* Fix all tests
* Add comments
* A few renamings
* go mod tidy
* 1) Calculate pruning point incrementally
2) Add IsValidPruningPoint to pruning manager and consensus
3) Use reachability children for selected child iterator
* Add request IBD root hash flow
* Fix UpdatePruningPointByVirtual and IsValidPruningPoint
* Regenerate messages.pb.go
* Make the pruning point the earliest chain block with finality interval higher than the previous pruning point
* Fix merge errors
* Copy over the CLI wallet from Kasparov.
* Fix trivial compilation errors.
* Reimplement the balance command.
* Extract isUTXOSpendable to a separate function.
* Reimplement the send command.
* Fix bad transaction ID parsing.
* Add a missing newline in a log.
* Don't use msgTx in send().
* Fix isUTXOSpendable not checking whether a UTXO is of a coinbase transaction.
* Add --devnet, --testnet, etc. to command line flags.
* In `create`, only print the public key of the active network.
* Use coinbase maturity in isUTXOSpendable.
* Add a readme.
* Fix formatting in readme.
* [NOD-1575] Implement Clone and Equal for all model types
* [NOD-1575] Add assertion for transaction ID equality
* [NOD-1575] Use DomainTransaction.Equal to compare to expected coinbase transaction
* [NOD-1575] Add TestDomainBlockHeader_Clone
* [NOD-1575] Don't clone nil values
* [NOD-1575] Add type assertions
* [NOD-1575] Don't clone nil values
* [NOD-1575] Add missing Equals
* [NOD-1575] Add length checks
* [NOD-1575] Update comment
* [NOD-1575] Check length for TransactionAcceptanceData
* [NOD-1575] Explicitly clone nils where needed
* [NOD-1575] Clone tx id
* [NOD-1575] Flip condition
* Nod 1576 make coverage tests for equal clone inside model externalapi (#1177)
* [NOD-1576] Make coverage tests for equal and clone inside model and externalapi
* Some formatting and naming fixes
* Made transactionToCompare type exported
* Added some tests and made some changes to the tests code
* No changes made
* Some formatting and naming changes made
* Made better test coverage for externalapi clone and equal functions
* Changed expected result for two cases
* Added equal and clone functions tests for ghostdag and utxodiff
* Added tests
* [NOD-1576] Implement reachabilitydata equal/clone unit tests
* [NOD-1576] Full coverage of reachabilitydata equal/clone unit tests
* Made changes and handling panic to transaction_equal_clone_test.go and formating of utxodiff_equal_clone_test.go
* Added recoverForEqual2 for handling panic to transaction_equal_clone_test.go
* [NOD-1576] Full coverage of transaction equal unit test
* [NOD-1576] Add expects panic
* [NOD-1576] Allow composites in go vet
* [NOD-1576] Code review fixes (#1223)
* [NOD-1576] Code review fixes
* [NOD-1576] Code review fixes part 2
* [NOD-1576] Fix wrong name
Co-authored-by: karim1king <karimkaspersky@yahoo.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: Karim <karim1king@users.noreply.github.com>
* Fix merge errors
* Use Equal where possible
* Use Equal where possible
* Use Equal where possible
Co-authored-by: andrey-hash <74914043+andrey-hash@users.noreply.github.com>
Co-authored-by: karim1king <karimkaspersky@yahoo.com>
Co-authored-by: Karim <karim1king@users.noreply.github.com>
* comments on default constants
* more comments on default constants
* more comments on default constants
* more comments on default constants
* gofmt
* small typos
* Replace default hasher (Double-SHA256) with domain seperated blake2b
* Replace all hashes with domain seperated blake2b
* Update the genesis blocks
* Replace OP_HASH256 with OP_BLAKE2B
* Fix the merkle tree by appending zeros instead of duplicating the hash when there is 1 branch left
* Update tests
* Add a payloadHash function
* Update gitignore to ignore binaries
* Fix a bug in the blake2b opcode
* Add a test script
* add gh action for build and test
* added all the test
* Change github workflow to use the new test script
* Change the docker file to use the new test script
* Add doc comment for ProtocolError.Unwrap()
* Use another github action to increase windows page size
* Run the action after any edit to the PR metadata/base
* Change go version from 1.15 to 1.14
* Rename test.sh to build_and_test.sh
Co-authored-by: Isabella Liu <isabellaliu77@gmail.com>
* Get rid of insertMode
* Rename AddBlockToVirtual->AddBlock
* When F is not in the future of P, enforce finality with P and not with F.
* Don't allow blocks with invalid parents or with missing block body
* Check finality violation before checking block status
* Implement CalculateIndependentPruningPoint
* Move checkBlockStatus to validateBlock
* Add ValidateBlock to block processor interface
* Adjust SetPruningPoint to the new IBD flow
* Add pruning store to CSM's constructor
* Flip wrong condition on AddHeaderTip
* Fix func (hts *headerSelectedTipStore) Has
* Fix block stage order
* Call to ValidateBodyInContext from validatePostProofOfWork
* Enable overrideDAGParams
* Update log
* Rename SetPruningPoint to ValidateAndInsertPruningPoint and move most of its logic inside block processor
* Rename hasValidatedHeader->hasValidatedOnlyHeader
* Fix typo
* Name return values for fetchMissingUTXOSet
* Add comment
* Return ErrMissingParents when block body is missing
* Add logs and comments
* Fix merge error
* Fix pruning point calculation to be by virtual selected parent
* Replace CalculateIndependentPruningPoint to CalculatePruningPointByHeaderSelectedTip
* Fix isAwaitingUTXOSet to check pruning point by headers
* Change isAwaitingUTXOSet indication
* Remove IsBlockInHeaderPruningPointFuture from BlockInfo
* Fix LowestChainBlockAboveOrEqualToBlueScore
* Add validateNewPruningPointTransactions
* Add validateNewPruningAgainstPastUTXO
* Rename set_pruning_utxo_set.go to update_pruning_utxo_set.go
* Check missing block body hashes by missing block instead of status
* Validate pruning point against past UTXO with the pruning point as block hash
* Remove virtualHeaderHash
* Fix comment
* Fix imports
* Expose CheckProofOfWork from model/pow
* Update blockvalidator to call the new CheckProofOfWork
* Update genesis blocks
* Update tools to use the new CheckProofOfWork
* Update tests with new PoW
* Change DifficultyAdjustmentWindowSize and TimestampDeviationTolerance from uint64 to int
* refactor block_heap for readability and usage
* Add a new SizedUpHeap
* Refactor BlueWindow with the new DAA
* Update TestBlueBlockWindow with the new DAA window
* Fix review requested changes
* [NOD-1579] Remove selected tip hash messages.
* [NOD-1579] Start moving IBD stuff into blockrelay.
* [NOD-1579] Rename relaytransactions to transactionrelay.
* [NOD-1579] Move IBD files into blockrelay.
* [NOD-1579] Remove flow stuff from ibd.go.
* [NOD-1579] Bring back IsInIBD().
* [NOD-1579] Simplify block relay flow.
* [NOD-1579] Check orphan pool for missing parents to avoid unnecessary processing.
* [NOD-1579] Implement processOrphan.
* [NOD-1579] Implement addToOrphanSetAndRequestMissingParents.
* [NOD-1579] Fix TestIBD.
* [NOD-1579] Implement isBlockInOrphanResolutionRange.
* [NOD-1579] Implement limited block locators.
* [NOD-1579] Add some comments.
* [NOD-1579] Specifically check for StatusHeaderOnly in blockrelay.
* [NOD-1579] Simplify runIBDIfNotRunning.
* [NOD-1579] Don't run IBD if it is already running.
* [NOD-1579] Fix a comment.
* [NOD-1579] Rename mode to syncInfo.
* [NOD-1579] Simplify validateAndInsertBlock.
* [NOD-1579] Fix bad SyncStateSynced condition.
* [NOD-1579] Implement validateAgainstSyncStateAndResolveInsertMode.
* [NOD-1579] Use insertModeHeader.
* [NOD-1579] Add logs to TrySetIBDRunning and UnsetIBDRunning.
* [NOD-1579] Implement and use dequeueIncomingMessageAndSkipInvs.
* [NOD-1579] Fix a log.
* [NOD-1579] Fix a bug in createBlockLocator.
* [NOD-1579] Rename a variable.
* [NOD-1579] Fix a slew of bugs in missingBlockBodyHashes and selectedChildIterator.
* [NOD-1579] Fix bad chunk size in syncMissingBlockBodies.
* [NOD-1579] Remove maxOrphanBlueScoreDiff.
* [NOD-1579] Fix merge errors.
* [NOD-1579] Remove a debug log.
* [NOD-1579] Add logs.
* [NOD-1579] Make various go quality tools happy.
* [NOD-1579] Fix a typo in a variable name.
* [NOD-1579] Fix full blocks over header-only blocks not failing the missing-parents validation.
* [NOD-1579] Add an error log about a condition that should never happen.
* [NOD-1579] Check all antiPast hashes instead of just the lowHash's anticone to filter for header-only blocks.
* [NOD-1579] Remove the nil stuff from GetBlockLocator.
* [NOD-1579] Remove superfluous condition in handleRelayInvsFlow.start().
* [NOD-1579] Return a boolean from requestBlock instead of comparing to nil.
* [NOD-1579] Fix a bad log.Debugf.
* [NOD-1579] Remove a redundant check.
* [NOD-1579] Change an info log to a warning log.
* [NOD-1579] Move OnNewBlock out of relayBlock.
* [NOD-1579] Remove redundant exists check from runIBDIfNotRunning.
* [NOD-1579] Fix bad call to OnNewBlock.
* [NOD-1579] Remove an impossible check.
* [NOD-1579] Added a log.
* [NOD-1579] Rename insertModeBlockWithoutUpdatingVirtual to insertModeBlockBody.
* [NOD-1579] Add a check for duplicate headers.
* [NOD-1579] Added a comment.
* [NOD-1579] Tighten a stop condition.
* [NOD-1579] Simplify a log.
* [NOD-1579] Clarify a log.
* [NOD-1579] Move a log.
* Replace blueScore with blueWork in ghostDAG SelectedParent selection
* Add blueWork to protopuf ghostdag data
* Auto generate protobuf go code
* Serialize/Deserialize blueWork when converting to protobuf
* pass block header store to ghostdagmanager
* Convert tal's ghostdag2 implementation to blueWork
* Change finality test to check the blueWork instead of blueScore
* Update ghostdag_test to pass blockHeaderStore to ghostdag, and test all networks genesis headers
* Add sanity blueWork check to ghostdag_test
* [NOD-1582] Fix multiple request per missing ancestor
* [NOD-1582] Don't remove peer on routerpkg.ErrRouteClosed from RPC
* [NOD-1582] Use LogAndMeasureExecutionTime where possible
commit 3830df34b2
Merge: 46dc2e97717e7819c2
Author: Elichai Turkel <elichai.turkel@gmail.com>
Date: Tue Dec 1 16:29:51 2020 +0200
Merge pull request #1170 from kaspanet/tal-ghost-fix
Fix GhostDAG tests and jsons
commit 17e7819c27
Author: Elichai Turkel <elichai.turkel@gmail.com>
Date: Tue Dec 1 16:24:01 2020 +0200
Remove non-json ghostdag tests
commit 4bebb1d96a
Author: Elichai Turkel <elichai.turkel@gmail.com>
Date: Tue Dec 1 13:26:06 2020 +0200
Add a coment above tal's ghostdag2 impl
commit faf21a042e
Author: Elichai Turkel <elichai.turkel@gmail.com>
Date: Tue Dec 1 13:20:08 2020 +0200
fix the interfaces after merge
commit a8b7a25b2e
Merge: af91b69b2f1c6df48c
Author: Elichai Turkel <elichai.turkel@gmail.com>
Date: Tue Dec 1 13:19:08 2020 +0200
Merge branch 'v0.8.2-dev' into tal-ghost-fix
commit af91b69b20
Author: Elichai Turkel <elichai.turkel@gmail.com>
Date: Tue Dec 1 13:18:41 2020 +0200
Fix the non-json tests
commit c56f34b73b
Author: Elichai Turkel <elichai.turkel@gmail.com>
Date: Tue Dec 1 13:18:17 2020 +0200
Fix the jsons
commit 46dc2e9773
Author: tal <tal@daglabs.com>
Date: Mon Nov 30 17:15:20 2020 +0200
[NOD - 1143] Cosmetics changes.
commit b28e5ce816
Author: tal <tal@daglabs.com>
Date: Mon Nov 30 15:48:08 2020 +0200
[#1126] Place selectedParent to be first on blueMergeSet.
commit 4b56ed2da9
Author: tal <tal@daglabs.com>
Date: Mon Nov 30 14:51:50 2020 +0200
[#1126] Change pacement between blockRight and blockLeft .
commit b09f31be93
Merge: e17a98b7b0db39833f
Author: talelbaz <63008512+talelbaz@users.noreply.github.com>
Date: Mon Nov 30 14:30:22 2020 +0200
Merge pull request #1162 from kaspanet/new-jsons
Update the dag json tests
commit e17a98b7ba
Author: tal <tal@daglabs.com>
Date: Mon Nov 30 14:08:25 2020 +0200
[#1126] Use WALK function in tests & cosmetic changes.
commit 0db39833f3
Author: Elichai Turkel <elichai.turkel@gmail.com>
Date: Mon Nov 30 12:20:13 2020 +0200
Update the dag json tests
commit 5a3da43dd4
Author: tal <tal@daglabs.com>
Date: Sun Nov 29 12:03:37 2020 +0200
[NOD-1433] Remove unneccessry code.
commit a6cde558ac
Author: tal <tal@daglabs.com>
Date: Mon Nov 23 17:05:56 2020 +0200
[NOD-1433] Change "Stage" sig function according to the new interface - added error as a return type.
commit 07859b6218
Author: tal <tal@daglabs.com>
Date: Mon Nov 23 17:03:26 2020 +0200
[NOD-1433] Print formats changed & Cosmetics code changes.
commit e1a851664e
Author: tal <tal@daglabs.com>
Date: Sun Nov 15 17:34:59 2020 +0200
[NOD-1433] Travers the tests dir and run each test.
commit 4c7474edc1
Author: tal <tal@daglabs.com>
Date: Mon Nov 9 12:44:53 2020 +0200
[NOD-1433] Travers the tests dir and run each test.
commit 89dd1e61d3
Author: tal <tal@daglabs.com>
Date: Mon Nov 9 11:48:36 2020 +0200
[NOD-1433] Change implementation to adjust genesis's score 0.
Also, keep changing the test file to fit the new implementation.
commit 6acdcd17de
Author: tal <tal@daglabs.com>
Date: Sun Nov 8 17:07:22 2020 +0200
[NOD-1433] New test was added(Test 6).
commit bf23889317
Author: tal <tal@daglabs.com>
Date: Sun Nov 8 14:59:36 2020 +0200
Fix golint errors
commit 79ff990b5f
Author: tal <tal@daglabs.com>
Date: Sun Nov 8 14:47:12 2020 +0200
added "Optimize imports".
commit 73d0128f63
Author: tal <tal@daglabs.com>
Date: Sun Nov 8 13:03:22 2020 +0200
Added an implementation factory.
commit 61ca8b2e7e
Author: tal <tal@daglabs.com>
Date: Thu Nov 5 16:03:18 2020 +0200
1. impl - choose the highest hash.
2. test - changed the test accordingly.
commit ef0943ca29
Author: tal <tal@daglabs.com>
Date: Thu Oct 29 18:00:45 2020 +0200
Update Tests
commit 6e5936abff
Author: tal <tal@daglabs.com>
Date: Tue Oct 27 10:22:45 2020 +0200
Change to the new API
commit 5a70dc48b3
Author: tal <tal@daglabs.com>
Date: Mon Oct 26 18:35:31 2020 +0200
1. Added tests for ori
commit 2b9f78353f
Author: tal <tal@daglabs.com>
Date: Mon Oct 26 13:04:37 2020 +0200
1. Added structure "isolatedTest" {k, test}
2. Added for loop on the tests.
3. New test - Test 5.
commit c026d7b7a2
Author: tal <tal@daglabs.com>
Date: Thu Oct 22 17:35:56 2020 +0300
Fix bugs in the GHOSTDAG : counters, conntains and isAncestorOf.
Added more tests.
commit 74493b27d2
Author: tal <tal@daglabs.com>
Date: Thu Oct 22 16:49:27 2020 +0300
added compare between Hashes
commit f689253463
Author: tal <tal@daglabs.com>
Date: Thu Oct 22 11:49:01 2020 +0300
added compare between Hashes
commit 66be07f616
Author: tal <tal@daglabs.com>
Date: Mon Oct 19 18:42:40 2020 +0300
First test - pass.
commit 327f34f2dc
Author: tal <tal@daglabs.com>
Date: Mon Oct 19 15:20:27 2020 +0300
Add alternative implementation for ghostdag.
change all function's signatures (add error type)
commit fd2ea3d84a
Author: tal <tal@daglabs.com>
Date: Mon Oct 19 11:57:05 2020 +0300
add alternative implementation for ghostdag
* Update go-secp256k1 to v0.0.3
* Update the txscript engine to support only 32 bytes pubkeys
* Update the txscript engine tests
* Update txscript/sign.go to use the new Schnorr KeyPair API
* Update txscript sign_test to use the new schnorr
* Update sigcache tests to use new schnorr pubkey
* Update integration tests to use the new txscript and new schnorr pubkey
* [NOD-1126]
1. Change function name in BlockValidator interface from: "ValidateProofOfWorkAndDifficulty" to "ValidatePruningPointViolationAndProofOfWorkAndDifficulty".
2. Add to the blockValidator struct the pruningManager (also added to the function "New" Respectively).
3. Added new function "checkPruningPointViolation" of blockValidator type.
4. Add new internal check - "checkPruningPointViolation", on the function "ValidateProofOfWorkAndDifficulty".(The third check).
5. Add new error rule - "ErrPruningPointViolation".
* [Issue-1126]
1. Remove the function "PruningPoint" from PruningManager interface.
2. Changes in blockValidator struct - remove pruningManager, and adding pruningStore.
3. Reads for "pruningPoint" function from pruningStore instead of pruningManager (because of note 1 above) in the functions: * "checkPruningPointViolation" of type blockValidator.
* "FindNextPruningPoint" of type pruningManager.
* [Issue-1126]
1. Add missing error handling.
* [Issue-1126] Changes in function "checkPruningPointViolation": If header = genesis, stop checking and return nil.
* [Issue-1126] In function "checkPruningPointViolation" - change from a for loop to the "IsAncestorOfAny" function.
* [#1126] "FindNextPruningPoint" - save the pruning point in case the point is the genesis and change code internal order.
* [#1126] "FindNextPruningPoint" - cosmetics change.
* [#1126] "FindNextPruningPoint" - remove "return nil" when there is no pruning point on the if expression.
Co-authored-by: tal <tal@daglabs.com>
* [NOD-1551] Add NewTestConsensusWithDataDir to factory
* [NOD-1551] Cache transaction ID
* [NOD-1551] Should return err if err != nil
* [NOD-1551] BuildBlockWithParents returns the blocks pastUTXOData
* [NOD-1551] Set BlockCoinbaseMaturity to 0 in TestDoubleSpends
* [NOD-1551] Fix comments
* --amend
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* Test bounded merge depth
* Fix a bug in GetBlockInfo, where trying to use reachability on an invalid block
* Add a test to reproduce and test the GetBlockInfo bug
* [NOD-1566] Add a dependency to golang-lru.
* [NOD-1566] Add caching to blockstore.go.
* [NOD-1566] Add LRUCache to all store objects and initialize them.
* [NOD-1566] Add caching to acceptanceDataStore.
* [NOD-1566] Add caching to blockHeaderStore.
* [NOD-1566] Implement a simpler LRU cache.
* [NOD-1566] Use the simpler cache implementation everywhere.
* [NOD-1566] Remove dependency in golang-lru.
* [NOD-1566] Fix object reuse issues in store Get functions.
* [NOD-1566] Add caching to blockRelationStore.
* [NOD-1566] Add caching to blockStatusStore.
* [NOD-1566] Add caching to ghostdagDataStore.
* [NOD-1566] Add caching to multisetStore.
* [NOD-1566] Add caching to reachabilityDataStore.
* [NOD-1566] Add caching to utxoDiffStore.
* [NOD-1566] Add caching to reachabilityReindexRoot.
* [NOD-1566] Add caching to pruningStore.
* [NOD-1566] Add caching to headerTipsStore.
* [NOD-1566] Add caching to consensusStateStore.
* [NOD-1566] Add comments explaining why we don't discard staging at the normal location in consensusStateStore.
* [NOD-1566] Make go vet happy.
* [NOD-1566] Fix merge errors.
* [NOD-1566] Add a missing break statement.
* [NOD-1566] Run go mod tidy.
* [NOD-1566] Remove serializedUTXOSetCache.
* [NOD-1557] Add trace logs in add_block_to_virtual.go.
* [NOD-1557] Add trace logs in resolve_block_status.go.
* [NOD-1557] Add trace logs in calculate_past_utxo.go.
* [NOD-1557] Add trace logs in finality.go.
* [NOD-1557] Add trace logs in multisets.go.
* [NOD-1557] Fix compilation errors.
* [NOD-1557] Add trace logs to verify_and_build_utxo.go.
* [NOD-1557] Add trace logs to update_virtual.go.
* [NOD-1557] Add trace logs to set_pruning_utxo_set.go.
* [NOD-1557] Add trace logs to populate_tx_with_utxo_entries.go.
* [NOD-1557] Add trace logs to pick_virtual_parents.go.
* [NOD-1557] Make go vet happy.
* [NOD-1557] Clarify that some logic in AddBlockToVirtual is there for the sake of logging alone.
* [NOD-1557] Call blockStatusStore directly in AddBlockToVirtual when refetching the block status.
* [NOD-1556] Add logs regarding block status and virtual blue score
* [NOD-1556] UTXODiffAlgebra: add the offending outpoint to the text of errors
* [NOD-1556] Make checkIntersectionWithRule return ok as well
* [NOD-1538] Implement a simple orphan pool.
* [NOD-1538] Connect the orphan pool to the appropriate flows.
* [NOD-1538] Make UnorphanBlocks actually unorphan blocks.
* [NOD-1538] Fix logs.
* [NOD-1538] Make unorphaned blocks call LogBlock.
* [NOD-1538] Fix a log and some bad names.
* [NOD-1538] Don't return an error from LogBlock.
* [NOD-1538] Pass a pointer to hash in findChildOrphansOfBlock.
* [NOD-1538] Extract addChildOrphansToProcessQueue to a separate function.
* [NOD-1538] Correct messages.proto.
* [NOD-1538] Fix invalid message handling.
* [NOD-1538] Fix mempool not wrapping consensus errors.
* [NOD-1538] Extract wrapping logic to a separate function.
* [NOD-1538] Extract wrapping logic to an even better separate function.
* [NOD-1538] Fix minimal net adapter.
* [NOD-1538] Don't insert block relation until we've validated that the block's parents exist.
* [NOD-1538] Don't hold addressManager in MinimalNetAdapter.
* [NOD-1538] Fix a comment in messages.proto.
* Revert "[NOD-1500] Delete integration tests"
This reverts commit fcb57a2066.
* [NOD-1518] hashserialization -> consenusserialization
* [NOD-1518] Fix add genesis to virtual
* [NOD-1518] Fix a bug in SerializeCoinbasePayload.
* [NOD-1518] Fix a loop error and make pastMedianTime behave correctly everywhere on genesis.
* [NOD-1518] Fix another bug and an infinite loop.
* [NOD-1518] Fix uninitialized slice.
* [NOD-1518] Fix bad should-commit checks and another infinite loop.
* [NOD-1518] Fix nil serialization.
* [NOD-1518] Rename blockHash to currentBlockHash.
* [NOD-1518] Move the check whether stagedVirtualUTXOSet != nil to the top of commitVirtualUTXODiff.
* [NOD-1518] Simplify utxoDiffStore.Commit.
* [NOD-1518] Unextract resolveBlockStatusAndCheckFinality.
* [NOD-1518] Move no-transactions logic into CalculateIDMerkleRoot.
* [NOD-1518] Remove redundant is-staged check.
* [NOD-1518] Fix merge errors.
* [NOD-1518] Don't write anything if utxoDiffChild is nil.
* [NOD-1518] Stage virtualAcceptanceData and virtualMultiset.
* [NOD-1518] Fix bugs in getBlockTemplate and submitBlock.
* [NOD-1518] Fix bad validation order in validateHeaderInContext.
* [NOD-1518] Fix bug in Next().
* [NOD-1518] Fix nil dereference of subnetworks in AddressCache.
* [NOD-1518] Fix multisetStore.Get returning a pointer to a multiset that is changed in place.
* [NOD-1518] Break on genesis in countSubtrees.
* [NOD-1518] Fix createBlockLocator.
* [NOD-1518] Fix MsgTxToDomainTransaction.
* [NOD-1518] Set MaxTxVersion to 1.
* [NOD-1518] Fix missing error handling, bug in MsgTxToDomainTransaction, and bad subnetwork equality check.
* [NOD-1518] Fix bug in hasUTXOByOutpointFromStagedVirtualUTXODiff.
* [NOD-1518] Remove irrelevant comments.
* [NOD-1518] Generate transactions with sufficient fee in tx_relay_test.
* [NOD-1518] Fix broken RPC handlers.
* [NOD-1518] Fix merge errors.
* [NOD-1518] Fix bad exists check in restorePastUTXO and missing genesis check in CalculatePastUTXOAndAcceptanceData.
* [NOD-1518] Add a comment.
* [NOD-1518] Use a regular mutex instead of a read-write mutex in consensus to avoid dealing with sneaky not-actually-read functions.
* [NOD-1518] Fix a deadlock in GetVirtualSelectedParent.
* [NOD-1518] Fix missing handler registration for CmdHeader.
* [NOD-1518] Fix processHeader calling OnNewBlock and LogBlock. Also fix conversion errors in IBDRootUTXOSetAndBlock.
* [NOD-1518] Fix bad Command() in MsgIBDRootUTXOSetAndBlock.
* [NOD-1518] Fix bad SyncStateMissingUTXOSet logic in resolveSyncState.
* [NOD-1518] Rename mode to syncState.
* [NOD-1518] Fix headers-only blocks coming in after the consensus thinks it's synced.
* [NOD-1518] Fix selectedChildIterator.Next not ignoring virtual, infinite loop in HashSet.Length().
* [NOD-1518] Fix not-properly wrapped IBD blocks.
* [NOD-1518] Fix bad conversion in RequestIBDBlocks.
* [NOD-1518] Fix bad string for CmdRequestHeaders.
* [NOD-1518] Fix bad string for CmdDoneHeaders.
* [NOD-1518] Fix bad Command() for MsgIBDRootNotFound.
* [NOD-1518] Fix bad areHeaderTipsSyncedMaxTimeDifference value.
* [NOD-1518] Add missing string for CmdRequestIBDBlocks.
* [NOD-1518] Fix bad check for SyncStateMissingBlockBodies.
* [NOD-1518] Fix bad timeout durations in tests.
* [NOD-1518] Fix IBD blocks not calling OnNewBlock.
* [NOD-1518] Change when IBD finishes.
* [NOD-1518] Properly clone utxoDiffChild.
* [NOD-1535] Fix reachability tests
* [NOD-1518] Fix merge errors.
* [NOD-1518] Move call to LogBlock to into OnNewBlock.
* [NOD-1518] Return "not implemented" in unimplemented RPC handlers.
* [NOD-1518] Extract cloning of hashes to a method over DomainHash.
* [NOD-1518] Use isHeaderOnlyBlock.
* [NOD-1518] Use constants.TransactionVersion.
* [NOD-1518] Break immediately if we reached the virtual in SelectedChildIterator.
* [NOD-1518] Don't stage nil utxoDiffChild.
* [NOD-1518] Properly check the genesis hash in CalculatePastUTXOAndAcceptanceData.
* [NOD-1518] Explain why we break on current == nil in countSubtrees.
* [NOD-1518] Add a comment explaining why we check against StatusValid in resolveSyncState.
* [NOD-1535] Add external reachability tests
* [NOD-1535] Fix reachability tests and fix related bugs
* [NOD-1535] Add setters fox reindex slack and window
* [NOD-1535] Remove redundant line
* [NOD-1535] Add comment
* [NOD-1535] Fix comments
* [NOD-1535] Rename DBReader->DatabaseContext
* [NOD-1535] Check that reindex root is changed
* [NOD-1535] Fix calculateNewTips
Co-authored-by: Mike Zak <feanorr@gmail.com>
Co-authored-by: stasatdaglabs <stas@daglabs.com>
* Revert "[NOD-1500] Delete integration tests"
This reverts commit fcb57a2066.
* [NOD-1518] hashserialization -> consenusserialization
* [NOD-1518] Fix add genesis to virtual
* [NOD-1518] Fix a bug in SerializeCoinbasePayload.
* [NOD-1518] Fix a loop error and make pastMedianTime behave correctly everywhere on genesis.
* [NOD-1518] Fix another bug and an infinite loop.
* [NOD-1518] Fix uninitialized slice.
* [NOD-1518] Fix bad should-commit checks and another infinite loop.
* [NOD-1518] Fix nil serialization.
* [NOD-1518] Rename blockHash to currentBlockHash.
* [NOD-1518] Move the check whether stagedVirtualUTXOSet != nil to the top of commitVirtualUTXODiff.
* [NOD-1518] Simplify utxoDiffStore.Commit.
* [NOD-1518] Unextract resolveBlockStatusAndCheckFinality.
* [NOD-1518] Move no-transactions logic into CalculateIDMerkleRoot.
* [NOD-1518] Remove redundant is-staged check.
* [NOD-1518] Fix merge errors.
* [NOD-1518] Don't write anything if utxoDiffChild is nil.
* [NOD-1518] Stage virtualAcceptanceData and virtualMultiset.
* [NOD-1518] Fix bugs in getBlockTemplate and submitBlock.
* [NOD-1518] Fix bad validation order in validateHeaderInContext.
* [NOD-1518] Fix bug in Next().
* [NOD-1518] Fix nil dereference of subnetworks in AddressCache.
* [NOD-1518] Fix multisetStore.Get returning a pointer to a multiset that is changed in place.
* [NOD-1518] Break on genesis in countSubtrees.
* [NOD-1518] Fix createBlockLocator.
* [NOD-1518] Fix MsgTxToDomainTransaction.
* [NOD-1518] Set MaxTxVersion to 1.
* [NOD-1518] Fix missing error handling, bug in MsgTxToDomainTransaction, and bad subnetwork equality check.
* [NOD-1518] Fix bug in hasUTXOByOutpointFromStagedVirtualUTXODiff.
* [NOD-1518] Remove irrelevant comments.
* [NOD-1518] Generate transactions with sufficient fee in tx_relay_test.
* [NOD-1518] Fix broken RPC handlers.
* [NOD-1518] Fix merge errors.
* [NOD-1518] Fix bad exists check in restorePastUTXO and missing genesis check in CalculatePastUTXOAndAcceptanceData.
* [NOD-1518] Add a comment.
* [NOD-1518] Use a regular mutex instead of a read-write mutex in consensus to avoid dealing with sneaky not-actually-read functions.
* [NOD-1518] Fix a deadlock in GetVirtualSelectedParent.
* [NOD-1518] Fix missing handler registration for CmdHeader.
* [NOD-1518] Fix processHeader calling OnNewBlock and LogBlock. Also fix conversion errors in IBDRootUTXOSetAndBlock.
* [NOD-1518] Fix bad Command() in MsgIBDRootUTXOSetAndBlock.
* [NOD-1518] Fix bad SyncStateMissingUTXOSet logic in resolveSyncState.
* [NOD-1518] Rename mode to syncState.
* [NOD-1518] Fix headers-only blocks coming in after the consensus thinks it's synced.
* [NOD-1518] Fix selectedChildIterator.Next not ignoring virtual, infinite loop in HashSet.Length().
* [NOD-1518] Fix not-properly wrapped IBD blocks.
* [NOD-1532] Add TestMultiset
* [NOD-1518] Fix bad conversion in RequestIBDBlocks.
* [NOD-1518] Fix bad string for CmdRequestHeaders.
* [NOD-1518] Fix bad string for CmdDoneHeaders.
* [NOD-1518] Fix bad Command() for MsgIBDRootNotFound.
* [NOD-1532] Add TestPastUTXOMultiset
* [NOD-1518] Fix bad areHeaderTipsSyncedMaxTimeDifference value.
* [NOD-1532] Added TestDoubleSpends
* [NOD-1518] Add missing string for CmdRequestIBDBlocks.
* [NOD-1518] Fix bad check for SyncStateMissingBlockBodies.
* [NOD-1518] Fix bad timeout durations in tests.
* [NOD-1518] Fix IBD blocks not calling OnNewBlock.
* [NOD-1518] Change when IBD finishes.
* [NOD-1518] Properly clone utxoDiffChild.
* [NOD-1532] Update hashes of blocks
* [NOD-1532] Fix genesis blocks and a few more bugs
* [NOD-1532] Bugfix: incorrect key passed to dbTx.Put
* [NOD-1532] Make sure there's no nil payloads
* [NOD-1532] Fix AddBlockToVirtual
* [NOD-1532] Update tips and virtualDiffParents properly
* [NOD-1532] Allow nil payload
* [NOD-1532] Check for actual error and not just some RuleError
* [NOD-1532] Get rid of SimpleCoinbaseData and make OpTrueScript P2SH
* [NOD-1532] If coinbaseData is nil - fill in with generic coinbaseData
Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: stasatdaglabs <stas@daglabs.com>
* Revert "[NOD-1500] Delete integration tests"
This reverts commit fcb57a2066.
* [NOD-1518] hashserialization -> consenusserialization
* [NOD-1518] Fix add genesis to virtual
* [NOD-1518] Fix a bug in SerializeCoinbasePayload.
* [NOD-1518] Fix a loop error and make pastMedianTime behave correctly everywhere on genesis.
* [NOD-1518] Fix another bug and an infinite loop.
* [NOD-1518] Fix uninitialized slice.
* [NOD-1518] Fix bad should-commit checks and another infinite loop.
* [NOD-1518] Fix nil serialization.
* [NOD-1518] Rename blockHash to currentBlockHash.
* [NOD-1518] Move the check whether stagedVirtualUTXOSet != nil to the top of commitVirtualUTXODiff.
* [NOD-1518] Simplify utxoDiffStore.Commit.
* [NOD-1518] Unextract resolveBlockStatusAndCheckFinality.
* [NOD-1518] Move no-transactions logic into CalculateIDMerkleRoot.
* [NOD-1518] Remove redundant is-staged check.
* [NOD-1518] Fix merge errors.
* [NOD-1518] Don't write anything if utxoDiffChild is nil.
* [NOD-1518] Stage virtualAcceptanceData and virtualMultiset.
* [NOD-1518] Fix bugs in getBlockTemplate and submitBlock.
* [NOD-1518] Fix bad validation order in validateHeaderInContext.
* [NOD-1518] Fix bug in Next().
* [NOD-1518] Fix nil dereference of subnetworks in AddressCache.
* [NOD-1518] Fix multisetStore.Get returning a pointer to a multiset that is changed in place.
* [NOD-1518] Break on genesis in countSubtrees.
* [NOD-1518] Fix createBlockLocator.
* [NOD-1518] Fix MsgTxToDomainTransaction.
* [NOD-1518] Set MaxTxVersion to 1.
* [NOD-1518] Fix missing error handling, bug in MsgTxToDomainTransaction, and bad subnetwork equality check.
* [NOD-1518] Fix bug in hasUTXOByOutpointFromStagedVirtualUTXODiff.
* [NOD-1518] Remove irrelevant comments.
* [NOD-1518] Generate transactions with sufficient fee in tx_relay_test.
* [NOD-1518] Fix broken RPC handlers.
* [NOD-1518] Fix merge errors.
* [NOD-1518] Fix bad exists check in restorePastUTXO and missing genesis check in CalculatePastUTXOAndAcceptanceData.
* [NOD-1518] Add a comment.
* [NOD-1518] Use a regular mutex instead of a read-write mutex in consensus to avoid dealing with sneaky not-actually-read functions.
* [NOD-1518] Fix a deadlock in GetVirtualSelectedParent.
* [NOD-1518] Fix missing handler registration for CmdHeader.
* [NOD-1518] Fix processHeader calling OnNewBlock and LogBlock. Also fix conversion errors in IBDRootUTXOSetAndBlock.
* [NOD-1518] Fix bad Command() in MsgIBDRootUTXOSetAndBlock.
* [NOD-1518] Fix bad SyncStateMissingUTXOSet logic in resolveSyncState.
* [NOD-1518] Rename mode to syncState.
* [NOD-1518] Fix headers-only blocks coming in after the consensus thinks it's synced.
* [NOD-1518] Fix selectedChildIterator.Next not ignoring virtual, infinite loop in HashSet.Length().
* [NOD-1518] Fix not-properly wrapped IBD blocks.
* [NOD-1518] Fix bad conversion in RequestIBDBlocks.
* [NOD-1518] Fix bad string for CmdRequestHeaders.
* [NOD-1518] Fix bad string for CmdDoneHeaders.
* [NOD-1518] Fix bad Command() for MsgIBDRootNotFound.
* [NOD-1518] Fix bad areHeaderTipsSyncedMaxTimeDifference value.
* [NOD-1518] Add missing string for CmdRequestIBDBlocks.
* [NOD-1518] Fix bad check for SyncStateMissingBlockBodies.
* [NOD-1518] Fix bad timeout durations in tests.
* [NOD-1518] Fix IBD blocks not calling OnNewBlock.
* [NOD-1518] Change when IBD finishes.
* [NOD-1518] Properly clone utxoDiffChild.
* [NOD-1518] Fix merge errors.
* [NOD-1518] Move call to LogBlock to into OnNewBlock.
* [NOD-1518] Return "not implemented" in unimplemented RPC handlers.
* [NOD-1518] Extract cloning of hashes to a method over DomainHash.
* [NOD-1518] Use isHeaderOnlyBlock.
* [NOD-1518] Use constants.TransactionVersion.
* [NOD-1518] Break immediately if we reached the virtual in SelectedChildIterator.
* [NOD-1518] Don't stage nil utxoDiffChild.
* [NOD-1518] Properly check the genesis hash in CalculatePastUTXOAndAcceptanceData.
* [NOD-1518] Explain why we break on current == nil in countSubtrees.
* [NOD-1518] Add a comment explaining why we check against StatusValid in resolveSyncState.
Co-authored-by: Mike Zak <feanorr@gmail.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* [NOD-1526] Fix compilation errors
* [NOD-1526] Make MsgTx.PayloadHash non-pointer
* [NOD-1526] Fixed many tests
* [NOD-1526] Fix reference_test.go
* [NOD-1526] Removed last instances of appmessage in consensus
* [NOD-1526] No need to check for subnetwork
* [NOD-1529] Add all stores and processes to consensus, and add access to TestConsensus
* [NOD-1529] Move the getters of TestConsensus to separate file
* [NOD-1529] Add AddBlock to TestConsensus
* [NOD-1529] Update NewTestConsensus to be more all-encompassing
* [NOD-1529] Remove test directory in teardown
* [NOD-1529] Add ForAllNets function
* [NOD-1529] Add comment
* [NOD-1528] Make data stores copy data on stage
* [NOD-1528] Add proto objects to serialize consensus state objects
* [NOD-1528] Fix receiver names
* [NOD-1528] Add copy to block store and utxo diff staging
* [NOD-1528] Return errors where needed
* [NOD-1521] Use static check as part of jenkins to check for swallowed errors
* [NOD-1521] added staticcheck installation
* [NOD-1521] Fix static check errors
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* [NOD-1500] Added Domain type and Constructor
* [NOD-1500] Replaced dag+txpool with domain in flowContext
* [NOD-1500] Replaced dag+txpool with domain in flowContext
* [NOD-1500] Converters: domain objects from/to appmessage
* [NOD-1500] Convert hashes to DomainHashes in appmessages
* [NOD-1500] Remove references to daghash in dagconfig
* [NOD-1500] Fixed all appmessage usages of hashes
* [NOD-1500] Update all RPC to use domain
* [NOD-1500] Big chunk of protocol flows re-wired to domain
* [NOD-1500] Finished re-wiring all protocol flows to new Domain
* [NOD-1500] Fix some mempool and kaspaminer compilation errors
* [NOD-1500] Deleted util/{block,tx,daghash} and dbaccess
* [NOD-1500] util.CoinbaseTransactionIndex -> transactionhelper.CoinbaseTransactionIndex
* [NOD-1500] Fix txsigner
* [NOD-1500] Removed all references to util/subnetworkid
* [NOD-1500] Update RpcGetBlock related messages
* [NOD-1500] Many more compilation fixes
* [NOD-1500] Return full list of missing blocks for orphan resolution
* [NOD-1500] Fixed handshake
* [NOD-1500] Fixed flowcontext compilation
* [NOD-1500] Update users of StartIBDIfRequired to handle error
* [NOD-1500] Removed some more fields from RPC
* [NOD-1500] Fix the getBlockTemplate flow
* [NOD-1500] Fix HandleGetCurrentNetwork
* [NOD-1500] Remove redundant code
* [NOD-1500] Remove obsolete notifications
* [NOD-1500] Split MiningManager and Consensus to separate fields in Domain
* [NOD-1500] Update two wrong references to location of txscript
* [NOD-1500] Added comments
* [NOD-1500] Fix some tests
* [NOD-1500] Removed serialization logic from appmessage
* [NOD-1500] Rename database/serialization/messages.proto to dbobjects.proto
* [NOD-1500] Delete integration tests
* [NOD-1500] Remove txsort
* [NOD-1500] Fix tiny bug
* [NOD-1500] Remove rogue dependancy on bchd
* [NOD-1500] Some stylistic fixes
* [NOD-1512] Implement UTXO set deserialization
* [NOD-1512] Remove redundant file
* [NOD-1512] Don't use big endian for serialization
* [NOD-1512] Use Read/Write element
* [NOD-1512] Unexport ReadElement
* [NOD-1512] Fix StageVirtualUTXOSet
* [NOD-1512] Get rid of dagParams in consensusStateManager
* [NOD-1512] Get rid of dagParams in consensusStateManager
* Refactor the Difficulty adjastment to the new design
* Add the necessary things to the factory for the DAA constructor
* Add missing dagParams to difficultymanager constructor
* Use DAGTraversal for blueBlockWindow, and don't store PowMax compactBits
* Update pruningmanager interface
* Add a ProtoUTXOSet to hashserialization
* Update miningmanager with all the necessary stores and managers
* Implement mining manager
* Prune P.AC not in V.Past
* PruningManager fix all review comments
* [NOD-1496] Implement headers only verification
* [NOD-1496] Add checkParentsExist
* [NOD-1496] Stage block statuses in block processor
* [NOD-1496] Rename AddBlock->AddHeaderTip
* [NOD-1496] Return early from validateAndInsertBlock on header only and put ValidateProofOfWorkAndDifficulty inside validateBlock
* [NOD-1420] Start working on ConsensusStateManager. Might be redundant due to recent changes
* [NOD-1420] Convert model to externalapi in utxo_algerbra helpers
* [NOD-1420] Add UTXO-diff algebra
* [NOD-1420] Prepare skeleton of calculateAcceptanceDataAndMultiset
* [NOD-1420] Added skeleton for AddBlockToVirtual
* [NOD-1420] Implement PopulateTransactionWithUTXOEntries
* [NOD-1420] Implement restorePastUTXO
* [NOD-1420] Implement finality check
* [NOD-1420] Move handling of tips to consensusStateManager
* [NOD-1420] Implement calculateAcceptanceDataAndMultiset
* [NOD-1420] Start implementing resolveBlockStatus
* [NOD-1420] Implement resolveBlockStatus
* [NOD-1420] Update related fields in end of resolveSingleBlockStatus
* [NOD-1420] Start working on selectVirtualParents
* [NOD-1420] Implemented BlockHeap
* [NOD-1420] Implement selectVirtualParents
* [NOD-1420] Implement updateVirtual
* [NOD-1420] Added comments where they were missing
* [NOD-1420] Place all consensusStateManager functions in correct files
* [NOD-1420] Return the missing outpoints from populateTransactionWithUTXOEntriesFromVirtualOrDiff
* [NOD-1420] Outpoint.ID -> TransactionID
* [NOD-1420] Fix Stringer tests
* [NOD-1420] Copy hash.FromString into utils
* [NOD-1420] SetParents should return an error
* [NOD-1420] Remove all reachabilityManager references from consensusStateManager
* [NOD-1420] Remove VirtualData. Get the info from the stores where needed
* [NOD-1420] Invert parameters to IsAncestorOf
* [NOD-1420] Use model.AcceptanceData
* [NOD-1420] Don't return accumulatedMassBefore in error cases
* [NOD-1420] Don't expect store functions to return nil when the requested data was found - instead add HasXXX functions
* [NOD-1420] addTransactionToMultiset sets isCoinbase properly
* [NOD-1420] expected hash string length is externalapi.DomainHashSize * 2
* [NOD-1420] Rename reachabilityTree -> reachabilityManager + updateReindexRoot if isNextVirtualSelectedParent
* [NOD-1420] ValidateCoinbaseTransaction in csm.verifyAndBuildUTXO
* [NOD-1420] Re-write HAsUTXODiffChild
* [NOD-1420] delete past_utxo.go.bak
* [NOD-1420] Implement validateCoinbaseTransaction in CSM
* [NOD-1420] Imlemented missing functionality in ValidateTransactionAndPopulateWithConsensusData
* [NOD-1420] Moved merge depth logic to MergeDepthManager
* [NOD-1420] Add logs
* Make TransactionOutputEstimatedSerializedSize public
* Update the mempool interface
* Refactor the mempool to the new design
* refactor txselection and blocktemplatebuilder to the new design
* Update the mining manager
* Update the MiningManager factory
* mempool fix requested changed
* Add new ErrMissingTxOut error
* Add tests for ruleError wrapping
* Update consensus to use new ErrMissingTxOut type where appropriate
* Add new ErrInvalidTransactionsInNewBlock error
* Add wrapping tests for ErrInvalidTransactionsInNewBlock
* Fix Review suggestions
* Fix broken serialization(add pointer redirection)
* [NOD-1492] Implement some data stores
* [NOD-1492] Remove pointers to acceptance data
* [NOD-1492] Fix receiver names
* [NOD-1492] Implement delete for acceptanceDataStore
* [NOD-1492] In blockRelationStore rename IsAnythingStaged to IsStaged
* [NOD-1492] Rename bucket name
* [NOD-1476] Add dependency to BlockRelationStore in BlockProcessor.
* [NOD-1476] Add dependency to BlockStatusStore in BlockValidator.
* [NOD-1476] Add dependency to GHOSTDAGManager in BlockValidator.
* [NOD-1476] Rename CalculateConsensusStateChanges to AddBlockToVirtual.
* [NOD-1476] Remove RestoreDiffFromVirtual.
* [NOD-1476] Remove RestorePastUTXOSet.
* [NOD-1476] Add dependency to GHOSTDAGDataStore in ConsensusStateManager.
* [NOD-1476] Rename CalculateAcceptanceDataAndUTXOMultiset to just CalculateAcceptanceData.
* [NOD-1476] Remove UTXODiffManager and add dependencies to AcceptanceManager.
* [NOD-1476] Rename CalculateAcceptanceData to CalculateAcceptanceDataAndMultiset.
* [NOD-1476] Add dependency to DAGTopologyManager from ConsensusStateManager.
* [NOD-1476] Add dependency to BlockStore from ConsensusStateManager.
* [NOD-1476] Add dependency to PruningManager from ConsensusStateManager.
* [NOD-1476] Remove unnecessary stuff from ConsensusStateChanges.
* [NOD-1476] Add dependency to UTXODiffStore from ConsensusStateManager.
* [NOD-1476] Add tips to BlockRelationsStore.
* [NOD-1476] Add dependency to BlockRelationsStore from ConsensusStateManager.
* [NOD-1476] Remove Tips() from ConsensusStateStore.
* [NOD-1476] Remove acceptanceManager.
* [NOD-1476] Remove irrelevant functions out of ConsensusStateManager.
* [NOD-1461] Change the external api interface to not having anything besides DomainTransactions and DomainBlocks.
* [NOD-1462] Move external api types to a separate package.
* [NOD-1462] Clarify which model we're using in miningmanager.
* [NOD-1462] Extract coinbase data to its own struct.
* [NOD-1462] Add a comment above CoinbaseData.
* [NOD-1462] Fix the comment above CoinbaseData.
* [NOD-1458] Rename RestoreUTXOSet to RestorePastUTXOSet.
* [NOD-1458] Make CalculateAcceptanceDataAndMultiset take BlockGHOSTDAGData and nothing else.
* [NOD-1458] Make ConsensusStateStore's Update take ConsensusStateChanges instead of just UTXODiff.
* [NOD-1458] Add Tips() to ConsensusStateStore.
* [NOD-1458] Make all implementation structs private.
* [NOD-1458] Remove BlockAtDepth and add highHash to ChainBlockAtBlueScore.
* [NOD-1458] Rename CalculateAcceptanceDataAndMultiset to CalculateAcceptanceDataAndUTXOMultiset.
* [NOD-1458] Add a dependency to GHOSTDAGManager from ConsensusStateManager.
* [NOD-1458] Add ChooseSelectedParent to GHOSTDAGManager.
* [NOD-1458] Add DifficultyManager.
* [NOD-1458] Add PastMedianTimeManager.
* [NOD-1458] Add Hash() to Multiset.
* [NOD-1458] Add a dependency to ghostdagManager from blockProcessor.
* [NOD-1458] Add errors to all interfaces that need them.
* [NOD-1458] Uppercasify types in comments.
* [NOD-1458] Fix a bad comment.
* [NOD-1458] Fix a comment.
* [NOD-1458] Rename ChainBlockAtBlueScore to HighestChainBlockBelowBlueScore.
* [NOD-1458] Replace BlockAndTransactionValidator with an anonymous interface.
* [NOD-1460] Move the miningmanager interfaces into its model package.
* [NOD-1460] Decouple miningmanager model from appmessage.
* [NOD-1460] Decouple miningmanager model from util.
* [NOD-1460] Make miningmanager implementation structs unexported.
* [NOD-1457] Pass DomainDBContext to all constructors, instead of passing a general dbContext
* [NOD-1457] Add NewTx to DomainDBContext
* [NOD-1457] Added comment
* [NOD-1413] Remove /cmd/addblock
* [NOD-1413] Define and implement TransactionValidator.
* [NOD-1413] Make changes to ConsensusStateManager's interface.
* [NOD-1413] Make changes to PruningManager's interface.
* [NOD-1413] Make changes to DAGTraversalManager's interface.
* [NOD-1413] Make changes to MultisetStore's interface.
* [NOD-1413] Make changes to UTXODiffStore's interface.
* [NOD-1413] Make changes to UTXODiffStore's interface harder.
* [NOD-1413] Make changes to AcceptanceDataStore's interface harder.
* [NOD-1413] Make changes to PruningStore's interface.
* [NOD-1413] Delete BlockIndex.
* [NOD-1413] Add FeeDataStore.
* [NOD-1413] Update BlockMessageStore's interface.
* [NOD-1413] Fix interface violations.
* [NOD-1413] Add FeeDataStore to BlockProcessor.
* [NOD-1413] Make go vet happy.
* [NOD-1413] Add missing fields to ConsensusStateChanges.
* [NOD-1413] Add another missing field to ConsensusStateChanges.
* [NOD-1413] Add a reference to blockStore in consensusStateManager.
* [NOD-1413] Add missing methods to UTXODiffStore.
* [NOD-1413] Rename pruningPointStore to pruningStore everywhere.
* [NOD-1413] Remove superfluous parameters from CalculateConsensusStateChanges.
* [NOD-1413] Add missing dependencies to PruningManager.
* [NOD-1413] Remove implementation-y functions from TransactionValidator's interface.
* [NOD-1413] Make go vet happy.
* [NOD-1413] Add a couple of methods to DAGTopologyManager.
* [NOD-1413] Fix a typo in a file name.
* [NOD-1413] Remove non-interface functions from Validator.
* [NOD-1416] Move processes/datastructures interfaces into the model package.
* [NOD-1416] Decouple the model from dbaccess.
* [NOD-1413] Implement DomainBlock and DomainTransaction.
* [NOD-1413] Decouple model from appmessage.
* [NOD-1413] Decouple model from util.
* [NOD-1413] Decouple model from subnetworkid.
* [NOD-1413] Remove an unused const.
* [NOD-1413] Add DomainHash and DomainTransactionID.
* [NOD-1413] Decouple model from daghash.
* [NOD-1413] Decouple model from mstime.
* [NOD-1413] Decouple model from go-secp256k1.
* [NOD-1413] Add a proxy over dbaccess.
* [NOD-1413] Add comments over all added types.
* [NOD-1413] Fix a comment.
* [NOD-1413] Get rid of DomainTime.
* [NOD-1413] Simplify BlockGHOSTDAGData.
* [NOD-1444] Implement getHeaders RPC command
* [NOD-1444] Fix tests and comments
* [NOD-1444] Fix error message
* [NOD-1444] Make GetHeaders propagate header serialization errors
* [NOD-1444] RLock the dag on GetHeaders
* [NOD-1444] Change the error field number to 1000
* [NOD-1414] Add interfaces for Factory and State.
* [NOD-1414] Create interfaces for algorithms and data stores.
* [NOD-1414] Create empty implementations for algorithms and data stores.
* [NOD-1414] Add new functions for all the implementations.
* [NOD-1414] Begin filling in the interfaces.
* [NOD-1414] Fill in the interfaces for the data structures.
* [NOD-1414] Fill in the interfaces for the algorithms.
* [NOD-1414] Fix a bug in package names.
* [NOD-1414] Connect up the various interfaces.
* [NOD-1414] Add stubs to all the implementations.
* [NOD-1414] Create MiningManager and its Factory.
* [NOD-1414] Add interfaces for mempool and blockTemplateBuilder.
* [NOD-1414] Add implementation structs for miningManager.
* [NOD-1414] Add stub implementations for mempool and blockTemplateBuilder.
* [NOD-1414] Rename state to kaspadState.
* [NOD-1414] Restructure where interfaces sit.
* [NOD-1414] Restructure where interfaces sit in the algorithms package as well.
* [NOD-1414] Move remaining models out of models.go.
* [NOD-1414] Modified some interfaces.
* [NOD-1414] Make go vet happy.
* [NOD-1414] Move SerializedUTXOSet into PruningManager.
* [NOD-1414] Modify FindNextPruningPoint to return found and nextPruningPointUTXOSet.
* [NOD-1414] Add IsDAGAncestorOf.
* [NOD-1414] Add PruningPoint().
* [NOD-1414] Add Entry() to ReadOnlyUTXOSet.
* [NOD-1414] Add MergeSet() to BlockGHOSTDAGData.
* [NOD-1414] Write comments for all the exported types and functions in miningmanager.
* [NOD-1414] Add comments to the upper levels of KaspadState.
* [NOD-1414] Replace AddNode with ReachabilityChangeset.
* [NOD-1414] Add payAddress and extraData to GetBlockTemplate.
* [NOD-1414] Add scriptPublicKey and extraData to BuildBlock.
* [NOD-1414] Rename algorithms to processes.
* [NOD-1414] Rename kaspadState to consensus.
* [NOD-1414] Add ValidateAgainstPastUTXO and ValidateFinality.
* [NOD-1414] Add BlockGHOSTDAGData to ReachabilityChangeset.
* [NOD-1414] Fix the comment over Mempool.
* [NOD-1414] Fix the comment over ValidateTransaction.
* [NOD-1414] Fill up the data structures.
* [NOD-1414] Add comments to remaining uncommented items in miningmanager.
* [NOD-1414] Add comments to structs and constructors.
* [NOD-1414] Rename Set to Insert.
* [NOD-1414] Add comments to everything inside datastructures.
* [NOD-1414] Add comments to everything inside models.
* [NOD-1414] Add comments to the interfaces in processes.
* [NOD-1414] Add comments to everything in processes.
* [NOD-1414] Make go vet happy.
* [NOD-1414] Rename scriptPublicKey to coinbaseScriptPublicKey.
* [NOD-1414] Add handlers to the consensus.
* [NOD-1414] Add highHash to blockAtDepth.
* [NOD-1414] Add resolveFinalityConflict.
* [NOD-1414] Reorg BlockValidator.
* [NOD-1414] In ResolveFinalityConflicts, rename blockHash to newFinalityBlockHash.
* [NOD-1414] Fix a comment.
* [NOD-1414] Make reachability structs public.
* [NOD-1414] Make UTXO structs public.
* [NOD-1447] checkEntryAmounts should check against totalSompiInAfter, not totalSompiInBefore
* [NOD-1447] Remove lastSompiIn, and use totalSompiInBefore instead
* [NOD-1420] Moved setting limits to executor
* [NOD-1420] Moved all code dealing with windows service to separate package
* [NOD-1420] Move practically all main to restructured app package
* [NOD-1420] Check for running as interactive only after checking if we are doing any service operation
* [NOD-1420] Add comments
* [NOD-1420] Add a comment
* [NOD-1412] Remove ffldb, and make ldb implement all the database
interfaces
* [NOD-1412] Removed any reference to dataStore and updated block dbaccess to work directly with key/values
* [NOD-1320] Flush UTXOs to disk.
* [NOD-1320] Minor improvements and fixes.
* FullUTXOSet: change size type from int64 to uint64.
* Rename FullUTXOSet.size to FullUTXOSet.estimatedSize
* Fill NewFullUTXOSetFromContext with db context on virtual block
creation.
* Typo fixes.
* [NOD-1320] Stylystic fixes.
* [NOD-1320] Update tests. Improvements and fixes.
* Update blockdag/dag tests: prepare DB for tests.
* Update blockdag/utxoset tests: prepare DB for tests.
* Update blockdag/test_utils utils.
* Update blockdag/common tests.
* FullUTXOSet: remove embedded utxoCollection type. Rename
utxoCollection to utxoCache.
* Fix blockdag/block_utxo genesisPastUTXO func.
* Minor fixes and improvements.
* [NOD-1404] Remove most of the notification manager to fix a deadlock.
* [NOD-1404] Rename a couple of fields.
* [NOD-1404] Fix merge errors.
* [NOD-1404] Remove most of the notification manager to fix a deadlock (#935)
* [NOD-1404] Remove most of the notification manager to fix a deadlock.
* [NOD-1404] Rename a couple of fields.
* [NOD-1388] Write a deterministic test that reproduces the crash.
* [NOD-1388] Fix a typo.
* [NOD-1388] Use isInPastOfAny in addValidTip instead of just checking the parents.
* [NOD-1388] Add a json representation of the crashing DAG.
* [NOD-1388] Remove crash_test.go.
* [NOD-1388] Change variable name, add a comment.
* [NOD-1388] Rephrase a comment.
* [NOD-1249] Add pruning related constants (#869)
* [NOD-1249] Add pruning related constants
* [NOD-1249] Change status suspect to UTXONotVerified
* [NOD-1249] Add TestPruningDepth
* [NOD-1249] Add comment to pruningDepth
* [NOD-1249] Add pruning helper functions (#875)
* [NOD-1249] Added node.blockAtDepth
* [NOD-1249] Added node.finalityPoint()
* [NOD-1249] Add hasFinalityPointInOthersSelectedChain
* [NOD-1249] Add nonFinalityViolatingBlues
* [NOD-1249] Added isInPastOfAny
* [NOD-1249] Updated all calls to blockNode functions that require dag
* [NOD-1249] Add blockNode.reds field and persist it
* [NOD-1249] Add checkObjectiveFinality
* [NOD-1249] Add isViolatingSubjectiveFinality
* [NOD-1249] Added to TestGHOSTDAG check that reds are as expected
* [NOD-1249] Add checkMergeLimit and checkDAGRelations
* [NOD-1249] Invert condition in blockInDepth
* [NOD-1249] Make isInPastOfAny resemble isInPast
* [NOD-1249] Added comments to isInPast and isInPastOfAny
* [NOD-1252] Remove any references to legacy finality (#876)
* [NOD-1032] validateParents: check number of parents and that no parents were manually rejected (#877)
* [NOD-1254] Block verification changes (#882)
* [NOD-1254] Call checkDAGRelations and move it to correct place
* [NOD-1254] Use blockStatuses properly
* [NOD-1254] Add support for setting node's verification flag and set it to UTXONotVerified once block passes basic verification
* [NOD-1254] Check for subjctiveFinality, and for node not being in the selectedParentChain
* [NOD-1254] Make blockStatus an ordinary value - not bit flags
* [NOD-1254] Isolate all utxo-requiring validation into a single separate if branches
* [NOD-1254] Re-arrange connectBloc so that things that happen in UTXO-validated blocks only are all grouped together
* [NOD-1254] Resolve and check selectedParent's status before validatingUTXO
* [NOD-1254] Separate virtualUTXODiff from utxoVerificationOutput
* [NOD-1254] Stylistic fixes
* [NOD-1254] Use dag.index.(Set)BlockNodeStatus instead of accessing node.status
* [NOD-1288] Sub-routinize checkConnectToPastUTXO
* [NOD-1288] Re-write checkConnectToPastUTXO in a way that allows to filter-out invalid transactions
* [NOD-1288] Make checkTxSequenceLock use already calculated utxo outputs
* [NOD-1288] Make checkTxMass use already calculated utxo outputs
* [NOD-1288] Use dag.sigCache for ValidateTransactionScripts
* [NOD-1288] Use checkConnectTransactionToPastUTXO in applyBlueBlocks
* [NOD-1288] Clean-up old code-path from no longer used functions
* [NOD-1288] Skip any irrelevant parts of txo verification if block is genesis
* [NOD-1288] Set where it should have been
* [NOD-1288] Fix reachability checks to never use the new node + make isInSelectedParentChainOf return true if node == other
* [NOD-1288] invert the condition for isNewSelectedTip
* [NOD-1288] Separate checkIsAccepted to own function, and properly handle coinbase
* [NOD-1288] Don't update utxo-diff for UTXONotVerified parents/tips + Make PrepareBlockForTest resolve the selectedParent's UTXOSet if needed
* [NOD-1288] Include mass off coinbase transactions
* [NOD-1288] Move comment to correct position
* [NOD-1288] If blockAtDepth should return genesis - do it immidiately
* [NOD-1288] Comment-out instead of removeing scriptval_test.go
* [NOD-1288] Rename: entry -> utxoEntry
* [NOD-1288] Remove special function for calcCoinbaseTxMass
* [NOD-1288] Remove redundant check from checkEntryAmounts
* [NOD-1288] Rename: MaxMassPerBlock -> MaxMassAcceptedByBlock
* [NOD-1255] Implement boundedMergeBreakingParents
* [NOD-1255] Implement selectAllowedTips
* [NOD-1255] Integrate virtual parent selection into block verification process
* [NOD-1255] Add node to tips all the time, remove it from candidates and add it's parents if it's disqualified
* [NOD-1255] remove tips from virtaulBlock
* [NOD-1255] Rename: didVirtualParentsChanged -> didVirtualParentsChange
* [NOD-1255] Remove redundant sanity check
* [NOD-1255] Handle a forgotten error
* [NOD-1255] Prettify selectVirtualParents
* [NOD-1255] UpdateTipsUTXO should run over all UTXO-Verified tips, even if they are not parents of virtual
* [NOD-1311] Make isInPast inclusive
* [NOD-1032] Handle finality conflicts (#904)
* [NOD-1312] AddTip should not include finalityViolating and manuallyRejected blocks
* [NOD-1312] Implement resolveFinalityConflict
* [NOD-1312] Implement dag notifications for finalityChanges + updateing DAG state
* [NOD-1312] Added finality conflict rpc boilerplate
* [NOD-1312] Implement handling of getFinalityConflicts + resolveFinalityConflict RPCs
* [NOD-1312] Implement finality conflict related notifications
* [NOD-1312] Move all time to millisecond time
* [NOD-1312] Add comments + unexport some methods
* [NOD-1312] Add clarification in comments
* [NOD-1312] Move updateFinalityConflictResolution to finality_conflicts.go
* [NOD-1312] Rename: currentSelectedTip -> selectedTip
* [NOD-1312] Add long comment to ResolveFinalityConflict
* [NOD-1312] Convert areAllInSelectedParentChainOf into isInSelectedParentChainOfAll
* [NOD-1312] Rename chainUpdates (type) -> selectedParentChainUpdates, to distinguish from the variable chainUpdates
* [NOD-1032] Make all blockdag tests compile
* [NOD-1278] Fix finality-related tests (#910)
* [NOD-1032] Don't return node.dag.genesis from blockAtDepth because it might still not exist
* [NOD-1032] Actually add a tip in dag.addTip
* [NOD-1278] Add transaction to utxo set if it's coinbase
* [NOD-1278] Use VirtualParentHashes instead of TipHashes where appropriate
* [NOD-1278] If no valid virtual parent candidates - return error, don't wait for panic
* [NOD-1278] Transition TestCalcSequenceLock from newTestDAG to DAGSetup
* [NOD-1278] Fix .bluest() tie-breaker
* [NOD-1278] Remove feeData structure, as it no longer works, store feeData in acceptanceData
* [NOD-1278] Remove dag parameter from blockNode methods
* [NOD-1278] Fix TestBlueBlockWindow
* [NOD-1278] Don't subject selectedParent to MaxMergeSet
* [NOD-1278] se PrepareAndProcessBlockForTest instead of .addTip in TestSelectedPath
* [NOD-1278] Fixed TestDAGStateSerialization
* [NOD-1278] Fix TestAcceptanceIndexRecover
* [NOD-1278] Fix TestCheckConnectBlockTemplate
* [NOD-1278] Fix TestChainUpdates
* [NOD-1278] Fix and rename TestVirtualBlock -> TestTips
* [NOD-1278] Rename checkIsAccepted -> maybeAcceptTx
* [NOD-1278] Re-activate TestDoubleSpends
* Revert "[NOD-1278] Fixed TestDAGStateSerialization"
This reverts commit 845095d6de.
* [NOD-1278] Remove dag parameter from expectedCoinbaseTransaction
* [NOD-1348] Implemented simplified Finality Conflict Resolution scheme (#911)
* [NOD-1348] Rename functions according to Research spec
* [NOD-1348] Added blockSet.areAllIn
* [NOD-1348] Implemented simplified finality conflict resolution scheme
* [NOD-1348] Refactorings and fixes in selectVirtualParents
* [NOD-1278] Fix bugs discovered by unit-tests + Fix unit-tests (#916)
* Updated to version v0.3.1
* [NOD-858] Don't switch sync peer if the syncing process hasn't yet started with the current sync peer (#700)
* [NOD-858] Don't switch sync peer if the syncing process hasn't yet started with the current sync peer
* [NOD-858] SetShouldSendBlockLocator(false) on OnBlockLocator
* [NOD-858] Rename shouldSendBlockLocator->wasBlockLocatorRequested
* [NOD-858] Move panic to shouldReplaceSyncPeer
* [NOD-869] Add a print after os.Exit(1) to see if it is ever called (#701)
* [NOD-1238] Fix acceptance index never being initialized. (#859)
* [NOD-1278] Genesis never violates finality
* [NOD-1348] Refactorings and fixes in selectVirtualParents
* [NOD-1278] Don't call dag.selectVirtualParents for genesis
* [NOD-1278] Properly organize errors in maybeAcceptBlock
* [NOD-1278] updateTipsUTXO should only run on tips whose status is
* [NOD-1278] updateTipsUTXO should only run on tips whose status is `valid`
* [NOD-1278] Fix TestDoubleSpends
* [NOD-1278] Fix TestDAGIndexFailedStatus
* [NOD-1278] IsFinalizedTransaction should use uint64 everywhere
* [NOD-1278] If tx is coinbase and not selectedParent - don't update pastUTXO
* [NOD-1278] Store tips and validTips separately
* [NOD-1278] Store ValidTips and VirtualParents in dagState
* [NOD-1278] Fix TestProcessOrphans
* [NOD-1278] Fix TestProcessOrphans
* [NOD-1278] Fix TestOrderInDiffFromAcceptanceData
* [NOD-1278] Fix TestHelp
* [NOD-1278] Remove mining.PrepareBlockForTest; use blockdag.PrepareBlockForTest instead
* [NOD-1278] Explicitly disallow chained transactions
* [NOD-1278]
* [NOD-1278] Fix some comments
Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
Co-authored-by: Yuval Shaul <yuval.shaul@gmail.com>
* [NOD-1355] Add unit-test for finality + When resolving finalityConflict - make sure the block that will come out selectedTip is statusValid (#919)
* [NOD-1355] Added test for finality
* [NOD-1355] When resolving finalityConflict - make sure the block that will come out selectedTip is statusValid
* [NOD-1032] Renames: anything about inputsWithReferencedUTXOEntries -> remove 'Referenced'
* [NOD-1032] Don't ignore non-rule errors
* [NOD-1032] Fix comment
* [NOD-1032] Enhanced comments on TestChainUpdates
* [NOD-1032] Remove scriptval_test.go
* [NOD-1032] Extracted isNewSelectedTip to a method
* [NOD-1032] Use dag.Now() instead of mstime.Now()
* [NOD-1032] Print block status when accepting block
* [NOD-1032] Add comment explaining boundedMergeBreakingParents
* [NOD-1032] Enhanced test and imporved comments in TestFinality
* [NOD-1032] Rename: Objective finality -> bounded merge depth
* [NOD-1032] No need to check that validTips are valid
* [NOD-1032] Remove dag from arguments of updateDiffAndDiffChild
* [NOD-1032] Correct variable names in LookupNodes
[NOD-1032] Correct variable names in LookupNodes
* [NOD-1032] Fix some comments
* [NOD-1032] Some style changes
* [NOD-1032] Refactor isAnyInPastOf
* [NOD-1032] Enhance comment in updateVirtualParents
* [NOD-1032] Flip condition in updateVirtualParents
* [NOD-1032] Stylistic and grammatic fixes in dag.go and dag_test.go
* [NOD-1032] Explain why updateSelectedParentSet receives geneses on init
* [NOD-1032] Remove ErrParentManuallyRejected
* [NOD-1032] Added wrapper for resolveNodeStatus that creates a special transaction for it
* [NOD-1032] Rename: statusUTXONotVerified -> statusUTXOPendingVerification
* [NOD-1032] Use virtual parents in CurrentBits()
* [NOD-1032] rename: isViolatingSubjectiveFinality -> isViolatingFinality
* [NOD-1032] Move updateVirtualAndTips to applyDAGChanges
* [NOD-1032] Invert condition for isFinalityPointInPast
* [NOD-1032] Fix antiPastBetween isInPast became inclusive
* [NOD-1032] Remove redundant call for addTip
* [NOD-1032] Use calcCoinbaseTxMass where appropriate
* [NOD-1032] Remove time fields from conflict notifications
* [NOD-1032] Assign the correct thing to i
* [NOD-1032] unify checkOutputsAmounts and checkTxOutputAmounts
* [NOD-1032] Cleanup in CheckTransactionInputsAndCalulateFee
* [NOD-1032] Fixed some style and comments
* [NOD-1032] If selectedParent is disqualifiedFromChain - validateAndApplyUTXOSet should return this as a ruleError
* [NOD-1032] Set the status in resolveNodeStatus
* [NOD-1032] Correct comment on boundedMergeBreakingParents
* [NOD-1032] Fix a typo.
* [NOD-1032] Update a variable name.
* [NOD-1032] Fix a comment.
* [NOD-1032] Fix merge errors.
* [NOD-1032] Add VirtualParentHashes to getBlockDagInfo.
* [NOD-1032] Update handleGetBlockTemplate.
* [NOD-1032] Comment out all the old RPC stuff.
* [NOD-1032] Remove irrelevant type.
* [NOD-1032] Implement ResolveFinalityConflict.
* [NOD-1032] Remove irrelevant comments.
* [NOD-1032] Implement NotifyFinalityConflicts.
* [NOD-1032] Add FinalityConflictNotification and FinalityConflictResolvedNotification.
* [NOD-1032] Finish implementing finality conflict notifications.
* [NOD-1032] Remove old RPC stuff.
* [NOD-1032] Fix grammar in a comment.
Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
Co-authored-by: Yuval Shaul <yuval.shaul@gmail.com>
Co-authored-by: stasatdaglabs <stas@daglabs.com>
* [NOD-1151] Added gRPC server for seeding peers
* [NOD-1151] Fix branch after rebase
* [NOD-1151] Lint infrastructure/config/config.go
* [NOD-1151] Use Warnf instead of Infof
* [NOD-1151] Check if a.cfg.GRPCSeed is defined
* [NOD-1151] Delete Makefile
- Use go generate instead of Makefile
* [NOD-1151] Panic in case of GRPCSeed and DNSSeed are both defined
* [NOD-1151] Add generate.go file
* [NOD-1151] Allow dnsseed and grpcseed to work together
Co-authored-by: Bogdan Ovsiannikov <takahawkkun@gmail.com>
Co-authored-by: yaroslavr-itd <yaroslav.r@it-dimension.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* [NOD-1319] Create the protorpc package.
* [NOD-1319] Use a general ClientStream in closeSend.
* [NOD-1319] Decouple p2pServer from gRPCServer.
* [NOD-1319] Begin implementing rpcServer.
* [NOD-1319] Move grpcStream to grpc_connection.go.
* [NOD-1319] Fold the rpc messages.proto into a common message.proto.
* [NOD-1319] Remove code duplication in MessageStream.
* [NOD-1319] Rename methods in netadapter.
* [NOD-1319] Rename message_xxx to p2p_xxx.
* [NOD-1319] Section off p2p messages from rpc messages in messages.proto.
* [NOD-1319] Split toPayload to a p2p part and and rpc part.
* [NOD-1319] Rename msgxxx.go to p2p_msgxx.go in the appmessage package.
* [NOD-1319] Implement GetCurrentVersionRequestMessage and GetCurrentVersionResponseMessage.
* [NOD-1319] Implement toAppMessage and fromAppMessage for getCurrentNetwork
* [NOD-1319] Make a temporary workaround so that tests pass.
* [NOD-1319] Begin implementing the rpc manager.
* [NOD-1319] Implement an initial routerInitializer for rpc.
* [NOD-1319] Rename the spawn in routerInitializer.
* [NOD-1319] Implement an RPC context.
* [NOD-1319] Move the actual handlers to a separate package.
* [NOD-1319] Use the correct value for the GetCurrentNetwork response.
* [NOD-1319] Fix some names.
* [NOD-1319] Begin implementing a cli rpc client.
* [NOD-1319] Implement connecting to the RPC server.
* [NOD-1319] Make sure that connecting to the server and sending/receiving messages works.
* [NOD-1319] Make kaspactl2 speak in json strings.
* [NOD-1319] Finish implementing kaspactl2.
* [NOD-1319] Remove debug messages.
* [NOD-1319] Properly handle errors in rpc.go.
* [NOD-1319] Move the grpc client to a separate package.
* [NOD-1319] Extract Post out of PostString.
* [NOD-1319] Implement PostAppMessage.
* [NOD-1319] Stub out submitBlock.
* [NOD-1319] Stub out getBlockTemplate.
* [NOD-1319] Combine request and reponse files.
* [NOD-1319] Implement submitBlock.
* [NOD-1319] Implement returning errors from RPC.
* [NOD-1319] Begin implementing getBlockTemplate.
* [NOD-1319] Add missing field in GetBlockTemplateRequestMessage.
* [NOD-1319] Implement a minimal getBlockTemplate.
* [NOD-1319] Add getBlockTemplate stuff to grpc.
* [NOD-1319] Implement the rest of getBlockTemplate.
* [NOD-1319] Add block/transaction added handlers to the protocol manager.
* [NOD-1319] Implement NotifyTransactionAddedToMempool.
* [NOD-1319] Implement NotifyBlockAddedToDAG.
* [NOD-1319] Connect block/transaction added handlers.
* [NOD-1319] Add notifyBlockAdded.
* [NOD-1319] Add a notification system.
* [NOD-1319] Improve the notification system.
* [NOD-1319] Add a block added listener stub.
* [NOD-1319] Add BlockAddedNotificationMessage.
* [NOD-1319] Finish implementing HandleNotifyBlockAdded.
* [NOD-1319] Println instead of Print in kaspactl2.
* [NOD-1319] Remove unused flags in kaspactl2.
* [NOD-1319] Make kaspaminer work with the new RPC.
* [NOD-1319] Fix a bad log.
* [NOD-1319] Make kaspaminer work.
* [NOD-1319] Disconnect the old RPC.
* [NOD-1319] Move grpcclient.go.
* [NOD-1319] Begin generalizing the rpcClient.
* [NOD-1319] Move errors to the side of the payload.
* [NOD-1319] Add errors to appmessage.
* [NOD-1319] Move AttachRouter to grpcclient.
* [NOD-1319] Fix kaspaminer not handling responses.
* [NOD-1319] Properly handle blockAddedNotifications.
* [NOD-1319] Move errors into individual response objects.
* [NOD-1319] Begin replacing the RPC client in the integration tests.
* [NOD-1319] Implement GetPeerAddresses.
* [NOD-1319] Implement GetPeerAddresses.
* [NOD-1319] Fix setOnBlockAddedHandler.
* [NOD-1319] Remove the old kaspactl.
* [NOD-1319] Move ConvertGetBlockTemplateResultToBlock to the mining package.
* [NOD-1319] Implement getSelectedTipHash.
* [NOD-1319] Simplify testRPCRouter.
* [NOD-1319] Write stubs for the required test RPC commands.
* [NOD-1319] Implement a minimal getMempoolEntry.
* [NOD-1319] Implement a minimal getMempoolEntry.
* [NOD-1319] Implement getConnectedPeerInfo.
* [NOD-1319] Delete the old RPC.
* [NOD-1319] Fix a fromAppMessage.
* [NOD-1319] Implement connectToPeer.
* [NOD-1319] Fix a bug in registerForBlockAddedNotifications.
* [NOD-1319] Fix a deadlock in closing notification listeners.
* [NOD-1319] Fix merge errors.
* [NOD-1319] Fix an import.
* [NOD-1319] Properly handle errors in grpcclient.
* [NOD-1319] Fix TestIBD.
* [NOD-1319] Prevent kaspaminer from running when not connected.
* [NOD-1319] Implement sendRawTransaction.
* [NOD-1319] Implement sendRawTransaction in the client.
* [NOD-1319] Extract a general RPC client from the integration test RPC client.
* [NOD-1319] Use the general RPC client for the miner.
* [NOD-1319] Move the rpcclient package out of netadapter.
* [NOD-1319] Normalize onBlockAdded.
* [NOD-1319] Begin implementing notifyChainChanged.
* [NOD-1319] Implement the model for notifyChainChanged.
* [NOD-1319] Implement conversions for notifyChainChanged.
* [NOD-1319] Implement HandleNotifyChainChanged.
* [NOD-1319] Normalize notifications.
* [NOD-1319] Implement RegisterForChainChangedNotifications.
* [NOD-1319] Begin connecting blockdag's chain-changed notification with the RPC.
* [NOD-1319] Finish implementing notifyChainChanged.
* [NOD-1319] Implement getBlockHex.
* [NOD-1319] Rename getBlockHex to getBlock.
* [NOD-1319] Implement the verbose model for getBlock.
* [NOD-1319] Implement buildBlockVerboseData.
* [NOD-1319] Implement buildTransactionVerboseData.
* [NOD-1319] Move verboseData stuff to verbosedata.go.
* [NOD-1319] Add includeTransactionVerboseData.
* [NOD-1319] Begin implementing getSubnetwork.
* [NOD-1319] Finish implementing getSubnetwork.
* [NOD-1319] Begin implementing getChainFromBlock.
* [NOD-1319] Finish implementing getChainFromBlock.
* [NOD-1319] Begin implementing getBlocks.
* [NOD-1319] Finish implementing getBlocks.
* [NOD-1319] Fix bad responses in HandleNotifyChainChanged.
* [NOD-1319] Fix bugs in verbosedata.go.
* [NOD-1319] Fix more bugs in verbosedata.go.
* [NOD-1319] Make go vet happy.
* [NOD-1319] Extract handleBlockDAGNotifications to a method.
* [NOD-1319] Add a newline.
* [NOD-1319] Use peers instead of connections to check if connected.
* [NOD-1319] Add a comment.
* [NOD-1319] Lock the dag lock in getBlock.
* [NOD-1319] Rename netAdapter.connections to p2pConnections.
* [NOD-1319] In protowire, rename wireXXX to protoXXX.
* [NOD-1319] Rename PostString to PostJSON.
* [NOD-1319] Disallow empty transactions in SendRawTransaction.
* [NOD-1319] Disallow empty blocks in SubmitBlocks.
* [NOD-1319] Add SetLogger.
* [NOD-1319] Fix an error message.
* [NOD-1319] Fix an error message.
* [NOD-1319] Rename testTimeout to rpcTimeout.
* [NOD-1319] Rename SendRawTransaction to SubmitTransaction.
* [NOD-1319] Rename ConnectToPeer to AddPeer.
* [NOD-1319] Add missing longPollID to request.
* [NOD-1319] Rename ChainChangedChainBlock to ChainBlock.
* [NOD-1319] Rename Vin and Vout.
* [NOD-1319] Implement RPCErrorf.
* [NOD-1319] Fix RPCErrorf's comment.
* [NOD-1319] Remove unused flags in kaspaminer.
* Add VirtualBlueHashes to BlockDAG
* Refactor TestGHOSTDAG to read DAGs from json files
* Added a new DAG for the ghostdag test suite
* Pass BehaviorFlags to delayed blocks
* [NOD-592] Remove TODOs related to fake nonces.
* [NOD-592] Remove irrelevant TODOs from handleRescanBlocks and parseTxAcceptedVerboseNtfnParams.
* [NOD-592] Fix TODO in handleGetTxOut.
* [NOD-592] Remove irrelevant TODO from updateAddress.
* [NOD-592] Move StandardVerifyFlags to a separate file.
* [NOD-592] Remove TODOs in sign.go.
* [NOD-592] Remove TODO in scriptval_test.go.
* [NOD-592] Remove TODO in reachabilitystore.go.
* [NOD-592] Remove XXXs.
* [NOD-592] Fix a comment.
* [NOD-557] Move AddAddressByIP out of AddressManager since it's used only for tests..
* [NOD-557] Remove rescan blocks.
* [NOD-592] Fix handleGetTxOut.
* [NOD-557] Remove regTest network.
* [NOD-557] Remove remaining references to regTest.
* [NOD-557] Move newHashFromStr from params.go to params_test.go.
* [NOD-557] Rename test to network in register_test.go.
* [NOD-557] Replaced removed tests in TestDecodeAddressErrorConditions.
* [NOD-1293] Use addressManager's GetBestLocalAddress.
* [NOD-1293] Copy the initListeners function from the old p2p to the address manager.
* [NOD-1293] Remove debug logs.
* [NOD-1293] Remove unused import.
* [NOD-1293] Fix a comment.
* [NOD-1286] Close router from netConnection.Disconnect
* [NOD-1286] Close router in grpc errors as well
* [NOD-1286] Fix typo
* [NOD-1286] Rename isConnected->isRouterClosed
* [NOD-1223] Delete unused files/packages.
* [NOD-1223] Move signal and limits to the os package.
* [NOD-1223] Put database and dbaccess into the db package.
* [NOD-1223] Fold the logs package into the logger package.
* [NOD-1223] Rename domainmessage to appmessage.
* [NOD-1223] Rename to/from DomainMessage to AppMessage.
* [NOD-1223] Move appmessage to the app packge.
* [NOD-1223] Move protocol to the app packge.
* [NOD-1223] Move the network package to the infrastructure packge.
* [NOD-1223] Rename cmd to executables.
* [NOD-1223] Fix go.doc in the logger package.
* [NOD-1259] All rule-errors should be protocol-errors
* [NOD-1259] Handle submitting of coinbase transactions properly
* Revert "[NOD-1259] All rule-errors should be protocol-errors"
This reverts commit 2fd30c1856.
* [NOD-1259] Don't panic on non-protocol errors in ProtocolManager.AddTransaction/AddBlock
* [NOD-1259] Implement subnetworkid.IsBuiltInOrNative and use where appropriate
* [NOD-1220] Add network name to the version message.
* [NOD-1220] Ban peers from the wrong network.
* [NOD-1220] Add the network parameter to protowire.
* [NOD-1220] Add "kaspa-" to network names.
* [NOD-1129] Implement TestIncestousNewBlockTemplate.
* [NOD-1129] Add some debug logs to TestIncestousNewBlockTemplate.
* [NOD-1129] Fix merge errors.
* [NOD-1129] Narrow down on the failure.
* [NOD-1129] Fix bad initial value for child.interval in reachabilityTreeNode.addChild.
* [NOD-1129] Rewrite the test to be specific to reachability.
* [NOD-1223] Move all network stuff into a new network package.
* [NOD-1223] Delete the unused package testutil.
* [NOD-1223] Move infrastructure stuff into a new instrastructure package.
* [NOD-1223] Move domain stuff into a new domain package.
* Add Hash Writers
* Add the hash writers to the tests
* Add the DoubleHash Writer to the benchmarks
* Remove buffers from hashing by using the Hash Writer
* Replace empty slice with nil in mempool test payload
* [NOD-1201] Panic if necessary callback are not set in gRPCConnection and gRPCServer
* [NOD-1201] Fix comment and change return order
* [NOD-1201] Return nil instead of error on gRPCServer.Start
* [NOD-1201] Fix typo
* [NOD-1246/NOD-1248] Add unit test for NetAdapter
* [NOD-1246/NOD-1248] Do not ignore OK
* [NOD-1248] Lint code
- Move `t *testing.T` to be first parameter in test-helper function
- Rename `getRouterInitializer` to `routerInitializerForTest`
- Make test data constants
Co-authored-by: Yaroslav Reshetnyk <yaroslav.r@it-dimension.com>
* [NOD-1190] Move non-processBlock stuff out of process.go.
* [NOD-1190] Move everything out of accept.go.
* [NOD-1190] Move all processBlock functions to process.go.
* [NOD-1190] Move orphan stuff to orphan.go.
* [NOD-1190] Remove thresholdstate stuff.
* [NOD-1190] Move isSynced to sync_rate.go.
* [NOD-1190] Move delayed block stuff to delayed_blocks.go.
* [NOD-1190] Rename orphans.go to orphaned_blocks.go.
* [NOD-1190] Move non-BlockDAG structs out of dag.go.
* [NOD-1190] Remove unused fields.
* [NOD-1190] Fixup BlockDAG.New a bit.
* [NOD-1190] Move sequence lock stuff to sequence_lock.go
* [NOD-1190] Move some multiset stuff out of dag.go.
* [NOD-1190] Move finality stuff out of dag.go.
* [NOD-1190] Move blocklocator stuff out of dag.go.
* [NOD-1190] Move confirmation stuff out of dag.go.
* [NOD-1190] Move utxo and selected parent chain stuff out of dag.go.
* [NOD-1190] Move BlockDAG lock functions to the beginning of dag.go.
* [NOD-1190] Move verifyAndBuildUTXO out of process.go.
* [NOD-1190] Extract handleProcessBlockError to a function.
* [NOD-1190] Remove daglock unlock in notifyBlockAccepted.
* [NOD-1190] Extract checkDuplicateBlock to a method.
* [NOD-1190] Fix merge errors.
* [NOD-1190] Remove unused parameter from CalcSequenceLock.
* [NOD-1190] Extract processBlock contents into functions.
* [NOD-1190] Fix parent delayed blocks not marking their children as delayed
* [NOD-1190] Fix TestProcessDelayedBlocks.
* [NOD-1190] Extract stuff in maybeAcceptBlock to separate functions.
* [NOD-1190] Rename handleProcessBlockError to handleConnectBlockError.
* [NOD-1190] Remove some comments.
* [NOD-1190] Use lowercase in error messages.
* [NOD-1190] Rename createNewBlockNode to createBlockNodeFromBlock.
* [NOD-1190] Rename orphaned_blocks.go to orpan_blocks.go.
* [NOD-1190] Extract validateUTXOCommitment to a separate function.
* [NOD-1190] Fix a bug in validateUTXOCommitment.
* [NOD-1190] Rename checkBlockTxsFinalized to checkBlockTransactionsFinalized.
* [NOD-1190] Add a comment over createBlockNodeFromBlock.
* [NOD-1190] Fold validateAllTxsFinalized into checkBlockTransactionsFinalized.
* [NOD-1190] Return parents from checkBlockParents.
* [NOD-1190] Remove the processBlock prefix from the functions that had it.
* [NOD-1190] Begin extracting functions out of checkTransactionSanity.
* [NOD-1190] Finish extracting functions out of checkTransactionSanity.
* [NOD-1190] Remove an unused parameter.
* [NOD-1190] Fix merge errors.
* [NOD-1190] Added an explanation as to why we change the nonce in TestProcessDelayedBlocks.
* [NOD-1190] Fix a comment.
* [NOD-1190] Fix a comment.
* [NOD-1190] Fix a typo.
* [NOD-1190] Replace checkBlockParents with handleLookupParentNodesError.
* [NOD-1233] Remove HandleNewBlockOld.
* [NOD-1233] Make ErrRouteClosed not a protocol error.
* [NOD-1233] Fix ambiguous comments.
* [NOD-1233] Remove a no-longer-relevant comment.
* [NOD-1233] Remove some of the TODOs.
* [NOD-1233] Replace fakeSourceAddress with a real sourceAddress.
* [NOD-1233] Remove a no-longer-relevant TODO.
* [NOD-1233] Remove TODO from handleGetNetTotals.
* [NOD-1233] Remove a no-longer-relevant TODO.
* [NOD-1233] Disconnect if connected to wrong partial/full type.
* [NOD-1233] Get rid of mempool tags.
* [NOD-1233] Remove TODOs.
* [NOD-1233] Simplify a test.
* [NOD-1190] Remove getNetTotals.
* [NOD-1259] All rule-errors should be protocol-errors
* [NOD-1259] Handle submitting of coinbase transactions properly
* Revert "[NOD-1259] All rule-errors should be protocol-errors"
This reverts commit 2fd30c1856.
* [NOD-1259] Don't panic on non-protocol errors in ProtocolManager.AddTransaction/AddBlock
* [NOD-1259] Implement subnetworkid.IsBuiltInOrNative and use where appropriate
* [NOD-1220] Add network name to the version message.
* [NOD-1220] Ban peers from the wrong network.
* [NOD-1220] Add the network parameter to protowire.
* [NOD-1220] Add "kaspa-" to network names.
* [NOD-1229] Fix node crashing if AntiPastHashesBetween lowHigh or highHash are not found in the DAG
* [NOD-1229] Rename InvalidParameterError to ErrInvalidParameter.
* [NOD-1229] Lowercasify errors.
* [NOD-1224] Make block already existing a ruleError
* [NOD-1224] Remove block from pendingBlocks list only after it was processed
* [NOD-1224] AddToPeers should have a Write Lock, not Read Lock
* [NOD-1224] Check for unrequested before processing
* [NOD-1214] Added test for 64 incoming connections to single node
* [NOD-1214] Expand comments, and a small rename
* [NOD-1214] Make sure no bully reports blockAdded twice
* [NOD-1162] Separate kaspad to it's own package, so that I can use it out of integration test
* [NOD-1162] Begin integration tests
* [NOD-1162] [FIX] Assign cfg to RPCServer
* [NOD-1162] Basic integration test ready
* [NOD-1162] Wait for connection for real
* [NOD-1162] [FIX] Connection manager should run the moment it adds a request
* [NOD-1162] Make connect something that can be invoked in middle of test
* [NOD-1162] Complete first integration test
* [NOD-1162] Undo refactor error
* [NOD-1162] Rename Kaspad to App
* [NOD-1162] Convert checking connection to polling
* [NOD-1162] [FIX] Set peerID on handshake
* [NOD-1162] [FIX] Broadcast should send to outgoing route, not incoming
* [NOD-1162] [FIX] Add CmdInvRelayBlock to MakeEmptyMessage
* [NOD-1162] [FIX] Initialize Hash before decoding MsgInvRelayBlock
* [NOD-1162] [FIX] Invert condition
* [NOD-1162] [FIX] Fixes to encoding of MsgGetRelayBlocks
* [NOD-1162] [FIX] Add MsgGetRelayBlocks to MakeEmptyMessage
* [NOD-1162] [FIX] Connection manager should run the moment it adds a request
* [NOD-1162] [FIX] Set peerID on handshake
* [NOD-1162] [FIX] Broadcast should send to outgoing route, not incoming
* [NOD-1162] [FIX] Add CmdInvRelayBlock to MakeEmptyMessage
* [NOD-1162] [FIX] Initialize Hash before decoding MsgInvRelayBlock
* [NOD-1162] [FIX] Invert condition
* [NOD-1162] [FIX] Fixes to encoding of MsgGetRelayBlocks
* [NOD-1162] [FIX] Add MsgGetRelayBlocks to MakeEmptyMessage
* [NOD-1162] Add comment
* [NOD-1162] Added support for 3 nodes and clients in integration tests
* [NOD-1162] Add third node to integration test
* [NOD-1192] Use lock-less functions in TxPool.HandleNewBlock
* [NOD-1192] Broadcast transactions only if there's more then 0
* [NOD-1162] Removed double waitTillNextIteration
* [NOD-1192] Rename: broadcastTransactions -> broadcastTransactionsAfterBlockAdded
* [NOD-1162] Call NotifyBlocks on client3 as well
* [NOD-1162] ErrTimeout and ErrRouteClosed should be ProtocolErrors
* [NOD-1162] Added comment and removed redundant type PeerAddedCallback
* [NOD-1162] Revert overly eager rename
* [NOD-1162] Move DisalbeTLS to common config + minimize call for ioutil.TempDir()
* [NOD-1162] Add some clarifications in code
* [NOD-1193] Skip closed connections in NetAdapter.Broadcast
* [NOD-1193] Make sure to protect connectionsToRouters from concurrent access
* [NOD-1162] Add _test to all files in integration package
* [NOD-1162] Introduced appHarness to better encapsulate a single node
* [NOD-1162] Removed onChainChanged handler
* [NOD-1162] Remove redundant closure
* [NOD-1162] Correctly mark integration_test config as Simnet
* [NOD-1162] Rename app.ID -> app.P2PNodeID
* [NOD-1162] Move TestIntegrationBasicSync to basic_sync_test.go
* [NOD-1210] Made it possible to setup any number of harnesses needed
* [NOD-1210] Rename appHarness1/2 to incoming/outgoing in connect function
* [NOD-1210] Add the 117-incoming-connections integration test
* [NOD-1210] Delete 117-incoming-connections test because it opens too much files
* [NOD-1210] Added function to notify of blocks conveniently
* [NOD-1210] Added function to mine a block from-A-to-Z
* [NOD-1210] Added IBD integration test
* [NOD-1210] Finish test for IBD and fix bug where
requestSelectedTipsIfRequired ran in handshake's goroutine
* [NOD-1210] Set log level to debug
* [NOD-1211] Add test for transaction relay
* [NOD-1211] Compare fix incorrect comaprison in KaspadMessage_RequestTransactions.fromWireMessage
* [NOD-1211] Return ok instead of err from FetchTxDesc and FetchTransaction
* [NOD-1211] Added MsgTransactionNotFound type
* [NOD-1211] Added HandlRequestedTransactions flow
* [NOD-1211] Wait for blocks to be accepted before moving forward
* [NOD-1211] Rename CmdNotFound to CmdTransactionNotFound
* [NOD-1211] Rename: requestAndSolveTemplate -> mineNextBlock
* [NOD-1211] Renamed incoming/outgoing to appHarness1/appHarness2 in isConnected
* [NOD-1211] Move check of Hash == nil to outside wireHashToProto
* [NOD-1211] Instantiate payloadHash before *x
* [NOD-1162] Separate kaspad to it's own package, so that I can use it out of integration test
* [NOD-1162] Begin integration tests
* [NOD-1162] [FIX] Assign cfg to RPCServer
* [NOD-1162] Basic integration test ready
* [NOD-1162] Wait for connection for real
* [NOD-1162] [FIX] Connection manager should run the moment it adds a request
* [NOD-1162] Make connect something that can be invoked in middle of test
* [NOD-1162] Complete first integration test
* [NOD-1162] Undo refactor error
* [NOD-1162] Rename Kaspad to App
* [NOD-1162] Convert checking connection to polling
* [NOD-1162] [FIX] Set peerID on handshake
* [NOD-1162] [FIX] Broadcast should send to outgoing route, not incoming
* [NOD-1162] [FIX] Add CmdInvRelayBlock to MakeEmptyMessage
* [NOD-1162] [FIX] Initialize Hash before decoding MsgInvRelayBlock
* [NOD-1162] [FIX] Invert condition
* [NOD-1162] [FIX] Fixes to encoding of MsgGetRelayBlocks
* [NOD-1162] [FIX] Add MsgGetRelayBlocks to MakeEmptyMessage
* [NOD-1162] [FIX] Connection manager should run the moment it adds a request
* [NOD-1162] [FIX] Set peerID on handshake
* [NOD-1162] [FIX] Broadcast should send to outgoing route, not incoming
* [NOD-1162] [FIX] Add CmdInvRelayBlock to MakeEmptyMessage
* [NOD-1162] [FIX] Initialize Hash before decoding MsgInvRelayBlock
* [NOD-1162] [FIX] Invert condition
* [NOD-1162] [FIX] Fixes to encoding of MsgGetRelayBlocks
* [NOD-1162] [FIX] Add MsgGetRelayBlocks to MakeEmptyMessage
* [NOD-1162] Add comment
* [NOD-1162] Added support for 3 nodes and clients in integration tests
* [NOD-1162] Add third node to integration test
* [NOD-1192] Use lock-less functions in TxPool.HandleNewBlock
* [NOD-1192] Broadcast transactions only if there's more then 0
* [NOD-1162] Removed double waitTillNextIteration
* [NOD-1192] Rename: broadcastTransactions -> broadcastTransactionsAfterBlockAdded
* [NOD-1162] Call NotifyBlocks on client3 as well
* [NOD-1162] ErrTimeout and ErrRouteClosed should be ProtocolErrors
* [NOD-1162] Added comment and removed redundant type PeerAddedCallback
* [NOD-1162] Revert overly eager rename
* [NOD-1162] Move DisalbeTLS to common config + minimize call for ioutil.TempDir()
* [NOD-1162] Add some clarifications in code
* [NOD-1193] Skip closed connections in NetAdapter.Broadcast
* [NOD-1193] Make sure to protect connectionsToRouters from concurrent access
* [NOD-1162] Add _test to all files in integration package
* [NOD-1162] Introduced appHarness to better encapsulate a single node
* [NOD-1162] Removed onChainChanged handler
* [NOD-1162] Remove redundant closure
* [NOD-1162] Correctly mark integration_test config as Simnet
* [NOD-1162] Rename app.ID -> app.P2PNodeID
* [NOD-1162] Move TestIntegrationBasicSync to basic_sync_test.go
* [NOD-1210] Made it possible to setup any number of harnesses needed
* [NOD-1210] Rename appHarness1/2 to incoming/outgoing in connect function
* [NOD-1210] Add the 117-incoming-connections integration test
* [NOD-1210] Delete 117-incoming-connections test because it opens too much files
* [NOD-1210] Added function to notify of blocks conveniently
* [NOD-1210] Added function to mine a block from-A-to-Z
* [NOD-1210] Added IBD integration test
* [NOD-1210] Finish test for IBD and fix bug where
requestSelectedTipsIfRequired ran in handshake's goroutine
* [NOD-1210] Set log level to debug
* [NOD-1210] A bunch of renamings
* [NOD-1162] Separate kaspad to it's own package, so that I can use it out of integration test
* [NOD-1162] Begin integration tests
* [NOD-1162] [FIX] Assign cfg to RPCServer
* [NOD-1162] Basic integration test ready
* [NOD-1162] Wait for connection for real
* [NOD-1162] [FIX] Connection manager should run the moment it adds a request
* [NOD-1162] Make connect something that can be invoked in middle of test
* [NOD-1162] Complete first integration test
* [NOD-1162] Undo refactor error
* [NOD-1162] Rename Kaspad to App
* [NOD-1162] Convert checking connection to polling
* [NOD-1162] [FIX] Set peerID on handshake
* [NOD-1162] [FIX] Broadcast should send to outgoing route, not incoming
* [NOD-1162] [FIX] Add CmdInvRelayBlock to MakeEmptyMessage
* [NOD-1162] [FIX] Initialize Hash before decoding MsgInvRelayBlock
* [NOD-1162] [FIX] Invert condition
* [NOD-1162] [FIX] Fixes to encoding of MsgGetRelayBlocks
* [NOD-1162] [FIX] Add MsgGetRelayBlocks to MakeEmptyMessage
* [NOD-1162] [FIX] Connection manager should run the moment it adds a request
* [NOD-1162] [FIX] Set peerID on handshake
* [NOD-1162] [FIX] Broadcast should send to outgoing route, not incoming
* [NOD-1162] [FIX] Add CmdInvRelayBlock to MakeEmptyMessage
* [NOD-1162] [FIX] Initialize Hash before decoding MsgInvRelayBlock
* [NOD-1162] [FIX] Invert condition
* [NOD-1162] [FIX] Fixes to encoding of MsgGetRelayBlocks
* [NOD-1162] [FIX] Add MsgGetRelayBlocks to MakeEmptyMessage
* [NOD-1162] Add comment
* [NOD-1162] Added support for 3 nodes and clients in integration tests
* [NOD-1162] Add third node to integration test
* [NOD-1192] Use lock-less functions in TxPool.HandleNewBlock
* [NOD-1192] Broadcast transactions only if there's more then 0
* [NOD-1162] Removed double waitTillNextIteration
* [NOD-1192] Rename: broadcastTransactions -> broadcastTransactionsAfterBlockAdded
* [NOD-1162] Call NotifyBlocks on client3 as well
* [NOD-1162] ErrTimeout and ErrRouteClosed should be ProtocolErrors
* [NOD-1162] Added comment and removed redundant type PeerAddedCallback
* [NOD-1162] Revert overly eager rename
* [NOD-1162] Move DisalbeTLS to common config + minimize call for ioutil.TempDir()
* [NOD-1162] Add some clarifications in code
* [NOD-1193] Skip closed connections in NetAdapter.Broadcast
* [NOD-1193] Make sure to protect connectionsToRouters from concurrent access
* [NOD-1162] Add _test to all files in integration package
* [NOD-1162] Introduced appHarness to better encapsulate a single node
* [NOD-1162] Removed onChainChanged handler
* [NOD-1162] Remove redundant closure
* [NOD-1162] Correctly mark integration_test config as Simnet
* [NOD-1162] Rename app.ID -> app.P2PNodeID
* [NOD-1120] Separate flow registration and running
* [NOD-1120] Extract executeFunc to separate function
* [NOD-1120] Move the registration of flows out of goroutine
* [NOD-1120] Return after handleError
* [NOD-1120] Rename: addXXXFlow -> registerXXXFlow
* Rename: stop -> errChan
* [NOD-1120] Fix name of goroutine
* [NOD-1198] Make router a property of netConnection, and remove map from connection to router in netAdapter
* [NOD-1198] Moved all router logic from netAdapter to netConnection
* [NOD-1198] Move disconnect to NetConnection
* [NOD-1198] Unexport netConnection.start
* [NOD-1198] Remove error from Disconnect functions
* [NOD-1198] Make sure OnDisconnectedHandler doesn't run when it shouldn't
* [NOD-1194] Make error handling more centralized, and ignore ErrRouteClosed
* [NOD-1194] Ignore ErrRouteClosed in connection_loops as well
* [NOD-1194] Enhance comment
* [NOD-1194] Return after any HandleError
* [NOD-1194] Rephrased comment
* [NOD-1079] Fix block rejects over "Already have block" (#783)
* [NOD-1079] Return regular error instead of ruleError on already-have-block in ProcessBlock.
* [NOD-1079] Fix bad implementation of IsSelectedTipKnown.
* [NOD-1079] In shouldQueryPeerSelectedTips use selected DAG tip timestamp instead of past median time.
* [NOD-1079] Remove redundant (and possibly buggy) clearing of sm.requestedBlocks.
* [NOD-684] change simnet block rate to block per ms (#782)
* [NOD-684] Get rid of dag.targetTimePerBlock and use finality duration in dag params
* [NOD-684] Fix regtest genesis block
* [NOD-684] Set simnet's TargetTimePerBlock to 1ms
* [NOD-684] Shorten simnet finality duration
* [NOD-684] Change isDAGCurrentMaxDiff to be written as number of blocks
* [NOD-684] Fix NextBlockMinimumTime to be add one millisecond after past median time
* [NOD-1004] Make AddrManager.getAddress use only 1 loop to check all address chances and pick one of them (#741)
* [NOD-1004] Remove code duplication in Good().
* [NOD-1004] Remove some more code duplication in Good().
* [NOD-1004] Remove some more code duplication in Good().
* [NOD-1004] Remove code duplication in GetAddress().
* [NOD-1004] Remove code duplication in updateAddress.
* [NOD-1004] Remove some more code duplication in updateAddress.
* [NOD-1004] Remove redundant check in expireNew.
* [NOD-1004] Remove superfluous existence check from updateAddress.
* [NOD-1004] Make triedBucket use a slice instead of a list.
* [NOD-1004] Remove code duplication in getAddress.
* [NOD-1004] Remove infinite loops out of getAddress.
* [NOD-1004] Made impossible branch panic.
* [NOD-1004] Remove a mystery comment.
* [NOD-1004] Remove an unnecessary sort.
* [NOD-1004] Make AddressKey a type alias.
* [NOD-1004] Added comment for AddressKey
* [NOD-1004] Fix merge errors.
* [NOD-1004] Fix merge errors.
* [NOD-1004] Do some renaming.
* [NOD-1004] Do some more renaming.
* [NOD-1004] Rename AddrManager to AddressManager.
* [NOD-1004] Rename AddrManager to AddressManager.
* [NOD-1004] Do some more renaming.
* [NOD-1004] Rename bucket to addressBucketArray.
* [NOD-1004] Fix a comment.
* [NOD-1004] Rename na to netAddress.
* [NOD-1004] Bring back an existence check.
* [NOD-1004] Fix an error message.
* [NOD-1004] Fix a comment.
* [NOD-1004] Use a boolean instead of -1.
* [NOD-1004] Use a boolean instead of -1 in another place.
Co-authored-by: Mike Zak <feanorr@gmail.com>
* Fix merge errors.
* [NOD-1181] Move isBanned logic into addressManager.
* [NOD-1181] Persist bans to disk.
* [NOD-1181] Add comments.
* [NOD-1181] Add an additional exit condition to the connection loop.
* [NOD-1181] Add a TODO.
* [NOD-1181] Wrap not-found errors in addressManager.
* [NOD-1181] Fix a comment.
* [NOD-1181] Rename banned to isBanned.
* [NOD-1181] Fix bad error handling in routerInitializer.
* [NOD-1181] Remove a TODO.
Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: Mike Zak <feanorr@gmail.com>
* [NOD-1079] Fix block rejects over "Already have block" (#783)
* [NOD-1079] Return regular error instead of ruleError on already-have-block in ProcessBlock.
* [NOD-1079] Fix bad implementation of IsSelectedTipKnown.
* [NOD-1079] In shouldQueryPeerSelectedTips use selected DAG tip timestamp instead of past median time.
* [NOD-1079] Remove redundant (and possibly buggy) clearing of sm.requestedBlocks.
* [NOD-684] change simnet block rate to block per ms (#782)
* [NOD-684] Get rid of dag.targetTimePerBlock and use finality duration in dag params
* [NOD-684] Fix regtest genesis block
* [NOD-684] Set simnet's TargetTimePerBlock to 1ms
* [NOD-684] Shorten simnet finality duration
* [NOD-684] Change isDAGCurrentMaxDiff to be written as number of blocks
* [NOD-684] Fix NextBlockMinimumTime to be add one millisecond after past median time
* [NOD-1004] Make AddrManager.getAddress use only 1 loop to check all address chances and pick one of them (#741)
* [NOD-1004] Remove code duplication in Good().
* [NOD-1004] Remove some more code duplication in Good().
* [NOD-1004] Remove some more code duplication in Good().
* [NOD-1004] Remove code duplication in GetAddress().
* [NOD-1004] Remove code duplication in updateAddress.
* [NOD-1004] Remove some more code duplication in updateAddress.
* [NOD-1004] Remove redundant check in expireNew.
* [NOD-1004] Remove superfluous existence check from updateAddress.
* [NOD-1004] Make triedBucket use a slice instead of a list.
* [NOD-1004] Remove code duplication in getAddress.
* [NOD-1004] Remove infinite loops out of getAddress.
* [NOD-1004] Made impossible branch panic.
* [NOD-1004] Remove a mystery comment.
* [NOD-1004] Remove an unnecessary sort.
* [NOD-1004] Make AddressKey a type alias.
* [NOD-1004] Added comment for AddressKey
* [NOD-1004] Fix merge errors.
* [NOD-1004] Fix merge errors.
* [NOD-1004] Do some renaming.
* [NOD-1004] Do some more renaming.
* [NOD-1004] Rename AddrManager to AddressManager.
* [NOD-1004] Rename AddrManager to AddressManager.
* [NOD-1004] Do some more renaming.
* [NOD-1004] Rename bucket to addressBucketArray.
* [NOD-1004] Fix a comment.
* [NOD-1004] Rename na to netAddress.
* [NOD-1004] Bring back an existence check.
* [NOD-1004] Fix an error message.
* [NOD-1004] Fix a comment.
* [NOD-1004] Use a boolean instead of -1.
* [NOD-1004] Use a boolean instead of -1 in another place.
Co-authored-by: Mike Zak <feanorr@gmail.com>
* Fix merge errors.
Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: Mike Zak <feanorr@gmail.com>
* [NOD-1192] Use lock-less functions in TxPool.HandleNewBlock
* [NOD-1192] Broadcast transactions only if there's more then 0
* [NOD-1192] Rename: broadcastTransactions -> broadcastTransactionsAfterBlockAdded
* [NOD-1162] [FIX] Connection manager should run the moment it adds a request
* [NOD-1162] [FIX] Set peerID on handshake
* [NOD-1162] [FIX] Broadcast should send to outgoing route, not incoming
* [NOD-1162] [FIX] Add CmdInvRelayBlock to MakeEmptyMessage
* [NOD-1162] [FIX] Initialize Hash before decoding MsgInvRelayBlock
* [NOD-1162] [FIX] Invert condition
* [NOD-1162] [FIX] Fixes to encoding of MsgGetRelayBlocks
* [NOD-1162] [FIX] Add MsgGetRelayBlocks to MakeEmptyMessage
* [NOD-1162] Add comment
* [NOD-1123] Bubble bad-message errors up to the protocol level.
* [NOD-1123] Implement Banning.
* [NOD-1123] Properly use &stopped.
* [NOD-1123] Ban by IP rather than IP and port.
* [NOD-1123] Don't initiate connections to banned peers.
* [NOD-1123] Fix infinite loop in checkOutgoingConnections.
* [NOD-1123] Fix bannedAddresses key.
* [NOD-1123] Rename onBadMessageHandler to onInvalidMessageHandler.
* [NOD-1175] Get rid of something weird.
* [NOD-1175] Implement AddBlock.
* [NOD-1175] Implement BFDisallowOrphans.
* [NOD-1175] Pass flags into AddBlock.
* [NOD-1175] Remove isOrphan and isDelayed handling from AddBlock.
* [NOD-1175] Use default return values in error.
* [NOD-1175] Bring back a comment.
* [NOD-1175] Add ErrOrphanBlockIsNotAllowed to errorCodeStrings.
* [NOD-1170] Return a custom error when a route is closed
* [NOD-1170] Return ErrRouteClosed directly from route methods
* [NOD-1170] Fix comment location
* [NOD-1164] Defined DatabaseContext as the basic object of dbaccess
* [NOD-1164] Update everything to use databaseContext
* [NOD-1164] Fix tests
* [NOD-1164] Add comments
* [NOD-1164] Removed databaseContext from blockNode
* [NOD-1164] Enforce DatabaseContext != nil
* [NOD-1164] Remove redundant and wrong comment line
* [NOD-1125] Write a skeleton for starting IBD.
* [NOD-1125] Add WaitForIBDStart to Peer.
* [NOD-1125] Move functions around.
* [NOD-1125] Fix merge errors.
* [NOD-1125] Fix a comment.
* [NOD-1125] Implement sendGetBlockLocator.
* [NOD-1125] Begin implementing findIBDLowHash.
* [NOD-1125] Finish implementing findIBDLowHash.
* [NOD-1125] Rename findIBDLowHash to findHighestSharedBlockHash.
* [NOD-1125] Implement downloadBlocks.
* [NOD-1125] Implement msgIBDBlock.
* [NOD-1125] Implement msgIBDBlock.
* [NOD-1125] Fix message types for HandleIBD.
* [NOD-1125] Write a skeleton for requesting selected tip hashes.
* [NOD-1125] Write a skeleton for the rest of the IBD requests.
* [NOD-1125] Implement HandleGetBlockLocator.
* [NOD-1125] Fix wrong timeout.
* [NOD-1125] Fix compilation error.
* [NOD-1125] Implement HandleGetBlocks.
* [NOD-1125] Fix compilation errors.
* [NOD-1125] Fix merge errors.
* [NOD-1125] Implement selectPeerForIBD.
* [NOD-1125] Implement RequestSelectedTip.
* [NOD-1125] Implement HandleGetSelectedTip.
* [NOD-1125] Make go lint happy.
* [NOD-1125] Add minGetSelectedTipInterval.
* [NOD-1125] Call StartIBDIfRequired where needed.
* [NOD-1125] Fix merge errors.
* [NOD-1125] Remove a redundant line.
* [NOD-1125] Rename shouldContinue to shouldStop.
* [NOD-1125] Lowercasify an error message.
* [NOD-1125] Shuffle statements around in findHighestSharedBlockHash.
* [NOD-1125] Rename hasRecentlyReceivedBlock to isDAGTimeCurrent.
* [NOD-1125] Scope minGetSelectedTipInterval.
* [NOD-1125] Handle an unhandled error.
* [NOD-1125] Use AddUint32 instead of LoadUint32 + StoreUint32.
* [NOD-1125] Use AddUint32 instead of LoadUint32 + StoreUint32.
* [NOD-1125] Use SwapUint32 instead of AddUint32.
* [NOD-1125] Remove error from requestSelectedTips.
* [NOD-1125] Actually stop IBD when it should stop.
* [NOD-1125] Actually stop RequestSelectedTip when it should stop.
* [NOD-1125] Don't ban peers that send us delayed blocks during IBD.
* [NOD-1125] Make unexpected message type messages nicer.
* [NOD-1125] Remove Peer.ready and make HandleHandshake return it to guarantee we never operate on a non-initialized peer.
* [NOD-1125] Remove errors associated with Peer.ready.
* [NOD-1125] Extract maxHashesInMsgIBDBlocks to a const.
* [NOD-1125] Move the ibd package into flows.
* [NOD-1125] Start IBD if required after getting an unknown block inv.
* [NOD-1125] Don't request blocks during relay if we're in the middle of IBD.
* [NOD-1125] Remove AddBlockLocatorHash.
* [NOD-1125] Extract runIBD to a seperate function.
* [NOD-1125] Extract runSelectedTipRequest to a seperate function.
* [NOD-1125] Remove EnqueueWithTimeout.
* [NOD-1125] Increase the capacity of the outgoingRoute.
* [NOD-1125] Fix some bad names.
* [NOD-1125] Fix a comment.
* [NOD-1125] Simplify a comment.
* [NOD-1125] Move WaitFor... functions into their respective run... functions.
* [NOD-1125] Return default values in case of error.
* [NOD-1125] Use CmdXXX in error messages.
* [NOD-1125] Use MaxInvPerMsg in outgoingRouteMaxMessages instead of MaxBlockLocatorsPerMsg.
* [NOD-1125] Fix a comment.
* [NOD-1125] Disconnect a peer that sends us a delayed block during IBD.
* [NOD-1125] Use StoreUint32 instead of SwapUint32.
* [NOD-1125] Add a comment.
* [NOD-1125] Don't ban peers that send us delayed blocks.
* [NOD-1148] Add lock around route's close operation
* [NOD-1148] Added tracing of incoming and outgoing messages
* [NOD-1148] Cast to MsgPing should have been to MsgPong
* [NOD-1148] Check for NeedMoreAddresses before sending GetAddr message
and invert condition
* [NOD-1163] Combine seperated flows into single packages
* [NOD-1163] Move handshake.go to handshake package
* [NOD-1163] Use single logger prefix for everything under protocol
* [NOD-1163] Add comment
* [NOD-1163] Fix refactor error
* [NOD-1120] Removed closure in NetAdapter.onConnectedHanlder
* [NOD-1120] Implement all connection manager methods
* [NOD-1120] Integrated connmanager into kaspad + added call for dnsseeder
* [NOD-1120] Allow buffer to not be bytes.Buffer
* [NOD-1120] Added timeout to connect
* [NOD-1120] Don't enter connections to add loop if none needed
* [NOD-1120] Add call for addressManager.Good
* [NOD-1120] Minor bug fixes
* [NOD-1120] Remove errChan from grpcConnection
* [NOD-1120] Add comments to exported methods
* [NOD-1120] cancel the context for DialContext in gRPCServer.Connect
* [NOD-1120] Don't try to remove from connSet a connection that doesn't exist
* [NOD-1120] add ok bool to connectionSet.get
* [NOD-1120] Remove overuse of if-else in checkConnectionRequests
* [NOD-1120] Made some order in ConnectionManager
* [NOD-1120] Moved checkIncomingConnections to it's own file
* [NOD-1120] cleanup in checkOutgoingConnections
* [NOD-1120] Cleanup in SeedDNS, and move call outside of connection manager
* [NOD-1120] Add check that both --connect and --addpeer aren't used
* [NOD-1120] Move dial timeout to constant
* [NOD-1120] Enhance comment
* [NOD-1120] Log connection failure out of initiateConnection
* [NOD-1148] Reshuffle checkRequestedConnections to make more sense
* [NOD-1120] Move continue to correct place + reshuffle logging code
* [NOD-1120] Don't expose server.Connection outside netAdapter - expose a wrapper instead
* [NOD-1120] Add comments
* [NOD-1120] Don't return the connection from netAdapter.Connect()
* [NOD-1120] Use .Address as key for connectionSet
* [NOD-1120] Fix minRetryDuration usage
* [NOD-1120] Remove the correct number of incoming connections
* [NOD-1120] Add comment
* [NOD-1120] Rename connSet -> incomingConnectionSet
* [NOD-1120] fix grammar
* [NOD-1142] Implement EnqueueWithTimeout and DequeueWithTimeout.
* [NOD-1142] Use DequeueWithTimeout in readMsgBlock.
* [NOD-1142] Add comments about the new methods.
* [NOD-1124] Move Router to the router package.
* [NOD-1124] Implement SetOnRouteCapacityReachedHandler.
* [NOD-1124] Use Routes instead of bare channels.
* [NOD-1124] Fix merge errors.
* [NOD-1124] Connect the Router to the Connection.
* [NOD-1124] Fix merge errors.
* [NOD-1124] Move some variables around.
* [NOD-1124] Fix unreachable code.
* [NOD-1124] Fix a variable name.
* [NOD-1124] Rename AddRoute to AddIncomingRoute.
* [NOD-1124] Rename SetRouter to Start.
* [NOD-1124] Make AddIncomingRoute create a Route by itself.
* [NOD-1124] Replace IncomingRoute with EnqueueIncomingMessage.
* [NOD-1124] Make Enqueue and Dequeue return isOpen instead of err.
* [NOD-1124] Remove writeDuringDisconnectLock.
* [NOD-1124] In sendLoop, move outgoingRoute to outside the loop.
* [NOD-1124] Start the connection loops only when Start is called.
* [NOD-1124] Replace OnIDReceivedHandler with AssociateRouterID.
* [NOD-1124] Add isOpen to Enqueue and Dequeue.
* [NOD-1124] Protect errChan from writing during disconnect.
* [NOD-1118] Added protobufs for the MessageStream
* [NOD-1118] Implement some of the basic grpc methods
* [NOD-1118] Implemented gRPCConnection send and receive
* [NOD-1118] Implemented basic connection loops
* [NOD-1118] gRPC server implementation ready
* [NOD-1118] Add connection management
* [NOD-1118] Sort out the connection loops
* [NOD-1118] Add temporary testConnection
* [NOD-1118] Send to c.errChan whether error was recieved or not
* [NOD-1118] Call OnConnectHandler in time
* [NOD-1118] Handle closing connections properly
* [NOD-1118] Add comments to exported functions
* [NOD-1118] Call server.addConnection on newConnection
* [NOD-1118] Add a TODO comment
* [NOD-1118] Add a TODO comment
* [NOD-1118] Make connection a Stringer
* [NOD-1118] Made the connection loops 100% synchronic
* [NOD-1118] Make connection.isConnected uint32
* [NOD-1118] Move the Add/Remove connection from grpcConnection to register/unregister connection
* [NOD-1118] Convert error messages to lower case
* [NOD-1118] Remove protoc inline dependency
* [NOD-1118] Fix comment
* [NOD-1118] Exit if there was an error starting the protocol manager
* [NOD-1118] Fix error message
* [NOD-1118] Fixed a few comments
* [NOD-1118] Extract listenOn to a method
* [NOD-1118] Use !=0 for isConnected
* [NOD-1118] Refactor listenOn
* [NOD-1118] Add lock on channelWrites in gRPCConnection
* [NOD-1118] Rename channelWriteLock -> writeDuringDisconnectLock
* [NOD-1118] Reshuffle a comment
* [NOD-1118] Add a TODO comment
* [NOD-1124] Rename Peer to Connection (because Peer is a business logic term)
* [NOD-1124] Implement Close for Router.
* [NOD-1124] Add SetPeerDisconnectedHandler.
* [NOD-1124] Remove mentions of "peer" from the netadapter package.
* [NOD-1124] Handle errors/stopping in netadapter.
* [NOD-1124] Remove netadapter.Connection.
* [NOD-1124] Add startSendLoop.
* [NOD-1124] Implement network IDs.
* [NOD-1124] Implement a map between IDs and routes.
* [NOD-1124] Implement Broadcast.
* [NOD-1124] Fix rename error.
* [NOD-1124] Fix copy+paste error.
* [NOD-1124] Change the type of NetAdapter.stop to uint32.
* [NOD-1124] If NetAdapter is stopped more than once, return an error.
* [NOD-1124] Add an error case to RouteInputMessage.
* [NOD-1124] Rename CreateID to NewID.
* [NOD-1124] Spawn from outside startReceiveLoop and startSendLoop.
* [NOD-1124] Fix a comment.
* [NOD-1124] Replace break with for condition.
* [NOD-1124] Don't disconnect from disconnected peers.
* [NOD-1124] Fix a for condition.
* [NOD-1124] Handle an error.
* [NOD-1124] Begin implementing netadapter.
* [NOD-1124] Implementing a stub gRPC server..
* [NOD-1124] Construct the server inside the netadapter.
* [NOD-1124] Rewrite protocol.go to fit with the new netAdapter model.
* [NOD-1124] Wrap a connection in Peer.
* [NOD-1124] Add a peerstate object.
* [NOD-1124] Remove the peerstate object.
* [NOD-1124] Remove router out of Peer.
* [NOD-1124] Tag a TODO.
* [NOD-1124] Return an error out of AddRoute if a route already exists for some message type.
* [NOD-1124] Rename the package grpc to grpcserver.
* [NOD-1124] Extracted newConnectionHandler into a type.
* [NOD-1124] Extract routerInitializer into a type.
* [NOD-1124] Panic/Add TODOs everywhere that isn't implemented.
* [NOD-1124] Improve the NetAdapter comment.
* [NOD-1124] Rename NewConnectionHandler to PeerConnectedHandler.
* [NOD-1124] Rename buildRouterInitializer to newRouterInitializer.
* [NOD-1124] Remove unreachable code.
* [NOD-1124] Make go vet happy.
* [NOD-1119] Removed all p2p server from all the initialization of server
* [NOD-1119] Removed any calling for p2p server in main
* [NOD-1119] Simplified some functions to not take both dag and dagParams
* [NOD-1119] Simplify creation of mempool and rpc server
* [NOD-1119] Setup indexes in separate function
* [NOD-1119] Some cleanup in NewServer
* [NOD-1119] Fix mempool test
* [NOD-1119] Fix go format
* [NOD-1119] Unexport dag.timeSource
* [NOD-1119] Removed server package + renamed the Server object to Kaspad, and made it minimal
* [NOD-1119] Delete redundant functions
* Unexported kaspad and related methods
* [NOD-1119] Unexported newKaspad
* [NOD-1119] Revise comments and remove redundant function
* [NOD-1119] Make comments of unexported methods lower-case
* [NOD-1119] Some more refactoring in newKaspad
* [NOD-1098] Change timestamps to be millisecond precision
* [NOD-1098] Change lock times to use milliseconds
* [NOD-1098] Use milliseconds precision everywhere
* [NOD-1098] Implement type mstime.Time
* [NOD-1098] Fix block 100000 timestamp
* [NOD-1098] Change orphan child to be one millisecond delay after its parent
* [NOD-1098] Remove test that checks if header timestamps have the right precision, and instead add tests for mstime, and fix genesis for testnet and devnet
* [NOD-1098] Fix comment
* [NOD-1098] Fix comment
* [NOD-1098] Fix testnet genesis
* [NOD-1098] Rename UnixMilli->UnixMilliseconds
* [NOD-1105] Don't use a database transaction when recovering acceptance index.
* Revert "[NOD-1105] Don't use a database transaction when recovering acceptance index."
This reverts commit da550f8e
* [NOD-1105] When recovering acceptance index, use a database transaction per block instead of for the entire recovery.
commit e303efef42
Author: stasatdaglabs <stas@daglabs.com>
Date: Mon Jun 29 11:59:36 2020 +0300
[NOD-1063] Rename a test.
commit bfecd57470
Author: stasatdaglabs <stas@daglabs.com>
Date: Mon Jun 29 11:57:36 2020 +0300
[NOD-1063] Fix a comment.
commit b969e5922d
Author: stasatdaglabs <stas@daglabs.com>
Date: Sun Jun 28 18:14:44 2020 +0300
[NOD-1063] Convert modifiedTreeNode to an out param.
commit 170f9872f4
Author: stasatdaglabs <stas@daglabs.com>
Date: Sun Jun 28 17:05:01 2020 +0300
[NOD-1063] Fix a bug in which a block is added to the selected parent chain below the reindex root.
* [NOD-1063] Fix a bug in which a block is added to the selected parent chain below the reindex root.
* [NOD-1063] Convert modifiedTreeNode to an out param.
* [NOD-1063] Fix a comment.
* [NOD-1063] Rename a test.
* Add hashMerkleRoot field to GetBlockTemplateResult
* Use hashMerkleRoot from template instead of recalculating
* Move ParseBlock from kaspaminer into rpcclient
* Rename ParseBlock to ConvertGetBlockTemplateResultToBlock and wrap errors
* [NOD-1055] Give higher priority for requesting missing ancestors when sending a getdata message (#767)
* [NOD-1063] Remove the remainingInterval field.
* [NOD-1063] Add helper functions to reachabilityTreeNode.
* [NOD-1063] Add reachabilityReindexRoot.
* [NOD-1063] Start implementing findNextReachabilityReindexRoot.
* [NOD-1063] Implement findCommonAncestor.
* [NOD-1063] Implement findReachabilityTreeAncestorInChildren.
* [NOD-1063] Add reachabilityReindexWindow.
* [NOD-1063] Fix findReachabilityTreeAncestorInChildren.
* [NOD-1063] Remove BlockDAG reference in findReachabilityTreeAncestorInChildren.
* [NOD-1063] Extract updateReachabilityReindexRoot to a separate function.
* [NOD-1063] Add reachabilityReindexSlack.
* [NOD-1063] Implement splitReindexRootChildrenAroundChosen.
* [NOD-1063] Implement calcReachabilityTreeNodeSizes.
* [NOD-1063] Implement propagateChildIntervals.
* [NOD-1063] Extract tightenReachabilityTreeIntervalsBeforeChosenReindexRootChild and tightenReachabilityTreeIntervalsAfterChosenReindexRootChild to separate functions.
* [NOD-1063] Implement expandReachabilityTreeIntervalInChosenReindexRootChild.
* [NOD-1063] Finished implementing concentrateReachabilityTreeIntervalAroundReindexRootChild.
* [NOD-1063] Begin implementing reindexIntervalsBeforeReindexRoot.
* [NOD-1063] Implement top-level logic of reindexIntervalsBeforeReindexRoot.
* [NOD-1063] Implement reclaimIntervalBeforeChosenChild.
* [NOD-1063] Add a debug log for reindexIntervalsBeforeReindexRoot.
* [NOD-1063] Rename reindexIntervalsBeforeReindexRoot to reindexIntervalsEarlierThanReindexRoot.
* [NOD-1063] Implement reclaimIntervalAfterChosenChild.
* [NOD-1063] Add a debug log for updateReachabilityReindexRoot.
* [NOD-1063] Convert modifiedTreeNodes from slices to sets.
* [NOD-1063] Fix findCommonAncestor.
* [NOD-1063] Fix reindexIntervalsEarlierThanReindexRoot.`
* [NOD-1063] Remove redundant nil conditions.
* [NOD-1063] Make map[*reachabilityTreeNode]struct{} into a type alias with a copyAllFrom method.
* [NOD-1063] Remove setInterval.
* [NOD-1063] Create a new struct to hold reachability stuff called reachabilityTree.
* [NOD-1063] Rename functions under reachabilityTree.
* [NOD-1063] Move reachabilityStore into reachabilityTree.
* [NOD-1063] Move the rest of the functions in reachability.go into the reachabilityTree struct.
* [NOD-1063] Update newReachabilityTree to take an instance of reachabilityStore.
* [NOD-1063] Fix merge errors.
* [NOD-1063] Fix merge errors.
* [NOD-1063] Pass a reference to the dag into reachabilityTree.
* [NOD-1063] Use Wrapf instead of Errorf.
* [NOD-1063] Merge assignments.
* [NOD-1063] Disambiguate a varaible name.
* [NOD-1063] Add a test case for intervalBefore.
* [NOD-1063] Simplify splitChildrenAroundChosenChild.
* [NOD-1063] Fold temporary variables into newReachabilityInterval.
* [NOD-1063] Fold more temporary variables into newReachabilityInterval.
* [NOD-1063] Fix a bug in expandIntervalInReindexRootChosenChild.
* [NOD-1063] Remove blockNode from futureCoveringBlock.
* [NOD-1063] Get rid of futureCoveringBlock.
* [NOD-1063] Use findIndex directly in findAncestorAmongChildren.
* [NOD-1063] Make findIndex a bit nicer to use. Also rename it to findAncestorIndexOfNode.
* [NOD-1063] Rename childIntervalAllocationRange to intervalRangeForChildAllocation.
* [NOD-1063] Optimize findCommonAncestor.
* [NOD-1063] In reindexIntervalsBeforeChosenChild, use chosenChild.interval.start - 1 instead of childrenBeforeChosen[len(childrenBeforeChosen)-1].interval.end + 1.
* [NOD-1063] Rename reindexIntervalsBeforeChosenChild to reindexIntervalsBeforeNode.
* [NOD-1063] Add a comment explain what "the chosen child" is.
* [NOD-1063] In concentrateIntervalAroundReindexRootChosenChild, rename modifiedTreeNodes to allModifiedTreeNodes.
* [NOD-1063] Extract propagateIntervals to a function.
* [NOD-1063] Extract interval "contains" logic to a separate function.
* [NOD-1063] Simplify "looping up" logic in reclaimIntervalXXXChosenChild.
* [NOD-1063] Add comments to reclaimIntervalXXXChosenChild.
* [NOD-1063] Rename copyAllFrom to addAll.
* [NOD-1063] Rename reachabilityStore (the variable) to just store.
* [NOD-1063] Fix an error message.
* [NOD-1063] Reword a comment.
* [NOD-1063] Don't return -1 from findAncestorIndexOfNode.
* [NOD-1063] Extract slackReachabilityIntervalForReclaiming to a constant.
* [NOD-1063] Add a missing condition.
* [NOD-1063] Call isAncestorOf directly in insertNode.
* [NOD-1063] Rename chosenReindexRootChild to reindexRootChosenChild.
* [NOD-1063] Rename treeNodeSet to orderedTreeNodeSet.
* [NOD-1063] Add a disclaimer to orderedTreeNodeSet.
* [NOD-1063] Implement StoreReachabilityReindexRoot and FetchReachabilityReindexRoot.
* [NOD-1063] Move storing the reindex root to within reachabilityTree.
* [NOD-1063] Remove isAncestorOf from reachabilityInterval.
* [NOD-1063] Add a comment about graph theory conventions.
* [NOD-1063] Fix tests.
* [NOD-1063] Change inclusion in isAncestorOf functions.
* [NOD-1063] Rename a test.
* [NOD-1063] Implement TestIsInFuture.
* [NOD-1063] Fix error messages in TestIsInFuture.
* [NOD-1063] Fix error messages in TestIsInFuture.
* [NOD-1063] Rename isInSelectedParentChain to isInSelectedParentChainOf.
* [NOD-1063] Rename isInFuture to isInPast.
* [NOD-1063] Expand on a comment.
* [NOD-1063] Rename modifiedTreeNodes.
* [NOD-1063] Implement test: TestReindexIntervalsEarlierThanReindexRoot.
* [NOD-1063] Implement test: TestUpdateReindexRoot.
* [NOD-1063] Explain a check.
* [NOD-1063] Use a method instead of calling reachabilityStore.loaded directly.
* [NOD-1063] Lowercasified an error message.
* [NOD-1063] Fix failing test.
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* [NOD-1038] Give higher priority for requesting missing ancestors when sending a getdata message (#767)
* [NOD-1060] Don't sync from peers that break the netsync protocol
* [NOD-1034] Use reachability to check finality
* [NOD-1034] Add comments and rename variables
* [NOD-1034] Fix comments
* [NOD-1034] Rename checkFinalityRules->checkFinalityViolation
* [NOD-1034] Change isAncestorOf to be exclusive
* [NOD-1034] Make isAncestorOf exclusive and also more explicit, and add TestReachabilityTreeNodeIsAncestorOf
* [NOD-530] Remove coinbase inputs and add blue score to payload
* [NOD-1042] Ignore very high orphans
* [NOD-1042] Add ban score to an orphan with malformed blue score
* [NOD-1042] Fix log
* [NOD-614] Copy bitcoin-core ban score policy
* [NOD-614] Add ban score to disconnects
* [NOD-614] Fix wrong branch of AddBanScore
* [NOD-614] Add ban score on sending too many addresses
* [NOD-614] Add comments
* [NOD-614] Remove redundant reject messages
* [NOD-614] Fix log message
* [NOD-614] Ban every node that sends invalid invs
* [NOD-614] Make constants for ban scores
* [NOD-1007] Split checkBlockSanity subroutines.
* [NOD-1007] Put back the comments about performance.
* [NOD-1007] Make all the functions in checkBlockSanity take a *util.Block.
* [NOD-1007] Rename checkBlockTransactionsOrderedBySubnetwork to checkBlockTransactionOrder.
* [NOD-1007] Move a comment up a scope level.
* [NOD-1048] Make leveldb compaction much less frequent. Also, allocate an entire gigabyte for leveldb's blockCache and writeBuffer.
* [NOD-1048] Implement changing the options for testing purposes.
* [NOD-1048] Rename originalOptions to originalLDBOptions.
* [NOD-1048] Add a comment.
* [NOD-1040] Don't remove DAG tips from the diffStore's loaded set
* [NOD-1040] Fix TestClearOldEntries.
* Revert "[NOD-1040] Fix TestClearOldEntries."
This reverts commit e0705814
* Revert "[NOD-1040] Don't remove DAG tips from the diffStore's loaded set"
This reverts commit d3eba1c1
* [NOD-1040] Increase maxBlueScoreDifferenceToKeepLoaded to 1500.
* [NOD-1012] Disallow non-native/coinbase transactions.
* [NOD-1012] Fix logic error.
* [NOD-1012] Fix/skip tests and remove --subnetwork.
* [NOD-1012] Disconnect from non-native peers.
* [NOD-1012] Don't skip subnetwork tests.
* [NOD-1012] Use EnableNonNativeSubnetworks in peer.go.
* [NOD-1012] Set EnableNonNativeSubnetworks = true in the tests that need them rather than by default in Simnet.
* [NOD-965] Make dag.index.LookupNode return false if node is not found
* [NOD-965] Rename blockDAG->dag
* [NOD-965] Remove irrelevant test
* [NOD-965] Use bi.index's ok in LookupNode
* [NOD-975] Don't include block transactions inside its UTXO commitment (#711)
* [NOD-975] Don't include block transactions inside its UTXO commitment.
* Revert "[NOD-975] Don't include block transactions inside its UTXO commitment."
This reverts commit b1a2ae66
* [NOD-975] Implement a (currently failing) TestUTXOCommitment.
* [NOD-975] Remove the block's own transactions from calcMultiset.
* [NOD-975] Simplify calcMultiset.
* [NOD-975] Add a comment on top of selectedParentMultiset.
* [NOD-975] Use pastUTXO instead of selectedParentUTXO in calcMultiset.
* [NOD-975] Use selected parent's pastUTXO instead of this block's pastUTXO in calcMultiset.
* [NOD-975] Extract selectedParentPastUTXO to a separate function.
* [NOD-975] Remove selectedParentUTXO from pastUTXO's return values.
* [NOD-975] Add txs to TestUTXOCommitment.
* [NOD-975] Remove debug code.
* [NOD-975] In pastUTXOMultiSet, copy the multiset to avoid modifying the original.
* [NOD-975] Add a test: TestPastUTXOMultiSet.
* [NOD-975] Improve TestPastUTXOMultiSet.
* [NOD-976] Implement tests for UTXO commitments (#715)
* [NOD-975] Don't include block transactions inside its UTXO commitment.
* Revert "[NOD-975] Don't include block transactions inside its UTXO commitment."
This reverts commit b1a2ae66
* [NOD-975] Implement a (currently failing) TestUTXOCommitment.
* [NOD-975] Remove the block's own transactions from calcMultiset.
* [NOD-975] Simplify calcMultiset.
* [NOD-975] Add a comment on top of selectedParentMultiset.
* [NOD-975] Use pastUTXO instead of selectedParentUTXO in calcMultiset.
* [NOD-975] Use selected parent's pastUTXO instead of this block's pastUTXO in calcMultiset.
* [NOD-975] Extract selectedParentPastUTXO to a separate function.
* [NOD-975] Remove selectedParentUTXO from pastUTXO's return values.
* [NOD-975] Add txs to TestUTXOCommitment.
* [NOD-976] Generate new blockDB blocks for tests.
* [NOD-976] Fix TestBlueBlockWindow.
* [NOD-976] Fix TestIsKnownBlock.
* [NOD-976] Fix TestGHOSTDAG.
* [NOD-976] Fix TestUTXOCommitment.
* [NOD-976] Remove kaka.
* [NOD-990] Save utxo diffs of past UTXO (#724)
* [NOD-990] Save UTXO diffs of past UTXO
* [NOD-990] Check for block double spends with its past instead of building its UTXO
* [NOD-990] Call resetExtraNonceForTest in TestUTXOCommitment
* [NOD-990] Remove redundant functions diffFromTx and diffFromAcceptedTx
* [NOD-990] Rename i->j to avoid confusion
* [NOD-990] Break long lines
* [NOD-990] Rename ErrDoubleSpendsWithBlockTransaction -> ErrDoubleSpendInSameBlock
* [NOD-990] Make ErrDoubleSpendInSameBlock more detailed
* [NOD-990] Add testProcessBlockRuleError
* [NOD-990] Fix comment
* [NOD-990] Add test for duplicate transactions on the same block
* [NOD-990] Use pkg/errors on panic
* [NOD-990] Make cloneWithoutBase method
* [NOD-990] Break long lines
* [NOD-990] Fix comment
* [NOD-990] Fix wrong variable names
* [NOD-990] Fix comment
* [NOD-974] Generate new test blocks.
* [NOD-974] Fix TestIsKnownBlock and TestGHOSTDAG.
* [NOD-974] Fix TestUTXOCommitment.
* [NOD-974] Fix comments
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
Co-authored-by: stasatdaglabs <stas@daglabs.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* [NOD-1001] Move side-effects of connection out of OnVersion
* [NOD-1001] Make AssociateConnection synchronous
* [NOD-1001] Wait for 2 veracks in TestPeerListeners
* [NOD-1001] Made AssociateConnection return error
* [NOD-1001] Remove temporary logs
* [NOD-1001] Fix typos and find-and-replace errors
* [NOD-1001] Move example_test back out of peer package + fix some typos
* [NOD-1001] Use correct remote address in setupPeersWithConns and return to address string literals
* [NOD-1001] Use separate verack channels for inPeer and outPeer
* [NOD-1001] Make verack channels buffered
* [NOD-1001] Removed temporary sleep of 1 second
* [NOD-1001] Removed redundant //
* [NOD-877] In UTXOEntry serialization, extract packedFlags out to a separate Uint8.
* [NOD-877] Generate new test blocks.
* [NOD-877] Fix TestIsKnownBlock.
* [NOD-877] Fix TestBlueBlockWindow.
* [NOD-877] Fix TestUTXOSerialization and TestGHOSTDAG.
* [NOD-877] Fix TestVirtualBlock.
* [NOD-1006] Make CompactToBig take an out param so that we can reuse the same big.Int in averageTarget.
* [NOD-1006] Fix merge errors.
* [NOD-1006] Use CompactToBigWithDestination only in averageTarget.
* [NOD-1006] Fix refactor errors.
* [NOD-1006] Fix refactor errors.
* [NOD-1006] Optimize averageTarget with a big.Int pool.
* [NOD-1006] Defer releasing bigInts.
* [NOD-1006] Use a pool for requiredDifficulty as well.
* [NOD-1006] Move the big int pool to utils.
* [NOD-1006] Remove unnecessary line.
* [NOD-1005] Moved isSyncedForMining to netsync manager, and renamed to isSynced + removed isCurrent
* [NOD-1005] Use sm.isSynced to check whether should request blocks from invs
* [NOD-1005] Use private version of isSynced to avoid infinite loop
* [NOD-1005] Fix a few typos
* [NOD-993] Use %+v when printing errors
* [NOD-993] Get rid of AssertError
* [NOD-993] Made ruleError use github.com/pkg/errors
* [NOD-993] remove redundant TODO
* [NOD-993] remove redundant Comment
* [NOD-993] Removed DeploymentError
* [NOD-982] Log message with level WARN when getting MsgReject.
* [NOD-982] Fix wrong logLevel in Write and Writef.
* [NOD-982] Use Write and Writef inside Trace, Tracef, Debug, Debugf, etc...
* [NOD-982] Move peer message logging to a separate file.
* [NOD-820] Add IsSynced to GetBlockTemplateResult.
* [NOD-820] Add isSynced to the help file.
* [NOD-820] Add MineWhenNotSynced to the kaspaminer config.
* [NOD-820] Implement miner MineWhenNotSynced logic.
* [NOD-820] Fixed capitalization in an error message.
* [NOD-847] Fix CIDR protection and prevent connecting to the same address twice
* [NOD-847] Fix Tests
* [NOD-847] Add TestDuplicateOutboundConnections and TestSameOutboundGroupConnections
* [NOD-847] Fix TestRetryPermanent, TestNetworkFailure and wait 10 ms before restoring the previous active config
* [NOD-847] Add "is" before boolean methods
* [NOD-847] Fix Connect's lock
* [NOD-847] Make numAddressesInAddressManager an argument
* [NOD-847] Add teardown function for address manager
* [NOD-847] Add stack trace to ConnManager errors
* [NOD-847] Change emptyAddressManagerForTest->createEmptyAddressManagerForTest and fix typos
* [NOD-847] Fix wrong test name for addressManagerForTest
* [NOD-847] Change error message if New fails
* [NOD-847] Add new line on releaseAddress
* [NOD-847] Always try to reconnect on disconnect
* [NOD-849] Cover ffldb/transaction with tests.
* [NOD-849] Cover cursor.go with tests.
* [NOD-849] Cover ldb/transaction with tests.
* [NOD-849] Cover location.go with tests.
* [NOD-849] Write TestFlatFileMultiFileRollback.
* [NOD-849] Fix merge errors.
* [NOD-849] Fix a comment.
* [NOD-849] Fix a comment.
* [NOD-849] Add a test that makes sure that files get deleted on rollback.
* [NOD-849] Add a test that makes sure that serializeLocation serialized to an expected value.
* [NOD-849] Improve TestFlatFileLocationDeserializationErrors.
* [NOD-849] Fix a copy+paste error.
* [NOD-849] Explain maxFileSize = 16.
* [NOD-849] Remove redundant RollbackUnlessClosed call.
* [NOD-849] Extract bucket to a variable in TestCursorSanity.
* [NOD-849] Rename TestKeyValueTransactionCommit to TestTransactionCommitForLevelDBMethods.
* [NOD-849] Extract prepareXXX into separate functions.
* [NOD-849] Simplify function calls in TestTransactionCloseErrors.
* [NOD-849] Extract validateCurrentCursorKeyAndValue to a separate function.
* [NOD-849] Add a comment over TestCursorSanity.
* [NOD-849] Add a comment over function in TestCursorCloseErrors.
* [NOD-849] Add a comment over function in TestTransactionCloseErrors.
* [NOD-849] Separate TestTransactionCloseErrors to TestTransactionCommitErrors and TestTransactionRollbackErrors.
* [NOD-849] Separate TestTransactionCloseErrors to TestTransactionCommitErrors and TestTransactionRollbackErrors.
* [NOD-849] Fix copy+paste error in comments.
* [NOD-849] Fix merge errors.
* [NOD-849] Merge TestTransactionCommitErrors and TestTransactionRollbackErrors into TestTransactionCloseErrors.
* [NOD-849] Move prepareDatabaseForTest into ffldb_test.go.
* [NOD-849] Add cursorKey to Value error messages in validateCurrentCursorKeyAndValue.
* [NOD-934] Fix addresses not getting their retry attempt counter incremented if they fail to connect.
* [NOD-922] Inline parseNetAddress.
* [NOD-922] Fix debug logs.
* [NOD-863] Write TestCursorNext.
* [NOD-863] Write TestCursorFirst.
* [NOD-863] Fix merge errors.
* [NOD-863] Add TestCursorSeek.
* [NOD-863] Add TestCursorCloseErrors.
* [NOD-863] Add TestCursorCloseFirstAndNext.
* [NOD-863] Add TestDataAccessorPut.
* [NOD-863] Add TestDataAccessorGet.
* [NOD-863] Add TestDataAccessorHas.
* [NOD-863] Add TestDatabaseDelete.
* [NOD-863] Add TestDatabaseAppendToStoreAndRetrieveFromStore.
* [NOD-863] Add TestTransactionAppendToStoreAndRetrieveFromStore.
* [NOD-863] Add TestTransactionDelete.
* [NOD-863] Add TestTransactionHas.
* [NOD-863] Add TestTransactionGet.
* [NOD-863] Add TestTransactionPut.
* [NOD-863] Move cursor tests to the bottom of interface_test.go.
* [NOD-863] Move interface_test.go to a database_test package.
* [NOD-863] Make each test in interface_test.go run for every database driver. Currently, only ffldb.
* [NOD-863] Make each cursor test in interface_test.go run for every database driver. Currently, only ffldb.
* [NOD-863] Split interface_test.go into separate files.
* [NOD-863] Rename interface_test.go to common_test.go.
* [NOD-863] Extract testForAllDatabaseTypes to a separate function.
* [NOD-863] Reorganize how test data gets added to the database.
* [NOD-863] Add explanations about testForAllDatabaseTypes.
* [NOD-863] Add tests that make sure that database changes don't affect previously opened transactions.
* [NOD-863] Extract databasePrepareFunc to a type alias.
* [NOD-863] Fix comments.
* [NOD-863] Add cursor exhaustion test to testCursorFirst.
* [NOD-863] Add cursor Next clause to testCursorSeek.
* [NOD-863] Add additional varification to testDatabasePut.
* [NOD-863] Add an additional verification into to testTransactionGet.
* [NOD-863] Add TestTransactionCommit.
* [NOD-863] Add TestTransactionRollback.
* [NOD-863] Add TestTransactionRollbackUnlessClosed.
* [NOD-863] Remove equals sign from databasePrepareFunc declaration.
* [NOD-858] Don't switch sync peer if the syncing process hasn't yet started with the current sync peer
* [NOD-858] SetShouldSendBlockLocator(false) on OnBlockLocator
* [NOD-858] Rename shouldSendBlockLocator->wasBlockLocatorRequested
* [NOD-858] Move panic to shouldReplaceSyncPeer
* [NOD-858] Don't switch sync peer if the syncing process hasn't yet started with the current sync peer
* [NOD-858] SetShouldSendBlockLocator(false) on OnBlockLocator
* [NOD-858] Rename shouldSendBlockLocator->wasBlockLocatorRequested
* [NOD-858] Move panic to shouldReplaceSyncPeer
* [NOD-909] Add tests for double spends
* [NOD-909] Add prepareAndProcessBlock that gets parent hashes and transactions as argument
* [NOD-909] Use PrepareAndProcessBlockForTest where possible
* [NOD-909] Use more meaningful names
* [NOD-909] Change a comment
* [NOD-909] Fix comment
* [NOD-909] Fix comment
* [NOD-899] Inside the database, in case we're out of disk space, panic without printing the stack trace.
* [NOD-899] Fix bad variable name.
* [NOD-899] Reduce code duplication.
* [NOD-900] Fix Seek not working at expected.
* [NOD-900] Wrap error messages.
* [NOD-900] Use ldbIterator.Key instead of LevelDBCursor.Key.
* [NOD-900] Add a comment.
* [NOD-828] Reimplement FFLDB (#663)
* [NOD-828] Create the database2 package that will some day replace the database package.
* [NOD-828] Implement a "bucket" key mechanism.
* [NOD-828] Move bucket.go into the ffldb2 package.
* [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway.
* [NOD-828] Copy over + fixup flat file structs from the old ffldb.
* [NOD-828] Implement flatFilePath.
* [NOD-828] Implement flatFileStore.write().
* [NOD-828] Implement flatFileStore.read().
* [NOD-828] Implement flatFileStore.rollback().
* [NOD-828] Sync the file to disk at the end of write().
* [NOD-828] Extract crc32ByteOrder to a separate variable.
* [NOD-828] Add a sanity test.
* [NOD-828] Remove context-unrelated methods from the Database interface.
* [NOD-828] Create an ffldb object. Simply work against a context.
* [NOD-828] Open the new database on start.
* [NOD-828] Create the leveldb package.
* [NOD-828] Implement opening/closing leveldb.
* [NOD-828] Implement get/put out of/into leveldb.
* [NOD-828] Implement transactions and make them implement a generic database interface.
* [NOD-828] Write sanity tests for leveldb with and without transactions.
* [NOD-828] Add another case to the transaction sanity test.
* [NOD-828] Implement AppendBlock/RetrieveBlock.
* [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package.
* [NOD-828] Add RollbackFlatData to the database interface.
* [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket.
* [NOD-828] Implement StoreBlock.
* [NOD-828] Implement FetchBlock.
* [NOD-828] Implement HasBlock.
* [NOD-828] Write a sanity test for block insertion.
* [NOD-828] Implement CurrentFlatDataLocation.
* [NOD-828] Implement storing the current block location.
* [NOD-828] Implement initializing/syncing the flat file block store and the "metadata".
* [NOD-828] Add InitBlockStore to TestBlockStoreSanity.
* [NOD-828] Fix rename errors.
* [NOD-828] Fix lint errors in the root database package.
* [NOD-828] Fix lint errors in the ffldb.go.
* [NOD-828] Fix lint errors in the flatfile/db.go.
* [NOD-828] Rename packages in such a way to make the linter happy.
* [NOD-828] Finish satisfying the linter.
* [NOD-828] Fix doc.go.
* [NOD-828] Fix comments in block.go.
* [NOD-828] Move dbaccess out of the database package.
* [NOD-828] Move opening/closing the database to dbaccess.
* [NOD-828] Move the Database interface to the root database package, since it's meant to be generic.
* [NOD-828] Make ffldb generic to later support additional databases.
* [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported.
* [NOD-828] Fix comments.
* [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore.
* [NOD-828] Make buckets nicer to use.
* [NOD-828] Implement cursors that iterate over some bucket.
* [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb.
* [NOD-828] Write a test making sure that flat file repair works.
* [NOD-828] Properly close the database in TestRepairFlatFiles.
* [NOD-828] Add a comment warning against putting and getting the same data within the same transaction.
* [NOD-828] Fix the flatFilesBucket description.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Fix the ffldb description.
* [NOD-828] Rename Database to DataAccessor and rename Handle to Database.
* [NOD-828] Make Get return nil if the value doesn't exist.
* [NOD-828] Attempt to close leveldb even if closing ffdb failed.
* [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket.
* [NOD-828] Fix not updating the store location in ffldb transactions.
* [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found.
* [NOD-828] Update the README and doc.go.
* [NOD-828] Simplify Bucket.Path().
* [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly.
* [NOD-828] Combine two lines into one.
* [NOD-828] Combine another two lines into one.
* [NOD-828] Move a misplaced comment.
* [NOD-828] Use Wrapf instead of Errorf where appropriate.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Fix Wrapf calls.
* [NOD-828] Fix comments.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Add a comment explaining the use of batches and snapshots.
* [NOD-828] Implement RollbackUnlessClosed().
* [NOD-828] Return both errors in StoreBlock rollback.
* [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor.
* [NOD-828] Make bucket a type alias instead of a struct.
* [NOD-828] Fix a typo.
* [NOD-828] Use copy instead of append in Bucket.
* [NOD-828] Extract flatFileLocationSerializedSize to a const.
* [NOD-828] Debugf -> Warnf in rollback.go.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Remove data length from flat file data format.
* [NOD-828] Rearrange TestLevelDBTransactionSanity a bit.
* [NOD-828] Add stack traces to all errors that come out of library functions.
* [NOD-828] Return errors from rollback().
* [NOD-828] Remove an irrelevant comment.
* [NOD-828] Remove redundant whitespace.
* [NOD-828] Handle nil in FetchBlock.
* [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct.
* Revert "[NOD-828] Make bucket a type alias instead of a struct."
This reverts commit 1fd39652
* [NOD-828] Fix revert errors.
* Revert "[NOD-828] Remove data length from flat file data format."
This reverts commit ef408e32
* [NOD-862] Move Cursor() into the DataAccessor interface.
* [NOD-828] Add Delete to DataAccessor.
* [NOD-828] Fix a couple of places that erroneously referenced blocks.
* [NOD-828] Add a comment on top of flatFileLocationSerializedSize.
* [NOD-828] Add Seek to Cursor.
* [NOD-828] Add First to Cursor.
* [NOD-828] Rename db to accessor in Context.
* [NOD-828] Make Get/Fetch calls return a boolean to indicate whether the requested item was found.
* [NOD-828] Name the output parameters of all Get functions.
* [NOD-828] Make RetrieveFromStore return whether the data was found.
* [NOD-887] Add a couple of QoL features to Cursor (#674)
* [NOD-887] Changed First to not return an error.
* [NOD-887] Fix merge error.
* [NOD-887] Make Cursor.Key not return the entire key path.
* [NOD-888] Add RollbackUnlessClosed to Context (#676)
* [NOD-888] Add RollbackUnlessClosed to Context.
* [NOD-888] Fix copy+paste error.
* [NOD-889] Instead of returning a boolean for not-found, return an error (#677)
* [NOD-889] Instead of returning a boolean for not-found, return an error.
* [NOD-889] Wrapped ErrNotFound for Get calls with nicer error messages.
* [NOD-889] Fix format.
* [NOD-889] Fix double space in a comment.
* [NOD-889] Add IsNotFoundError to dbaccess.
* [NOD-862] Replace calls to Tx.StoreBlock, Tx.HasBlock, Tx.FetchBlock with appropriate calls in dbaccess (#672)
* [NOD-828] Create the database2 package that will some day replace the database package.
* [NOD-828] Implement a "bucket" key mechanism.
* [NOD-828] Move bucket.go into the ffldb2 package.
* [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway.
* [NOD-828] Copy over + fixup flat file structs from the old ffldb.
* [NOD-828] Implement flatFilePath.
* [NOD-828] Implement flatFileStore.write().
* [NOD-828] Implement flatFileStore.read().
* [NOD-828] Implement flatFileStore.rollback().
* [NOD-828] Sync the file to disk at the end of write().
* [NOD-828] Extract crc32ByteOrder to a separate variable.
* [NOD-828] Add a sanity test.
* [NOD-828] Remove context-unrelated methods from the Database interface.
* [NOD-828] Create an ffldb object. Simply work against a context.
* [NOD-828] Open the new database on start.
* [NOD-828] Create the leveldb package.
* [NOD-828] Implement opening/closing leveldb.
* [NOD-828] Implement get/put out of/into leveldb.
* [NOD-828] Implement transactions and make them implement a generic database interface.
* [NOD-828] Write sanity tests for leveldb with and without transactions.
* [NOD-828] Add another case to the transaction sanity test.
* [NOD-828] Implement AppendBlock/RetrieveBlock.
* [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package.
* [NOD-828] Add RollbackFlatData to the database interface.
* [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket.
* [NOD-828] Implement StoreBlock.
* [NOD-828] Implement FetchBlock.
* [NOD-828] Implement HasBlock.
* [NOD-828] Write a sanity test for block insertion.
* [NOD-828] Implement CurrentFlatDataLocation.
* [NOD-828] Implement storing the current block location.
* [NOD-828] Implement initializing/syncing the flat file block store and the "metadata".
* [NOD-828] Add InitBlockStore to TestBlockStoreSanity.
* [NOD-828] Fix rename errors.
* [NOD-828] Fix lint errors in the root database package.
* [NOD-828] Fix lint errors in the ffldb.go.
* [NOD-828] Fix lint errors in the flatfile/db.go.
* [NOD-828] Rename packages in such a way to make the linter happy.
* [NOD-828] Finish satisfying the linter.
* [NOD-828] Fix doc.go.
* [NOD-828] Fix comments in block.go.
* [NOD-828] Move dbaccess out of the database package.
* [NOD-828] Move opening/closing the database to dbaccess.
* [NOD-828] Move the Database interface to the root database package, since it's meant to be generic.
* [NOD-828] Make ffldb generic to later support additional databases.
* [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported.
* [NOD-828] Fix comments.
* [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore.
* [NOD-828] Make buckets nicer to use.
* [NOD-828] Implement cursors that iterate over some bucket.
* [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb.
* [NOD-828] Write a test making sure that flat file repair works.
* [NOD-828] Properly close the database in TestRepairFlatFiles.
* [NOD-828] Add a comment warning against putting and getting the same data within the same transaction.
* [NOD-862] Use dbaccess.HasBlock instead of Tx.HasBlock in initDAGState.
* [NOD-862] Use dbaccess.StoreBlock instead of dbStoreBlock.
* [NOD-862] Use dbaccess.FetchBlock instead of various block fetching mechanisms.
* [NOD-828] Fix the flatFilesBucket description.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Fix the ffldb description.
* [NOD-828] Rename Database to DataAccessor and rename Handle to Database.
* [NOD-828] Make Get return nil if the value doesn't exist.
* [NOD-828] Attempt to close leveldb even if closing ffdb failed.
* [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket.
* [NOD-828] Fix not updating the store location in ffldb transactions.
* [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found.
* [NOD-828] Update the README and doc.go.
* [NOD-828] Simplify Bucket.Path().
* [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly.
* [NOD-828] Combine two lines into one.
* [NOD-828] Combine another two lines into one.
* [NOD-828] Move a misplaced comment.
* [NOD-828] Use Wrapf instead of Errorf where appropriate.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Fix Wrapf calls.
* [NOD-828] Fix comments.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Add a comment explaining the use of batches and snapshots.
* [NOD-828] Implement RollbackUnlessClosed().
* [NOD-828] Return both errors in StoreBlock rollback.
* [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor.
* [NOD-828] Make bucket a type alias instead of a struct.
* [NOD-828] Fix a typo.
* [NOD-828] Use copy instead of append in Bucket.
* [NOD-828] Extract flatFileLocationSerializedSize to a const.
* [NOD-828] Debugf -> Warnf in rollback.go.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Remove data length from flat file data format.
* [NOD-828] Rearrange TestLevelDBTransactionSanity a bit.
* [NOD-828] Add stack traces to all errors that come out of library functions.
* [NOD-828] Return errors from rollback().
* [NOD-828] Remove an irrelevant comment.
* [NOD-828] Remove redundant whitespace.
* [NOD-828] Handle nil in FetchBlock.
* [NOD-828] Implement a dbaccess.BlockNode struct.
* [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct.
* [NOD-828] Implement toDBBlockNode and fromDBBlockNode.
* Revert "[NOD-828] Make bucket a type alias instead of a struct."
This reverts commit 1fd39652
* [NOD-828] Fix revert errors.
* Revert "[NOD-828] Remove data length from flat file data format."
This reverts commit ef408e32
* [NOD-862] Implement storing index blocks.
* [NOD-862] Use database transactions where appropriate.
* [NOD-862] Fix tests failing on DAGSetup.
* [NOD-862] Fix bad make call.
* [NOD-862] Fix remaining database opening problems in tests.
* [NOD-862] Move Cursor() into the DataAccessor interface.
* [NOD-862] Move Cursor() into the DataAccessor interface.
* [NOD-862] Iterate over the new block index in dagio.
* [NOD-862] Fix block index key.
* [NOD-828] Add Delete to DataAccessor.
* [NOD-862] Pass byte slices to dbaccess instead of objects.
* [NOD-862] Fix errors.
* [NOD-862] Fix maybeAcceptBlock not checking block existence.
* [NOD-862] Fix TestAcceptanceIndexRecover.
* [NOD-862] Add comments to StoreBlockIndex and BlockIndexCursor.
* [NOD-828] Fix a couple of places that erroneously referenced blocks.
* [NOD-828] Add a comment on top of flatFileLocationSerializedSize.
* [NOD-828] Add Seek to Cursor.
* [NOD-828] Add First to Cursor.
* [NOD-828] Rename db to accessor in Context.
* [NOD-828] Make Get/Fetch calls return a boolean to indicate whether the requested item was found.
* [NOD-828] Name the output parameters of all Get functions.
* [NOD-828] Make RetrieveFromStore return whether the data was found.
* [NOD-862] Fix merge errors.
* [NOD-862] Fix DAGSetup using bad temp directories.
* [NOD-862] Fix TestProcessDelayedBlocks not closing the database properly.
* [NOD-862] Fix merge errors.
* [NOD-862] Merge flushToDBWithContext and flushToDB.
* [NOD-862] Remove TODO.
* [NOD-862] Add prefix to the temp dir in DAGSetup.
* [NOD-862] Bring back dbFetchBlockByHash.
* [NOD-862] Use BlockDAG.BlockByHash in p2p and rpc.
* [NOD-862] Use daghash.Hash in dbaccess.
* [NOD-862] Add defer to RollbackUnlessClosed after NewTx().
* [NOD-862] Extract dbStoreBlock to a separate function.
* [NOD-862] Fix grammar in comment.
* [NOD-862] Fix merge errors.
* [NOD-867] Migrate database logic in blockdag/dagio.go to dbaccess (#675)
* [NOD-828] Create the database2 package that will some day replace the database package.
* [NOD-828] Implement a "bucket" key mechanism.
* [NOD-828] Move bucket.go into the ffldb2 package.
* [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway.
* [NOD-828] Copy over + fixup flat file structs from the old ffldb.
* [NOD-828] Implement flatFilePath.
* [NOD-828] Implement flatFileStore.write().
* [NOD-828] Implement flatFileStore.read().
* [NOD-828] Implement flatFileStore.rollback().
* [NOD-828] Sync the file to disk at the end of write().
* [NOD-828] Extract crc32ByteOrder to a separate variable.
* [NOD-828] Add a sanity test.
* [NOD-828] Remove context-unrelated methods from the Database interface.
* [NOD-828] Create an ffldb object. Simply work against a context.
* [NOD-828] Open the new database on start.
* [NOD-828] Create the leveldb package.
* [NOD-828] Implement opening/closing leveldb.
* [NOD-828] Implement get/put out of/into leveldb.
* [NOD-828] Implement transactions and make them implement a generic database interface.
* [NOD-828] Write sanity tests for leveldb with and without transactions.
* [NOD-828] Add another case to the transaction sanity test.
* [NOD-828] Implement AppendBlock/RetrieveBlock.
* [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package.
* [NOD-828] Add RollbackFlatData to the database interface.
* [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket.
* [NOD-828] Implement StoreBlock.
* [NOD-828] Implement FetchBlock.
* [NOD-828] Implement HasBlock.
* [NOD-828] Write a sanity test for block insertion.
* [NOD-828] Implement CurrentFlatDataLocation.
* [NOD-828] Implement storing the current block location.
* [NOD-828] Implement initializing/syncing the flat file block store and the "metadata".
* [NOD-828] Add InitBlockStore to TestBlockStoreSanity.
* [NOD-828] Fix rename errors.
* [NOD-828] Fix lint errors in the root database package.
* [NOD-828] Fix lint errors in the ffldb.go.
* [NOD-828] Fix lint errors in the flatfile/db.go.
* [NOD-828] Rename packages in such a way to make the linter happy.
* [NOD-828] Finish satisfying the linter.
* [NOD-828] Fix doc.go.
* [NOD-828] Fix comments in block.go.
* [NOD-828] Move dbaccess out of the database package.
* [NOD-828] Move opening/closing the database to dbaccess.
* [NOD-828] Move the Database interface to the root database package, since it's meant to be generic.
* [NOD-828] Make ffldb generic to later support additional databases.
* [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported.
* [NOD-828] Fix comments.
* [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore.
* [NOD-828] Make buckets nicer to use.
* [NOD-828] Implement cursors that iterate over some bucket.
* [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb.
* [NOD-828] Write a test making sure that flat file repair works.
* [NOD-828] Properly close the database in TestRepairFlatFiles.
* [NOD-828] Add a comment warning against putting and getting the same data within the same transaction.
* [NOD-862] Use dbaccess.HasBlock instead of Tx.HasBlock in initDAGState.
* [NOD-862] Use dbaccess.StoreBlock instead of dbStoreBlock.
* [NOD-862] Use dbaccess.FetchBlock instead of various block fetching mechanisms.
* [NOD-828] Fix the flatFilesBucket description.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Fix the ffldb description.
* [NOD-828] Rename Database to DataAccessor and rename Handle to Database.
* [NOD-828] Make Get return nil if the value doesn't exist.
* [NOD-828] Attempt to close leveldb even if closing ffdb failed.
* [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket.
* [NOD-828] Fix not updating the store location in ffldb transactions.
* [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found.
* [NOD-828] Update the README and doc.go.
* [NOD-828] Simplify Bucket.Path().
* [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly.
* [NOD-828] Combine two lines into one.
* [NOD-828] Combine another two lines into one.
* [NOD-828] Move a misplaced comment.
* [NOD-828] Use Wrapf instead of Errorf where appropriate.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Fix Wrapf calls.
* [NOD-828] Fix comments.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Add a comment explaining the use of batches and snapshots.
* [NOD-828] Implement RollbackUnlessClosed().
* [NOD-828] Return both errors in StoreBlock rollback.
* [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor.
* [NOD-828] Make bucket a type alias instead of a struct.
* [NOD-828] Fix a typo.
* [NOD-828] Use copy instead of append in Bucket.
* [NOD-828] Extract flatFileLocationSerializedSize to a const.
* [NOD-828] Debugf -> Warnf in rollback.go.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Remove data length from flat file data format.
* [NOD-828] Rearrange TestLevelDBTransactionSanity a bit.
* [NOD-828] Add stack traces to all errors that come out of library functions.
* [NOD-828] Return errors from rollback().
* [NOD-828] Remove an irrelevant comment.
* [NOD-828] Remove redundant whitespace.
* [NOD-828] Handle nil in FetchBlock.
* [NOD-828] Implement a dbaccess.BlockNode struct.
* [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct.
* [NOD-828] Implement toDBBlockNode and fromDBBlockNode.
* Revert "[NOD-828] Make bucket a type alias instead of a struct."
This reverts commit 1fd39652
* [NOD-828] Fix revert errors.
* Revert "[NOD-828] Remove data length from flat file data format."
This reverts commit ef408e32
* [NOD-862] Implement storing index blocks.
* [NOD-862] Use database transactions where appropriate.
* [NOD-862] Fix tests failing on DAGSetup.
* [NOD-862] Fix bad make call.
* [NOD-862] Fix remaining database opening problems in tests.
* [NOD-862] Move Cursor() into the DataAccessor interface.
* [NOD-862] Move Cursor() into the DataAccessor interface.
* [NOD-862] Iterate over the new block index in dagio.
* [NOD-862] Fix block index key.
* [NOD-828] Add Delete to DataAccessor.
* [NOD-862] Pass byte slices to dbaccess instead of objects.
* [NOD-862] Fix errors.
* [NOD-862] Fix maybeAcceptBlock not checking block existence.
* [NOD-862] Fix TestAcceptanceIndexRecover.
* [NOD-862] Add comments to StoreBlockIndex and BlockIndexCursor.
* [NOD-828] Fix a couple of places that erroneously referenced blocks.
* [NOD-828] Add a comment on top of flatFileLocationSerializedSize.
* [NOD-828] Add Seek to Cursor.
* [NOD-828] Add First to Cursor.
* [NOD-828] Rename db to accessor in Context.
* [NOD-828] Make Get/Fetch calls return a boolean to indicate whether the requested item was found.
* [NOD-828] Name the output parameters of all Get functions.
* [NOD-828] Make RetrieveFromStore return whether the data was found.
* [NOD-862] Fix merge errors.
* [NOD-862] Fix DAGSetup using bad temp directories.
* [NOD-862] Fix TestProcessDelayedBlocks not closing the database properly.
* [NOD-867] Remove blockIndexBucket from dagio.
* [NOD-867] Fix wrong key in StoreIndexBucket.
* [NOD-867] Migrate DAG state to dbaccess.
* [NOD-867] Remove utxoSetVersionKeyName.
* [NOD-862] Fix merge errors.
* [NOD-867] Move localSubnetworkID into dagState.
* [NOD-867] Fix a comment.
* [NOD-867] Remove an unused function.
* [NOD-867] Migrate the database's UTXO set to dbaccess.
* [NOD-867] Add missing error check.
* [NOD-867] Changed First to not return an error.
* [NOD-867] Make Cursor.Key not return the entire key path.
* [NOD-887] Fix the comment above BlockIndexCursorFrom.
* [NOD-862] Merge flushToDBWithContext and flushToDB.
* [NOD-862] Remove TODO.
* [NOD-862] Add prefix to the temp dir in DAGSetup.
* [NOD-862] Bring back dbFetchBlockByHash.
* [NOD-862] Use BlockDAG.BlockByHash in p2p and rpc.
* [NOD-862] Use daghash.Hash in dbaccess.
* [NOD-862] Add defer to RollbackUnlessClosed after NewTx().
* [NOD-862] Extract dbStoreBlock to a separate function.
* [NOD-867] Remove TODOs.
* [NOD-867] Fix merge errors.
* [NOD-867] Fix comments and errors.
* [NOD-867] Unexport blockIndexKey.
* [NOD-867] Fix merge errors.
* [NOD-867] Move a misplaced comment.
* [NOD-867] Fix an error message.
* [NOD-867] Remove preallocation in initDAGState.
* [NOD-866] Migrate database logic in blockdag/indexers package to dbaccess (#682)
* [NOD-865] Delete blockidhash.go.
* [NOD-865] Remove a lot of no-longer relevant logic from indexers.
* [NOD-865] Pass TxContext to ConnectBlock.
* [NOD-865] Migrate the acceptance index to dbaccess.
* [NOD-865] Fix a block not being sent to ConnectBlock.
* [NOD-865] Pass the block's hash instead of the whole block.
* [NOD-865] Add forgotten Commit call.
* [NOD-865] Add comments.
* [NOD-866] Fix a comment.
* [NOD-866] Fix a comment.
* [NOD-866] Remove pointless indirection in acceptanceindex.
* [NOD-866] Fix comment over ForEachHash.
* [NOD-866] Rename ClearAcceptanceIndex to DropAcceptanceIndex.
* [NOD-866] Explain collecting keys before deleting them.
* [NOD-865] Move misc db logic to db access (#681)
* [NOD-828] Create the database2 package that will some day replace the database package.
* [NOD-828] Implement a "bucket" key mechanism.
* [NOD-828] Move bucket.go into the ffldb2 package.
* [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway.
* [NOD-828] Copy over + fixup flat file structs from the old ffldb.
* [NOD-828] Implement flatFilePath.
* [NOD-828] Implement flatFileStore.write().
* [NOD-828] Implement flatFileStore.read().
* [NOD-828] Implement flatFileStore.rollback().
* [NOD-828] Sync the file to disk at the end of write().
* [NOD-828] Extract crc32ByteOrder to a separate variable.
* [NOD-828] Add a sanity test.
* [NOD-828] Remove context-unrelated methods from the Database interface.
* [NOD-828] Create an ffldb object. Simply work against a context.
* [NOD-828] Open the new database on start.
* [NOD-828] Create the leveldb package.
* [NOD-828] Implement opening/closing leveldb.
* [NOD-828] Implement get/put out of/into leveldb.
* [NOD-828] Implement transactions and make them implement a generic database interface.
* [NOD-828] Write sanity tests for leveldb with and without transactions.
* [NOD-828] Add another case to the transaction sanity test.
* [NOD-828] Implement AppendBlock/RetrieveBlock.
* [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package.
* [NOD-828] Add RollbackFlatData to the database interface.
* [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket.
* [NOD-828] Implement StoreBlock.
* [NOD-828] Implement FetchBlock.
* [NOD-828] Implement HasBlock.
* [NOD-828] Write a sanity test for block insertion.
* [NOD-828] Implement CurrentFlatDataLocation.
* [NOD-828] Implement storing the current block location.
* [NOD-828] Implement initializing/syncing the flat file block store and the "metadata".
* [NOD-828] Add InitBlockStore to TestBlockStoreSanity.
* [NOD-828] Fix rename errors.
* [NOD-828] Fix lint errors in the root database package.
* [NOD-828] Fix lint errors in the ffldb.go.
* [NOD-828] Fix lint errors in the flatfile/db.go.
* [NOD-828] Rename packages in such a way to make the linter happy.
* [NOD-828] Finish satisfying the linter.
* [NOD-828] Fix doc.go.
* [NOD-828] Fix comments in block.go.
* [NOD-828] Move dbaccess out of the database package.
* [NOD-828] Move opening/closing the database to dbaccess.
* [NOD-828] Move the Database interface to the root database package, since it's meant to be generic.
* [NOD-828] Make ffldb generic to later support additional databases.
* [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported.
* [NOD-828] Fix comments.
* [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore.
* [NOD-828] Make buckets nicer to use.
* [NOD-828] Implement cursors that iterate over some bucket.
* [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb.
* [NOD-828] Write a test making sure that flat file repair works.
* [NOD-828] Properly close the database in TestRepairFlatFiles.
* [NOD-828] Add a comment warning against putting and getting the same data within the same transaction.
* [NOD-828] Fix the flatFilesBucket description.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Fix the ffldb description.
* [NOD-828] Rename Database to DataAccessor and rename Handle to Database.
* [NOD-828] Make Get return nil if the value doesn't exist.
* [NOD-828] Attempt to close leveldb even if closing ffdb failed.
* [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket.
* [NOD-828] Fix not updating the store location in ffldb transactions.
* [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found.
* [NOD-828] Update the README and doc.go.
* [NOD-828] Simplify Bucket.Path().
* [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly.
* [NOD-828] Combine two lines into one.
* [NOD-828] Combine another two lines into one.
* [NOD-828] Move a misplaced comment.
* [NOD-828] Use Wrapf instead of Errorf where appropriate.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Fix Wrapf calls.
* [NOD-828] Fix comments.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Add a comment explaining the use of batches and snapshots.
* [NOD-828] Implement RollbackUnlessClosed().
* [NOD-828] Return both errors in StoreBlock rollback.
* [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor.
* [NOD-828] Make bucket a type alias instead of a struct.
* [NOD-828] Fix a typo.
* [NOD-828] Use copy instead of append in Bucket.
* [NOD-828] Extract flatFileLocationSerializedSize to a const.
* [NOD-828] Debugf -> Warnf in rollback.go.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Remove data length from flat file data format.
* [NOD-828] Rearrange TestLevelDBTransactionSanity a bit.
* [NOD-828] Add stack traces to all errors that come out of library functions.
* [NOD-828] Return errors from rollback().
* [NOD-828] Remove an irrelevant comment.
* [NOD-828] Remove redundant whitespace.
* [NOD-828] Handle nil in FetchBlock.
* [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct.
* Revert "[NOD-828] Make bucket a type alias instead of a struct."
This reverts commit 1fd39652
* [NOD-828] Fix revert errors.
* Revert "[NOD-828] Remove data length from flat file data format."
This reverts commit ef408e32
* [NOD-862] Move Cursor() into the DataAccessor interface.
* [NOD-828] Add Delete to DataAccessor.
* [NOD-865] Move fee data db operations to dbaccess
* [NOD-865] Move reachability data db operations to dbaccess
* [NOD-865] Move UTXO diff data db operations to dbaccess
* [NOD-865] Move subnetwork data db operations to dbaccess
* [NOD-865] Fix createDAGState
* [NOD-865] Remove old Get signature with "exists"
* [NOD-865] Move multiset db operations to dbaccess
* [NOD-865] Use dbaccess transactions where possible
* [NOD-865] Remove old Get signature with "exists"
* [NOD-881] Recover TestGHOSTDAGErrors
* [NOD-865] Create function for db keys
* [NOD-865] Change Exists to Has, and use accessor.Has where possible
* [NOD-865] Make ClearReachabilityData transactive
* [NOD-865] Don't iterate cursors while changing db data
* [NOD-865] Rename RegisterSubnetwork -> StoreSubnetwork
* [NOD-865] Change bucket from utxodiffs to utxo-diffs
* [NOD-865] Rename SubnetworkExists->HasSubnetwork
* [NOD-865] Change a comment
* [NOD-865] Fix tests
* [NOD-865] Fix comment
* [NOD-865] Remove the prefix "db" from some functions
* [NOD-865] Remove redundant comments
* [NOD-865] Make clearBucket function
* [NOD-865] Make clear functions get a dbTx as an arg
* [NOD-865] Remove erroneous tx commit
Co-authored-by: stasatdaglabs <stas@daglabs.com>
* [NOD-868] Delete the old database package (#683)
* [NOD-828] Create the database2 package that will some day replace the database package.
* [NOD-828] Implement a "bucket" key mechanism.
* [NOD-828] Move bucket.go into the ffldb2 package.
* [NOD-828] Delete the un-interfaced ffldb package from database2, since we aren't going to be using it anyway.
* [NOD-828] Copy over + fixup flat file structs from the old ffldb.
* [NOD-828] Implement flatFilePath.
* [NOD-828] Implement flatFileStore.write().
* [NOD-828] Implement flatFileStore.read().
* [NOD-828] Implement flatFileStore.rollback().
* [NOD-828] Sync the file to disk at the end of write().
* [NOD-828] Extract crc32ByteOrder to a separate variable.
* [NOD-828] Add a sanity test.
* [NOD-828] Remove context-unrelated methods from the Database interface.
* [NOD-828] Create an ffldb object. Simply work against a context.
* [NOD-828] Open the new database on start.
* [NOD-828] Create the leveldb package.
* [NOD-828] Implement opening/closing leveldb.
* [NOD-828] Implement get/put out of/into leveldb.
* [NOD-828] Implement transactions and make them implement a generic database interface.
* [NOD-828] Write sanity tests for leveldb with and without transactions.
* [NOD-828] Add another case to the transaction sanity test.
* [NOD-828] Implement AppendBlock/RetrieveBlock.
* [NOD-828] Refactor so that concepts such as "block" and "metadata" don't leak into the database package.
* [NOD-828] Add RollbackFlatData to the database interface.
* [NOD-828] Remove anything from dbaccess that I'm not planning to implement as part of this ticket.
* [NOD-828] Implement StoreBlock.
* [NOD-828] Implement FetchBlock.
* [NOD-828] Implement HasBlock.
* [NOD-828] Write a sanity test for block insertion.
* [NOD-828] Implement CurrentFlatDataLocation.
* [NOD-828] Implement storing the current block location.
* [NOD-828] Implement initializing/syncing the flat file block store and the "metadata".
* [NOD-828] Add InitBlockStore to TestBlockStoreSanity.
* [NOD-828] Fix rename errors.
* [NOD-828] Fix lint errors in the root database package.
* [NOD-828] Fix lint errors in the ffldb.go.
* [NOD-828] Fix lint errors in the flatfile/db.go.
* [NOD-828] Rename packages in such a way to make the linter happy.
* [NOD-828] Finish satisfying the linter.
* [NOD-828] Fix doc.go.
* [NOD-828] Fix comments in block.go.
* [NOD-828] Move dbaccess out of the database package.
* [NOD-828] Move opening/closing the database to dbaccess.
* [NOD-828] Move the Database interface to the root database package, since it's meant to be generic.
* [NOD-828] Make ffldb generic to later support additional databases.
* [NOD-828] Make ffldb.Open return DatabaseHandle, since ffldb is no longer exported.
* [NOD-828] Fix comments.
* [NOD-828] Rename AppendFlatData to AppendToStore and RetrieveFlatData to RetrieveFromStore.
* [NOD-828] Make buckets nicer to use.
* [NOD-828] Implement cursors that iterate over some bucket.
* [NOD-828] Generalize flat-file repairing and move block database repairing into ffldb.
* [NOD-828] Write a test making sure that flat file repair works.
* [NOD-828] Properly close the database in TestRepairFlatFiles.
* [NOD-828] Add a comment warning against putting and getting the same data within the same transaction.
* [NOD-828] Fix the flatFilesBucket description.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Fix the ffldb description.
* [NOD-828] Rename Database to DataAccessor and rename Handle to Database.
* [NOD-828] Make Get return nil if the value doesn't exist.
* [NOD-828] Attempt to close leveldb even if closing ffdb failed.
* [NOD-828] Fix a bug where the wrong location would be written to the current store location bucket.
* [NOD-828] Fix not updating the store location in ffldb transactions.
* [NOD-828] Make scanFlatFiles return an error if os.Stat fails for any reason other than file-not-found.
* [NOD-828] Update the README and doc.go.
* [NOD-828] Simplify Bucket.Path().
* [NOD-828] Since LevelDBCursor satisfied the database2.Cursor interface, use it directly.
* [NOD-828] Combine two lines into one.
* [NOD-828] Combine another two lines into one.
* [NOD-828] Move a misplaced comment.
* [NOD-828] Use Wrapf instead of Errorf where appropriate.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Fix Wrapf calls.
* [NOD-828] Fix comments.
* [NOD-828] Remove superfluous whitespace.
* [NOD-828] Add a comment explaining the use of batches and snapshots.
* [NOD-828] Implement RollbackUnlessClosed().
* [NOD-828] Return both errors in StoreBlock rollback.
* [NOD-828] Move rollback-on-error logic into ffldb. Remove CurrentStoreLocation and RollbackStore from DataAccessor.
* [NOD-828] Make bucket a type alias instead of a struct.
* [NOD-828] Fix a typo.
* [NOD-828] Use copy instead of append in Bucket.
* [NOD-828] Extract flatFileLocationSerializedSize to a const.
* [NOD-828] Debugf -> Warnf in rollback.go.
* [NOD-828] Fix a comment.
* [NOD-828] Fix a comment.
* [NOD-828] Remove data length from flat file data format.
* [NOD-828] Rearrange TestLevelDBTransactionSanity a bit.
* [NOD-828] Add stack traces to all errors that come out of library functions.
* [NOD-828] Return errors from rollback().
* [NOD-828] Remove an irrelevant comment.
* [NOD-828] Remove redundant whitespace.
* [NOD-828] Handle nil in FetchBlock.
* [NOD-828] Move the explanation about batches and snapshots to the LevelDBTransaction struct.
* Revert "[NOD-828] Make bucket a type alias instead of a struct."
This reverts commit 1fd39652
* [NOD-828] Fix revert errors.
* Revert "[NOD-828] Remove data length from flat file data format."
This reverts commit ef408e32
* [NOD-862] Move Cursor() into the DataAccessor interface.
* [NOD-828] Add Delete to DataAccessor.
* [NOD-865] Move fee data db operations to dbaccess
* [NOD-865] Move reachability data db operations to dbaccess
* [NOD-865] Move UTXO diff data db operations to dbaccess
* [NOD-865] Move subnetwork data db operations to dbaccess
* [NOD-865] Fix createDAGState
* [NOD-865] Remove old Get signature with "exists"
* [NOD-865] Move multiset db operations to dbaccess
* [NOD-865] Use dbaccess transactions where possible
* [NOD-865] Remove old Get signature with "exists"
* [NOD-881] Recover TestGHOSTDAGErrors
* [NOD-865] Create function for db keys
* [NOD-865] Change Exists to Has, and use accessor.Has where possible
* [NOD-865] Make ClearReachabilityData transactive
* [NOD-865] Don't iterate cursors while changing db data
* [NOD-865] Rename RegisterSubnetwork -> StoreSubnetwork
* [NOD-865] Change bucket from utxodiffs to utxo-diffs
* [NOD-865] Rename SubnetworkExists->HasSubnetwork
* [NOD-865] Change a comment
* [NOD-868] Remove all tests from old database.
* [NOD-868] Remove all unused methods from the old database's interfaces.
* [NOD-865] Fix tests
* [NOD-868] Remove references to DB.
* [NOD-865] Fix comment
* [NOD-868] Remove the old ffldb besides the interface and errors.go.
* [NOD-868] Remove errors.go.
* [NOD-868] Remove the old database package.
* [NOD-868] Add openDB to DAGSetup to emulate the old dbpath in dag.config.
* [NOD-868] Rename database2 to database.
* [NOD-868] Use NewTx instead of NoTx where required.
* [NOD-868] Fix merge errors.
* [NOD-868] Rename dbXXX functions to just xxx.
* [NOD-868] Rename putDAGState to saveDAGState.
* [NOD-868] Replace comments in initDAGState with logs.
* [NOD-868] Explain the openDB parameter in DAGSetup.
* [NOD-868] Fixup doc.go and README.md.
* [NOD-868] Remove pointless transactions.
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* [NOD-805] Fix merge errors.
* [NOD-805] Fix a comment.
* [NOD-805] Don't return virtualTxsAcceptanceData from applyDAGChanges.
* [NOD-805] Add missing error handling in TestAcceptanceDataIndexRecover.
* [NOD-805] Rename blockDAG to dag in indexers/manager.go.
* [NOD-805] Defer cursor.Close() everywhere.
* [NOD-805] Rename scanFlatFiles to findCurrentLocation.
* [NOD-805] Extract crc32ChecksumLength and dataLengthLength to constants.
* [NOD-805] Handle open files properly in rollback.go.
* [NOD-805] Remove unnecessary func wrapper.
* [NOD-805] Remove unnecessary trimming in initialize.
* [NOD-805] Made StoreBlock accept only TxContext.
* [NOD-805] Changed the log level of an error message to Error.
* [NOD-805] Add a note about holding mutexes over deleteFile.
* [NOD-805] Remove a false comment.
* [NOD-805] Fix a comment.
* [NOD-805] Rename blk to block.
* [NOD-805] Extract utxoKey to a separate function.
* [NOD-805] Move dbaccess.xxxKey functions to the tops of their respective files.
* [NOD-805] Fix grammar in dbaccess/db.go.
* [NOD-805] Wrap a failed database corruption recovery error.
* [NOD-805] Split lines with WithStack in them.
* [NOD-805] Fix the comment over initialize.
* [NOD-805] Rename ffdb to flatFileDB and ldb to levelDB.
* [NOD-805] Fix a comment.
* [NOD-805] Fix a comment.
* [NOD-805] Use s.writeCursor instead of cursor.
* [NOD-805] Embed file in lockableFile.
* [NOD-805] the the -> the
* [NOD-805] openDB -> db
* [NOD-805] Use TxContext in all flushToDB functions.
* [NOD-805] Rename context -> dbContext.
* [NOD-805] Reword the comment at the beginning on initDAGState.
* [NOD-805] Explain cursor key trimming.
* [NOD-805] Remove Error from Cursor.
* [NOD-805] Return ErrNotFound from done Cursor Key and Value.
* [NOD-805] Add missing error handling.
* [NOD-805] Fix a comment.
* [NOD-805] Fix a variable name.
* [NOD-805] Remove pointless underscore.
* [NOD-805] Fix a comment.
* [NOD-805] Fix a variable name.
Co-authored-by: Mike Zak <feanorr@gmail.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* [NOD-874] If the node is not current call sm.restartSyncIfNeeded() on handleInvMsg
* [NOD-874] Check haveUnknownInvBlock before restartSyncIfNeeded
* [NOD-874] Fix comment
* [NOD-874] Fix comment
* [NOD-874] Fix comment
* [NOD-881] Don't recalculate subtreesize for children
* [NOD-881] Make BenchmarkReindexInterval clearer
* [NOD-881] Use b.ResetTimer
* [NOD-881] Fix BenchmarkReindexInterval to use b.N
* [NOD-855] Save ECMH for each block UTXO
* [NOD-855] Remove UpdateExtraNonce method
* [NOD-855] Remove multiset data from UTXO diffs
* [NOD-855] Fix to fetch multiset of selected parent
* [NOD-855] Don't remove coinbase inputs from multiset
* [NOD-855] Create multisetBucketName on startup
* [NOD-855] Remove multiset from UTXO diff tests
* [NOD-855] clear new entries from multisetstore on saveChangesFromBlock
* [NOD-855] Fix tests
* [NOD-855] Use UnacceptedBlueScore when adding current block transactions to multiset
* [NOD-855] Hash utxo before adding it to multiset
* [NOD-855] Pass isCoinbase to NewUTXOEntry
* [NOD-855] Do not use hash when adding entries to multiset
* [NOD-855] When calculating multiset, replace the unaccepted blue score of selected parent transaction with the block blue score
* [NOD-855] Manually add a chained transaction to a block in TestChainedTransactions
* [NOD-855] Change name and comments
* [NOD-855] Use FindAcceptanceData to find a specific block acceptance data
* [NOD-855] Remove redundant copy of txIn.PreviousOutpoint
* [NOD-855] Use fmt.Sprintf when creating internalRPCError
* [NOD-848] Optimize allocations when serializing UTXO diffs
* [NOD-848] Use same UTXO serialization everywhere, and use compression as well
* [NOD-848] Fix usage of wrong buffer
* [NOD-848] Fix tests
* [NOD-848] Fix wire tests
* [NOD-848] Fix tests
* [NOD-848] Remove VLQ
* [NOD-848] Fix comments
* [NOD-848] Add varint for big endian encoding
* [NOD-848] In TestVarIntWire, assume the expected decoded value is the same as the serialization input
* [NOD-848] Serialize outpoint index with big endian varint
* [NOD-848] Remove p2pk from compression support
* [NOD-848] Fix comments
* [NOD-848] Remove p2pk from decompression support
* [NOD-848] Make entry compression optional
* [NOD-848] Fix tests
* [NOD-848] Fix comments and var names
* [NOD-848] Remove UTXO compression
* [NOD-848] Fix tests
* [NOD-848] Remove big endian varint
* [NOD-848] Fix comments
* [NOD-848] Rename ReadVarIntLittleEndian->ReadVarInt and fix WriteVarInt comment
* [NOD-848] Add outpointIndexByteOrder variable
* [NOD-848] Remove redundant comment
* [NOD-848] Fix outpointMaxSerializeSize to the correct value
* [NOD-848] Move subBuffer to utils
* [NOD-778] Add WithDiffInPlace
* [NOD-778] Fix bug in WithDiffInPlace
* [NOD-778] Add comment to WithDiffInPlace
* [NOD-778] Add double dag.restoreUTXO to benchmark, to remove time for hard-disk loading
* [NOD-778] Also test WithDiffInPlace in TestUTXODiffRules
* [NOD-778] Add tests for all cases possible in TestUTXODiffRules
* [NOD-778] Fix test-case 'first in toAdd in this, second in toRemove in this and toAdd in other'
* [NOD-778] Fixed in WithDiffInPlace
* [NOD-778] Update error messages when diffFrom(withDiffResult) fails in TestUTXODiffRules
* [NOD-778] diffFrom: disallow utxos both in d.toAdd, other.toAdd, and only one of d.toRemove and other.toRemove
* [NOD-778] Fix expected value in 'first in toRemove in this, second in toRemove in other'
* [NOD-778] diffFrom: Disallow situations where utxo both in d.toRemove and other.toRemove with different blue scores and no corresponding utxo in d.toAdd
* [NOD-778] WithDiff: Fix faulty logic that allows updates to blue scores
* [NOD-778] Fix WithDiffInPlace to pass all tests
* [NOD-778] Deleted temporary prints
* [NOD-778] Sorted TestUTXODiffRules tests according to spreadsheet
* [NOD-778] Delete deeputxo_test.go
* [NOD-778] Updated comments
* [NOD-778] Re-order
* [NOD-778] Re-order test-cases to be according to spreadsheet
* [NOD-778] Simplified case when both d.toRemove and other.toRemove have the same outpoint in diffFrom
* [NOD-778] Change a few error messages that say 'transaction' instead of 'outpoint'
* [NOD-778] Rename: utxoToAdd/Remove -> entryToAdd/Remove
* [NOD-788] Remove redundant else
* [NOD-778] Rename: existingUTXO -> existingEntry + remove redundant else
* [NOD-778] Correct test name
* [NOD-806] After panic, gracefully stop logs, and then exit immediately
* [NOD-806] Convert non-kaspad applications to use the new spawn
* [NOD-806] Fix disabled log at rpcclient
* [NOD-806] Refactor HandlePanic
* [NOD-806] Cancel Logger interface
* [NOD-806] Remove redundant spawn checks from waitgroup_test.go
* [NOD-806] Use caller subsystem when logging panics
* [NOD-806] Fix go vet errors
* [NOD-552] Add NormalizeRPCServerAddress and use it where needed
* [NOD-552] Make NormalizeAddress return an error for an invalid address
* [NOD-552] Use longer lines for a comment
* [NOD-656] Log hashrate in kaspaminer
* [NOD-656] Measure hash rate in kilohashes
* [NOD-656] Show hash rate once in 10 seconds
* [NOD-656] Put hash rate logic in a separate function
* [NOD-656] Create logHashRateInterval constant
* [NOD-719] Added defers to unlocks (#618)
* [NOD-719] Added defers to unlocks
* [NOD-719] Added another defer to another Unlock
* [NOD-719] Added yet another defer to yet another Unlock
* [NOD-747] Change FinalityInterval to be 24 hours, isCurrent to be true if the DAG's time is less than 12 hours than the present, and change MaxInvPerMsg to be 1 << 17 (#625)
* [NOD-769] Add a log for when a reachability reindex occurs.
Co-authored-by: Svarog <feanorr@gmail.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
* [NOD-719] Added defers to unlocks (#618)
* [NOD-719] Added defers to unlocks
* [NOD-719] Added another defer to another Unlock
* [NOD-719] Added yet another defer to yet another Unlock
* [NOD-747] Change FinalityInterval to be 24 hours, isCurrent to be true if the DAG's time is less than 12 hours than the present, and change MaxInvPerMsg to be 1 << 17 (#625)
* [NOD-744] Wrap go routines with spawn
* [NOD-747] Wrap some more go routines with spawn
* [NOD-744] Some more missing go routines
* [NOD-744] Break lines so make code more readable
* [NOD-744] Declare a local scope variable so the func would use it.
* [NOD-744] Fix type and update comment.
* [NOD-744] Declare local var so go routine would use it
* [NOD-744] Rename variable, use normal assignment;
* [NOD-744] Rename variable.
* [NOD-641] Upgrade to github.com/pkg/errors v0.9.1 and use errors.As where needed
* [NOD-641] Fix find and replace error
* [NOD-641] Use errors.As for error type checking
* [NOD-641] Fix errors.As for pointer types
* [NOD-641] Use errors.As where needed
* [NOD-641] Rename rErr->ruleErr
* [NOD-641] Rename derr->dbErr
* [NOD-641] e->flagsErr where necessary
* [NOD-641] change jerr to more appropriate name
* [NOD-641] Rename cerr->bdRuleErr
* [NOD-641] Rename serr->scriptErr
* [NOD-641] Use errors.Is instead of testutil.AreErrorsEqual in TestNewHashFromStr
* [NOD-641] Rename bdRuleErr->dagRuleErr
* [NOD-641] Rename mErr->msgErr
* [NOD-641] Rename dErr->deserializeErr
* [NOD-648] Add TestProcessDelayedBlocks
* [NOD-648] Add one second to secondsUntilDelayedBlockIsValid to make sure the delayedBlock timestamp will be valid, and add comments
* [NOD-648] Remove redundant import
* [NOD-648] Use fakeTimeSource instead of time.Sleep
* [NOD-648] Rename dag.HaveBlock->dag.IsKnownBlock, dag.BlockExists->dag.IsInDAG
* [NOD-648] Add comment
* [NOD-641] Rename HaveBlock->IsKnownBlock, BlockExists->IsInDAG
* [NOD-702] Fix netsync slowing down significantly due to excessive allocs in serializeUTXO.
* [NOD-702] Fix bad make statement.
* [NOD-702] Move writeBuffer to flushToDB.
* [NOD-636] Scope err so defer anonymous defer function will get it.
* [NOD-636] Add comment to explain why this line is needed.
* [NOD-636] Edit comment.
* [NOD-700] Convert blockSet to map[*blockNode]struct{}.
* [NOD-700] Rename bluestNode to bluestBlock in bluest().
* [NOD-700] Make IsInSelectedParentChain not use the now-slower containsHash.
* [NOD-700] Rename block to node in blockset.go.
* [NOD-700] Remove containsHash and hashesEqual.
* [NOD-700] Add a comment to IsInSelectedParentChain about how it'll fail if the given blockHash is not within the block index.
* [NOD-675] Rename startHash/stopHash to lowHigh/stopHash
* [NOD-675] Fix typo
* [NOD-675] Undo go.mod go.sum conflicts
* [NOD-675] revert back to startHash for getChainFromBlock.
* [NOD-675] Revet back to startHash in getChainFromBlock leftovers.
* [NOD-675] Fix test name.
* [NOD-629] Change GHOSTDAG K to a a single byte using type
* [NOD-629] Rename variable
* [NOD-629] Rename K to KSize
* [NOD-629] Remove redundant casting
* [NOD-629] Add test for KSize
* [NOD-629] Seperate block serialization and db store
* [NOD-629] Make sure K is serialized as uint8
* [NOD-629] Rename KSize to KType
* [NOD-629] Comment for test
* [NOD-629] Change fail message
* [NOD-629] Remove newlines
* [NOD-629] Fix test
* [NOD-629] Do not use maxuint8, but !0 instead
* [NOD-629] Fix test
* [NOD-629] Merge conflict
* [NOD-629] Fix test; Update comment
* [NOD-652] Add selectedTip and getSelectedTip messages
* [NOD-652] Remove peerSyncState.isSelectedTipKnown
* [NOD-652] Do nothing on OnSelectedTip if the peer selected tip hasn't changed
* [NOD-652] Handle selected tip message with block handler
* [NOD-652] Add comments
* [NOD-652] go mod tidy
* [NOD-652] Fix TestVersion
* [NOD-652] Use dag.AdjustedTime instead of dag.timeSource.AdjustedTime
* [NOD-652] Create shouldQueryPeerSelectedTips and queueMsgGetSelectedTip functions
* [NOD-652] Change selectedTip to selectedTipHash where needed
* [NOD-652] add minDAGTimeDelay constant
* [NOD-652] add comments
* [NOD-652] Fix names and comments
* [NOD-652] Put msg.reply push in the right place
* [NOD-652] Fix comments and names
* [NOD-640] Revamp blueBlocksBetween to return up to maxEntries from highNode's antiFuture.
* [NOD-640] Fix bad traversal.
* [NOD-640] Use more accurate len.
* [NOD-640] Use more appropriate len in another place.
* [NOD-640] Remove the whole business with highNode's anticone.
* [NOD-640] Rename highNodeAntiFuture to candidateNodes.
* [NOD-640] Explain the highNode.blueScore-lowNode.blueScore+1 approximation.
* [NOD-640] UpHeap -> upHeap.
* [NOD-640] Fix off-by-one error.
* [NOD-640] Rename blueBlocksBetween to antiPastBetween,
* [NOD-640] upHeap -> up-heap.
* [NOD-640] Use a classic for to populate nodes.
* [NOD-640] Reworded a comment.
* [NOD-640] Clarify a comment.
* [NOD-640] Fix nodes declaration.
* [NOD-669] Remove the "get" from getBlueBlocksBetween.
* [NOD-669] Remove the "Get" from GetBlueBlocksHeadersBetween.
* [NOD-669] In blueBlocksBetween, rename startHash to lowHash and stopHash to highHash.
* [NOD-669] Rename startHash to lowHash and stopHash to highHash in blockLocator logic.
* [NOD-669] Remove zeroHash logic in blockLocator.
* [NOD-669] Finish renaming startHash and stopHash in blockdag.
* [NOD-669] Rename startHash and stopHash in blockdag where I previously missed it.
* [NOD-669] Rename startHash and stopHash in blockdag where I previously missed it some more.
* [NOD-669] Rename startHash and stopHash in blockdag where I previously missed it some more some more.
* [NOD-669] Fix bad grammar in method names.
* [NOD-669] Rename lowHash to blockHash in SelectedParentChain.
* [NOD-669] Fix a comment.
* [NOD-669] Fix startSync sending a blockLocatorMsg with a zeroHash instead of the peer's selectedTip.
* [NOD-669] Rename bestPeer to syncPeer.
* [NOD-669] Fix comments.
* [NOD-616] Remove unused methods from BlockDAG.
* [NOD-616] Remove Height from GetRawMempoolVerboseResult and TxDesc.
* [NOD-616] Replaced BlockDAG.ChainHeight with SelectedTipBlueScore.
* [NOD-616] Remove the unused BlockChainHeightByHash.
* [NOD-616] Remove the unused blockChainHeight from checkBlockHeaderContext.
* [NOD-616] Remove chainHeight from util.Block.
* [NOD-616] Remove TestChainHeight.
* [NOD-616] Update unknown rule activation warning to use blueScore.
* [NOD-616] Update thresholdState to use blueScore instead of chainHeight.
* [NOD-616] Update blockLocator to use blueScore instead of chainHeight.
* [NOD-616] Remove blockNode.chainHeight.
* [NOD-616] Fix comments and variable names.
* [NOD-616] Replace a weird for loop with a while loop.
* [NOD-616] Fix a comment.
* [NOD-616] Remove pre-allocation in blockLocator.
* [NOD-616] Coalesce checks that startHash and stopHash are not the same into the same condition.
* [NOD-616] Fix a comment.
* [NOD-616] Remove weird blueScore logic around childHashStrings.
* [NOD-616] Fix hash pointer comparison.
* [NOD-616] Fix a comment.
* [NOD-616] Add ban score to peers misusing GetBlockLocator.
* [NOD-616] Replace adding ban score with disconnecting.
* [NOD-616] Add blueScore to FilteredBlockAddedNtfn.
* [NOD-570] Separate genesis variables for different netwroks
* [NOD-570] Make Testnet genesis
* [NOD-570] Make simnet and regtest genesis
* [NOD-570] Remake devnet genesis
* [NOD-570] Rename regNet -> regTest testnet->testNet
* [NOD-570] Change network names to one word instead of camel case
* [NOD-570] Change network names to one word instead of camel case
* [NOD-570] Fix test names
* [NOD-570] Fix TestGHOSTDAG
Co-authored-by: Dan Aharoni <dereeno@protonmail.com>
* [NOD-540] Implement reachability (#545)
* [NOD-540] Begin implementing reachability.
* [NOD-540] Finish implementing reachability.
* [NOD-540] Implement TestIsFutureBlock.
* [NOD-540] Implement TestInsertFutureBlock.
* [NOD-540] Add comments.
* [NOD-540] Add comment for interval in blockNode.
* [NOD-540] Updated comments over insertFutureBlock and isFutureBlock.
* [NOD-540] Implement interval splitting methods.
* [NOD-540] Begin implementing tree manipulation in blockNode.
* [NOD-540] Implement countSubtreesUp.
* [NOD-540] Add a comment explaining an impossible condition.
* [NOD-540] Implement applyIntervalDown.
* [NOD-540] Moved the reachability tree stuff into reachability.go.
* [NOD-540] Add some comments.
* [NOD-540] Add more comments, implement isInPast.
* [NOD-540] Fix comments.
* [NOD-540] Implement TestSplitFraction.
* [NOD-540] Implement TestSplitExact.
* [NOD-540] Implement TestSplit.
* [NOD-540] Add comments to structs.
* [NOD-540] Implement TestAddTreeChild.
* [NOD-540] Fix a comment.
* [NOD-540] Rename isInPast to isAncestorOf.
* [NOD-540] Rename futureBlocks to futureCoveringSet.
* [NOD-540] Rename isFutureBlock to isInFuture.
* [NOD-540] move reachabilityInterval to the top of reachability.go.
* [NOD-540] Change "s.t." to "such that" in a comment.
* [NOD-540] Fix indentation.
* [NOD-540] Fix a potential bug involving float inaccuracy.
* [NOD-540] Wrote a more descriptive error message.
* [NOD-540] Fix error messsage.
* [NOD-540] Fix the recursive countSubtreesUp.
* [NOD-540] Rename countSubtreesUp to countSubtrees and applyIntervalDown to propagateInterval.
* [NOD-540] Implement updating reachability for a valid new block.
* [NOD-540] Implement a disk storage for reachability data.
* [NOD-540] Fix not all tree nodes being written to the database.
* [NOD-540] Implement serialization for reachabilityData.
* [NOD-540] Implement some deserialization for reachabilityData.
* [NOD-540] Implement restoring the reachabilityStore on node restart.
* [NOD-540] Made interval and remainingInterval pointers.
* [NOD-540] Rename setTreeInterval to setInterval.
* [NOD-540] Rename reindexTreeIntervals to reindexIntervals and fixed the comment above it.
* [NOD-540] Expand the comment above reindexIntervals.
* [NOD-540] Fix comment above countSubtrees.
* [NOD-540] Fix comment above countSubtrees some more.
* [NOD-540] Fix comment above split.
* [NOD-540] Fix comment above isAncestorOf.
* [NOD-540] Fix comment above reachabilityTreeNode.
* [NOD-540] Fix weird condition in addTreeChild.
* [NOD-540] Rename addTreeChild to addChild.
* [NOD-540] Fix weird condition in splitFraction.
* [NOD-540] Reverse the lines in reachabilityTreeNode.String().
* [NOD-540] Renamed f to fraction and x to size.
* [NOD-540] Fix comment above bisect.
* [NOD-540] Implement rtn.isAncestorOf().
* [NOD-540] Use treeNode isAncestorOf instead of treeInterval isAncestorOf.
* [NOD-540] Use newReachabilityInterval instead of struct initialization.
* [NOD-540] Make reachabilityTreeNode.String() use strings.Join.
* [NOD-540] Use sync.RWMutex instead of locks.PriorityMutex.
* [NOD-540] Rename thisTreeNode to newTreeNode.
* [NOD-540] Rename setTreeNode to addTreeNode.
* [NOD-540] Extracted selectedParentAnticone to a separate function.
* [NOD-540] Rename node to this.
* [NOD-540] Move updateReachability and isAncestorOf from dag.go to reachability.go.
* [NOD-540] Add whitespace after multiline function signatures in reachability.go.
* [NOD-540] Make splitFraction return an error on empty interval.
* [NOD-540] Add a comment about rounding to splitFraction.
* [NOD-540] Replace sneaky tabs with spaces.
* [NOD-540] Rename split to splitExponential.
* [NOD-540] Extract exponentialFractions to a separate function.
* [NOD-540] Rename bisect to findIndex.
* [NOD-540] Add call to reachabilityStore.clearDirtyEntries at the end of saveChangesFromBlock.
* [NOD-540] Explain the dirty hack in reachabilityStore.init().
* [NOD-540] Split the function signature for deserializeReachabilityData to two lines.
* [NOD-540] Add a comment about float precision loss to exponentialFractions.
* [NOD-540] Corrected a comment about float precision loss to exponentialFractions.
* [NOD-540] Fixed a comment about float precision loss to exponentialFractions some more.
* [NOD-540] Added further comments above futureCoveringBlockSet.
* [NOD-540] Rename addTreeNode to setTreeNode.
* [NOD-540] Rename splitExponential to splitWithExponentialBias.
* [NOD-540] Fix object references in reachabilityData deserialization (#563)
* [NOD-540] Fix broken references in deserialization.
* [NOD-540] Fix broken references in futureCoveringSet deserialization. Also add comments.
* [NOD-540] Don't deserialize on the first pass in reachabilityStore.init().
* [NOD-540] Remove redundant assignment to loaded[hash].
* [NOD-540] Use NewHash instead of SetBytes. Rename data to destination.
* [NOD-540] Preallocate futureCoveringSet.
* [NOD-541] Implement GHOSTDAG (#560)
* [NOD-541] Implement GHOSTDAG
* [NOD-541] Replace the old PHANTOM variant with GHOSTDAG
* [NOD-541] Move dag.updateReachability to the top of dag.applyDAGChanges to update reachability before the virtual block is updated
* [NOD-541] Fix blueAnticoneSize
* [NOD-541] Initialize node.bluesAnticoneSizes
* [NOD-541] Fix pastUTXO and applyBlueBlocks blues order
* [NOD-541] Add serialization logic to node.bluesAnticoneSizes
* [NOD-541] Fix GHOSTDAG to not count the new block and the blue candidates anticone, add selected parent to blues, and save to node.bluesAnticoneSizes properly
* [NOD-541] Fix test names in inner strings
* [NOD-541] Writing TestGHOSTDAG
* [NOD-541] In blueAnticoneSize change node->current
* [NOD-541] name ghostdag return values
* [NOD-541] fix ghostdag to return slice
* [NOD-541] Split k-cluster violation rules
* [NOD-541] Add missing space
* [NOD-541] Add comment to ghostdag
* [NOD-541] In selectedParentAnticone rename past->selectedParentPast
* [NOD-541] Fix misrefernces to TestChainUpdates
* [NOD-541] Fix ghostdag comment
* [NOD-541] Make PrepareBlockForTest in blockdag package
* [NOD-541] Make PrepareBlockForTest in blockdag package
* [NOD-541] Assign to selectedParentAnticone[i] instead of appending
* [NOD-541] Remove redundant forceTransactions arguments from PrepareBlockForTEST
* [NOD-541] Add non-selected parents to anticoneHeap
* [NOD-541] add test for ghostdag
* [NOD-541] Add comments
* [NOD-541] Use adjusted time for initializing blockNode
* [NOD-541] Rename isAncestorOf -> isAncestorOfBlueCandidate
* [NOD-541] Remove params from PrepareBlockForTest
* [NOD-541] Fix TestChainHeight
* [NOD-541] Remove recursive lock
* [NOD-541] Fix TestTxIndexConnectBlock
* [NOD-541] Fix TestBlueBlockWindow
* [NOD-541] Put prepareAndProcessBlock in common_test.go
* [NOD-541] Fix TestConfirmations
* [NOD-541] Fix TestAcceptingBlock
* [NOD-541] Fix TestDifficulty
* [NOD-541] Fix TestVirtualBlock
* [NOD-541] Fix TestSelectedPath
* [NOD-541] Fix TestChainUpdates
* [NOD-541] Shorten TestDifficulty test time
* [NOD-541] Make PrepareBlockForTest use minimal valid block time
* [NOD-541] Remove TODO comment
* [NOD-541] Move blockdag related mining functions to mining.go
* [NOD-541] Use NextBlockCoinbaseTransaction instead of NextBlockCoinbaseTransactionNoLock in NextCoinbaseFromAddress
* [NOD-541] Remove useMinimalTime from BlockForMining
* [NOD-541] Make MedianAdjustedTime a *BlockDAG method
* [NOD-541] Fix ghostdag to use anticone slice instead of heap
* [NOD-541] Fix NewBlockTemplate locks
* [NOD-541] Fix ghostdag comments
* [NOD-541] Convert MedianAdjustedTime to NextBlockTime
* [NOD-541] Fix ghostdag comment
* [NOD-541] Fix TestGHOSTDAG comment
* [NOD-541] Add comment before sanity check
* [NOD-541] Explicitly initialize .blues in ghostdag
* [NOD-541] Rename *blockNode.lessThan to *blockNode.less
* [NOD-541] Remove redundant check if block != chainBlock
* [NOD-541] Fix comment
* [NOD-541] Fix comment
* [NOD-497] Add comment; General refactoring
* [NOD-497] General refactoring.
* [NOD-497] Use isAncestor of the tree rather than the node
* [NOD-497] Remove reachability mutex lock as it is redundant (dag lock is held so no need); General refactoring.
* [NOD-497] Update comment
* [NOD-497] Undo test blocktimestamp
* [NOD-497] Update comments; Use BlockNode.less for blockset;
* [NOD-497] Change processBlock to return boolean and not the delay duration (merge conflict)
* [NOD-497] Undo change for bluest to use less; Change blocknode less to use daghash.Less
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
Co-authored-by: Dan Aharoni <dereeno@protonmail.com>
* [NOD-420] Delay blocks with valid timestamp (non-delayed) that point to a delayed block.
* [NOD-420] Mark block as requested when setting as delayed.
* [NOD-420] Merge master; Use dag.timeSource.AdjustedTime() instead of time.Now;
* [NOD-420] Return nil when not expecting an error
* [NOD-420] Initialise delyaed blocks mapping
* [NOD-420] Trigger delayed blocks processing every time we process a block.
* [NOD-420] Hold the read lock in processDelayedBlocks
* [NOD-420] Add delayed blocks heap sorted by their process time so we could process them in order.
* [NOD-420] Update debug log
* [NOD-420] Fix process blocks loop
* [NOD-420] Add comment
* [NOD-420] Log error message
* [NOD-420] Implement peek method for delayed block heap. extract delayed block processing to another function.
* [NOD-420] Trigger process delayed blocks only in process block
* [NOD-420] Move delayed block addition to process block
* [NOD-420] Use process block to make sure we fully process the delayed block and deal with orphans.
* [NOD-420] Unexport functions when not needed; Return isDelayed boolean from ProcessBlock instead of the delay duration
* [NOd-420] Remove redundant delayedBlocksLock
* [NOD-420] Resolve merge conflict; Return delay 0 instead of boolean
* [NOD-420] Do not treat delayed block as orphan
* [NOD-420] Make sure block is not processed if we have already sa delayed.
* [NOD-420] Process delayed block if parent is delayed to make sure it would not be treated as orphan.
* [NOD-420] Rename variable
* [NOD-420] Rename function. Move maxDelayOfParents to process.go
* [NOD-420] Fix typo
* [NOD-420] Handle errors from processDelayedBlocks properly
* [NOD-420] Return default values if err != nil from dag.addDelayedBlock
* [NOD-420] Return default values if err != nil from dag.addDelayedBlock in another place
Co-authored-by: Svarog <feanorr@gmail.com>
* [NOD-549] Update version to 0.1.0 and allow injection of appBuild
* [NOD-549] Fixed peer tests
* [NOD-549] Fixed wire tests
* [NOD-549] Remove any mention of semVer.
* [NOD-549] Don't include appBuild at all if it includes invalid characters
* [NOD-549] Panic if appBuild contains invalid characters
* [NOD-549] Move checkAppBuild into
* [NOD-549] Update comment
* [NOD-546] Report build failures to Discord instead of Telegram.
* [NOD-546] Make a temporary compilation error.
* [NOD-546] Make a couple of temporary print outs.
* [NOD-546] Remove temporary debug stuff.
* [NOD-546] Make notify_discord() return early if Discord variables are not set.
* [NOD-549] Update version to 0.1.0 and allow injection of appBuild
* [NOD-549] Fixed peer tests
* [NOD-549] Fixed wire tests
* [NOD-549] Remove any mention of semVer.
* [NOD-549] Don't include appBuild at all if it includes invalid characters
* [NOD-549] Panic if appBuild contains invalid characters
* [NOD-549] Move checkAppBuild into
* [NOD-517] Remove copyright notices from all doc.go.
* [NOD-517] Updated the root doc.go.
* [NOD-517] Remove all cov_report.sh and test_coverage.txt.
* [NOD-517] Make all doc.go use the same style of comment.
* [NOD-517] Update dagconfig doc.go.
* [NOD-517] Update blockdag doc.go.
* [NOD-517] Update doc.go in connmgr.
* [NOD-517] Update doc.go in fullblocktests.
* [NOD-517] Update doc.go in database.
* [NOD-517] Update doc.go in ecc.
* [NOD-517] Update doc.go in rpctest.
* [NOD-517] Removed superfluous license in logs.
* [NOD-517] Update doc.go in mempool.
* [NOD-517] Updated doc.go in peer.
* [NOD-517] Update doc.go in rpcclient.
* [NOD-517] Update doc.go in txscript.
* [NOD-517] Update doc.go in util.
* [NOD-517] Update doc.go in base58.
* [NOD-517] Update doc.go in bech32.
* [NOD-517] Update doc.go in txsort.
* [NOD-517] Update doc.go in wire.
* [NOD-517] Fix indentation.
* [NOD-517] Add a copyright notice to the main doc.go.
* [NOD-517] Add Conformal to the license notices.
* [NOD-517] Remove superfluous language from a doc.
* [NOD-517] Fix bad example.
* [NOD-401] Created CLI-Wallet base structure and new command
* [NOD-401] Switched to go-flags sub-command parsing
* [NOD-401] Added config for all sub-commands
* [NOD-401] Work in progress for send command in cli-wallet
* [NOD-401] Allow to pass PrefixUnknown to ParseAddress + add .Prefix() to addresses
* [NOD-401] Finished implementing all wallet commands
* [NOD-401] some refactorings to sendTx
* [NOD-401] Moved wallet to kasparov repo + updated tests with new prefixes
* [NOD-495] Remove txgen to separate repository
* [NOD-495] Remove DNSSeeder to separate repository
* [NOD-495] Remove kasparov to separate repository
* [NOD-495] Remove miningsimulator to separate repository
* [NOD-495] httpserverutils should use kaspad logger package
* [NOD-495] Remove faucet to separate repository
* [NOD-495] httpserverutils should use kasparov logger
* [NOD-532] Change chain to DAG in the root package.
* [NOD-532] Change chain to DAG in checkpoints.go.
* [NOD-532] Change chain to DAG in blockdag.
* [NOD-532] Change chain to DAG in cmd.
* [NOD-532] Change chain to DAG in dagconfig.
* [NOD-532] Change chain to DAG in database.
* [NOD-532] Change chain to DAG in mempool.
* [NOD-532] Change chain to DAG in mempool.
* [NOD-532] Change chain to DAG in netsync.
* [NOD-532] Change chain to DAG in rpcclient.
* [NOD-532] Change chain to DAG in server.
* [NOD-532] Change chain to DAG in txscript.
* [NOD-532] Change chain to DAG in util.
* [NOD-532] Change chain to DAG in wire.
* [NOD-532] Remove block heights in dagio.go examples.
* [NOD-532] Rename fakeChain to fakeDAG.
* [NOD-532] Fix comments, remove unused EnableBCInfoHacks flag.
* [NOD-532] Fix comments and variable names.
* [NOD-532] Fix comments.
* [NOD-532] Fix merge errors.
* [NOD-532] Formatted project.
* [NOD-390] Add faucet Dockerfile
* [NOD-390] Allow running migration without -api-server-url and --private-key arguments
* [NOD-390] Change kasparov-server to kasparovd in its Dockerfile
* [NOD-548] Remove default DNS seed from devnet
* [NOD-390] Add faucet Dockerfile
* [NOD-390] Allow running migration without -api-server-url and --private-key arguments
* [NOD-390] Change kasparov-server to kasparovd in its Dockerfile
* [NOD-390] Change API server and Kasparov server to kasparovd
* [NOD-494] Updated main README.md
* [NOD-494] Updated blockdag/README.md
* [NOD-494] Aligned text length in main README.md
* [NOD-494] Updated most remaining packages READMEs + deleted util/coinset
* [NOD-494] Update integration README
* [NOD-494] Did a final pass over all readmes
* [NOD-494] Updated README for DNSSeeder with more info on how to create a functioning setup
* [NOD-494] Remove all double spaces from readmes
* [NOD-494] Minor fixes in READMEs + update license to kaspanet developers
* [NOD-494] Add backtick around ecc and util in hdkeychain README
* [NOD-502] Remove checkpoints.
* [NOD-502] Remove remaining references to checkpoints.
* [NOD-500] Split RejectFinality to RejectDifficulty.
* [NOD-500] Remove support for headers-first in p2p.
* [NOD-500] Panic in newHashFromStr in case of an error.
* [NOD-525] renamed folders server and syncd to kasparovserver and kasparovsync respectively
* [NOD-525] Fixed references to kasparov sub-apps
* [NOD-525] Renamed kasparovserver -> kasparovd
* [NOD-471] Make AddTx return false for duplicate coinbase, and make pastUTXO return accepted transaction with the accepting block blue score
* [NOD-471] Remove diffFromAcceptanceData
* [NOD-471] Make fetchBlueBlocks return also selected parent
* [NOD-471] Skip adding coinbase transactions on applyBlueBlocks
* [NOD-471] Use tx.IsCoinbase() instead of i == util.CoinbaseTransactionIndex
* [NOD-492] Split ApiServer to server and syncd.
* [NOD-492] Add missing file.
* [NOD-492] Remove references to --migrate from common config.
* [NOD-492] Move MQTT to the sync daemon.
* [NOD-492] Fix server Dockerfile and create one for syncd.
* [NOD-492] Rename ApiServer to Kasparov.
* [NOD-492] Fix packages.
* [NOD-492] Fix more packages.
* [NOD-492] Fix comments and package names.
* [NOD-492] Move blank import packages to main.
* [NOD-492] Move common logging logic out of individual config.go files.
* [NOD-492] Move database models to a package called dbmodels.
* [NOD-492] Rename models package to apimodels.
* [NOD-487] Implement a mechanism to gracefully shut down after a panic.
* [NOD-487] Fixed bad log.
* [NOD-487] Removed unused import.
* [NOD-487] Convert panic handlers from anonymous functions to methods.
* [NOD-488] Make getBlueBlocksBetween return error if start hash is not in the selected parent chain of stop hash
* [NOD-488] Convert for to while style
* [NOD-479] Separate max outbound connections and max inbound connections
* [NOD-479] Fix merge
* [NOD-479] Renames and add function countinboundPeers
* [NOD-479] Remove redundant check on maximum outbound peers
* [NOD-479] Rename countinboundPeers -> countInboundPeers
* [NOD-428] Required RPC user and password, and do not create a default config file for btcctl if rpc login details were provided
* [NOD-428] Don't check rpc user and password if rpc is disabled
* [NOD-428] Fix error message
* [NOD-484] Fix deadlock between p2p server and sync manager during shutdown.
* [NOD-484] Fix quitWaitGroup.Wait() potentially not waiting in some scenarios.
* [NOD-484] Add a comment explaining quitWaitGroup.
* [NOD-484] Fix typo.
* [NOD-484] Add etc to comment.
* [NOD-443] Immediately request missing parents in API server sync
* [NOD-443] Add rpc client log to api server
* [NOD-443] Fix wrong ordering of pendingHeaders queue
* [NOD-443] Fix error comparision at TestNewHashFromStr
* [NOD-443] Make a separate handleMissingParentHeaders function
* [NOD-443] Change log level
* [NOD-443] Put handleMissingParentHeaders next to handleBlockHeader
* [NOD-443] Make handleBlockAddedMsg function
* [NOD-443] Make reusable missingParentsHashesStr string
* [NOD-443] Remove redundant 's'
* [NOD-443] Refactor to first get all blocks and then add them to database
* [NOD-443] Rename variables and functions, and remove redundant logs
* [NOD-443] Make fetchBlockAndMissingAncestors use block hash as an argument
* [NOD-443] Add log only for first orphan block
* [NOD-443] Fix wrong order of adding blocks to pendingBlocks
* [NOD-443] Write logs for all orphans
* [NOD-443] Log only missing parents that are not already fetched
* [NOD-443] Rename rawVerboseBlockTuple -> rawVerboseBlock
* [NOD-443] Make fetchBlock return *rawVerboseBlock
* [NOD-443] Rename rawVerboseBlock -> rawAndVerboseBlock
* [NOD-472] Don't fetch accepting block and tx confirmations for getBlocks
* [NOD-472] Don't fetch accepting block and tx confirmations in any block verbose result
* [NOD-472] Add stringPointerToString function
* [NOD-382] Add notification for accepted transactions
* [NOD-382] Remove print statement
* [NOD-426] Publish notifications for unaccepted transactions
* [NOD-426] Load DB in controller
* [NOD-426] Remove function name from error message
* [NOD-426] Add input addresses for transactions notifications
* [NOD-426] Remove function name from error message
* [NOD-426] Change method name to accepted transactions
* [NOD-426] Remove newlines
* [NOD-426] Use join instead of separate query
* [NOD-426] Remove new line
* [NOD-447] Fix deadlocks and hanging goroutines
* [NOD-447] Add tests
* [NOD-447] Add unpatch to spawnPatch
* [NOD-447] Don't send to releaseWait if waitingCounter is zero
* [NOD-447] Change waitingCounter to boolean and rename to isReleaseWaitWaiting, change checkIfRunningSpawnsAreLeft to return only one function, and lock critical code related to wg.isReleaseWaitWaiting
* [NOD-447] Rename txConfirmations -> txConfirmationsNoLock, txConfirmationsWithLock -> txConfirmations
* [NOD-447] Add documentation and delete redundant spawn
* [NOD-447] Fix comments
* [NOD-447] Fix comments
* [NOD-445] Added option to mining simulator to get address list from AWS
* [NOD-445] Add support to get miningsimulator addresslist from AWS
* [NOD-445] Added mechanism to update when new servers come online
* [NOD-445] Set config in connectionManager
* [NOD-445] Invert DisableTLS condition in readCert
* [NOD-381] Publish transaction messages to MQTT
* [NOD-381] Remove redundant variable
* [NOD-381] Send payload as string
* [NOD-381] Add Error handling
* [NOD-381] Respond with TransactionResponse
* [NOD-381] Use transactionResponse for notifications
* [NOD-381] Move code to appropriate places
* [NOD-381] Pass raw block instead of txId
* [NOD-381] Add comments to public functions
* [NOD-381] Remove print statement
* [NOD-381] Pass transaction instead of block; Use pointers so default will be nil;
* [NOD-381] Use pointers so value could be nil
* [NOD-381] Change variable name
* [NOD-381] Set QoS to 2
* [NOD-381] Move isConnected to MQTT, so client won't have to worry about it; General code refactors;
* [NOD-448] Change GetBlocksCmd to be able to include both raw and verbose block data.
* [NOD-448] Update sync logic to only make one getBlocks call per page.
* [NOD-448] Make GetBlocks get each block only once.
* [NOD-423] Rename BestBlock to SelectedTip
* [NOD-423] Implement GetSelectedTip RPC command
* [NOD-423] Add help to getSelectedTip command
* [NOD-423] Fix getSelectedTip test
* [NOD-423] Fix tests so they would compile. These tests will need to be rewriten at some point.
* [NOD-423] Make integration test compile. Test need to be revisited
* [NOD-423] Rename variables
* [NOD-423] Change comment s about best block to selected tip.
* [NOD-423] Update comment
* [NOD-423] Change height to bluescore
* [NOD-434] Add the same enqueue/process mechanism as chainChangedMsgs for blockAddedMsgs.
* [NOD-434] Clean up after merge.
* [NOD-434] Implement mechanism for re-requesting missing parent blocks.
* [NOD-434] Fixed bad error message.
* [NOD-434] Split processBlockAddedMsgs.
* [NOD-434] Name return values in canHandleBlockAddedMsg.
* [NOD-434] Rename canHandleBlockAddedMsg to missingParentHashes and fix bad loop break.
* [NOD-434] Rename the variable missingParentHashes to missingHashes.
* [NOD-434] Rename a couple of variables.
* [NOD-434] Rename outerloop to outerLoop.
* [NOD-434] Fix typo and remove superfluous continue.
* [NOD-412] Change Warnf to Errorf where appropriate.
* [NOD-430] Print hashes of missing parents in case can't insert block into DB of API-Server
* [NOD-430] Use continue OUTER_LOOP instead of break
* [NOD-430] Use lowerCamelCase for label
* [NOD-380] Add MQTT to the project.
* [NOD-380] Add MQTT params to config.
* [NOD-380] Implement connecting to an mqtt broker.
* [NOD-380] Fix a comment.
* [NOD-380] Removed unnecessary option.
* [NOD-380] Added comments to MQTT functions.
* [NOD-380] Fix copy+paste error.
* [NOD-380] Make it so that all the mqtt flags must be passed together.
* [NOD-380] Use activeConfig instead of passing it everywhere.
* [NOD-395] Write a test for the diffFromAcceptanceData crash.
* [NOD-395] Converted MultiBlockTxsAcceptanceData into a slice.
* [NOD-395] Fix failing test.
* [NOD-395] Populate multiBlockTxsAcceptanceData bottom-to-top.
* [NOD-395] Add comment to FindAcceptanceData.
* [NOD-395] Remove no-longer relevant note about probability in TestOrderInDiffFromAcceptanceData.
* [NOD-413] Make "Max failed connection attempts reached" less frequent
* [NOD-413] Throttle only certain types of logs.
* [NOD-413] Add a comment for shouldWriteConnFailedLog.
* [NOD-413] Fix lint error.
* [NOD-413] Make ErrNoAddress a special type to support error wrapping.
* [NOD-413] Make throttledConnFailedLogInterval 10 minutes.
* [NOD-413] Move p2p errors into variables.
* [NOD-413] Reorganize throttled stuff to be next to each other.
* [NOD-386] Extract net parsing functionality to a shared place.
* [NOD-386] Add extract ActiveNetParams to cmdconfig
* [NOD-386] Adding comments so go-vet won't shout at me
* [NOD-386] Rename package name to config
* [NOD-386] Rename commandConfig to configFlags
* [NOD-386] Rename function to ResolveNetwork
* [NOD-386] Fix renaming errors
* [NOD-386] Refactor network config to btcd level so APIserver and btcd could use it
* [NOD-386] Refactor network config to config package
* [NOD-386] Move ActiveNetParams to network section
* [NOD-386] Explictly return nil
* [NOD-386] Reuse activeNetParams from netwrok config
* [NOD-386] Set ActiveNetworkFlags instance to be global
* [NOD-386] Remove redundant newline
* [NOD-386] Init ActiveNetParams in address manager test
* [NOD-386] Add dnsseeder network config
* [NOD-386] Use ActiveConfig() method to access configuration
* [NOD-373] Implement Schnorr digital signatures and remove ECDSA (based on code from gcash/bchd)
* [NOD-374] Add new error to list; Update comments.
* [NOD-373] Remove leftovers of verifyMessage RPC command (which was deleted)
* [NOD-373] Remove redundant test, add Schnorr tests, and fix tests where needed
* [NOD-373] Fix tests and remove redundant ones
* [NOD-373] Refactor functions names
* [NOD-373] Remove empty line
* [NOD-373] Fix comments, rename functions to more meaningful names
* [NOD-373] Additional data in nonceRFC6979 should not be nil
* [NOD-373] Refactor function name
* [NOD-373] Add permalinks for links to bchd code
* [NOD-385] Make confirmations be calculated as dag.selectedTip().blueScore - acceptingBlock.blueScore + 2
* [NOD-385] Fix comments
* [NOD-385] Make more explicit check in accepting block for selected tip
* [NOD-385] Put only non accepted transactions in areTxsInBlock
* [NOD-385] fetchSelectedTip only if needed
* [NOD-404] Calculate mass in API server
* [NOD-404] Fix uninitialized maps
* [NOD-404] Use txID instead of prevDBTransactionsOutput.Transaction.TransactionID
* [NOD-398] Change API server type HandlerError to work with errors instead of error strings
* [NOD-398] Rename OriginalError -> Cause and isHandleError -> ok
* [NOD-350] Implement testnet faucet
* [NOD-350] Add JSON annotations to api server response types
* [NOD-350] Fix IP check query, update IP usage with upsert, and make IP a primary key
* [NOD-377] Remove redundant float conversion
* [NOD-377] Change not current database error message
* [NOD-377] change API route from /money_request to /request_money
* [NOD-377] Add a constant for 24 hours
* [NOD-377] Remove redundant call for getWalletUTXOSet()
* [NOD-377] Condition refactoring
* [NOD-377] Fix POST request to API server content type
* [NOD-350] Rename day -> timeBetweenRequests
* [NOD-377] Rename timeBetweenRequests -> minRequestInterval, timeBefore24Hours -> minRequestInterval
* [NOD-350] Rename file responsetypes -> response_types
* [NOD-350] Rename convertTxModelToTxResponse -> convertTxDBModelToTxResponse
* [NOD-350] Explicitly select blue_score in fetchSelectedTipBlueScore
* [NOD-350] Refactor and add comments
* [NOD-350] Make calcFee use MassPerTxByte
* [NOD-350] Convert IP column to varchar(39) to allow ipv6 addresses
* [NOD-350] Add comments to isFundedAndIsChangeOutputRequired
* [NOD-350] Remove approximateConfirmationsForCoinbaseMaturity
* [NOD-350] Fix comments
* [NOD-394] Rename --only-build to --no-run.
* [NOD-394] Allow --rm and --no-build to be run together with no-run.
* [NOD-394] Make --cleanup alias for --rm --no-run --no-build.
* [NOD-394] Fix typo in usage string.
* [NOD-394] Set docker/docker-compose.yaml to use devnet instead of testnet.
* [NOD-376] Made bad unorphaned blocks not reject the original block.
* [NOD-376] Fix wording in a comment.
* [NOD-376] Add a test to make sure that bad child blocks don't invalidate valid parent blocks.
* [NOD-376] Clarify comments and don't check PoW for child block (it's irrelevant for this test case).
* [NOD-368] correct text output of address generator (address instead of public key)
* [NOD-368] add hash160 to output of genaddr
* [NOD-368] change encoding to hexadecimal
* [NOD-368] fix formatting
* change finality interval to 1000 ( ~16.6 minutes interval)
* [NOD-357] define finality interval in dagParams instead of using a constant.
* use dagParams for FinalityInterval instead of constant
* override parameter so test would pass on CI (Jenkins machine runs out of memory if we use 1000)
* formating the code
* [NOD-134] Don't connect to an address from the same 16 CIDR
* [NOD-134] Rename outboundPeerConnected variables
* [NOD-134] Change newConnMtx to newConnReqMtx
* [NOD-338] Recover indexer if it didn't work for a while
* [NOD-338] Recover indexer if it didn't work for a while
* [NOD-338] Recover indexer if it didn't work for a while
* [NOD-338] Add tests and move blockidhash.go to blockdag package
* [NOD-338] Delete index current block id when dropping index, and do some refactoring
* [NOD-338] Change comments
* [NOD-338] Change recover error messages
* [NOD-338] Fix comments
* [NOD-338] Fix comments and fix test name
* [NOD-337] In CheckTransactionSanity, made max mass of transaction to be half of block max mass.
* [NOD-337] Added a comment for MaxMassPerTx.
* [NOD-337] Fixed a couple of comments.
* [NOD-275] Moved getBlockTemplate and related functionality to a separate file.
* [NOD-275] Started moving handlers to separate files.
* [NOD-275] Fixed merge errors.
* [NOD-275] Moved all handlers out of rpcserver.go.
* [NOD-275] Moved non-shared functions out of rpcserver.go.
* [NOD-275] Moved handleGetAllManualNodesInfo to a separate file.
* [NOD-275] Moved handlers out of rpcwebsocket.go to separate files.
* [NOD-275] Fixed import error.
* [NOD-275] Renamed all handler files to include underscores.
* [NOD-275] Moved common rpc helper functions to common.go.
* [NOD-289] Implemented database isCurrent checking and connection.
* [NOD-289] Added GetChainFromBlock to RPCClient.
* [NOD-289] Limited the amount of blocks in GetChainFromBlockResponse.
* [NOD-289] Fixed various issues that were keeping GetChainFromBlocks from working properly.
* [NOD-289] Created blockloop.go.
* [NOD-289] Updated go.mod after merge.
* [NOD-289] Implemented collection of current selected parent chain.
* [NOD-289] Fixed test. Reverted not deleting utxoDiffData from the DB.
* [NOD-289] Implemented GetBlocks.
* [NOD-289] Added comment to BlockHashesFrom.
* [NOD-289] Added GetBlocks to rpcclient.
* [NOD-289] Added verboseBlocks to GetBlocks.
* [NOD-289] Implemented block insertion.
* [NOD-289] Added AUTO_INCREMENT to tables that were missing it.
* [NOD-289] Made gasLimit in subnetwork nullable.
* [NOD-289] Renamed transactions_outputs to transaction_outputs.
* [NOD-289] Fixed weird coinbase behavior in vin.
* [NOD-289] Made collectCurrentBlocks start from the most recent startHash.
* [NOD-289] Added IsChainBlock to GetBlockVerboseResult.
* [NOD-289] Implemented adding a block from onBlockAdded.
* [NOD-289] Added removedParentChainHashes to getChainFromBlock.
* [NOD-289] Implemented updating the selected parent chain from onChainChanged.
* [NOD-289] Implemented some initial logic for updating the UTXO.
* [NOD-289] Fixed merge errors.
* [NOD-326] Fixed some more merge errors.
* [NOD-289] Added error handling for missing required records.
* [NOD-289] Implemented handling removedChainHashes.
* [NOD-289] Implemented handling addedChainBlocks.
* [NOD-289] Fixed incorrect coinbase check.
* [NOD-289] Implemented inserting the transaction output address.
* [NOD-289] Added updating block.IsChainBlock.
* [NOD-289] Split insertBlock into many small functions.
* [NOD-289] Split updateSelectedParentChain into smaller functions.
* [NOD-289] Fixed pointer errors.
* [NOD-289] Fixed a bad exists check.
* [NOD-289] Fixed a couple of small bugs.
* [NOD-289] Fixed a TxID/Hash mixup.
* [NOD-289] Added block/tx mass to getBlockVerboseResponse.
* [NOD-289] Renamed blockLoop.go to sync.go. Added comments.
* [NOD-289] Deleted apiserver README.
* [NOD-289] Fixed golint errors.
* [NOD-289] Renamed findMostRecentBlockHash to findHashOfBluestBlock.
* [NOD-289] Fixed style in syncBlocks and fixed a comment.
* [NOD-289] Copied NewErrorFromDBErrors over from NOD-324.
* [NOD-289] Created a couple of utils to make error handling with gorm slightly less painful.
* [NOD-289] Added error handling for database calls.
* [NOD-289] Fixed some more style/comments.
* [NOD-289] Fixed comments.
* [NOD-289] Renamed TransactionInput.TransactionOutput to TransactionInput.PreviousTransactionOutput.
* [NOD-289] Added a commends about pagination in getBlocks and getChainFromBlock.
* [NOD-289] Removed the coinbase field from Vin.
* [NOD-289] Deferred handling chainChangedMsgs until we have the appropriate data.
* [NOD-289] Optimized queries in updateRemovedChainHashes and updateAddedChainBlocks.
* [NOD-289] Optimized queries in insertBlockParents.
* [NOD-289] Optimized queries in insertTransactionInput.
* [NOD-289] Split Where calls to separate lines.
* [NOD-289] Fixed merge errors.
* [NOD-289] Exited early from insertBlockParents if we're the genesis block.
* [NOD-289] Improved nextChainChangedChan mechanism.
* [NOD-289] Fixed the above sync mechanism a bit.
* [NOD-289] Renamed IsDBRecordNotFoundError to HasDBRecordNotFoundError and IsDBError to HasDBError.
* [NOD-289] Replaced old error handling for db errors with the lovely new stuff.
* [NOD-289] Exited early if we already inserted a block. This saves us checking if a record already exists for some record types.
* [NOD-289] Decoupled syncBlocks from syncSelectedParentChain.
* [NOD-289] Made a comment more explicit.
* [NOD-289] Extracted net resolution to a separate function.
* [NOD-289] Extracted syncing to a separate function.
* [NOD-289] Fixed a comment.
* [NOD-289] Fixed merge erros.
* [NOD-289] Fixed a couple of bugs.
* [NOD-289] Fixed another bug.
* [NOD-289] Extracted ChainChangedMsg conversion to a separate function.
* [NOD-289] Optimized queries in canHandleChainChangedMsg.
* [NOD-289] Moved the sync function closer to its call site.
* [NOD-289] Renamed HasDBRecordNotFoundError to IsDBRecordNotFoundError.
* [NOD-289] Used count instead of first.
* [NOD-289] Renamed address to hexAddress.
* [NOD-315] Created acceptanceindex.go including boilerplate.
* [NOD-315] Disallowed calls to notifyChainChanges and getChainFromBlock if the acceptance index is not on.
* [NOD-315] Implemented the acceptance index.
* [NOD-315] Fixed serialization/deserialization. Added test.
* [NOD-315] Fixed/added comments.
* [NOD-315] Fixed copy/paste errors.
* [NOD-315] Added an empty line for readability.
* [NOD-335] Made it not write a stack trace if the command line flags are wrong.
* [NOD-335] Fixed panic not printing the right error.
* [NOD-335] Removed code duplication.
* [NOD-319] Add query params to api server route handler
* Temp commit
* [NOD-322] Make database.DB a function
* [NOD-322] Move context to be the first parameter in all functions
* [NOD-322] Set db to nil on database.Close()
* [NOD-322] Tidy go.mod/go.sum
* [NOD-323] Move rpc-client to separate package
* [NOD-309] Add controller for POST /transaction
* [NOD-309] Added route for POST /transaction
* [NOD-309] in POST /transaction: Forward reject errors to client
* [NOD-309] Added custom client messages to errors in POST /transaction
* [NOD-309] Use utils.NewInternalServerHandlerError where appropriate
* [NOD-324] Properly handle GORM errors in API server
* [NOD-324] Handle RecordNotFound error in GetBlockByHashHandler
* [NOD-324] Make a separate function for NewErrorFromDBErrors
* [NOD-327] Add --migrate cli flag to API server
* [NOD-327] Change log messages
* [NOD-327] Remove `required` flag from API server RPC CLI arguments
* [NOD-327] Add database version in migrations logs
* [NOD-307] Implement API-Server GET /blocks
* [NOD-307] Implement API-Server GET /blocks
* [NOD-307] Add comments to exported constants
* [NOD-307] Flatten GET query values and check that 'order' value is valid
* [NOD-307] Validate order values in GetBlocksHandler
* [NOD-307] Add convertQueryParamToInt function
* [NOD-322] Make database.DB a function
* [NOD-322] Move context to be the first parameter in all functions
* [NOD-322] Set db to nil on database.Close()
* [NOD-322] Tidy go.mod/go.sum
* [NOD-322] Use http package const + message for StatusInternalServerError
* [NOD-299] Add waitgroup to wait for all `spawn`s to complete before calling teardown
* [NOD-299] Restore spawn on teardown + mark spawn done in the correct thread
* [NOD-303] Implement get transactions by address for API server
* [NOD-304] Implement get transactions by address
* [NOD-304] Implement get transactions by address
* [NOD-304] Use structs for where if possible
* [NOD-304] Auto increment IDs
* [NOD-304] Make defaultGetTransactionsLimit constant
* [NOD-304] Delete db directory
* [NOD-304] change db var to query
* [NOD-304] Extract route handle function from addRoutes
* [NOD-304] Order transactions by ID
* [NOD-304] Add error for passing arrays to GET
* [NOD-303] Implement get transaction by id for api server
* [NOD-303] Make routeParamTxID a constant
* [NOD-303] Change database is not current error.
* [NOD-303] Add ID to TransactionInput and TransactionOutput models
* [NOD-303] Change transactions_outputs table name to transaction_outputs and transactions_inputs to transaction_inputs
* [NOD-303] Add json annotations to transaction response types
* [NOD-303] Split server package
* [NOD-303] Add GetTransactionByHashHandler
* [NOD-303] Add comments to exported functions and variables
* [NOD-303] Put response types in a separate file
* [NOD-303] Rename functions
* [NOD-310] Implement REST server in API server
* [NOD-310] MetaData -> Metadata
* [NOD-310] Make custom context methods instead of custom request functions
* [NOD-310] change "Request ID" prefix to "RID" and convert to apiServerContext with newAPIServerContext everywhere
* [NOD-301] Don't sync with peer if the rendezvous point is below finality
* [NOD-301] Add block hash and peer address for the warn message
* [NOD-301] Fix perrLog.Warnf arguments order
* [NOD-300] If node has invalid ancestor set the according status in blockindex
* [NOD-300] Test that status is also updated for grand child of an invalid block
* [NOD-300] change make(blockSet) to newSet()
* [NOD-292] In accept.go, made dbStoreBlock and flushToDB occur within the same transaction.
* [NOD-292] Implemented processing blocks that were not validated on BTCD start.
* [NOD-292] Fixed processing logic on init. Added a test for it.
* [NOD-292] Fixed some comments.
* [NOD-292] Made unlocks deferred in a couple of places.
* [NOD-292] Made unprocessed block reprocess via ProcessBlock rather than maybeAcceptBlock.
* [NOD-292] Fixed grammar in comment. Added an explanation to TestAcceptingInInit.
* [NOD-292] Split flushToDB into two versions.
* [NOD-292] Fixed a bad assignment.
* [NOD-292] Fixed bad spacing.
* [NOD-297] Fix onChainChanged on rpcclient
* [NOD-285] create gorm models for db (#378)
* [NOD-285] Map API-Server database using GORM
* [NOD-285] Add accepting block to transactions and blocks models, and remove accepting block model
* [NOD-285] Define model relations
* [NOD-285] Fix many to many for Transaction and Block models
* [NOD-285] Remove redundant main file
* [NOD-296] Send SyncMgr.SubmitBlock errors as rpc errors (#381)
* [NOD-296] Send SyncMgr.SubmitBlock errors as rpc errors
* [NOD-296] Add error message prefix
* [NOD-298] Add comments to gorm models (#382)
* [NOD-294] Fix golint in deploy.sh and fix all lint warnings (#380)
* [NOD-294] Fix golint in deploy.sh and fixed all lint errors
* [NOD-294] Fix typos in comments
* [NOD-294] Convert VirtualForTest into alias of *virtualBlock
* [NOD-294] Fixed some more typos in comments
* [NOD-295] Limit the length of GetData to 50 (#383)
* [NOD-295] Fixed bad break condition in addInvsToGetDataMessageFromQueue.
* [NOD-295] Fixed the fix for bad break condition in addInvsToGetDataMessageFromQueue.
* [NOD-295] Made the check for max invs refer to invsNum instead of MaxInvPerGetDataMsg.
* [NOD-297] Fix onChainChanged on rpcclient
* [NOD-286] Implement API-Server base structure (#379)
* [NOD-286] Implement API-Server base structure
* [NOD-286] Add rpc user and password as command line arguments
* [NOD-286] Make log directory a CLI argument
* [NOD-286] Add db login details as CLI arguments
* [NOD-297] Fix onChainChanged on rpcclient and server
* [NOD-297] Fix variables and functions names
* [NOD-297] Fix AcceptedTxIds -> AcceptedTxIDs
* [NOD-286] Implement API-Server base structure
* [NOD-286] Add rpc user and password as command line arguments
* [NOD-286] Make log directory a CLI argument
* [NOD-286] Add db login details as CLI arguments
* [NOD-295] Fixed bad break condition in addInvsToGetDataMessageFromQueue.
* [NOD-295] Fixed the fix for bad break condition in addInvsToGetDataMessageFromQueue.
* [NOD-295] Made the check for max invs refer to invsNum instead of MaxInvPerGetDataMsg.
* [NOD-294] Fix golint in deploy.sh and fixed all lint errors
* [NOD-294] Fix typos in comments
* [NOD-294] Convert VirtualForTest into alias of *virtualBlock
* [NOD-294] Fixed some more typos in comments
* [NOD-285] Map API-Server database using GORM
* [NOD-285] Add accepting block to transactions and blocks models, and remove accepting block model
* [NOD-285] Define model relations
* [NOD-285] Fix many to many for Transaction and Block models
* [NOD-285] Remove redundant main file
* [NOD-280] Create database for API server
* [NOD-280] Rename public_key_script to pk_script
* [NOD-280] Change indexes names
* [NOD-280] Add accepting block to blocks and transactions table and remove accepting_blocks table
* [NOD-280] Add readme
* [NOD-280] Change VARCHAR(32) to CHAR(64)
* [NOD-280] Rename location_in_block to index
* [NOD-270] Added notifyChainChanges and related commands.
* [NOD-270] Added NTChainChanged to blockdag.
* [NOD-270] Implemented collection and sending of ChainChanged notifications.
* [NOD-270] Fixed an improperly named test.
* [NOD-270] Added a test: TestChainChangedNotification.
* [NOD-270] Fixed a couple copy+paste errors.
* [NOD-270] Added a couple of comments for TestChainChangedNotification.
* [NOD-270] Fixed formatting error.
* [NOD-270] Fixed a comment.
* [NOD-270] Uncoupled chain updates inside blockdag from the concept of a notification.
* [NOD-270] Removed intermediary ChainUpdates object from ChainChangedNotificationData.
* [NOD-256] Add error log
* [NOD-256] Add error log
* [NOD-256] Fix typo and comment
* [NOD-256] Remove btclog dir
* [NOD-256] Format project
* [NOD-256] Add error log files
* [NOD-256] Add an option to add a log file to write into to an existing backend logger
* [NOD-256] Get rid of redundant logs initialization
* [NOD-256] rename initLogRotators to initLog
* [NOD-256] Get rid ExampleSignTxOutput and convert ExampleBlockDAG_ProcessBlock to a regular test
* [NOD-256] Show error message if os.Exiting from initLog
* [NOD-241] Implement lower resolution peer rendezvous point discovery
* [NOD-241] Implement lower resolution peer rendezvous point discovery
* [NOD-241] Find exact rendezvous point
* [NOD-241] Find exact rendezvous point
* [NOD-241] Fix tests
* [NOD-241] Remove hash stop from MsgBlockLocator and add tests to MsgBlockLocator and MsgGetBlockLocator
* [NOD-241] Change everywhere startHash to hashStart and change comments
* [NOD-241] Fix locateBlockNodes to stop at hashStop
* [NOD-241] Formatted locatorSummary.
* [NOD-241] Fix node reversal
* [NOD-241] Fix hash start and hash stop order, and don't include startNode in dag.blockLocator
* [NOD-241] rename locateBlockNodes -> getBlueBlocksBetween and add a comment to it
* [NOD-241] change hash start to start hash and hash stop to stop hash
* [NOD-241] Move block locator stuff to a different file
* [NOD-241] Rename msggetblocks.go to msggetblockinvs.go
* [NOD-241] Format project
* [NOD-241] Rename rpcserverSyncManager.LocateHeaders to GetBlueBlocksHeadersBetween
* [NOD-241] Move the logic of finding the highest shared block to OnBlockLocator
* [NOD-241] Rename chainHeight -> nextChainHeight
* [NOD-241] Fix typo in comment
* [NOD-264] Implemented calcTxSelectionValue.
* [NOD-264] Fixed bad subnetworkID in calcTxSelectionValue.
* [NOD-264] Implemented sorting the txDescs by value.
* [NOD-264] Got rid of txPrioItem.
* [NOD-264] Moved transaction selection to a separate file.
* [NOD-264] Renamed the result object to txsForBlockTemplate.
* [NOD-264] Implemented tx selection.
* [NOD-264] Fixed trying to get the gas limit for built-in subnetworks.
* [NOD-264] Wrote comments where appropriate.
* [NOD-264] Moved calcTxSelectionValue to the mining package. (Non-mining nodes shouldn't be forced to calc selection value for every transaction)
* [NOD-264] Wrote a test for selectTxs.
* [NOD-264] Fixed a comment.
* [NOD-264] Fixed misunderstood test.
* [NOD-264] Added zero fee check. Added a couple more tests.
* [NOD-264] Added probabilistic tests. Fixed a couple of bugs in tx selection.
* [NOD-264] Fixed tests with missing fees.
* [NOD-264] Added a test over a range of txs with different gas/mass.
* [NOD-264] Added expected probability to the rest of the test cases.
* [NOD-264] Tightened bounds in probability test.
* [NOD-264] Fixed values in probabily test.
* [NOD-264] Added a comments for alpha and rebalanceThreshold.
* [NOD-264] Fixed a couple of comments, renamed result to txsForBlockTemplate.
* [NOD-264] Removed an irrelevant comment. Changed Tracef to Warnf in some logs.
* [NOD-264] Renamed selectionValue -> txValue.
* [NOD-264] Moved rebalancing to the start of the tx selection loop.
* [NOD-264] Added overflow check for gasUsage.
* [NOD-264] Renamed blockSigOps and blockMass to totalSigOps and totalMass.
* [NOD-264] Removed the need to pass usedCount to reblanaceCandidates. Also relaxed bounds in a test.
* [NOD-264] Split selectTxs into smaller functions. Also relaxed bounds in a test some more.
* [NOD-264] Added a comment for findTx.
* [NOD-264] Ordered candidateTxs by subnetwork instead of txValue.
* [NOD-264] Disallowed zero tx fees in mempool and config. Renamed iterateCandidateTxs to populateTemplateFromCandidates.
* [NOD-264] Changed isFinalizedTransaction log level from Warn to Debug.
* [NOD-264] Removed references to SigOps in txSelection.
* [NOD-264] Removed SigOps validation. Validating mass should suffice.
* [NOD-264] Renamed wasUsed to isMarkedForDeletion.
* [NOD-264] Renamed markCandidateTxUsed to markCandidateTxForDeletion.
* [NOD-264] Made some probabilistic tests less likely to fail when they shouldn't.
* [NOD-264] Added a message warning people about probabilistic tests.
* [NOD-264] Rephrased a comment about rebalanceThreshold.
* [NOD-264] Removed IsCoinBase, CheckTransactionInputsAndCalulateFee, and ValidateTransactionScripts from txSelection.
* [NOD-264] Removed a condition that is no longer relevant.
* [NOD-264] "which's" -> "whose"
* [NOD-264] Removed wasteful preallocations.
* [NOD-264] Fixed a comment referring to "used" transactions.
* [NOD-269] Added a skeleton for getChainFromBlock.
* [NOD-269] Made startHash and includeBlocks optional.
* [NOD-269] Implemented chainBlock collection.
* [NOD-269] Extracted GetBlockVerboseResult building to its own method.
* [NOD-269] Implemented the IncludeBlocks part of GetChainFromBlock.
* [NOD-269] Added a comment for NewGetChainFromBlockCmd.
* [NOD-269] Made IsInSelectedPathChain return an error.
* [NOD-269] Fixed a very wrong comment.
* [NOD-269] Made SelectedPathChain allocate only the required amount of space.
* [NOD-269] Renamed pathChain to parentChain.
* [NOD-269] Split handleGetChainFromBlock to separate functions.
* [NOD-269] Fixed some grammar.
* [NOD-234] Added an IsCurrent check to handleGetBlockTemplate.
* [NOD-234] Removed IsCurrent check from handleGetBlockTemplateRequest. Added an explanation for why we're checking the chainHeight.
* [NOD-234] Added ShouldMineOnGenesis to the IsCurrent check.
* [NOD-234] Flipped && operands to fail fast.
* [NOD-240] Removed references to the wallet in rpcserver.go.
* [NOD-240] Began removing btcwalletxxx.go.
* [NOD-240] Got rid of rpcclient/wallet.go and walletsvrcmds.go.
* [NOD-240] Moved GetBestBlockResult to dagsvrresults.go.
* [NOD-240] Finished removing walletsvrXXX.go.
* [NOD-240] Removed wallet stuff from btcctl.
* [NOD-240] Removed a few last things that I've missed.
* [NOD-255] When orphan blocks arrive from netsync - don't write log unless we are in Debug
* [NOD-255] If there are more than K*2 orphans in pool - report as a potential problem anyway
* [NOD-255] Update comment to explain the K*2 figure
* [NOD-237] Implemented transaction mass.
* [NOD-237] Added transaction mass validation to the mempool.
* [NOD-237] Made blockMaxMassMax not rely on MaxBlockPayload.
* [NOD-237] Added comments describing the new constants in validate.go.
* [NOD-237] Changed the default blockmaxmass to 10,000,000.
* [NOD-237] Fixed a comment that erroneously didn't refer to mass.
* [NOD-237] Added comments to ValidateTxMass and CalcTxMass.
* [NOD-237] Renamed "size" to "byte". Made validateBlockMass exit early if validation fails. Fixed unit names in comments. In CalcTxMass, moved summing of mass to the bottom of the function.
* [NOD-237] Instead of ErrMassTooHigh, renamed ErrBlockTooBig and ErrTxTooBig. Replaced wire.MaxBlockPayload with MaxMassPerBlock.
* [NOD-237] Fixed sanity checks related to block size in commands.
* [NOD-237] To use up less memory during testing, made the mass in the "too big" test come from pkScripts rather than input bytes.
* [NOD-237] Added an overflow check to validateBlockMass.
* [NOD-222] Added constant: UnacceptedBlueScore.
* [NOD-222] Made it so that block transactions always have UnacceptedBlueScore.
* [NOD-222] Implemented updating unaccepted UTXO entries with accepted ones in the virtual.
* [NOD-222] Fixed an unclear comment.
* [NOD-222] Fixed diffFromAcceptanceData not receiving the right blue score.
* [NOD-222] Fixed various issues with the implementation. It appears to work now.
* [NOD-222] Removed debug logs.
* [NOD-222] Fixed tests that relied on utxoCollection.String().
* [NOD-222] Fixed TestChainedTransactions.
* [NOD-222] Fixed tests that relied on GetVirtualFromParentsForTest.
* [NOD-222] Fixed having identical entries in toAdd and toRemove.
* [NOD-222] Fixed logic in diffFrom that I previously broke.
* [NOD-222] Fixed a wrong check.
* [NOD-222] Figured out the magical invocation to make everything work.
* [NOD-222] Fixed blockDB tests.
* [NOD-222] Removed debug method.
* [NOD-222] Fixed comments related to setting coinbase maturity to 0.
* [NOD-222] Fixed a typo in a comment.
* [NOD-222] Added a comment that explains the new addition in GetVirtualFromParentsForTest.
* [NOD-222] Added a comment to DiffUTXOSet.Get().
* [NOD-222] Fixed a nuance in DiffUTXOSet.containsInputs.
* [NOD-222] Replaced nonsense in GetVirtualFromParentsForTest with diffFromAcceptanceData.
* [NOD-222] Renamed newVirtualUTXO -> newVirtualPastUTXO.
* [NOD-222] Fixed a comment.
* [NOD-222] Extracted checking utxoCollection with blueScore to a method.
* [NOD-222] Added tests where the same entry is in both toAdd and toRemove.
* [NOD-222] Used Add/RemoveEntry inside diffFromAcceptedTx.
* [NOD-222] Removed superfluous test for UnacceptedBlueScore.
* [NOD-222] Added/Updated comments.
* [NOD-222] Added tests to TestUTXODiffRules.
* [NOD-222] Added appropriate protection against impossible "from"s in diffFrom.
* [NOD-222] Added a comment explaining why we diffFrom acceptanceData in verifyAndBuildUTXO.
* [NOD-222] Fixed comments and equal() in utxoset.
* [NOD-225] Finalize nodes below finality point
* [NOD-225] finalizeNodesBelowFinalityPoint only if dag.lastFinalityPoint is changed
* [NOD-225] change comment in validateParents
* [NOD-225] add string to ErrInvalidParentsRelation error
* [NOD-225] Change comment in validateParents
* [NOD-225] Change comment in validateParents
* [NOD-225] change comment in validateParents
* [NOD-225] Delete diff data from db directly from finalizeNodesBelowFinalityPoint
* [NOD-225] Refactor updateFinalityPoint
* [NOD-208] Added blockBlueScore to UTXOEntry.
* [NOD-208] Added blueBlockScore to NewUTXOEntry.
* [NOD-208] Fixed compilation errors in policy, utxoset, and dag tests.
* [NOD-208] Changed validateBlockRewardMaturity and CheckTransactionInputsAndCalulateFee to use blueScore.
* [NOD-208] Changed CalcBlockSubsidy to use blueScore.
* [NOD-208] Changed SequenceLockActive to use blueScore.
* [NOD-208] Removed ExtractCoinbaseHeight.
* [NOD-208] Removed reference to block height in ensureNoDuplicateTx.
* [NOD-208] Changed IsFinalizedTransaction to use blueScore.
* [NOD-208] Fixed merge errors.
* [NOD-208] Made UTXOEntry serialization use blueScore.
* [NOD-208] Changed CalcPriority and calcInputValueAge to use blueScore.
* [NOD-208] Changed calcSequenceLock to use blueScore.
* [NOD-208] Removed blockChainHeight from UTXOEntry.
* [NOD-208] Fixed compilation errors in feeEstimator. Fixed a bug in the test pool hardness.
* [NOD-208] Fixed oldestChainBlockWithBlueScoreGreaterThan not handling an extreme case.
* [NOD-208] Fixed TestDiffFromTx.
* [NOD-208] Got rid of priority and support of free transactions.
* [NOD-208] Fixed TestProcessTransaction.
* [NOD-208] Fixed TestTxFeePrioHeap.
* [NOD-208] Fixed TestAddrIndex and TestFeeEstimatorCfg.
* [NOD-208] Removed unused rateLimit parameter from ProcessTransaction.
* [NOD-208] Fixed tests that rely on CreateTxChain.
* [NOD-208] Fixed tests that rely on CreateSignedTxForSubnetwork.
* [NOD-208] Fixed TestFetchTransaction.
* [NOD-208] Fixed TestHandleNewBlock. Fixed HandleNewBlock erroneously processing fee transactions.
* [NOD-208] Fixed TestTxIndexConnectBlock.
* [NOD-208] Removed the use of Height() from the fee estimator.
* [NOD-208] Removed unused methods from rpcwebsocket.go.
* [NOD-208] Removed Height from util.Block.
* [NOD-208] Removed ErrForkTooOld. It doesn't make sense in a DAG.
* [NOD-208] Made blockHeap use blueScore instead of height.
* [NOD-208] Removed fee estimator.
* [NOD-208] Removed DAG.Height.
* [NOD-208] Made TestAncestorErrors test chainHeight instead of height.
* [NOD-208] Fixed a couple of comments that were still speaking about block height.
* [NOD-208] Replaced all uses of HighestTipHash with SelectedTipHash.
* [NOD-208] Remove blockNode highest and some remaining erroneous uses of height.
* [NOD-208] Fixed a couple of comments. Fixed outPoint -> outpoint merge error.
* [NOD-208] Fixed a couple more comments.
* [NOD-208] Used calcMinRequiredTxRelayFee instead of DefaultMinRelayTxFee for mempool tests.
* [NOD-208] Renamed mempool Config BestHeight to DAGChainHeight.
* [NOD-208] Fixed a bug in oldestChainBlockWithBlueScoreGreaterThan. Made calcSequenceLock use the node's selected parent chain rather than the virtual block's.
* [NOD-208] Removed chainHeight from blockNode String().
Renamed checkpointsByHeight to checkpointsByChainHeight and prevCheckpointHeight to prevCheckpointChainHeight.
Removed reference to chainHeight in blockIndexKey.
Fixed comments in dagio.go.
* [NOD-208] Removed indexers/blocklogger.go, as no one was using it.
* [NOD-208] Made blocklogger.go log blueScore instead of height.
* [NOD-208] Fixed typo.
* [NOD-208] Fixed comments, did minor renaming.
* [NOD-208] Made a "common sense" wrapper around sort.Search.
* [NOD-208] Fixed comment in SearchSlice.
* [NOD-213] Add customization to txgen
* [NOD-213] Add fee rate as an argument
* [NOD-213] Don't delay transaction emission if there's no need
* [NOD-213] enqueueTransactions -> queueTransactions
* [NOD-213] reuse delay variable
* [NOD-213] Add ExtractGasLimit function
* [NOD-213] Use time.Ticket in sendTransactionLoop
* [NOD-195] Made fee tx implicit
* [NOD-195] Removed redundant checks for fee transactions
* [NOD-195] Add fee tx data into acceptence data and fee data
* [NOD-195] Fix some tests
* [NOD-195] Update Block100000 with new data
* [NOD-195] Fixed remaining tests
* [NOD-195] Save and load feeTx to/from database
* [NOD-195] Remove DisconnectBlock methods from indexers, since they are not used anywhere
* [NOD-195] Add fee tx to addrindex
* [NOD-195] Don't populate inputs for fee transactions
* [NOD-195] Delete feeTxBucket in removeDAGState
* [NOD-195] Got rid of util.FeeTRansactionIndex
* [NOD-201] Implemented the AddSubnetwork CLI tool.
* [NOD-201] Fixed various bugs in AddSubnetwork.
* [NOD-201] Fixed mempool maybeAcceptTransaction verifying gasLimit for a subnetwork registry transaction.
* [NOD-201] Fixed serialization/deserialization bugs in addrIndex.
* [NOD-201] Fixed BlockConfirmationsByHash not handling the zeroHash.
* [NOD-201] Used btclog instead of go log.
* [NOD-201] Made gasLimit a command-line flag. Made waitForSubnetworkToBecomeAccepted only return an error.
* [NOD-201] Filtered out mempool transactions.
* [NOD-201] Fixed embarrassing typos.
* [NOD-201] Added subnetwork registry tx fee + appropriate cli flag.
* [NOD-201] Skipped TXOs that can't pay for registration.
* [NOD-192] Add method to compute confirmations of a single transaction (#306)
* [NOD-192] Implemented txConfirmations.
* [NOD-192] Renamed acceptedBy -> acceptingBlock and ConfirmationsByHash -> BlockConfirmationsByHash.
* [NOD-194 + NOD-199] Update all JSON-RPC methods to use new methods for computing confirmations + Remove the x1.5 factor when counting confirmations in txgen (#309)
* [NOD-194] Connected JSON-RPC commands with new confirmations logic.
* [NOD-194] Fixed failing tests.
* [NOD-194] Removed x1.5 from isTxMatured.
* [NOD-194] Made isTxMatured panic if it receives nil confirmations.
* [NOD-194] Added isInMempool to RPC methods that require it.
* [NOD-194] Fixed a typo.
* [NOD-194] Made the declaration of isInMempool more clear.
* [NOD-194] Removed some unnecessary complexity from isTxMatured.
* [NOD-193] Update Tx-Index to accomodate correct Confirmations structure (#308)
* [NOD-193] Uploaded BlockID to be uint64 in txIndex and addrIndex.
* [NOD-193] Removed the inclusion of current block transactions to txsAcceptanceData.
* [NOD-193] Implemented writing to the tx index txs with the virtual as the accepting block.
* [NOD-193] Added test for txs accepted by the virtual block.
* [NOD-193] Removed the requirement for subnetwork registry transactions to be accepted.
* [NOD-194] Made in-memory the txsAcceptedByVirtual part of txIndex.
* [NOD-193] Optimized txsAcceptedByVirtual initialization.
* [NOD-193] Fixed weird loop in txsAcceptedByVirtual initialization.
* [NOD-190] Fixed merge errors.
* [NOD-189] Made UTXODiff WithDiff and DiffFrom allocate collections with appropriate sizes.
In mempool HandleNewBlock, Replaced removeTransaction loop with removeTransactions.
* [NOD-189] Removed code duplication between removeTransaction and removeTransactions.
* [NOD-189] Fixed a merge error.
* [NOD-189] Fixed another merge error.
* [NOD-189] Renamed removeRedeemers to removeDependants.
* [NOD-189] Removed superfluous check inside removeTransactionWithDiff.
* [NOD-189] Added a comment to removeTransactions detailing what it optimizes.
* [NOD-189] Added documentation to removeTransactionWithDiff and split it into smaller methods.
* [NOD-172] Port EMCH from bchd
* [NOD-172] Fix hdkeychain.TestErrors and add btcec.TestRecoverCompact
* [NOD-172] Make ECMH immutable
* [NOD-172] Fix gofmt errors
* [NOD-172] Add TestMultiset_NewMultisetFromDataSlice and fix Point to be immutable
* [NOD-172] Fix gofmt errors
* [NOD-172] Add test for checking that the Union of a multiset and its inverse is zero
* [NOD-179] Add ECMH Point to all UTXO-structs
* [NOD-179] Fix utxo set tests
* [NOD-179] Fix mempool tests
* [NOD-179] Remove RemoveTxOuts
* [NOD-179] Move serializeBlockUTXODiffData to the top of the file
* [NOD-179] Fix serializeBlockUTXODiffData comment format
* [NOD-179] Fix AddTx comment and name return values
* [NOD-180] Validate utxo commitments
* [NOD-179] Fix TestAcceptingBlock and TestConfirmations to not use the block hash as phantom break even
* [NOD-180] Fix typo
* [NOD-180] move most of the logic in calcUTXOCommitment to UTXOSet.WithTransactions
* [NOD-180] Optionally return error when a transaction in WithTransactions is double spent
* [NOD-180] Rename allowDoubleSpends to ignoreDoubleSpends
* [NOD-172] Port EMCH from bchd
* [NOD-172] Fix hdkeychain.TestErrors and add btcec.TestRecoverCompact
* [NOD-172] Make ECMH immutable
* [NOD-172] Fix gofmt errors
* [NOD-172] Add TestMultiset_NewMultisetFromDataSlice and fix Point to be immutable
* [NOD-172] Fix gofmt errors
* [NOD-172] Add test for checking that the Union of a multiset and its inverse is zero
* [NOD-179] Add ECMH Point to all UTXO-structs
* [NOD-179] Fix utxo set tests
* [NOD-179] Fix mempool tests
* [NOD-179] Remove RemoveTxOuts
* [NOD-179] Move serializeBlockUTXODiffData to the top of the file
* [NOD-179] Fix serializeBlockUTXODiffData comment format
* [NOD-179] Fix AddTx comment and name return values
* [NOD-191] Added selectedPathChainSlice to virtualBlock.
* [NOD-191] Implemented acceptingBlock().
* [NOD-191] Implemented confirmations().
* [NOD-191] Added selectedPathChainSlice tests to TestSelectedPath.
* [NOD-191] Fixed a bug in acceptingBlock(). Written tests for confirmations().
* [NOD-191] Written tests for acceptingBlock().
* [NOD-191] Added test to make sure that acceptingBlock(tip) returns the virtual block.
* [NOD-191] Added a panic if we somehow feed a childless block that isn't the virtual to acceptingBlock.
* [NOD-191] Fixed comments.
* [NOD-191] Fixed a bug in acceptingBlock. Added red block tests for acceptingBlock.
* [NOD-191] Added red block tests for confirmations.
* [NOD-191] Fixed misleading comment and error message.
* [NOD-175] Added BlockAddedNotificationData and sent it instead of just a block on BlockAdded.
* [NOD-175] Added BFWasUnorphaned and raised it when an unorphaned block was to be accepted.
* [NOD-175] Fixed a typo.
* [NOD-175] Made it so that only the mempool gets updated if we're not current or the block was just now unorphaned.
* [NOD-177] Removed references to idMerkleRoot.
* [NOD-177] Generated new genesis hashes.
* [NOD-177] Generated new blk_ blocks.
* [NOD-177] Fixed TestHaveBlock.
* [NOD-177] Fixed The rest of the tests.
* [NOD-177] Fixed a couple of comments and a duplicate test.
* [NOD-177] Fixed blocks1-256.bz2.
* [NOD-172] Port EMCH from bchd
* [NOD-172] Fix hdkeychain.TestErrors and add btcec.TestRecoverCompact
* [NOD-172] Make ECMH immutable
* [NOD-172] Fix gofmt errors
* [NOD-172] Add TestMultiset_NewMultisetFromDataSlice and fix Point to be immutable
* [NOD-172] Fix gofmt errors
* [NOD-172] Add test for checking that the Union of a multiset and its inverse is zero
* [NOD-164] Added validation routine
* [NOD-167] Extracted acceptedIDMerkleRoot calculation to its own method and implemented NextAcceptedIDMerkleRoot.
* [NOD-164] Fixed TestValidateFeeTransaction.
* [NOD-164] Fixed TestFinality.
* [NOD-164] Fixed blk_ tests.
* [NOD-164] Fixed if -> iff in a comment.
* [NOD-164] Minor style changes in comments.
* [NOD-164] Moved validateAcceptedIDMerkleRoot to before its population with the block's own transactions.
Replaced heavy call to verifyAndBuildUTXO in NextBlockFeeTransaction and NextAcceptedIDMerkleRoot with a call to pastUTXO on the virtual.
* [NOD-164] Fixed erroneous comment.
* [NOD-164] Inserted the logic from buildAndSortAcceptedTxs into calculateAcceptedIDMerkleRoot, since the former was meaningless on its own.
* [NOD-164] Changed looping over txsAcceptanceData instead of over node.blues.
* [NOD-13] Added notify_telegram to deploy.sh.
* [NOD-13] Made a test fail for testing.
* [NOD-13] Added some temporary logging.
* [NOD-13] Wrote a nice message for the bot to send.
* [NOD-13] Made the message nicer.
* [NOD-13] Made the message nicer still.
* [NOD-13] Added the build log as an attachment.
* [NOD-13] Actually added the build log as an attachment.
* [NOD-13] Added a delay to allow the build log to properly flush.
* [NOD-13] Disowned notify_telegram.
* [NOD-13] Disowning doesn't work. Using the "at" command instead.
* [NOD-13] Properly using the at command.
* [NOD-13] Actually properly using the at command.
* [NOD-13] Added a couple of prints to see whether the script is even being called.
* [NOD-13] More printouts...
* [NOD-13] Added a command to start atd if it stopped for some reason.
* [NOD-13] Added slashes in multiline echo command.
* [NOD-13] Added quotes where required and removed debug comments.
* [NOD-13] Revert "[NOD-13] Made a test fail for testing."
This reverts commit 9701e30e
* [NOD-13] Added some comments.
* [NOD-144] Use chainHeight in SelectedAncestor, and update all logic that uses it
* [NOD-144] Moved UnminedHeight to blockdag, and updated all references
* [NOD-96] Convert txid to pointer where possible
* [NOD-96] Make msgTx.TxID return a pointer
* [NOD-96] observedTransaction.id -> observedTransaction.txID
* [NOD-152] Stopped pushBlockMsg from sending tip inv to syncing nodes.
* [NOD-152] Fixed restartSyncIfNeeded not restarting if sync is needed.
* [NOD-152] Removed continueHash, as it is no longer required.
* [NOD-140] Converted DNSSeeder to use btclog.
* [NOD-140] Converted MiningSimulator to use btclog.
* [NOD-140] Converted TxGen to use btclog.
* [NOD-140] Fixed log level in handlePanic in txgen.
* [NOD-140] Renamed logger to log everywhere. Removed superfluous flag-setting to go-log.
* [NOD-139] Made processOrphans not turn an error if one of the parents is still missing.
* [NOD-139] Made addOrphanBlock and removeOrphanBlock process all parents instead of only the selected parent.
* [NOD-139] Made addOrphanBlock remove excess orphans by their timestamp rather than their discovery time. Fixed orphans being added more than once.
* [NOD-139] Simplified removal from slice in removeOrphanBlock.
* [NOD-139] Made check for no-orphans-left come before assignment to prevOrphans.
* [NOD-139] Added Timestamp() to util.Block.
* [NOD-139] Fixed merge errors.
* [NOD-149] Store phantom values in the database
* [NOD-149] Explain when zero hash is used when serializing blockNode
* [NOD-149] make deserializeBlockNode return a blockNode
* [NOD-149] use blockNode initializer instead of lots of assignments
* [NOD-142] Updated util.FastLog2Floor to work on uint64
* [NOD-142] Convert height and chainHeight to uint64
* [NOD-142] A couple fixes in comments of TestFastLog2Floor
* [NOD-142] Make spendableOutOffset uint64 too
* [NOD-131] Allow override of dnsseed by command line or config
* [NOD-131] Moved tor.go from connmgr to util/network, to prevent dependancy loop
* [NOD-131] Typo fix
* [NOD-131] Clarify description for --dnsseed cli flag
* [NOD-131] Removed redundant line that somehow got into go.sum
* [NOD-121] Do not handle transaction inputs for reward transactions in SearchRawTransactions RPC call
* [NOD-121] Do not get transaction inputs for fee transactions in SearchRawTransactions RPC call
* [NOD-66] Created TX generator
* [NOD-66] Created transaction generator
* [NOD-66] Improved TX generator against double spend. Created genaddr utility. Refactored
* [NOD-66] Save chenges before branch switch
* [NOD-66] Use log package instead of fmt
* [NOD-66] Fixed/restored docker files
* [NOD-66] Changed according to new WithLock/NoLock convention
* [NOD-122] Handle each message in rpcclient with a separate goroutine
* [NOD-122] Stop listening to new blocks when not mining
* [NOD-122] Made RPC logging in mining simulator more explicit + some styling enhencement
* [NOD-115] Add timeout to rpcclient requests
* [NOD-115] Add timeout of half a second to mining simulator requests
* [NOD-115] Remove redundant allocation of responseChan
* [NOD-114] Added a minimum address amount GetAddrs.
* [NOD-114] Added smallNetwork intervals for when the network is small.
* [NOD-114] Fixed bad minimum address calculation.
* [NOD-95] Added docker file and scripts to run dev instances for debugging
* [NOD-95] Enabled debugging from dockers
* [NOD-95] Removed redundant entrypoint.sh file
* [NOD-95] Removed multi-stage build artifacts from Dockerfile.dev
* [NOD-95] Add --help to run-dev.sh
* [NOD-82] Added RPC files to the BTCD Docker container.
* [NOD-82] Generated new cert and key files that include *.daglabs.com.
* [NOD-82] Added a dagConfig parameter for accepting unroutable IPs.
* [NOD-83] Modify block parents' children only after validation
* [NOD-83] Creating CheckConnectBlockTemplateWithLock with RLock
* [NOD-83] Put updateParentsChildren inside updateParents
* [NOD-83] create updateParentsDiffs function
* [NOD-58] Replace lastBlock with selected tip in version message (#210)
* [NOD-58] Replace lastBlock with selected tip in version message
* [NOD-58] Fix typo in comment
* [NOD-58] Add mutex to SelectedTipHash
* [NOD-58] Remove redundant comment
* [NOD-58] Remove wantStartingHeight from peerStats
* [NOD-58] Remove lock from SelectedTipHash
* Nod 53 change getheaders message to handle new block locator (#213)
* [NOD-53] Change getheaders message to handle the new block locator mechanism
* [NOD-53] Use heap in locateHeaders
* [NOD-53] Create a constructor for each heap direction
* [NOD-57] Check if a node is synced only by timestamps (#214)
* [NOD-60] implement isSyncCandidate (#218)
* [NOD-60] Implement isSyncCandidate
* [NOD-60] Fix typo
* [NOD-65] Fix netsync related tests and remove fields optionality from… (#220)
* [NOD-65] Fix netsync related tests and remove fields optionality from msgversion
* [NOD-65] gofmt rpcserver.go
* [NOD-65] add missing test for verRelayTxFalse
* [NOD-62] Change getblocks message to handle the new block locator mechanism (#219)
* [NOD-62] Change getblocks message to handle the new block locator mechanism
* [NOD-62] Add locateBlockNodes function
* [NOD-68] Adjust orphan parents requesting for a DAG (#222)
* [NOD-68] Adjust orphan parents requesting for a DAG
* [NOD-68] add sendInvsFromRequestedQueue and trigger it when requested blocks slice is empty, or immediatly if we're not in sync mode
* [NOD-68] Prevent duplicates from entering to state.requestQueue and add wrapping locks to addBlocksToRequestQueue
* [NOD-68] Fix Lock -> Unlock in sendInvsFromRequestedQueue
* [NOD-74] Starts syncing again when the current sync peer is done (#225)
* [NOD-74] Starts syncing again when the current sync peer is done
* [NOD-74] Unlock mtx before netsync is restarted
* [NOD-74] Fix name isSyncPeerFree -> isWaitingForBlocks
* [NOD-75] fixing netsync bugs (#227)
* [NOD-74] Starts syncing again when the current sync peer is done
* [NOD-74] Unlock mtx before netsync is restarted
* [NOD-75] Fixing netsync bugs
* [NOD-80] Request block data from block propagation just after you are… (#231)
* [NOD-80] Request block data from block propagation just after you are current
* [NOD-80] Fix adding to both queues in addInvToRequestQueue
* [NOD-81] Start to mine on top of genesis iff all peers selected tip is genesis (#232)
* [NOD-81] Start to mine on top of genesis only if all of your peers' selected tip is genesis
* [NOD-81] Explain forAllPeers/forAllOutboundPeers shouldContinue behaviour in comments
* [NOD-81] Add forAllInboundPeers and add return values for forAllPeers/forAllOutboundPeers/forAllInboundPeers functions
* [NOD-16] Add pushSet to the BlockHeap type
* [NOD-16] Fixed syntax error
* [NOD-78] Use transfer some read-only functions from dagLock.Lock to dagLock.RLock
* [NOD-78] Make mining.NewBlockTemplate lock the dag up until the point it calls CheckConnectBlockTemplate
* [NOD-78] Removed locking from functions that are concurrency-safe, and added WithLock version to those that are not
* [NOD-78] Remove unused isLockHeld from CalcNextBlockVersion
* [NOD-78] Renamed UTXORLock/UTXORUnlock to RLock/RUnlock
* [NOD-78] Revesed dagUnlocket to isDagLocked
* [NOD-77] AddrManager.Good now updates subnetwork in case it was modified
* [NOD-77] Do not update subnetworkID in updateAddr if address already known
* [NOD-77] Restructure case where ka.tried = true for more readable code
* [NOD-77] Some corrections to comments
* [NOD-77] Fixd typo
* [NOD-70] Added GetBlockTemplate method to rpcclient
* [NOD-70] Basic infrastructure for mining simulator
* [NOD-70] Fix txFees in NewBlockTempalte: include value for fee transaction + don't set fee for coinbase = -totalFees.
* [NOD-70] Added capabilities parameter to Client.GetBlockTemplate call
* [NOD-70] Dirty version of mining simulator complete
* [NOD-70] cleaned up mining simulator
* [NOD-70] Added dockerfile to mining simulator
* [NOD-70] Updated base config path of mining simulator to mining_simulator
* [NOD-70] Remove error return from msgblock.AddTransaction - it never returns one
* [NOD-70] Renamed r -> random
* [NOD-70] Move paths initialization of mining simulator to main
* [NOD-70] Cleaned up mining simulator dockerfile
* [NOD-70] Add '--' to tini argument
* [NOD-56] Created devnet
* [NOD-56] Fixed tests
* [NOD-56] Fixed go vet errors
* [NOD-56] Added TestSolveGenesisBlock function
* [NOD-56] Created command line tool for solving genesis blocks
* [NOD-63] Merge BlockAccepted and BlockConnected notifications into BlockAdded + remove BlockDisconnected notifications
* [NOD-63] Many instances of chain->DAG and similar
* [NOD-63] Some more chian -> DAG
* [NOD-48] Update wire.NewMsgTx to recieve all fields in msgTx
* [NOD-48] Fix all compilation errors resulting from modification of wire.NewMsgTx
* [NOD-48] Calculate payloadHash iff subnetworkID is not native
* [NOD-48] Update all places the instantiate wire.MsgTx to use wire.NewMsgTx
* [NOD-48] Remove 'wire.' calls inside wire package
* [NOD-48] Made newMsgTx with all parameters private, and added a few public functions that take various arguments for all common use-cases
* [NOD-48] Explicitly pass SubnetworkIDNative instead of nil to newMsgTx
* [NOD-48] Remove option to pass nil to newMsgTx
* [NOD-42] Split checkFinalityRules and newFinalityPoint
* [NOD-42] Rename connectToDAG -> addBlock + move anything that is not actually connecting block to DAG out of connectBlock
* [NOD-42] Extract methods from PastUTXO
* [NOD-42] Give names to outputs in verifyAndBuildUTXO and propagate name
changes up the call tree
* [NOD-42] Split loop that creates UTXODiff and updates acceptance data into 2 separate methods
* [NOD-42] Removed from applyUTXOChanges any validation logic, moved in any logic related to updating the DAG and renamed to applyDAGChanges
* [NOD-42] Rename: CheckTransactionInputs -> CheckTransactionInputsAndCalculateFee
* [NOD-42] Revise some comments
* [NOD-42] Removed finalityErr constant - it's not needed
* Multiple chain -> dag corrections in comments
* [NOD-42] Removed redundant declaration of feeData
* [NOD-42] Reworded some comments
* [NOD-42] Rename MultiblockTxsAcceptanceData -> MultiBlockTxsAcceptanceData
* [NOD-26] Zero payload for signature hash calculation
* [NOD-26] Zero payload in shallowCopyTx function
* [NOD-26] Zero payload in calcSignatureHash function
* [NOD-26] Updated comment
* [NOD-46] Immediately associate received address to their subnetwork
* [NOD-46] fix comment
* [NOD-46] Disconnect peer if it sends an MsgAddr with no subnetwork
* [NOD-46] Disconnect peer if it sends an MsgAddr with incorrect subnetwork id
* [NOD-46] add parenthesis to condition
* [NOD-46] change order of conditions
* [DEV-379] Refactored checkBlockContext: Mainly extracted methods and re-organized variable use to minimize clutter
* [DEV-379] Simplify the condition according to which we increment blockCount
* [DEV-379] Move all logic to save new block data to separate method
* [DEV-379] Refactored the checking of finality point
* [DEV-379] Minor styling changes
* [DEV-379] Extracted method in applyUTXOChanges subroutines
* [NOD-41] Moved update of finality point to after block was validated + some minor style fixes
* [NOD-41] call dag.checkFinalityRulesAndGetFinalityPoint(node) even if fastAdd
* [NOD-41] Fix in a comment
* [NOD-41] Some methods of dag that could have been just functions converted to function
* [DEV-363] Added testnet seeders
* [DEV-363] Prepare BTCD to work with DNS seeder
* [DEV-363] Finished BTCD part of DNS seeder
* [DEV-363] Reverted protocol changes
* [DEV-363] Reverted protocol changes
* [DEV-363] Reverted protocol changes and fixed tests
* [DEV-363] Added tests and refactored
* [DEV-363] Small refactoring of p2p changes
* [DEV-363] Update NeedMoreAddresses function to take into account full peers + small refactoring
* [DEV-363] Removed IsDNSSeederPeer flag
* [DEV-363] Fixed comment
* [DEV-378] Added feeAccumulator structures
* [DEV-378] Added logic to create and store fee data when validating a block
* [DEV-378] Renamed feeAccumulator to compactFeeData, and all related entities accordingly
* [DEV-378] Converted MsgTx.TxHash() to pointer to hash and not hash
* [DEV-378] Restructured parameters to buildFeeTransaction and related entities
* [DEV-378] Finished the code that calculates fees for blocks
* [DEV-378] Fix TxIndex after changing the structure of AcceptedTxsData
* [DEV-378] For genesis block: Return empty AcceptedTxsData instead of nil
* [DEV-378] Off-by-one error
* [DEV-378] Length of compactFeeData should be determined by specific method, not
* [DEV-378] Multiple bugfixes in tx fee calculation
* [DEV-378] Calculate fee even if fastAdd, to save feeData
* [DEV-378] use IsEqual instead of == when comparing TxHash
* [DEV-378] txindex: if including block is the new block - don't fetch id from DB
* [DEV-378] Fixed a few typos and made some vars consts
* [DEV-378] Re-organized fee functions, removed redundant functions and constants, and revised a few comments
* [DEV-378] Recovered fmt string changes lost in merge
* [DEV-378] Renamed acceptedTxsData and related types and vars to txsAcceptanceData
* [DEV-378] Some comment fixes
* [DEV-378] Remove redundant .ToString()
* [DEV-376] Changed any instance of %v in format strings with a more specific format token
* [DEV-376] Fixed some more wrong formatting strings + removed redundant
cast
* [DEV-376] Added fmt.Sprintf where it was missing
* [DEV-376] use %s for util.Amount, to invoke .String()
* [DEV-376] Some more fixes in format strings
* [DEV-376] fixed mruinvmap_test to expect the correct behaviour
* [DEV-259] Allow to spend genesis coinbase, and use ProcessBlock to add genesis to the DAG like any other block
* [DEV-259] fix IsCurrent to check genesis timestamp if needed
* [DEV-259] add genesisPastUTXO as separate function
* [DEV-316] Update JSON-RPC API to include new transaction fields
* [DEV-316] Fix tests
* [DEV-316] Add txAcceptedVerbose test with subnetwork, gas and paylaod
* [DEV-364] Add fee transactions validation
* [DEV-364] make NextBlockFeeTransactions for creating block templates
* [DEV-364] apply coinbase rules to fee transaction is some cases
* [DEV-364] Add comments
* [DEV-364] put getTXO as separate function
* [DEV-364] Make getParentsFeeData a separate function
* [DEV-364] fix calculateFees
* [DEV-364] force maximum sequence for fee transactions
* [DEV-364] add TestValidateFeeTransactions
* [DEV-364] change fee transaction to be one tx per block rather than one tx for each blue
* [DEV-364] fix tests
* [DEV-364] Use constants instead of inline numbers
* [DEV-336] Split addresses in address manager by subnetwork id
* [DEV-336] Load DAG with subnetwork from the config file
* [DEV-336] Remove redundant checks in updateAddrNew and updateAddrTried
* [DEV-92] Covered blocknode.go with tests.
* [DEV-92] Added test for blockSet.highest. Fixed a bug in it.
* [DEV-92] Added tests for blockSet.subtract and blockSet.addSet.
* [DEV-92] Covered blockset.go with tests.
* [DEV-92] Got rid of some old stuff related to STXOs.
* [DEV-92] Covered error.go with tests.
* [DEV-92] Covered utxoSet with tests.
* [DEV-92] Fixed formatting.
* [DEV-361] Create type TxID as alias to daghash.Hash. Use it for transaction IDs
* [DEV-361] Fixed missed renames
* [DEV-361] Removed usage of zeroHash
* [DEV-361] Fixed more missed renames
* [DEV-319] Implemented transaction data sending logic.
* [DEV-319] Implemented NotifyNewTransactions command validation.
* [DEV-319] Reduced some duplication in notifyForNewTx.
* [DEV-319] Renamed a parameter for clarity.
* [DEV-319] Added a test for marshalling/unmarshalling the new varient of notifyNewTransactions.
* [DEV-319] Added a check in handleNotifyNewTransactions to avoid unnecessary validation.
* [DEV-319] Added comments to explain the initialization of marshalledJSONVerboseFull and marshalledJSONVerbosePartial.
* [DEV-311] Moved subnetwork storage from directly in DAG to subnetworkStore
* Added gas validation in CheckBlockSanity
* [DEV-311] Add SubnetworkStore to last remaining call for CheckBlockSanity
* [DEV-311] Added subnetworkID to config in TestcheckBlockSanity
* [DEV-311] Moved CheckBlockSanity to be method of BlockDAG, and removed subnetworkStore argument
* [DEV-311] Removed SubnetworkID as parameter to CheckBlockSanity
* [DEV-311] Update gas usage before
* [DEV-311] some chain=>DAG updates in comments
* [DEV-311] Removed remaining dag-specific parameters from checkBlockSanity
* [DEV-339] Handling of dependent transactions in mempool
* [DEV-339] Small fixes after code review
* [DEV-339] Fixed compilation
* [DEV-339] Removed extra loop in addTransaction function
* [DEV-339] Changed addTransaction do not loop on inputs second time
* [DEV-332] Added MerkleProof to MsgBlock and rejected full-node/partial-block type misbehaviors.
* [DEV-332] Fixed merge issues.
* [DEV-332] Got rid of MerkleProof. Turns out we no longer need it.
* [DEV-332] Got rid of NTBlockDisconnected, as no one was ever triggering it. (It was part of reorg)
* [DEV-332] Implemented clearing out the payloads of transactions of outgoing blocks for partial nodes.
* [DEV-332] Extracted ConvertToPartial to its own method. Added a test. Added a condition for converting to a partial block.
* [DEV-332] Fixed bad ConvertToPartial condition.
* [DEV-348] Removed a couple of unused methods.
* [DEV-348] Implemented validating incoming transactions for bad partial transactions.
* [DEV-348] Added a (incomplete) filter for propogation of transactions.
* [DEV-348] Implemented filtering inventory by subnetwork.
* [DEV-348] Fixed broken tests.
* [DEV-348] Added test for non-zero payload partial transactions.
* [DEV-348] Added a comment for Config.SubnetworkID.
* [DEV-348] Fixed formatting.
* [DEV-348] Renamed isRemoteTransactionFull to shouldTxBeFull.
* [DEV-348] Added a check for invalid transaction in maybeAcceptTransaction. Added handling for native networks.
* [DEV-348] Fixed formatting.
* [DEV-348] Fixed a bug in transaction validation.
* [DEV-348] Rephrased a comment.
* [DEV-348] Extracted subnetwork compatibility to a method. Wrote a test for it.
* [DEV-348] Removed an unnecessary check over the native subnetwork.
* [DEV-333] Added subnetwork to local peer version message.
* [DEV-333] Fixed broken references.
* [DEV-333] Added serialization/deserialization of the version message.
* [DEV-333] Added rejection for peers with wrong subnetwork.
* [DEV-333] Fixed bad comment.
* [DEV-333] Renamed ChainParams to DAGParams.
* [DEV-333] Fixed partial nodes disconnecting from full nodes.
* [DEV-333] chain -> DAG
* [DEV-333] Possibly fixed outbound msgVersion rules.
* [DEV-333] Combined the two incompatible subnetwork cases into one if.
* [DEV-333] Reformatted the condition.
* [DEV-338] Remove provisional nodes, and panic in case there's an error after verifying the block is valid
* [DEV-338] Improved deffered blockNode cleanup and cosmetic changes
* [DEV-338] Fixed dag.index.SetStatusFlags for parents + cosmetic changes
* [DEV-338] Fixed build
* [DEV-338] Fixed comments
* [DEV-345] Validate that gas and payload are 0 when required by sub-network
* [DEV-345] Remove check for txOut.Value < 0, since txOut.Value is a uint64
* [DEV-345] Added tests for CheckTransactionSanity
* [DEV-345] Remove checks for Gas and Payload validity in wire.MsgTx.Decode
* [DEV-345] Verify that payload in Gas sub-network is always 8 bytes (uint64).
* [DEV-345] Renamed tstCheck{Script/Rule}Error to check{Script/Rule}Error
* [DEV-345] Improved formatting
* [DEV-343] Made sub-network registry use subNetworkIDs.
* [DEV-343] Removed an unnecessary clone.
* [DEV-343] Renamed buildSubNetworkID to txToSubNetworkID. Broke out of a loop when it was known that no further processing is required. Handled error cases from dbGetNetwork separately from the "not-found" case/
* [DEV-343] Added an error case in GasLimit() for where the sub-network is nil.
* [DEV-343] Fixed return nil instead of err. Used a better way to check whether we should continue checking accepted transactions.
* [DEV-312] Take in account subnetwork's GAS limit, when adding
transactions to block. Try to do that optimally.
* [DEV-312] Fixed GAS overusage calculation
* [DEV-337] Make sure that a transaction that uses more gas than the total allowed for sub-network
* [DEV-337] Moved transaction GAS check to mempool
* [DEV-337] Added Unit test for gas usage in transaction
* [DEV-337] Fixed build
* [DEV-337] Fixed tests stuff
* [DEV-337] Removed TODO comment
* [DEV-326] Worked around a parallelism issue in go test. Fixed a test where a file was being deleted while still open.
* [DEV-326] Worked around a timer-related oddity.
* [DEV-326] Fixed closing a file twice.
* [DEV-326] Using pdb.closed instead of an additional variable.
* [DEV-355] Disallowed mining-related RPC commands from running on any sub-network besides the "all" sub-network.
* [DEV-335] Fixed formatting errors.
* [DEV-335] Fixed some more formatting errors.
* [DEV-335] Changed error type to ErrRPCInvalidRequest and rephrased the error message.
* [DEV-335] Fixed formatting.
* [DEV-335] Removed error from getGenerate.
* [DEV-312] Take in account subnetwork's GAS limit, when adding
transactions to block. Try to do that optimally.
* [DEV-312] Fixed GAS overusage calculation
* [DEV-312] Prepare to merge with master
* [DEV-312] Fix after merging
* [DEV-307] Implemented adding pending registry transactions.
* [DEV-307] Implemented a skeleton for the sub-network registry algorithm.
* [DEV-307] Implemented the serialization/deserialization logic for pending sub-network transactions.
* [DEV-307] Implemented marking sub-network registry transactions as successfully registered.
* [DEV-307] Implemented clearing pending sub-network entries and writing registered sub-networks.
* [DEV-307] Added comments for all the dagio functions.
* [DEV-307] Fixed a couple of bugs and added a test for checking serialization/deserialization of pending sub-network transactions.
* [DEV-307] Implemented getting a registered sub-network's gas limit.
* [DEV-307] Updated sub-network announcement transaction validation rules.
* [DEV-307] Specified what is validated.
* [DEV-307] Added initialization for lastSubNetworkID.
* [DEV-307] Renamed extractValidSubNetworkRegistryTxs to validateAndExtractSubNetworkRegistryTxs to better reflect what it does.
* Dev 303 implement block finality (#139)
* [DEV-303] Implement block finality
* [DEV-303] Add finality tests
* [DEV-303] Make finality tests use maybeAcceptBlock
* [DEV-303] Only check finality rules if we are fastAdd mode
* [DEV-303] replace hasBiggerFinalityScoreThan checks with .finalityScore()
* [DEV-307] Forgot to actually create the sub-network buckets.
* [DEV-307] Wrote a full-flow test for sub-network registry and fixed a couple of bugs that it had uncovered.
* [DEV-307] Took firstSubNetworkID out of dagconfig.Params. It's unnecessary there.
* [DEV-307] Added persistance for dag.lastSubNetworkID.
* [DEV-307] Moved sub-network stuff to their own files. Got rid of firstSubNetworkID. Replaced SubNetworkReservedLast with SubNetworkUnreservedFirst. Added ErrSubNetworkRegistry.
* [DEV-307] Added a newline at the end of dag_test.go.
* [DEV-307] Renamed previousFinalityPoint to initialFinalityPoint.
* [DEV-307] Moved sub-network IO stuff to subnetworks.go and subnetworks_test.go.
* [DEV-307] Merged dbPutRegisteredSubNetworkTx and dbRegisterSubNetwork. Fixed a too-big allocation and a block double-processing bug.
* [DEV-307] Simplified the serialized representation of a slice of transactions.
* [DEV-307] Fixed comments.
* [DEV-331] Disallowed node to run in reserved sub-networks
* [DEV-331] Added test to make sure that constant are not modified without modifying help text
* [DEV-314] Added tests for DisasmPC and DisasmScript
* [DEV-314] Re-wrote TestCheckErrorCondition to cover the whole method
* [DEV-314] Fixed error message
* [DEV-330] Covered all methods in daghash with tests
* [DEV-330] Added tests for NewHashFromString
* [DEV-330] Added test case for TestNewHashFromStr that is not the default daghash.Hash value
* [DEV-303] Implement block finality
* [DEV-303] Add finality tests
* [DEV-303] Make finality tests use maybeAcceptBlock
* [DEV-303] Only check finality rules if we are fastAdd mode
* [DEV-303] replace hasBiggerFinalityScoreThan checks with .finalityScore()
* [DEV-308] Enforce tx order in block to be by sub-network ID
* [DEV-308] Use mutable block in TestCheckBlockSanity
* [DEV-308] Fixed comment
* [DEV-308] Fixed TestCheckBlockSanity: use invalid block instead of modifying good block
* [DEV-308] Changed name of block in TestCheckBlockSanity
* [DEV-301] implement blockNode.chainHeight as the height of the selected parent chain
* [DEV-301] add description to TestChainHeight
* [DEV-301] Delete comment on TestChainHeight and expalain about the dag
* [DEV-257] Include transaction acceptance status in getBlock RPC call
* [DEV-257] change txRawResult-accepted to camel case
* [DEV-257] gofmt
* [DEV-257] change getBlock-acceptedtx help entry to getBlock-acceptedTx
* [DEV-257] delete TxRawResult.accepted and instead create TxRawResult.acceptedBy
* [DEV-81] Overwrite maxOpenFiles for testInterface to force tests to check the LRU-mechanism in openFile
* [DEV-81] Added database.UseLogger test
* [DEV-81] Completed coverage of reconcileDB()
* [DEV-81] Added some tests for dbcache
* [DEV-81] Moved init and UseLogger to separate file to make them more easily-testable + added tests
* [DEV-81] Added tests for deleteFile
* [DEV-81] Added tests to cursor.Delete + made sure it returns error when transaction is not writable
* [DEV-81] Moved database/error_test.go from database_test package to database package + added test for IsErrorCode
* [DEV-81] Added tests for handleRollback error-cases
* [DEV-81] Added tests for cursor.skipPendingUpdates
* [DEV-81] Added tests for various cursor edge-cases
* [DEV-81] tx.putKey no longer returns error, because there is no case when it does
* [DEV-81] Added tests to CreateBucket error cases
* [DEV-81] Added tests to bucket.Get and .Delete error cases + .Delete now returns error on empty key
* [DEV-81] Added test for ForEachBucket
* [DEV-81] Added tests to StoreBlock
* [DEV-81] Added test for deleting a double nested bucket
* [DEV-81] Removed log_test, as it is no longer necessary with the logging system re-design
* [DEV-81] Added test to some of writePendingAndCommit error-cases
* [DEV-81] Update references from btcutil to btcd/util
* [DEV-81] Add tests for dbCacheIterator{.Next(), .Prev(), .Key, .Value()} in cases when iterator is exhausted
* [DEV-81] Added tests for ldbIterator placeholder functions
* [DEV-81] Added test name to Error messsages in TestSkipPendingUpdates
* [DEV-81] Begin writing TestSkipPendingUpdatesCache
* [DEV-81] Added error-cases for DBCache.flush() and DBCache.commitTreaps()
* [DEV-81] Use monkey.patch from bou.ke and not from github
* [DEV-81] Rewrote IsErrorCode in both database and txscript packages to be more concise
* [DEV-81] Rename any database.Tx to dbTx instead of tx - to remove confusion with coin Tx
* [DEV-81] Fix typo
* [DEV-81] Use os.TempDir() instead of /tmp/ to be cross-platform
* [DEV-81] use SimNet for database tests + Error if testDB exists after deleting it
* [DEV-81] Removed useLogger - it's redundant
* [DEV-81] Added comment on how CRC32 checksums are calculated in reconcile_test.go
* [DEV-81] Added comment that explains what setWriteRow does
* [DEV-81] Use constant instead of hard-coded value
* [DEV-81] Fixed some typo's + better formatting
* [DEV-242] Modified some help functionality to convert to lowercase camel case instead of just lowercase.
* [DEV-242] Corrected help functionality for struct field names.
* [DEV-242] Corrected help functionality for struct names.
* [DEV-242] Cleaned up toLowercaseCamelCase.
* [DEV-242] Renamed toLowercaseCamelCase to toCamelCase.
* [DEV-242] Converted the rest of the stuff in rpcserverhelp.go to camelCase. Fixed a bug in the camelCase converter.
* [DEV-242] camelCase-ified the last few RPC parameter names.
* [DEV-242] Fixed an off-by-one bug in toCamelCase.
* [DEV-242] Changed back from "jsonRpc" to "jsonrpc".
* [DEV-242] Moved toCamelCase into utils, wrote unit tests for it, and fixed an off-by-one bug.
* [DEV-242] Re-exported DefaultHomeDir because it's required in windows_service.go.
* [DEV-242] Added a comment above DefaultHomeDir to satisfy golint.
* [DEV-242] Formatted config/config.go.
* [DEV-234] add TxAcceptedInBlock and TxBlocks
* [DEV-234] test TxAcceptedInBlock and TxBlocks
* [DEV-234] test TxAcceptedInBlock and TxFirstBlockRegion
* [DEV-234] rename selectedPathSet to selectedPathChain
* [DEV-234] set indexers db as part of index manager initialization
* [DEV-234] remove redudant dag instance in txindex
* [DEV-234] fix TestTxIndexConnectBlock and add DAGParams as part of config in DAGSetup
* [DEV-234] TestTxIndexConnectBlock make K=1 to make calculations easier
* [DEV-234] rename TxAcceptingBlock to BlockThatAcceptedTx
* [DEV-234] update block fields names in txindex_test.go
* [DEV-234] rename selectedPathChain -> selectedPathChainSet
* [DEV-236] add counter to blockdag that will increment for each valid block that is connected to the dag
* [DEV-236] increment dag.blockCount while building block index in initDagState
* [DEV-236] changed dag.BlockCount to return an unsigned value
* [DEV-255] create checkConnectToPastUTXO and move the required functionalities to it from checkConnectBlock
* [DEV-255] get rid of checkConnectBlock
* [DEV-255] rename pNode -> node
* [DEV-255] add comment to describe ErrWithDiff
* [DEV-134] Implement Continuous Integration
Squashed commit:
[5e41d830] Dev 223 fix txindex (#100)
* [DEV-201] In handleGetBlockDAGInfo calculate difficulty by the tip with the lowest bits
* [DEV-202] Move VirtualBlock.GetUTXOEntry to BlockDAG
* [DEV-203] Move VirtualBlock.SelectedTip() to BlockDAG
* [DEV-203] Move VirtualBlock.SelectedTip() to BlockDAG
* [DEV-204] Unexport VirtualBlock() and add CalcMedianTime method for DAG
* [DEV-204] add explanation about difficulty in CurrentBits() comment
* [DEV-204] unexport VirtualBlock type
* [DEV-223] make applyUTXOChanges return pastUTXOResults
* [DEV-223] add bluestxdata for current block as well
* [DEV-223] re-design tx index
* [DEV-223] edit txindex comments
* [DEV-223] rename BluesTxData -> AcceptedTxData, and return from applyUTXOChanges only transactions that got accepted
* [DEV-223] add unit test for txindex
* [DEV-223] fix comments and unite blueTransaction and AcceptedTxData to one type
* [DEV-223] use bucket cursor for dbFetchFirstTxRegion
* [DEV-223] use the same cursor instance for dbFetchFirstTxRegion
* [DEV-223] write in dbFetchFirstTxRegion's comment that it returns the first block region
* [DEV-223] rename type BlueBlockTransaction to TxWithBlockHash
* [DEV-223] add named returned value for applyUTXOChanges
[4c95e293] [DEV-134] Made golint ignore the vendor directory.
[21736dbc] [DEV-134] Renamed ExampleBlockChain_ProcessBlock to ExampleBlockDAG_ProcessBlock to satisfy go vet.
[beea6486] [DEV-134] Removed pushing the built docker to a remove repository. That's unnecessary at this stage.
[bee911ed] [DEV-134] Made all precompilation checks run on everything instead of only the root dir.
[585f92ae] [DEV-134] Added "github.com/pkg/errors" to dep.
[5f02f570] [DEV-134] -vendor-only is written with only one hyphen.
[3eee7f95] [DEV-134] go vet instead of go tool vet.
[0c2d4343] [DEV-134] Split all the pre-compile checks to separate lines to be able to tell which of them is failing.
[780519c8] [DEV-134] Ran gofmt on everything.
[8247146b] Dev 223 fix txindex (#100)
* [DEV-201] In handleGetBlockDAGInfo calculate difficulty by the tip with the lowest bits
* [DEV-202] Move VirtualBlock.GetUTXOEntry to BlockDAG
* [DEV-203] Move VirtualBlock.SelectedTip() to BlockDAG
* [DEV-203] Move VirtualBlock.SelectedTip() to BlockDAG
* [DEV-204] Unexport VirtualBlock() and add CalcMedianTime method for DAG
* [DEV-204] add explanation about difficulty in CurrentBits() comment
* [DEV-204] unexport VirtualBlock type
* [DEV-223] make applyUTXOChanges return pastUTXOResults
* [DEV-223] add bluestxdata for current block as well
* [DEV-223] re-design tx index
* [DEV-223] edit txindex comments
* [DEV-223] rename BluesTxData -> AcceptedTxData, and return from applyUTXOChanges only transactions that got accepted
* [DEV-223] add unit test for txindex
* [DEV-223] fix comments and unite blueTransaction and AcceptedTxData to one type
* [DEV-223] use bucket cursor for dbFetchFirstTxRegion
* [DEV-223] use the same cursor instance for dbFetchFirstTxRegion
* [DEV-223] write in dbFetchFirstTxRegion's comment that it returns the first block region
* [DEV-223] rename type BlueBlockTransaction to TxWithBlockHash
* [DEV-223] add named returned value for applyUTXOChanges
[bff68aa3] [DEV-134] Gave executable permission to deploy.sh
[638a99d9] [DEV-134] Added jenkinsfile and deploy script.
* [DEV-134] Added a robust testing script.
* [DEV-134] Fixed a bash-ism.
* [DEV-134] Disabled testing with coverage for now.
* [DEV-134] Disabled golint and removed removing debug symbols.
* [DEV-134] Disabled aligncheck.
* [DEV-134] Disabled structcheck and varcheck.
* [DEV-134] Added "don't inline functions" to compiler flags for testing.
* [DEV-134] Made build fail if gofmt prints out anything.
* [DEV-134] Fixed misleading comment.
* [DEV-134] Added comments to test.sh.
* [DEV-134] Renamed tm to measure_runtime and removed do_ prefixes from functions.
* [DEV-134] Fixed gofmt line in build script.
* [DEV-134] Fixed gofmt some more.
* [DEV-134] Fixed gofmt not actually failing due to logical or.
* [DEV-91] add TestAddrIndex
* [DEV-91] add TestFeeEstimator
* [DEV-91] rename TestFeeEstimator -> TestFeeEstimatorCfg
* [DEV-91] added TestCount
* [DEV-91] add TestExtractRejectCode in mempool_test.go
* [DEV-91] get rid of fakeErr type and defer unpatching monkey patch
* [DEV-91] use the result of monkey.patch to unpatch
* [DEV-222] Changed type of TxOut.Value and util.Amount to uin64
* [DEV-222] Replaced math.MaxUint64 with 0
* [DEV-222] Fixed comment to reflect uint64 instead of int64
* [DEV-222] Fixed overflow comment
* [DEV-201] In handleGetBlockDAGInfo calculate difficulty by the tip with the lowest bits
* [DEV-202] Move VirtualBlock.GetUTXOEntry to BlockDAG
* [DEV-203] Move VirtualBlock.SelectedTip() to BlockDAG
* [DEV-203] Move VirtualBlock.SelectedTip() to BlockDAG
* [DEV-204] Unexport VirtualBlock() and add CalcMedianTime method for DAG
* [DEV-204] add explanation about difficulty in CurrentBits() comment
* [DEV-204] unexport VirtualBlock type
* [DEV-223] make applyUTXOChanges return pastUTXOResults
* [DEV-223] add bluestxdata for current block as well
* [DEV-223] re-design tx index
* [DEV-223] edit txindex comments
* [DEV-223] rename BluesTxData -> AcceptedTxData, and return from applyUTXOChanges only transactions that got accepted
* [DEV-223] add unit test for txindex
* [DEV-223] fix comments and unite blueTransaction and AcceptedTxData to one type
* [DEV-223] use bucket cursor for dbFetchFirstTxRegion
* [DEV-223] use the same cursor instance for dbFetchFirstTxRegion
* [DEV-223] write in dbFetchFirstTxRegion's comment that it returns the first block region
* [DEV-223] rename type BlueBlockTransaction to TxWithBlockHash
* [DEV-223] add named returned value for applyUTXOChanges
* [DEV-167] Created Dockerfile, dockerignore, and docker-compose.
* [DEV-167] Updated docker-compose to use remote image.
* [DEV-167] Added --addrindex to docker-compose.
* [DEV-167] Switched to testnet and plugged in the correct address.
* [DEV-167] Removed the third and fourth nodes from docker-compose.
* [DEV-167] Exposed RPC port, added rpcuser and rpcpass.
* [DEV-167] Wrangled RPC stuff into shape. Moved docker stuff into btcd/docker.
* [DEV-167] Moved dockerignore back to root. Corrected path for rpc.cert and rpc.key.
* [DEV-167] Added Jenkins job stuff.
* [DEV-167] Added deploy.sh.
* [DEV-167] Removed .travis.yaml and a couple of files that shouldn't be in this branch.
* [DEV-224] Fix getblockcount RPC command to show real data
* [DEV-227] provisionalNode.commit() should not update the original parents if the provisionalNode was created with withRelatives=false
* [DEV-224] remove debug info from TestBlockCount and BlockDAG.BlockCount()
* [DEV-201] In handleGetBlockDAGInfo calculate difficulty by the tip with the lowest bits
* [DEV-202] Move VirtualBlock.GetUTXOEntry to BlockDAG
* [DEV-203] Move VirtualBlock.SelectedTip() to BlockDAG
* [DEV-203] Move VirtualBlock.SelectedTip() to BlockDAG
* [DEV-204] Unexport VirtualBlock() and add CalcMedianTime method for DAG
* [DEV-204] add explanation about difficulty in CurrentBits() comment
* [DEV-204] unexport VirtualBlock type
* [DEV-206] allow block timestamp to be equal to past median time
* [DEV-206] make more specific error messages
* [DEV-206] make independent nodes in TestPastMedianTime
* [DEV-149] Add test case for CVE-2018-17144 (Bitcoin DoS/Double Spend bug)
* [DEV-149] change t.Errorf + return to t.Fatalf
* [DEV-149] fix malformed blocks
* [DEV-149] change test blocks to use simnet genesis
* [DEV-92] Covered lookupPreviousNodes in tests.
* [DEV-92] Covered accept.go in tests.
* [DEV-92] Fixed a typo.
* [DEV-92] Covered blockindex.go in tests.
* [DEV-92] Replaced Errorf + return with Errorf. Added the test case to the error messages.
* [DEV-92] Fixed grammar in a comment.
* [DEV-92] Split casting error checking and ErrorCode checking into separate tests.
* [DEV-92] Improved errors.
* [DEV-92] Made errors a tiny bit more descriptive.
* [DEV-189] invert arguments order or daghash.Less
* [DEV-189] invert arguments order of daghash.Less in blockset.highest
* [DEV-189] change to equivalent condition in Hash.Less to make it prettier
* [DEV-194] change prevblocks order to be from low hash to high hash
* [DEV-189] invert arguments order or daghash.Less
* [DEV-189] invert arguments order of daghash.Less in blockset.highest
* [DEV-189] change to equivalent condition in Hash.Less to make it prettier
* [DEV-169] add to BlockDAG TipHashes() and HighestTipHash() and remove Tiphashes() and SelectedTipHash from VirtualBlock
* [DEV-169] move highest node logic to separate method of blockset
* [DEV-66] fix endless loop in validateParents
* [DEV-166] add TestValidateParents
* [DEV-166] use generateNode in TestValidateParents to avoid repetition
* [DEV-166] use blockDAG.genesis instead of blockDAG.virtual.SelectedTip()
* [DEV-132] change TargetTimePerBlock to 10 seconds
* [DEV-132] change pow limits to regtest limits in all networks, enable cpu mining in testnet, and get rid of dns seeds in testnet
* [DEV-91] add tests for RemoveOrphansByTag
* [DEV-91] Add tests for orphan tx expiration
* [DEV-91] add TestDoubleSpends
* [DEV-91] add TestFetchTransaction
* [DEV-91] added a test for MinHighPriority
* [DEV-91] test unparsable scripts
* [DEV-91] test MaxOrphanTxs=0
* [DEV-91] add TestRemoveTransaction
* [DEV-105] use utxodiff in mempool instead of utxoview
* [DEV-105] get rid of utxoview
* [DEV-105] fix tests to use utxoset
* [DEV-105] remove utxoview type
* [DEV-105] move DagSetup to test_utils.go
* [DEV-105] add comments and add blockdag/test_utils_test.go
* [DEV-105] change checkPoolDoubleSpend to check utxodiff
* [DEV-105] add restoreInputs arg to removeTransaction
* [DEV-91] add restoreInputs arg to removeTransaction in unit tests
* [DEV-105] add restoreInputs arg to removeTransaction
* [DEV-105] give more descriptive names to vars
* [DEV-115] close txChan outside of HandleNewBlock
* [DEV-105] rename DagSetup -> DAGSetup
* [DEV-91] remove IsSpentInDiff
* [DEV-91] fix comment
* [DEV-91] Make IsPushOnlyScript return an error if the script cannot be parsed
* [DEV-91] add more tests for IsPushOnlyScript
* [DEV-91] fix comments
* [DEV-91] fix NewAddressPubKeyHash
* [DEV-91] add tests for RemoveOrphansByTag
* [DEV-91] Add tests for orphan tx expiration
* [DEV-91] add TestDoubleSpends
* [DEV-91] add TestFetchTransaction
* [DEV-91] added a test for MinHighPriority
* [DEV-91] test unparsable scripts
* [DEV-91] test MaxOrphanTxs=0
* [DEV-91] add TestRemoveTransaction
* [DEV-105] use utxodiff in mempool instead of utxoview
* [DEV-105] get rid of utxoview
* [DEV-105] fix tests to use utxoset
* [DEV-105] remove utxoview type
* [DEV-105] move DagSetup to test_utils.go
* [DEV-105] add comments and add blockdag/test_utils_test.go
* [DEV-105] change checkPoolDoubleSpend to check utxodiff
* [DEV-105] add restoreInputs arg to removeTransaction
* [DEV-91] add restoreInputs arg to removeTransaction in unit tests
* [DEV-105] add restoreInputs arg to removeTransaction
* [DEV-105] give more descriptive names to vars
* [DEV-115] close txChan outside of HandleNewBlock
* [DEV-105] rename DagSetup -> DAGSetup
* [DEV-91] remove IsSpentInDiff
* [DEV-91] fix comment
* [DEV-91] Make IsPushOnlyScript return an error if the script cannot be parsed
* [DEV-91] add more tests for IsPushOnlyScript
* [DEV-91] fix comments
* [DEV-105] use utxodiff in mempool instead of utxoview
* [DEV-105] get rid of utxoview
* [DEV-105] fix tests to use utxoset
* [DEV-105] remove utxoview type
* [DEV-105] move DagSetup to test_utils.go
* [DEV-105] add comments and add blockdag/test_utils_test.go
* [DEV-105] add restoreInputs arg to removeTransaction
* [DEV-105] give more descriptive names to vars
* [DEV-115] close txChan outside of HandleNewBlock
* [DEV-105] rename DagSetup -> DAGSetup
* [DEV-75] Removed fetchEntryByHash, which was no longer used.
* [DEV-75] Removed a few functions in manager.go that weren't used by anything.
* [DEV-75] checkConnectBlock will soon not accept a utxoViewpoint, so removed the call to view.Tip() so that it could be deleted.
* [DEV-75] Got rid of tips in UtxoViewpoint.
* [DEV-75] Added the full UTXO set to the virtual block.
* [DEV-75] Implemented UTXO-wrangling when adding a new block.
* [DEV-75] Added isCoinbase to utxoEntry creation.
* [DEV-75] Added blockHeight to utxoEntry creation.
* Implemented fetching the fullUTXOSet from the database.
* [DEV-75] Simplified DAGState because almost all of the state in it was unnecessary. Also got rid of dbDAGState.
* [DEV-75] Made the process around adding a new block, UTXO-wise, much safer.
* [DEV-75] Implemented melding the virtual UTXO diff to the database.
* [DEV-75] Fixed utxoSet loading from the wrong bucket.
* [DEV-75] Began pruning utxoviewpoint.go. Replaced FetchUtxoEntry with a fullUTXOSet-based GetUTXOEntry.
* [DEV-75] Removed fetchUtxos.
* [DEV-75] Moved GetUTXOEntry into the virtual block.
* [DEV-75] Updated IndexManager to not use utxoViewpoints.
* [DEV-75] Fixed some bad login in restoreUTXO involving nodeDiffs.
* [DEV-75] Got rid of the UTXO spend journal, which wasn't used anywhere.
* [DEV-75] Moved some STXO-related validation logic out of connectToDAG and into checkConnectBlock.
* [DEV-75] Renamed UtxoXxx to UTXOXxx. Removed a bunch of methods that were no longer used by anything.
* [DEV-75] Another Utxo -> UTXO rename.
* [DEV-75] Removed IsModified from UTXOView, which was not used anywhere.
* [DEV-75] Renamed nodeDiff to provisionalNode. Added a bunch of comments.
* [DEV-75] Removed the test for genesis in pastUTXO, since it can never happen.
* [DEV-75] Wrote tests for errors in pastUTXO.
* [DEV-75] Wrote tests for errors in restoreUTXO.
* [DEV-75] Improved testErrorThroughPatching.
* [DEV-75] Wrote tests for errors in verifyAndBuildUTXO.
* [DEV-75] Used TipHashes instead of tips().hashes(), fixed comments in a few places, and brought back an error return that I erroneously removed.
* [DEV-75] Removed UTXO set logs.
* [DEV-75] Recreated add/remove/contains functions for utxoCollection.
* [DEV-75] Changed newUTXOEntry to use an object initializer.
* [DEV-75] Changed the UTXO bucket version to 1.
* Added a comment that clarifies that the index is not initialized before initDAGState is called.
* [DEV-75] Renamed GetVirtualBlock to VirtualBlock.
* [DEV-75] Removed superfluous variables.
* [DEV-75] Combined connectBlockToParents with updateParentsDiffs.
* [DEV-75] Removed another superfluous variable.
* [DEV-75] In pastUTXO, first fetch transactions from the database and only then add them.
* [DEV-75] Reworded the comment for commit().
* [DEV-75] Made all the connectUTXO subfunctions not BlockDAG methods.
* [DEV-75] Updated the comment for connectUTXO.
* [DEV-75] Updated the comment explaining why we're creating provisionalNodes.
* [DEV-75] Removed a couple of unnecessary calls to toProvisionalNode.
* [DEV-75] Renamed connectUTXO to applyUTXOChanges.
* [DEV-75] Replaced allProvisionalNodes with provisionalNodeSet, an object that holds provisionalNodes for this particular call to applyUTXOChanges.
* [DEV-75] Changed createProvisionalNode to accept a boolean "withParents" instead of relying on the caller to supply the node's parents or an empty set.
* [DEV-75] Made most applyUTXOChanges subfunctions be methods on provisionalNode.
* [DEV-75] Fixed a couple of bad comments.
* [DEV-75] Added descriptive error messages to callers of applyUTXOChanges.
* [DEV-75] Fixed weird English.
* [DEV-75] Replaced DAGState with a slice of DAG tip hashes.
* [DEV-75] In createProvisionalNode, changed withParents to withRelatives to avoid certain kinds of attacks.
* [DEV-75] Renamed createdProvisionalNode to newProvisionalNode.
* [DEV-75] Added precalculating the amount of transactions inside a new block's blue set.
* [DEV-75] Pruned unnecessary variable.
* [DEV-107] Disallow situation where a parent is also an ancestor of another parent
* [DEV-107] Add comment for validateParents
* [DEV-107] move validateParents to checkBlockContext
* [DEV-107] break validateParents error to 2 lines
* [DEV-107] remove TestProcessBlock
* [DEV-107] fix comment that explains block 3c test
* [DEV-107] remove blk_3C from TestCheckConnectBlockTemplate
* [DEV-76] handleGetBlockTemplateLongPoll to use tips instead of tip
* [DEV-76] move concatHashesToString to daghash package as JoinHashesStrings
* [DEV-76] use strings.Join for JoinHashesStrings
* [DEV-76] rename templateID -> longPollID
* [DEV-76] add function prefix to errors
* [DEV-98] Move script flags from relay rules to consensus
* [DEV-98] remove flags from script_tests.json
* [DEV-98] fix multisig and remove test that assume no minimal data rule
* [DEV-98] rename bip16 bool to isP2sh
* [DEV-98] add sighash type to overly long signature in script_tests.json
* [DEV-98] add test for NUMEQUAL for non equal numbers script_tests.json
* [DEV-98] remove debugging if
* [DEV-98] remove ErrCleanStack from EVAL_FALSE
* [DEV-98] change isP2sh to isP2SH to comply with Go style
* [DEV-98] add ScriptNoFlags to explictly indicate for empty ScriptFlags
* [DEV-98] rename ErrPubKeyType -> ErrPubKeyFormat
* [DEV-98] rename PUBKEYTYPE -> PUBKEYFORMAT
* [DEV-97] Moved github.com/daglabs/btcutil into github.com/daglabs/btcd/btcutil.
* [DEV-97] Updated Gopkg.toml to no longer refer to btcutil.
* [DEV-97] Renamed btcutil to util.
* [DEV-82] break down main to packages
* [DEV-82] separate rpcserver and server
* [DEV-82] Fixed Windows-related code that failed to compile.
* [DEV-82] remove params.go and use only dagconfig.Params
* [DEV-82] fix log.go license
* [DEV-74] Implemented and written tests for utxoIterator.
* [DEV-74] Improved utxoIterator tests.
* [DEV-74] Implemented utxoCollection tests.
* [DEV-74] Implemented utxoDiff and its tests.
* [DEV-74] Implemented utxoSet.
* [DEV -74] Added tests for fullUTXOSet.
* [DEV-74] Added some tests for diffUTXOSet.
* [DEV-74] Wrote tests for diffUTXOSet iterator.
* [DEV-74] Added a negative test for diffUTXOSet.withDiff.
* [DEV-74] Wrote tests for addTx.
* [DEV-74] Wrote a toRemove test for addTx.
* [DEV-74] Changed blockNode.utxoDiff to be of type utxoDiff.
* [DEV-74] Removed superfluous whitespace.
* [DEV-74] Renamed confusing "previousHash" to "hash".
* [DEV-74] Fixed bad test and corrected failing test.
* [DEV-74] Moved confusing "negatives" test to be part of the general utxoCollection test.
* [DEV-74] Removed utxoDiff.inverted.
* [DEV-74] Renamed blockNode.utxoDiff to blockNode.diff.
* [DEV-74] Renamed diff to diffFrom for clarity's sake.
* [DEV-74] Converted the type of utxoCollection from map[daghash.Hash]map[uint32]*wire.TxOut to map[wire.OutPoint]*UtxoEntry.
* [DEV-74] Corrected test names in utxoCollection_test.
* [DEV-74] Removed superfluous utxoCollection iterator and moved utxoIterator into utxoset.go.
* [DEV-74] Renamed variables in utxoset.go.
* [DEV-74] Renamed verifyTx to areInputsInUTXO and removed a superfulous test.
* [DEV-74] Fixed bad test logic in TestDiffUTXOSet_addTx.
* [DEV-74] Added a few comments. Added reference-equals checks to clone functions.
* [DEV-74] Moved utxoCollection and utxoDiff into utxoset.go.
* [DEV-74] Wrote explanations for utxoCollection and utxoDiff tests.
* [DEV-74] Wrote explanations for all utxoSet tests besides addTx.
* [DEV-74] Wrote explanations for TestDiffUTXOSet_addTx.
* [DEV-74] Moved the documentation for utxoDiff into utxoset.go.
* [DEV-74] Wrote an explanation on utxoSet.
* [DEV-75] Found a typo.
* [DEV-75] Renamed dag -> virtual, dagView -> virtualBlock.
* [DEV-75] Renamed newDAGView to newVirtualBlock.
* [DEV-75] Moved queries for the genesis block from virtualBlock to BlockDAG.
* [DEV-75] Got rid of chainView height and findFork.
* [DEV-75] Renamed receivers from c to v.
* [DEV-75] Updated initBlockNode to allow for virtual (headerless) nodes, updated dbDAGState to contain multiple tip hashes, implemented virtualBlock.setTips.
* [DEV-75] Got rid of virtualBlock.equals, which was not used anywhere.
* [DEV-75] Got rid of virtualBlock.tip().
* [DEV-75] Got rid of SetTip everywhere except for tests.
* [DEV-75] Got rid of Next().
* [DEV-75] Got rid of Contains().
* [DEV-75] Got rid of HeightRange(), as no one was using it.
* [DEV-75] Made verifyDAG in rpcserver.go not use block height for iteration.
* [DEV-75] Got rid of the part of Manager.Init() that handled "catching up" for side chains, which allowed me to get rid of BlockDAG.BlockByHeight().
* [DEV-75] Dropped support for the RPC command getblockhash since it was getting blocks by their height.
* [DEV-75] Dropped getnetworkhashps since it was reliant on height, fixed another couple of RPC commands to return nextHashes instead of a nextHash, and got rid of nodeByHeight in virtualBlock.
* [DEV-75] Got rid of setTip().
* [DEV-75] Moved blockLocator() out of virtualBlock and into BlockDAG. Also removed TestLocateInventory().
* [DEV-75] Implemented addTip().
* [DEV-75] Cleaned up virtualblock.go a bit.
* [DEV-75] Erased irrelevant tests in virtualblock_test.go. Moved dag-related tests into dag_test.go.
* [DEV-75] Removed unnecessary nil check.
* [DEV-75] Wrote tests for virtualBlock.
* [DEV-75] Fixed bad test, added explanations to tests.
* [DEV-89] Fixed a comment.
* [DEV-89] Fixed another comment.
* [DEV-89] Removed the section in Manager::Init that handled rolling back indexes to the main chain if their tip is an orphaned fork. This could only happen during reorg, which no longer exists. Also removed BlockDAG::MainChainHasBlock, which was no longer used by anyone.
* [DEV-89] Removed the nil check inside initBlockNode() and amended the one place that called it with nil.
* [DEV-89] Renamed the receiver param for BlockDAG from b to dag.
* [DEV-89] Moved fastLog2Floor from dag.go to btcutil/btcmath.go.
* [DEV-89] Renamed tstTip to testTip.
* [DEV-89] Renamed phanom_test.go to phantom_test.go.
* [DEV-89] Fixed comments, renamed mainChainHeight to dagHeight.
* [DEV-89] Rewrote virtualBlock.addTip().
* [DEV-89] Fixed a comment. (chain -> DAG)
* [DEV-89] Fixed another chain -> DAG comment.
* [DEV-74] Implemented and written tests for utxoIterator.
* [DEV-74] Improved utxoIterator tests.
* [DEV-74] Implemented utxoCollection tests.
* [DEV-74] Implemented utxoDiff and its tests.
* [DEV-74] Implemented utxoSet.
* [DEV -74] Added tests for fullUTXOSet.
* [DEV-74] Added some tests for diffUTXOSet.
* [DEV-74] Wrote tests for diffUTXOSet iterator.
* [DEV-74] Added a negative test for diffUTXOSet.withDiff.
* [DEV-74] Wrote tests for addTx.
* [DEV-74] Wrote a toRemove test for addTx.
* [DEV-74] Changed blockNode.utxoDiff to be of type utxoDiff.
* [DEV-74] Removed superfluous whitespace.
* [DEV-74] Renamed confusing "previousHash" to "hash".
* [DEV-74] Fixed bad test and corrected failing test.
* [DEV-74] Moved confusing "negatives" test to be part of the general utxoCollection test.
* [DEV-74] Removed utxoDiff.inverted.
* [DEV-74] Renamed blockNode.utxoDiff to blockNode.diff.
* [DEV-74] Renamed diff to diffFrom for clarity's sake.
* [DEV-74] Converted the type of utxoCollection from map[daghash.Hash]map[uint32]*wire.TxOut to map[wire.OutPoint]*UtxoEntry.
* [DEV-74] Corrected test names in utxoCollection_test.
* [DEV-74] Removed superfluous utxoCollection iterator and moved utxoIterator into utxoset.go.
* [DEV-74] Renamed variables in utxoset.go.
* [DEV-74] Renamed verifyTx to areInputsInUTXO and removed a superfulous test.
* [DEV-74] Fixed bad test logic in TestDiffUTXOSet_addTx.
* [DEV-74] Added a few comments. Added reference-equals checks to clone functions.
* [DEV-74] Moved utxoCollection and utxoDiff into utxoset.go.
* [DEV-74] Wrote explanations for utxoCollection and utxoDiff tests.
* [DEV-74] Wrote explanations for all utxoSet tests besides addTx.
* [DEV-74] Wrote explanations for TestDiffUTXOSet_addTx.
* [DEV-74] Moved the documentation for utxoDiff into utxoset.go.
* [DEV-74] Wrote an explanation on utxoSet.
* [DEV-74] Moved diffChild next to diff, improved their comments, moved the explanations for diffFrom and withDiff to the appropriate methods, got rid of utxoIterator, and renamed areInputsInUTXO to containsInputs.
* [DEV-74] Replaced boring old-fashioned reference equality with special, fancy reference equality for maps, slices, and channels.
* [DEV-74] Wrote additional explanations for test cases.
* [DEV-62] add phantom constructs to blocknode
* [DEV-62] add phantom constructs to blocknode
* [DEV-72] write blues()
* [DEV-72] write blues()
* [DEV-72] write blues()
* [DEV-62] add comments to new phantom constructs in blocknode
* Fixed dbIndexConnectBlock. (#33)
* Fixed dbIndexConnectBlock.
* Removed redundant check in storeFilter.
* Created a new method to BlockHeader: IsGenesis.
* [DEV-71] Implement BlockHeap (#35)
* [DEV-71] Implemented BlockHeap.
* [DEV-71] Removed irrelevant comment.
* [DEV-71] Renamed variables in Pop() and split Less() to multiple lines.
* [DEV-72] write blues()
* [DEV-72] write blues()
* [DEV-72] write blues()
* [DEV-72] write blues tests
* [DEV-72] write blues tests
* [DEV-72] remove relevant past
* [DEV-72] write blues tests
* [DEV-72] write blues tests
* [DEV-72] write blues tests
* [DEV-72] write functions to order blockSet by hash and write blue tests
* [DEV-72] add secret mining and censorship attack tests
* [DEV-72] remove prints
* [DEV-72] remove K from dagconfig.Params
* [DEV-72] remove K from dagconfig.Params
* [DEV-72] change blueScore to uint64
* [DEV-72] block V was missing, so renamed w -> v, x -> w etc
* [DEV-72] use node.String instead of %v
* [DEV-72] block V was missing, so renamed w -> v, x -> w etc
* [DEV-72] add K to dagconfig.Params, and add expected reds to all phantom tests
* [DEV-72] set K=10 and add comments to phantom and phantom tests
* [DEV-72] fix formatting and add comments to TestPhantom
* [DEV-72] fix grammar
* [DEV-77] Added tests for ScriptFreeList
* [DEV-77] Fixed formatting in BlockHeader .SelectedPrevBlock and IsGenesis
* [DEV-77] Restructured returnScriptBuffers to be more idiomatic and testable
* [DEV-77] Added tests for binaryFreeList
* [DEV-77] Added test-cases to TestVarIntWireErrors to check for errors when writing the varint length
* [DEV-77] Added MsgSendHeaders and MsgFeeFilter test-cases for TestMessage
* [DEV-77] Added test for BlockHeader.IsGenesis()
* [DEV-77] using binaryFreeListMaxItems instead of freeListMaxItems in TestBinaryFreeList
* [DEV-77] Fixed error message copy-paste typo
* [DEV-42] Fixed rpcserverhelp.go.
* [DEV-42] Fixed GetBlockHeaderVerboseResult, GetBlockVerboseResult, and GetBlockTemplateResult.
* [DEV-42] Fixed rpcserver.go.
* [DEV-42] Got rid of the rescan command. Apparently it was deprecated and succeeded by the rescanBlocks command.
* [DEV-42] Modified handleRescanBlocks to satisfy compilation.
* [DEV-42] Fixed failing tests in rpcserver.go and rpcserverhelp.go.
* [DEV-42] Removed tests for rescan.
* [DEV-42] Removed rescan from notify.go.
* [DEV-42] Fixed a couple of comments I've missed.
* [DEV-36] Fixed import.go.
* [DEV-65] Renamed BestState to State and GetSnapshot to GetCurrentState.
* [DEV-65] Implemented State and SelectedTip, updated dagState (the serializable version of State), and updated State creations.
* [DEV-65] Fixed references to GetCurrentState and fixed tests.
* [DEV-40] Created a new method AreEquals for comparing hash slices. Updated mining package to work with primitive DAG.
* [DEV-65] Renamed State to DAGState, corrected comments, created a new thread-safe method setDAGState, fixed broken references I missed.
* [DEV-65] Converted dagState (the object) to dbDAGState, and made it serialize/deserialize into/out of JSON.
* [DEV-65] Renamed Txns to Txs and fixed a couple of comments.
* [DEV-65] Renamed some more "Txns" and removed some dead code.
* [DEV-36] Fixed findcheckpoint.go.
* [DEV-36] Fixed indentation.
* [DEV-36] Fixed weird phrasing.
* [DEV-36] Fixed some more weird phrasing.
* [DEV-36] Removed haveBlocks variable from the function HaveBlocks.
* [DEV-65] Renamed BestState to State and GetSnapshot to GetCurrentState.
* [DEV-65] Implemented State and SelectedTip, updated dagState (the serializable version of State), and updated State creations.
* [DEV-65] Fixed references to GetCurrentState and fixed tests.
* [DEV-40] Created a new method AreEquals for comparing hash slices. Updated mining package to work with primitive DAG.
* [DEV-65] Renamed State to DAGState, corrected comments, created a new thread-safe method setDAGState, fixed broken references I missed.
* [DEV-65] Converted dagState (the object) to dbDAGState, and made it serialize/deserialize into/out of JSON.
* [DEV-65] Renamed Txns to Txs and fixed a couple of comments.
* [DEV-65] Renamed some more "Txns" and removed some dead code.
* [DEV-40] Fixed formatting errors...
* [DEV-40] Reworded confusing test failure messages.
* [DEV-40] Fixed merge error.
* [DEV-40] Fixed style for AreEqual and TestAreEqual.
* [DEV-65] Renamed BestState to State and GetSnapshot to GetCurrentState.
* [DEV-65] Implemented State and SelectedTip, updated dagState (the serializable version of State), and updated State creations.
* [DEV-65] Fixed references to GetCurrentState and fixed tests.
* [DEV-65] Renamed State to DAGState, corrected comments, created a new thread-safe method setDAGState, fixed broken references I missed.
* [DEV-65] Converted dagState (the object) to dbDAGState, and made it serialize/deserialize into/out of JSON.
* [DEV-65] Renamed Txns to Txs and fixed a couple of comments.
* [DEV-65] Renamed some more "Txns" and removed some dead code.
* [DEV-18] changed CSV/CLTV to be regular op codes, and returned nop2 and nop3 to be regular nops
* [DEV-18] remove csv/cltv flags - part 1
* [DEV-18] remove csv/cltv flags - part 2
* [DEV-18] remove csv/cltv activation rules
* [DEV-18] remove csv/cltv activation rules
* [DEV-18] csv_fork_test fixes
* [DEV-18] readd chain params
* [DEV-18] readd chain params and remove csv activation rules
* [DEV-18] returned build flags to integration test
* [DEV-18] make csv/cltv to pop the the first element of the stack instead of peeking it
* [DEV-18] fix comments related to CSV/CLTV to remove any reference to soft fork
* [DEV-18] fix comments related to CSV/CLTV to remove any reference to soft fork
* [DEV-18] rename csv_fork_test.go to csv_fork.go
* [DEV-18] change mTx location
* [DEV-18] remove BIP0065Height
* [DEV-18] add function isUpgradableNop for readability
* [DEV-45] Updated the BlockDB files to contain blocks that could be deserialized.
* [DEV-45] Fixed TestHaveBlock (finally)
* [DEV-45] Fixed TestLocateInventory and everything that's reliant on chainview_test::chainedNodes.
* [DEV-45] Fixed TestChainViewNil.
* [DEV-45] Fixed TestNotifications.
* [DEV-45] Fixed ExampleBlockChain_ProcessBlock.
* [DEV-45] Fixed TestCheckBlockScripts.
* [DEV-45] Fixed TestCheckConnectBlockTemplate.
* [DEV-45] Renamed the BlockDBs to their original names.
* [DEV-45] Skipping TestFullBlocks for until we have implemented Phantom.
* [DEV-45] Deleted a couple of methods that are no longer used. (They were previously used for reorganization)
* [DEV-56] Remove any occurance of OP_CODESEPARATOR
* [DEV-56] Modified sighash tests to exclude OP_CODESEPARATOR
* [DEV-56] Fixed sighash.json to not include any codeseparators at all
* [DEV-56] Reformatted sighash.json
* [DEV-56] Rename any instance of subScript to script
* [DEV-56] Extracted check for opUnknown to separate function
* [DEV-58] removed addition of multisig dummy in signMultiSig
* [DEV-58] Removed temporary code for test cleanup
* [DEV-56] Remove any occurance of OP_CODESEPARATOR
* [DEV-56] Modified sighash tests to exclude OP_CODESEPARATOR
* [DEV-56] Fixed sighash.json to not include any codeseparators at all
* [DEV-56] Reformatted sighash.json
* [DEV-56] Rename any instance of subScript to script
* [DEV-56] Extracted check for opUnknown to separate function
* [DEV-58] Removed temporary code for test cleanup
* [DEV-50] Remove the Multisig bug requiring a dummy push
* [DEV-53] SigHashSingle now errors when the index is wrong
* [DEV-53] Fixed tests for SIGHASH_SINGLE
* [DEV-55] Removed FindAndDelete functionality that deletes sognatures
* [DEV-55] Removed some more redundant tests
* [DEV-55] Fixed redundant comment
* [DEV-50] Remove the Multisig bug requiring a dummy push
* [DEV-53] SigHashSingle now errors when the index is wrong
* [DEV-53] Fixed tests for SIGHASH_SINGLE
* [DEV-53] Removed redundant part of comment
* [DEV-50] Remove the Multisig bug requiring a dummy push
* [DEV-50] Removed redundant comment, since dummy bug was fixed
* [DEV-50] Removed some more dummy-related comments and error message
* [DEV-27] Fixed dbtool to work with multiple previous blocks
* [DEV-27] Fix Example_blockStorageAndRetrieval expected bytes
* [DEV-27] Added generator for testdata, and generated new testdata to conform to DAG
* [DEV-27] Fixed FetchBlockHeader(s) and database interface test for DAG
This modifies the utxoset in the database and related UtxoViewpoint to
store and work with unspent transaction outputs on a per-output basis
instead of at a transaction level. This was inspired by similar recent
changes in Bitcoin Core.
The primary motivation is to simplify the code, pave the way for a
utxo cache, and generally focus on optimizing runtime performance.
The tradeoff is that this approach does somewhat increase the size of
the serialized utxoset since it means that the transaction hash is
duplicated for each output as a part of the key and some additional
details such as whether the containing transaction is a coinbase and the
block height it was a part of are duplicated in each output.
However, in practice, the size difference isn't all that large, disk
space is relatively cheap, certainly cheaper than memory, and it is much
more important to provide more efficient runtime operation since that is
the ultimate purpose of the daemon.
While performing this conversion, it also simplifies the code to remove
the transaction version information from the utxoset as well as the
spend journal. The logic for only serializing it under certain
circumstances is complicated and it isn't actually used anywhere aside
from the gettxout RPC where it also isn't used by anything important
either. Consequently, this also removes the version field of the
gettxout RPC result.
The utxos in the database are automatically migrated to the new format
with this commit and it is possible to interrupt and resume the
migration process.
Finally, it also updates the tests for the new format and adds a new
function to the tests to convert the old test data to the new format for
convenience. The data has already been converted and updated in the
commit.
An overview of the changes are as follows:
- Remove transaction version from both spent and unspent output entries
- Update utxo serialization format to exclude the version
- Modify the spend journal serialization format
- The old version field is now reserved and always stores zero and
ignores it when reading
- This allows old entries to be used by new code without having to
migrate the entire spend journal
- Remove version field from gettxout RPC result
- Convert UtxoEntry to represent a specific utxo instead of a
transaction with all remaining utxos
- Optimize for memory usage with an eye towards a utxo cache
- Combine details such as whether the txout was contained in a
coinbase, is spent, and is modified into a single packed field of
bit flags
- Align entry fields to eliminate extra padding since ultimately
there will be a lot of these in memory
- Introduce a free list for serializing an outpoint to the database
key format to significantly reduce pressure on the GC
- Update all related functions that previously dealt with transaction
hashes to accept outpoints instead
- Update all callers accordingly
- Only add individually requested outputs from the mempool when
constructing a mempool view
- Modify the spend journal to always store the block height and coinbase
information with every spent txout
- Introduce code to handle fetching the missing information from
another utxo from the same transaction in the event an old style
entry is encountered
- Make use of a database cursor with seek to do this much more
efficiently than testing every possible output
- Always decompress data loaded from the database now that a utxo entry
only consists of a specific output
- Introduce upgrade code to migrate the utxo set to the new format
- Store versions of the utxoset and spend journal buckets
- Allow migration process to be interrupted and resumed
- Update all tests to expect the correct encodings, remove tests that no
longer apply, and add new ones for the new expected behavior
- Convert old tests for the legacy utxo format deserialization code to
test the new function that is used during upgrade
- Update the utxostore test data and add function that was used to
convert it
- Introduce a few new functions on UtxoViewpoint
- AddTxOut for adding an individual txout versus all of them
- addTxOut to handle the common code between the new AddTxOut and
existing AddTxOuts
- RemoveEntry for removing an individual txout
- fetchEntryByHash for fetching any remaining utxo for a given
transaction hash
This commit adds a logic to the addSpentRequests that inspects the
mempool for any spends of the outputs. Before this commit, a spend would
only be checked when a transaction was first accepted into the mempool.
The index will hold three types of entries for each filter type, block
pair: filter, header, and hash. Since they all have similar methods
and implementations, refactor to reduce duplication.
In this commit, we modify the fee estimation to use vsize as a base
rather than size. A recent commit landed to track the fee rate using
vsize in the mempool, and also correct some incorrect unit math. This is
a follow up to that commit to ensure that fee estimation is uniform
throughout.
This commit changes the value of bytesPerKb to 1000 from 1024.
This is done to ensure consistency between the fee estimator
and the mempool, where the feeRate is set to
fee * 1000 / serializedSize
This commit extends the work started by roasbeef in the
previous commit to bring full cancellation of pending
connection requests. It also adds minor refactors to
channel send/receives to help cleanup potentially
lingering go routines.
This commit adds the ability for callers to remove pending connections
via a call to the Remove() method. With this change, upstream users of
this package can use the connmgr for more elaborate connectivity needs
as they can now cancel pending connections that are no longer needed.
new txs that it observes. The block manager alerts the fee estimator
of new and orphaned blocks.
Check for invalid state and recreate FeeEstimator if necessary.
Rollback takes a block hash rather than a BlockStamp.
Increase rounds in TestEstimateFeeRollback to test dropping txs that have been in the mempool too long.
Since the tx hash has moved to the basic filter, generating an extended filter
can result in `ErrNoData`. This is handled by writing a nil filter and giving
it a zero hash.
The cfilter BIP specifies that the filter type is a uint8. The
current code encodes it correctly on the wire, but everywhere else,
it's treated as a boolean (false for basic filter, true for
extended). This commit corrects that to account for possible
additional filter types in the future. All package changes are
done in one commit as they're all interdependent. The following
packages are updated:
* blockchain/indexers
* btcjson
* peer
* wire
* main (server.go and rpcserver.go)
The btclog package has been changed to defining its own logging
interface (rather than seelog's) and provides a default implementation
for callers to use.
There are two primary advantages to the new logger implementation.
First, all log messages are created before the call returns. Compared
to seelog, this prevents data races when mutable variables are logged.
Second, the new logger does not implement any kind of artifical rate
limiting (what seelog refers to as "adaptive logging"). Log messages
are outputted as soon as possible and the application will appear to
perform much better when watching standard output.
Because log rotation is not a feature of the btclog logging
implementation, it is handled by the main package by importing a file
rotation package that provides an io.Reader interface for creating
output to a rotating file output. The rotator has been configured
with the same defaults that btcd previously used in the seelog config
(10MB file limits with maximum of 3 rolls) but now compresses newly
created roll files. Due to the high compressibility of log text, the
compressed files typically reduce to around 15-30% of the original
10MB file.
In this commit, we update the glide.lock file to be pinned against the
latest btcutil commit hash. btcutil has recently been updated to pull in
all changes from roasbeef's fork. Notably, it now includes the code
necessary for creating GCS filters (BIP 158).
Go 1.10 made some changes such that json.Unmarshal can no longer
unmarshal into exported fields that are themselves embedded via an
uninitialized unexported pointer.
Since rpcclient previously relied on this behavior, this updates the
client to create the pointers before unmarshalling into the struct.
The block index now tracks the set of dirty block nodes with status
changes that haven't been persisted and flushes the changes to the DB
at the appropriate times.
Now that all headers are stored in the in-memory index, the database
bucket managed by blockchain, and in the flat files, it makes sense to
drop the redundant data from the block index bucket in ffldb.
To avoid modifying the database interface, this reimplements
FetchBlockHeader(s) to use header data stored in flat files. This can
be trivially implemented by delegating to FetchBlockRegion.
Currently only the blocks in the active chain are loaded into the
block index on initialization. This instead iterates over the entire
block index bucket in LevelDB and loads all nodes.
The bucket contains block headers keyed by the block height encoded as
big-endian concatenated with the block hash. This allows block headers
to be fetched from the DB in height order with a cursor.
These method allows safe concurrent access to reading and modifying
block node statuses. When block statuses get persisted in a later
change, the setter methods can be used to mark block nodes as dirty.
Each node in the block index records some flags about its validation
state. This is just stored in memory for now, but can save effort if
attempting to reconnect a block that failed validation or was
disconnected.
Previously, rpctest would start a btcd node using the btcd executable
in the environment PATH. This caused difficult-to-find issues where
the code would be tested against an older version of btcd, or another
fork entirely. Now it compiles btcd the first time it is needed and
uses that fresh version when launching nodes.
This was only used to test block proposals, which has been changed to
instead use CheckConnectBlockTemplate. The flag complicated the
implementation of some chain processing routines and would be
difficult to implement with headers-first syncing.
This renames CheckConnectBlock to CheckConnectBlockTemplate and
modifies it to be easily consumable by the getblocktemplate RPC
handler. Performs full block validation now instead of partial
validation.
This propagates the interrupt channel through to blockchain and the
indexers so that it is possible to interrupt long-running operations
such as catching up indexes.
The helper function parseListeners has been changed to return a
slice of net.Addrs with Network() returning tcp4 or tcp6 instead of
returning two slices of IPv4 and IPv6 addresses to simplify calling
code. Also improves how local addresses are added to the address
manager when listening on wildcard addresses.
Also splits some newServer logic into new method initListeners.
The modifies the encoding of witness stacks in JSON responses to use a
slice of strings instead of a single space-separated string for
compatibility with Core.
This updates the README to remove the mailing lists which no longer
exist and to link the GPG public key used to sign the release tags
directly from the repository instead of the no longer active website.
This update adds additional callsite logging options via btclog and
fixes an error with the rotator package that caused it to stop running
when creating any log messages larger than 4096 bytes.
While here, switch to the new Write method of the Rotator object as
this is more efficient than using the Reader interface with a pipe.
Changes from @jrick.
This refactors the code that locates blocks (inventory discovery) out of
server and into blockchain where it can make use of the new much more
efficient chain view and more easily be tested. As an aside, it really
belongs in blockchain anyways since it's purely dealing with the block
index and best chain.
Since the majority of the network has moved to header-based semantics,
this also provides an additional optimization to allow headers to be
located directly versus needing to first discover the hashes and then
fetch the headers.
The new functions are named LocateBlocks and LocateHeaders. The former
returns a slice of located hashes and the latter returns a slice of
located headers.
Finally, it also updates the RPC server getheaders call and related
plumbing to use the new LocateHeaders function.
A comprehensive suite of tests is provided to ensure both functions
behave correctly for both correct and incorrect block locators.
This modifies calcSignatureHash to use a shallow copy of the transaction
versus a deep copy since the actual scripts themselves are not modified
and therefore don't need to be copied.
This is being done because profiling the most overall allocated space
shows that the deep copy performed in calcSignatureHash accounts for
nearly 20% of all allocations on a synced running instance. Also,
copying all of the additional data makes it more time consuming as well.
With this change, that figure drops from ~20% to ~5% of all allocations.
The following benchmark shows the relative speedups and allocation
reduction as a result of the optimization on my system. In particular,
the changes result in approximately a 15% speedup and a whopping 99.89%
reduction in allocations when using a large transaction with thousands
of inputs which was the worst case scenario.
benchmark old allocs new allocs delta
--------------------------------------------------------------------
BenchmarkCalcSignatureHash 11151 12 -99.89%
benchmark old ns/op new ns/op delta
--------------------------------------------------------------------
BenchmarkCalcSignatureHash 3599845 3056359 -15.10%
- Remove inMainChain from block nodes since that can now be efficiently
determined by using the chain view
- Track the best chain via a chain view instead of a single block node
- Use the tip of the best chain view everywhere bestNode was used
- Update chain view tip instead of updating best node
- Change reorg logic to use more efficient chain view fork finding logic
- Change block locator code over to use more efficient chain view logic
- Remove now unused block-index-based block locator code
- Move BlockLocator definition to chain.go
- Move BlockLocatorFromHash and LatestBlockLocator to chain.go
- Update both to use more efficient chain view logic
- Rework IsCheckpointCandidate to use block index and chain view
- Optimize MainChainHasBlock to use chain view instead of hitting db
- Move to chain.go since it no longer involves database I/O
- Removed error return since it can no longer fail
- Optimize BlockHeightByHash to use chain view instead of hitting db
- Move to chain.go since it no longer involves database I/O
- Removed error return since it can no longer fail
- Optimize BlockHashByHeight to use chain view instead of hitting db
- Move to chain.go since it no longer involves database I/O
- Removed error return since it can no longer fail
- Optimize HeightRange to use chain view instead of hitting db
- Move to chain.go since it no longer involves database I/O
- Optimize BlockByHeight to use chain view for main chain check
- Optimize BlockByHash to use chain view for main chain check
The purpose is to remove the dependency of blockmanager on serverPeer,
which is defined in the main package. Instead, we split out some of
the fields from serverPeer into a separate struct called peerSyncState
in blockmanager.go. While they are in the same package now, this
change makes it easier to move blockManager into its own package along
with peerSyncState. The blockManager tracks a map of Peer pointers to
the peer state and keeps it updated as peers connect and disconnect.
This removes the DisableVerify function and related state since nothing
uses it anymore since the command line option was removed. It is a
remnant of initial development.
This exposes the ability to more efficiently create a block locator from
a chain view for a given block node by using their ability to do O(1)
lookups.
It also adds tests to ensure the behavior is correct.
This significantly optimizes and simplifies the generation of block
locators by making use of the fact that all block nodes are now in
memory and therefore it is no longer necessary to consult the database
for the hashes or worry about issues related to dynamic loading of nodes.
Also, it slightly modifies the algorithm so that the doubling doesn't
start for one additional iteration in order to mirror other prominent
clients on the network. Due to the way block locators are used, this
does not change any semantics in terms of requesting and locating
blocks.
Finally, the semantics of BlockLocatorFromHash have been changed to
return a locator for the current tip in the case the hash is unknown.
This is far preferable since only including the passed block hash, when
it isn't known, could end up leading to causing a redownload of the
entire chain under certain circumstances.
Putting the test code in the same package makes it easier for forks
since they don't have to change the import paths as much and it also
gets rid of the need for internal_test.go to bridge.
While here, remove the reorganization test since it is much better
handled by the full block tests and is no longer needed and do some
light cleanup on a few other tests.
The full block tests had to remain in the separate test package since it
is a circular dependency otherwise. This did require duplicating some
of the chain setup code, but given the other benefits this is
acceptable.
This introduces the concept of a synthetic block chain that can be used
in the tests to avoid needing setup a full blown chain instance with a
database and generate valid blocks and converts the sequence lock tests
in TestCalcSequenceLock to use it.
Not only does this speed up the test execution time, but it allows the
dependency on rpctest to be removed which will allow the sequence locks
tests to be consolidated into the main package without creating a
circular dependency.
This modifies the function to set the tip in the new chainview code to
bulk copy existing nodes when it needs to expand the cap rather than
simply creating a new empty slice and allowing the walk code below it to
repopulate it. This is a nice optimization since, in practice, most of
the time expanding the cap is only required when the active chain is
being extended after having run for a while which means the end result
is that it will be able to bulk copy all the nodes and just add the most
recent one versus having to walk them all and add them back.
Also, while here expand the tests for setting the tip to ensure the
nodes contained in the resulting view are correct after forcing the
resizes and correct a bug they exposed where changing between a
longer-shorter-longer chain where the longer chain is the same chain
could result in not populating the view correctly.
Finally, update the fake nodes generated by the tests to use a
nonce generated by a deterministic prng in order to ensure the hashes of
all fake nodes are unique, but reproducible.
This regenerates the precomputed secp256k1 byte points used to optimize
scalar multiplication. This should have been done as part of the
normalization correction.
This implements a new type in the blockchain package that takes
advantage of the fact that all block nodes are now in memory to provide
a flat view of a specific chain of blocks (a specific branch of the
overall block tree) from a given tip all the way back to the genesis
block along with several convenience functions such as efficiently
comparing two views, quickly finding the fork point (if any) between two
views, and O(1) lookup of the node at a specific height.
The view is not currently used, but the intent is that the code will be
refactored to make use of these views to simplify and optimize several
areas such as best chain selection and reorg logic and finding successor
nodes. They will also greatly simplify the process of disconnecting the
download logic from the connection logic.
A comprehensive suite of tests is provided to ensure the chain views
behave correctly.
This cleans up the test in TestCalcSequenceLock in the following ways:
- Use calculated values instead of magic value so it is easier to update
the tests as needed
- Make tests match the comments
- Change comments to be more consistent and fix some grammar errors
- Set mempool flag for unconfirmed tx tests since they are intended to
mimic transactions in the mempool
This commit contains the entire btcrpcclient repository along with
several changes needed to move all of the files into the rpcclient
directory in order to prepare it for merging. This does NOT update btcd
or any of the other packages to use the new location as that will be
done separately.
- All import paths in the old btcrpcclient files have been changed to
the new location
- All references to btcrpcclient as the package name have been changed to
rpcclient
This modifies the code that determines the most recently known
checkpoint to take advantage of recent changes which make the entire
block index available in memory by only storing a reference to the
specific node in the index that represents the latest known checkpoint.
Previously, the entire block was stored and new checkpoints required
loading it from the database.
This completely removes the threshold state database caching code since
it can very quickly be calculated at startup now that the entire block
index is loaded first.
This reworks the block index code such that it loads all of the headers
in the main chain at startup and constructs the full block index
accordingly.
Since the full index from the current best tip all the way back to the
genesis block is now guaranteed to be in memory, this also removes all
code related to dynamically loading the nodes and updates some of the
logic to take advantage of the fact traversing the block index can
longer potentially fail. There are also more optimizations and
simplifications that can be made in the future as a result of this.
Due to removing all of the extra overhead of tracking the dynamic state,
and ensuring the block node structs are aligned to eliminate extra
padding, the end result of a fully populated block index now takes quite
a bit less memory than the previous dynamically loaded version.
The main downside is that it now takes a while to start whereas it was
nearly instant before, however, it is much better to provide more
efficient runtime operation since that is its ultimate purpose and the
benefits far outweigh this downside.
Some benefits are:
- Since every block node is in memory, the recent code which
reconstructs headers from block nodes means that all headers can
always be served from memory which is important since the majority of
the network has moved to header-based semantics
- Several of the error paths can be removed since they are no longer
necessary
- It is no longer expensive to calculate CSV sequence locks or median
times of blocks way in the past
- It will be possible to create much more efficient iteration and
simplified views of the overall index
- The entire threshold state database cache can be removed since it is
cheap to construct it from the full block index as needed
An overview of the logic changes are as follows:
- Move AncestorNode from blockIndex to blockNode and greatly simplify
since it no longer has to deal with the possibility of dynamically
loading nodes and related failures
- Rename RelativeNode to RelativeAncestor, move to blockNode, and
redefine in terms of AncestorNode
- Move CalcPastMedianTime from blockIndex to blockNode and remove no
longer necessary test for nil
- Change calcSequenceLock to use Ancestor instead of RelativeAncestor
since it reads more clearly
Instead of having both server and blockManager be aware of the
txProcessed and blockProcessed channels, now the server passed them as
method arguments to blockProcessor.
This takes care of a few minor nits on the recently merged subscribe
code. In particular:
- Avoid extra unnecessary allocation on notifications slice
- Avoid defer overhead on notification mutex in sendNotifications
- Make test function comment start with the name of the function per
Effective Go guidelines
- Use constant for number of subscribers in test
- Don't exceed column 80 in test print
This refactors the RPC server to accept and take ownership of already
configured listeners and refactors the logic to setup those listeners to
the server. This mirrors the logic used by the connection manager and
is desirable since it is another step closer to being able to split the
RPC server code out into a separate package and will make it much easier
to internally test since it allows creating mock listeners.
This modifies all of the RPC code to use the chain parameters that are
associated with the RPC server instead of the global activeNetParams and
thus moves one step closer to being able to split the RPC server out
into a separate package.
This decouples the RPC server from the internal btcd server to move
closer to being able to split it out into a separate package.
In order to accomplish this, it introduces an rpcserverConfig type and
several new interfaces, named rpcserverPeer, rpcserverConnManager, and
rpcserverBlockManager, which are necessary to break the direct
dependencies on the main server and block manager instances.
It also adds concrete implementations of the new interfaces and uses
them to configure the RPC server.
Ultimately, the RPC server should ideally be decoupled even more such
that all of the types in the configuration struct use interfaces instead
of the concrete types. Doing this would make the RPC server much easier
to internally test since it would allow creating lightweight stubs for
the various pieces.
Instead of having the block manager notify the RPC server about
accepted, connected, and disconnected blocks, the RPC server will
directly listen for notifications from the blockchain.
The objective is to remove the reference from blockManager to
server. Since the blockManager is responsible for keeping the mempool
in sync, it should have a direct reference to it.
The BlockChain struct emits notifications for various events, but
it is only possible to register one listener. This changes the
interface and implementations to allow multiple listeners.
This removes the standardness check to reject transactions with a lock
time greater than a maxint32 because the old bitcoind nodes which it was
designed to protect against are no longer valid for other reasons and
thus there are no longer any of them on the network to worry about.
This makes the code for getheaders more consistent with the rest of the
code in terms of making use of existing error functions and using the
same RPC error codes as other handlers.
While here, it also performs the fetching of headers directly instead of
using a function from server which makes it more tightly coupled.
This replaces the ErrDoubleSpend and ErrMissingTx error codes with a
single error code named ErrMissingTxOut and updates the relevant errors
and expected test results accordingly.
Once upon a time, the code relied on a transaction index, so it was able
to definitively differentiate between a transaction output that
legitimately did not exist and one that had already been spent.
However, since the code now uses a pruned utxoset, it is no longer
possible to reliably differentiate since once all outputs of a
transaction are spent, it is removed from the utxoset completely.
Consequently, a missing transaction could be either because the
transaction never existed or because it is fully spent.
If no existing btcctl.conf file exists, btcctl creates a default one
using the RPC username and password from the btcd.conf. If the
--wallet flag is passed, however, it should read from btcwallet.conf
instead.
https://github.com/btcsuite/btcd/issues/875.
This commit updates the block template generation logic to only include
witness transactions once the soft-fork has activated and to also
include the OP_RETURN witness commitment (with additional block weight
accounting).
This commit updates the new segwit validation logic within block
validation to be guarded by an initial check to the version bits state
before conditionally enforcing the logic based off of the state.
This commit adds set of BIP0009 (Version Bits) deployment parameters
for all networks detailing the activation parameters for the segwit
soft-fork.
Additionally, the BIP0009 integration test has been updated to test for
the proper transitioning of version bits state for the segwit soft
fork. Finally, the `getblockchaininfo` test has also been updated to
properly display the state of segwit.
This commit modifies the existing block selection logic to limit
preferentially by weight instead of serialized block size, and also to
adhere to the new sig-op cost limits which are weighted according to
the witness discount.
This commit implements the new block validation rules as defined by
BIP0141. The new rules include the constraints that if a block has
transactions with witness data, then there MUST be a commitment within
the conies transaction to the root of a new merkle tree which commits
to the wtxid of all transactions. Additionally, rather than limiting
the size of a block by size in bytes, blocks are now limited by their
total weight unit. Similarly, a newly define “sig op cost” is now used
to limit the signature validation cost of transactions found within
blocks.
This commit implements the new “weight” metric introduced as part of
the segwit soft-fork. Post-fork activation, rather than limiting the
size of blocks and transactions based purely on serialized size, a new
metric “weight” will instead be used as a way to more accurately
reflect the costs of a tx/block on the system. With blocks constrained
by weight, the maximum block-size increases to ~4MB.
This commit adds verification of the post-segwit standardness
requirement that all pubkeys involved in checks operations MUST be
serialized as compressed public keys. A new ScriptFlag has been added
to guard this behavior when executing scripts.
This commit modifies the op-code execution for OP_IF and OP_NOTIF to
enforce the additional “minimal if” constraints which require the
top-stack item when the op codes are encountered to be either an empty
vector, or exactly [0x01].
This commit adds a new function to btcec: IsCompressedPubKey. This
function returns true iff the passed serialized public key is encoded
in compressed format.
This commit implements the flag activation portion of BIP 0147. The
verification behavior triggered by the NULLDUMMY script verification
flag has been present within btcd for some time, however it wasn’t
activated by default.
With this commit, once segwit has activated, the ScriptStrictMultiSig
will also be activated within the Script VM. Additionally, the
ScriptStrictMultiSig is now a standard script verification flag which
is used unconditionally within the mempool.
This commit implements full witness program validation for the
currently defined version 0 witness programs. This includes validation
logic for nested p2sh, p2wsh, and p2wkh. Additionally, when in witness
validation mode, an additional set of constrains are enforced such as
using the new sighash digest algorithm and enforcing clean stack
behavior within witness programs.
This commit fixes an off-by-one error which is only manifested by the
new behavior of OP_CODESEPARATOR within sig hashes triggered by the
segwit behavior. The current behavior within the Script VM
(txscript.Engine) is known to be fully correct to the extent that it has
been verified. However, once segwit activates a consensus divergence
would emerge due to *when* the program counter was incremented in the
previous code (pre-this-commit).
Currently (pre-segwit) when calculating the pre-image to a transaction
sighash for signature verification, *all* instances of OP_CODESEPARATOR
are removed from the subScript being signed before generating the final
sighash. SegWit has additional nerfed the behavior of OP_CODESEPARATOR
by no longer removing them (and starting after the last instance), but
instead simply starting the subScript to be directly *after* the last
instance of an OP_CODESEPARATOR within the pkScript.
Due to this new behavior, without this commit, an off-by-one error
(which only matters post-segwit), would cause txscript to generate an
incorrect subScript since the instance of OP_CODESEPARATOR would remain
as part of the subScript instead of being sliced off as the new behavior
dictates. The off-by-one error itself is manifested due to a slight
divergence in txscript.Engine’s logic compared to Bitcoin Core. In
Bitcoin Core script verification is as follows: first the next op-code
is fetched, then program counter is incremented, and finally the op-code
itself is executed. Before this commit, btcd flipped the order
of the last two steps, executing the op-code *before* the program
counter was incremented.
This commit fixes the post-segwit consensus divergence by incrementing
the program-counter *before* the next op-code is executed. It is
important to note that this divergence is only significant post-segwit,
meaning that txscript.Engine is still consensus compliant independent of
this commit.
This commit introduces a series of internal and external helper
functions which enable the txscript package to be aware of the new
standard script templates introduced as part of BIP0141. The two new
standard script templates recognized are pay-to-witness-key-hash
(P2WKH) and pay-to-witness-script-hash (P2WSH).
This commit implements most of BIP0143 by adding logic to implement the
new sighash calculation, signing, and additionally introduces the
HashCache optimization which eliminates the O(N^2) computational
complexity for the SIGHASH_ALL sighash type.
The HashCache struct is the equivalent to the existing SigCache struct,
but for caching the reusable midstate for transactions which are
spending segwitty outputs.
This commit modifies the logic within the block manager and service to
preferentially fetch transactions and blocks which include witness data
from fully upgraded peers.
Once the initial version handshake has completed, the server now tracks
which of the connected peers are witness enabled (they advertise
SFNodeWitness). From then on, if a peer is witness enabled, then btcd
will always request full witness data when fetching
transactions/blocks.
This commit modifies the base peer struct to ascertain when a peer is
able to understand the new witness encoding, and specify the peer’s
supported encoding explicitly before/after the version handshake.
This commit implements the new witness encoding/decoding for
transactions as specified by BIP0144. After segwit activation, a
special transaction encoding is used to signal to upgraded nodes that
the transaction being deserialized bares witness data. The prior
BtcEncode and BtcDecode methods have been extended to be aware of the
new signaling bytes and the encoding of witness data within
transactions.
Additionally, a new method has been added to calculate the “stripped
size” of a transaction/block which is defined as the size of a
transaction/block *excluding* any witness data.
This commit adds the new inventory types for segwit which are used by
the responder to explicitly request that transactions/blocks sent for a
particular inv hash should include all witness data.
This commit modifies the existing wire.Message interface to introduce a
new MessageEncoding variant which dictates the exact encoding to be
used when serializing and deserializing messages. Such an option is now
necessary due to the segwit soft-fork package, as btcd will need to be
able to optionally encode transactions/blocks without witness data to
un-upgraded peers.
Two new functions have been introduced: ReadMessageWithEncodingN and
WriteMessageWithEncodingN which wrap BtcDecode/BtcEncode with the
desired encoding format.
This commit introduces the new SFNodeWitness service bit which has been
added to the protocol as part of BIP0144. The new service bit allows
peers on the network to signal their acceptance and adherence to the
new rules defined as part of the segwit soft-fork package.
The IsBech32SegwitPrefix method takes a string prefix and
determines if it is a valid prefix for a Bech32 encoded segwit
address for any of the default or registered networks.
Version 0.15.0 of Bitcoin Core will include a new RPC command that will
allow us to obtain the amount of time (in seconds) that the server has
been running.
This modifies the goclean.sh script that is executed on Travis to
only run the tests without the race detector.
While it is nice to run the race detector on the tests, unfortunately
there is a limit to the number of goroutines that can be launched while
running it. Since Travis is now much slower than it once was, this
causes a ton of false positive failures.
This corrects the assertion in the decodeSpentTxOut function so it does
not improperly cause a panic when unwinding transactions during a reorg
under certain circumstances. In particular, the provided transaction
version that is passed when a stxo entry does not exist is now -1 in
order to properly distinguish it from the zero value.
It also updates the tests accordingly.
This was discovered by the reorg on testnet from block
00000000000018c58c2d2816f03dac327d975a18af6edf1a369df67ecddaf816 to
0000000000001c1161a367156465cc6226e9f862d9c585f94db5779fdf5455ff.
The btclog package has been changed to defining its own logging
interface (rather than seelog's) and provides a default implementation
for callers to use.
There are two primary advantages to the new logger implementation.
First, all log messages are created before the call returns. Compared
to seelog, this prevents data races when mutable variables are logged.
Second, the new logger does not implement any kind of artifical rate
limiting (what seelog refers to as "adaptive logging"). Log messages
are outputted as soon as possible and the application will appear to
perform much better when watching standard output.
Because log rotation is not a feature of the btclog logging
implementation, it is handled by the main package by importing a file
rotation package that provides an io.Reader interface for creating
output to a rotating file output. The rotator has been configured
with the same defaults that btcd previously used in the seelog config
(10MB file limits with maximum of 3 rolls) but now compresses newly
created roll files. Due to the high compressibility of log text, the
compressed files typically reduce to around 15-30% of the original
10MB file.
This slightly optimizes the NAF function by avoiding returning the
unused bit when there is not a carry.
It also adds a bunch of additional unit tests which I made while
debugging.
This modifies the normalize function of the internal field value to
both optimize it and address an issue where the reduction could
lead to an incorrect result with a small range of values. It also adds
tests to ensure the behavior is correct.
The following benchmark shows the relative speedups as a result of the
optimization on my system. In particular, the changes result in
approximately a 14% speedup in Normalize, which ultimately translates to
a 2% speedup in signature verifies.
benchmark old ns/op new ns/op delta
--------------------------------------------------------------------
BenchmarkAddJacobian 1364 1289 -5.50%
BenchmarkAddJacobianNotZOne 3150 3091 -1.87%
BenchmarkScalarBaseMult 134117 132816 -0.97%
BenchmarkScalarBaseMultLarge 135067 132966 -1.56%
BenchmarkScalarMult 411218 402217 -2.19%
BenchmarkSigVerify 671585 657833 -2.05%
BenchmarkFieldNormalize 36.0 31.0 -13.89%
The github markdown interpreter has been changed such that it no longer
allows spaces in between the brackets and parenthesis of links. This
updates the markdown files accordingly.
While here, it also corrects a couple of inconsistencies in regards to
other README.md files in the project.
The github markdown interpreter has been changed such that it no longer
allows spaces in between the brackets and parenthesis of links and now
requires a newline in between anchors and other formatting. This
updates all of the markdown files accordingly.
While here, it also corrects a couple of inconsistencies in some of the
README.md files.
This updates the GetNetworkInfoResult structure to include the latest
fields added to Core for compatibility purposes.
While here, also move the definitions of the subtypes for the result
before their use for consistency.
This changes the nonce generated to detect self connections over to use
pseudo randoms instead of a cryptographically random nonce.
There is really not a good reason for it to be cryptographically strong,
using the prng is much faster, and the prng also doesn't burn entropy.
This removes the field that tracks whether the version has been sent
since it is no longer used after the version negotiation was separated
from the main read and write code.
This commit modifies the existing block validation logic to examine the
current version bits state of the CSV soft-fork, enforcing the new
validation rules (BIPs 68, 112, and 113) accordingly based on the
current `ThesholdState`.
This commit publicly exports the CreateBlock function as it can be very
useful for generating blocks for tests. Additionally, the behavior of
the function has been modified slightly to build off of the genesis
block for the specified chain if the `prevBlock` paramter is nil.
This commit adds BIP-9 deployment parameters for all registered
networks for the CSV soft-fork package.
The mainnet and testnet parameters have been set in accordance to the
finalized BIPs.
For simnet, and the regression net, the activation date is back-dated
in order to allow signaling for the soft-fork at any time. Additionally
the expiration time for simnet and regrets has been set to
math.MaxInt64, meaning they’ll never expire.
Now that glide is used for version management and a specific commit of
the upstream repository can be locked it is no longer necessary to
maintain a fork of the package specifically to keep a stable dependency.
While here, update the glide dependency for btcutil as well since it was
switched to use the upstream path as well.
Invalid tokens: github had made some changes the past months, the old style is not rendering at all, so I fixed that for you.
Other contributors can do the same for all of the project's documents.
Thanks.
This modifies the goclean.sh script that is executed on Travis to
only run the tests once.
While it is nice to see coverage reports in the log, unfortunately it
appears that both the -race and -cover flags can't be used together, and
the tests have grown in complexity such that they are starting to get
close to TravisCI time limits.
This simplifies the code based on the recommendations of the gosimple
lint tool.
Also, it increases the deadline for the linters to run to 10 minutes and
reduces the number of threads that is uses. This is being done because
the Travis environment has become increasingly slower and it also seems
to be hampered by too many threads running concurrently.
This modifies the blockNode and BestState structs in the blockchain
package to store hashes directly instead of pointers to them and updates
callers to deal with the API change in the exported BestState struct.
In general, the preferred approach for hashes moving forward is to store
hash values in complex data structures, particularly those that will be
used for cache entries, and accept pointers to hashes in arguments to
functions.
Some of the reasoning behind making this change is:
- It is generally preferred to avoid storing pointers to data in cache
objects since doing so can easily lead to storing interior pointers
into other structs that then can't be GC'd
- Keeping the hash values directly in the block node provides better
cache locality
Since the code base is currently in the process of changing over to
decouple download and connection logic, but not all of the necessary
parts are updated yet, ensure blocks that are in the database, but do
not have an associated main chain block index entry, are treated as if
they do not exist for the purposes of chain connection and selection
logic.
This refactors the block index logic into a separate struct and
introduces an individual lock for it so it can be queried independent of
the chain lock.
This modifies the block node structure to include a couple of extra
fields needed to be able to reconstruct the block header from a node,
and exposes a new function from chain to fetch the block headers which
takes advantage of the new functionality to reconstruct the headers from
memory when possible. Finally, it updates both the p2p and RPC servers
to make use of the new function.
This is useful since many of the block header fields need to be kept in
order to form the block index anyways and storing the extra fields means
the database does not have to be consulted when headers are requested if
the associated node is still in memory.
The following timings show representative performance gains as measured
from one system:
new: Time to fetch 100000 headers: 59ms
old: Time to fetch 100000 headers: 4783ms
This removes the CalcPastMedianTime since it is now exposed much more
efficiently via the MedianTime field of the BestState snapshot returned
from the BestSnapshot function.
This modifies the block nodes used in the blockchain package for keeping
track of the block index to use int64 for the timestamps instead of
time.Time.
This is being done because a time.Time takes 24 bytes while an int64
only takes 8 and the plan is to eventually move the entire block index
into memory instead of the current dynamically-loaded version, so
cutting the number of bytes used for the timestamp by a third is highly
desirable.
Also, the consensus code requires working with unix-style timestamps
anyways, so switching over to them in the block node does not seem
unreasonable.
Finally, this does not go so far as to change all of the time.Time
references, particularly those that are in the public API, so it is
purely an internal change.
This modifies the rpctest harness and its associated memwallet to make
use of the new filter-based notifications since the old notifications
are now deprecated.
It also updates the glide.lock file to require the necessary
btcrpcclient version.
This modifies the blockchain code to store all blocks that have passed
proof-of-work and contextual validity tests in the database even if they
may ultimately fail to connect.
This eliminates the need to store those blocks in memory, allows them to
be available as orphans later even if they were never part of the main
chain, and helps pave the way toward being able to separate the download
logic from the connection logic.
This modifies the .travis.yml file to install a release version of glide
instead of using its latest master branch. This will prevent upstream
changes from inadvertently breaking the CI builds.
This contains a bit of cleanup and additional logic to improve the
recently-added ability to specify additional checkpoints via the
--addcheckpoint option.
In particular:
- Improve error messages in the checkpoint parsing
- Correct the mergeCheckpoints function to weed out duplicate height
checkpoints while using the most-recently provided one as described by
its comment
- Add an assertion to blockchain.New that the provided checkpoints are
sorted as required
- Keep comments to 80 columns and use two spaces after periods in them to
be consistent with the rest of the code base
- Make the entry in doc.go match the actual btcd -h output
The most recent version of glide creates a working directory named
.glide that includes .go files and thus must be excluded from the find
operation that generates merged test coverage of all packages.
A DNS lookup was being attempted on onion addresses causing
connections to fail. This has been fixed by introducing type
onionAddr (which implements a net.Addr interface) and passing
it to btcdDial.
Also, the following onion related fixes have been made:
* getaddednodeinfo - updated to handle onion addrs.
* TorLookupIP - fixed err being shadowed.
* newServer - rename tcpAddr to netAddr
* addrStringToNetAddr - skip if host is already an IP addr.
* addrStringToNetAddr - err if tor is disabled
* getaddednodeinfo - check if host is already an IP addr.
ScriptVerifyNullFail defines that signatures must be empty if a
CHECKSIG or CHECKMULTISIG operation fails.
This commit also enables ScriptVerifyNullFail at the mempool policy
level.
This updates the data driven transaction script tests to use the most
recent format and test data as implemented by Core so the test data can
more easily be updated and help prove cross-compatibility correctness.
In particular, the new format combines the previously separate valid and
invalid test data files into a single file and adds a field for the
expected result. This is a nice improvement since it means tests can
now ensure script failures are due to a specific expected reason as
opposed to only generically detecting failure as the previous format
required.
The btcd script engine typically returns more fine grained errors than
the test data expects, so the test adapter handles this by allowing
expected errors in the test data to be mapped to multiple txscript
errors.
It should also be noted that the tests related to segwit have been
stripped from the data since the segwit PR has not landed in master yet,
however the test adapter does recognize the new ability for optional
segwit data to be supplied, though it will need to properly construct
the transaction using that data when the time comes.
This converts the majority of script errors from generic errors created
via errors.New and fmt.Errorf to use a concrete type that implements the
error interface with an error code and description.
This allows callers to programmatically detect the type of error via
type assertions and an error code while still allowing the errors to
provide more context.
For example, instead of just having an error the reads "disabled opcode"
as would happen prior to these changes when a disabled opcode is
encountered, the error will now read "attempt to execute disabled opcode
OP_FOO".
While it was previously possible to programmatically detect many errors
due to them being exported, they provided no additional context and
there were also various instances that were just returning errors
created on the spot which callers could not reliably detect without
resorting to looking at the actual error message, which is nearly always
bad practice.
Also, while here, export the MaxStackSize and MaxScriptSize constants
since they can be useful for consumers of the package and perform some
minor cleanup of some of the tests.
If a host is down and doesn't send a TCP RST, the net.Dial function
blocks until the OS times out the connection. Convert to using
DialTimeout with a 30 second default timeout.
Fail fast if glide or gometalinter isn't available.
And also fail if running glide or gometalinter fails.
Before goclean.sh could pass even if glide or gometalinter wasn't
available at all. It seems that even though we use `set -ex`, their
failure was hidden either inside the $() or behind the pipe.
This adds tests which exercise the BIP0009 state transitions using the
newly available soft fork status information in the getblockchaininfo
RPC.
The following is an overview of the tests added:
- Assert the chain height is 0 and the state is ThresholdDefined
- Generate 1 fewer blocks than needed to reach the first state
transition
- Assert chain height is expected and state is still ThresholdDefined
- Generate 1 more block to reach the first state transition
- Assert chain height is expected and state moved to ThresholdStarted
- Generate enough blocks to reach the next state transition window, but
only signal support in 1 fewer than the required number to achieve
ThresholdLockedIn
- Assert chain height is expected and state is still ThresholdStarted
- Generate enough blocks to reach the next state transition window with
only the exact number of blocks required to achieve locked in status
signalling support
- Assert chain height is expected and state moved to ThresholdLockedIn
- Generate 1 fewer blocks than needed to reach the next state transition
- Assert chain height is expected and state is still ThresholdLockedIn
- Generate 1 more block to reach the next state transition
- Assert chain height is expected and state moved to ThresholdActive
In addition, it updates the existing BIP0009 mining tests to include
extra assertions that the chain height is at the expected height in
addition to checking that the bits are correctly set according to the
expected state.
This commit updates the fields of GetBlockChainInfoResult to reflect
the current state of the RPC as widely implemented by other full-node
implementations.
Replace assignments to individual fields of wire.NetAddress with
creating the entire object at once, as one would do if the type was
immutable.
In some places this replaces the creation of a NetAddress with a
high-precision timestamp with a call to a 'constructor' that converts
the timestamp to single second precision. For consistency, the tests
have also been changed to use single-precision timestamps.
Lastly, the number of allocations in readNetAddress have been reduced by
reading the services directly into the NetAddress instead of first into
a temporary variable.
The thresholdState and deploymentState functions expect the block node
for the block prior to which the threshold state is calculated, however
the startup code which checked the threshold states was using the
current best node instead of its parent.
While here, also update the comments and rename a couple of variables to
help make this fact more clear.
The CSV consensus rules dictate that the opcode fails when the
transaction version is not at least version 2, however that only applies
if the disable flag is not set in the sequence.
This is not an issue at the current time because we do not yet enforce
CSV at a consensus level, however, I noticed this discrepancy when doing
a thorough audit of the CSV paths due to the ongoing work to add full
consensus-enforced CSV support.
As a result, this must be merged prior to enabling consensus enforcement
for CSV or it would open up the potential for a hard fork.
This modifies the code to only enforce the fairly expensive BIP0030
(duplicate transcactions) checks when the chain has not yet reached the
BIP0034 activation height (and is not one of the 2 special historical
blocks that break the rule and prompted BIP0034 to being with) since
that BIP made it impossible to create duplicate coinbases and thus
removed the possibility of creating transactions that overwrite older
ones.
This is a rather large optimization because the check is expensive due
to involving a ton of cache misses in the utxoset. For example, the
following are times it took to perform the BIP0030 check on blocks
425490 - 425502 and a system with a relatively old Hitachi spinner HDD:
block 425490: 674.5857ms
block 425491: 726.5923ms
block 425492: 827.6051ms
block 425493: 680.0863ms
block 425494: 722.0917ms
block 425495: 700.0889ms
block 425496: 647.5823ms
block 425497: 445.0565ms
block 425498: 602.5765ms
block 425499: 375.0476ms
block 425500: 771.0979ms
block 425501: 461.5586ms
block 425502: 603.0766ms
As can be seen from these numbers, this reduces the block validation
time by an average of just over half a second for the given
representative data set and hardware.
Signed-off-by: Dave Collins <davec@conformal.com>
Now that all softforking is done via BIP0009 versionbits, replace the
old isMajorityVersion deployment mechanism with hard coded historical
block heights at which they became active.
Since the activation heights vary per network, this adds new parameters
to the chaincfg.Params struct for them and sets the correct heights at
which each softfork became active on each chain.
It should be noted that this is a technically hard fork since the
behavior of alternate chain history is different with these hard-coded
activation heights as opposed to the old isMajorityVersion code. In
particular, an alternate chain history could activate one of the soft
forks earlier than these hard-coded heights which means the old code
would reject blocks which violate the new soft fork rules whereas this
new code would not.
However, all of the soft forks this refers to were activated so far in
the chain history there is there is no way a reorg that long could
happen and checkpoints reject alternate chains before the most recent
checkpoint anyways. Furthermore, the same change was made in Bitcoin
Core so this needs to be changed to be consistent anyways.
This commit adds all of the infrastructure needed to support BIP0009
soft forks.
The following is an overview of the changes:
- Add new configuration options to the chaincfg package which allows the
rule deployments to be defined per chain
- Implement code to calculate the threshold state as required by BIP0009
- Use threshold state caches that are stored to the database in order
to accelerate startup time
- Remove caches that are invalid due to definition changes in the
params including additions, deletions, and changes to existing
entries
- Detect and warn when a new unknown rule is about to activate or has
been activated in the block connection code
- Detect and warn when 50% of the last 100 blocks have unexpected
versions.
- Remove the latest block version from wire since it no longer applies
- Add a version parameter to the wire.NewBlockHeader function since the
default is no longer available
- Update the miner block template generation code to use the calculated
block version based on the currently defined rule deployments and
their threshold states as of the previous block
- Add tests for new error type
- Add tests for threshold state cache
addrmgr.GetAddress() had a parameter `class string` originally intended
to support looking up addresses according to some type of filter such as
IPv4, IPv6, and only those which support specific wire.ServiceFlags
(full nodes, nodes that support bloom filters, nodes that support
segwit, etc). But currently the parameter is unused and also has an
inappropriate type `string`.
If it would ever be used, it's easy to add back and should then get an
appropriate type such as something that allows bitflags to be set so
that the caller could request combinations such as peers that support
IPv6, are full nodes, and support bloom filters.
This corrects an issue introduced by commit
e8f63bc295 where a failure to lookup a
hostname could lead to a panic in certain circumstances. An error is
now returned in that case as expected.
This modifies the error handling path in the peer read loop such that it
will no longer log an error when the error is io.ErrUnexpectedEOF. This
is being done because that error is almost always the result of a peer
being remotely disconnected and thus it isn't useful information to log.
However, since it might actually be due to a malformed message, a reject
message is still queued up to be sent back to the peer (which will
simply be discarded if the peer disconnected) before it is disconnected.
While it would be ideal to only log if it's not due to a disconnect, and
the code already attempts to handle that, it's not 100% possible to
detect upon the read returning an error without attempting to read again
which will not happen until the next loop iteration.
This commit modifies the `ConnManager` to use the `net.Add` interface
through the package instead of a plain string to represent and
manipulate addresses. This change makes the package much more general as
users of the package can possibly utilize custom implementations of the
`net.Addr` interface to establish connections.
More precisely, the `ConnReq` struct has been modified to use a net.Addr
instance explicitly, and the `DialFunc` type has also been modified to
take a `net.Addr` directly. This latter change gives functions that
adhere to the `DialFunc` type more flexibility as to exactly how the
connection is established.
Additionally, the `connmgr.Config.GetNewAddress` configuration option
now directly returns a `net.Addr. This change allows the `connmgr` to be
decoupled from all DNS queries which allows callers to preferentially
select more secure methods like performing DNS lookups over a Tor proxy.
This modifies the signatures of all serverPeer callbacks that are
provided as peer.Listeners to use _ for the first parameter name which
ensures the passed peer can't be used within the function and updates
all references to the server peer.
This helps ensure any overridden methods that might be defined on a
serverPeer will be invoked where directly calling methods on the passed
peer would not.
Also, while here, add a comment to the OnFeeFilter function.
This modifies the connection manager to provide support for accepting
inbound connections on a caller-provided set of listeners and notify the
caller via a callback.
This is only the minimum work necessary to get inbound support into the
connection manager. The intention for future commits is to move more
connection-related logic such as limiting the maximum number of overall
connections and banned peer tracking into the connection manager.
This removes the type definitions for the callback functions in favor of
declaring them directly in the Config struct. This is more consistent
with the rest of the code base and is preferred since it means callers
reviewing the documentation don't have to follow another level of
indirection to figure out the signature.
Rather than accepting a net.Addr interface and returning an error when
it's not specifically a *net.TCPConn, just accept a *net.TCPConn
directly so the compiler will assert it. Also, remove the error return
since it can no longer occur.
This function is a legacy function from way back during initial
development. Nothing actually uses it and it's not very useful anyways
since it requires the connection to be a specific type (net.TCPConn) and
therefore doesn't work right with things that typically provide their
own net.Conn implementation like proxies.
This modifies the goclean.sh script to include the unconvert lint tool
in the gometalinter configuration.
It also bumps the deadline to 45 seconds to give slower TravisCI
build servers more breathing room.
This makes two modifications based on the results of staticcheck.
The first is to use a simplified zero value for the time in the addrmgr
package tests.
The second is check any errors when opening a directory while attempting
to perform old version upgrades.
This switches `goclean` to use `gometalint` instead of running each checking tool one at a time. The `gometalint` tool runs them concurrently.
More importantly it will allow us to easily add additional linters as desired.
The current max orphan transaction size causes problems with dependent
transaction relay due to its artificially small size in relation to the
max standard transaction size.
Consequently, this modifies the orphan transaction policy by increasing
the max size of each orphan to the same value allowed for standard
non-orphan transactions and reducing the default max allowed number of
orphans to 100.
From a memory usage standpoint, the worst case max mem usage prior to
this change was 5MB plus structure and tracking overhead (1000 max
orphans * 5KB max each). With this, that is raised to 10MB (100 max
orphans * 100KB max each) in the worst case.
It is important to note that the values were originally implemented as a
naive means to control the size of the orphan pool before several of the
recent enhancements which more aggressively remove orphans from the
orphan pool were added, so they needed to be evaluated again.
For a very long time prior to recent changes, the orphan pool would
quickly reach the max allowed worst-case usage and effectively stay
there forever whereas with more recent changes, the actual run-time
orphan pool usage is usually much smaller.
Finally, as another point in favor of this change, as the network has
evolved, nodes have generally become better about orphan management and
as such missing ancestors will typically either be broadcast or mined
fairly quickly resulting in fewer overall orphans.
This removes any remaining orphan transactions that were sent by a peer
when it disconnects since it is extremely unlikely that the missing
parents will ever materialize from elsewhere.
This allows a caller-provided tag to be associated with orphan
transactions. This is useful since the caller can use the tag for
purposes such as keeping track of which peers orphans were first seen
from.
Also, since a parameter is required now anyways, it associates the peer
ID with processed transactions from remote peers.
This does the minimum work necessary to refactor the CPU miner code into
its own package. The idea is that separating this code into its own
package will improve its testability and ultimately be useful to other
parts of the codebase such as the various tests which currently
effectively have their own stripped-down versions of this code.
The API will certainly need some additional cleanup and changes to make
it more usable outside of the specific circumstances it was originally
designed to support (namely the generate RPC), however it is better to
do that in future commits in order to keep the changeset as small as
possible during this refactor.
Overview of the major changes:
- Create the new package
- Move cpuminer.go -> cpuminer/cpuminer.go
- Update mining logging to use the new cpuminer package logger
- Rename cpuminerConfig to Config (so it's now cpuminer.Config)
- Rename newCPUMiner to New (so it's now cpuminer.New)
- Update all references to the cpuminer to use the package
- Add a skeleton README.md
This moves the error check for an attempt to call the generate RPC when
on a network that there is virtually no chance of mining a block with
the CPU into the RPC server where it more naturally belongs. The caller
of the CPU should be the one to determine if it wants to allow mining or
not. While here, use a more accurate RPC error code of ErrDifficulty
instead of ErrInternal.
This change is a step towards being able to separate the CPU mining code
into its own subpackage.
The inability for a peer to negotiate is not something that should be a
warning which implies something is wrong. On the contrary, it is quite
expected that various peers will connect (or be connected to) that are
unable to properly negotiate for a variety of reasons. One example would
be a peer that is too old.
Also, while here take care of a few style nits.
This corrects a few issues introduced with the connection manager where
the server was not notifying the connection manager when a connection
request is available again.
The cases resolved are:
- Unable to initialize a server peer instance in response to the connection
- Failure to associate the connection with the server peer instance
- Disconnection of a non-persistent outbound peer
It also changes the log message to a debug in the former case because
it's not something that should be shown to the user as an error given
it's not due to anything the user has misconfigured nor is it even
unexpected if an invalid address is provided.
This adds a new field to the peer struct which stores the protocol
version advertised by the remote peer and updates the StatsSnapshot to
return the advertised version instead of the negotiated version.
Now that glide is used for version management the deps.txt file is no
longer needed. It will still be available in the git history if it is
ever needed.
This modifies the NewMsgTx function to accept the transaction version as
a parameter and updates all callers.
The reason for this change is so the transaction version can be bumped
in wire without breaking existing tests and to provide the caller with
the flexibility to create the specific transaction version they desire.
This does the minimum work necessary to refactor the block template
generation code into the mining package. The idea is that separating
this code into the mining package will greatly improve its testability,
allow independent benchmarking and profiling, and open up some
interesting opportunities for future development related to mining.
There are some areas related to policy and other configuration that
could be further refactored, however it is better to do that in future
commits in order to keep the changeset as small as possible during this
refactor.
Overview of the major changes:
- Move mining.go -> mining/mining.go
- Move mining_test.go -> mining/mining_test.go
- Add logger to mining package
- Update the MINR subsystem to use the new mining package logger
- Export CoinbaseFlags from the mining package
- BlkTmplGenerator is now mining.BlkTmplGenerator
- Update all references to the mining code to use the package
This move the export for MinHighPriority from the mempool package to the
mining package. This should have been done when the priority
calculation code was moved to the mining package.
This modifies the block template generate for the mining code such that
it takes chain instance and params instead of requiring a fully
initialized blockManager instance.
Also, in preparation for being able to more easily separate the code, it
exposes and makes use of two new functions:
- BestSnapshot which returns the state snapshot from the underlying
chain instance
- TxSource which returns the underlying transaction source
This is a step towards being able to separate the mining code into its
own package. No functional change.
This introduces a cpuminerConfig type which contains the necessary
information to break the direct dependency on the main server instance.
This change is a step towards being able to separate the cpu miner into
its own subpackage. No functional change.
This commit adds a new option to the mempool’s policy configuration
which determines which transaction versions should be accepted as
standard.
The default version set by the policy within the server is 2; this
allows accepting transactions which have version 2 enabled in order to
utilize the new sequence locks feature.
This commit adds a new function to the package: `LockTimeToSequence`.
The function is a simple utility function which aides the caller to
mapping a targeted time or block based relative lock-time to the
appropriate sequence number.
This commit introduces the concept of “sequence locks” borrowed from
Bitcoin Core for converting an input’s relative time-locks to an
absolute value based on a particular block for input maturity
evaluation.
A sequence lock is computed as the most distant maturity height/time
amongst all the referenced outputs within a particular transaction.
A transaction with sequence locks activated within any of its inputs
can *only* be included within a block if from the point-of-view of that
block either the time-based or height-based maturity for all referenced
inputs has been met.
A transaction with sequence locks can only be accepted to the mempool
iff from the point-of-view of the *next* (yet to be found block) all
referenced inputs within the transaction are mature.
This introduces a new type named BlkTmplGenerator which encapsulates the
various state needed to generate block templates.
This is useful since it means code that needs to generate block
templates can simply accept the generator rather than needing access to
all of the additional state which in turn will ultimately make it easier
to split the mining code into its own package.
This moves the priority-related code from the mempool package to the
mining package and also exports a new constant named UnminedHeight which
takes the place of the old unexported mempoolHeight.
Even though the mempool makes use of the priority code to make decisions
about what it will accept, priority really has to do with mining since
it influences which transactions will end up into a block. This change
also has the side effect of being a step towards enabling separation of
the mining code into its own package which, as previously mentioned,
needs access to the priority calculation code as well.
Finally, the mempoolHeight variable was poorly named since what it
really represents is a transaction that has not been mined into a block
yet. Renaming the variable to more accurately reflect its purpose makes
it clear that it belongs in the mining package which also needs the
definition now as well since the priority calculation code relies on it.
This will also benefit an outstanding PR which needs access to the same
value.
This fixes a bug where a transaction would lose reference to other
transactions dependant on it when being considered for inclusion in a
new block template. The issue only occurs when the transaction being
considered triggers a change of priority queue ordering to ordering by
fee. It results in none of the dependant transactions being considered
for inclusion in the new block template.
This implements orphan expiration in the mempool such that any orphans
that have not had their ancestors materialize within 15 minutes of their
initial arrival time will be evicted which in turn will remove any other
orphans that attempted to redeem it.
In order to perform the evictions with reasonable efficiency, an
opportunistic scan interval of 5 minutes is used. That is to say that
there is not a hard deadline on the scan interval and instead it runs
when a new orphan is added to the pool if enough time has passed.
The following is an example of running this code against the main
network for around 30 minutes:
23:05:34 2016-10-24 [DBG] TXMP: Expired 3 orphans (remaining: 254)
23:10:38 2016-10-24 [DBG] TXMP: Expired 112 orphans (remaining: 231)
23:15:43 2016-10-24 [DBG] TXMP: Expired 95 orphans (remaining: 206)
23:20:44 2016-10-24 [DBG] TXMP: Expired 90 orphans (remaining: 191)
23:25:51 2016-10-24 [DBG] TXMP: Expired 71 orphans (remaining: 191)
23:30:55 2016-10-24 [DBG] TXMP: Expired 70 orphans (remaining: 105)
23:36:19 2016-10-24 [DBG] TXMP: Expired 55 orphans (remaining: 107)
As can be seen from the above, without orphan expiration on this data
set, the orphan pool would have grown an additional 496 entries.
This introduces a new package to house tests which use the rpctest
package to programmatically drive end-to-end integration testing and
move the existing RPC integration tests into the new package.
This modifies the way orphan removal and processing is done to more
aggressively remove orphans that can no longer be valid due to other
transactions being added or removed from the primary transaction pool.
The net effect of these changes is that orphan pool will typically be
much smaller which greatly improves its effectiveness. Previously, it
would typically quickly reach the max allowed worst-case usage and
effectively stay there forever.
The following is a summary of the changes:
- Modify the map that tracks which orphans redeem a given transaction to
instead track by the specific outpoints that are redeemed
- Modify the various orphan removal and processing functions to accept
the full transaction rather than just its hash
- Introduce a new flag on removeOrphans which specifies whether or not
to remove the transactions that redeem the orphan being removed as
well which is necessary since only some paths require it
- Add a new function named removeOrphanDoubleSpends that is invoked
whenever a transaction is added to the main pool and thus the outputs
they spent become concrete spends
- Introduce a new flag on maybeAcceptTransaction which specifies whether
or not duplicate orphans should be rejected since only some paths
require it
- Modify processOrphans as follows:
- Make use of the modified map
- Use newly available flags and logic work more strictly work with tx
chains
- Recursively remove any orphans that also redeem any outputs redeemed
by the accepted transactions
- Several new tests to ensure proper functionality
- Removing an orphan that doesn't exist is removed both when there is
another orphan that redeems it and when there is not
- Removing orphans works properly with orphan chains per the new
remove redeemers flag
- Removal of multi-input orphans that double spend an output when a
concrete redeemer enters the transaction pool
This introduces a new pool membership test function to the mempool
testing infrastructure and refactors the tests to make use of it.
It is useful since it is common logic that is not only needed in the
existing tests, but will be needed by most mempool-related tests.
Avoid compatibility issues with software that relies on the behavior of
bitcoind's JSON-RPC implementation.
The JSON-RPC 1.0 spec defines that notifications must have their "id"
set to null and states that notifications do not have a response.
A JSON-RPC 2.0 notification is a request with "json-rpc":"2.0", and
without an "id" member. The specification states that notifications
must not be responded to. JSON-RPC 2.0 permits the null value as a
valid request id, therefore such requests are not notifications.
Bitcoin Core serves requests with "id":null or even an absent "id", and
responds to such requests with "id":null in the response.
Btcd does not respond to any request without and "id" or with "id":null,
regardless the indicated JSON-RPC protocol version.
In order to avoid compatibility issues with software relying on
Core's behavior, this commit implements "quirks mode" as follows:
- quirks mode can be enabled via configuration (disabled by default)
- If no JSON-RPC version is indicated in the request, accept and
respond to request with "id":null
- If no JSON-RPC version is indicated in the request, accept and
respond to requests without an "id" member
- In both cases above, use "id":null in the response
- Do not respond to request without an "id" or with "id":null when
JSON-RPC version is indicated in the request (process as notification)
This optimizes the way in which the mempool oprhan map is limited in the
same way the server block manager maps were previously optimized.
Previously the code would read a cryptographically random value large
enough to construct a hash, find the first entry larger than that value,
and evict it.
That approach is quite inefficient and could easily become a
bottleneck when processing transactions due to the need to read from a
source such as /dev/urandom and all of the subsequent hash comparisons.
Luckily, strong cryptographic randomness is not needed here. The primary
intent of limiting the maps is to control memory usage with a secondary
concern of making it difficult for adversaries to force eviction of
specific entries.
Consequently, this changes the code to make use of the pseudorandom
iteration order of Go's maps along with the preimage resistance of the
hashing function to provide the desired functionality. It has
previously been discussed that the specific pseudorandom iteration order
is not guaranteed by the Go spec even though in practice that is how it
is implemented. This is not a concern however because even if the
specific compiler doesn't implement that, the preimage resistance of the
hashing function alone is enough.
The following is a before and after comparison of the function for both
speed and memory allocations:
benchmark old ns/op new ns/op delta
----------------------------------------------------------------
BenchmarkLimitNumOrphans 3727 243 -93.48%
benchmark old allocs new allocs delta
-----------------------------------------------------------------
BenchmarkLimitNumOrphans 4 0 -100.00%
This renames the mempool.Config.RelayNonStd option to AcceptNonStd which
more accurately describes its behavior since the mempool was refactored
into a separate package.
The reasoning for this change is that the mempool is not responsible for
relaying transactions (nor should it be). Its job is to maintain a pool
of unmined transactions that are validated according to consensus and
policy configuration options which are then used to provide a source of
transactions that need to be mined.
Instead, it is the server that is responsible for relaying transactions.
While it is true that the current server code currently only relays txns
that were accepted to the mempool, this does not necessarily have to
be the case. It would be entirely possible (and perhaps even a good
idea as something do in the future), to separate the relay policy from
the mempool acceptance policy (and thus indirectly the mining policy).
Older nodes previously added the IP and port information to the address
manager which proved to be unreliable as an inbound connection from a
peer didn't necessarily mean the peer itself accepted inbound
connections.
This also fixes a bug where the peer package was incorrectly sending
the peer's services as its own.
This commit introduces package connmgr which contains connection
management related functionality.
The following is an overview of the features the package provides:
- Maintain fixed number of outbound connections
- Optional connect-only mode
- Retry persistent connections with increasing back-off
- Source peers from DNS seeds
- Use Tor to resolve DNS
- Dynamic ban scores
- Test coverage
In addition, btcd has been refactored to make use of the new package by
extending the connection manager to work with the server to source and
maintain peer connections. The following is a broad overview of the
changes to integrate the package:
- Simplify peer state by removing pending, retry peers
- Refactor to remove retries which are now handled by connmgr
- Use callback to add addresses sourced from the DNS seed
Finally the following connection-related things have been improved as a
part of this refactor:
- Fixes 100% cpu usage when network is down (#129)
- Fixes issues with max peers (#577)
- Simplify outbound peer connections management
This modifies the recently-added NullDataScript function in several
ways in an effort to make them more consistent with the tests in the
rest of the code base and improve/correct the logic:
- Use the hexToBytes and mustParseShortForm functions
- Consistently format the test errors
- Replace the valid bool flag with an expected error and test against it
- Ensure the returned script type is the expected type in all cases
This adds a new function named NullDataScript to the txscript package that returns a provably-pruneable OP_RETURN script with the provided data. The function will return an error if the provided data is larger than the maximum allowed length for a nulldata script to be be considered standard.
This modifies the rpctest framework to start btcd with the appropriate
network flags depending on the provided parameters.
Previously, it always started btcd with --simnet even if other
parameters, such as those for the regression test network, were
provided.
This modifies the ExtractCoinbaseHeight function to recognize small
canonically serialized block heights in coinbase scripts of blocks
higher than version 2.
This allows regression test chains in which blocks encode the serialized
height in the coinbase starting from block 1.
Putting the test code in the same package makes it easier for forks
since they don't have to change the import paths as much and it also
gets rid of the need for internal_test.go to bridge.
Also, do some light cleanup on a few tests while here.
This corrects the isNullData standard transaction type test to work
properly with canonically-encoded data pushes. In particular, single
byte data pushes that are small integers (0-16) are converted to the
equivalent numeric opcodes when canonically encoded and the code failed
to detect them properly.
It also adds several tests to ensure that both canonical and
non-canonical nulldata scripts are recognized properly and modifies the
test failure print to include the script that failed.
This does not affect consensus since it is just a standardness check.
This coincides with the mempool only, policy change which enforces
transaction finality according to the median-time-past rather than
blockheader timestamps. The behavior is pre-cursor to full blown BIP
113 consensus deployment, and subsequent activation.
As a result, the TimeSource field in the mempoolConfig is no longer
needed so it has been removed. Additionally, checkTransactionStandard has been
modified to instead take a time.Time as the mempool is no longer explicitly
dependant on a Chain instance.
This commit adds an additional closure function to the mempool’s config
which computes the median time past from the point of view of the best
node in the chain. The mempool test harness has also been updated to allow
setting a mock median time past for testing purposes.
In addition to increasing the testability of the mempool, this commit
should also speed up transaction and block validation for BIP 113 as
the MTP no longer needs to be re-calculated each time from scratch.
Putting the test code in the same package makes it easier for forks
since they don't have to change the import paths as much and it also
gets rid of the need for internal_test.go to bridge.
Also, remove the exception from the lint checks about returning the
unexported type since it is no longer required.
This adds new tests to the TestNormalize, TestMul, TestAdd2 functions
which trigger an issue with modular reduction that was fixed in the
prevous commit to prevent regressions.
As noted in issue #706, the existing code had an issue where the
normalized result was > P when both the first and second words of the
field representation being normalized were BOTH greater than or equal to
the first and second words of P. Although this condition is rare in
practice, it needs to be handled properly.
This resolves the issue by comparing the low words in the final
reduction step against the normalized low order prime bits to ensure the
final subtraction occurs correctly any time they're > P. This approach
retains the constant time property as well.
This adds a full-blown testing infrastructure in order to test consensus
validation rules. It is built around the idea of dynamically generating
full blocks that target specific rules linked together to form a block
chain. In order to properly test the rules, each test instance starts
with a valid block that is then modified in the specific way needed to
test a specific rule.
Blocks which exercise following rules have been added for this initial
version. These tests were largely ported from the original Java-based
'official' block acceptance tests as well as some additional tests
available in the Core python port. It is expected that further tests
can be added over time as consensus rules change.
* Enough valid blocks to have a stable base of mature coinbases to spend
for futher tests
* Basic forking and chain reorganization
* Double spends on forks
* Too much proof-of-work coinbase (extending main chain, in block that
forces a reorg, and in a valid fork)
* Max and too many signature operations via various combinations of
OP_CHECKSIG, OP_MULTISIG, OP_CHECKSIGVERIFY, and OP_MULTISIGVERIFY
* Too many and max signature operations with offending sigop after
invalid data push
* Max and too many signature operations via pay-to-script-hash redeem
scripts
* Attempt to spend tx created on a different fork
* Attempt to spend immature coinbase (on main chain and fork)
* Max size block and block that exceeds the max size
* Children of rejected blocks are either orphans or rejected
* Coinbase script too small and too large
* Max length coinbase script
* Attempt to spend tx in blocks that failed to connect
* Valid non-coinbase tx in place of coinbase
* Block with no transactions
* Invalid proof-of-work
* Block with a timestamp too far in the future
* Invalid merkle root
* Invalid proof-of-work limit (bits header field)
* Negative proof-of-work limit (bits header field)
* Two coinbase transactions
* Duplicate transactions
* Spend from transaction that does not exist
* Timestamp exactly at and one second after the median time
* Blocks with same hash via merkle root tricks
* Spend from transaction index that is out of range
* Transaction that spends more that its inputs provide
* Transaction with same hash as an existing tx that has not been
fully spent (BIP0030)
* Non-final coinbase and non-coinbase txns
* Max size block with canonical encoding which exceeds max size with
non-canonical encoding
* Spend from transaction earlier in same block
* Spend from transaction later in same block
* Double spend transaction from earlier in same block
* Coinbase that pays more than subsidy + fees
* Coinbase that includes subsidy + fees
* Invalid opcode in dead execution path
* Reorganization of txns with OP_RETURN outputs
* Spend of an OP_RETURN output
* Transaction with multiple OP_RETURN outputs
* Large max-sized block reorganization test (disabled by default since
it takes a long time and a lot of memory to run)
Finally, the README.md files in the main and docs directories have been
updated to reflect the use of the new testing framework.
This removes the exported CalcPastTimeMedian function from the
blockchain package as it is no longer needed since the information is
now available via the BestState snapshot.
Also, update the only known caller of this, which is the chain state in
block manager, to use the snapshot instead. In reality, now that
everything the block manager chain state provides is available via the
blockchain BestState snapshot, the entire thing can be removed, however
that will be done in a separate to commit to keep the changes targeted.
This makes the enforcement of the bloom filter service bit much more
strict. In particular, it does the following:
- Moves the enforcement of the bloom filter service bit out of the peer
package and into the server so the server can ban as necessary
- Disconnect peers that send filter commands when the server is
configured to disable them regardless of the protocol version
- Bans peers that are a high enough protocol version that they are
supposed to observe the service bit is disabled, but ignore it and
send filter commands regardless.
As an added bonus, this fixes the old logic which had a bug in that it
was examining the *remote* peer's supported services in order to choose
whether or not to disconnect instead of the *local* server's supported
services.
This modifies the blockchain.ProcessBlock function to return an
additional boolean as the first parameter which indicates whether or not
the block ended up on the main chain.
This is primarily useful for upcoming test code that needs to be able to
tell the difference between a block accepted to a side chain and a block
that either extends the main chain or causes a reorganize that causes it
to become the main chain. However, it is also useful for the addblock
utility since it allows a better error in the case a file with out of
order blocks is provided.
allAddr was being allocated with counters instead of the actual size
of the address map. This led to the possibility of including nils
in the returned slice, which resulted in a panic.
This adds a new build tag named rpctest which must be set in order for
rpctest-based tests to be executed. The new build tag is also added to
the goclean.sh script which is executed by Travis during continuous
integration builds.
This change is being made because the rpctest framework requires
additional careful user configuration to ensure the version of btcd
under test can be programmatically launched from the system path with
all of the necessary ports open whereas all of the other tests are
self-contained within the test binary itself.
Since said additional configuration is typically not done, it leads to a
lot of false positives. Putting the tests behind a build tag allows
them to remain to be available and run during continuous integration
without imposing the additional configuration requirements on users.
Use os.Getpid() to get the process ID, not os.Getppid(), which returns
the parent process ID. This resulted in multiple calls to
generateListeningAddresses() getting the same listening ports.
This modifies the ports that are selected for use for the p2p and rpc
ports to start with a port that is based on the process id instead of a
hard-coded value. The chosen ports are incremented for each running
instance similar to the previous code except the p2p and rpc ports and
now split into ranges instead of being 2 apart.
This is being done because the previous code only worked for a single
process which means it prevented the ability to run tests in parallel.
The new approach will work with multiple processes, however it must be
stated that there is still a very small probability that the stars could
align resulting in the same ports being selected.
Finally, this also reverts the recent change to run tests serially since
this fixes the underlying cause for that change.
This modifies the goclean.sh to execute all the tests amongst
the packages serially. The default behavior of the `go test` command is
to execute all tests in parallel amongst the listed packages. This
behavior can at times cause tests which use the `rpctest` package to
fail due to multiple `btcd` nodes attempting to bind to the same port
simultaneously. As only one node can successfully bind to the port, the
btcd processes for the other concurrent harness instances exit silently
causing the RPC clients to fail with connection timeouts as their
target process no longer exists. Executing all tests serially
eliminates such a race condition which can cause non-deterministic test
failures.
This modifies the code which handles failed server responses to attempt
to unmarshal the response bytes as a regular JSON-RPC response
regardless of the HTTP status code and, if that fails, return an error
that includes the status code as well as the raw response bytes.
This is being done because some JSON-RPC servers, such as Bitcoin Core,
don't follow the intended HTTP status code meanings. For example, if
you request a block that doesn't exist, it returns a status code of 500
which is supposed to mean internal server error instead of a standard
200 status code (since the HTTP request itself was successful) along
with the JSON-RPC error response.
The result is that errors from Core will now show up as an actual
RPCError instead of an error with the raw JSON bytes.
This also has the benefit of returning the HTTP status code in the
error for real HTTP failure cases such as 401 authentication failures,
which previously would just be an empty error when used against Core
since it doesn't return the actual response along with the status code
as it should.
This changes the GetBlockVersion API to not send a third parameter.
The third parameter is a boolean that expands the transaction data
structures as well. However, Bitcore Core does not recognize this
feature.
GetBlockVerbose now only takes a hash value.
Users of the GetBlockVerbose(hash, true) must now use
GetBlockVerboseTx(hash).
This modifies the return value of the GetBlock and GetBlockAsync
functions to a raw *wire.MsgBlock instead of *btcutil.Block.
This is being done because a btcutil.Block assumes there is a height
associated with the block and the block height is not part of a raw
serialized block as returned by the underlying non-verbose RPC API
getblock call and thus is not available to populate a btcutil.Block
correctly.
A raw wire.Block correctly matches what the underlying RPC call is
actually returning and thus avoids the confusion that could occur.
GetBlockHeader returns the blockheader of the specified blockhash.
GetBlockHeaderVerbose returns the data structure with information about
the blockheader of the specified blockhash.
Fixes#89
This modifies the rpctest harness to call the TearDown function in the
SetUp error path. This ensures any resources, such as temp directories,
that were created during setup before whatever the failure that caused
the error are properly cleaned up.
This corrects the case where any errors that might have happened when
creating the config file were not being displayed due to the logging
system not being initialized yet.
While here, consistently use fmt.Fprint{f,ln} throughout the loadConfig
function even after the logging system is initialized since it will
help prevent copy/paste errors such as the one that induced this change
to begin with.
This changes the transaction input standardness checks as follows:
- Allow any script in a pay-to-script-hash transaction to be relayed and
mined so long as it has no more than 15 signature operations
- Remove the obsolete checks which naively calculated the number of
expected inputs in favor of the more accurate ScriptVerifyCleanStack
and ScriptVerifySigPushOnly functionality of the script engine that was
added after these checks.
This commit modifies the current set of integration tests to ensure
that that the main harness always rejects non-standard transactions.
With this in place, even though we’re using simnet parameters, we
ensure that transaction acceptance/validation is identical to that of
main net.
This commit adds two new cli flags: one for accepting non-std
transactions, and the other for rejecting non-std transactions.
The two flag are rejected when using concurrently. Config parsing is
set up such that, the desired policy expressed via the config always
overrides the policy set by default for a particular chain.
The doc.go files and the sample-btcd.conf file have been updated to document
the new flags exposing further policy control.
This adds a basic test harness infrastructure for the mempool package
which aims to make writing tests for it much easier.
The harness provides functionality for creating and signing transactions
as well as a fake chain that provides utxos for use in generating valid
transactions and allows an arbitrary chain height to be set. In order
to simplify transaction creation, a single signing key and payment
address is used throughout and a convenience type for spendable outputs
is provided.
The harness is initialized with a spendable coinbase output by default
and the fake chain height set to the maturity height needed to ensure
the provided output is in fact spendable as well as a policy that is
suitable for testing.
Since tests are in the same package and each harness provides a unique
pool and fake chain instance, the tests can safely reach into the pool
policy, or any other state, and change it for a given harness without
affecting the others.
In order to be able to make use of the existing blockchain.Viewpoint
type, a Clone method has been to the UtxoEntry type which allows the
fake chain instance to keep a single view with the actual available
unspent utxos while the mempool ends up fetching a subset of the view
with the specifically requested entries cloned.
To demo the harness, this also contains a couple of tests which make use
of it:
- TestSimpleOrphanChain -- Ensures an entire chain of orphans is
properly accepted and connects up when the missing parent transaction
is added
- TestOrphanRejects -- Ensure orphans are actually rejected when the
flag on ProcessTransactions is set to reject them
This modifies the config for the new mempool package such that it takes
a callback function to obtain the best chain height instead of requiring
a fully initialized blockchain.BlockChain instance.
This will make it much easier to test the mempool since the tests will
be able to provide their own height function to test various
functionality without having create and manipulate full blocks and chain
instances.
This adds a new field to the best chain state snapshot for the
calculated past median time as returned by the calcPastMedianTime
function. This is useful since it provides fast access to it without
having to acquire the chain lock which is needed to recalculate it.
This will ultimately allow the associated exported function to be
removed since it only exists to be able to calculate this exact value,
however this commit only introduces the new field in order to keep the
changes minimal.
This commit adds a `defer` statement at the top of `TestRpcServer`
which will attempt a `recover` which tears down all active harnesses in
the event that one of the tests causes a panic in the main goroutine.
Before this commit, if a buggy test caused a panic while all integration
tests were being executed, then any active harnesses would fail to be
properly torn down. This would cause the running btcd processes to be
leaked, possibly interfering with future test runs until the process was
manually killed. This commit fixes such behavior.
In order to aide in debugging, when a test panics, the test number is
printed out along with a full stack-trace from the start of the test to
the panic point.
This corrects several issues with the new rpctest package.
The following is an overview of the issues that have been corrected:
- The JoinNodes code was incorrect for both mempool and blocks.
- For mempool it was only checking the first node against itself
- For blocks it was only testing the height instead of hash and height
which means the chains could be different and it would claim they're
synced
- The test for mempool joins was inaccurate in a few ways:
- It was generating separate chains such that the generated
transaction was invalid (an orphan) on one chain, but not
the other
- Mempools are not automatically synced when nodes are connected and
there is a large random delay before any transaction rebroadcast
happens, so it can't be relied on for the purposes of this test
- The test for block joins was generating two independent chains of the
same height with the same difficulty and was only passing due to the
aforementioned bug in JoinNodes
- All of the ConnectNode calls were connecting the main harness outbound
to the local test harness instances
- This is not correct because ConnectNode makes the outbound
connection persistent, which means once the local test harness is
gone, it would keep trying to connect for the remainder of the tests
to a node that is never coming back and instead ends up connecting to
an independent test harness.
This commit modifies the existing Travis configuration to also install
the btcd binary within the container’s PATH.
This change is necessary in order for tests written against the new
rpctest package to work properly within the Travis testing environment.
This commit introduces a new file: rpcserver_test.go dedicated for
including integration tests for btcd using the new rpctest package.
The tests are created using a TestMain instance first creates a single
main harness which is intended to be re-used across tests instances.
Afterwards all registered RPC tests are executed, with proper clean up
being executed regardless of the passing state of the tests.
The following RPC calls are excessed by the initial set of tests added:
* getbestblock
* getblockcount
* getblockhash
This commit adds a new package (rpctest) which provides functionality
for writing automated black box tests to exercise the RPC interface.
An instance of a rpctest consists of an active btcd process running in
(typically) --simnet mode, a btcrpcclient instance connected to said
node, and finally an embedded in-memory wallet instance (the memWallet)
which manages any created coinbase outputs created by the mining btcd
node.
As part of the SetUp process for an RPC test, a test author can
optionally opt to have a test blockchain created. The second argument
to SetUp dictates the number of mature coinbase outputs desired. The
btcd process will then be directed to generate a test chain of length:
100 + numMatureOutputs.
The embedded memWallet instance acts as a minimal, simple wallet for
each Harness instance. The memWallet itself is a BIP 32 HD wallet
capable of creating new addresses, creating fully signed transactions,
creating+broadcasting a transaction paying to an arbitrary set of
outputs, and querying the currently confirmed balance.
In order to test various scenarios of blocks containing arbitrary
transactions, one can use the Generate rpc call via the exposed
btcrpcclient connected to the active btcd node. Additionally, the
Harness also exposes a secondary block generation API allowing callers
to create blocks with a set of hand-selected transactions, and an
arbitrary BlockVersion or Timestamp.
After execution of test logic TearDown should be called, allowing the
test instance to clean up created temporary directories, and shut down
the running processes.
Running multiple concurrent rpctest.Harness instances is supported in
order to allow for test authors to exercise complex scenarios. As a
result, the primary interface to create, and initialize an
rpctest.Harness instance is concurrent safe, with shared package level
private global variables protected by a sync.Mutex.
Fixes#116.
The protocol was silently upgrade in Core some time ago to enforce a
limit of 256 for the user agent in version messages. This updates wire
to coincide with that change and consequently includes a wire version
bump to 0.4.1.
This does the minimum work necessary to refactor the mempool code into
its own package. The idea is that separating this code into its own
package will greatly improve its testability, allow independent
benchmarking and profiling, and open up some interesting opportunities
for future development related to the memory pool.
There are likely some areas related to policy that could be further
refactored, however it is better to do that in future commits in order
to keep the changeset as small as possible during this refactor.
Overview of the major changes:
- Create the new package
- Move several files into the new package:
- mempool.go -> mempool/mempool.go
- mempoolerror.go -> mempool/error.go
- policy.go -> mempool/policy.go
- policy_test.go -> mempool/policy_test.go
- Update mempool logging to use the new mempool package logger
- Rename mempoolPolicy to Policy (so it's now mempool.Policy)
- Rename mempoolConfig to Config (so it's now mempool.Config)
- Rename mempoolTxDesc to TxDesc (so it's now mempool.TxDesc)
- Rename txMemPool to TxPool (so it's now mempool.TxPool)
- Move defaultBlockPrioritySize to the new package and export it
- Export DefaultMinRelayTxFee from the mempool package
- Export the CalcPriority function from the mempool package
- Introduce a new RawMempoolVerbose function on the TxPool and update
the RPC server to use it
- Update all references to the mempool to use the package.
- Add a skeleton README.md
This reduces the mempool lock contention by removing an unnecessary
check when responding to a "mempool" request.
In particular, the code first gets a list of all transactions from the
mempool and then iterates them in order to construct the inventory
vectors and apply bloom filtering if it is enabled. Since it is
possible that the transaction was removed from the mempool by another
thread while that list is being iterated, the code was checking if each
transaction was still in the mempool. This is a pointless check because
the transaction might still be removed at any point after the check
anyways. For example, it might be removed after the mempool response
has been sent to the remote peer or even while the loop is still
iterating.
This refactors the notification state mutex out of the state itself to
the client. This is being done since the code makes a copy of the
notification state and accesses that copy immutably, and therefore there
is no need for it to have its own mutex.
This makes vet happy by manually copying the notification state fields
during the deep copy instead of copying the entire struct which contains
an embedded mutex.
This exposes a new function on the ScriptBuilder type named AddOps that
allows multiple opcodes to be added via a single call and adds tests to
exercise the new function.
Finally, it updates a couple of places in the signing code that were
abusing the interface by setting its private script directly to use the
new public function instead.
This rewrites the shutdown logic to simplify the shutdown signalling.
All cleanup is now run from deferred functions in the main function and
channels are used to signal shutdown either from OS signals or from
other subsystems such as the RPC server and windows service controller.
The RPC server has been modified to use a new channel for signalling
shutdown that is exposed via the RequestedProcessShutdown function
instead of directly calling Stop on the server as it previously did.
Finally, it adds a few checks for early termination during the main
start sequence so the process can be stopped without starting all the
subsystems if desired.
This is a backport of the equivalent logic from Decred with a few slight
modifications. Credits go to @jrick.
This moves several of the chain constants to the Params struct in the
chaincfg package which is intended for that purpose. This is mostly a
backport of the same modifications made in Decred along with a few
additional things cleaned up.
The following is an overview of the changes:
- Comment all fields in the Params struct definition
- Add locals to BlockChain instance for the calculated values based on
the provided chain params
- Rename the following param fields:
- SubsidyHalvingInterval -> SubsidyReductionInterval
- ResetMinDifficulty -> ReduceMinDifficulty
- Add new Param fields:
- CoinbaseMaturity
- TargetTimePerBlock
- TargetTimespan
- BlocksPerRetarget
- RetargetAdjustmentFactor
- MinDiffReductionTime
This updates all code to make use of the new chainhash package since the
old wire.ShaHash type and related functions have been removed in favor
of the abstracted package.
Also, while here, rename all variables that included sha in their name
to include hash instead.
This is mostly a backport of some of the same modifications made in
Decred along with a few additional things cleaned up. In particular,
this updates the code to make use of the new chainhash package.
Also, since this required API changes anyways and the hash algorithm is
no longer tied specifically to SHA, all other functions throughout the
code base which had "Sha" in their name have been changed to Hash so
they are not incorrectly implying the hash algorithm.
The following is an overview of the changes:
- Remove the wire.ShaHash type
- Update all references to wire.ShaHash to the new chainhash.Hash type
- Rename the following functions and update all references:
- wire.BlockHeader.BlockSha -> BlockHash
- wire.MsgBlock.BlockSha -> BlockHash
- wire.MsgBlock.TxShas -> TxHashes
- wire.MsgTx.TxSha -> TxHash
- blockchain.ShaHashToBig -> HashToBig
- peer.ShaFunc -> peer.HashFunc
- Rename all variables that included sha in their name to include hash
instead
- Update for function name changes in other dependent packages such as
btcutil
- Update copyright dates on all modified files
- Update glide.lock file to use the required version of btcutil
This is a backport of the chainhash package made in Decred along with a
few additional things cleaned up, finished test coverage, and rewording
of some documentation to make it more generic.
In particular, the new package provides the definition of the hash type
and associated hashing functions which will allow the rest of the code to be
agnostic to the specific hash algorithm.
This only implements the package and does not change any of the code
base over to use it.
Putting the test code in the same package makes it easier for forks
since they don't have to change the import paths as much and it also
gets rid of the need for internal_test.go to bridge.
This same thing should probably be done for the majority of the code
base.
Rather than making the caller to pass in the median time source on
ProcessBlock and IsCurrent, modify the Config struct to include the
median time source and associate it with the chain instance when it is
created.
This is being done because both the ProcessBlock and IsCurrent functions
require access to the blockchain state already, it is a little bit safer
to ensure the time source matches the chain instance state, it
simplifies the caller logic, and it also allows its use within the logic
of the blockchain package itself which will be required by upcoming
rule change warning logic that is part of BIP9.
This commit drastically reduces the number of allocations needed to
deserialize a transaction and its scripts by using the combination of a
free list for initially deserializing the individual scripts along with
copying them into a single contiguous byte slice after the final size is
known and modifying each script in the transaction to point to its
location within the contiguous blob.
The end result is only a single allocation that holds all of the scripts
for a transaction regardless of the total number of scripts it has.
The script free list allows a maximum of 12,500 items with each buffer
being 512 bytes. This implies it will have a peak usage of 6.1MB. The
values were chosen based on profiling data and a desire to allow at
least 100 scripts per transaction to be simultaneously deserialized by
125 peers.
Also, while optimizing, decode directly into the existing previous
outpoint structure of each transaction input in order to avoid the extra
allocation per input that is otherwise caused when the local escapes to
the heap.
The following is a before and after comparison of the allocations
with the benchmarks that did not change removed:
benchmark old allocs new allocs delta
-----------------------------------------------------------
ReadTxOut 1 0 -100.00%
ReadTxIn 2 0 -100.00%
DeserializeTxSmall 7 5 -28.57%
DeserializeTxLarge 11146 6 -99.95%
The current code involves a ton of small allocations which is harsh on
the garbage collector and in turn causes a lot of addition runtime
overhead both in terms of additional memory and processing time.
In order to improve the situation, this drasticially reduces the number
of allocations by creating contiguous slices of objects and
deserializing into them. Since the final data structures consist of
slices of pointers to the objects, they are constructed by pointing them
into the appropriate offset of the contiguous slice.
This could be improved upon even further by converting all of the data
structures provided the wire package to be slices of contiguous objects
directly, however that would be a major breaking API change and would
end up requiring updating a lot more code in every caller. I do think
that ultimately the API should be changed, but the changes in this
commit already makes a massive difference and it doesn't require
touching any of the callers, so it is a good place to begin.
The following is a before and after comparison of the allocations
with the benchmarks that did not change removed:
benchmark old allocs new allocs delta
-----------------------------------------------------------
DeserializeTxLarge 16715 11146 -33.32%
DecodeGetHeaders 501 2 -99.60%
DecodeHeaders 2001 2 -99.90%
DecodeGetBlocks 501 2 -99.60%
DecodeAddr 3001 2002 -33.29%
DecodeInv 50003 3 -99.99%
DecodeNotFound 50002 3 -99.99%
DecodeMerkleBlock 107 3 -97.20%
Since the protocol encodes timestamps differently depending on the
message, the code currently decodes into a local variable and then
converts it to a time.Time. However, this causes an allocation due to
the local having to escape to the heap in order for the readElement
function to write to it.
So, in order to avoid that, this introduces two new types for a
timestamp named uint32Time and int64Time that are encoded as the
respective type on the read. When calling the readElements function,
the time.Time field in the message is cast to a pointer of the
appropriate type which effectively allows the allocations to be avoided.
The following is a before and after comparison of the allocations
with the benchmarks that did not change removed:
benchmark old allocs new allocs delta
----------------------------------------------------------------------
ReadBlockHeader 1 0 -100.00%
DecodeHeaders 4001 2001 -49.99%
DecodeAddr 4001 3001 -24.99%
DecodeMerkleBlock 108 107 -0.93%
This introduces a new binary free list which provides a concurrent safe
list of unused buffers for the purpose of serializing and deserializing
primitive integers to their raw binary bytes.
For convenience, the type also provides functions for each of the
primitive unsigned integers that automatically obtain a buffer from the
free list, perform the necessary binary conversion, read from or write
to the given io.Reader or io.Writer, and return the buffer to the free
list.
A global instance of the type has been introduced with a maximum number
of 1024 items. Since each buffer is 8 bytes, it will consume a maximum
of 8KB. Theoretically, this value would only allow up to 1024 peers
simultaneously reading and writing without having to resort to burdening
the garbage collector with additional allocations. However, due to the
fact the code is designed in such a way that the buffers are quickly
used and returned to the free list, in practice it can support much more
than 1024 peers without involving the garbage collector since it is
highly unlikely every peer would need a buffer at the exact same time.
The following is a before and after comparison of the allocations
with the benchmarks that did not change removed:
benchmark old allocs new allocs delta
-------------------------------------------------------------
WriteVarInt1 1 0 -100.00%
WriteVarInt3 1 0 -100.00%
WriteVarInt5 1 0 -100.00%
WriteVarInt9 1 0 -100.00%
ReadVarInt1 1 0 -100.00%
ReadVarInt3 1 0 -100.00%
ReadVarInt5 1 0 -100.00%
ReadVarInt9 1 0 -100.00%
ReadVarStr4 3 2 -33.33%
ReadVarStr10 3 2 -33.33%
WriteVarStr4 2 1 -50.00%
WriteVarStr10 2 1 -50.00%
ReadOutPoint 1 0 -100.00%
WriteOutPoint 1 0 -100.00%
ReadTxOut 3 1 -66.67%
WriteTxOut 2 0 -100.00%
ReadTxIn 5 2 -60.00%
WriteTxIn 3 0 -100.00%
DeserializeTxSmall 15 7 -53.33%
DeserializeTxLarge 33428 16715 -50.00%
SerializeTx 8 0 -100.00%
ReadBlockHeader 7 1 -85.71%
WriteBlockHeader 10 4 -60.00%
DecodeGetHeaders 1004 501 -50.10%
DecodeHeaders 18002 4001 -77.77%
DecodeGetBlocks 1004 501 -50.10%
DecodeAddr 9002 4001 -55.55%
DecodeInv 150005 50003 -66.67%
DecodeNotFound 150004 50002 -66.67%
DecodeMerkleBlock 222 108 -51.35%
TxSha 10 2 -80.00%
This adds two new flags, --txindex and --addrindex, to the addblock
utility which mirror the flags on btcd. They serve to to specify that
the transaction index and/or address index, respectively, should be
built while importing from the bootstrap file.
This is technically not 100% required since btcd will build the indexes
on the first load (when enabled) if they aren't already built, however
it is much faster to build the indexes as the blocks are being validated
(particularly for the address index), so this makes the capability
available.
This converts the project to allow btcd to be used with the glide
package manager in order to provide stable and reproducible builds
without the user having to jump through all of the hoops as they do
today.
It consists of adding a glide.yaml file which identifies the project
dependencies and locations along with a glide.lock file which contains
the complete dependency tree pinned to specific versions. Glide uses
these files to download the packages (or updates) to a local vendor
directory and checkout the correct pinned versions. The go tool, in
turn, is used to build/install btcd and will use the pinned versions in
the vendor directory.
This also updates TravisCI to build using glide, removes some of the
exceptions in the lint checks which are no longer required, and updates
the README.md with the new instructions needed to build the project with
glide.
This reduces the target ratio of freshly allocated data to live data to
10% in order to limit excessive overallocations by the garbage collector
during data bursts such as processing complex blocks or rapidly
receiving a lot of large transactions.
When an OS reboots or shuts down, it sends all processes SIGTERM before
sending SIGKILL. This allows btcd to do a proper shutdown which most
importantly closes the database.
This adds support for serving headers instead of inventory messages in
accordance with BIP0130. btcd itself does not yet make use of the
feature when receiving data.
This adds decode benchmarks for several of the messages that profiling
has identified to cause a lot of allocations in addition to those that
already exist. By adding these benchmarks, it makes it easier to get
allocation and speed statistics which can in turn be used to compare
future improvements.
The following bencharmarks have been added:
DecodeGetHeaders, DecodeHeaders, DecodeGetBlocks, DecodeAddr, DecodeInv,
DecodeNotFound, and DecodeMerkleBlock
For reference, here is the benchmark data as of this commit.
DecodeGetHeaders 93261 ns/op 24120 B/op 1004 allocs/op
DecodeHeaders 2071263 ns/op 368399 B/op 18002 allocs/op
DecodeGetBlocks 92486 ns/op 24120 B/op 1004 allocs/op
DecodeAddr 850608 ns/op 136202 B/op 9002 allocs/op
DecodeInv 17107172 ns/op 3601447 B/op 150004 allocs/op
DecodeNotFound 17522225 ns/op 3601444 B/op 150004 allocs/op
DecodeMerkleBlock 21062 ns/op 5192 B/op 222 allocs/op
This modifies the benchmarks in the wire package to avoid creating a new
reader for each iteration. This is useful since it means that showing
the memory allocations will only show the function under test instead of
the allocation for the benchmark setup as well.
The following is a before and after comparison of the allocations
with the benchmarks that did not change removed:
benchmark old allocs new allocs delta
------------------------------------------------------------
ReadVarInt1 2 1 -50.00%
ReadVarInt3 2 1 -50.00%
ReadVarInt5 2 1 -50.00%
ReadVarInt9 2 1 -50.00%
ReadVarStr4 4 3 -25.00%
ReadVarStr10 4 3 -25.00%
ReadOutPoint 2 1 -50.00%
ReadTxOut 4 3 -25.00%
ReadTxIn 6 5 -16.67%
DeserializeTxSmall 16 15 -6.25%
DeserializeTxLarge 33430 33428 -0.01%
ReadBlockHeader 8 7 -12.50%
This adds a benchmark for deserializing a large transaction that is
often referred to as the megatransaction since it is the largest Bitcoin
transaction mined to date. It consists of 5569 inputs and 1 output and
its hash is:
bb41a757f405890fb0f5856228e23b715702d714d59bf2b1feb70d8b2b4e3e08.
This is being done so there is a benchmark that tests more of a
worst-case scenario which is a better candidate for identifying and
testing improvements.
The following benchmark results shows the how much more intensive this
transaction is over the existing mock transaction:
DeserializeTxSmall 1000000 1751 ns/op 376 B/op 16 allocs/op
DeserializeTxLarge 300 5093980 ns/op 1672829 B/op 33430 allocs/op
This removes the root field and all references to it from the BlockChain
since it is no longer required.
It was previously required because the chain state was not initialized
when the instance was created. However, that is no longer the case, so
there is no reason to keep it around any longer.
This changes the script template parsing function to use a pointer into
the constant global opcode array for parsed opcodes as opposed to making
a copy of the opcode entries which causes unnecessary allocations.
Profiling showed that after roughly 48 hours of operation, this
copy was the culprit of 207 million unnecessary allocations.
This removes the logging functions that are now implemented in the peer
package as they are no longer used by btcd itself and should have been
removed when they were copied into the peer package.
It is not the responsibility of mempool to relay transactions, so
return a slice of transactions accepted to the mempool due to the
passed transaction to the caller.
This improves the tests of the priority queue to include the secondary
sort ordering as well as adds some manual entries to ensure the edge
conditions are properly tested.
This also brings the priority queue test coverage up to 100%.
having 3 int32s above the uint64s in the struct
will cause misalignment for some 32-bit architectures.
see https://golang.org/pkg/sync/atomic/#pkg-note-BUG
This aligns bytesReceived and bytesSent.
Profiles discovered that lookups into the signature cache included an
expensive comparison to the stored `sigInfo` struct. This lookup had the
potential to be more expensive than directly verifying the signature
itself!
In addition, evictions were rather expensive because they involved
reading from /dev/urandom, or equivalent, for each eviction once the
signature cache was full as well as potentially iterating over every
item in the cache in the worst-case.
To remedy this poor performance several changes have been made:
* Change the lookup key to the fixed sized 32-byte signature hash
* Perform a full equality check only if there is a cache hit which
results in a significant speed up for both insertions and existence
checks
* Override entries in the case of a colliding hash on insert Add an
* .IsEqual() method to the Signature and PublicKey types in the
btcec package to facilitate easy equivalence testing
* Allocate the signature cache map with the max number of entries in
order to avoid unnecessary map re-sizes/allocations
* Optimize evictions from the signature cache Delete the first entry
* seen which is safe from manipulation due to
the pre image resistance of the hash function
* Double the default maximum number of entries within the signature
cache due to the reduction in the size of a cache entry
* With this eviction scheme, removals are effectively O(1)
Fixes#575.
The current code is needlessly checking the number of bytes needed to
serialize the unspentness bitmap in the utxo against a maximum value
that could never be returned because the function takes a uint32 output
index which is treated as a bit offset, and converts it bytes, which
will necessarily be less than a max uint32.
This check also causes a compile error on arm where native integers are
32 bits.
This simply removes the unneeded check.
This adds a test to ensure the priority queue works properly both for
sorting by fee per KB and priorities.
Thanks to @ceejep for the original test code and idea which was
subsequently modified and cleaned up a bit to the code here.
This introduces a new indexing infrastructure for supporting optional
indexes using the new database and blockchain infrastructure along with
two concrete indexer implementations which provide both a
transaction-by-hash and a transaction-by-address index.
The new infrastructure is mostly separated into a package named indexers
which is housed under the blockchain package. In order to support this,
a new interface named IndexManager has been introduced in the blockchain
package which provides methods to be notified when the chain has been
initialized and when blocks are connected and disconnected from the main
chain. A concrete implementation of an index manager is provided by the
new indexers package.
The new indexers package also provides a new interface named Indexer
which allows the index manager to manage concrete index implementations
which conform to the interface.
The following is high level overview of the main index infrastructure
changes:
- Define a new IndexManager interface in the blockchain package and
modify the package to make use of the interface when specified
- Create a new indexers package
- Provides an Index interface which allows concrete indexes to plugin
to an index manager
- Provides a concrete IndexManager implementation
- Handles the lifecycle of all indexes it manages
- Tracks the index tips
- Handles catching up disabled indexes that have been reenabled
- Handles reorgs while the index was disabled
- Invokes the appropriate methods for all managed indexes to allow
them to index and deindex the blocks and transactions
- Implement a transaction-by-hash index
- Makes use of internal block IDs to save a significant amount of
space and indexing costs over the old transaction index format
- Implement a transaction-by-address index
- Makes use of a leveling scheme in order to provide a good tradeoff
between space required and indexing costs
- Supports enabling and disabling indexes at will
- Support the ability to drop indexes if they are no longer desired
The following is an overview of the btcd changes:
- Add a new index logging subsystem
- Add new options --txindex and --addrindex in order to enable the
optional indexes
- NOTE: The transaction index will automatically be enabled when the
address index is enabled because it depends on it
- Add new options --droptxindex and --dropaddrindex to allow the indexes
to be removed
- NOTE: The address index will also be removed when the transaction
index is dropped because it depends on it
- Update getrawtransactions RPC to make use of the transaction index
- Reimplement the searchrawtransaction RPC that makes use of the address
index
- Update sample-btcd.conf to include sample usage for the new optional
index flags
This commit is the first stage of several that are planned to convert
the blockchain package into a concurrent safe package that will
ultimately allow support for multi-peer download and concurrent chain
processing. The goal is to update btcd proper after each step so it can
take advantage of the enhancements as they are developed.
In addition to the aforementioned benefit, this staged approach has been
chosen since it is absolutely critical to maintain consensus.
Separating the changes into several stages makes it easier for reviewers
to logically follow what is happening and therefore helps prevent
consensus bugs. Naturally there are significant automated tests to help
prevent consensus issues as well.
The main focus of this stage is to convert the blockchain package to use
the new database interface and implement the chain-related functionality
which it no longer handles. It also aims to improve efficiency in
various areas by making use of the new database and chain capabilities.
The following is an overview of the chain changes:
- Update to use the new database interface
- Add chain-related functionality that the old database used to handle
- Main chain structure and state
- Transaction spend tracking
- Implement a new pruned unspent transaction output (utxo) set
- Provides efficient direct access to the unspent transaction outputs
- Uses a domain specific compression algorithm that understands the
standard transaction scripts in order to significantly compress them
- Removes reliance on the transaction index and paves the way toward
eventually enabling block pruning
- Modify the New function to accept a Config struct instead of
inidividual parameters
- Replace the old TxStore type with a new UtxoViewpoint type that makes
use of the new pruned utxo set
- Convert code to treat the new UtxoViewpoint as a rolling view that is
used between connects and disconnects to improve efficiency
- Make best chain state always set when the chain instance is created
- Remove now unnecessary logic for dealing with unset best state
- Make all exported functions concurrent safe
- Currently using a single chain state lock as it provides a straight
forward and easy to review path forward however this can be improved
with more fine grained locking
- Optimize various cases where full blocks were being loaded when only
the header is needed to help reduce the I/O load
- Add the ability for callers to get a snapshot of the current best
chain stats in a concurrent safe fashion
- Does not block callers while new blocks are being processed
- Make error messages that reference transaction outputs consistently
use <transaction hash>:<output index>
- Introduce a new AssertError type an convert internal consistency
checks to use it
- Update tests and examples to reflect the changes
- Add a full suite of tests to ensure correct functionality of the new
code
The following is an overview of the btcd changes:
- Update to use the new database and chain interfaces
- Temporarily remove all code related to the transaction index
- Temporarily remove all code related to the address index
- Convert all code that uses transaction stores to use the new utxo
view
- Rework several calls that required the block manager for safe
concurrency to use the chain package directly now that it is
concurrent safe
- Change all calls to obtain the best hash to use the new best state
snapshot capability from the chain package
- Remove workaround for limits on fetching height ranges since the new
database interface no longer imposes them
- Correct the gettxout RPC handler to return the best chain hash as
opposed the hash the txout was found in
- Optimize various RPC handlers:
- Change several of the RPC handlers to use the new chain snapshot
capability to avoid needlessly loading data
- Update several handlers to use new functionality to avoid accessing
the block manager so they are able to return the data without
blocking when the server is busy processing blocks
- Update non-verbose getblock to avoid deserialization and
serialization overhead
- Update getblockheader to request the block height directly from
chain and only load the header
- Update getdifficulty to use the new cached data from chain
- Update getmininginfo to use the new cached data from chain
- Update non-verbose getrawtransaction to avoid deserialization and
serialization overhead
- Update gettxout to use the new utxo store versus loading
full transactions using the transaction index
The following is an overview of the utility changes:
- Update addblock to use the new database and chain interfaces
- Update findcheckpoint to use the new database and chain interfaces
- Remove the dropafter utility which is no longer supported
NOTE: The transaction index and address index will be reimplemented in
another commit.
mempoolPolicy contains the values that configure the mempool policy.
This decouples the values from the internals of btcd to move closer
to a mempool package.
Putting the test code in the same package makes it easier for forks
since they don't have to change the import paths as much and it also
gets rid of the need for internal_test.go to bridge.
This same thing should probably be done for the majority of the code
base.
This modifies the chaincfg package to register the default network
params via the init function instead of manually hard coding their data
into the maps. This is less error prone when adding new default
networks.
A new function named mustRegister has been introduced that panics if
there are any errors when registering the network that the new code
makes use of and appropriate tests have been added.
This optimizes the way in which the maps are limited by the block
manager.
Previously the code would read a cryptographically random value large
enough to construct a hash, find the first entry larger than that value,
and evict it.
That approach is quite inefficient and could easily become a bottleneck
when processing transactions due to the need to read from a source such
as /dev/urandom and all of the subsequent hash comparisons.
Luckily, strong cryptographic randomness is not needed here. The primary
intent of limiting the maps is to control memory usage with a secondary
concern of making it difficult for adversaries to force eviction of
specific entries.
Consequently, this changes the code to make use of the pseudorandom
iteration order of Go's maps along with the preimage resistance of the
hashing function to provide the desired functionality. It has
previously been discussed that the specific pseudorandom iteration order
is not guaranteed by the Go spec even though in practice that is how it
is implemented. This is not a concern however because even if the
specific compiler doesn't implement that, the preimage resistance of the
hashing function alone is enough.
Thanks to @Roasbeef for pointing out the efficiency concerns and the
fact that strong cryptographic randomness is not necessary.
This simply exports and adds some comments to the fields of the
BlockTemplate struct.
This is primarily being done as a step toward being able to separate the
mining code into its own package, but also it makes sense on its own
because code that requests new block template necessarily examines the
returned fields which implies they should be exported.
This prevents the node from repeatedly requesting and rejecting the
same transaction as different peers inv the same transaction.
Idea from Bitcoin Core commit 0847d9cb5fcd2fdd5a21bde699944d966cf5add9
Also, limit the number of both requested blocks and transactions.
The --blocksonly configuration option disables accepting transactions
from remote peers. It will still accept, relay, and rebroadcast
valid transactions sent via RPC or websockets.
This updates the Go versions using when running the TravisCI integration
tests to reflect the latest supported Go versions.
Also, the vet tool moved into the Go source tree as of Go 1.5. Its previous
location in the x/tools repo was deprecated at that time and has now
been removed.
Finally, old tool path is no longer needed, so it has been removed.
The vet tool moved into the Go source tree as of Go 1.5. Its previous
location in the x/tools repo was deprecated at that time and has now
been removed.
This commit updates the .travis.yml configuration to avoid fetching vet
from the old location and to simply use the version now available as
part of the standard Go install.
Also, while here, remove the check for changing the tool path since it
is no longer needed.
This modifies the peer package to add support for the sendheaders
protocol message introduced by BIP0030.
NOTE: This does not add support to btcd itself. That requires the server
and sync code to make use of the new functionality exposed by these
changes. As a result, btcd will still be using protocol version 70011.
This ensures the channel passed to QueueMessage is writable and that
QueueMessage will not read from the channel (write-only).
This change is merely a safety change. If a user of the API passes
a read-only channel to QueueMessage, it will now be caught at compile
time instead of panicking during runtime.
Also update internal functions.
When the RPC server is not running a buffered transaction notification
channel fills and eventually blocks. This commit ensures that the
channel continues to be drained irrespective of the RPC server status.
Since the latest golint no longer works with on Go 1.4 and Go 1.4 is no
longer officially supported by btcsuite, remove it from the
configurations tested by TravisCI.
This commit updates the main README.md and docs/README.md files to
replace the references to the now dead btcgui project with the
Windows-only Paymetheus project.
While here, it also updates some information to make it more current and
accurately describe the current status.
The existing check to only log errors from websocket connections when
the error is not because the connection is being shutdown only detects
the single case where the remote client was shutdown.
This commit modifies that logic to include when the connection is being
forcibly shutdown via closing the quit channel and when it has been
shutdown from a client side close.
Now that Go 1.6 has been released, update the required Go version in the
README to 1.5 and add Go 1.6 to the configurations tested by TravisCI.
Also, while here, update the Go 1.4 and 1.5 versions tested by TravisCI
to the latest point releases.
This updates the findcheckpoint utility to work when there are not
already any checkpoints. This doesn't really matter for Bitcoin at the
current time, but if a new testnet is created it will not have any
checkpoints to start with and this change also means the utility can
work for alts.
While here, switch a couple of error prints to ensure they contain a
final newline.
This removes the intermediate transaction log that was introduced as a
part of the database cache as a workaround for leveldb batches causing
massive memory usage spikes in favor of the recently introduced leveldb
transaction interface which no longer has the memory usage issues.
This approach is preferred because it can avoid the extra memory needed
for the transaction log and therefore all of the intermediate states as
well. As a result, the default cache size has been doubled since it
equals roughly the same amount of overall memory usage and the flush
interval has been raised as well.
Dynamic ban scores consist of a persistent and a decaying component. The
persistent score can be used to create simple additive banning policies
simlar to those found in other bitcoin node implementations. The
decaying score enables the creation of evasive logic which handles
misbehaving peers (especially application layer DoS attacks) gracefully
by disconnecting and banning peers attempting various kinds of flooding.
Dynamic ban scores allow these two approaches to be used in tandem.
This pull request includes the following:
- Dynamic ban score type & functions, with tests for core functionality
- Ban score of connected peers can be queried via rpc (getpeerinfo)
- Example policy with decaying score increments on mempool and getdata
- Logging of misbehavior once half of the ban threshold is reached
- Banning logic can be disabled via configuration (enabled by default)
- User defined ban threshold can be set via configuration
This updates a couple of logging statements to use the serverPeer
instance instead of the embedded peer.Peer so they are consistent with
all of the other log statements.
This commit does not change functionality. It makes the creation of inbound and outbound peers more homogeneous. As a result the Start method of peer was removed as it was found not to be necessary. This is the first of several pull requests/commits designed to make the peer public API and internals less complex.
This implements the wire protocol encoding portion of a new
sendheaders message as described by BIP0130. It purpose is to request
that a peer sends header commands instead of inv commands when
announcing new blocks. This includes a protocol version bump to 70012
and a wire version bump to 0.4.0.
Note that this does not implement logic to handle the command in btcd,
rather it only makes the command available at the wire protocol level.
A future commit which honors the command and therefore provides full
BIP0130 support is still required.
This optimizes the filter address handling code in the RPC server
handleSearchRawTransasctions path in a few ways:
- Only normalize the filter addresses provided in the command once
rather than for every transaction being returned
- Reset the passes filter flag just before it's used
- Use a local for the encoded address to avoid the bounds checking
overhead of accessing it from the slice
- Avoiding subsequent map lookups once the filter has already passed
Also, while here, modify a few of the related comments to match code
style and consistency.
This modifies the conversion of the output index from the JSON-based
test data for valid and invalid transactions as well as the signature
hash type for signature hash tests to first convert to a signed int and
then to an unsigned int. This is necessary because the result of a
direct conversion of a float to an unsigned int is implementation
dependent and doesn't result in the expected value on all platforms.
Also, while here, change the function names in the error prints to match
the actual names.
Fixes#600.
This modifies the test for createrawtransaction to specify the constant
size passed as an int64. This is necessary because the NewCmd function
accepts the parameters as interfaces in order to support varargs and a
raw numeric constant is treated as an integer. Since the constant value
is larger than an int32, this causes certain platforms like ARM which
treat a raw integer as a 32-bit integer to fail to compile.
This is work towards #600.
This commit adds a database cache layer to the ffldb database backend so
that callers can commit multiple transactions without having to incur
the overhead of a disk sync on every new block.
This commit contains a complete redesign and rewrite of the database
package that approaches things in a vastly different manner than the
previous version. This is the first part of several stages that will be
needed to ultimately make use of this new package.
Some of the reason for this were discussed in #255, however a quick
summary is as follows:
- The previous database could only contain blocks on the main chain and
reorgs required deleting the blocks from the database. This made it
impossible to store orphans and could make external RPC calls for
information about blocks during the middle of a reorg fail.
- The previous database interface forced a high level of bitcoin-specific
intelligence such as spend tracking into each backend driver.
- The aforementioned point led to making it difficult to implement new
backend drivers due to the need to repeat a lot of non-trivial logic
which is better handled at a higher layer, such as the blockchain
package.
- The old database stored all blocks in leveldb. This made it extremely
inefficient to do things such as lookup headers and individual
transactions since the entire block had to be loaded from leveldb (which
entails it doing data copies) to get access.
In order to address all of these concerns, and others not mentioned, the
database interface has been redesigned as follows:
- Two main categories of functionality are provided: block storage and
metadata storage
- All block storage and metadata storage are done via read-only and
read-write MVCC transactions with both manual and managed modes
- Support for multiple concurrent readers and a single writer
- Readers use a snapshot and therefore are not blocked by the writer
- Some key properties of the block storage and retrieval API:
- It is generic and does NOT contain additional bitcoin logic such spend
tracking and block linking
- Provides access to the raw serialized bytes so deserialization is not
forced for callers that don't need it
- Support for fetching headers via independent functions which allows
implementations to provide significant optimizations
- Ability to efficiently retrieve arbitrary regions of blocks
(transactions, scripts, etc)
- A rich metadata storage API is provided:
- Key/value with arbitrary data
- Support for buckets and nested buckets
- Bucket iteration through a couple of different mechanisms
- Cursors for efficient and direct key seeking
- Supports registration of backend database implementations
- Comprehensive test coverage
- Provides strong documentation with example usage
This commit also contains an implementation of the previously discussed
interface named ffldb (flat file plus leveldb metadata backend). Here
is a quick overview:
- Highly optimized for read performance with consistent write performance
regardless of database size
- All blocks are stored in flat files on the file system
- Bulk block region fetching is optimized to perform linear reads which
improves performance on spindle disks
- Anti-corruption mechanisms:
- Flat files contain full block checksums to quickly an easily detect
database corruption without needing to do expensive merkle root
calculations
- Metadata checksums
- Open reconciliation
- Extensive test coverage:
- Comprehensive blackbox interface testing
- Whitebox testing which uses intimate knowledge to exercise uncommon
failure paths such as deleting files out from under the database
- Corruption tests (replacing random data in the files)
In addition, this commit also contains a new tool under the new database
directory named dbtool which provides a few basic commands for testing the
database. It is designed around commands, so it could be useful to expand
on in the future.
Finally, this commit addresses the following issues:
- Adds support for and therefore closes#255
- Fixes#199
- Fixes#201
- Implements and closes#256
- Obsoletes and closes#257
- Closes#247 once the required chain and btcd modifications are in place
to make use of this new code
This implementation ensures that all addresses have an equal chance of
being included in the addr message. It also moves the pseudorandom number
generator seeding to package level so that it can be overridden for
testing if required.
When a persistent peer is disconnected (for example due to a
network timeout), a connection retry is issued. The logic for
doing so failed to remove the peer from the peerState, causing
dead peer connections to fill the peerState. Since connections
in the peerState are counted towards the maxPeers limit, this
would cause btcd to eventually stop retrying connection.
This commit fixes the issue by properly removing the peer from
the peerState.
The getaddr msg is usually replied to with an addr msg, but if
the other peer does not have any addresses to share it will not
reply at all (instead of replying with an addr msg with 0 addresses).
Therefore, the getaddr msg is not guaranteed a reply, so this commit
removes it from stall detection to avoid incorrectly kicking
such peers.
In case of an error during protocol negotiation in handleVersionMsg,
immediately break out and prevent further processing of OnVersion
listener which generally depends upon peer attributes like NA to be set
during the negotiation. Fixes#579.
The CPU miner relies on the mempool, so the mempool has to be created
before calling the function to create the CPU miner. When PR #568
introduced the mempool config struct, it moved the mempool creation
after the miner creation, which leads to the CPU miner crashing due to
trying to access a nil mempool.
This move the CPU miner creation after the mempool creation
appropriately.
This creates a skeleton mining package that simply contains a few of the
definitions used by the mining and mempool code.
This is a step towards decoupling the mining code from the internals of
btcd and ultimately will house all of the code related to creating block
templates and CPU mining.
The main reason a skeleton package is being created before the full
blown package is ready is to avoid blocking mempool separation which
relies on these type definitions.
This introduces the concept of a new interface named TxSource which aims
to generically provide a concurrent safe source of transactions to be
considered for inclusion in a new block. This is a step towards
decoupling the mining code from the internals of btcd. Ultimately the
intent is to create a separate mining package.
The new TxSource interface relies on a new struct named miningTxDesc,
which describes each entry in the transaction source. Once this code is
refactored into a separate mining package, the mining prefix can simply
be dropped leaving the type exported as mining.TxDesc.
To go along with this, the existing TxDesc type in the mempool has been
renamed to mempoolTxDesc and changed to embed the new miningTxDesc type.
This allows the mempool to efficiently implement the MiningTxDescs
method needed to satisfy the TxSource interface.
This approach effectively separates the direct reliance of the mining
code on the mempool and its data structures. Even though the memory
pool will still be the default concrete implementation of the interface,
making it an interface offers much more flexibility in terms of testing
and even provides the potential to allow more than one source (perhaps
multiple independent relay networks, for example).
Finally, the memory pool and all of the mining code has been updated to
implement and use the new interface.
This does three things:
- Splits the priority calculation logic from the TxDesc type
- Modifies the calcPriority function to perform the value age
calculation instead of accepting it as a parameter
- Changes the starting priority to be calculated when the transaction is
added to the pool
The first is useful as it is a step towards decoupling the mining code
from the internals of the memory pool. Also, the concept of priority is
related to mining policy, so it makes more sense to have the
calculations separate than being defined on the memory pool tx
descriptor.
The second change has been made because everywhere that uses the
calcPriority function first has to calculate the value age anyways and
by making it part of the function it can be avoided altogether in
certain circumstances thereby provided a bit of optimization.
The third change ensure the starting priority is safe for reentrancy
which will be important once the mempool is split into a separate
package.
This fixes an issue introduced during the peer refactor where persistent
peers that failed the initial connection are not retried as intended.
It also improves the retry logic as follows:
- Make the retry handler goroutine simply use a for loop instead of
launching a new goroutine for each backoff attempt. Even though
goroutines are fairly cheap to create, it is much more efficient to
simply loop
- Change the retry handler to accept a flag if it is the initial attempt
- Rather than dividing the const interval by 2 everywhere and passing
the retry duration in, just half the constant and set the initial
duration to it in the retry handler
Finally, include the address of the peer in the error message when a new
outbound peer can't be created.
This introduces the concept of a mining policy struct which is used to
control block template generation instead of directly accessing the
config struct. This is a step toward decoupling the mining code from
the internals of btcd. Ultimately the intent is to create a separate
mining package.
Now that the memory pool minimum fee calculation code is also
calculating a more precise value instead of rounding up to the nearest
kilobyte boundary, the comment in NewBlockTemplate regarding this
behavior is no longer accurate. Thus, this removes the comment.
Also, while here, change the calculation to use an int64 instead of
float since it matches the precision of the new calculation code used by
the memory pool and can also avoid the need for the slower floating
point math.
When the peer code was refactored, the lists of peers were converted to
maps however the code which runs when a peer disconnects still iterates
them like a slice. This is no longer necessary since they are maps
which means the peer can simply be looked up by its ID.
Also, the old code was comparing the map entry and the peer being
removed by their pointers which could lead to potentially not properly
finding the peer. This issue is also resolved by this change since it
looks up the peer by its ID.
This defines the peer stat fields directly in the Peer struct instead of
defining them in a separate struct and embedding them. It is slightly
more efficient this way and there is really no reason to have them
defined separately anyways since they are not passed around or otherwise
used independently.
Don't log "Established connection" message on new when
DisableConnectOnNew is set and no connection was actually established.
Do log that same message when Connect() successfully connects.
Fixes a rare hang during peer tests due to the same connection being
used for different outbound peers. Also fixed passing a chan to
QueueMessage(..) which was not being waited on, so was not being used.
500 tests with various transactions and scripts, verifying that
calcSignatureHash generates the expected hash in each case.
This requires changing SigHashType to uint32; that won't affect the
standard use-cases, but will make calcSignatureHash behave more like the
Core counterpart for non-standard SigHashType settings, like those in
some of these tests.
This is more efficient and prevents duplicate entries which can
lead to no-longer-orphans being attempted be added to the mempool
multiple times.
Bug found by me, debugged with @davecgh, and patch from @davecgh
This refactors the checkTransactionStandard function, along with related
constants, from the mempool to the policy.go file since it is strictly
related to policy.
In addition, it adds tests for the function which cover all code paths.
ProcessTransaction could have accepted a new transaction into mempool
but could have returned a reject message if a no-longer-orphan
transaction failed to be accepted. This would also skip any
additional no-longer-orphans, keeping them in the orphan pool.
Instead of returning an error incorrectly, log the error and skip
the no-longer-orphan transaction. This allows the rest of the
no-longer-orphans to be processed as well.
This modifies the IP parsing code to work with IPv6 zone ids. This is
needed since the net.ParseIP function does not allow zone ids even
though net.Listen does.
rpcserver:
If the locktime is given, the transaction inputs will be set to a
non-max value, activating the locktime. The locktime for the
new transaction will be set to the given value.
This mimics Bitcoin Core commit 212bcca92089f406d9313dbe6d0e1d25143d61ff
This commit modifies the peer logging so that it will not log an error
when it's due to the remote peer disconnecting or the peer being
forcibly disconnected or shutdown.
First, it removes the documentation section from all the README.md files
and instead puts a web-based godoc badge and link at the top with the
other badges. This is being done since the local godoc tool no longer
ships with Go by default, so the instructions no longer work without
first installing godoc. Due to this, pretty much everyone uses the
web-based godoc these days anyways. Anyone who has manually installed
godoc won't need instructions.
Second, it makes sure the ISC license badge is at the top with the other
badges and removes the textual reference in the overview section.
Finally, it's modifies the Installation section to Installation and
Updating and adds a '-u' to the 'go get' command since it works for both
and thus is simpler.
This commit implements stall detection logic at the peer level to detect
and disconnect peers that are either not following the protocol in
regards to expected response messages or have otherwise stalled. This
is accomplished by setting deadlines for each message type which expects
a response and periodically checking them while properly taking into
account processing time.
There are an increasing number of nodes on the network which claim to be
full nodes, but don't actually properly implement the entire p2p
protocol even though they implement it enough to cause properly
implemented nodes to make data requests to which they never respond.
Since btcd currently only syncs new blocks via single sync peer and,
prior to this commit only had very basic stall detection, this could
lead to a situation where the block download became stalled indefinitely
due to one of these misbehaving peers. This commit fixes that issue
since the stalled peer will now be detected and disconnected which leads
to a new sync peer being selected.
This logic will also fit nicely with the future multi-peer sync model
which is on the roadmap and for which infrastructure work is underway.
Fixes#486 and fixes#229.
This commit modifies the ping logic in the peer to ping on an interval
regardless of what other messages are being sent versus the previous
method of delaying the ping each time a message that is expected to
receive data is sent.
This helps improve the ping statistics and simplifies its logic.
This fleshes out the doc.go documentation which is shown on godoc, the
README.md shown on github, and improves a couple of comments for the
fields in the Config struct.
This commit introduces package peer which contains peer related features
refactored from peer.go.
The following is an overview of the features the package provides:
- Provides a basic concurrent safe bitcoin peer for handling bitcoin
communications via the peer-to-peer protocol
- Full duplex reading and writing of bitcoin protocol messages
- Automatic handling of the initial handshake process including protocol
version negotiation
- Automatic periodic keep-alive pinging and pong responses
- Asynchronous message queueing of outbound messages with optional
channel for notification when the message is actually sent
- Inventory message batching and send trickling with known inventory
detection and avoidance
- Ability to wait for shutdown/disconnect
- Flexible peer configuration
- Caller is responsible for creating outgoing connections and listening
for incoming connections so they have flexibility to establish
connections as they see fit (proxies, etc.)
- User agent name and version
- Bitcoin network
- Service support signalling (full nodes, bloom filters, etc.)
- Maximum supported protocol version
- Ability to register callbacks for handling bitcoin protocol messages
- Proper handling of bloom filter related commands when the caller does
not specify the related flag to signal support
- Disconnects the peer when the protocol version is high enough
- Does not invoke the related callbacks for older protocol versions
- Snapshottable peer statistics such as the total number of bytes read
and written, the remote address, user agent, and negotiated protocol
version
- Helper functions for pushing addresses, getblocks, getheaders, and
reject messages
- These could all be sent manually via the standard message output
function, but the helpers provide additional nice functionality such
as duplicate filtering and address randomization
- Full documentation with example usage
- Test coverage
In addition to the addition of the new package, btcd has been refactored
to make use of the new package by extending the basic peer it provides to
work with the blockmanager and server to act as a full node. The
following is a broad overview of the changes to integrate the package:
- The server is responsible for all connection management including
persistent peers and banning
- Callbacks for all messages that are required to implement a full node
are registered
- Logic necessary to serve data and behave as a full node is now in the
callback registered with the peer
Finally, the following peer-related things have been improved as a part
of this refactor:
- Don't log or send reject message due to peer disconnects
- Remove trace logs that aren't particularly helpful
- Finish an old TODO to switch the queue WaitGroup over to a channel
- Improve various comments and fix some code consistency cases
- Improve a few logging bits
- Implement a most-recently-used nonce tracking for detecting self
connections and generate a unique nonce for each peer
isMultiSig was not verifying the number of pubkeys specified matched
the number of pubkeys provided. This caused certain non-standard
scripts to be considered multisig scripts.
However, the script still would have failed during execution.
NOTE: This only affects whether or not the script is considered
standard and does NOT affect consensus.
Also, add a test for this check.
Also, update TravisCI goclean script to remove the special casing which
ignored 'Id' from the lint output since that exception is no longer
needed. It was previously required due to the old version of btcjson,
but that is no longer in the repo.
This commit exports the ReadVarString and WriteVarString functions so
they are available for callers to use.
A variable length string is encoded as a variable length integer
containing the length of the string followed by the bytes that represent
the string itself.
See https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki for
more information.
This commit mimics Bitcoin Core commit bc60b2b4b401f0adff5b8b9678903ff8feb5867b
and includes additional tests from Bitcoin Core commit
cb54d17355864fa08826d6511a0d7692b21ef2c9
We've already been generating lowS sigs for quite a while. This removes
the malleability vector.
This mimics Bitcoin Core commit 49dd5c629df0a08cf3b1ea8085c03312d1a81696
Introduce an ECDSA signature verification into btcd in order to
mitigate a certain DoS attack and as a performance optimization.
The benefits of SigCache are two fold. Firstly, usage of SigCache
mitigates a DoS attack wherein an attacker causes a victim's client to
hang due to worst-case behavior triggered while processing attacker
crafted invalid transactions. A detailed description of the mitigated
DoS attack can be found here: https://bitslog.wordpress.com/2013/01/23/fixed-bitcoin-vulnerability-explanation-why-the-signature-cache-is-a-dos-protection/
Secondly, usage of the SigCache introduces a signature verification
optimization which speeds up the validation of transactions within a
block, if they've already been seen and verified within the mempool.
The server itself manages the sigCache instance. The blockManager and
txMempool respectively now receive pointers to the created sigCache
instance. All read (sig triplet existence) operations on the sigCache
will not block unless a separate goroutine is adding an entry (writing)
to the sigCache. GetBlockTemplate generation now also utilizes the
sigCache in order to avoid unnecessarily double checking signatures
when generating a template after previously accepting a txn to the
mempool. Consequently, the CPU miner now also employs the same
optimization.
The maximum number of entries for the sigCache has been introduced as a
config parameter in order to allow users to configure the amount of
memory consumed by this new additional caching.
This commit modifies the SearchRawTransactions and
SearchRawTransactionsVerbose functions to work properly with the latest
searchrawtransactions API in btcd.
In particular, this involves adding a new parameter for the reverse
option which specifies whether or not to include the results in reverse
order.
While current existing numeric opcodes are limited to 4 bytes, new
opcodes may need different limits.
This mimics Bitcoin Core commit 99088d60d8a7747c6d1a7fd5d8cd388be1b3e138
This commit optimizes the createVinList function which is used to
generate the JSON list of transaction inputs. It also makes it more
consistent with the createVinListPrevOut function.
In particular, it entails the following changes:
- Only do a single coinbase check and return right away instead of
checking multiple times inside the loop over the inputs
- Use a pointer for populating the details of each entry to avoid
multiple unnecessary array lookups and bounds checks
- Group all fields that populate the entry for better readability
SFNodeBloom is a new service flag that a node is required to use to
indicate that it supports bloom filtering. This includes a protocol
version bump to 70011 and a wire version bump to 0.3.0.
btcd:
The SFNodeBloom flag is set by default. A new configuration option
--nopeerbloomfilters has been added to to disable bloom filtering.
Any node advertising a version greater than or equal to 70011 that
attempts to use bloom filtering will be disconnected if bloom
filtering is disabled.
This mimics Bitcoin Core commit afb0ccaf9c9e4e8fac7db3564c4e19c9218c6b03
This commit modifies the DisasmString function to use a bytes buffer for
constructing the disassembled string instead of naive string
concatenation. This makes a huge difference when disassembling scripts
with large numbers of opcodes.
The Bitcoin wire protocol includes several fields with their lengths
encoded according to a variable length integer encoding scheme that does
not enforce a unique encoding for all numbers.
This can lead to a situation where deserializing and re-serializing the
same data can result in different bytes. There are no currently known
issues due to this, but it is safer to reject such subtle differences as
they could potentially lead to exploits.
Consequently, this commit modifies the varint decoding function to error
when the value is not canonically encoded which effectively means that
all messages with varints that are not canonically encoded will now be
rejected. This will not cause issues with old client versions in
regards to blocks and transactions since the data is deserialized into
memory and then reserialized before being relayed thereby effectively
erasing any non-canonical encodings.
Also, new tests have been added to ensure non-canonical encodings are
properly rejected and exercise the new code, and the default user agent
version for wire has been bumped to version 0.2.1 to differentiate the
new behavior.
The equivalent logic was implemented in Bitcoin Core by PR 2884.
This commit improves the most-recently used inventory map human readable
string to only show the inventory vectors. and adds tests for the entire
structure to bring its coverage to 100%.
In addition, removes the type assertion check in the Add function since
the internal inventory list is only managed by the type itself and the
tests would now catch any mistakes in the type of entries in the list.
This commit modifies the SearchRawTransactions and
SearchRawTransactionsVerbose functions to work properly with the latest
searchrawtransactions API in btcd.
In particular, this involves changing the return value from
[]*btcjson.TxRawResult to []*btcjson.SearchRawTransactionResult and
adding the additional optional parameter which specifies whether or not
to include information about the previous outputs.
This commit converts all block height references to int32 instead of
int64. The current target block production rate is 10 mins per block
which means it will take roughly 40,800 years to reach the maximum
height an int32 affords. Even if the target rate were lowered to one
block per minute, it would still take roughly another 4,080 years to
reach the maximum.
In the mean time, there is no reason to use a larger type which results
in higher memory and disk space usage. However, for now, in order to
avoid having to reserialize a bunch of database information, the heights
are still serialized to the database as 8-byte uint64s.
This is being mainly being done in preparation for further upcoming
infrastructure changes which will use the smaller and more efficient
4-byte serialization in the database as well.
IsUnspendable takes a public key script and returns whether it is
spendable.
Additionally, hook this into the mempool isDust function, since
unspendable outputs can't be spent.
This mimics Bitcoin Core commit 0aad1f13b2430165062bf9436036c1222a8724da
The comment says "only allow recent nodes (10mins) after we failed 30 times",
but the server actually did the opposite and allowed only recent nodes before
30 failed connection attempts. This corrects the server's behavior.
This commit modifies the createTxRawResult code path along with callers
to work with block headers as opposed to btcutil.Blocks. This in turn
allows the code in handleGetRawTransaction and
handleSearchRawTransactions to perform a much cheaper block header load
as opposed to a full block load.
While here, also very slightly optimize the createVinList function to
avoid creating a util.Tx wrapper and to take advantage of the
btcutil.Amount type added after the function was originally written
This commit updates the merkle block handling to for the latest changes
to the btcutil API and optimizes it along the way.
Previously, the code was inefficiently reloading the transactions for
the matched hashes from the database instead of simply pulling them from
the full block that was used to create the merkle block.
At the current time, there is no difference between the wire encoding
at protocol version 0 and the stable long-term storage format. These
methods are simply for consistency with the other types.
This commit updates the wire tests for transactions which force
serialization and deserialization errors to force an error in the the
transaction lock time path.
This brings the wire test coverage back up to 100%.
This commit correctly replaces persistent peers that are being retried in
the list of persistent peers so it will continue to be retried as
intended.
Also, limit the maximum retry interval for persistent peers to 5 minutes.
Fixes#463.
This change moves IsFinalizedTransaction to txscript and also changes
the first argument to take a wire.MsgTx instead of btcutil.Tx. This
is needed for an upcoming diff in which txscript will require
IsFinalizedTransaction and we do not want to import the btcd/blockchain.
This allows clients watching for both to know when all mined
transaction notifications for the block have been received.
Otherwise, clients would be aware that there is a new block, see the
exact same block hash/height in the next tx notifications, but never
know when all txs from the block have been received and processed.
This fixes a synchronization issue in btcwallet where the wallet would
mark itself synced to some block height before any of those blocks'
transactions were processed. If the RPC connection is lost between
sending the blockconnected notification and receiving the last
transaction notification, the wallet would not know of this and
continue with missing transactions.
The configured onion proxy was not being used due to checking if the
passed string had suffix '.onion', which never matched because the
port number is part of the string.
This commit adds an additional step to the README.md install section to
run the go version command and check the version so people that are
installing it for the first time and ensure they are running a high
enough version and have GOROOT and GOPATH set correctly.
This commit corrects an issue where the reject message sent in response
to rejected transactions had the Cmd field set to "block" instead of
"tx".
Fixes#436.
Tor stream isolation randomizes proxy user credentials resulting in
Tor creating a new circuit for each connection. This makes it more
difficult to correlate connections.
Idea from Wladimir J. van der Laan via Bitcoin Core.
The option -maxorphantx allows the user to specify the number of
orphan transactions to keep in memory.
Also, lower the default max orphan count from 10000 to 1000.
This commit refactors the consensus rule checks for block headers and
blocks in the blockchain package into separate functions. These changes
contain no modifications to consensus rules and the code still passes all
block consensus tests. It is only a refactoring.
This is being done to help pave the way toward supporting concurrent
downloads. While the package already supports headers-first mode up
through the latest checkpoint through the use of the BFFastAdd flag and
hard-coded checkpoints, it currently only works when downloading from a
single peer. In order to support concurrent downloads from multiple
peers, the ability for the caller to do things such as independently
checking a block header (both context-free and full-context checks) will
be needed.
There are several more changes that will be necessary to support
concurrent downloads as well, such as making the package concurrent safe,
modifying it to make use of the new database API, etc. Those changes are
planned for future commits.
The following changes were made to ListTransactionsResult (which
models the long result format used by listtransactions,
listsinceblock, etc.):
- Fee made optional (float64 -> *float64 + omitempty)
- BlockIndex made optional (int64 + omitempty -> *int64 + omitempty)
- InvolvesWatchOnly added (bool + omitempty)
- Vout added (uint32)
The following changes were made to GetTransactionDetailsResult (which
models the short result format of listtransactions):
- InvolvesWatchOnly added (bool + omitempty)
- Fee added (*float64 + omitempty)
- Vout added (uint32)
The combination of pointer types and the omitempty struct tag allow
excluding the field from the JSON object, or including it with the
zero value. This is useful in particular for the fee fields, which
should be included whenever the category is "send" even if the fee is
zero. Other optional fields which are only added to the result object
with non-zero values (such as includeswatchonly) can be reduced to
simply an omitempty tag without the pointer type.
This commit contains fixes from the results of a thorough audit of
txscript to find any cases of script evaluation which doesn't match the
required consensus behavior. These conditions are fairly obscure and
highly unlikely to happen in any real scripts, but they could have
nevertheless been used by a clever attacker with malicious intent to
cause a fork.
Test cases which exercise these conditions have been added to the
reference tests and will contributed upstream to improve the quality for
the entire ecosystem.
- Create FutureGenerateResult type with Receive() method
- Create GenerateAsync method for Client which returns a
FutureGenerateResult
- Create Generate method for Client which calls GenerateAsync
and then calls Receive() on the returned FutureGenerateResult
Unlike OP_IF and OP_NOTIF which interpret the top stack item as a
number, OP_IFDUP interprets it as a boolean. This has important
consequences because numbers are imited to int32s while booleans can be
an arbitrary number of bytes.
The offending script was found and reported by Jonas Nick through the
use of fuzzing.
Create GenerateCmd in btcjson v2. Update tests to check GenerateCmd.
Update chaincfg/params.go with a new bool in Params, GenerateSupported,
with true values in SimNetParams and RegressionNetParams and false in
the others.
Create new flag, discreteMining, in CPUMiner struct.
Add GenerateNBlocks function to cpuminer.go and handleGenerate
function to rpcserver.go.
Update documentation for the RPC calls.
- Move reference tests to test package since they are intended to
exercise the engine as callers would
- Improve the short form script parsing to allow additional opcodes:
DATA_#, OP_#, FALSE, TRUE
- Make use of a function to decode hex strings rather than manually
defining byte slices
- Update the tests to make use of the short form script parsing logic
rather than manually defining byte slices
- Consistently replace all []byte{} and [][]byte{} with nil
- Define tests only used in a specific function inside that func
- Move invalid flag combination test to engine_test since that is what
it is testing
- Remove all redundant script tests in favor of the JSON-based tests in
the data directory.
- Move several functions from internal_test.go to the test files
associated with what the tests are checking
This commit moves all code related to standard scripts into a separate
file named standard.go as well as the associated tests into
standard_test.go. Since the code in address.go and address_test.go is
only related to standard scripts, it has been combined into the new
files and the old files deleted.
The intent here is to make it clear that the code in standard.go is not
related to consensus.
This commit implements a new type, named scriptNum, for handling all
numeric values used in scripts and converts the code over to make use of
it. This is being done for a few of reasons.
First, the consensus rules for handling numeric values in the scripts
require special handling with subtle semantics. By encapsulating those
details into a type specifically dedicated to that purpose, it
simplifies the code and generally helps prevent improper usage.
Second, the new type is quite a bit more efficient than big.Ints which
are designed to be arbitrarily large and thus involve a lot of heap
allocations and additional multi-precision bookkeeping. Because this
new type is based on an int64, it allows the numbers to be stack
allocated thereby eliminating a lot of GC and also eliminates the extra
multi-precision arithmetic bookkeeping.
The use of an int64 is possible because the consensus rules dictate that
when data is interpreted as a number, it is limited to an int32 even
though results outside of this range are allowed so long as they are not
interpreted as integers again themselves. Thus, the maximum possible
result comes from multiplying a max int32 by itself which safely fits
into an int64 and can then still appropriately provide the serialization
of the larger number as required by consensus.
Finally, it more closely resembles the implementation used by Bitcoin
Core and thus makes is easier to compare the behavior between the two
implementations.
This commit also includes a full suite of tests with 100% coverage of
the semantics of the new type.
This commit contains a lot of cleanup on the txscript code to make it
more consistent with the code throughout the rest of the project. It
doesn't change any operational logic.
The following is an overview of the changes:
- Add a significant number of comments throughout in order to better
explain what the code is doing
- Fix several comment typos
- Move a couple of constants only used by the engine to engine.go
- Move a variable only used by the engine to engine.go
- Fix a couple of format specifiers in the test prints
- Reorder functions so they're defined before/closer to use
- Make the code lint clean with the exception of the opcode definitions
- Remove all redundant opcode tests in favor of the JSON-based tests
in the data directory.
- Remove duplicate stack nip test
- Add new tests to data/script_invalid.json to exercise additional
negative error paths
- Remove old unneeded pubkey trace code from opcodeCheckSig
- Simplify and improve the disassembly print function
- Add new tests to directly test all individual opcode disassembly
- Add new tests to directly test opcode disabled function which does not
get invoked during ordinary execution
- Improve test coverage of opcode.go
This commit moves the opcode execution logic from the opcode type to the
engine type because execution of an opcode modifies the engine state
(primarily the main and alternate data stacks) as opposed to the state
of the opcode. Making the engine the receiver more clearly indicates
this fact.
This commit very slightly optimizes the cryptographic hashing performed
by the script opcodes by calling the hash sum routines directly (for
those that support it) rather than allocating a new generic hash.Hash
hasher instance for them.
This commit unexports the Stack type since it is only intended to be
used internally during script execution. Further, the engine exposes
the {G,S}etStack and {G,S}etAltStack functions which return the items as
a slice of byte slices ([][]byte) for caller access while stepping.
* The cases for the 'addnode' command were previously
stacked on top the new cases for the 'node' command.
The intended behavior was to create a fall through and
handle both commands. However, trying to use this
syntax with a type switch caused the first case to be
ignored.
* addnode' specific functions and structs in the server
have been removed. Instead, the 'add' and 'del' subcommands
are now proxied to the matching 'node' cmd functions.
This commit removes the NewOutPointFromWire function from the btcjson
version 2 package so it can be used without needing the wire package as a
dependency. It also updates the test accordingly.
This results in the package only depending on core Go packages.
Closes#401.
This commit improves the way the conditional execution stack is handled in
a few ways.
First, the current execution state is now pushed onto the end of the slice
rather than the front of it. This has been done because it results in
fewer allocations and is therefore more efficient.
Second, the need for allocating and setting an initial true in the
conditional stack has been eliminated. The vast majority of scripts don't
contain any conditionals, so there is no reason to allocate a slice when
it isn't needed.
Third, a new function has been added to the engine to determine if the
current conditional branch is executing named isBranchExecuting which
handles the fact the conditional execution stack can now be empty and
improves the readability of the code.
Finally, it removes a couple of TODOs which I have verified do not apply.
This commit exports a new map named OpcodeByName which can be used to
lookup an opcode value given a human-readable opcode name.
It also modifies the test function which does short form parsing to use
the new map instead of the internal array.
Closes#267.
This commit converts the opcode map to an array to improve performance.
Benchmark of executing a standard p2pk transaction:
New: BenchmarkExecute 2000 784349 ns/op
Old: BenchmarkExecute 2000 792600 ns/op
The time is dominated by the signature checking as expected, however there
is still an increase in speed.
This commit modifies the definition of the opcodes to their hex
counterparts rather than decimal since it is far more common to see
scripts in hex. This makes it easier when manually looking at script
dumps to correlate opcodes. However, since there are also cases where it
is useful to see the decimal value of the opcode, the decimal value has
been left as a comment. Obviously converting the numbers is trivial, but
it is handy when looking at the opcode definitions to already have it
there.
In addition, it syncs the opcodes with the latest Bitcoin Core internal
opcodes for completeness and modifies the tests accordingly.
Rather than storing a separate bool for whether or not each flag is set in
every script engine instance, store the flags and check if the relevant
flag is set from each specific location.
This reduces the memory needed by each script engine instance and means
future flags will not require new fields.
This commit renames the Script type to Engine to better reflect its
purpose. It also renames the NewScript function to NewEngine to match.
This is being done because name Script for the engine is confusing since
it implies it is an actual script rather than the execution environment
for the script. It also paves the way for eventually supplying a
ParsedScript type which will be less likely to be confused with the
execution environment.
While moving the code, some additional variable names and comments have
been updated to better match the style used throughout the rest of the
code base. In addition, an attempt has been made to use consistent naming
of the engine as 'vm' instead of using different variables names as it was
previously.
Finally, the relevant engine code has been moved into a new file named
engine.go and related tests moved to engine_test.go.
This commit separates the test functions and associated helper functions
which are used to execute the reference transaction and script tests from
Bitcoin Core into a separate file named reference_test.go.
Also, add a few comments and fix a couple of typos along the way.
This commit removes the unnecessary sigScript parameter from the
txscript.NewScript function. This has bothered me for a while because it
can and really should be obtained from the provided transaction and input
index. The way it was, the passed script could technically be different
than what is in the transaction. Obviously that would be an improper use
of the API, but it's safer and more convenient to simply pull it from the
provided transaction and index.
Also, since the function signature is changing anyways, make the input
index parameter come after the transaction which it references.
This commit removes the error returns from the BlockHeader.BlockSha,
MsgBlock.BlockSha, and MsgTx.TxSha functions since they can never fail and
end up causing a lot of unneeded error checking throughout the code base.
It also updates all call sites for the change.
* Gives node operators full control of peer connectivity
* RPC adds ability to disconnect all matching non-persistent peers,
remove persistent peers, and connect to peers making them either
temporary or persistent.
This change reduces fingerprinting via timestamps in addr messages.
Previously, the last seen time for an address was updated when
certain protocol commands were received. Now, the last seen time
is set when the peer disconnects if the peer had sent a verack
message and was connected for more than 20 minutes.
This mimics Bitcoin Core commit:
9c2737901b5203f267d21d728019d64b46f1d9f3
Also, add additional sanity checking before updating the peer's
timestamp. These include:
- Do not mark a peer as connected if we never received
a version message.
- Disconnect a peer for sending a verack before btcd
sent a version
- Disconnect a peer for sending multiple verack's
This change was suggested as Countermeasure 2 in
Eclipse Attacks on Bitcoin's Peer-to-Peer Network, Ethan
Heilman, Alison Kendler, Aviv Zohar, Sharon Goldberg. ePrint Archive
Report 2015/263. March 2015.
This mimics Bitcoin Core commit c6a63ceeb4956933588995bcf01dc3095aaeb1fc
This change was suggested as Countermeasure 2 in Eclipse Attacks on
Bitcoin's Peer-to-Peer Network, Ethan Heilman, Alison Kendler, Aviv
Zohar, Sharon Goldberg. ePrint Archive Report 2015/263. March 2015.
This mimics Bitcoin Core commit
f68ba3f67bd500a64fb8932c6b41924ddc31d76f
Because FetchTransactionStore in GetBlockTemplate occasionally accesses
the internal blockchain memory structure while it is being read or modified,
a race can occur. To prevent this, FetchTransactionStore is instead
routed through the internal channel for blockchain requests.
The limited user is specified with the --rpclimituser and
--rpclimitpass options (or the equivalent in the config file).
The config struct and loadConfig() are updated to take the
new options into account. The limited user can have neither
the same username nor the same password as the admin user.
The package-level rpcLimit map in rpcserver.go specifies
the RPC commands accessible by limited users. This map
includes both HTTP/S and websocket commands.
The checkAuth function gets a new return parameter to
signify whether the user is authorized to change server
state. The result is passed to the jsonRPCRead function and
to the WebsocketHandler function in rpcwebsocket.go.
The wsClient struct is updated with an "isAdmin" field
signifying that the client is authorized to change server
state, written by WebsocketHandler and handleMessage.
The handleMessage function also checks the field to
allow or disallow an RPC call.
The following documentation files are updated:
- doc.go
- sample-btcd.conf
- docs/README.md
- docs/json_rpc_api.md
- docs/configure_rpc_server_listen_interfaces.md
This commit adds a new function which is similar to the DoubleSha256
function except it returns a ShaHash copy instead of a byte slice. It
also adds a new benchmark for it.
This can be a slight optimization in certain cases where the caller
ultimately wants a ShaHash since it can avoid a heap allocation and
additional copy to convert the result to a ShaHash (the function simply
performs a type cast against the returned array which is not possible
against a []byte).
existing: DoubleSha256 500000 3081 ns/op 32 B/op 1 allocs/op
new: DoubleSha256SH 500000 2939 ns/op 0 B/op 0 allocs/op
The hashing functions for blocks and transactions have also been updated
to make use of the new function since they directly return the ShaHash.
The transaction change in particular is quite useful since transactions
are frequently hashed and this change allows all of those hashes to avoid
an additional heap allocation.
This commit contains three classes of optimizations:
- Reducing the number of unnecessary hash copies
- Improve the performance of the DoubleSha256 function
- A couple of minor optimizations of the ShaHash functions
The first class is a result of the Bytes function on a ShaHash making a
copy of the bytes before returning them. It really should have been named
CloneBytes, but that would break the API now.
To address this, a comment has been added to the function which explicitly
calls out the copy behavior. In addition, all call sites of .Bytes on a
ShaHash in the code base have been updated to simply slice the array when
a copy is not needed. This saves a significant amount of data copying.
The second optimization modifies the DoubleSha256 function to directly use
fastsha256.Sum256 instead of the hasher interface. This reduces the
number of allocations needed. A benchmark for the function has been added
as well.
old: BenchmarkDoubleSha256 500000 3691 ns/op 192 B/op 3 allocs/op
new: BenchmarkDoubleSha256 500000 3081 ns/op 32 B/op 1 allocs/op
The final optimizations are for the ShaHash IsEqual and SetBytes functions
which have been modified to make use of the fact the type is an array and
remove an unneeded subslice.
This change was suggested as Countermeasure 6 in
Eclipse Attacks on Bitcoin’s Peer-to-Peer Network, Ethan
Heilman, Alison Kendler, Aviv Zohar, Sharon Goldberg. ePrint Archive
Report 2015/263. March 2015.
This mimics Bitcoin Core commit 1d21ba2f5ecbf03086d0b65c4c4c80a39a94c2ee
This commit contains several changes needed to update the client to use
the latest version of btcjson. In addition, it contains a couple of other
minor changes along the way.
While the underlying changes are quite large, the public API of this
package is still the same, so caller should generally not have to update
their code due to that. However, the underlying btcjson package API has
changed significantly. Since this package hides the vast majority of that
from callers, it should not afffect them very much. However, one area in
particular to watch out for is that the old btcjson.Error is now
btcjson.RPCError, so any callers doing any type assertions there will need
to update.
The following is a summary of the changes:
- The underlying btcjson significantly changed how commands work, so the
internals of this package have been reworked to be based off of requests
instead of the now non-existant btcjson.Cmd interface
- Update all call sites of btcjson.New<Foo>Cmd since they can no longer
error or take varargs
- The ids for each request are part of the request now instead of the
command to match the new btcjson model and are strict uint64s so type
assertions are no longer needed (slightly improved efficiency)
- Remove the old temporary workaround for the getbalance command with an
account of "*" since btcwallet has since been fixed
- Change all instances of JSONToAmount to btcutil.NewAmount since that
function was removed in favor of btcutil.Amount
- Change all btcws invocations to btcjson since they have been combined
In order to avoid prior situations of stalled syncs due to
outdated peer height data, we now update block heights up peers in
real-time as we learn of their announced
blocks.
Updates happen when:
* A peer sends us an orphan block. We update based on
the height embedded in the scriptSig for the coinbase tx
* When a peer sends us an inv for a block we already know
of
* When peers announce new blocks. Subsequent
announcements that lost the announcement race are
recognized and peer heights are updated accordingly
Additionally, the `getpeerinfo` command has been modified
to include both the starting height, and current height of
connected peers.
Docs have been updated with `getpeerinfo` extension.
- Delete spent TX in setclearSpentData when unspent by block
disconnect on reorg; return an error when there's more than
one record to delete in the spent TX as that should never
happen.
- Test spent TX deletion when reorg causes block disconnect.
- Test for correct NewestSha results after DropAfterBlockBySha.
- Fix DropAfterBlockBySha to update info for NewestSha.
- Updated copyright statements in modified files
Fix#303 by changing the addrindex key prefix to 3 characters so that
it's easy to check length when dropping the index. To drop the old
index, check to make sure we aren't dropping any entries that end in
"sx" or "tx" as those aren't part of the addrindex. Update test to
deal with the new prefix length.
Fix#346 by changing the pointers in the mempool's addrindex map to
wire.ShaHash 32-byte values. This lets them be deleted even if the
transaction data changes places in memory upon expanding the maps.
Change the way addrindex uint32s are stored to big-endian in order to
sort the transactions on disk in chronological/dependency order.
Change the "searchrawtransactions" RPC call to return transactions
from the database before the memory pool so that they're returned in
order. This commit DOES NOT do topological sorting of the memory pool
transactions to ensure they're returned in dependency order. This may
be a good idea for a future enhancement.
Add addrindex versioning to automatically drop the old/incompatible
version of the index and rebuild with the new sort method and key
prefix.
- Use explicit index values for byte slices
- Fix a bug in FetchTxsForAddr that allocated an extra 10 bytes
for each address index
- Add missing iterator release in error path
- Check for iterator errors.
The ScriptVerifyLowS flag defines that script signatures must
comply with the DER format as well as have an S value less than
or equal to the half order.
The ScriptVerifyCleanStack flag requires that only a single
stack element remains after evaluation and that when interpreted
as a bool, it must be true. This is BIP0062, rule 6.
This mimics Bitcoin Core commit b6e03cc59208305681745ad06f2056ffe6690597
This commit modifies the argument handling for btcctl to treat a
parameter that is a single dash as an indicator to read that paramter from
stdin instead.
This change allows commands, such as the submitblock, to accept data piped
from stdin for any parameter. This, in turn, allows large arguments, such
as blocks, which can often be too big for a single argument due to
Operating System limitations to be submitted by putting them into a file
and redirecting stdin.
For example:
btcctl submitblock - <block.hex
cat block.hex | btcctl submitblock -
btcctl sendrawtransaction - <tx.hex
cat tx.hex | btcctl sendrawtransaction -
This commit adds a new function to the blockchain package named
IsCoinBaseTx which performs the same function as IsCoinBase except it
takes raw wire transactions as opposed to the higher level util
transactions.
While here, it also adds a file for benchmarks along with a couple of
benchmarks for the IsCoinBase and IsCoinBaseTx functions.
Finally, the function was very slightly optimized:
BenchmarkIsCoinBaseOld 100000000 10.7 ns/op 0 B/op 0 allocs/op
BenchmarkIsCoinBaseNew 200000000 6.05 ns/op 0 B/op 0 allocs/op
For every transaction in a newly accepted block, process the orphan
pool moving now no longer orphan transactions to the mempool.
Previously, no longer orphan transactions would remain in the orphan
pool.
This change increases the maximum allowed bytes allowed in pushed
data to be considered a nulldata transaction. This matches the current
value the reference implementation uses by default.
This commit provides a new function named PkScriptLocs on the MsgTx type
which can be used to efficiently retrieve a list of offsets for the public
key scripts for the serialized form of the transaction.
This is useful for certain applications which store fully serialized
transactions and want to be able to quickly index into the serialized
transaction to extract a give public key script directly thereby avoiding
the need to deserialize the entire transaction.
This commit modifies finalized transaction check used by the memory pool
and block templates to use the network adjusted time instead of the
unadjusted local time. This helps keep the transactions accepted to the
memory pool, and hence allowed to relay, more consistent across nodes.
When the fields in the command for the getnetworkhashps RPC don't have the
fields set, use the intended default values.
Since the btcjson package sets these fields to the default values when a
command is unmarshaled from the wire, this typically isn't necessary.
However, when the RPC server calls the handler internally with optional
command fields set to nil, as is the case in getmininginfo, the defaults
need to be set as well.
By exporting StandardVerifyFlags, clients can ensure they create
transactions that btcd will accept into its mempool.
This flag doesn't belong in txscript. It belongs in a
policy package. However, this is currently the least worse place.
This modifies the recently added code which rejects free/low-fee
transactions with insufficient priority to ignore resurrected transactions
from disconnected blocks. It also exempts resurrected transactions from
the free/low-fee rate limiting.
This commit moves the definition of the flags which are needed to check
transaction scripts higher up the call stack to pave the way for adding
support for v3 blocks. While here, also spruce up a couple of sections.
There are no functional changes in this commit.
This commit contains what is essentially a complete rewrite of the btcctl
utility to make use of the new features provided by the latest version
btcjson and improve several things along the way. The following
summarizes the changes:
- The supported commands and handling now come directly from btcjson, so
it is no longer necessary to manually add new commands. Once a command
has been registered with btcjson, it will automatically become usable by
btcctl complete with full error handling (once it is re-compiled of
course)
- Rather than dumping the entire list of commands on every error, the user
now must specifically request the list of command via the -l option
- The list of commands is now categorized by chain and wallet and
alphabetized
- The help flag now only shows the help options instead of also dumping
all of the commands
- The error display on valid commands with invalid parameters has been
greatly improved to show the specific parameter number, reason, and
error code
- When a valid command is specified with invalid parameter, only the usage
for that specific command is shown now
- It is now possible to use a SOCKS5 proxy for connection
- The output of commands has been improved in the following ways:
- Strings on commands such as getbestblockhash no longer have quotes
wrapped around them
- Fields that are integers no longer show in scientific notation when
they are large (timestamps for example)
This closes#305 as a side effect.
This commit updates the SearchRawTransactionsCmd verbose parameter in the
latest version of btcjson to an integer to match recent changes to the
previous version of btcjson.
By default, have the mempool reject free and low-fee transactions that
have insufficient priority to be mined in the next block.
Addtionally, add a new configuration option, -norelaypriority, to
disable the check.
This commit converts the RPC server over to use the new features available
in the latest version of btcjson and improve a few things along the way.
This following summarizes the changes:
- All btcjson imports have been updated to the latest package version
- The help has been significantly improved
- Invoking help with no command specified now provides an alphabetized
list of all supported commands along with one-line usage
- The help for each command is automatically generated and provides much
more explicit information such as the type of each parameter, whether
or not it's optional or required, etc
- The websocket-specific commands are now provided when accessing the
help when connected via websockets
- Help has been added for all websocket-specific commands and is only
accessible when connected via websockets
- The error returns and handling of both the standard and websocket
handlers has been made consistent
- All RPC errors have been converted to the new RPCError type
- Various variables have been renamed for consistency
- Several RPC errors have been improved
- The commands that are marked as unimplemented have been moved into the
separate map where they belong
- Several comments have been improved
- An unnecessary check has been removed from the createrawtransaction
handler
- The command parsing has been restructured a bit to pave the way for
JSON-RPC 2.0 batching support
- When invalid HTTP Basic Access Authentication details are provided to
the websocket endpoint, return a WWW-Authenticate header just like the
non-websocket endpoint
- When a connection to the websocket endpoint fails to properly upgrade,
return a 400 Bad Request HTTP error status code
This commit reorders the RPC server code to more closely match the
ordering used by the rest of the code base such that functions are
typically defined before they are used and near their call site.
The ScriptVerifyMinimalData enforces that all push operations use the
minimal data push required. This is part of BIP0062.
This commit mimics Bitcoin Core commit
698c6abb25c1fbbc7fa4ba46b60e9f17d97332ef
This changes the behavior of the rescan RPC to automatically set the
client up for transaction notifications for transactions paying to any
rescanned address and spending outputs in the final rescan UTXO set
after a rescanned is performed through the best block in the chain.
Remove ScriptCanonicalSignatures and use the new
ScriptVerifyDERSignatures flag. The ScriptVerifyDERSignatures
flag accomplishes the same functionality.
`searchrawtransactions` can return multiple transactions. Previously the documentation erroneously had return examples showing only a single tx for each case.
* The documentation previously presented the output for the `non-verbose` case as a single json string, instead of an array of json strings.
* Similarly, for the `verbose` mode, the documentation detailed the output as a single json object, instead of an array of json objects.
This commit contains the entire btcjson repository along with several
changes needed to move all of the files into the btcjson directory in
order to prepare it for merging. This does NOT update btcd or any of the
other packages to use the new location as that will be done separately.
- All import paths in the old btcjson test files have been changed to the
new location
- The coveralls badge has been removed since it unfortunately doesn't
support coverage of sub-packages
This is ongoing work toward #214.
- Also update to use the new container-based builds
- Add coveralls.io support
- Update build badge to use SVG
- Add badges for coverage, license, and godoc
- For now, set it to change to he v2 directory
- This will need to be changed once the v1 code is replaced
This commit implements a reimagining of the way the btcjson package
functions based upon how the project has evolved and lessons learned while
using it since it was first written. It therefore contains significant
changes to the API. For now, it has been implemented in a v2 subdirectory
to prevent breaking existing callers, but the ultimate goal is to update
all callers to use the new version and then to replace the old API with
the new one.
This also removes the need for the btcws completely since those commands
have been rolled in.
The following is an overview of the changes and some reasoning behind why
they were made:
- The infrastructure has been completely changed to be reflection based instead
of requiring thousands and thousands of lines of manual, and therefore error
prone, marshal/unmarshal code
- This makes it much easier to add new commands without making marshalling
mistakes since it is simply a struct definition and a call to register that
new struct (plus a trivial New<foo>Cmd function and tests, of course)
- It also makes it much easier to gain a lot of information from simply
looking at the struct definition which was previously not possible
such as the order of the parameters, which parameters are required
versus optional, and what the default values for optional parameters
are
- Each command now has usage flags associated with them that can be
queried which are intended to allow classification of the commands such
as for chain server and wallet server and websocket-only
- The help infrastructure has been completely redone to provide automatic
generation with caller provided description map and result types. This
is in contrast to the previous method of providing the help directly
which meant it would end up in the binary of anything that imported the
package
- Many of the structs have been renamed to use the terminology from the
JSON-RPC
specification:
- RawCmd/Message is now only a single struct named Request to reflect the fact
it is a JSON-RPC request
- Error is now called RPCError to reflect the fact it is specifically an RPC
error as opposed to many of the other errors that are possible
- All RPC error codes except the standard JSON-RPC 2.0 errors have been
converted from full structs to only codes since an audit of the codebase
has shown that the messages are overridden the vast majority of the time
with specifics (as they should be) and removing them also avoids the
temptation to return non-specific, and therefore not as helpful, error
messages
- There is now an Error which provides a type assertable error with
error codes so callers can better ascertain failure reasons
programatically
- The ID is no longer a part of the command and is instead specified at the time
the command is marshalled into a JSON-RPC request. This aligns better with
the way JSON-RPC functions since it is the caller who manages the ID that is
sent with any given _request_, not the package
- All <Foo>Cmd structs now treat non-pointers as required fields and pointers as
optional fields
- All New<Foo>Cmd functions now accept the exact number of parameters, with
pointers to the appropriate type for optional parameters
- This is preferrable to the old vararg syntax since it means the code will
fail to compile if the optional arguments are changed now which helps
prevent errors creep in over time from missed modifications to optional args
- All of the connection related code has been completely eliminated since this
package is not intended to used a client, rather it is intended to provide
the infrastructure needed to marshal/unmarshal Bitcoin-specific JSON-RPC
requests and replies from static types
- The btcrpcclient package provides a robust client with connection management
and higher-level types that in turn uses the primitives provided by this
package
- Even if the caller does not wish to use btcrpcclient for some reason, they
should still be responsible for connection management since they might want
to use any number of connection features which the package would not
necessarily support
- Synced a few of the commands that have added new optional fields that
have since been added to Bitcoin Core
- Includes all of the commands and notifications that were previously in
btcws
- Now provides 100% test coverage with parallel tests
- The code is completely golint and go vet clean
This has the side effect of addressing nearly everything in, and therefore
closes#26.
Also fixes#18 and closes#19.
* Decouple logging and handling of distinct error messages for cases of
failed tx look up.
* This also fixes a bug wherein the `addrindexer` failed to lookup a tx
(since it didn't exist), but continued since the returned error was
nil.
This commit adds two new verification flags to txscript named
ScriptVerifyStrictEncoding and ScriptVerifyDerSignatures.
The ScriptVerifyStrictEncoding flag enforces signature scripts
and public keys to follow the strict encoding requirements.
The ScriptVerifyDerSignatures flag enforces signature scripts
to follow the strict encoding requirements.
These flags mimic Bitcoin Core's SCRIPT_VERIFY_STRICTENC and
SCRIPT_VERIFY_DERSIG flags and brings the Bitcoin Core test scripts up
to date.
This commit modifies the pre-computed table used to optimize the secp256k1
scalar multiplication to a string instead of a byte slice. This change
makes the compile more efficient since the Go compiler internally
represents bytes slices inefficiently.
This reduces the memory needed to compile btcec to 3MB versus the previous
40MB before this change.
In addition, it modifies the code which loads the pre-computed table to
deserialize directly into the table instead of into locals that are then
copied.
Fixes#297.
This commit corrects a case in the OP_CHECKMULTISIG handling where it was
possible to improperly validate a transaction that had a combination of
valid and malformed signatures.
It also adds a new test to ensure this case is properly handled and nukes
a superfluous comment.
Fixes#293.
This commit adds a new example to the txscript package that demonstrates
creating a new transaction which redeems funds and signing the referenced
transaction output the SignTxOutput function.
This commit contains various modifications for code and comment
consistency in the btcec package:
- Call out references at the top and reference them by their identifier in
the other comments
- Remove a TODO that no longer applies
- Add comments to the fields in the KoblitzCurve struct and reorder them
slightly
- Make comments wrap to 80
- Cleanup code that was far exceeding col 80 (only function declarations
typically do this)
- Extend block comments to use as much of the 80 cols as available
- Add a bit more explanation in a couple of places
- Update copyright year on secp256k1.go
- Fix a couple of typos in the comments
The benchmarks are still only compiled when running 'go test' so this has
no effect on regular usage. This is being done because benchmarks often
need access to internal state.
Normal tests are kept in a separate package since they also serve to
exercise the public API, and by intentionally making it more difficult to
reach into the internals, it helps ensure the public API is sanely usable.
Since the benchmarks can now access the internals directly, this commit
also removes the functions which exposed the internals to the test package
from internal_test.go which were only used by the benchmarks.
Also, it removes a duplicate benchmark.
This commit converts the initialization of the constants to use a function
which panics on error instead of just ignoring the error. This is
acceptable since they are hard-coded constants and should never fail.
This commit adds code which generates the linearly independent vectors
used by the secp256k1 endomorphism code. These value are hard-coded into
the curve already, but having the code used to generate them is handy
should any future curves be added which can also make use of the same
class of endomorphism.
This commit contains the entire btcnet repository along with several
changes needed to move all of the files into the chaincfg directory in
order to prepare it for merging. This does NOT update btcd or any of the
other packages to use the new location as that will be done separately.
- All import paths in the old btcnet test files have been changed to the
new location
- All references to btcnet as the package name have been changed to
chaincfg
- The coveralls badge has been removed since it unfortunately doesn't
support coverage of sub-packages
This is ongoing work toward #214.
* Address index is built up concurrently with the `--addrindex` flag.
* Entire index can be deleted with `--dropaddrindex`.
* New RPC call: `searchrawtransaction`
* Returns all transacitons related to a particular address
* Includes mempool transactions
* Requires `--addrindex` to be activated and fully caught up.
* New `blockLogger` struct has been added to factor our common logging
code
* Wiki and docs updated with new features.
Use Non-Adjacent Form (NAF) of large numbers to reduce ScalarMult computation times.
Preliminary results indicate around a 8-9% speed improvement according to BenchmarkScalarMult.
The algorithm used is 3.77 from Guide to Elliptical Curve Crytography by Hankerson, et al.
This closes#3
This implements a speedup to ScalarMult using the endomorphism available to secp256k1.
Note the constants lambda, beta, a1, b1, a2 and b2 are from here:
https://bitcointalk.org/index.php?topic=3238.0
Preliminary tests indicate a speedup of between 17%-20% (BenchScalarMult).
More speedup can probably be achieved once splitK uses something more like what fieldVal uses. Unfortunately, the prime for this math is the order of G (N), not P.
Note the NAF optimization was specifically not done as that's the purview of another issue.
Changed both ScalarMult and ScalarBaseMult to take advantage of curve.N to reduce k.
This results in a 80% speedup to large values of k for ScalarBaseMult.
Note the new test BenchmarkScalarBaseMultLarge is how that speedup number can
be checked.
This closes#1
This commit reworks the way that the pre-computed table which is used to
accelerate scalar base multiple is generated and loaded to make use of the
go generate infrastructure and greatly reduce the memory needed to compile
as well as speed up the compile.
Previously, the table was being generated using the in-memory
representation directly written into the file. Since the table has a very
large number of entries, the Go compiler was taking up to nearly 1GB to
compile. It also took a comparatively long period of time to compile.
Instead, this commit modifies the generated table to be a serialized,
compressed, and base64-encoded byte slice. At init time, this process is
reversed to create the in-memory representation. This approach provides
fast compile times with much lower memory needed to compile (16MB versus
1GB). In addition, the init time cost is extremely low, especially as
compared to computing the entire table.
Finally, the automatic generation wasn't really automatic. It is now
fully automatic with 'go generate'.
This commit contains the entire btcwire repository along with several
changes needed to move all of the files into the wire directory in
order to prepare it for merging. This does NOT update btcd or any of the
other packages to use the new location as that will be done separately.
- All import paths in the old btcwire test files have been changed to the
new location
- All references to btcwire as the package name have been chagned to
wire
- The coveralls badge has been removed since it unfortunately doesn't
support coverage of sub-packages
This is ongoing work toward #214.
This commit causes TravisCI to run several tools on each pull request and
commit to help ensure the code quality remains high. This includes gofmt,
goimports, golint, go vet, the race detector, and coverage stats.
Also, it instructs TravisCI to use nicer container-based builds.
This change converts the leveldb database's ExistsSha() and
ExistsTxSha to use the goleveldb API. Has() only returns if
the key exists and does not need to read the entire value into
memory resulting in less disk i/o and much less GC.
This commit contains the entire btcchain repository along with several
changes needed to move all of the files into the blockchain directory in
order to prepare it for merging. This does NOT update btcd or any of the
other packages to use the new location as that will be done separately.
- All import paths in the old btcchain test files have been changed to
the new location
- All references to btcchain as the package name have been changed to
blockchain
* When an inv is to be sent to the server for relaying, the sender
already has access to the underlying data. So
instead of requiring the relay to look up the data by
hash, the data is now coupled in the request message.
This commit contains the entire btcscript repository along with several
changes needed to move all of the files into the txscript directory in
order to prepare it for merging. This does NOT update btcd or any of the
other packages to use the new location as that will be done separately.
- All import paths in the old btcscript test files have been changed to the
new location
- All references to btcscript as the package name have been chagned to
txscript
This is ongoing work toward #214.
The ScriptVerifySigPushOnly flag enforces signature scripts to
only contain pushed data. This is rule 2 of BIP0062.
Mimics Bitcoin Core commit d752ba86c1872f64a4641cf77008826d32bde65f
This commit makes use of the new ScriptDiscourageUpgradableNops flag to
reject execution of NOP1 through NOP10 for transactions that are
considered standard.
This mirrors the behavior added to Bitcoin Core via pull request 5000.
This commit modifies various regression tests to make them more consistent
with other tests throughout the code base.
Also, it allows of all the tests to run in parallel.
NOP1 through NOP10 are reserved for future soft-fork upgrades. When
such an upgrade occurs, the NOP argument will then require verification.
Rejecting transactions that contain these NOPs into the mempool will
discourage those transactions from being mined elsewhere and ensure
btcd will never mine such transactions. This prevents now
invalid scripts (according to the majority of hashing power) even if the
client has not yet upgraded.
Non-executed upgradable NOPs are still allowed as they will still be
valid post-upgrade.
Mimics Bitcoin Core commit 03914234b3c9c35d66b51d580fe727a0707394ca
BIP0062 defines specific rules and canonical encodings for data pushes.
The existing script builder code already conformed to all but one of the
canonical data push rules that was added after it was originally
implemented (adding a single byte of 0x81 must be converted to
OP_1NEGATE). This commit implements that case and expands the existing
tests to explicitly cover all cases mentioned in BIP0062.
In addition, as a part of this change, the AddData function has been
modified so that any attempt to push more than the maximum script element
size bytes (520) in one push or any pushes the would cause the script to
exceed the maximum script bytes allowed by the script engine (10000) will
result in the final call to the Script function to only return the script
up to the point of the first error along with the error. This change
should have little effect on existing callers since they are almost
positively not creating scripts which violate these rules as they could
never be executed, however it does mean they need to check the new error
return.
Since the regression tests intentionally need to be able to exceed that
limit, a new function named AddFullData has been added which does not
enforce the limits, but still provides canonical encoding of the pushed
data.
Note that this commit does not affect consensus rules nor modify the
script engine.
Also, the tests have been marked so they can run in parallel.
This utility is a relic from initial development before it was possible to
request blocks and transactions via RPC. The correct way to do this now
is by using RPC since that works while btcd is operating unlike the
utility which requires an exclusive lock on the database.
Currently, the reference client bans peers that send alerts not signed
with its key. We could verify against their key, but since the
reference client developers are currently unwilling to support other
implementations' alert messages, we will not relay theirs.
This commit contains the entire btcdb repository along with several
changes needed to move all of the files into the database directory in
order to prepare it for merging. This does NOT update btcd or any of the
other packages to use the new location as that will be done separately.
- All import paths in the old btcdb test files have been changed to the
new location
- All references to btcdb as the package name have been chagned to
database
- The coveralls badge has been removed since it unfortunately doesn't
support coverage of sub-packages
This is ongoing work toward #214.
This was previously hard-coded to zero instead of using the offset
provided by the median time source which takes time samples from the other
connected nodes.
Only two operations are performed with this data structure: adding to
the back and removing from the front. Because middle inserts and
deletions are never needed, a linked list results in overall worse
performance due to an extra allocation for each element's node, worse
cache locality, and the runtime cost of boxing/unboxing each item
during accesses.
On top of the performance gains, a slice is more type safe as it is a
true generic data structure making it is impossible to insert or
access an element with the wrong type.
* Previously both searchrawtransaction and getrawtransaction would omit
the 'Confirmations' field by default if the transaction was found in
the mempool.
* By removing `omitempty`, a mempool transaction will now have
'Confirmations:0'
Also update to use the new container-based builds
For now, golint has been disabled since it fails due to wanting
HTTPPostMode. This change will be made as a part of the upgrade to use
btcjson2 since it involves an API change.
The addition of the pre-computed values for the ScalarBaseMult
optimizations added a benign race condition since a pointer to each
pre-computed Jacobian point was being used in the call to addJacobian
instead of a local stack copy.
This resulted in the code which normalizes the field values to check for
further optimization conditions such as equal Z values to race against the
IsZero checks when multiple goroutines were performing EC operations since
they were all operating on the same point in memory.
In practice this was benign since the value was being replaced with the
same thing and thus it was the same before and after the race, but it's
never a good idea to have races.
The writeElement function provides faster serialization for primitives.
This commit modifies all instances that call writeElement with a
pointer to a primitive or a byte slice to instead use the primitive /
writeVarBytes function so the faster serialization paths are used.
The benchmarks are still only compiled when running 'go test' so this has
no effect on regular usage. This is being done because benchmarks often
need access to internal state.
Normal tests are kept in a separate package since they also serve to
exercise the public API, and by intentionally making it more difficult to
reach into the internals, it helps ensure the public API is sanely usable.
Since the benchmarks can now access the internals directly, this commit
also removes the functions which exposed the internals to the test package
from internal_test.go which were only used by the benchmarks.
The comment for the RPC listen section in the sample-btcd.conf incorrectly
claimed that the default for the RPC server listener is to listen on all
interfaces by default. In reality, it only listens on localhost for IPv4
and IPv6 by default.
Closes#208.
The mempool's MaybeAcceptTransaction methods have also been modified
to return a slice of transaction hashes referenced by the transaction
inputs which are unknown (totally spent or never seen). While this is
currently used to include the first hash in a ProcessTransaction error
message if inserting orphans is not allowed, it may also be used in
the future to request orphan transactions from peers.
This commit adds a test for checking known good block scripts in a block
are valid. As a part of this, it adds the infastructure needed to load a
saved transaction store from a file which contains all of the input
transactions needed.
It also contains some changes from running goimports as well as some other
cleanup.
This commit modifies the getblocktemplate RPC to correctly handle the
curtime field.
Currently, the RPC server will refuse to serve a block template if the
current time is before the minimum allowed time for a block even though
the real generated block template already accounted for it.
This consists of three changes:
- Remove the unnecessary and incorrect check in the RPC invocation for the
current time against the min required time since it is already handled
properly by the block template generation.
- Expose the network time source to the RPC work state and use the
adjusted time for checking against the maximum allowed time instead of
the current time.
- Set the returned RPC result curtime field to the time of the generated
block header.
Fixes#209.
This commit modifies the creation of RPC results to ignore errors in
script disassembly since they already contain the error string inline and
the RPC results must still be generated regardless.
This was already done in the decodescript RPC, however the same thing was
not being done in the higher level verbose transaction results. This
applies to both the createrawtransaction and decoderawtransaction RPCs.
Also, since this was the only thing that could error within those
functions, the error returns and caller checking of now non-existent
errors have been removed.
Fixes#210.
The hash is not displayed big endian, but rather as a byte-reversed
hash as a hexidecimal string, so document it as such.
While here, speed up the function. Some benchmarks:
BenchmarkShaHashString 2000000 996 ns/op
BenchmarkShaHashStringOld 100000 22701 ns/op
This commit introduces a new flag, --notls, which can be used to disable
TLS for the RPC server. However, the flag can only be used when the RPC
server is bound to localhost interfaces. This is intended to prevent the
situation where someone decides they want to expose the RPC server to the
web for remote management/access, but forgot they have TLS disabled.
This commit corrects the error check from the createTxRawResult call in
handleGetRawTransaction. It was previously given a different name which
resulted in the wrong error being checked.
Fixes#196.
This commit improves a couple of issues surrounding the creation of the
btcd home directory.
First, the code was previously attempting to log any errors that occurred
while creating the directory using the logging system which is not
initialized at that point. Thus, nothing was displayed to the user.
Second, if any component of btcd home directory path already exists, but
is not a directory, such as in the case of symlinks, the error returned
from the os.MkDirAll call indicates the directory can't be created. While
this is true, it's not always the most helpful error to display to the
user. So, this commit adds logic to detect when the failure case is due
to an existing symlink and displays a nicer error message suggesting the
user check if the destination of the link is mounted.
Fixes#193.
This prevents a downgrade attack to the vulnerable SSLv3. While here,
go ahead and require at least TLS 1.2 since TLS 1.0 and 1.1 have their
own set of issues and it's only a matter of time before those would
need to be completely avoided as well.
ok @davecgh
Our policy is to only ensure the code compiles for the latest Go release
minus one version. Since Go 1.4 has now been released, this commit
updates TravisCI to remove Go 1.2 from the build matrix and add Go 1.3.
Also, go vet changed locations in between Go 1.3 and Go 1.4, so a check
has been added to download the appropriate version based on which version
of Go is running the integration tests.
Finally, the previous removed Go tip entry in the build matrix has been
reintroduced since it will now work again.
Now that Go 1.4 has been released, this commit reverts the recent changes
which dealt with allowing TravisCI to work with go tip in between the Go
1.3 and Go 1.4 release cycle and updates the .travis.yml to invoke test
coverage tool from the new path in Go 1.4.
As pointed out in #189, according to the Go documentation, a ticker must
be stopped to release associated resources. This commit adds a defer call
to stop two tickers there were previously not being stopped as well as
changes two others that were being stopped over to use defer so it's more
consistent.
The other ticker in ScheduleShutdown is replaced and already calls Stop
before replacing it, so it has not been modified.
Closes#189.
ok @jrick
This commit addresses a few nitpicks in the recent getrawmppol update
which populates the starting and current priority fields.
In particular:
- Move the new calcInputValueAge function before the function which
invokes it so it is consistent with the rest of the mempool code
- Double space after periods for consistency
- Correct the comments for calcInputValueAge to indiciate that inputs
which are in the the memory pool count as zero toward the value age
rather than the incorrect claim that that the overal input value age is
zero when one of them does
- Rename endingPriority to currentPriority to match the RPC field and its
actual function
- Make the comment about using zero when input transactions can't be found
for some reason more accurate since there can be (and frequently is)
more than one input transaction
This commit modifies the .travis.yml to invoke a new script which has also
been added that gets the test coverage tool from the new path needed by go
tip.
This commit modifies the RandomUint64 function so that rather than
returning an io.ErrShortBuffer when the system does not have enough
entropy available, it now blocks until it does have enough. This means
that RandomUint64 will now always eventually succeed unless the entropy
source is closed (which only really ever happens when the operating system
is shutting down).
The tests have also been updated for the change in semantics to maintain
100% coverage.
Closes#23.
Since the result might not be set when the response does not contain one,
such as in the case when the command is not implemented by the server,
don't attempt to modify it until ensuring it exists.
This commit extends the same logic in the previous commit to the
comparison of offsets returned from the median time source in the tests.
In particular it modifies the tests to allow for the offsets to differ by
a second due to a boundary condition to prevent false positives.
The new median time tests perform a comparsion of the adjusted time
returned from the median time source to the expected value. However,
since time is always moving, it is possible the current time between the
call to the adjusted time function and the current time taken in the tests
for comparison just after the call differ by a second due to a boundary
condition. So, this commit modifies the tests to allow for this
condition.
This commit changes the internal testing mechanism from returning the
unexported timeSorter type directly to insted returning sort.Interface.
This has been done because the latest version of golint complains about
return unexported types.
This commit provides a new interface, MedianTimeSource, along with a
concrete implementation which allows improved accuracy of time by making
use of the median network time as calculated from multiple time samples.
The time samples are to be provided by callers and are intended to come
from remote clients.
The calculations performed in this implementation exactly mirror those in
Bitcoin Core because time calculations are part of the consensus rules and
hence need to match exactly.
This commit uses the new MedianTimeSource API in btcchain to create a
median time source which is stored in the server and is fed time samples
from all remote nodes that are connected. It also modifies all call sites
which now require the the time source to pass it in.
Code uses a windowing/precomputing strategy to minimize ECC math.
Every 8-bit window of the 256 bits that compose a possible scalar multiple has a complete map that's pre-computed.
The precomputed data is in secp256k1.go and the generator for that file is in gensecp256k1.go
Also fixed a spelling error in a benchmark test.
Results so far seem to indicate the time taken is about 35% of what it was before.
Closes#2
This change removes transactions from a newly connected block
from the orphan pool if they exist. Additionally, any orphan
transactions that are no longer orphan transactions are moved
to the mempool and inv'd to the currently connected peers.
This commit adds a couple of options which were not details in the sample
config file. It also fixes a couple of typos and makes the example default
maxpeers setting in the config file match the actual default used in btcd.
The go vet command complains about untagged struct initializers when
defining a ShaHash directly. This seems to be a limitation where go vet
does not exclude the warning for types which are a constant size byte
array like it does for normal constant size byte array definition.
This commit simply modifies the code to use a constant definition cast to
a ShaHash to overcome the limitation of go vet.
Add the word 'string' to the error string so if this is printed, it
will be clearer that it was a hash string decode that failed (as
opposed to other errors creating a ShaHash from a byte slice) and
change 'chars' to 'bytes', since the string length is measured in
bytes, not UTF-8 code points.
ok @davecgh
The go vet command complains about untagged struct initializers when
defining a ShaHash directly. This seems to be a limitation where go vet
does not exclude the warning for types which are a constant size byte array
like it does for normal constant size byte array definition.
This commit simply modifies the tests to use a constant definition cast to a
ShaHash to overcome the limitation of go vet.
This commit explicitly ignores errors that were already intentionally
ignored in btcctl by using an _ to make the errcheck utility happy. It is
also nice to make it explicit that the error is being ignored
intentionally rather than leave questions of whether it was accidentally
forgotten.
This commit resolves a minor issue where an error in the config file would
prevent the help from being shown until the config file error was
resolved.
This results in the following behavior:
- With a malformed header:
$ ./btcd
Error parsing config file: ~/btcd/btcd.conf:14: malformed section header
Use btcd -h to show usage
- With an invalid option:
$ ./btcd
Error parsing config file: unknown option: bogus
Use btcd -h to show usage
- Invoking help with an error in the config file:
$ ./btcd -h
Usage:
...
- Invoking with an invalid command line option:
$ ./btcd --bogus=bogus
unknown flag `bogus'
Use btcd -h to show usage
ok @jrick
This commit builds on the recent limiter updates. The recent changes
introduced a new function named calcTxFee which could easily be confused
with calculating the fees of the transaction rather than its intended
purpose of calculating the minimum transaciton relay fee.
Rather than taking that approach, this commit instead renames the existing
function to calcMinRequiredTxRelayFee and uses a consistent variable name
for the serialized size of a transaction. It then moves the check for
whether or not the check should be applied based on the serialized size of
the transcation and block priority to the call site.
This approach also has the benefit of avoiding two calls to the
calculation function since it's a local at the call site.
ok @jrick, @dajohi
Before, btcd was rate limiting all transactions that had a minimum
fee of zero. Now, btcd only rate limits transactions that contain
a fee less than the calculated fee based on size.
Closes#163
This commit adds an example test file so it integrates nicely with Go's
example tooling.
This allows the example output to be tested as a part of running the
normal Go tests to help ensure it doesn't get out of date with the code.
It is also nice to have the examples in one place rather than repeating it
in doc.go and README.md.
Links and information about the examples have been included in README.md in
place of the examples and doc.go has been updated accordingly.
This ensures we backoff when reconnecting to peers for which we don't
understand the replies, just like we do for peers we fail to connect to.
Closes#103
The notification queue used for websocket client notifications had a
bug which caused the next queued item in some situations to not be
sent, but instead send a previously sent item. This change fixes this
by setting the 'next' variable to the next item which must be
dequeued, if the queue length is non-zero.
This bug did not always manifest itself as if the receiving goroutine
was ready, a queued item could be sent directly to it rather than
waiting in the queue to be sent at a later time.
This change modifies the behavior of the gettxout RPC to match the
behavior of the reference client. If a transaction output is spent by
a mined transaction, the handler will now return nil (JSON null).
While here, avoid indexing some slices multiple times, by creating a
local variable instead.
The sync.atomic requires alignment of variables used atomically on ARM.
This commit moves the connected and disconnect variables in the peer
struct up so they are aligned.
Fixes#157.
This commit prevents a race that could happen on shutdown if a sigint was
received between adding the first the interrupt handler and and future
handlers. This code is loosely based on initial pull request #154 which
was not accepted due to introducing a race of its own.
Thanks to @tuxcanfly for the intial pull request and finding the issue.
This commit introduces an HDPrivateKeyID and HDPublicKeyID field to the
params struct which are used by hierarchical deterministic extended keys
as defined by BIP0032.
In addition, a new function named HDPrivateKeyToPublicKeyID has been added
to allow the caller to get the associated public key ID given a private
key ID.
The tests have also been updated to maintain 100% test coverage.
ok @jrick
Rather than showing the usage when an error is encounted during options
parsing, show a message that describes how to invoke help instead. This
is useful because the help is long enough now that the error is often
overlooked since it scrolls out of view.
ok @jrick
This commit implements reject handling as defined by BIP0061 and bumps the
maximum supported protocol version to 70002 accordingly.
As a part of supporting this a new error type named RuleError has been
introduced which encapsulates and underlying error which could be one of
the existing TxRuleError or btcchain.RuleError types.
This allows a single high level type assertion to be used to determine if
the block or transaction was rejected due to a rule error or due to an
unexpected error. Meanwhile, an appropriate reject error can be created
from the error by pulling the underlying error out and using it.
Also, a check for minimum protocol version of 209 has been added.
Closes#133.
Although it is possible to get the command name for each msg type by
creating an instances of the type and calling the Command method against
it, it's slightly more efficient to simply allows callers to have direct
access to the exported constants.
This is currently really useful for the reject message since callers need
to be able to examine the command type to determine whether or not the
hash field needs to be included.
This commit splits the two rule validation errors related to the timestamp
being too old into two distince errors rather than grouping them under the
same one. Thus there is now a new ErrCheckpointTimeTooNew in addition to
the existing ErrTimeTooNew error.
This allows the caller to detect the when a block is rejected due to a
time-related checkpoint failure whereas before the combined error did not.
The done and wait channels used to throttle outgoing data are being used
as semaphores. As mentioned in the previous commit, it's more efficient
to use a 0-byte type and allow compiler optimizations for the specific use
case.
This commit implements a portion of the mutations section of BIP0023.
In particular, it adds the mutable, mintime, maxtime, and noncerange keys
to the returned block template along with indicating support for the time,
transactions/add, prevblock, and coinbase/append mutations. Also, the
addition of the mintime and maxtime fields imply support for the
time/decrement and time/increment mutations. Further, if the caller
indicates the coinbasevalue capability, the coinbasetxn field will be
omitted thereby implying support for the coinbase and generation
mutations.
Closes#124.
This commit implements the long polling portion of the getblocktemplate
RPC as defined by BIP0022. Per the specification, each block template is
returned with a longpollid which can be used in a subsequent
getblocktemplate request to keep the connection open until the server
determines the block template associated with the longpollid should be
replaced with a new one.
This is work towards #124.
This commit implements the non-optional and template tweaking support for
the getblocktemplate RPC as defined by BIP0022. This implementation does
not yet include long polling support.
This is work towards #124.
This commit adds the Move, MoveAsync, MoveMinConf, MoveMinConfAsync,
MoveComment, and MoveCommentAsync functions for issuing the "move" RPC
with various optional parameters.
Closes#16.
This commit implements full support for filtering based on the filterload,
filteradd, filterclear, and merkleblock messages introduced by BIP0037.
This allows btcd to work seamlessly with SPV wallets such as BitcoinJ.
Original code by @dajohi. Cleanup, bug fixes, and polish by @davecgh.
This commit has no effect on the effective functionality since Go
automatically deferences pointers, but there is no reason to have a
double indirection when returning the reply for getmininginfo.
This commit adds an example test file so it integrates nicely with Go's
example tooling.
This allows the example output to be tested as a part of running the
normal Go tests to help ensure it doesn't get out of date with the code.
It is also nice to have the example in one place rather than repeating it
in doc.go and README.md.
Links and information about the example have been included in README.md in
place of the example.
It seems the godoc example extraction does not recognize the same syntax
used for functions with receivers when working with interfaces.
So, instead, categorize the example on the interface.
This commit moves the example to a test file so it integrates nicely with
Go's example tooling.
This allows the example output to be tested as a part of running the
normal Go tests to help ensure it doesn't get out of date with the code.
It is also nice to have the example in one place rather than repeating it
in doc.go and README.md.
Links and information about the example have been included in README.md in
place of the example.
This commit moves the ProcessBlock example to a test file so it integrates
nicely with Go's example tooling.
This allows the example output to be tested as a part of running the
normal Go tests to help ensure it doesn't get out of date with the code.
It is also nice to have the example in one place rather than repeating it
in doc.go and README.md.
Links and information about the example have been incldued in doc.go and
README.md in place of the example.
Previously the exported Disconnect and Shutdown functions called each other
and therefore needed to release and reacquire the locks which could
potentionally allow a request to sneak in between the lock and unlock.
This commit changes the exported Shutdown function so that everything is
done under the request lock to prevent this possibility. In order to
support this, the shared code between the Disconnect and Shutdown
functions has been refactored into unexported functions and the locking
has been altered accordingly.
Also, replace the sendMessage function with a select over the send and
disconnect channels to prevent any issues with queued up messages.
joint debugging effort between myself and @jrick
Previously, requests could still be sent to a shutdown client and
added to the client's internal data structures, without ever
responding to the future with an error for a shutdown client (causing
hangs when blocking on the future receive). This change fixes this by
performing a non-blocking read of the client's shutdown channel before
adding a request, and responding with the shutdown error if the client
has begun or completed its shutdown.
ok @davecgh
This commit adds error returns to all of the Db interface methods except
for FetchTxByShaList and FetchUnSpentTxByShaList since they expose the
errors on each individual transaction.
It also updates all tests and code for both the leveldb and memdb drivers
for the changes.
Closes#5.
ok @drahn
Since the underlying database driver can now return an error when looking
up if blocks and transactions exist, the HaveBlock function now includes
an error return to allow any underlying errors to be exposed.
This ability for multiple database backends has been available for a long,
long time. Looks like this was simply not updated when the interface was
added.
These changes are a joint effort between myself and @dajohi.
- Separate IP address range/network code into its own file
- Group all of the RFC range declarations together
- Introduces a new unexported function to simplify the range declarations
- Add comments for all exported functions
- Use consistent variable casing in refactored code
- Add initial doc.go package overview
- Bump serialize interval to 10 minutes
- Correct GroupKey to perform as intended
- Make AddLocalAddress return error instead of just a debug message
- Add tests for AddLocalAddress
- Add tests for GroupKey
- Add tests for GetBestLocalAddress
- Use time.Time to improve readability
- Make address manager code golint clean
- Misc cleanup
- Add test coverage reporting
This commit does just enough to move the address manager into its own
package. Since it was not originally written as a package, it will
require a bit of refactoring and cleanup to turn it into a robust
package with a friendly API.
By putting each DNS seed in its own go routine, btcd can start connecting
to nodes as they are found instead of waiting for all seeds to respond. This
significantly speeds up startup time.
Additionally, logging was added to show how many addresses were fetched from
each seed.
BitcoinJ, and possibly other wallets, don't follow the spec of sending an
inventory message and allowing the remote peer to decide whether or not
they want to request the transaction via a getdata message. Unfortuantely
the reference implementation permits unrequested data, so it has allowed
wallets that don't follow the spec to proliferate.
While this is not ideal, this commit removes the functionality which
disconnects peers for sending unsolicited transactions to provide
interoperability.
This commit corrects an issue where the data requested by getdata was not
being properly throttled which could lead to higher than desired memory
usage on large requests.
This commit modifies the error handling for websocket connections to fall
back to returning the status text returned from the server if the
handshake fails and it's not due to an authentication or invalid endpoint.
This change set equips the RPC client with handling of non successful
HTTP responses. An HTTP response is considered non successful when its
status code is not in the range 200..299
This commits removes a number of golint warnings. There is a class of
warnings which I can't fix due to unsufficient knowledge of the domain
at this point. These are listed here:
addrmanager.go:907:1: comment on exported method AddrManager.Attempt
should be of the form "Attempt ..."
addrmanager.go:1048:1: exported function RFC1918 should have comment or
be unexported
addrmanager.go:1058:1: exported function RFC3849 should have comment or
be unexported
addrmanager.go:1065:1: exported function RFC3927 should have comment or
be unexported
addrmanager.go:1073:1: exported function RFC3964 should have comment or
be unexported
addrmanager.go:1081:1: exported function RFC4193 should have comment or
be unexported
addrmanager.go:1089:1: exported function RFC4380 should have comment or
be unexported
addrmanager.go:1097:1: exported function RFC4843 should have comment or
be unexported
addrmanager.go:1105:1: exported function RFC4862 should have comment or
be unexported
addrmanager.go:1113:1: exported function RFC6052 should have comment or
be unexported
addrmanager.go:1121:1: exported function RFC6145 should have comment or
be unexported
addrmanager.go:1128:1: exported function Tor should have comment or be
unexported
addrmanager.go:1143:1: exported function Local should have comment or be
unexported
addrmanager.go:1228:2: exported const InterfacePrio should have comment
(or a comment on this block) or be unexported
discovery.go:26:2: exported var ErrTorInvalidAddressResponse should have
comment or be unexported
limits/limits_unix.go:19:1: exported function SetLimits should have
comment or be unexported
limits/limits_windows.go:7:1: exported function SetLimits should have
comment or be unexported
util/dropafter/dropafter.go:22:6: exported type ShaHash should have
comment or be unexported
util/dropafter/dropafter.go:38:2: exported const ArgSha should have
comment (or a comment on this block) or be unexported
util/dropafter/dropafter.go:128:5: exported var ErrBadShaPrefix should
have comment or be unexported
util/dropafter/dropafter.go:129:5: exported var ErrBadShaLen should have
comment or be unexported
util/dropafter/dropafter.go:130:5: exported var ErrBadShaChar should
have comment or be unexported
util/showblock/showblock.go:24:6: exported type ShaHash should have
comment or be unexported
util/showblock/showblock.go:46:2: exported const ArgSha should have
comment (or a comment on this block) or be unexported
util/showblock/showblock.go:163:1: exported function DumpBlock should
have comment or be unexported
util/showblock/showblock.go:211:5: exported var ErrBadShaPrefix should
have comment or be unexported
util/showblock/showblock.go:212:5: exported var ErrBadShaLen should have
comment or be unexported
util/showblock/showblock.go:213:5: exported var ErrBadShaChar should
have comment or be unexported
The hex package requires an even number of characters in hex encoded
strings. Some of the handlers already prepended a zero if necessary to
make this condition true, however other did not. This commit extends this
functionality to all handlers and also makes the error return consistent.
ok @jrick
This commit creates and exports a new constant, MaxTimeOffsetSeconds,
which is the maximum number of seconds a block timestamp is allowed to be
ahead of the current time.
Previously this value was hard coded into the consensus rule path, however
it is useful better to have it defined as a constant and exported so other
callers can access it.
No consensus rules have been changed with this commit.
This commit adds a new behavior flag, BFDryRun which allows the caller
to indicate all checks should be performed against the block as normal
except it will not modify any state. This is useful to test that a block
is valid without actually modifying the current chain or memory state.
This commit also adds a few additional checks which were elided before
since they are implicitly handled by btcwire. However, with the ability
to propose blocks which didn't necessarily come through the btcwire path,
these checks need to be enforced in the chain code as well.
As a part of adding the checks, three new error codes named
ErrBlockTooBig, ErrTooManyTransactions, and ErrTxTooBig have been
introduced.
Closes#5.
This commit adds a new function named SerializeSize to the public API for
MsgBlock which can be used to determine how many bytes the serialized data would
take without having to actually serialize it. In addition, it makes the
exported BlockVersion an untyped constant as well as changes the block and
tx versions to a signed integer to more closely match the protocol.
Finally, this commit also adds tests for the new function.
The following benchmark shows the difference between using the new
function to get the serialize size for a typical block and serializing
into a temporary buffer and taking the length of it:
Bufffer: BenchmarkBlockSerializeSizeBuffer 200000 27050 ns/op
New: BenchmarkBlockSerializeSizeNew 100000000 34 ns/op
Closes#19.
There are certain cases such as getblocktemplate which allow external
callers to be repsonsible for creating their own coinbase to replace the
generated one. By allowing the pay address to be nil in such cases, the
need to specify mining addresses via --miningaddr can be avoided thereby
leaving the payment address management up to the caller.
This commit adds a new result type for the getblocktemplate RPC which
provides the fields as defined by BIP0022. The extension fields defined
by BIP0023 are not included yet.
ok @jcvernaleo
BIP0022 defines optional fields in a getblocktemplate request for long
polling and template tweaking.
In addition, for template tweaking, there are two fields, sigoplimit and
sizelimit, which are atypical in that they are allowed to be either
booleans or numeric. This requires the fields to be represented as
interfaces which means any code making use of the struct will need to use
type assertions or a type switch.
This commit updates GetBlockTemplateCmd accordingly.
ok @jcvernaleo
This commit modifies the RPC server such that all handlers now receive a
channel which will be notified when a client disconnects. This
notification can then be used to stop long-running operations early when a
client disconnects.
This capability was already present for websocket clients, but this commit
exposes it to standard HTTP clients as well.
This commit modifies the error return type for errors during script
validation to use the RuleError so they are consistent with the rest of
the errors. This also helps the calling code differentiate blocks
rejected due to script parsing and validation errors as opposed to
internal issues such as inability to read from the disk.
To accomplish this, two new two new RuleErrors, ErrScriptMalformed and
ErrScriptValidation, have been added.
Also, the errors for script parsing issues and script validation errors
have been improved to include both transaction hashes and indexes involved
in the validation effort. Previously script parsing issues had almost no
additional information as to which transaction input/outputs the failing
script came from.
The string needs to be unmarshaled rather than returned directly otherwise
it will have an extra set of quotes plus it's safter to properly unmarshal
it.
Pointed out by @jrick.
The submitblock RPC returns a string containing an error if it failed for
any reason. This comment detects a non-null return from submitblock and
converts that string to an error which is returned.
This commit adds a new behavior flag, BFNoPoWCheck which allows the caller
to indicate the check which ensures a block hashes to a value less than
required target should not be performed.
This commit change the ProcessBlock function to accept a new type named
BehaviorFlags in place of the current fastAdd parameter.
This has been done to pave the way for adding more control over the checks
that are performed such as avoiding the proof of work checks which will be
in an upcoming commit. A bitmask was chosen because it will allow the
ProcessBlock API to remain a little more stable since new flag additions
that change the behavior are only likely to be used by new code because
adding flags does not change the existing behavior.
ok @jrick
This commit modifies the TLS setup to only override the RootCAs for the
TLS connection if certificates are specified. This allows the
Certificates parameter to be ommitted from the connection config to use
the system CAs.
Now that the ProcessBlock function returns whether or not the block was an
orphan, the code which requests the parent blocks from the peer that sent
them has been moved to directly after the call to it.
This makes the handling more obvious and has allowed removal of the
blockPeer map which was kept so the notification handler could identify
which peer to request the blocks from.
ok @jrick
This commit changes the way that orphan blocks are identified by adding a
new boolean return value on ProcessBlock and removing the notification for
NTOrphanBlock.
This allows the calling code to identify orphan blocks immediately instead
of having to setup a seperate callback handler and implementing some type
of state tracking. This, in turn, allows cleaner code for handling them.
In addition, the tests have been updated for the new function signature
and also now check that each block is or is not an orphan as expected
which makes the tests more robust.
ok @jrick
This commit changes the RuleError type to a struct which consists of an
error code and human-readable description.
From a usage perspective, existing code should not break since type
asserting an error to a RuleError still works in the same manner. The
difference is the caller can now take that type asserted RuleError and
access the .ErrorCode field on it to programmatically identify the
specific rule that was violated.
ok @jrick
If connecting to a bitcoin RPC server as an HTTP POST client, full
response objects rather than just the raw result bytes were being
passed to the specific result unmarshalers, causing unmarshal errors
for the incorrect JSON types. To fix this, first unmarshal the
response body into a rawResponse, and pass only the raw result bytes
(or an error) to the specific handlers.
This was reported by nskelsey on IRC.
ok @davecgh
This reasons for this change follow:
- All instances of the same key should be consistent amongst the commands
and returns
- Output indices can't be negative, so rather than adding more code to
check for a negative after unmarshal, just allow the unmarshal to weed
out negatives
ok @jcvernaleo
While here, fix a bug found through testing. Register will now return
ErrDuplicateNet if the caller attempts to register any of the standard
network parameters provided by this package.
ok @davecgh
There are certain RPCs where an address should be sent as the raw string
instead of the encoded bitcoin address since the RPC handler on the other
end expects "keys" instead of Bitcoin addresses.
For example, the multisignature RPCs addmultisigaddress and createmultisig
can work with pubkeys (compressed, uncompressed, or hybrid) as well as
Bitcoin addresses.
The original issue which prompted these changes was report by Paul Snow on
IRC.
This value matches testnet and mainnet and makes more sense because it
allows way more coins to be generated which is useful during simulation
testing.
NOTE: this will invalidate existing simnet chains, but since they
are only intended to be short lived for the duration of a simulation test,
this is a non-issue.
Now that btcd support CPU mining, update the getgenerate, setgenerate,
gethashespersec, and getmininginfo RPC handlers to return the appropriate
information.
Also, remove the various extra help addenda about btcd not supporting
mining since it is no longer true.
This commit implements a built-in concurrent CPU miner that can be enabled
with the combination of the --generate and --miningaddr options. The
--blockminsize, --blockmaxsize, and --blockprioritysize configuration
options wich already existed prior to this commit control the block
template generation and hence affect blocks mined via the new CPU miner.
The following is a quick overview of the changes and design:
- Starting btcd with --generate and no addresses specified via
--miningaddr will give an error and exit immediately
- Makes use of multiple worker goroutines which independently create block
templates, solve them, and submit the solved blocks
- The default number of worker threads are based on the number of
processor cores in the system and can be dynamically changed at
run-time
- There is a separate speed monitor goroutine used to collate periodic
updates from the workers to calculate overall hashing speed
- The current mining state, number of workers, and hashes per second can
be queried
- Updated sample-btcd.conf file has been updated to include the coin
generation (mining) settings
- Updated doc.go for the new command line options
In addition the old --getworkkey option is now deprecated in favor of the
new --miningaddr option. This was changed for a few reasons:
- There is no reason to have a separate list of keys for getwork and CPU
mining
- getwork is deprecated and will be going away in the future so that means
the --getworkkey flag will also be going away
- Having the work 'key' in the option can be confused with wanting a
private key while --miningaddr make it a little more clear it is an
address that is required
Closes#137.
Reviewed by @jrick.
in go-flags/ini_private.go in function readIni there is no mechanism for
adding inline comments. Only comments that have a semicolon as the first
non-whitespace character are ignored.
According to Wikipedia http://en.wikipedia.org/wiki/INI_file#Comments
inline comments are not universally supported.
This change adds some white space for readability, but
name := strings.TrimSpace(line[1 : len(line)-1]) removes accidental whitespace
left in later.
Closes#135.
Also, since the new websoscket package allows the message type to be set
independently from the type of the variable, remove the casts between
strings and []byte in the websocket read/write paths. This avoids extra
copies thereby reducing the garbage generated.
Closes#134.
Rather than using bytes.NewBuffer, which is a read/write entity
(io.ReadWriter), use bytes.NewReader which is only a read entitiy
(io.Reader) in all cases where it is possible. Benchmarking shows it's
slightly faster and it's also technically more accurate since it ensures
the data is read-only.
There are a few cases where bytes.NewBuffer must still be used since a
buffer with a known length is required for those instances.
Rather than using bytes.NewBuffer, which is a read/write entity
(io.ReadWriter), use bytes.NewReader which is only a read entitiy
(io.Reader). Benchmarking shows it's slightly faster and it's also
technically more accurate since it ensures the data is read-only.
This commit resolves an issue where it was possible the block manager
could hang on shutdown due to inventory rebroadcasting. In particular, it
adds checks to prevent modification of the of rebroadcast inventory during
shutdown and adds a drain on the channel to ensure any outstanding
messages are discarded.
Found by @dajohi who also provided some of the code.
This commit updates doc.go to include the new simulation test network
magic constant as well as remove the information about BIP0037 not being
supported since that is no longer true.
This commit helps prevent transaction malleability by enforcing that the
extra dummy value on multisig transaction script contains no data for a
transaction . This syncs with a recent change in Bitcoin Core to remain
compatible.
As part of this change a new constant has been introduced which is used to
specify the script flags which are used for standard transactions. This
constant is then used in both the memory pool and the mining code to
ensure they remain in sync with one another.
Closes#131.
ok @jrick, @dajohi
This flag works similar to the existing --testnet flag does. That is to
say it selects the default simulation test network RPC port for either
btcd or btcwallet depending on whether or not the --wallet flag is also
present.
This commit, along with recent commits to btcnet and btcwire, expose a new
network that is intended to provide a private network useful for
simulation testing. To that end, it has the special property that it has
no DNS seeds and will actively ignore all addr and getaddr messages. It
will also not try to connect to any nodes other than those specified via
--connect. This allows the network to remain private to the specific
nodes involved in the testing and not simply become another public
testnet.
The network difficulty is also set extremely low like the regression test
network so blocks can be created extremely quickly without requiring a lot
of hashing power.
The default network ports are not really part of the wire protocol rather
they are part of a network parameters. Thus, this commit moves them to
the btcnet package.
The genesis blocks are not really part of the wire protocol rather they
are part of a network parameters. Thus, this commit moves the all of the
gensis blocks and tests to the btcnet package.
Also, create variables in the test package for the mainnet genesis hash,
merkle root, and coinbase transaction for use throughout the tests since
they the exported values are no longer available.
The genesis block for each network is a parameter for the network. As
such, it makes more sense to define them in this package instead of in the
wire protocol package.
ok @jrick
The examples uses a btcutil API that was recently updated from passing
btcwire.BitcoinNet to *btcnet.Params. This change updates the btcutil
call in the examples, as well as modifying the example in the README
to match that in doc.go (where errors were handled slightly
differently).
The ParamsForNet function was removed, so likewise this change removes
ErrUnknownNet error that it used to return.
As network registration is now necessary for correct handling of
alternate network encoding magics, and therefore the ErrDuplicateNet
error returned by Register is here to stay, kill the comment about the
error being removed later.
This commit refactors the code to make use of the btcnet package for
network-specific parameters instead of switching on the specific network.
For example, the percentage of the network that needs to run version 2
blocks is different between testnet and mainnet. Previously the code was
using a switch to choose these values. With this refactor, those
parameters are part of the network parameters provided when creating a new
chain instance.
Another example is checkpoints, which have been moved to btcnet so they
can be specified by the caller instead of hard coded into this package.
As a result, the checkpoints for the standard networks are now specified
in the btcnet package.
This makes it easier to add new networks such as a testnet4 should it
become needed. It also allows callers to define their own custom network
parameters without having to modify the code of the package to add new
switch cases for the custom network.
This commit adds more chain-related fields to the parameters as a part of
converting btcchain to work with btcnet parameters instead of coding the
network-specific knowledge into the package itself.
It also moves the checkpoints from btcchain since checkpoints are a
network parameter and make more sense here.
ok @jrick
This commit raises the maximum allowed size for a standard signature
script to cover a 15-of-15 multi-signature pay-to-script-hash with
compressed pubkeys.
This mirrors a recent change to remain compatible with Bitcoin Core.
ok @owinga who also helped verify and correct the script math.
Closes#128.
This change modifies the params struct to embed a *btcnet.Params,
removing the old parameter fields that are handled by the btcnet
package.
Hardcoded network checks have also been removed in favor of modifying
behavior based on the current active net's parameters.
Not all library packages, notable btcutil and btcchain, have been
updated to use btcnet yet, but with this change, each package can be
updated one at a time since the active net's btcnet.Params are
available at each callsite.
ok @davecgh
This change adds two parameters to the Params struct which are used by
btcd to alter behavior based on the active network, rather than
hardcoding checks for a particular network.
The first, ResetMinDifficulty, specifies whether the target difficulty
can change between the retarget intervals.
The second, RelayNonStdTxs, specifies whether standard transaction
checks should not be performed when accepting mempool transactions.
The zero values (false) for both of these bools are the correct
parameter values for mainnet.
While here, rename the test network version 3 to "testnet3", as both
btcd and btcwallet will include additional handling to rename this to
"testnet" for directory names, and a switch to "testnet3" is planned
in the future.
The btcutil package recently exposed a WIF type to provide more
functionality and simplify working with WIF strings. This commit changes
the DumpPrivKey and ImportPrivKey RPCs to use the new WIF type.
There are several RPCs which accept a pointer to a hash, transaction,
block, etc. Previously not all RPCs handled being passed a nil pointer
consistently.
Closes#4.
PushedData returns an array of byte slices containing any pushed data
found in the passed script. This includes OP_0, but not OP_1 - OP_16.
help from and ok @owainga
- Keep comments to 80 cols for consistency with the rest of the code base
- Made verify a method off of Signature instead of PublicKey since one
verifies a signature with a public key as opposed to the other way
around
- Return new signature from Sign function directly rather than creating a
local temporary variable
- Modify a couple of comments as recommended by @owainga
- Update sample usage in doc.go for both signing messages and verifying
signatures
ok @owainga
This change removes the internal pad function in favor a more opimized
paddedAppend function. Unlike pad, which would always alloate a new
slice of the desired size and copy the bytes into it, paddedAppend
only appends the leading padding when necesary, and uses the builtin
append to copy the remaining source bytes. pad was also used in
combination with another call to the builtin copy func to copy into a
zeroed byte slice. As the slice is now created using make with an
initial length of zero, this copy can also be removed.
As confirmed by poking the bytes with the unsafe package, gc does not
zero array elements between the len and cap when allocating slices
with make(). In combination with the paddedAppend func, this results
in only a single copy of each byte, with no unnecssary zeroing, when
creating the serialized pubkeys. This has not been tested with other
Go compilers (namely, gccgo and llgo), but the new behavior is still
functionally correct regardless of compiler optimizations.
The TestPad function has been removed as the pad func it tested has
likewise been removed.
ok @davecgh
This commit adds logic to track all registered notifications that have
been registered by the client in a notification state when the default
automatic reconnect is enabled.
The notification state is then used to reregister for all previously
registered notifications on reconnect. This allows the caller to
continue receiving notifications across reconnect cycles.
Also, since the new package exposes more connection related error
information, add a new ErrInvalidEndpoint which is returned if the
specified enpoint does not appear to be a valid websocket provider and
only return the ErrInvalidAuth error when HTTP authorization failure
status codes are detected.
Closes#1.
Since the example illustrates callbacks for the OnBlockConnected and
OnBlockDisconnected callbacks, also register for the notifications with
NotifyBlocks.
While here, document the fact that most of the callbacks require
registration.
Ordinarily, getwork will return an error if btcd is not connected to any
other peers. This commit relaxes that requirement when running in
regression test mode since it is useful for development purposes.
While here, also improve check which returns an error from getwork is not
current to exclude the check when the best chain height is zero since the
code never believes it is current when at height 0.
This commit adds a full suite tests for the new reject message added in
protocol version 70002 to bring the overall test coverage of btcwire back
up to 100%.
Closes#9.
This commit adds the new reject protocol message added to recent versions
of the reference implementation. This message is intended to be used in
response to messages from a remote peer when it is rejected for some
reason such as blocks being rejected due to not conforming to the chain
rules, transactions double spending inputs, and version messages sent
after they're already sent.
This is work toward issue #9.
- Correct MsgFilterLoad max payload
- Enforce max flag bytes per merkle block
- Improve and finish tests to include testing all error paths
- Add fast paths for BloomUpdateType
- Convert all byte fields to use read/writeVarBytes
- Style and consistency updates
- README.md and doc.go updates
Closes#12.
- Group the new read/writeVarBytes functions together to be consistent
with the existing code
- Modify the comments on the new read/writeVarBytes to be a little more
descriptive and consistent with existing code
- Use "test payload" for field name in the tests for the
read/writeVarBytes functions which is more accurate
- Remove reserved param from NewAlert since there is no point is having
the caller deal with a reserved param
- Various comment tweaks for clarity and consistency
- Use camel case for fuction params for consistency
- Move the NewAlert and NewAlertFromPayload functions after the receiver
definitions for code layout consistency
Closes#11.
Along the same lines as the previous commit, the RPCs that return
serialized data structures should use the max protocol version btcd
supports as opposed to the maximum protocol version etcwire supports.
The getinfo RPC should return the max protocol version btcd supports as
opposed to the maximum protocol version btcwire supports. Currently they
are both the same value, so there is no issue. However, they will not
always be the same.
Since the github contributors list only lasts for a period of time, it's
better to also maintain a contributors list in the repository rather than
solely relying on github.
* Introduced common methods readVarBytes, writeVarBytes.
* Added type Alert which knows how to deserialize
the serialized payload and also serialize itself back.
* Updated MsgAlert BtcEncode/BtcDecode methods to handle the
new Alert.
* Sane limits are placed on variable length fields like SetCancel
and SetSubVer
This commit modifies InfoResult to remove all of the omitempty tags since
the fields should be present in the marshalled JSON regardless of them
being zero values or not.
This commit modifies the types of the GetMiningInfoResult to better match
their underlying types. While the information was all available and
accurate, it's nicer to treat numeric values as proper types instead of
all float64s.
Also, a new networkhashps field has been added for compatibility with the
reference implementation.
This commit updates the block manager's local chain state when a block
processed by submitting it directly to the block manager as opposed to
only when it comes from the network.
Also, it modifies the submitblock RPC to use the concurrent safe block
manager process block instead of the unsafe btcchain version.
The combination of these two fixes ensure the internal block manager chain
state is properly synced with the actual btcchain state regardless of how
blocks are added.
This change fixes rescan to include transactions that pay to the
pubkey for a rescanned pubkey hash address. This behavior was lost
when the rescan was optimized for specific types of the
btcutil.Address interface.
ok @davecgh
This commit correctly sets the error in the marhsalled reply if it is
already a *btcjson.Error. Previously it would only set the error if it
was not of that type which led to some RPC results showing no error when
they actually had one.
This change overrides the cmd return value of custom registered
methods to always return an unparsableCmd if the marshaling as a
JSON-RPC request succeeded, but the request was an invalid structure
for the custom method.
In Discover, the reponse was lowercased for comparison. However,
this caused a 404 - Not found when fetching the url provided by
the location header if the url contained uppercase.
ok @owainga
In practise the races caused by not protecting these quite simply didn't
matter, they couldn't actually cause any damage whatsoever. However, I
am sick of hearing about these essentially false positivies whenever
someone runs the race detector (yes, i know that race detector has no
false positives but this was effectively harmess).
verified to shut the detector up by dhill.
If we switch the knuth shuffle to the version that swaps the element
with an element between it and the end of the array, then once we have
gotten to the amount of elements we need they won't change later in the
algorithm. Terminating here means that we only do 23% of the length of
the array worth of random swaps at most.
We make ka.na immutable in the address manager. Whenever we would update
the structure we replace it with a new copy. This beats making a copy of
all addresses once per getaddr command (max is just over 23000 we would
be copying, compared to at most 2000 copies on a new getaddr that has
all addresses we know with newer dates).
On unknown inventory types, handleGetDataMsg would loop forever.
After fixing that, if a getdata request only had unknown inventory
types, it would block forever.
ok @davecgh
The code was updated to automatically handle the transaction count in the
block header without having the additional field some time ago. This
comment was outdated.
Copying the RIPEMD160 after SHA256 hash result into a new stack array
to be used as a map lookup key can be quite expensive, and this should
be avoided if possible on intensive tasks such as rescans. This
change takes advantage of the new Hash160 methods of the
AddressPubKeyHash and AddressScriptHash types to use the address's
underlying hash array directly, rather than creating a copy from the
ScriptAddress result.
Unfortunately, for AddressPubKey, ScriptAddress may return either a
byte slice of len 33 or 65 depending on whether the pubkey is
compressed or not, so no such straightforward optimization is
possible.
As a result of this change, I have seen rescans perform roughly 3.5x
faster than before.
the new function AddUserAgent adds the user agent to the stack
and formats it as per BIP 0014
e.g: "/btcwire:0.1.4/myclient:1.2.3(optional; comments)/"
the validation on UserAgent has been moved to a new function
validateUserAgent
This commit renames the notifyallnewtxs RPC to notifynewtransactions to be
more consistent with the standard RPC names and to more accurately
reflect its intention which is to register for new transactions as they
are accepted to the memory pool. In addition, the notifications produced
have been renamed to txaccepted and txacceptedverbose depending on whether
or not the verbose flag was set when the client registered to receive the
notifications via notifynewtransactions.
This closesconformal/btcd#98.
The websocket extension command to register for notifications when a new
transaction has been accepted to the memory pool and the resulting
notifications have been renamed. This commit catches up to the change.
This commit renamed the notifynewtxs RPC to notifyreceived since it more
accurately reflects its intention which is to register addresses to be
notified about when they receive funds.
This is work towards conformal/btcd#98.
ok @jrick.
Since a chain verification can take a long time depending on the
parameters, this commit adds a debug print to the RPC server at the info
level for how many blocks are being verified and at what level.
The logic was also slightly modified so the number of blocks being checked
can easily be calculated and shown.
This commit extends the RegisterCustomCmd function to also accept a reply
parser which will be invoked for replies to the custom command via
ReadResultCmd. This allows replies to custom commands to be returned a
concrete structs instead of map[string]interface{}.
ok @jcvernaleo, @jrick
This commit moves all of the structs which deal with command results and
related functions to a new file named jsonresults.go.
This is in preparation of allowing custom results to be registered.
ok @jcvernaleo
This commit marks the CreateMessage and CreateMessageWithId functions as
deprecated as they are no longer maintained due to the new concrete
command approach.
It also updates the package documentation to show the new preferred method
as well as add details about btcd.
ok @jcvernaleo
The existing RpcCommand and TlsRpcCommand were build with
CreateMessage(WithID) in mind since they return an already marshalled
command. The approach was changed to use concrete commands created via
the New<command>Cmd family of functions. However, this means the caller
had to manually marshal the command before calling (Tls)RpcCommand which
was not very user friendly.
This commit adds two new functions named RpcSend and TlsRpcSend which
directly accepts a btcjson.Cmd and handles the marshalling itself.
ok @jcvernaleo
The NewTxNtfn notification was previously using a map[string]interface.
This commit modifies it to use the new concrete type
btcjson.ListTransactionsResult instead which allows nicer marshalling and
unmarshalling.
ok @jrick
This is no longer required since the RawCmd params now consist of a slice
of raw JSON messages which the registered parser can directly unmarshal.
ok @jcvernaleo
This commit modifies the AllVerboseTxNtfn command to use a parser along
with the standard btcjson.RegisterCustomCmd instead of the now removed
RegisterCustomCmdGenerator function. This is possible due to the recent
changes that defers the unmarshalling of the parameters of a RawCmd to the
parser.
The unmarshaller for the GetAddressBalanceCmd was calling the wrong parse
function. It wasn't caught by the tests because the error on
UnmarshalJSON was being ignored, so this commit also updates the tests to check the
error.
This commit modifies all of the commands and notifications to work
properly with the latest btcjson.RawCmd Params field changes.
This new approach is superior to the old method of using a []interface{}
because it means that each parameter is now unmarshalled into the expected
concrete type directly instead of whatever it happens to be in the JSON.
Since it is now preferred to use full blown structs for individual
parameters, the RescanCmd type has been changed to use a new OutPoint
that can be used directly for marshalling and unmarshalling instead of a
*btcwire.OutPoint.
Also, all of the MarshalJSON functions now make use of the new
btcjson.NewRawCmd function rather than repeating the same common field
over and over and all of the MarshalJSON, UnmarshalJSON, and parseX
functions have been made more consistent.
ok @jrick
This commit modifies the RawCmd type to use a slice of json.RawMessages so
that unmarshalling each individual parameter can be deferred to the
handler for the command.
This is superior to the old method of using a []interface{} because it
means that each parameter is now unmarshalled into the expected concrete
type directly instead of whatever it happens to be in the JSON. Due to
this, the old behavior where every number was unmarshlled as a float64 and
converted to the appropriate final type, which could lead to odd behavior
when going through marshal/unmarshal cycles, has been eliminated. This
change also means each individual parameter can now be full blown structs
(JSON objects) that marshal and unmarshal themselves which greatly
simplifies complex parameters and removes the need for additional and
error prone code.
This commit contains a lot of changes because every command needed its
MarshalJSON and UnmarshalJSON methods updated to support the new
mechanism. As a part of changing these functions, a new common creation
function for the the RawCmd struct was added to avoid the previous
duplication of the common fields. Also, rather than repeating the method
string in each MarshJSON functon, it makes use of cmd.Method().
Finally, some of the error messages were incorrect and have been corrected
as well as made consistent.
ok @jcvernaleo, @jrick, @owainga
This commit addes a couple of tests to ensure performing math operations
on a 5-byte integer, even if the result is a 4-byte integer, is considered
invalid.
- Coalesce the new bytes into the max message size constant to stay
consistent
- Correct optional relay tx field handling
- Rename the relay transactions field to DisableRelayTx so the zero value
of false has the correct default behavior
- Add tests for new bool fast paths in read/writeElement
- Stay consistent with version order in tests
- Add a single entry to TestVersionWire to test the new functionality
instead of adding a whole new TextVersionRelayTx function.
- Use BIP0037 in tests instead of hard coding 70001
- Nuke XXX that 70001 is different since this is handled now
- Fix and cleanup some comments
- Update test coverage report
Since fields of a version message after the AddrMe field are optional,
this commit adds a series of tests which ensure parsing version messages
which omit each of the optional fields works properly.
This commit modifies the MsgVersion.BtcDecode function to match the
behavior where fields after the first address field (AddrYou) are optional
and only read if the buffer contains remaining bytes.
Unfortunately this means the reader for MsgVersion.BtcDecode must be a
*bytes.Buffer or an error is returned. This is not an issue for the vast
majority of cases since all of the message reading code which is the main
way messages are read is already using a *bytes.Buffer, however, this
change might affect external callers if they are doing something special
with custom readers.
Fixes#14.
This commit modifies peers to use a max protocol version that is specified
as a constant in the peer code as opposed to the btcwire.ProtocolVersion
constant.
This allows btcwire to be updated to support new protocol versions without
causing peers to claim they support a protocol version which they actually
don't.
This commit corrects the debug log print to use the hash of referenced
coinbase transaction instead of the hash of the current transation when a
coinbase maturity is not met.
This change periodically (about every 10 seconds) notifies the
connected websocket client of the height of the last processed block
as part of the rescan. This enables clients to listen for the
notification and keep track of how much progress a rescan has made
even without any results being found. If the websocket connection is
lost during the rescan, on reconnect, clients may safely start over at
the last notified height.
This commit cleans up and moves a couple of comments in the recent pull
request which implements a rebroadcast handler (#114) in order to avoid
discussing internal state in the exported function comment.
How a function actually accomplishes the stated functionality is not
something that a caller is concerned with. The details about the internal
state are better handled with comments inside the function body.
This commit implements a rebroadcast handler which deals with
rebroadcasting inventory at a random time interval between 0 and 30
minutes. It then uses the new rebroadcast logic to ensure transactions
which were submitted via the sendrawtransaction RPC are rebroadcast until
they make it into a block.
Closes#99.
Rather than using the deprecated TxShas function on a btcutil.Block,
convert handleGetBlock to use the newer preferred method of ranging over
the Transactions to obtain the cached hash of each transaction.
This is a little more efficient since it can avoid creating and caching an
extra slice to keep the hashes in addition to having the hash cached with
each transaction.
Rather than using the deprecated TxShas function on a btcutil.Block,
convert the checkBIP0030 to use the newer preferred method of ranging over
the Transactions to obtain the cached hash of each transaction.
This is also a little more efficient since it can avoid creating and
caching an extra slice to keep the hashes in addition to having the hash
cached with each transaction.
This commit adds a new function named NewBlockTemplate along with
supporting infrastructure which is part of the core functionality needed
to support mining.
In particular the function creates a new block template which contains a
fully populated block with a zero nonce that is ready to be solved as well
as additional information regarding the fees and number of signature
operations for each transaction included in the block. The specific
transaction selection logic mirrors the reference implementation.
Various cleanup, optimizations, and comment suggestions provided by
@owainga. Also contains some naming suggestions and comment fixes from
@flammit.
This change adds select statements to each channel write that may
occur from a non-RPCS goroutine to unblock once the server has begun
shutting down. This prevents issues in clean shutdown where non-RPCS
goroutines may block indefinitely on message sends to the RPC server.
This change adds rescan fast paths for all current implementations of
the btcutil.Address interface. Rather than encoding a payment address
string from a txout script, the minimum number of bytes (which depends
on the address type) are copied into a stack array and used as a
lookup key to a map exclusive for that address type. This performs
better than the previous implementation due to avoiding the base58
encoding and generating less garbage (so each GC cycle completes
faster).
If the address type switch falls into the default (unhandled) case,
the encoded payment address is used as a fallback. This keeps the
intended rescan behavior even when new implementations of
btcutil.Address are added without a rescan fast path.
Benchmarks indicate that when a fast path is followed, for 20 byte
RIPEMD-160 hashes (such as P2PKH and P2SH), skipping the payment
address encoding (and thus bypassing the base58 encoding code) can
result in map lookups up to 60x faster.
Rather than returning an error when creating the RPC server an it can't
listen on any of the specified interfaces, only error when it can't listen
on all of the specified interfaces.
Change FetchBlockByHeight to do the database lookup itself instead of
calling getBlkByHeight. This saves an allocation of the entire block,
which it wasn't using.
In OpenDB, use fetchBlockShaByHeight instead of getBlkByHeight since it
only needs the sha. This saves memory on startup.
ok drahn@
Rather than updating the new chain state with the hash and height of the
block that was just processed, query the database for the best block.
This is needed because the block that was just processed might be a side
chain block or have caused a reorg.
serialization buffer in MsgTx's TxSha().
Benchmarking shows this is slightly faster due to avoiding the extra
garbage collection in addition to less peak memory usage.
Before: BenchmarkTxShaOld 500000 5626 ns/op
After: BenchmarkTxShaNew 500000 5457 ns/op
This commit introduces a chain state that is updated as blocks are
processed into the block chain instance associated with the block manager.
This has been done because btcchain is currently not safe for concurrent
access and the block manager is typically quite busy processing block and
inventory. This approach allows fast access to most chain information in
a concurrent safe fashion.
Rather than doing a lot of if/else in the configuration load for btcctl,
set the default for the RPC server to localhost and the RPC cert to the
btcd cert, then override them as needed depending on the --wallet and
--testnet flags.
The code was not resetting the counters when logging a message, so the
stats kept increasing rather than showing the number of items processed
since the last log message.
This has been resolved by copying the log function directly from btcd and
modifying it slightly to work with the importer.
blockImporter now has these new properties:
txProcessed - number of transactions processed by the importer
lastHeight - height of the last imported block
lastBlockTime - block time of the last imported block
lastLogTime - last timestamp of when a progress message was shown
the p/progress option in addblocks now indicates how often you want the progress message to appear
Both the script tests (positive and negative) and tx texts (ditto) are
present. Some of the tx tests in the negative section have been
replaced by a comment line explaining why that test is elided, to add in
diffing. The reasons were always that they test things handled by other
parts of the btcd stack (normally chain). For example MAX_MONEY, number
of outputs, coinbase sizes etc.
Much of the inital test logic from @dajohi using hand transcribed tables
for selected tests. The json parsers, script format parser and a lot of
cleaning up/bugfixing from your truly. @davecgh had some input too.
This commit corrects the number of expected inputs for a multi-sig script
to include the additional item that is popped from the stack due to the
OP_CHECKMULTISIG consensus bug (which is required and properly performed).
Note this issue did NOT affect the consensus critical code and hence would
not cause a chain fork. It did however, cause standard p2sh multisig txns
to be rejected from the mempool as nonstandard.
The tx rejected as non-standard which prompted this was spotted by
@mbelshe on IRC.
ok @owainga
This commit removes the word "only" from the non-standard transaction
error message when the number of items on the stack does not match the
number of expected items.
This was suggested by @mbelshe.
This commit updates the install command under the Installation section in
README.md to include all subdirectories thereby including the utilities
such as addblock and btcctl.
This commit makes a slight variant of the existing calcPastMedianTime
function available to external callers. The new exported version
calculates the past median time from the end of the current main chain
versus an arbitrary block node.
Found by tests dhill is working on. We checked that ifs were closed at the end
of execution but not at script switching time, we now move this to just after
finishing a single script.
Rather than having a separate query channel for the block manager, use the
same channel so the block handler acts as a pure FIFO queue. This
prevents possible starvation of query related messages.
ok @owainga
This commit makes a slight variant of the existing checkConnectBlock
function available to external callers. The new exported version checks
if the passed block will connect to the end of the current main chain.
In order to support this, a few other small modifications have been made
to the initial index generation and the existing checkConnectBlock since
it previously made some assumptions about the state of genesis block which
can no longer be assumed due accepting blocks from callers directly.
Also, add a quick test to ensure the new function fails when checking if
the genesis block will connect again when it's already inserted.
This change modifies the RPC server's notifiation manager from a
struct with requests, protected by a mutux, to two goroutines. The
first maintains a queue of all notifications and control requests
(registering/unregistering notifications), while the second reads from
the queue and processes notifications and requests one at a time.
Previously, to prevent slowing down block and mempool processing, each
notification would be handled by spawning a new goroutine. This lead
to cases where notifications would end up being sent to clients in a
different order than they were created. Adding a queue keeps the
order of notifications originating from the same goroutine, while also
not slowing down processing while waiting for notifications to be
processed and sent.
ok @davecgh
This mirrors a recent change in the reference implementation. Since the
vast majority of the network still uses it and transactions which aren't
considered standard won't be relayed, the rules for standard transactions
need to be kept consistent.
Ideally this should be handled via floating fees, but we're rather limited
until we have miner support and a larger network share so the probability
of a transaction being relayed and confirmed is high.
Closes#100.
ok @owainga
This commit exports the VarIntSerializeSize function to provide callers
with an easy method to determine how many bytes it would take to serialize
the passed value as a variable length integer.
This commit exports a new variant of the existing internal
calcNextRequiredDifficulty function which calculates and returns the next
required difficulty for a block after the end of the current best chain
based on the difficulty retarget rules.
In order to support the exported function the internal one was slightly
modified to accept the block timestamp instead of an entire block since
the timestamp is all that is needed and the caller of the exported
function might next have a full block yet.
This commit modifies the BuildMerkleTreeStore function to accept a slice
of btcutil.Tx transactions as opposed to a full block. This allows more
flexibility when calculating merkle roots since a full block may not be
created yet (particularly when generating blocks that need to be solved in
mining).
Previously, the BuildMerkleTreeStore function accepted a btcutil.Block
because originally the block itself cached the transaction hashes and it
was necessary to have access to the block to make use of the cached
transactions. However, the code has since been improved such that it
caches transaction hashes directly in each btcutil.Tx. This means the
code can remain as efficient as before while allowing the individual
transacitons to be passed.
This commit simply alphabetizes the subsystem logger variables, map, and
use switch so the order in the code is consistent with the sorted output
displayed when using --debuglevel.
The debuglevel parameter has accepted subsystems in additional to an
overall level for quite some time, but the sample config file was not
updated to reflect that.
This commit updates the sample config file accordingly.
This commit changes the websocket client code to use a mutex for
disconnect since it's theoretically possible a non-blocking select on the
quit channel could fall through from two different goroutines thus causing
a second call to close.
ok @jrick.
This change removes the processedtx notification, replacing it with
recvtx, and the spenttx notification, replacing it with redeemingtx.
Both new transactions return the full serialized transaction (encoded
in hexadecimal) rather than details about the transaction.
The old txmined notification has also been completely removed as it is
unreliable due to transaction malleability and no code should be
depending on it.
This change improves the mechanism by which btcd notifies a websocket
client of transaction relating to watched address and unspent outputs
in the following ways:
1. The old processedtx notification has been replaced with the new
recvtx notification. This notification, rather than parsing out
details used by wallet clients, sends the serialized transaction
(as hexadecimal) and any block details (if mined) if any transaction
output sends to one of the websocket client's watched addresses.
2. The old txspent notification has been replaced with the new
redeemingtx notification. This notification, rather than parsing
out details used by wallet clients, sends the serialized transaction
(as hexadecimal) and any block details (if mined) if any transaction
input spends a watched output.
3. When processing notifications for transaction outputs, if any output
spends to a client's watched address, a corresponding spent request
is automatically registered.
4. Transaction notifications originating from mempool now include both
transaction inputs and outputs, rather than only processing
5. When processing notifications for transaction inputs, a client's
output spent request is only removed if the transaction being
processed has also been mined into a block. In combination with the
4th change, this results in two redeemingtx notifications for
transactions which first appear in mempool and are subsequently mined
into a block.
This commit adds an additional sanity check to ensure the block that is
being processed does not contain a timestamp with a precision higher than
one second. This is necessary because the consensus rules only deal with
whole seconds in the time comparisons whereas the internal data structures
make use of Go time.Time values which support up to nanosecond precision.
Also, add a test to ensure the new functionality works as expected.
ok @owainga
This commit changes all cases which generate default timestamps to
time.Now to limit the timestamp to one second precision. The code which
serializes and deserializes timestamps already does this, but it is useful
to make sure defaults don't exceed the precision of the protocol either.
With this change there is less chance that developers using defaults will
end up with structures that have a higher time precision than what will
ultimately be sent across the wire.
This commit moves the subsidy halving interval to the chain params so it
can be configured per network. With that change it sets the regression
test halving interval to 150 to match the regression test params of the
reference implementation.
This was pointed out by @flammit.
The name findLatestKnownCheckpoint is confusing and doesn't really convey
the fact the purpose of the function is to the find the checkpoint prior
to the current known block.
Therefore, rename the function to findPreviousCheckpoint for clarity.
Also, update some comments to follow suit.
This commit adds an additional check to the block acceptance rules which
prevents new blocks that fork the main chain before the previous known
good checkpoint. This prevents storage of new, otherwise valid, blocks
from building off of old blocks which are likely at a much easier
difficulty and therefore could be used to waste cache and disk space.
Note this is slightly different than the other existing check which
prevents blocks with a timestamp before the timestamp of the latest known
good checkpoint since that check effectively prevents old side chain
blocks that already existed (per the claimed timestamp).
ok drahn@
This commit tightens the check for a pay-to-pubkey script by ensuring the
length of the pubkey is one of the two valid values of 33 or 65. This
mirrors the checks in the multisig script type check as well.
ok @owainga
This changes the implementation of the sendrawtransaction RPC handler
to match bitcoind behavior by always returning a rejection error for
any error processing or accepting the tx by the mempool. Previously,
if the tx was rejected for a rule error rather than an actual failure,
a client would still receive the tx sha as a result with no error.
Also, unexport the functions to generate script types. Everything should
(and is) be using PayToAddrScript() with an address type instead of
throwing bytes around.
discussed with #@davecgh
This commit makes use of the new btcscript.HasCanonicalPushes to enforce
canonical data pushes for transactions that are considered standard.
A canonical data push is one where the fewest number of bytes possible to
encode the size of the data being pushed is used. This includes using the
small integer opcodes for single byte data that can be represented
directly.
This commit adds a new ScriptBuilder interface that can be used to build
custom scripts. It currently is fairly basic, but it allows you to push
raw opcodes, ints, and data while respecting canonical encoding. These
primitives are sufficient to build any script.
This could be improved upon with quite a few things. One example would be
functions for certain opcodes that take properly typed parameters to make
it harder to create invalid scripts.
For now though, it is already quite useful since it handles all of the
opcode selection for canonical data pushes and integer encoding.
The initial discussion took place in #5.
This commit builds off the previous commit which fixed the execution of
multi-signature scripts with zero required signatures.
It introduces the concept of a "small int" which is one of OP_0 or OP_1 -
OP_16. All areas of code that deal with multi-sig transactions now make
use of these to ensure consistent handling.
This fixes a few issues surrounding multi-sig zero required signature
transactions included proper detection as a multi-sig script, signature
counting for script statistics, and
ok @owainga
It is possible for a multisignature transaction to require zero
signatures. For example, input 2 of testnet transaction
b2d93dfd0b2c1a380e55e76a8d9cb3075dec9f4474e9485be008c337fd62c1f7
in block number 185117.
Previously the code was pushing a false to the stack when no
valid signatures were found. This commit remedies that by pushing true
when no valid signatures were found, but none are required. Otherwise it
still pushes false when no valid signatures were found, but some are
required.
Fixes#7.
ok @owainga
This commit increases the block priority size to 50000 and the max
standard tx size to 100k. This matches relatively recent changes in the
reference implementation. The max block size was also increased to
750000, but since btcd does not currently create blocks, there is no
constant for it. That constant will likely be added as a part of the
getwork implementation since it requires block creation.
Closes#71.
Previously the websocket notifications for addresses were limited to
pay-to-pubkey-hash only. This commit removes that restriction so
all btcutil.Address types are supported. This includes pay-to-pubkey,
pay-to-pubkey-hash, and pay-to-script-hash.
When a spent notification and address notification is removed, the
tracking entry in the client which is used to track what to remove on
shutdown needs to be removed as well.
This commit refactors the entire websocket client code to resolve several
issues with the previous implementation. Note that this commit does not
change the public API for websockets. It only consists of internal
improvements.
The following is the major issues which have been addressed:
- A slow websocket client could impede notifications to all clients
- Long-running operations such as rescans would block all other requests
until it had completed
- The above two points taken together could lead to apparant hangs since
the client doing the rescan would eventually run out of channel buffer
and block the entire group of clients until the rescan completed
- Disconnecting a websocket during certain operations could lead to a hang
- Stopping the rpc server with operations under way could lead to a hang
- There were no limits to the number of websocket clients that could
connect
The following is a summary of the major changes:
- The websocket code has been split into two entities: a
connection/notification manager and a websocket client
- The new connection/notification manager acts as the entry point from
the rest of the subsystems to feed data which potentially needs to
notify clients
- Each websocket client now has its own instance of the new websocket
client type which controls its own lifecycle
- The data flow has been completely redesigned to closely resemble the
peer data flow
- Each websocket now has its own long-lived goroutines for input, output,
and queuing of notifications
- Notifications use the new notification queue goroutine along with
queueing to ensure they dont't block on stalled or slow peers
- There is a new infrastructure for asynchronously executing long-running
commands such as a rescan while still allowing the faster operations to
continue to be serviced by the same client
- Since long-running operations now run asynchronously, they have been
limited to one at a time
- Added a limit of 10 websocket clients. This is hard coded for now, but
will be made configurable in the future
Taken together these changes make the code far easier to reason about and
update as well solve the aforementioned issues.
Further optimizations to improve performance are possible in regards to
the way the connection/notification manager works, however this commit
already contains a ton of changes, so they are being left for another
time.
This commit adds a new configuration option, --rpcmaxclients, to limit the
number of max standard RPC clients that are served concurrently. Note
that this value does not apply to websocket connections. A future commit
will add support for limiting those separately.
Closes#68.
Rather than using a type specifically in btcd for the getrawmempool, this
commit, along with a recent commit to btcjson, changes the code over to
use the type from btcjson. This is more consistent with other RPC results
and provides a few extra benefits such as the ability for btcjson to
automatically unmarshal the results into a concrete type with proper field
types as opposed to a generic interface.
Rather than using a type specifically in btcd for the getpeerinfo, this
commit, along with a recent commit to btcjson, changes the code over to
use the type from btcjson. This is more consistent with other RPC results
and provides a few extra benefits such as the ability for btcjson to
automatically unmarshal the results into a concrete type with proper field
types as opposed to a generic interface.
Recent commits to the reference implementation have changed the syncnode
field to be present in the getpeerinfo RPC even when it is false. This
commit changes btcd to match.
The wsContext was being locked twice when NewBlockNotifyCheckTxIn is
called. Fixed by changing handlers to assume lock is acquired and
renamed methods to not be exported.
Since the Z values are normalized (which ordinarily mutates them as
needed) before checking for equality, the race detector gets confused when
using a global value for the field representation of the value 1 and
passing it into the various internal arithmetic routines and reports a
false positive.
Even though the race was a false positive and had no adverse effects, this
commit silences the race detector by creating new variables at the top
level and passing them instead of the global fieldOne variable. The
global is still used for comparison operations since those have no
potential to mutate the value and hence don't trigger the race detector.
This commit changes the server byte counters over to use a mutex instead
of the atomic package. The atomic.AddUint64 function requires the struct
fields to be 64-bit aligned on 32-bit platforms. The byte counts are
fields in the server struct and are not 64-bit aligned. While it would be
possible to arrange the fields to be aligned through various means, it
would make the code too fragile for my tastes. I prefer code that doesn't
depend on platform specific alignment.
Fixes#96.
This commit adds a new option, --logdir, which works in the same fashion
as the --datadir option. Consequently, the logging directory is name
"namespaced" by the network as well. This resolves the issue where two
btcd instances running (one for mainnet and one for testnet) would
overwrite each other's log files by default.
It also provides the user with a method to change the logging location to
non-default locations if they prefer. For example, it enables multiple
btcd instances on the same network to specify unique logging directories
(even though running multiple btcd instances on the same network is not
the most sane configuration).
Closes#95.
Since the websocket handlers run in their own separate goroutines, it's
possible that they execute after the websocket connection has been
closed and cleaned up. This commit add the necessary checks to ensure
stale data isn't added to notification lists and that requests to closed
connections are ignored.
This closes#92.
The getwork command alters the output depending on whether or not the
optional data parameter was specified. It is a JSON object when no data
was provided, and a boolean indicating whether a solution was found
when data was provided.
The getwork RPC now only accepts a single optional parameter which, if
present, is a string containing the hex-encoded block data to solve.
Also update the tests for the new correct parameters.
ok @jcvernaleo
Access to connections map and associated notification maps in rpcServer
need to be protected with s.ws.Lock to prevent race with add/remove new
clients.
This closes#88.
This commit resolves an issue where the block node index was forcing
entire blocks to be kept in memory thereby forcing excessive memory usage.
For example, prior to this change, the memory usage could consume upwards
of 1.5GB while importing bootstrap.dat via the addblock utility. With
this change the entire import takes <150MB. This also has the same memory
reduction to btcd since it uses the same code path.
Changed mempool.MaybeAcceptTransaction to accept an additional parameter
to differentiate betwee new transactions and those added from
disconnected blocks.
Added new fields to requestContexts to indicate which clients want to
receive all new transaction notifications.
Added NotifyForNewTx to rpcServer to deliver approriate transaction
notification.
Sending NotifyAllNewTXsCmd will register websocket client to receive
notifications on all new transactions.
Once registered the client will receive either AllTxNtfn or
AllVerboseTxNtfn based on the required verbose field in the
NotifyAllNewTXsCmd.
Previously the getnettotals was just looping through all of the currently
connected peers to sum the byte counts and returning that. However, the
intention of the getnettotals RPC is to get all bytes since the server was
started, so this logic was not correct.
This commit modifies the code to keep an atomic counter on the server for
bytes read/written and has each peer update the server counters as well as
the per-peer counters.
Rather than using a dedicated channel for the sync peer request and reply,
use a single query channel that accepts a query type as well as a reply
channel. This will allow other queries to be added in the future without
the various queries being racy.
This commit adds byte counters to each peer using the new btcwire
ReadMessageN and WriteMessageN functions to obtain the number of bytes
read and written, respectively. It also returns those byte counters via
the PeerInfo struct which is used to populate the RPC getpeerinfo reply.
Closes#83.
This commit adds two new funtions named ReadMessageN and WriteMessageN
which return an additional paramter for the number of bytes read or written,
respectively.
It also adds tests to ensure the number of bytes read and written are the
expected values both for successful reads/writes and unsuccessful ones.
Closes#6.
When given badly formatted signature or pubkeys like in block
0000000000000001e4241fd0b3469a713f41c5682605451c05d3033288fb2244, transaction
fd9b541d23f6e9bddb34ede15c7684eeec36231118796b691ae525f95578acf1 we could fail
on strange scripts because we returned an error instead of failing the
opcode and putting a FALSE on the stack.
Fixes chainfork issue on the aforementioned block.
This commit moves the connection endpoint for websockets to /ws instead of
/wallet. First, the former is more standard, and second the latter
presumes how the websocket is to be used.
Closes#80.
This commit improves how the headers-first mode works in several ways.
The previous headers-first code was an initial implementation that did not
have all of the bells and whistles and a few less than ideal
characteristics. This commit improves the heaers-first code to resolve
the issues discussed next.
- The previous code only used headers-first mode when starting out from
block height 0 rather than allowing it to work starting at any height
before the final checkpoint. This means if you stopped the chain
download at any point before the final checkpoint and restarted, it
would not resume and you therefore would not have the benefit of the
faster processing offered by headers-first mode.
- Previously all headers (even those after the final checkpoint) were
downloaded and only the final checkpoint was verified. This resulted in
the following issues:
- As the block chain grew, increasingly larger numbers of headers were
downloaded and kept in memory
- If the node the node serving up the headers was serving an invalid
chain, it wouldn't be detected until downloading a large number of
headers
- When an invalid checkpoint was detected, no action was taken to
recover which meant the chain download would essentially be stalled
- The headers were kept in memory even though they didn't need to be as
merely keeping track of the hashes and heights is enough to provde they
properly link together and checkpoints match
- There was no logging when headers were being downloaded so it could
appear like nothing was happening
- Duplicate requests for the same headers weren't being filtered which
meant is was possible to inadvertently download the same headers twice
only to throw them away.
This commit resolves these issues with the following changes:
- The current height is now examined at startup and prior each sync peer
selection to allow it to resume headers-first mode starting from the
known height to the next checkpoint
- All checkpoints are now verified and the headers are only downloaded
from the current known block height up to the next checkpoint. This has
several desirable properties:
- The amount of memory required is bounded by the maximum distance
between to checkpoints rather than the entire length of the chain
- A node serving up an invalid chain is detected very quickly and with
little work
- When an invalid checkpoint is detected, the headers are simply
discarded and the peer is disconnected for serving an invalid chain
- When the sync peer disconnets, all current headers are thrown away
and, due to the new aforementioned resume code, when a new sync peer
is selected, headers-first mode will continue from the last known good
block
- In addition to reduced memory usage from only keeping information about
headers between two checkpoints, the only information now kept in memory
about the headers is the hash and height rather than the entire header
- There is now logging information about what is happening with headers
- Duplicate header requests are now filtered
Previously there was only a function to get the latest checkpoint. This
commit exposes a new function named Checkpoints which returns a slice of
checkpoints ordered by height for the active network or nil when
checkpoints are disabled.
Previously the logging function which reports on progress was called for
every block, regardless of whether it was an orphan or not. This could be
confusing since it could show a different number of blocks processed as
compared to the old versus new heights reported (orphans do not add to the
block height since they aren't extending the main chain). Further, the
database had to be consulted for the latest block since the block we just
processed might not be the latest one if it was an orphan. This is quite
a bit more time conusming than it should've been for progress reporting.
This commit modifies that to only include non-orphan blocks. As a result,
the latest height shown will match the number of blocks processed (even
when there are orphans) and the additional block lookup from the database
is avoided.
The recent addition of the fast add path to support headers first was not
running the block node index pruning code which removes unneeded block
nodes from memory. This resulted in higher memory usage than needed in
fast add mode.
Previously the code was only adding a new block node as a child in the
inernal node index for the cases it extended the main chain or a side
chain and not for a node which caused a reorg. This resulted in the block
node pruning code not clearing the parent link of reorged nodes which
ultimately led to a sanity check error accordingly.
This commit resolves the issue by ensuring new block nodes are added as
children of their respective parents in all cases.
Closes#4.
The getinfo RPC method requires access to information only available in
the wallet. Therefore, it has been moved to the list of methods which
return an error information the caller to send the request to the wallet
instead.
The getnewaddress RPC method deals with wallet-related functionality and
therefore has been moved to the list of methods which return an error
information the caller to send the request to the wallet instead.
Both of these RPC methods require access to information ony available in
the wallet. Therefore they have been moved to the list of methods which
return an error information the caller to send the request to the wallet
instead.
Rather than having the iterator functions a separate entities that access
the state to iterate, just expose the iterators as receivers on the state
itself. This is more consistent with the style used throughout the code
and the other receivers on the state such as Count, OutboundCount, etc.
This commit adds full support for the getaddednodeinfo RPC command
including DNS lookups which abide by proxy/onion/tor rules when the DNS
flag is specified. Note that it returns an array of strings when the DNS
flag is not set which is different than the current version of bitcoind
which is bugged and scheduled to be fixed per issue 3581 on the bitcoind
issue tracker.
The getaddednode command dns flag alters the output. It is a JSON object
when true and a slice of strings containing the addresses when false.
Note there is a bug in bitcoind as of version 0.8.6 which returns the
addresses as a JSON object with duplicate keys. This has been reported as
issue 3581 on the bitcoind issue tracker.
This commit allows the result for getaddednodeinfo to be either the JSON
object or the string slice.
The createencryptedwallet extension RPC request should not be used to
create wallets for additional accounts. Instead, all btcwallet
accounts should use the same passphrase as the default account's
wallet. This change removes the account name and description
parameters from createencryptedwallet, as it will only be used to
create the default account.
This commit improves how the legacy RPC server responds to authentication
failures so things like web browsers can react better. The following
changes have been made:
First, authentication failures were only printing the 401 error response
in the body instead of setting the http status code. This means the
response had a 200 OK header with a body of 401 Unauthorized. Therefore
the client would think everything was ok, but see the response as
malformed JSON.
Second, the spec for 401 Unauthorized responses state they must include a
WWW-Authenticate header to instruct the client how to authenticate.
Without this, browsers won't prompt the user for credentials.
The previous websocket code required HTTP auth headers to be sent in order
to use the websocket. While this makes sense for most applications, some
use cases such as javascript-based websockets from browsers do no have the
ability to send headers.
This commit modifies the authentication logic to allow an alternative
authentication mechanism when HTTP auth headers can't be sent. In
particular, it introduces a new JSON-RPC command named authenticate which
accepts the username and passphrase for the RPC server. The command is
only required if the connetion has not already been authenticated via HTTP
auth headers and it must be the first command to be received. Sending any
other command will immediately disconnect the websocket.
ok from @owainga and @jrick.
This closes#77.
This fixes two issues: first, the sendrawtransaction handler had an
extra character in the key in the websocket handler map, preventing
the handler from never running. Second, a nil pointer dereference was
removed from the handler.
This change fixes the minedtx notifications for btcwallet, since the
websocket-handler now runs instead of falling back to the legacy RPC
handler.
This commit adds the btcdb memdb backend as a supported database type.
Note that users will NOT want to run in this mode because, being memory
only, it obviously does not persist the database when shutdown.
It is being added for testing purposes to help prevent constant abuse to
developer's hard drive when churning the block database multiple times a
day.
The specific parameters required by a backend is better left up to the
backend itself. For example memdb has no need for a database path, while
ldb does. This commit modifies the OpenDB and CreateDB functions to take
a arbitrary arguments which are passed along to the driver. The driver is
expected to verify the correct type and number of arguments and error
accordingly.
The existing backends have been updated accordingly.
This commit prunes several unused functions from the Db interface and the
underlying implementations. For the most part these are holdovers from
the original sqlite implementation. It also removes the types associated
with those functions since they are no longer needed. The following
functions and types have been removed:
- InvalidateCache
- InvalidateBlockCache
- InvalidateTxCache
- SetDBInsertMode
- InsertMode type and associated constants
- NewIterateBlocks
- BlockIterator interface
The reasons for removing these are broken out below.
- Neither of two current implementations implement these functions nor
does any of the fully functional code using the interface invoke them.
- After contemplating and testing caching of blocks and transactions at
this layer, it doesn't seem to provide any real benefit unless very
specific assumptions about the use case are made. Making those
assumptions can make other use cases worse. For example, assuming a
large cache is harmful to memory-constrained use cases. Leaving it up
to the caller to choose when to cache block and transactions allows much
greater flexibility.
- The DB insert mode was an artifact of the original sqlite implementation
and probably should have only been exposed specifically on the
implementation as opposed through generic interface. If a specific
implementation wishes to provide functionality such as special modes,
that should be done through type assertions.
This commit changes the node index creation to use block headers instead
of full blocks. This speeds up the initial node index generation since it
doesn't require loading a bunch of full blocks at startup.
This commit modifies local variables that are used for more convenient
access to a block's header to use pointers. This avoids copying the
header multiple times.
This commit switches the handleGetHeadersMsg function to make use of the
new FetchBlockHeightBySha and FetchBlockHeaderBySha functions in btcdb.
Also, while here, nuke the header copy which is no longer required due to
the recent btcwire changes.
This commit introduces two new functions to the btcdb.Db interface named
FetchBlockHeightBySha and FetchBlockHeaderBySha.
The FetchBlockHeightBySha function is useful since previously it was only
possible to get the height of block by fetching the entire block with
FetchBlockBySha and pulling the height out of the returned btcutil.Block.
The FetchBlockHeaderBySha function will ultimately make it much more
efficient to fetch block headers. Currently, due to the database design
in the ldb backend, the entire block has to be loaded anyways, so the only
current benefit is to avoid the deserialize on all of the transactions.
However, ultimately btcdb will gain a more efficient backend which can
also avoid reading all of the extra transaction data altogether.
The sqlite3 backend has been deprecated for quite some time. As a result,
it has not been updated with many of the more recent changes which means
the behavior no longer conforms to the interface contract.
This commit introduces two new functions for Blockheader named Serialize
and Deserialize. The functions provide a stable mechanism for serializing
and deserializing block headers to and from disk. The main benefit here
is deserialization of the header since typically only full blocks are
serialized to disk. Then when a header is needed, only the header portion
of the block is read and deserialized.
This commit removes the TxnCount field from the BlockHeader type and
updates the tests accordingly. Note that this change does not affect the
actual wire protocol encoding in any way.
The reason the field has been removed is it really doesn't belong there
even though the wire protocol wiki entry on the official bitcoin wiki
implies it does. The implication is an artifact from the way the
reference implementation serializes headers (MsgHeaders) messages. It
includes the transaction count, which is naturally always 0 for headers,
along with every header. However, in reality, a block header does not
include the transaction count. This can be evidenced by looking at how a
block hash is calculated. It is only up to and including the Nonce field
(a total of 80 bytes).
From an API standpoint, having the field as part of the BlockHeader type
results in several odd cases.
For example, the transaction count for MsgBlocks (the only place that
actually has a real transaction count since MsgHeaders does not) is
available by taking the len of the Transactions slice. As such, having
the extra field in the BlockHeader is really a useless field that could
potentially get out of sync and cause the encode to fail.
Another example is related to deserializing a block header from the
database in order to serve it in response to a getheaders (MsgGetheaders)
request. If a block header is assumed to have the transaction count as a
part of it, then derserializing a block header not only consumes more than
the 80 bytes that actually comprise the header as stated above, but you
then need to change the transaction count to 0 before sending the headers
(MsgHeaders) message. So, not only are you reading and deserializing more
bytes than needed, but worse, you generally have to make a copy of it so
you can change the transaction count without busting cached headers.
This is part 1 of #13.
This commit reduces the initial idle timeout before version negotiation
has happened on a new peer to 30 seconds. Previously it could take 5
minutes due to the general idle timeout.
The websocket.Server used the by websocket.Handler type automatically adds
a handshake function which prevents connections when the Origin header is
not set. Not all clients send this information and we already require
authentication headers as the auth mechanism anyways.
The websocket package assumes binary blobs if []byte is used. Since we're
using JSON-RPC for all the websocket communications, it should be text
based.
This commit changes the websocket Send/Receive and associated channels to
strings accordingly.
This removes the last notification that was being sent unsolicited.
Since it is no longer needed, the code to duplicate notifications to
all clients has been removed.
Previously the code performed a database query for every checkpoint (going
backwards) to find the latest known checkpoint on every block. This was
particularly noticabled near the beginning of the block chain when there
are still several checkpoints that haven't been reached yet.
This commit changes the logic to cache the latest known checkpoint while
keeping track of when it needs to be updated once a new later known
checkpoint has been reached.
While here, also add a log message when a checkpoint has been reached and
verified.
The previous script validation logic entailed starting up a hard-coded
number of goroutines to process the transaction scripts in parallel. In
particular, one goroutine (up to 8 max) was started per transaction in a
block and another one was started for each input script pair in the
each transaction. This resulted in 64 goroutines simultaneously running
scripts and verifying cryptographic signatures. This could easily lead to
the overall system feeling sluggish.
Further the previous design could also result in bursty behavior since the
number of inputs to a transaction as well as its complexity can vary
widely between transactions. For example, starting 2 goroutines (one to
process the transaction and one for actual script pair validation) to
verify a transaction with a single input was not desirable.
Finally, the previous design validated all transactions and inputs
regardless of a failure in one of the other scripts. This really didn't
have a big impact since it's quite rare that blocks with invalid
verifications are being processed, but it was a potential way DoS vector.
This commit changes the logic in a few ways to improve things:
- The max number of validation goroutines is now based on the number of
cores in the system
- All transaction inputs from all transactions in the block are collated
into a single list which is fed through the aforementioned validation
goroutines
- The validation CPU usage is much more consistent due to the collation of
inputs
- A validation error in any goroutine immediately stops validation of all
remaining inputs
- The errors have been improved to include context about what tx script
pair failed as opposed to showing the information as a warning
This closesconformal/btcd#59.
The RPC server was performing some of the shutdown logic in the wrong
order, that is, logging the the server has shut down, waiting for all
server goroutines to finish, and then closing a channel to notify
server goroutines to stop. These three items have been reversed to
fix a hang where goroutines currently being waited on had not shut
down because they did not receive the notification.
While here, the server waitgroup was incremented for a goroutine that
was running without it, another select statement was added to stop a
duplicate close (which never occured last commit when I added the
select statements), and the "stopping rescan" logging was moved to
debug to make the ^C shutdown logging nicer.
It was pointed out in #76 that if you arrived to the Update section
of the README without seeing the Installation section, the requirement for
Go 1.2 is easy to miss. This commit builds the requirement in the
Installation section and adds it to the Updating section as well to
hopefully make it more noticable.
The addblock utility was originally written as a quick debug tool during
initial development to populate blocks into the database. However, now
that it has been designated as the standard way to import bootstrap.dat
(and indeed block data files in general), it was lacking a few features
such as properly checking against the chain rules and known good
checkpoints.
This commit reworks and improves the utility in several ways:
- Imported blocks are now checked against the chain rules including
checkpoints to ensure they match the known good chain
- The utility now properly shuts down after processing all blocks
- Attempting to import orphan blocks (blocks which build off a block you
don't yet have in the database) returns an error
- Blocks that are already known are now skipped instead of causing an
error which means you can stop and restart the import mid-way through
without issues or start it after you've already downloaded a
portion of the chain
- The block height is no longer assumed to start at 0 which means input
files that start later in the chain will work properly so long as you
already have the chain at least up to the point of the block just before
the first one in the input file
- Improved error handling and reporting
- How often the progress display is shown is now configurable
- Statistics about how many blocks were processed, imported, and already
known are now displayed after the input file has been fully processed
This resolves comments made in #60.
The warning about a missing config file should only be shown after all
other configuration has succeeded so it's not shown when there are invalid
options specified. Also add a comment about it where its intended
placement is for the future.
This changes the protocol between btcd and btcwallet to follow
JSON-RPC specifications sending notifications as requests with an
empty ID.
The notification request context handling has been greatly cleaned up
now that IDs no longer need to be saved when sending notifications.
This commit changes a couple of sections which deal with large lists of
inventory vectors to use the new size hint functions recently added to
btcwire. This allows a bit more efficiency since the size of the list is
known up front and we can therefore avoid dynamically growing the backing
array several times. This also helps avoid a Go bug that leaks memory on
appends and GC churn.
This commit adds two new functions named NewMsgGetDataSizeHint and
NewMsgInvSizeHint. These are intended to allow callers which know in
advance how large the inventory lists will grow the ability to provides
that information when creating the message. This in turn provides a
mechanism to avoid the need to perform several grow operations of the
backing array when adding large number of inventory vectors.
This commit changes all code which deals with extracting addresses from
scripts to use the btcscript API ExtractPkScriptAddrs which in turn makes
use of the new btcutil.Address interface.
This provides much cleaner code for dealing with arbitrary script
destinations which is extensible without having to churn the APIs if new
destination types are added.
This commit significantly changes the address extraction code. The
original code was written before some of the other newer code was written
and as a result essentially duplicated some of the logic for handling
standard scripts which is used elsewhere in the package.
The following is a summary of what has changed:
- CalcPkScriptAddrHashes, ScriptToAddrHash, and ScriptToAddrHashes have
been replaced by ExtractPkScriptAddresses
- The ScriptType type has been removed in favor of the existing
ScriptClass type
- The new function returns a slice of btcutil.Addresses instead of raw
hashes that the caller then needs to figure out what to do with to
convert them to proper addressses
- The new function makes use of the existing ScriptClass instead of an
nearly duplicate ScriptType
- The new function hooks into the existing infrastructure for parsing
scripts and identifying scripts of standard forms
- The new function only works with pkscripts to match the behavior of the
reference implementation - do note that the redeeming script from a p2sh
script is still considered a pkscript
- The logic combines extraction for all script types instead of using a
separate function for multi-signature transactions
- The new function ignores addresses which are invalid for some reason
such as invalid public keys
This implements --onion (and --onionuser/--onionpass) that enable a
different proxy to be used to connect to .onion addresses. If no main
proxy is supplied then no proxy will be used for non-onion addresses.
Additionally we add --noonion that blocks connection attempts to .onion
addresses entirely (and avoids using tor for proxy dns lookups).
the --tor option has been supersceded and thus removed.
Closes#47
This commit modifies the names of opcdoes shown in the oneline script
disassembly to match the reference implementation. In particular
OP_1NEGATE, and OP_0 through OP_16 are changed to the raw numbers
they represent when doing oneline disassembly. When doing full
disassembly, the full opcode names are still shown.
ok @owainga.
This commit modifies the new valid peer message to display the useragent.
Previously this information was only available by setting the PEER
subsystem debuglevel to debug or lower.
This was prompted by #64.
Rather than returning an empty string from DisasmString if a script fails
to parse, return the disassembly up to the point of the failure along with
[error] appended. The error is still returned in case the caller wants
more information about the script parse failure.
This commit adds two new functions named PayToScriptHashScript and
PayToAddrScript. The first one creates and returns a public-key script
which pays to the provided script hash and conforms to BIP0016.
The second function takes the new btcutil.Address interface type and
returns an appropriate script to pay to the address type in the interface.
It currently works for btcutil.AddressPubKeyHash and
btcutil.AddressScriptHash.
The inputs to the createrawtransaction command are JSON strings, however
the NewCreateRawTransactionCmd function takes concrete types. Thus,
callers which deal with the raw JSON need a method to perform the
validation and conversion. This logic is the same as what needs to happen
when unmashalling a full JSON-RPC command, so this commit factors that
logic out into a separate function and exports it.
Fixed up bad function comment headers
Added a small set of tests for the ScriptToAddrHashes function to test
functionality of a couple real multisig cases as well as error checking
The name handlers for a package level is a bit too generic and could
easily cause a name collision. Even though the compiler would catch it,
use something a bit more descriptive.
Since the command to handler mappings are the most often modified and
referenced code in rpcserver.go and rpcwebsocket.go, move them near the
top of their respective files.
This commit cleans up the standard RPC command hanlding a bit by removing
the websocket specific notification channel from the handlers. This was
previously required because the sendrawtransaction, when called from a
websocket enabled connection, needs to add a notification for when the
transaction is mined.
This commit modifies that to instead implement a websocket extended
version of sendrawtransaction which invokes the standard handler and adds
the notification. In addition, the main send was modified to first look
if the command has a websocket specific handler first, and then falls back
to standard commands, rather than the previous approach of first checking
for a standard command and falling through to websocket commands. This
essentially allows websockets connections to extend commands with the same
name with additional functionality such as what was done in this commit.
The rpcserver.go file is starting to get a bit unwieldy. This commit
moves the separable websocket specific bits into a separate file named
rpcwebsocket.go.
Added MarshalJSON method to Vin which properly omits the Vout field
from coinbase Vin's by using anonymous structs with the proper
subsets of fields outputted by bitcoind
Changed Vout.ScriptPubKey to add omitempty for ReqSigs and Addresses:
both fields are not shown when there is an error identifying to
scriptType or parsing addresses in bitcoind
Added error checking for script disassembley
Changed vout to handle errors in processing the way bitcoind does: the
type displayed is "nonstandard" when the calculated type is nonstandard
or nulltype and also when there is an error getting the address.
Still doesn't properly support multisig addresses, but now it should
return "nonstandard" since since address lookup fails for those cases.
Since the decoderawtransaction result makes use of the same vin and vout
lists, this commit also factors the logic for those out into separate
functions.
The fields of the PeerInfo should not have been marked omit as the only
ones that should be omitted to for compatibility are the SyncNode and
BanScore fields.
In order to match the Satohsi client, the return is supposed to be an
8-digit string representation of the services instead of the actual
services numeric value.
The ScriptSig field of the Vin type for TxRawResult is now a pointer in
btcjson so it can be properly omitted. This commit updates the code to
create the new ScriptSig object as needed.
The previous commit modified the Vin ScriptSig field to be a pointer to an
anonymous struct so it could be properly omitted. However, the callers
need to be able to create a new object to assign to the field, so this
commit makes the previous anonymous struct an exported type named
ScriptSig.
In order for a field which is a struct to be omitted from JSON
marshal/unmarshal, it must be a pointer so the json package can tell when
it's empty (when it's nil).
The getrawtransaction RPC call should return a hex-encoded string of the
transaction when verbose is false instead of a TxRawResult object with the
Hex field set to be compatible with the Sathoshi client. This commit,
along with a recent commit to btcjson corrects this.
Also, while here, do a bit of cleanup, finish a TODO to check for an
invalid hash, and optimize the handling of non-verbose slightly.
The getrawtransaction command has recently added a verbose flag which
alters the output. The previous code made use of the "Hex" field of the
TxRawResult to return the information, but this is not consistent with the
original getrawtransaction RPC call which returns a string when verbose is
false and the TxRawResult JSON object when it is true.
This commit corrects that by allowing the result for getrawtransaction to
be either form.
The getblock RPC call should return a hex-encoded string of the block when
verbose is false instead of a BlockResult object with a Hex field set to
be compatible with the Sathoshi client. This commit, along with a recent
commit to btcjson corrects this.
Also, while here, move code which only applies to verbose mode after the
call which handles the non-verbose logic. This saves a few cycles since
the non-verbose logic doesn't need the extra information.
The getblock command has recently added a verbose flag which alters the
output. The previous code added a new field "Hex" to the BlockResult, but
this is not consistent with the origin getblock RPC call which returns a
string when verbose is false and the BlockResult JSON object when it is
true.
This commit corrects that by first removing the Hex field from the
BlockResult and second allowing the result for getblock to be either form.
This commit adds a new function to btcctl that shows the results as
properly indented JSON instead of relying on spew and changes all of the
commands that used spew to the new function. The output of btcctl
should be more user-facing than developer-facing.
This commit modifies btcctl to show float values with %f instead of the
default %v. This means the values will show similar to 1180923195.260000
instead of 1.18092319526e+09 (scientific notation).
The fee field of the getrawmempool RPC JSON response should be in Bitcoins
instead of Satoshi. This commit corrects that issue.
Also, add a couple of comments and fix a comment typo while here.
Since there is already a variable for the current block height in addition
to the next block height, use the existing curHeight variable instead
doing nextBlockHeight-1 in mempool add.
This commit does some housekeeping on peer.go to make the code more
consistent, correct a few comments, and add new comments to explain the
peer data flow. A couple of examples are variables not using the standard
Go style (camelCase) and comments that don't match the style of other
comments.
Instead of one thread that queues and writes, we move to a two queue
model. The queueHandler muxes all the sources of outgoung packets and
drips them to the actual sender. This is done so that a large send
doesnt' allow the channels to fillup and cause blockmanager and server
to block, which delays other peers.
Most messages we handle as is. However, for getdata we do some manual
limiting and pipelining, we queue up three and then we load the next
into memory, not sending it until the otherp ackets have been sent. We
may want to change this later to queue the packet *then* wait so that we
don't completely drain the pipe.
A few misc tweaks to avoid deadlocking by ensuring the all channels will
always drain. mostly this relates to ensuring that we know no more data
will be coming before we drain the channel, and not queueing after we
are marked to disconnect.
Discussed heavily with drahn@ and davec@.
This commit modifies the code to use the new btcec Signature.Serialize API
instead of the internal sigDER which has now been removed. This closes#3.
ok @owainga
This commit exposes a new function named Serialize on the Signature type
which can be used to obtain a DER encoded signature. Previously this
function was named sigDer and was part of btcscript, but as @donovanhide
pointed out in issue btcscript/#3, it really should have been part of this
package.
ok @owainga
Since the function was only exported for use by the test package (and was
commented as such), just move it into the internal_test.go file so it is
only available when the tests run.
This commit essentially rewrites all of the primitives needed to perform
the arithmetic for ECDSA signature verification of secp256k1 signatures to
significantly speed it up. Benchmarking has shown signature verification
is roughly 10 times faster with this commit over the previous.
In particular, it introduces a new field value which is used to perform the
modular field arithmetic using fixed-precision operations specifically
tailored for the secp256k1 prime. The field also takes advantage of
special properties of the prime for significantly faster modular reduction
than is available through generic methods.
In addition, the curve point addition and doubling have been optimized
minimize the number of field multiplications in favor field squarings
since they are quite a bit faster. They routines also now look for
certain assumptions such as z values of 1 or equivalent z values which
can be used to further reduce the number of multiplicaitons needed when
possible.
Note there are still quite a few more optimizations that could be done
such as using precomputation for ScalarBaseMult, making use of the
secp256k1 endomorphism, and using windowed NAF, however this work already
offers significant performance improvements.
For example, testing 10000 random signature verifications resulted in:
New btcec took 15.9821565s
Old btcec took 2m34.1016716s
Closesconformal/btcd#26.
This changes notifications to JSON-RPC Requests, rather than
Responses, that also satisify the btcjson.Cmd interface and are
registered with btcjson's parser. This prevents issues where JSON-RPC
Response IDs clash due to a client using the same ID as what an old
notification used.
As this changes the API, and thus, requires notification handlers to
be modified, the remaining missing notifications used by btcwallet
have been implemented. Applications parsing these notifications, such
as btcgui, can now use a common handler function signature for all
notifications.
Test coverage for all notifications has been added (excluding testing
for badly-marshaled notifications with wrong numbers of parameters, or
wrong types).
Fixes#2.
The regression test does not work properly with the new headers-first
download approach, so force the old inv-based block download for
regression test mode.
It is not necessary to do all of the transaction validation on
blocks if they have been confirmed to be in the block chain leading
up to the final checkpoint in a given blockschain.
This algorithm fetches block headers from the peer, then once it has
established the full blockchain connection, it requests blocks.
Any blocks before the final checkpoint pass true for fastAdd on
btcchain operation, which causes it to do less valiation on the block.
It is not necessary to do all of the transaction validation on
blocks if they have been confirmed to be in the block chain leading
up to the final checkpoint in a given blockschain.
This algorithm fetches block headers from the peer, then once it has
established the full blockchain connection, it requests blocks.
Any blocks before the final checkpoint pass true for fastAdd on
btcchain operation, which causes it to do less valiation on the block.
The allows the tests to run without showing warning for malformed bits
(which are intentionally malformed for testing purposes). Also, the
tests would not compile since the new btclog backend was switched out.
This commit resolves that.
since we don't wait for peers, this largely just waits for the server procs
themselves to die. Unless the entire server is wedged (which is what kill -9 is
for) this should always shut down fairly swiftly.
This should mean we sync addrmanager and disestablish upnp correctly on
interrupt.
Discussed with davec.
This code borrows and fixes up a chunk of code to handle upnp from
Taipei-Torrent (https://github.com/jackpal/Taipei-Torrent), under
current versions of go none of the xml parsing was working correctly.
This fixes that and also refactors the SOAP code to be a little nicer by
stripping off the soap containers. It is still rather rough but seems to
correctly redirect ports and advertise the correct address.
Upnp is not run by default. --upnp will enable it, but it will still not
run if we are not listening or if --externalip is in use.
Closes#51
The you address is the one we already set up fo the user, so either waht
we connected to (this will work with tor, etc), or the ip the user
connect to us from otherwise. We must however check to see if it is the address
of the proxy and strip it.
The me addesss, we use the same address selection for local addresses as
always
This should mean that we pass our tor address out in the version message
and thus the peers should add us to their addressmanager.
This implements only the bare bones of external ip address selection
using very similar algorithms and selection methods to bitcoind. Every
address we bind to, and if we bind to the wildcard, every listening
address is recorded, and one for the appropriate address type of the
peer is selected.
Support for fetching addresses via upnp, external services, or via the
command line are not yet implemented.
Closes#35
Perform the requisite processing on .onion addresses to turn them into the tor
reserved ipv6 region (the same as bitcoind and onioncat). Furthermore,
when printing an ip address, reverse the conversion so we print it
nicely. base32 as standard is uppercase, but tor and bitcoind seem to
use lowercase so we first must for we force .onion addrs to uppercase
(and to lowercase on the reverse).
As a side effect we now should handle dns names on the command line (via tor if
required) and add them to the addressmanger as necessary.
The code to send an address messages in batches was previously clearing
all addresses from the existing message after queueing it to be sent.
Since the message is a pointer, this means it was removing the addresses
from the same message which might not have already been sent yet (from
another goroutine) which led to a race.
This commit modifies the code to create a new address message for each
batch as intended.
Fixes#58.
Updated handleGetRawTransaction to populate all the fields required to
match bitcoind. It still doesn't handle MULTISIG addresses correctly.
Changed handleGetBlock to implement new optional verbose (default true)
flag and also added a verboseTx flag to return TxRawDefault instead of
Txid. When verbose=false, GetBlock returns hex-encoded wire bytes for
the block.
Changed TxRawResult to omitempty block info for mempool tx as well as
moved Txid to Vin.TxId from Vin.ScriptSig.Txid (both match bitcoind
output)
Added support for new bitcoind [verbose=true] and added non-standard
optional verboseTx to return TxRawResults intead Txids
The vout field (as part of the getrawtransaction JSON reply) should be
set to the input's previous outpoint's index, not the current input
index.
Found by flam and reported on IRC. Thanks!
This switches a break to a continue if a txout does not include a
pay-to-pubkey-hash script type. btcwallet only supports
pay-to-pubkey-hash at the moment, and this fixes an issue where a tx
may have an different type of output, as well as pay-to-pubkey-hash,
which may be ignored by the wallet notification code.
Found by dhill.
I previously fixed the duplicate send (before seeing GH issue #54),
but forgot that btcwallet expects a nil reply when rescan has
finished. This adds the final reply back, but replies with nil.
Fixes#54.
This updates the replies for rescan and tx notifications with
additional information that is needed for wallet to properly support
the listtransactions command.
While here, drastically improve rescan performance by not looking up
every sha in rescan's block range.
This commit modifies btcd to run cleanly as a Windows service. btcd is
intended to be a long running process that stays synchronized with the
bitcoin block chain and provides chain services to multiple users. It
follows that a service is the best option on Windows for this
functionality.
A few key points are:
- Supports graceful shutdown via the service stop/shutdown commands
- Integrates cleanly with the Windows event log
- Adds a new /s flag that can be used to install/remove/start/stop the
service
One outstanding issue is that the application data directory is currently
user specific which means, by default, if you start btcd as a user, the
same data won't be used as when it's running as a service. This needs to
be resovled. The most likely approach will be to put all data into the
common appdata directory Windows provides, but it will require some
additional work to deal with permissions properly as user processes can't
write there by default.
Closes#42.
This commit modifies the logging to also log all output to a rolling log
file in the btcd home directory under the logs folder. It uses a maximum
size of 10MB per log file and a max rotation size of 3. This means the
log files will not exceed 30 megabytes.
The code was previously only changing the logging level if it wasn't the
default which is accurate for setting the log level once at startup time,
but it needs to set it unconditionally to allow dynamic updates.
Also, make every subsystem within btcd use its own logger instance so each
subsystem can have its own level specified independent of the others.
This is work towards #48.
Outbound we already have the exact same thing set up, and this should
quieten the race detector. Please note that this does *not* cause
problems with the service flags being wrong. Since by this point we have
already done every thing that would use the service flags from p.na in
addrmanager, and now p.Services is correct..
This commit adds version information to btcctl. The plan is to keep
it in lock step with the btcd version. It also updates the release script
so the version file is updated automatically with the btcd one.
This commit improves the configuration for btcctl in several ways:
- Add the ability to specify a config file
- Add a default entry to the rpc cert to point to the location
it will likely be in the btcd home directory
- Move the config bits into a separate file for easier maintenance
- Remove periods from --rpccert/--rpckey since none of the other options use
them
- Make it a little more clear that the --listen and --rpclisten options can be
used multiple times by changing the summary to start with "Add an ..."
which also matches the --addpeer style
All rpc sockets now listen using TLS by default, and this can not be
turned off. The keys (defauling to the datadirectory) may be provided by
--rpccert and --rpckey. If the keys do not exist we will generate a new
self-signed keypair with some sane defaults (hostname and all current
interface addresses).
Additionally add tls capability to btcctl so that it can still be used.
The certificate to use for verify can be provided on the commandline or
verification can be turned off (this leaves you susceptible to MITM
attacks)
Initial code from dhill (rpc tls support) and jrick (key generation),
cleanup, debugging and polishing from me.
These act the same as the !tls versions of this code but they take a PEM
encoded certificate chain to be used to verify certificates (ca verified
certs could use /etc/ssl/certs.pem) and a parameter to skip cert
verification and will use https internally.
- Lock the mempool when removing transactions during a notification as
intended
- When generating the inventory vectors to serve on a mempool request,
recheck the memory pool for each hash since it's possible another thread
could have removed an entry after the initial query for available
hashes
- When a block is connected, remove any transactions which are now double
spends as a result of the newly connected transactions
The regression test mode is special and therefore most likely will not
want to use the same settings that are in the configuration file. The -C
option can still be used to specify a config file in regression test mode
if desired.
When a transaction is being checked for acceptance into the transation
memory pool, it makes use of a chain function to ensure the invariant rules
for what transactions are allowed into a block are not violated. That
function returns a btcchain.RuleError if rules are violated. However,
since this code path is tailored to free-standing transactions, the error
needs to be converted to a TxRuleError so the caller can properly detect
the transaction as a rejected transaction instead of treating it like an
real error.
This commit modifies the transaction memory pool handling so that it does
not relay resurrected transactions. The other peers on the network will
also be reorganizing to the same block, so they already know about them.
This commit makes use of the new default-mask go-flags option in
conjunction with delaying the usage display until after the config file is
parsed. This has a couple of nice properties such as showing the actual
values that will be used as loaded from the specific config file instead
of the defaults specified in btcd itself, and also allows any config file
parsing errors to be shown prior to displaying the usage.
This commit adds a few more checks to restrict what transactions are
allowed into the transaction memory pool and therefore are candidates
to be mined and relayed.
In particular, the following changes were made to what is considered
standard:
- nulldata scripts are now supported and considered standard
- multi-signature transaction are now checked to ensure they only have a
max of 3 pubkeys and the number of signatures doesn't exceed the number
of pubkeys
- the number of inputs to a signature script must now match the expected
number of inputs for the script type (includes support for additional
pay-to-script-hash inputs)
- the number of inputs pushed onto the stack by a redeeming sig script
must match the number of inputs consumed by the referenced pk script
- there can now only be a max of one nulldata output per transaction
This allows the provision of address/port pairs to be listened on instead
of just providing the port. e.g.:
btcd --listen 1.2.3.4:4321 --listen 127.0.0.01 --listen [::1]:5432
When --proxy and --connect are used, we disable listening *unless* any --listen
arguments have been provided, when we will listen on those addresses as
requested.
Initial code by davec, integration by myself.
Closes#33
allow listens to fail, but warn. error if all failed
fmt
So far we only do level 0 and level 1 checks (precense and basic
sanity). The checks done at higher levels in bitcoind are closely
coupled with their database layout.
arguably Closes#13
This change allows wallet to record all transactions in a block before
receving the new block notification, and then process them all
together when the blockconnected notification arrives.
Previously, RemoveMinedTxRequest was being run from a caller which
held a reader lock for the websocket request contexts. When
RemoveMinedTxRequest tried to grab a writer lock, it would block.
This change creates a new function, removeMinedTxRequest, that does
not grab any locks, and the caller (NotifyBlockConnected) grabs a
writer lock instead of a reader lock.
Previously, on a blockconnected notification, the websocket context
reader lock was not always being given up properly. This change
defers the unlock so it will always happen.
This fixes an issue where wallet will stop responding (due to not
being able to complete its handshake) on reconnect.
This commit makes use of the new btcutil.AppDataDir function which chooses
appropriate data directories for each supported operating system. It also
adds code to the upgrade path to properly migrate existing data from the
old to new locations.
This is part of work toward issue #30.
This commit modifies the tests to setup a chain instance backed by the new
memory database backend for btcdb. This allows the tests to avoid
creating and cleaning up files and also allows the tests to run faster
since it can all happen in memory.
The chainSetup function has also been changed to provide logic to switch
on the database type to allow for easy changing of the backend to a
different database type as needed. For example, it could be useful to
provide extra testing against new database backends.
This commit adds a new backend driver which conforms to the btcdb
interface to provide a memory only database. This is primarily useful for
testing purposes as normal operations require a persistent block storage
mechanism.
Rather than defining CheckBlockSanity as a member of a BlockChain
instance, define it at the root level so it is truly context free as
intended. In order to make it context free, the proof of work limit is
now a required parameter.
This change reworks where the command parsing occurs to be done before
handlers are checked. Before, the websocket extension handler called
the standard handler with the same message, and if it was unhandled,
would unmarshal it a second time for checking extension handlers.
This commit adds tests to ensure the new "fast" paths in readElement and
writeElement work properly including proper fallback to the slower
reflection based read/write of the binary package.
This commit modifies the writeElement function to have a "fast path" which uses type
assertions for all of the types which btcwire write so the more expensive
reflection-based binary.Write can be avoided.
Also, this changes all cases that were writing raw ShaHash (32-byte) arrays (which
requires a stack copy) instead simply passing the pointer.
The following benchmark results show the results for serializing a block header
after these changes:
Before: BenchmarkWriteBlockHeader 500000 5566 ns/op
After: BenchmarkWriteBlockHeader 1000000 991 ns/op
This is part of the ongoing effort to optimize serialization as noted in
conformal/btcd#27.
The following benchmark results show the results for deserializing a block header:
Before: BenchmarkReadBlockHeader 500000 5916 ns/op
After: BenchmarkReadBlockHeader 1000000 2078 ns/op
This is part of the ongoing effort to optimize serialization as noted in
conformal/btcd#27.
The benchmark results for the current commit:
Before: BenchmarkDeserializeTx 500000 4018 ns/op
After: BenchmarkDeserializeTx 500000 3780 ns/op
The cumulative benchmark results since commit b7b700fd5a:
Before: BenchmarkDeserializeTx 200000 10665 ns/op
After: BenchmarkDeserializeTx 500000 3780 ns/op
This is part of the ongoing effort to optimize serialization as noted in
conformal/btcd#27.
The benchmark results for the current commit:
Before: BenchmarkSerializeTx 1000000 1233 ns/op
After: BenchmarkSerializeTx 1000000 1083 ns/op
The cumulative benchmark results since commit b7b700fd5a:
Before: BenchmarkSerializeTx 1000000 5437 ns/op
After: BenchmarkSerializeTx 1000000 1083 ns/op
This is part of the ongoing effort to optimize serialization as noted in
conformal/btcd#27.
Before: BenchmarkWriteTxIn 5000000 422 ns/op
After: BenchmarkWriteTxIn 5000000 389 ns/op
This is part of the ongoing effort to optimize serialization as noted in
conformal/btcd#27.
Before: BenchmarkReadTxIn 1000000 2435 ns/op
After: BenchmarkReadTxIn 1000000 1427 ns/op
This is part of the ongoing effort to optimize serialization as noted in
conformal/btcd#27.
Before: BenchmarkReadTxOut 500000 4576 ns/op
After: BenchmarkReadTxOut 2000000 871 ns/op
This is part ef the ongoing effort to optimize serialization as noted in
conformal/btcd#27.
Before: BenchmarkWriteTxOut 500000 4050 ns/op
After: BenchmarkWriteTxOut 10000000 248 ns/op
This is part ef the ongoing effort to optimize serialization as noted in
conformal/btcd#27.
persistentpeers and outbound(nonpersistent) peers get their own lists,
so iterating over them can be much simpler (and quicker when you have
125 peer of which 8 are outbound).
Redo the datastructures we search so that we only do one lookup per txin and
txout instead of doing a loop per wallet connection.
Don't send spent data on tx notifications, this can be worked out in wallet and
it is expensiveish to calculate. However we DO check upon getting a notification
request if the output is already spent, and in which case we send an immediate
notification to force a rescan.
MinedTxNotfications are handled separately to the connected block messages
largely to enable this to scale rather better.
Tested by jrick (who found one bug i had introduced, thanks!)
Additionally (accidentally squashed in):
Add handlers for all known commands.
We have handlers for all wallet-requiring commands that will return a suitable
error.
Unimplemented commands temporarily return an error stating so.
Before: BenchmarkReadOutPoint 500000 2946 ns/op
After: BenchmarkReadOutPoint 5000000 582 ns/op
This is part ef the ongoing effort to optimize serialization as noted in
conformal/btcd#27.
Before: BenchmarkWriteOutPoint 500000 2664 ns/op
After: BenchmarkWriteOutPoint 10000000 151 ns/op
This is part ef the ongoing effort to optimize serialization as noted in
conformal/btcd#27.
Before:
BenchmarkWriteVarStr4 1000000 1114 ns/op
BenchmarkWriteVarStr10 1000000 1352 ns/op
After:
BenchmarkWriteVarStr4 5000000 291 ns/op
BenchmarkWriteVarStr10 10000000 248 ns/op
This is part ef the ongoing effort to optimize serialization as noted in
conformal/btcd#27.
Before:
BenchmarkReadVarStr4 1000000 1698 ns/op
BenchmarkReadVarStr10 1000000 1812 ns/op
After:
BenchmarkReadVarStr4 2000000 853 ns/op
BenchmarkReadVarStr10 5000000 712 ns/op
This is part ef the ongoing effort to optimize serialization as noted in
conformal/btcd#27.
This commit slightly optimizes the readVarInt function in the case of
multiple-byte variable length integers. It also reduces the amount of
memory garbage it generates.
Before:
BenchmarkReadVarInt1 5000000 386 ns/op
BenchmarkReadVarInt3 5000000 693 ns/op
BenchmarkReadVarInt5 2000000 793 ns/op
BenchmarkReadVarInt9 5000000 709 ns/op
After:
BenchmarkReadVarInt1 5000000 387 ns/op
BenchmarkReadVarInt3 5000000 471 ns/op
BenchmarkReadVarInt5 5000000 575 ns/op
BenchmarkReadVarInt9 5000000 473 ns/op
This is part ef the ongoing effort to optimize serialization as noted in
conformal/btcd#27.
This change allows map lookups using address hashes (which are
returned as []byte) instead of either copying the hash into an array,
or doing a bytes.Equal().
A stupid range over a map until the right key is found was also just
changed to a single map lookup.
This change pads serialized (big endian) pubkey numbers to a length of
32 bytes. Previously, because serialized pubkey numbers are read
MSB-first, if a number could be serialized in less than 31 bytes, the
deserialized number would be incorrect.
The btcutil code was recently changed to provide a new Tx type which
provides hash caching (among other things). As a result, the notion of
obtaining a transaction hashes via TxShas was deprecated as well. This
commit updates the tests and backend implementation code accordingly.
This adds to the initial rescan implementation, but switches it to
rescan based on a group of addresses, rather than just one. Due to
how expensive database lookups are during a rescan, wallets should
take advantage of this to rescan once for all needed addresses for all
accounts.
This commit changes the various cases that were serializing transactions
into a buffer and taking the length to use the new faster SerializeSize
API. It also completes a TODO since the serialized size of a transaction
output is now available.
This commit adds tests for the new SerializeSize functions for variable
length integers and transactions (and indirectly transaction inputs and
outputs).
This commit adds a new function named SerializeSize to the public API for
MsgTx, TxOut, and TxIn which can be used to determine how many bytes the
serialized data would take without having to actually serialize it.
The following benchmark shows the difference between using the new
function to get the serialize size for a typical transaction and
serializing into a temporary buffer and taking the length of it:
Bufffer: BenchmarkTxSerializeSizeBuffer 200000 7050 ns/op
New: BenchmarkTxSerializeSizeNew 100000000 18 ns/op
This is part of the ongoing effort to optimize serialization as noted in
conformal/btcd#27.
Most variable length integers are smaller numbers, so this commit reverses
the order of the if checks in the writeVarInt to assume smaller numbers
are more common.
This is part of the ongoing effort to optimize serialization as noted in
conformal/btcd#27
Fix move need to convert amount to btc from satoshi
Fix settxfee need to convert amount to btc from satoshi
Fix for optional arg off by one in move.
Fix for gettxout in optional args along with typo in error message.
And lots of new tests.
Rather than showing all errors from ProcessTransaction as an error, check
if the error is a TxRuleError meaning the transaction was rejected as
opposed to something actually going wrong and log it accordingly.
Looking up transactions from the database is an expensive operation.
This commit modifies the NotifyNewTxListener code to simply iterate the
transactions in the block instead of looking them up from the db.
Currently the wallet code needs a spent flag which ultimately shouldn't be
required. For now, the spent data is simply created on the fly which is
still significantly faster than doing database transaction lookups.
Closes#24.
Test stop, signrawtransaction, setaccount, sendtoaddress, and a few
others.
Fix off by one error in the optional arguments for sendtoaddress.
Rename occurances of Minconf to MinConf for constitancy.
This change unbreaks the case where an unknown command is sent to the
RPC server. Instead of replying back with a nil JSON id, if the
initial unmarshal was successful (and thus, the message was valid
JSON-RPC), the unmarshaled id will be used in the error reply.
This changes the behavior of ParseMarshaledCmd to always return a
non-nil Cmd whenever unmarshaling a valid JSON-RPC message succeeds.
This is needed for RPC server code to reply back with detailed errors
to clients using the Method and Id methods of the Cmd interface.
We have a channel for queries and commands in server, where we pass in
args and the channel to reply from, let rpcserver use these interfaces
to provide the requistie information.
So far not all of the informaation is 100% correct, the syncpeer
information needs to be fetched from blockmanager, the subversion isn't
recorded and the number of bytes sent and recieved needs to be obtained
from btcwire. The rest should be correct.
This commit updates btcd to work with the new btcchain APIs which now
accept btcutil.Tx instead of raw btcwire.MsgTx. It also modifies the
transaction memory pool to store btcutil.Tx.
This is part of the ongoing transaction hash optimization effort noted in
conformal/btcd#25.
This change updates the doc.go documentation file with the correct use
of the new (since 8eead5217d) API used
for passing additional options when creating new script engines.
Spotted by davec@
This commit optimizes InsertBlock slightly by using the cached transaction
hashes instead of recomputing them. Also, while here, use the
ShaHash.IsEqual function to do comparisons for consistency.
This is ongoing work towards conformal/btcd#25.
ok @drahn.
This commit provides a new --cpuprofile flag that can be used to specify a
file path to write CPU profile data into. The resulting profile can then be
consumed by the 'go tool pprof' command.
Since the main SIGINT handler is running as a goroutine, the main
goroutine must be kept active long enough for it to finish or it will be
nuked when the main goroutine exits. This commit makes that happen by
slightly modifying how it waits for shutdown.
Rather than having the main goroutine only wait for the server to
shutdown, there is now a shutdown channel that is used to signal the main
goroutine to shutdown for all cases such as a graceful shutdown, a
scheduled shutdown, an RPC stop, or a SIGINT.
While here, also add a few prints to indicate a SIGINT was received and
the shutdown progress.
Profiling showed the duplicate transaction input check was taking around
6% of the total CheckTransactionSanity processing time. This was largely
due to using fmt.Sprintf to generate the map key.
This commit modifies the check instead to use the actual output as a map
key.
The following benchmark results show the difference:
Before: BenchmarkOldDuplicatInputCheck 100000 21787 ns/op
After: BenchmarkNewDuplicatInputCheck 2000000 937 ns/op
Closes#2
Profiling showed the MRU inventory handling was taking 5% of the total
block handling time. Upon inspection this is because the original
implementation was extremely inefficient using iteration to find the
oldest node for eviction.
This commit reworks it to use a map and list in order to achieve close to
O(1) performance for lookups, insertion, and eviction.
The following benchmark results show the difference:
Before: BenchmarkMruInventoryList 100 1069168 ns/op
After: BenchmarkMruInventoryList 10000000 152 ns/op
Closes#21
The ValidateTransactionScripts was recently changed to accept script flags
which pass through to the script engine in order to control its validation
behavior. This commit modifies the transaction memory pool script
validation code for this change and additionally adds the new flag to
perform canonical signtaure checking.
This removes the bip16 bool from NewScript and adds it to flags (with
the constant ScriptBip16), and also adds a new flag,
ScriptCanonicalSignatures, which will call btcec.ParseDERSignature
parsing a signature during an Execute. This is needed to emulate
bitcoind behavior, as bitcoind performs canonical signature validation
(DER format) in some places (like mempool acceptance) but not others
(like block validation).
Updated tests and test coverage file to reflect changes.
This change adds an additional signature parsing function which
performs additional checks to verify the signature is serialized in a
valid DER (and thus, unique) format, instead of allowing the less
strict BER signatures that ParseSignature will happily accept.
Added additional tests and updated test coverage to reflect changes.
Initial prototype for one command mostly from jrick, with discussion
with me and jcv. Other commands mocked up in a couple of long boring days by
yours truly.
Code isn't fully tested yet so may well still be buggy in the type conversions.
Test will be added over the next few days to get this up to snuff.
This commit modifies the ValidateTransactionScripts API to accept the
recently added ScriptFlags from btcscript. This provides flexibility to
the caller to choose validation behavior based on those new flags.
Several of the bitcoin data structures contain variable length entries,
many of which have well-defined maximum limits. However, there are still
a few cases, such as variable length strings and number of transactions
which don't have clearly defined maximum limits. Instead they are only
limited by the maximum size of a message.
In order to efficiently decode messages, space is pre-allocated for the
slices which hold these variable length pieces as to avoid needing to
dynamically grow the backing arrays. Due to this however, it was
previously possible to claim extremely high slice lengths which exceed
available memory (or maximum allowed slice lengths).
This commit imposes limits to all of these cases based on calculating
the maximum possible number of elements that could fit into a message
and using those as sane upper limits.
The variable length string case was found (and tests added to hit it) by
drahn@ which prompted an audit to find all cases.
This change allows btcwallet to keep a pool of transactions that have
not yet been mined into a block, notifying wallet when transactions
are mined, as well as introducing a new way to send the
btcd:blockconnected notification with wallet-specific information as
part of the same notification. When a transaction is sent using the
RPC call 'sendrawtransaction', a notification request will be
automatically registered with the connected wallet (if using
websockets) to notify the wallet when the transaction first appears in
a block.
To perform this notification, and to avoid requiring wallets from
waiting for seperate mined tx notifications (and resend after a
timeout) or from sending an additional tx mined request for every tx
in the pool after each new block, the blockconnected notification is
now created seperately for each wallet. If the notified wallet has
sent a transaction, an additional JSON field "minedtxs" will include
an array of transaction IDs that the wallet has created and which are
included in the new block.
This new unique blockconnected notification can also be used for
additional notifications that may happen each new block in the future,
and to cut down on existing notification handlers in btcwallet, such
as for transactions to a watched address.
Rather than relying on the http package's DefaultServeMux for the RPC
server, create a unique mux specifically for the RPC server. This ensures
things, such as the http profiling handlers, do not commingle.
Define errors (previously done in btcwallet and btcd's rpcserver) in
one place so they can just be called by name rather than generated in
code using btcjson.
Closes#5
This commit modifies the main processing loop for orphan dependencies
(orphans that are processed due to their parents showing up) to use an
index based for loop over range. Since the Go range statement does not
reevaluate on every iteration, it was previously possible under certain
circumstances for the slice to be changed out from under the range
statement while processing the orphan blocks.
This commit adds the infrastructure needed to re-run the integrity tests
after calling the InvalidBlockCache, InvalidateTxCache, and
InvalidateCache functions. The tests intentially randomize the order the
invalidate functions are called in to increase the likelyhood of finding
in issues realted to ordering of the calls.
This commit changes the interface test errors to all report the block
height and hash prior to the failure. Test failures that transactions
also now report the transaction number and hash. The makes it much
easier to track down where a test failure occurs.
The idea here is that interface_test.go will be directly usable in each
implementation to increase test coverage there as well, but also works at
the top-most level to test arbitrary backends.
This is simply at start at providing generic interface tests. The only
thing is tests so far is the empty database conditions on NewestSha, but
it adds infrastructure for creating and opening databases with special
type handling based on the database and necessary logic to teardown so
multiple backends can be tested simultaneously.
This and the next series of commits all discussed with drahn@.
This also adds tests to verify that the created error string follows
the expected format of "code: message" and tests to verify that Error
can be used as an error. Additionally, the idiom
var _, _ error = Error{}, &Error{}
was added so the build will fail if the builtin error interface is
never satisified in the future.
Rather than fetching the hash of each block individually 2k+ times, make
use of the FetchHeightRange function so all of the most recent hashes can
be fetched at once.
This commit modifies the code to make use of the new btcd APIs that allow
fetching of transaction lists which either do or do not include fully
spent transactions. It is more efficient to avoid fetching fully spent
transactions from the database when they aren't needed.
This commit modifies the errors that result from missing expected input
transactions to a RuleError. This allows the caller to detect a block was
rejected due to a rule violation as opposed to an unexpected error.
This commit adds a quick check to the transaction store fetch code which
simply returns an empty store if no hashes were requested rather than
bothering the db with an empty list.
This commit modifies the transaction lookup code to use a set instead of a
slice (list). This allows the lookup to automatically prevent duplicate
requests to the database.
Previously, the code simply added every referenced transaction to a list
without checking for duplicates, which led to multiple requests against
the database for the same transaction. It also meant the request list
could grow quite large with all of the duplicates using far more memory
than required.
While the end result was accurate, operating that way is not as efficient
as only requesting unique transactions.
This commit implents a basic infrastructure to be used throughout the
tests for creating a new chain instance that is ready to have tests run
against it. It also returns a teardown function the caller can use to
clean up after it is done testing. This paves the way for adding more
tests.
The stack data is normally sliced from the actual script and btcscript is not
supposed to ever change the tx passed into it.
Add a test (and fix the other leading zeros tests) to stop this happening again.
This commit changes the InvVect_* constants, which are not standard Go
style, to the InvType*. In order to preserve backwards compatibility, it
also adds a legacy.go file which maps the old public constant names to the
new ones.
Closes#1.
The original thought was that chain would also house the transaction
memory pool, but that ultimately was decided against. As a result,
it only makes sense to query chain for blocks rather than generic
inventory.
So add entries for them that disassemble and parse ok, but will fail
when executed with the appropriate error. Add a full suite of tests to confirm
that this happens.
Found by a strange transaction in testnet.
It is technically possible for the Read method on a reader to return zero
bytes read with a nil error even though that behavior is "discouraged" by
the interface documenation. This commit switches the read of the first
byte to use io.ReadFull which will always error in this case.
This commit corrects the reading of the serialized height in coinbase
transactions for block height of version 2 or greater. On mainnet, the
serialized height is always 3 bytes and will continue to be so for
something like another ~159 years, so there was no issue with mainnet.
However on testnet, there are some version 2 blocks which are low enough
in the chain to only take 2 bytes to serialize.
In addition, this commit adds a full tests for the relavant function
including negative tests and variable length serialized lengths for block
heights.
Closes#1.
btcdb was changed a while back to not insert the genesis block by default.
This commit modifies the reorg test to insert it as required so not all
blocks are orphans.
FetchTxBySha changes what it returns, it can now return a TxListReply and
and error if none are found.
FetchTxByShaList is renamed to FetchUnSpentTxByShaList to indicate that
it will (likey/eventually) only return Tx that have some unspent TxOuts.
Tx which are fully spent may not be (reliably) looked up using this API.
Rather than using a channel for notifictions, use a callback instead.
There are several issues that arise with async notifications via a
channel, so use a callback instead. The caller can always make the
notifications async by using channels in the provided callback if
needed.
This commit modifies CheckInputTransactions to ensure that not only must a
transaction exist in the transaction store, it must also not have any
errors associated with it or be nil.
PayToPubKeyHashScript generates a new pay to pubkey hash script to use
as the pkScript when creating new transactions. If the passed pubkey
hash is an invalid size, StackErrInvalidOpcode will be returned as an
error.
SignatureScript returns the signature script necessary to validate a
single input of a transaction.
This also adds sanity checking for serializing scripts into byte
slices. If the length of a serialized opcode does not equal the
expected length, StackErrInvalidOpcode will be returned when unparsing
a []parsedOpcode.
New internal tests were added to verify checks for valid and invalid
parsed opcodes.
This commit adds an exported function, IsPushOnlyScript, which can be used
to tell whether or not a script only pushes data (only contains opcodes
that push data).
Several of the functions require a map of contextual transaction data to
use as a source for referenced transactions. This commit exports the
underlying TxData type and creates a new type TxStore, which is a map of
points to the under TxData. In addition, this commit exposes a new
function, FetchTransactionStore, which returns a transaction store
(TxStore) containing all of the transactions referenced by the passed
transaction, as well as the existing transaction if it already exists.
This paves the way for subsequent commits which will expose some of the
functions which depend on this transaction store.
This commit modifies the code to choose sane defaults for the backing
arrays for slices that involve a lot of appends such block locators, hash
processing, and needed transactions. This is an optimization to avoid
the overhead of growing the backing arrays and copying the data multiple
times in the most common case. This also prevents a leak in Go GC which
will likely ultimatley be fixed, but the efficiecy gains alone are worth
the change.
Removing from the bottom of a stack (nipN(depth)) would leak the first
entry in the array by slicing [1:], leaving array[0] dangling an
inaccessible (but unable to be freed until the whole slice is gone). We
left it like this for a while, but best not to leak the memory. Happens
rarely so the performance hit shouldn't matter that much. Do the same
thing for condstack.
Several of the messages store the parts that have a variable number of
elements as slices. This commit modifies the code to choose sane defaults
for the backing arrays for the slices so when the entries are actually
appended, a lot of the overhead of growing the backing arrays and copying
the data multiple times is avoided.
Along the same lines, when decoding messages, the actual size is known and
now is pre-allocated instead of dynamically growing the backing array
thereby avoiding some overhead.
When we do append loops, make an educated guess as to the size and make an array
with that capacity to avoid extra copying.
Doesn't affect the speed of the tests, over 4 runs the difference was lost in
the noise.
After discussion with others and thinking about the notification channel
some more, we've decided to leave it up to the caller to quickly handle
notifications. While it is true that notification should be handled
quickly to not block the chain processing code unnecessarily, launching a
goroutine in chain means the notifications are no longer necessarily in
order. Also, if the caller is not properly handling the notifications,
the goroutines end up sicking around forever. By leaving it up to the
caller to quickly handle the notification or launch a goroutine as
necessary for the caller, it provides the flexibility to ensure proper
notification ordering as well as control over other things such as
how to handle backpressure.
improved error handling
Fix testing, no point in running tests multiple times
Fix error when no blocks in database, reopen would misreport NewstSha() return
Rather than sending the root of the an orphan chain when sending an orphan
notification, send the hash of the orphan block itself. The caller can
then call the GetOrphanRoot function with the hash to get the root of the
orphan chain as needed. This is being changed since it's cleaner and more
detministic sending the hash for the orphan block that was just processed
rather than sending a possibly different hash depending on whether there
is an orphan chain or not.
Since the notification channel is provided by the caller and it may or may
not be buffered, send notifications in their own goroutine so the chain
processing code does not have to wait around on the caller to process the
notification before continuing.
Rather than removing disconnected transactions from the node viewpoint
transaction store, clear the entry instead. This is needed for the
connect code to update transactions that were removed on the other side of
a chain fork.
This commit adds a new type, BlockLocator, and supporting infrastructure
to support generating block locators from a provided hash and getting
a full block locator for the latest known block. This will likely be
expanded in the future to provide the opposite functionality as well.
That is to say the ability to lookup a block using a block locator.
This commit modifies the GenerateInitialIndex function to update the best
chain with each node as it is loaded. This allows the best chain to be
set immediately upon generating the initial index which is a slight
performance optimization.
This commit provides a new exported function, HaveInventory, which can
be used to determine if the already has the item referenced by passed
inventory vector. Currently it only provides support for blocks and
cursory support for transactions in the main chain. Types that are
unrecognized are returned as if they unknown as expected.
This commit provides a new exported function, IsKnownOrphan, which can
be used to determine if the passed hash is currently already known to the
chain as an orphan.
This commit exposes GetOrphanRoot to callers. It also modifies the code
to allow the function to be called in a concurrent safe manner. This
allows the information to be safely obtained from multiple goroutines.
This code is still prototype at this time.
It appears to function correctly but something
consumes more memory than is considered reasonable
for the dataset comprised of the full bitcoind chain.
Not recommened for use at this time.
It is not safe to remove elements from a slice while iterating them with
the range statement since it does not reevaluate the slice on each
iteration nor does it adjust the index for the modified slice. This
commit modifies the code to use a for loop with an index (which does
reevaluate on every iteration) and manually adjusts the index when
elements are removed from the slice.
Add new RpcRawCommand to get []byte represtation of reply instead of
btcjson.Result.
Export ReadResultCommand to allow caller of RpcRawCommand to deal with
the reply in the same way RpcCommand does.
Adjust test code to for the above changes.
Add return types for serveral additional commands.
Fix required vs. optional arguments for createrawtransaction.
Update tests and test coverage for all changes.
This function is a convenience method to create a new NetAddress
from a net.IP and uint16 port as opposed to a net.Addr which must be of
type *net.TCPAddr. This allows callers to support connection types that
don't provide access to a concrete *net.TCPAddr implementation.
This commit updates the calls into btcutil, btcscript, and btcwire for the
latest API changes which remove the need for the protocol version for
serialization and deserialization of blocks and transactions.
This commit updates the calls into btcutil and btcwire for the latest API
changes which remove the need for the protocol version for serialization
and deserialization of blocks and transactions.
This commit modifies the code to no longer require a protocol version. It
does this by making use of the new Serialize function in btcwire.
Unfortuantely this does entail a public API change which I generally don't
like to do, but eliminating the usage of the protocol version throughout
the codebase was important enough to warrant the change.
Both of these depend on the serialized bytes which are dependent on the
version field in the block/transaction. They must be independent of the
protocol version so there is no need to require it.
This commit introduces two new functions for MsgBlock and MsgTx named
Serialize and Deserialize. The functions provide a stable mechanism for
serializing and deserializing blocks and transactions to and from disk
without having to worry about the protocol version. Instead these
functions use the Version fields in the blocks and transactions.
These new functions differ from BtcEncode and BtcDecode in that the latter
functions are intended to encode/decode blocks and transaction from the
wire which technically can differ depending on the protocol version and
don't even really need to use the same format as the stored data.
Currently, there is no difference between the two, and due to how
intertwined they are in the reference implementaiton, they may not ever
diverge, but there is a difference and the goal for btcwire is to provide
a stable API that is flexible enough to deal with encoding changes.
Fix error returns in InsertBlock and FetchBlockBySha
Give up on return by name in InsertBlock() and return explicit
err one location in FetchBlockBySha to return proper error value
This commit adds a new optional function named GenerateInitialIndex. It
generates the required number of initial block nodes in an optimized
fashion. Since the memory block index is sparse and previous nodes are
dynamically loaded as needed, this function is not stricty needed.
However, during initial startup (when there are no nodes in memory yet),
dynamically loading all of the required nodes on the fly in the usual way
is much slower than preloading them.
The recent pruning code made the parent hash for a node available directly
as a field in the node. Make use of this in the getPrevNodeFromNode
function to avoid having to load the full block data associated with the
node to get the parent hash.
This commit implements pruning for the block nodes that form the memory
chain which are no longer needed. The validation of a block only requires
information from a set number of previous nodes. The initial code did not
prune old nodes as they were no longer needed, but the vast majority of
the infrastructure to support it was already in place as that was always
the goal. This commit finishes implementing the missing bits to make it
a reality.
Previously, the code was using big rational numbers for work values which
resulted in carrying way too much precision around (and ultimately a lot
of extra memory and computation to carry that precision). This commit
converts the work values to big integers and calculates them with integer
division. This is acceptable because the numerator is multiplied by 2^256
which is higher than the maximum possible proof of work. Therefore
anything after the decimal is superfluous precision for the purposes of
chain selection.
Also, add a check for negative difficulty values when calculating the work
value. Negative values won't occur in practice with valid blocks, but
it's possible an invalid block could trigger the code path, so be safe and
check for it.
Previously the main network checkpoints were being used for unrecognized
networks. This commit changes the code so that no checkpoints are used in
that scenario.
Rather than converting the proof of work limit to its compact
representation multiple times during operation, do it once at package
initialization time and export it via the chain parameters.
This commit adds info level log statements when a block causes a chain
fork, extends a side chain (fork), or causes a reorganize. When a reorg
happens, the new and old chain heads along with the fork point are logged.
Even though the code currently makes heavy use of nodes which will have
the appropriate height, blocks which did not come from the database will
not have a height set. As a result, this could possibly result in a
height of 0 being used when unintended. By setting the block height in
the block as soon as its known (regardless of the source), we can future
proof against bugs that would likely be hard to track down.
The btcscript API for GetSigOpCount and GetPreciseSigOpCount
recently changed to no longer return an error. This change was necessary
to mirror the behavior of bitcoind signature operations counting. This
commit modifies the code accordingly to use the new API.
We do this by allowing the parser to return the list of parsed objects
(it is an internal only api anyway) and then we use this in the sigops
counting and ignore the error. This change due to bitcoind
compatability.
It is acceptable for a transaction input in a block to reference the
output of another transaction in the same block only if the referenced
transaction comes before the transaction referencing it.
This commit modifies the way initial database creation is handled.
Previously, the genesis for the main chain was inserted automatically upon
creation of the database. However, that approach caused an issue since
other networks such as the test network don't use the same genesis block
as the main network.
The new approach introduced by this commit is to leave it up to the caller
to insert the desired genesis block. In order to support this, the
InsertBlock function has been modified to allow the first (and only the
first) block to be inserted without having an existing parent. Also, the
NewestSha function has been modified to return a zero hash, -1 for the
height, and no error when the database does not yet have any blocks. This
allows the caller to determine the difference between no blocks and only
the genesis block (in which case the return values would be the genesis
hash and 0 for the height).
The go vet command complains about untagged struct initializers when
defining a ShaHash directly. This seems to be a limitation where go vet
does not exclude the warning for types which are a constant size byte array
like it does for normal constant size byte array definition.
This commit simply modifies the tests to use a constant definition cast to a
ShaHash to overcome the limitation of go vet.
This commit modifies the double spend detection to handle double spends
within the same block as well as side chains when doing the checks before
reorganizing the chain.
There was not much documentation about the difference between testnet and
testnet3, so make it clear that testnet is used for regression tests and
testnet3 is the public test network (version 3).
In addition to returning errors to the caller, log the error with a prefix
in a few key places that helps identify the origin for errors. In some
cases, the underlying error comes from a different subsystem such as the
SQL database driver and the error messages can be fairly generic.
This commit modifies the code to use params based on the active network
which paves the way for supporting the special rules and different genesis
blocks used by the test networks.
Since the same coinbase transaction is used for the genesis blocks of
all three currently supposed networks, separate it into its own var and
use a reference to it in each of the genesis block defintions.
After discussing the criteria used by the core developers on #btc-dev IRC
channel, gmaxwell indicated they like to see at least 2016 blocks. This
commit updates the checkpoint confirmations accordingly.
Following the example of bitcoind and the bitcoin wiki, provide a
function to convert floats (which are returned by the json-rpc server)
to int64 (which is the normal representation for most values involving
bitcoins).
btcscript contains an amount of debug logging that is very useful to
have. We have agreed that testing this isn't realy practical or indeed
really useful and thus is rather unlikely to grow test coverage any time
soon.
Turns out that there are some signatures in the bitcoin blockchain that have
trailing 0s, for example
12a1b29fd6c295075b6a66f5fd90f0126ceb1fda4f15e4b44d92518bd52a5cdf has a signature
length of 0x45 where there are 0x47 bytes following that length check (one is
hashtype and is supposed to be trimmed out prior to calling the function). We
relax the paranoid length check to permit traling data, but not to permit
buffers that are too short. Change the test to passing with a big comment
stating why this is now considered a valid case.
To be usd for validation. Most of the codepaths teste, a few tests
missing for cases needed tests in the validation codepaths too. To be
worked on in tree.
The only time we need to zero out scripts is for calcScriptHash which operates
on a deep copy anyway. This should make the tx passed to us unmodified now.
Although you can technically get at this value via the MaxPayloadLength
function on a block, it is less overhead for any consumers that need to
know the value to simply export it directly.
We were counting the number of push ops instead of the number of non
push ops. Add tests that found this (checking tha the max operations
check fires).
Use this to test the pubkey paths in checksig which return btcec errors
which we don't define. all of the other active tests know the return
code we need.
Use it to reduce code dpulication in the bip16 case.
In addition we export it so that that a user could run:
for !done && err == nil {
done, err = s.Step()
}
err = s.CheckErrorCondition()
manually instead of having to run Execute().
Since we already have the specific username and password we want to strip
from errors, use a specific search string rather than a generic regular
expression. This is quite a bit more efficient than using regular
expressions and also has the benefit of being more accurate.
Also, rather than using the added overhead of fmt to convert the error to
a string, just call Error() directly on it to get the string.
Finally, instead of just stripping it, replace it with the literal
string "<username>:<password>" to avoid any possible confusion in the
error messages where it might otherwise appear the url was being
constructed incorrectly.
ok jcv@
This commit updates the usage example as follows:
- Add a defer db.Close since the database should be closed after the
caller is done with it
- Correct the import path for the btcdb/sqlite3 package
- Add a db extension to the example database name
- Make the error handling and comments match the standard style
This commit attempts to clarify which functions in the Db interface may be
returning cached data that the InvalidateCache, InvalidateBlockCache, and
InvalidateTxCache functions are used to clear.
The Db interface is intended to work with block heights as opposed to
specific database ids which may or may not be the same as the block
height. This commits changes the function names to make that distinction
a little more clear.
This commit makes several changes to the operation tests as follows:
- Ensure the database is closed between each operational mode test as
otherwise the database is locked when trying to reopening it which
makes it unusable
- Make the insert tests error after a single failure since every test
afterwards is guaranteed to fail as well
- Remove some of the logging statements which cluttered up the failed test
prints making it harder to see why a given test failed
- Make a couple of the log messages more descriptive
The function is being added back in since the tests rely on it, but it is
marked as DEPRECATED since it really should not be a part of the public
generic db interface.
This commit changes MsgBlock to enforce a 1MB max payload per the spec.
Previously it was only limited to the max overall message size. While
here, also enforce max payloads per message type (instead of only the max
overall message payload) when writing messages.
This commit makes the examples a little more clear by providing complete
examples of the two different types of replies. These changes were
prompted by PaulCapes on IRC who pointed out the previous example was not
very clear.
ok jcv@
The functions for generating transaction and block hashes contained a few
error checks for conditions which could never fail without run-time
panics. This commit removes those superfluous checks and adds explanatory
comments.
The io.Writer.Write function always returns an error if the bytes written
is less than provided, so there is no reason to further check if the
payload length matches after a successful write.
This commit adds tests for the error paths when encoded and decoding
MsgHeaders. Also, while here modify the casing of the local vars to be
consistent.
This commit corrects the MsgGetBlocks tests so that the protocol version
in the multiLocators data matches the wire encoded test data. This will
help future proof the tests against protocol changes. Also, while here
modify the casing of the local vars to be consistent.
This commit corrects the tests so that the main API functions are tested
against the latest protocol version, but the TxSha and BlockSha functions
are run against the specific protocol version used to encode the test
data. This will help future proof the tests against protocol changes.
This commit adds a new MessageError type that is intended to allow the
caller to differentiate between general io errors and and errors that
resulted from malformed messages.
The io.ReadFull function always returns an error if the full payload can't
be read, so there is no reason to further check if the payload length
matches after a successful read.
This project is currently under active development and is in an Alpha state.
Kaspad is the reference full node Kaspa implementation written in Go (golang).
It currently properly downloads, validates, and serves the block chain using the
exact rules (including bugs) for block acceptance as the reference
implementation (bitcoind). We have taken great care to avoid btcd causing a
fork to the block chain. It passes all of the 'official' block acceptance tests
(https://github.com/TheBlueMatt/test-scripts).
This project is currently under active development and is in Beta state.
It also properly relays newly mined blocks, maintains a transaction pool,
and relays individual transactions that have not yet made it into a block. It
ensures all individual transactions admitted to the pool follow the rules
required into the block chain and also includes the vast majority of the more
strict checks which filter transactions based on miner requirements ("standard"
transactions).
## What is kaspa
One key difference between btcd and bitcoind is that btcd does *NOT* include
wallet functionality and this was a very intentional design decision. See the
blog entry [here](https://blog.conformal.com/btcd-not-your-moms-bitcoin-daemon)
for more details. This means you can't actually make or receive payments
directly with btcd. That functionality will be provided by the forthcoming
btcwallet and btcgui.
Kaspa is an attempt at a proof-of-work cryptocurrency with instant confirmations and sub-second block times. It is based on [the PHANTOM protocol](https://eprint.iacr.org/2018/104.pdf), a generalization of Nakamoto consensus.
## Requirements
Go 1.16 or later.
## Installation
#### Windows - MSI Available
https://github.com/conformal/btcd/releases
#### Linux/BSD/POSIX - Build from Source
#### Build from Source
- Install Go according to the installation instructions here:
http://golang.org/doc/install
-Run the following command to obtain btcd, all dependencies, and install it:
```$ go get github.com/conformal/btcd```
-Ensure Go was installed properly and is a supported version:
- btcd will now be installed in either ```$GOROOT/bin``` or ```$GOPATH/bin```
depending on your configuration. If you did not already add to your system
path during the installation, we recommend you do so now.
```bash
$ go version
```
## Updating
- Run the following commands to obtain and install kaspad including all dependencies:
#### Windows
```bash
$ git clone https://github.com/kaspanet/kaspad
$ cd kaspad
$ go install . ./cmd/...
```
Install a newer MSI
- Kaspad (and utilities) should now be installed in `$(go env GOPATH)/bin`. If you did
not already add the bin directory to your system path during Go installation,
you are encouraged to do so now.
#### Linux/BSD/POSIX - Build from Source
- Run the following command to update btcd, all dependencies, and install it:
```$ go get -u -v github.com/conformal/btcd/...```
## Getting Started
btcd has several configuration options avilable to tweak how it runs, but all
of the basic operations described in the intro section work with zero
configuration.
#### Windows (Installed from MSI)
Launch btcd from your Start menu.
#### Linux/BSD/POSIX/Source
Kaspad has several configuration options available to tweak how it runs, but all
of the basic operations work with zero configuration.
```bash
$ ./btcd
````
## Mailing lists
$ kaspad
```
- btcd: discussion of btcd and its packages.
- btcd-commits: readonly mail-out of source code changes.
## Discord
Join our discord server using the following link: https://discord.gg/YNYnNN5Pf2
To subscribe to a given list, send email to list+subscribe@opensource.conformal.com
## Issue Tracker
## TODO
The [integrated github issue tracker](https://github.com/kaspanet/kaspad/issues)
is used for this project.
The following is a brief overview of the next things we have planned to work on
for btcd. Note this does not include the separate btcwallet and btcgui which
are currently under heavy development:
Issue priorities may be seen at https://github.com/orgs/kaspanet/projects/4
- Documentation
- Code cleanup
- Add remaining missing RPC calls
- Add option to allow btcd run as a daemon/service
- Complete several TODO items in the code
- Offer 32-bit MSI as well as the 64-bit one
- Offer cross-compiled binaries for popular OSes (Fedora, Ubuntu, FreeBSD, OpenBSD)
## Documentation
## GPG Verification Key
All official release tags are signed by Conformal so users can ensure the code
has not been tampered with and is coming from Conformal. To verify the
signature perform the following:
- Download the public key from the Conformal website at
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.