149 Commits

Author SHA1 Message Date
Ori Newman
7d44275eb1
Add found to GetBlock (#2165) 2022-11-16 23:48:05 +02:00
Ori Newman
cf4073b773
Remove HF activation code (#2152)
* Remove HF activation code

* Remove unused var totalInputs
2022-10-02 19:17:33 +03:00
Ori Newman
7e9b5b9010
Security Patch + HF (#2142)
* HF

* Fix lint
2022-09-21 18:58:32 +03:00
Michael Sutton
1c9bb54cc2
Crucial fix for the UTXO difference mechanism (#2114)
* Illustrate the bug through prints

* Change consensus API to a single ResolveVirtual call

* nil changeset is not expected when err=nil

* Fixes a deep bug in the resolve virtual process

* Be more defensive at resolving virtual when adding a block

* When finally resolved, set virtual parents properly

* Return nil changeset when nothing happened

* Make sure the block at the split point is reversed to new chain as well

* bump to version 0.12.4

* Avoid locking consensus twice in the common case of adding block with updateVirtual=true

* check err

* Parents must be picked first before set as virtual parents

* Keep the flag for tracking virtual state, since tip sorting perf is high with many tips

* Improve and clarify resolve virtual tests

* Addressing minor review comments

* Fixed a bug in the reported virtual changeset, and modified the test to verify it

* Addressing review comments
2022-07-15 12:35:20 +03:00
Michael Sutton
d957a6d93a
Fix UTXO diff child error (#2084)
* Avoid creating the chain iterator if high hash is actually low hash

* Always use iterator in nextPruningPointAndCandidateByBlockHash

* Initial failing test

* Minimal failing test + some comments

* go lint

* Add simpler tests with two different errors

* Missed some error checks

* Minor

* A workaround patch for preventing the missing utxo child diff bug

* Make sure we fully resolve virtual

* Move ResolveVirtualWithMaxParam to test consensus

* Mark virtual not updated and loop in batches

* Refactor: remove VirtualChangeSet from functions return values

* Remove workaround comments

* If block has no body, virtual is still considered updated

* Remove special error ErrReverseUTXODiffsUTXODiffChildNotFound

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-06-15 02:52:14 +03:00
Ori Newman
c839337425
Add finality check to ResolveVirtual (#2041)
* Add finality check to ResolveVirtual

* Warn on finality conflict only if needed
2022-05-06 00:29:48 +03:00
Ori Newman
35a959b56f
Making a workaround for the UTXO diff child bug (#2020)
* Making a workaround for the UTXO diff child bug

* Fix error message

* Fix error message
2022-04-14 12:55:20 +03:00
Ori Newman
2b395e34b1
Use separate depth than finality depth for merge set calculations after HF (#2013)
* Use separate than finality depth for merge set calculations after HF

* Add comments and edit error messages

* Fix TestValidateTransactionInContextAndPopulateFee

* Don't disconnect from node if isViolatingBoundedMergeDepth

* Use new merge root for virtual pick parents; apply HF1 daa score split for validation only

* Use `blue work` heuristic to skip irrelevant relay blocks

* Minor

* Make sure virtual's merge depth root is a real block

* For ghostdag data we always use the non-trusted data

* Fix TestBoundedMergeDepth and in IBD use VirtualMergeDepthRoot instead of MergeDepthRoot

* Update HF1DAAScore

* Make sure merge root and finality are called + avoid calculating virtual root twice

* Update block version to 1 after HF

* Update to v0.12.0

Co-authored-by: msutton <mikisiton2@gmail.com>
2022-04-12 00:26:44 +03:00
Michael Sutton
80879cabe1
Clean up debug log level by moving many frequent logs to trace level (#2004)
* Clean up debug log level and move most/frequent logs to trace level

* Change function call traces to trace log level
2022-04-03 23:25:29 +03:00
Michael Sutton
21b82d7efc
Block template cache (#1994)
* minor text fix

* Implement a block template cache with template modification/reuse mechanism

* Fix compilation error

* Address review comments

* Added a through TestModifyBlockTemplate test

* Update header timestamp if possible

* Avoid copying the transactions when only the header changed

* go fmt
2022-03-31 16:37:48 +03:00
Ori Newman
061e65be93
Fix argument order for IsAncestorOf in boundedMergeBreakingParents (#1966) 2022-03-09 21:11:00 +02:00
Ori Newman
df573bba63
Remove unused args from CalcSubsidy (#1877) 2021-12-11 20:16:23 +02:00
Ori Newman
32e8e539ac
Apply ResolveVirtual diffs to the UTXO index (#1868)
* Apply ResolveVirtual diffs to the UTXO index

* Add comments

Co-authored-by: Ori Newman <>
2021-12-05 13:22:48 +02:00
Ori Newman
11103a36d3
Get rid of genesis's UTXO dump (#1867)
* Get rid of genesis's UTXO dump

* Allow known orphans when AllowSubmitBlockWhenNotSynced=true

* gofmt

* Avoid IBD without changing the pruning point when only genesis is available

* Add DisallowDirectBlocksOnTopOfGenesis=true for mainnet

* Remove any mention to nobanning to let stability tests run

* Rename ifGenesisSetUtxoSet to loadUTXODataForGenesis

Co-authored-by: Ori Newman <>
2021-12-05 12:46:41 +02:00
Elichai Turkel
dbf18d8052
Hard fork - new genesis with the utxo set of the last block (#1856)
* UTXO dump of block 0fca37ca667c2d550a6c4416dad9717e50927128c424fa4edbebc436ab13aeef

* Activate HF immediately and change reward to 1000

* Change protocol version and datadir location

* Delete comments

* Fix zero hash to muhash zero hash in genesis utxo dump check

* Don't omit genesis as direct parent

* Fix tests

* Change subsidy to 500

* Dont assume genesis multiset is empty

* Fix BlockReward test

* Fix TestValidateAndInsertImportedPruningPoint test

* Fix pruning point genesis utxo set

* Fix tests related to mainnet utxo set

* Dont change the difficulty before you have a full window

* Fix TestBlockWindow tests

* Remove global utxo set variable, and persist mainnetnet utxo deserialization between runs

* Fix last tests

* Make peer banning opt-in

* small fix for a test

* Fix go lint

* Fix Ori's review comments

* Change DAA score of genesis to checkpoint DAA score and fix all tests

* Fix the BlockLevel bits counting

* Fix some tests and make them run a little faster

* Change datadir name back to kaspa-mainnet and change db path from /data to /datadir

* Last changes for the release and change the version to 0.11.5

Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: Ori Newman <>
Co-authored-by: msutton <mikisiton2@gmail.com>
2021-11-25 20:18:43 +02:00
Michael Sutton
2a1b38ce7a
Fix mergeset limit violation bug (#1855)
* Added a test which reproduces a bug where virtual's mergeset exceeds the mergeset limit -- this test currently fails

* Added a simple condition to fix the mergeset limit bug

* Format issue
2021-11-24 19:34:59 +02:00
Ori Newman
765dd170e4
Optimizations and header size reduce hardfork (#1853)
* Modify DefaultTimeout to 120 seconds

A temporary workaround for nodes having trouble to sync (currently the download of pruning point related data during IBD takes more than 30 seconds)

* Cache existence in reachability store

* Cache block level in the header

* Fix IBD indication on submit block

* Add hardForkOmitGenesisFromParentsDAAScore logic

* Fix NumThreads bug in the wallet

* Get rid of ParentsAtLevel header method

* Fix a bug in BuildPruningPointProof

* Increase race detector timeout

* Add cache to BuildPruningPointProof

* Add comments and temp comment out go vet

* Fix ParentsAtLevel

* Dont fill empty parents

* Change HardForkOmitGenesisFromParentsDAAScore in fast netsync test

* Add --allow-submit-block-when-not-synced in stability tests

* Fix TestPruning

* Return fast tests

* Fix off by one error on kaspawallet

* Fetch only one block with trusted data at a time

* Update fork DAA score

* Don't ban for unexpected message type

* Fix tests

Co-authored-by: Michael Sutton <mikisiton2@gmail.com>
Co-authored-by: Ori Newman <>
2021-11-22 09:00:39 +02:00
stasatdaglabs
2dddb650b9
Switch to a fixed block subsidy after a certain work threshold (#1831)
* Implement isBlockRewardFixed.

* Fix factory.go.

* Call isBlockRewardFixed from calcBlockSubsidy.

* Fix bad call to ghostdagDataStore.Get.

* Extract blue score and blue work from the header instead of from the ghostdagDataStore.

* Fix coinbasemanager constructor arguments order

* Format consensus_defaults.go

* Check the mainnet switch from the block's point of view rather than the virtual's.

* Don't call newBlockPruningPoint twice in buildBlock.

* Properly handle new pruning point blocks in isBlockRewardFixed.

* Use the correct variable.

* Add a comment explaining what we do when the pruning point is not found in isBlockRewardFixed.

* Implement TestBlockRewardSwitch.

* Add missing error handling.

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-10-31 15:04:51 +02:00
stasatdaglabs
3dbc42b4f7
Implement the new block subsidy function (#1830)
* Replace the old blockSubsidy parameters with the new ones.

* Return subsidyGenesisReward if blockHash is the genesis hash.

* Traverse a block's past for the subsidy calculation.

* Partially implement SubsidyStore.

* Refer to SubsidyStore from CoinbaseManager.

* Wrap calcBlockSubsidy in getBlockSubsidy, which first checks the database.

* Fix finalityStore not calling GenerateShardingID.

* Implement calculateAveragePastSubsidy.

* Implement calculateMergeSetSubsidySum.

* Implement calculateSubsidyRandomVariable.

* Implement calcBlockSubsidy.

* Add a TODO about floats.

* Update the calcBlockSubsidy TODO.

* Use binary.LittleEndian in calculateSubsidyRandomVariable.

* Fix bad range in calculateSubsidyRandomVariable.

* Replace float64 with big.Rat everywhere except for subsidyRandomVariable.

* Fix a nil dereference.

* Use a random walk to approximate the normal distribution.

* In order to avoid unsupported fractional results from powInt64, flip the numerator and the denominator manually.

* Set standardDeviation to 0.25, MaxSompi to 10_000_000_000 * SompiPerKaspa and defaultSubsidyGenesisReward to 1_000.

* Set the standard deviation to 0.2.

* Use a binomial distribution instead of trying to estimate the normal distribution.

* Change some values around.

* Clamp the block subsidy.

* Remove the fake duplicate constants in the util package.

* Reduce MaxSompi to only 100m Kaspa to avoid hitting the uint64 ceiling.

* Lower MaxSompi further to avoid new and exciting ways for the uint64 ceiling to be hit.

* Remove debug logs.

* Fix a couple of failing tests.

* Fix TestBlockWindow.

* Fix limitTransactionCount sometimes crashing on index-out-of-bounds.

* In TrustedDataDataDAABlock, replace BlockHeader with DomainBlock

* In calculateAveragePastSubsidy, use blockWindow instead of doing a BFS manually.

* Remove the reference to DAGTopologyManager in coinbaseManager.

* Add subsidy to the coinbase payload.

* Get rid of the subsidy store and extract subsidies out of coinbase transactions.

* Keep a blockWindow amount of blocks under the virtual for IBD purposes.

* Manually remove the virtual genesis from the merge set.

* Fix simnet genesis.

* Fix TestPruning.

* Fix TestCheckBlockIsNotPruned.

* Fix TestBlockWindow.

* Fix TestCalculateSignatureHashSchnorr.

* Fix TestCalculateSignatureHashECDSA.

* Fix serializing the wrong value into the coinbase payload.

* Rename coinbaseOutputForBlueBlock to coinbaseOutputAndSubsidyForBlueBlock.

* Add a TODO about optimizing trusted data DAA window blocks.

* Expand on a comment in TestCheckBlockIsNotPruned.

* In calcBlockSubsidy, divide the big.Int numerator by the big.Int denominator instead of converting to float64.

* Clarify a comment.

* Rename SubsidyMinGenesisReward to MinSubsidy.

* Properly handle trusted data blocks in calculateMergeSetSubsidySum.

* Use the first two bytes of the selected parent's hash for randomness instead of math/rand.

* Restore maxSompi to what it used to be.

* Fix TestPruning.

* Fix TestAmountCreation.

* Fix TestBlockWindow.

* Fix TestAmountUnitConversions.

* Increase the timeout in many-tips to 30 minutes.

* Check coinbase subsidy for every block

* Re-rename functions

* Use shift instead of powInt64 to determine subsidyRandom

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-10-30 10:16:47 +03:00
Ori Newman
d9bc94a2a8
Replace header finality point with pruning point and enforce finality rules on IBD with headers proof (#1823)
* Replace header finality point with pruning point

* Fix TestTransactionAcceptance

* Fix pruning candidate

* Store all past pruning points

* Pass pruning points on IBD

* Add blue score to block header

* Simplify ArePruningPointsInValidChain

* Fix static check errors

* Fix genesis

* Renames and text fixing

* Use ExpectedHeaderPruningPoint in block builder

* Fix TestCheckPruningPointViolation
2021-08-31 08:01:48 +03:00
stasatdaglabs
837dac68b5
Update block headers to include multiple levels of parent blocks (#1822)
* Replace the old parents in the block header with BlockLevelParents.

* Begin fixing compilation errors.

* Implement database serialization for block level parents.

* Implement p2p serialization for block level parents.

* Implement rpc serialization for block level parents.

* Add DirectParents() to the block header interface.

* Use DirectParents() instead of Parents() in some places.

* Revert test_block_builder.go.

* Add block level parents to hash serialization.

* Use the zero hash for genesis finality points.

* Fix failing tests.

* Fix a variable name.

* Update headerEstimatedSerializedSize.

* Add comments in blocklevelparents.go.

* Fix the rpc-stability stability test.

* Change the field number for `parents` fields in p2p.proto and rpc.proto.

* Remove MsgBlockHeader::NumParentBlocks.
2021-08-24 12:06:39 +03:00
stasatdaglabs
d922ee1be2
Add header commitments for DAA score, blue work, and finality points (#1817)
* Add DAAScore, BlueWork, and FinalityPoint to externalapi.BlockHeader.

* Add DAAScore, BlueWork, and FinalityPoint to NewImmutableBlockHeader and fix compilation errors.

* Add DAAScore, BlueWork, and FinalityPoint to protowire header types and fix failing tests.

* Check for header DAA score in validateDifficulty.

* Add DAA score to buildBlock.

* Fix failing tests.

* Add a blue work check in validateDifficultyDAAAndBlueWork.

* Add blue work to buildBlock and fix failing tests.

* Add finality point validation to ValidateHeaderInContext.

* Fix genesis blocks' finality points.

* Add finalityPoint to blockBuilder.

* Fix tests that failed due to missing reachability data.

* Make blockBuilder use VirtualFinalityPoint instead of directly calling FinalityPoint with the virtual hash.

* Don't validate the finality point for blocks with trusted data.

* Add debug logs.

* Skip finality point validation for block whose finality points are the virtual genesis.

* Revert "Add debug logs."

This reverts commit 3c18f519ccbb382f86f63904dbb1c4cd6bc68b00.

* Move checkDAAScore and checkBlueWork to validateBlockHeaderInContext.

* Add checkCoinbaseBlueScore to validateBodyInContext.

* Fix failing tests.

* Add DAAScore, blueWork, and finalityPoint to blocks' hashes.

* Generate new genesis blocks.

* Fix failing tests.

* In BuildUTXOInvalidBlock, get the bits from StageDAADataAndReturnRequiredDifficulty instead of calling RequiredDifficulty separately.
2021-08-12 13:25:00 +03:00
Ori Newman
d748089a14
Update the virtual after overriding the virtual UTXO set (#1811)
* Update the virtual after overriding the virtual utxo set

* Put the updateVirtual inside importVirtualUTXOSetAndPruningPointUTXOSet

* Add pruningPoint to importVirtualUTXOSetAndPruningPointUTXOSet

* Remove sanity check
2021-08-02 17:02:15 +03:00
Ori Newman
f26a7fdedf
Return headers first (#1806)
* Return headers first

* Delete TestHandleRelayInvs

* resolve virtual only after IBD

* Fix ResolveVirtual

* Fix comments and variable names
2021-07-27 17:07:29 +03:00
Ori Newman
d207888b67
Implement pruned headers node (#1787)
* Pruning headers p2p basic structure

* Remove headers-first

* Fix consensus tests except TestValidateAndInsertPruningPointWithSideBlocks and TestValidateAndInsertImportedPruningPoint

* Add virtual genesis

* Implement PruningPointAndItsAnticoneWithMetaData

* Start fixing TestValidateAndInsertImportedPruningPoint

* Fix TestValidateAndInsertImportedPruningPoint

* Fix BlockWindow

* Update p2p and gRPC

* Fix all tests except TestHandleRelayInvs

* Delete TestHandleRelayInvs parts that cover the old IBD flow

* Fix lint errors

* Add p2p_request_ibd_blocks.go

* Clean code

* Make MsgBlockWithMetaData implement its own representation

* Remove redundant check if highest share block is below the pruning point

* Fix TestCheckLockTimeVerifyConditionedByAbsoluteTimeWithWrongLockTime

* Fix comments, errors ane names

* Fix window size to the real value

* Check reindex root after each block at TestUpdateReindexRoot

* Remove irrelevant check

* Renames and comments

* Remove redundant argument from sendGetBlockLocator

* Don't delete staging on non-recoverable errors

* Renames and comments

* Remove redundant code

* Commit changes inside ResolveVirtual

* Add comment to IsRecoverableError

* Remove blocksWithMetaDataGHOSTDAGDataStore

* Increase windows pagefile

* Move DeleteStagingConsensus outside of defer

* Get rid of mustAccepted in receiveBlockWithMetaData

* Ban on invalid pruning point

* Rename interface_datastructures_daawindowstore.go to interface_datastructures_blocks_with_meta_data_daa_window_store.go

* * Change GetVirtualSelectedParentChainFromBlockResponseMessage and VirtualSelectedParentChainChangedNotificationMessage to show only added block hashes
*  Remove ResolveVirtual
* Use externalapi.ConsensusWrapper inside MiningManager
* Fix pruningmanager.blockwithmetadata

* Set pruning point selected child when importing the pruning point UTXO set

* Change virtual genesis hash

* replace the selected parent with virtual genesis on removePrunedBlocksFromGHOSTDAGData

* Get rid of low hash in block locators

* Remove +1 from everywhere we use difficultyAdjustmentWindowSize and increase the default value by one

* Add comments about consensus wrapper

* Don't use separate staging area when resolving resolveBlockStatus

* Fix netsync stability test

* Fix checkResolveVirtual

* Rename ConsensusWrapper->ConsensusReference

* Get rid of blockHeapNode

* Add comment to defaultDifficultyAdjustmentWindowSize

* Add SelectedChild to DAGTraversalManager

* Remove redundant copy

* Rename blockWindowHeap->calculateBlockWindowHeap

* Move isVirtualGenesisOnlyParent to utils

* Change BlockWithMetaData->BlockWithTrustedData

* Get rid of maxReasonLength

* Split IBD to 100 blocks each time

* Fix a bug in calculateBlockWindowHeap

* Switch to trusted data when encountering virtual genesis in blockWithTrustedData

* Move ConsensusReference to domain

* Update ConsensusReference comment

* Add comment

* Rename shouldNotAddGenesis->skipAddingGenesis
2021-07-26 12:24:07 +03:00
talelbaz
aba44e7bfb
Disable relative time lock by time (#1800)
* ignore type flag

* Ignore type flag of relative time lock - interpret as DAA score

* Split verifyLockTime to functions with and without threshold.relative lockTimes dont need threshold check

* Change function name and order of the functions calls

Co-authored-by: tal <tal@daglabs.com>
2021-07-18 17:52:16 +03:00
Svarog
369a3bac09
Limit block mass instead of merge set limit + Introduce SigOpCount to TransactionInput (#1790)
* Update constants

* Add to transaction SigOpCount

* Update mass calculation, and move it from InContext to InIsolation

* Update block validation accordingly

* Add SigOpCount validation during TransactionInContext

* Remove checking of mass vs maxMassAcceptedByBlock from consensusStateManager

* Update mining manager with latest changes

* Add SigOpCount to MsgTx.Copy()

* Fix initTestTransactionAcceptanceDataForClone

* Fix all tests in transaction_equal_clone_test.go

* Fix TestBlockMass

* Fix tests in transactionvalidator package

* Add SigOpCount to sighash

* Fix TestPruningDepth

* Fix problems in libkaspawalelt

* Fix integration tests

* Fix CalculateSignatureHash tests

* Remove remaining places talking about block size

* Add sanity check to checkBlockMass to make sure all transactions have their mass filled

* always add own sigOpCount to sigHash

* Update protowire/rpc.md

* Start working on removing any remaining reference to block/tx size

* Update rpc transaction verbose data to include mass rather then size

* Convert verboseData and block size check to mass

* Remove remaining usages of tx size in mempool

* Move transactionEstimatedSerializedSize to transactionvalidator

* Add PopulateMass to fakeRelayInvsContext

* Move PopulateMass to beggining of ValidateAndInsertTransaction + fix in it

* Assign mass a new number for backward-compatibility
2021-07-14 14:21:57 +03:00
Ori Newman
edc459ae1b
Improve pickVirtualParents performance and add many-tips to the stability tests suite (#1725)
* First limit the candidates size to 3*csm.maxBlockParents before taking the bottom csm.maxBlockParents/2

* Change log level of printing all tips to Tracef

* Add many-tips to run-fast.sh and run-slow.sh

* Fix preallocation size

* Assign intermediate variables
2021-05-19 16:06:52 +03:00
talelbaz
d7f2cf81c0
Change merge set order to topological order (#1654)
* Change mergeSet to be ordered topologically.

* Add special condition for genesis.

* Add check that the coinbase is validated.

* Change names of variables(old: chainHash, blueHash).

* Fix the DAG diagram in the comment above the function.

* Fix variables names.

Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-05-19 14:40:55 +03:00
Svarog
dfd8b3423d
Implement new mechanism for updating UTXO Diffs (#1671)
* Use selectedParent instead of selectedTip for non-selectedTip blocks in restoreSingleBlockStatus

* Cache the selectedParent for re-use in a resolveSingleBlockStatus chain

* Implement and use reverseUTXOSet

* Reverse blocks in correct order

* Support resolveBlockStatus without separate stagingAreas for usage of testConsensus

* Handle the case where the tip of the resolved block is not the next selectedTip

* Unify isResolveTip

* Some minor fixes and cleanup

* Add full finality window re-org test to stability-slow

* rename: useSeparateStagingAreasPerBlock -> useSeparateStagingAreaPerBlock

* Better logs in resolveSingleBlockStatus

* A few retouches to reverseUTXODiffs

* TEMPORARY COMMIT: EXTRAT ALL DIFFFROMS TO SEPARATE METHODS

* TEMPORARY COMMIT: REMOVE DIFFICULTY CHECKS IN DEVNET

* Don't pre-allocate in utxo-algebra, since the numbers are not known ahead-of-time

* Add some logs to reverseUTXODiffs

* Revert "TEMPORARY COMMIT: REMOVE DIFFICULTY CHECKS IN DEVNET"

This reverts commit c0af9dc6ade78a914c970e11bc63c34605565f57.

* Revert "TEMPORARY COMMIT: EXTRAT ALL DIFFFROMS TO SEPARATE METHODS"

This reverts commit 4fcca1b48c3a1183598833a355b9bfaf169edba1.

* Remove redundant paranthesis

* Revise some logs messages

* Rename:oneBlockBeforeCurrentUTXOSet -> lastResolvedBlockUTXOSet

* Don't break if the block was resolved as invalid

* rename unverifiedBlocks to recentlyVerifiedBlcks in reverseUTXODiffs

* Add errors.New to the panic, for a stack trace

* Reverse the UTXODiffs after the main block has been commited

* Use the correct value for previousUTXODiff

* Add test for ReverseUTXODiff

* Fix some names and comments

* Update TestReverseUTXODiffs to use consensus.Config

* Fix comments mentioning 'oneBlockBeforeTip'
2021-04-20 10:26:55 +03:00
Elichai Turkel
83beae4463
Add consensus.Config as a wrapper for dagParams (#1680)
* Add a new consensus.Config wrapper to dagParams

* Update all tests to use consensus.Config
2021-04-19 09:07:34 +03:00
Svarog
acdc59b565
Add version file to database (#1678)
* Add version file to database

* Remove redundant code

* Check for version before opening the database, create version file after

* Create version file before opening the database
2021-04-14 12:35:39 +03:00
Svarog
347dd8fc4b
Support resolveBlockStatus without separate stagingAreas for usage of testConsensus (#1666) 2021-04-08 11:26:17 +03:00
talelbaz
70f3fa9893
Update miningManager test (#1593)
* [NOD-1429] add mining manager unit tests

* [NOD-1429] Add additional test

* found a bug, so stopped working on this test until the bug will be fix.

* Update miningmanager_test.go test.

* Delete payloadHash field - not used anymore in the current version.

* Change the condition for comparing slices instead of pointers.

* Fix due to review notes - change names, use testutils.CreateTransaction function and adds comments.

* Changes after fetch&merge to v0.10.0-dev

* Create a new function createChildTxWhenParentTxWasAddedByConsensus and add a comment

* Add an argument to create_transaction function and fix review notes

* Optimization

* Change to blockID(instead of the all transaction) in the error messages and fix review notes

* Change to blockID(instead of the all transaction) in the error messages and fix review notes

* Change format of error messages.

* Change name ofa variable

* Use go:embed to embed sample-kaspad.conf (only on go1.16)

* Revert "Use go:embed to embed sample-kaspad.conf (only on go1.16)"

This reverts commit bd28052b92b5a36f2ea826188d6876fddaf1802d.

Co-authored-by: karim1king <karimkaspersky@yahoo.com>
Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-03-30 13:52:40 +03:00
Svarog
4e18031483
Resolve each block status in it's own staging area (#1634) 2021-03-30 11:04:43 +03:00
Svarog
f1451406f7
Add support for multiple staging areas (#1633)
* Add StagingArea struct

* Implemented staging areas in blockStore

* Move blockStagingShard to separate folder

* Apply staging shard to acceptanceDataStore

* Update blockHeaderStore with StagingArea

* Add StagingArea to BlockRelationStore

* Add StagingArea to blockStatusStore

* Add StagingArea to consensusStateStore

* Add StagingArea to daaBlocksStore

* Add StagingArea to finalityStore

* Add StagingArea to ghostdagDataStore

* Add StagingArea to headersSelectedChainStore and headersSelectedTipStore

* Add StagingArea to multisetStore

* Add StagingArea to pruningStore

* Add StagingArea to reachabilityDataStore

* Add StagingArea to utxoDiffStore

* Fix forgotten compilation error

* Update reachability manager and some more things with StagingArea

* Add StagingArea to dagTopologyManager, and some more

* Add StagingArea to GHOSTDAGManager, and some more

* Add StagingArea to difficultyManager, and some more

* Add StagingArea to dagTraversalManager, and some more

* Add StagingArea to headerTipsManager, and some more

* Add StagingArea to constnsusStateManager, pastMedianTimeManager

* Add StagingArea to transactionValidator

* Add StagingArea to finalityManager

* Add StagingArea to mergeDepthManager

* Add StagingArea to pruningManager

* Add StagingArea to rest of ValidateAndInsertBlock

* Add StagingArea to blockValidator

* Add StagingArea to coinbaseManager

* Add StagingArea to syncManager

* Add StagingArea to blockBuilder

* Update consensus with StagingArea

* Add StagingArea to ghostdag2

* Fix remaining compilation errors

* Update names of stagingShards

* Fix forgotten stagingArea passing

* Mark stagingShard.isCommited = true once commited

* Move isStaged to stagingShard, so that it's available without going through store

* Make blockHeaderStore count be avilable from stagingShard

* Fix remaining forgotten stagingArea passing

* commitAllChanges should call dbTx.Commit in the end

* Fix all tests tests in blockValidator

* Fix all tests in consensusStateManager and some more

* Fix all tests in pruningManager

* Add many missing stagingAreas in tests

* Fix many tests

* Fix most of all other tests

* Fix ghostdag_test.go

* Add comment to StagingArea

* Make list of StagingShards an array

* Add comment to StagingShardID

* Make sure all staging shards are pointer-receiver

* Undo bucket rename in block_store

* Typo: isCommited -> isCommitted

* Add comment explaining why stagingArea.shards is an array
2021-03-29 10:34:11 +03:00
Ori Newman
d9449a32b8
Use DAA score where needed (#1602)
* Replace blue score with DAA score in UTXO entries

* Use DAA score for coinbase maturity

* Use DAA score for sequence lock

* Fix calcBlockSubsidy to use DAA score

* Don't pay to blocks that are not included in the DAA added blocks, and bestow red blocks reward to the merging block

* Fix TestGetPruningPointUTXOs

* Fix TestTransactionAcceptance

* Fix TestChainedTransactions

* Fix TestVirtualDiff

* Fix TestBlockWindow

* Fix TestPruning

* Use NewFromSlice instead of manually creating the hash set

* Add assert

* Add comment

* Remove redundant call to UpdateDAADataAndReturnDifficultyBits

* Add RequiredDifficulty, rename UpdateDAADataAndReturnDifficultyBits to StageDAADataAndReturnRequiredDifficulty and add comments

* Make buildUTXOInvalidHeader get bits as an argument

* Fix comments
2021-03-15 13:48:40 +02:00
Ori Newman
53781eed4d
Remove payload hash (#1583)
* Remove payload hash

* Fix tests
2021-03-08 15:15:03 +02:00
Elichai Turkel
7909480757
Merge big subdags in pick virtual parents (#1574)
* Refactor mergeSetIncrease to return the current BFS block to allow easier merging

* Remove unneeded Heap/HashSet usages

* Add new IsAnyAncestorOf to DagTopolyManager

* Check if the new candidate is in the future of any existing candidate

* Add comments and fix off-by-one in the mergeSetIncrease queue

* Fixed DAGToplogy test mock

* Fix review comments
2021-03-03 16:17:16 +02:00
Elichai Turkel
1f69f9eed9
Cleanup the logger and make it asynchronous (#1524)
* Remove Subsystems map and replace with RegisterSubSystem

* Clean up the logger

* Fix LOGFLAGS and make LongFile work correctly

* Parallelize the logger backend

* More logger cleanup

* Initialize and close the logger backend wherever it's needed

* Move the location where the backend is closed, also print the log if it panics while writing

* Add TestMain to reachability manager tests to preserve the same log level

* Fix review comments

Co-authored-by: Svarog <feanorr@gmail.com>
2021-03-01 14:04:40 +02:00
Elichai Turkel
6a12428504
Close iterators (#1542)
* Add Close() function to all the iterators

* Add defer iterator.Close() whenever we open an iterator

* Add isClosed to all iterators and panic/return error if used after closing

Co-authored-by: Svarog <feanorr@gmail.com>
2021-03-01 11:15:59 +02:00
Svarog
fb6c9c8f21
Remove virtual diff parents (#1550)
* resolveSingleBlockStatus: If the block being resolved is not going to be the next selectedTip - set it's diffParent to be the old selectedTip

* resolveSingleBlockStatus: If the block being resolved is going to be the next selectedTip - set it as old selectedTip's diffChild

* Remove any mentions of virtualDiffParents

* If block is genesis - don't do all the mumbo-jumbo with oldSelectedTip

* Check an unchecked error

* Write a better log message
2021-02-23 17:19:35 +02:00
Ori Newman
2adb4f5d0f
Fix UTXO index (#1548)
* Add VirtualUTXODiff and VirtualParents to block insertion result

* Add GetVirtualUTXOs

* Add OnPruningPointUTXOSetOverrideHandler

* Add recovery to UTXO index

* Add UTXO set override notification

* Fix compilation error

* Fix iterators in UTXO index and fix TestUTXOIndex

* Change Dialing to DEBUG

* Change LogBlock location

* Rename StopNotify to StopNotifying

* Add sanity check

* Add comment

* Remove receiver from serialization functions

Co-authored-by: Elichai Turkel <elichai.turkel@gmail.com>
2021-02-23 16:51:51 +02:00
Svarog
1222a555f2
Prune blocks below pruning point when moving pruning point during IBD (#1513)
* Split deletePastBlocks into sub-routines

* Remove SelectedParentIterator, and refactor SelectedChildIterator to support First and Error

* Implement PruneAllBlocks

* Prune all blocks in the store

* Prune only blocks that are below the pruning point

* Defer call onEnd of LogAndMeasureExecutionTime

* Handle a forgotten error

* Minor style fixes
2021-02-10 16:39:36 +02:00
Elichai Turkel
2d61a67592
Change some logs (#1511) 2021-02-09 14:00:02 +02:00
Elichai Turkel
3d0a2a47b2
Move testGHOSTDagSorter to testutils, and build a boilerplate for overriding specific managers (#1486)
* Move testGHOSTDagSorter to testutils

* Allow overriding managers in consensus, starting with ghostdag

* Add test prefix to SetDataDir and SetGHOSTDAGManager

Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2021-02-08 15:24:26 +02:00
Svarog
6bc7a4eb85
Allow GetMissingBlockBodyHashes return an empty list if the missing blocks were requested before IBD start (#1498)
* Allow GetMissingBlockBodyHashes return an empty list if the missing blocks were requested before IBD start

* Add link to issue in comment about error to be fixed
2021-02-07 16:12:15 +02:00
Ori Newman
dbababb978
Limit mempool size to million transactions and remove the least profitable transactions (#1483)
* Limit mempool size to million transactions and remove the least profitable transactions

* Simplify insert

* Fix typo

* Improve findTxIndexInOrderedTransactionsByFeeRate readability
2021-02-03 19:45:39 +02:00
Ori Newman
238950cb98
Add logs (#1484)
* Add logs

* Fix log name
2021-02-03 17:47:59 +02:00
stasatdaglabs
97be133cee
Add logs to help debug long virtual parent selection. (#1470)
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-02-01 19:15:50 +02:00