181 Commits

Author SHA1 Message Date
Ori Newman
6d46f5510e Merge tag 'patch10' into v0.11.13-rc1 2022-03-11 19:24:36 +02:00
Ori Newman
ad6310de81 Send pruning point and its anticone in batches 2022-03-11 18:00:19 +02:00
msutton
3f51f926bb apply Don't relay blocks in virtual anticone #1970 to v5 2022-03-11 15:30:16 +02:00
msutton
91875da221 Merge remote-tracking branch 'origin/dev' into fix-ibd-shared-past-search 2022-03-11 15:27:54 +02:00
msutton
fb95156cd1 rename pastdiff -> anticone 2022-03-11 15:27:03 +02:00
msutton
047fa30183 apply the traversal limit logic and sort headers 2022-03-11 15:05:26 +02:00
msutton
8f68d5c606 addressing simple comments 2022-03-11 14:11:08 +02:00
Ori Newman
b7be807167
Don't relay blocks in virtual anticone (#1970) 2022-03-11 13:24:45 +02:00
msutton
63d7dd9f62 missed one err check 2022-03-11 11:26:06 +02:00
msutton
48eb577cac remove unused methods 2022-03-11 11:22:29 +02:00
msutton
a388aefa37 go fmt 2022-03-11 11:18:25 +02:00
msutton
e67b3b65b4 implement ibd sync past diff of relay and selected tip 2022-03-11 11:04:28 +02:00
msutton
6231eaf231 handle and flow for new request past diff message - logic unimplemented yet 2022-03-11 09:19:09 +02:00
msutton
c684bf04bc address the case where pruning points disagree, now both IBD tests pass 2022-03-10 17:23:11 +02:00
msutton
947d427f40 new ibd shared block algo -- only basic test passing 2022-03-10 03:02:08 +02:00
msutton
9df231f810 added relay hash to the log print 2022-03-08 09:29:00 +02:00
msutton
09cebe6960 Perform side-chain check earlier to avoid IBD start 2022-03-08 09:18:24 +02:00
msutton
c903a65def a temp patch for fixing IBD issues for all side-chains 2022-03-08 03:51:06 +02:00
msutton
840b2a3353 wip 2022-03-08 03:15:20 +02:00
msutton
ea0586045d set all internal imports to v5 2022-03-07 16:37:42 +02:00
msutton
685c049a12 yet another checkpoint 2022-03-07 15:51:59 +02:00
msutton
35f86bc98a add p2p v5 which is currently identical to v4 2022-03-07 15:33:52 +02:00
msutton
cb5e9b55b7 Update checkpoint to yet another side-chain 2022-03-07 14:54:05 +02:00
msutton
20f16cf729 Update checkpoint to new side-chain 2022-03-06 01:03:44 +02:00
Ori Newman
4d3f504b73 Check checkpoint only if highestSharedBlockFound 2022-03-02 21:17:46 +02:00
Ori Newman
b5eda33488 remove count 2022-03-02 13:11:32 +02:00
Ori Newman
ef1a3c0dce remove debug log 2022-03-02 13:09:34 +02:00
Ori Newman
1cedc720ac patch 2022-03-02 12:31:14 +02:00
Ori Newman
6449b03034
Ignore transaction invs on IBD (#1960)
* Ignore transaction invs on IBD

* Add IsIBDRunning mock to TestHandleRelayedTransactionsNotFound

Co-authored-by: Ori Newman <>
2022-02-26 22:20:08 +02:00
Ori Newman
3f7e482291
Add progress indication for virtual resolution (#1949)
* Add progress indication for virtual resolution

* Avoid division by zero
2022-02-20 00:32:41 +02:00
Ori Newman
82f0a4d74f
Drop support for p2p v3 (#1942)
* Drop support for p2p v3

* Remove redundant aliases

* Remove redundant condition
2022-02-06 17:28:06 +02:00
Ori Newman
1cd712a63e
Increase headers proof timeout and add more progress logs (#1939)
* Increase timeout for pruning proof and add some logs

* Show resolving virtual progress as whole percents
2022-02-06 12:42:28 +02:00
stasatdaglabs
b1229f7908
Display progress of IBD process in Kaspad logs (#1938)
* Report progress percentage when downloading headers in IBD.

* Extract reporting logic to a separate type.

* Report progress for IBD missing block bodies.
2022-02-02 21:24:45 +02:00
Michael Sutton
4855d845b3
Extract IBD management from invs relay flow to a new separated flow (#1930)
* Separate IBD to a new flow (so now invs are handled concurrently and no route capacity errors)

* Invs messages should be queued while waiting for BlockLocator msg

* Close IBD channel so that HandleIBDFlow exits too

* Apply flow separation to p2p protocol v4

* Manage the IBDRequestChannel through the Peer struct

* Some IBDs take a little longer
2022-01-24 09:30:41 +02:00
Ori Newman
dadacdc0f4
Filter redundant blocks from daa window (#1925)
* Copy blockrelay flows to v4

* Remove duplicate sending of the same DAA blocks

* Advance testnet version

* Renames and add comments

* Add IBD test between v3 and v4

* Fix syncee v4 p2p version

* Check if netsync finished with selected tip
2022-01-09 16:58:51 +02:00
Ori Newman
d2379608ad
P2P upgrade mechanism (#1921)
* Implement upgrade mechanism for p2p

* Remove dependencies from flowcontext to v3

* Add p2p v4

* Add Ready flow

* Remove copy paste code of v3

* Register SendAddresses flow at the top level

* Add option to set protocol version from CLI and add TestAddressExchangeV3V4

* Send ready message on minimal net adapter

* Rename defaultMaxProtocolVersion->maxAcceptableProtocolVersion
2022-01-09 09:59:45 +02:00
Ori Newman
1d740e1eab
UTXO index fix (#1891)
* Update to version v0.11.8

* Call OnPruningPointUTXOSetOverride after committing the staging consensus

* use domain.Consensus() in utxo index so it'll always check the right consensus
2021-12-19 08:56:18 +02:00
Ori Newman
32e8e539ac
Apply ResolveVirtual diffs to the UTXO index (#1868)
* Apply ResolveVirtual diffs to the UTXO index

* Add comments

Co-authored-by: Ori Newman <>
2021-12-05 13:22:48 +02:00
Ori Newman
11103a36d3
Get rid of genesis's UTXO dump (#1867)
* Get rid of genesis's UTXO dump

* Allow known orphans when AllowSubmitBlockWhenNotSynced=true

* gofmt

* Avoid IBD without changing the pruning point when only genesis is available

* Add DisallowDirectBlocksOnTopOfGenesis=true for mainnet

* Remove any mention to nobanning to let stability tests run

* Rename ifGenesisSetUtxoSet to loadUTXODataForGenesis

Co-authored-by: Ori Newman <>
2021-12-05 12:46:41 +02:00
Elichai Turkel
606b781ca0
Fix bug in RequiredDifficulty and release another version 2021-11-25 21:49:15 +02:00
Elichai Turkel
dbf18d8052
Hard fork - new genesis with the utxo set of the last block (#1856)
* UTXO dump of block 0fca37ca667c2d550a6c4416dad9717e50927128c424fa4edbebc436ab13aeef

* Activate HF immediately and change reward to 1000

* Change protocol version and datadir location

* Delete comments

* Fix zero hash to muhash zero hash in genesis utxo dump check

* Don't omit genesis as direct parent

* Fix tests

* Change subsidy to 500

* Dont assume genesis multiset is empty

* Fix BlockReward test

* Fix TestValidateAndInsertImportedPruningPoint test

* Fix pruning point genesis utxo set

* Fix tests related to mainnet utxo set

* Dont change the difficulty before you have a full window

* Fix TestBlockWindow tests

* Remove global utxo set variable, and persist mainnetnet utxo deserialization between runs

* Fix last tests

* Make peer banning opt-in

* small fix for a test

* Fix go lint

* Fix Ori's review comments

* Change DAA score of genesis to checkpoint DAA score and fix all tests

* Fix the BlockLevel bits counting

* Fix some tests and make them run a little faster

* Change datadir name back to kaspa-mainnet and change db path from /data to /datadir

* Last changes for the release and change the version to 0.11.5

Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: Ori Newman <>
Co-authored-by: msutton <mikisiton2@gmail.com>
2021-11-25 20:18:43 +02:00
Ori Newman
b04292c97a Don't server parallel PruningPointAndItsAnticoneRequests 2021-11-22 09:56:30 +02:00
Ori Newman
765dd170e4
Optimizations and header size reduce hardfork (#1853)
* Modify DefaultTimeout to 120 seconds

A temporary workaround for nodes having trouble to sync (currently the download of pruning point related data during IBD takes more than 30 seconds)

* Cache existence in reachability store

* Cache block level in the header

* Fix IBD indication on submit block

* Add hardForkOmitGenesisFromParentsDAAScore logic

* Fix NumThreads bug in the wallet

* Get rid of ParentsAtLevel header method

* Fix a bug in BuildPruningPointProof

* Increase race detector timeout

* Add cache to BuildPruningPointProof

* Add comments and temp comment out go vet

* Fix ParentsAtLevel

* Dont fill empty parents

* Change HardForkOmitGenesisFromParentsDAAScore in fast netsync test

* Add --allow-submit-block-when-not-synced in stability tests

* Fix TestPruning

* Return fast tests

* Fix off by one error on kaspawallet

* Fetch only one block with trusted data at a time

* Update fork DAA score

* Don't ban for unexpected message type

* Fix tests

Co-authored-by: Michael Sutton <mikisiton2@gmail.com>
Co-authored-by: Ori Newman <>
2021-11-22 09:00:39 +02:00
Ori Newman
9b81f5145e
Increase p2p msg size and reset utxo after ibd (#1847)
* Don't build unnecessary binaries

* Reset UTXO index after IBD

* Enlarge max msg size to 1gb

* Fix UTXO chunks logic

* Revert UTXO set override change

* Fix sendPruningPointUTXOSet

* Increase tests timeout to 20 minutes

Co-authored-by: Kaspa Profiler <>
2021-11-11 09:27:07 +02:00
Ori Newman
ff72568d6b
Fix pruning point anticone order (#1836)
* Send pruning point anticone in topological order
Fix a UTXO pagination bug
Lengthen the stabilization time for the last DAA test

* Extend "sudden hash rate drop" test length to 45 minutes

Co-authored-by: Kaspa Profiler <>
2021-11-07 08:21:34 +02:00
Ori Newman
99aaacd649
Check blue score before requesting a pruning proof (#1835)
* Check blue score before requesting a pruning proof

* BuildPruningPointProof should return empty proof if the pruning point is genesis

* Don't fail many-tips if kaspad exits ungracefully
2021-10-31 12:48:18 +02:00
stasatdaglabs
77a344cc29
In IBD, validate the timestamps of the headers of the pruning point and selected tip (#1829)
* Implement validatePruningPointFutureHeaderTimestamps.

* Fix TestIBDWithPruning.

* Fix wrong logic.

* Add a comment.

* Fix a comment.

* Fix a variable name.

* Add a commment

* Fix TestIBDWithPruning.

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-10-30 20:32:49 +03:00
Ori Newman
5dbb1da84b
Implement pruning point proof (#1832)
* Calculate GHOSTDAG, reachability etc for each level

* Don't preallocate cache for dag stores except level 0 and reduce the number of connections in the integration test to 32

* Reduce the number of connections in the integration test to 16

* Increase page file

* BuildPruningPointProof

* BuildPruningPointProof

* Add PruningProofManager

* Implement ApplyPruningPointProof

* Add prefix and fix blockAtDepth and fill headersByLevel

* Some bug fixes

* Include all relevant blocks for each level in the proof

* Fix syncAndValidatePruningPointProof to return the right block hash

* Fix block window

* Fix isAncestorOfPruningPoint

* Ban for rule errors on pruning proof

* Find common ancestor for blockAtDepthMAtNextLevel

* Use pruning proof in TestValidateAndInsertImportedPruningPoint

* stage status and finality point for proof blocks

* Uncomment golint

* Change test timeouts

* Calculate merge set for ApplyPruningPointProof

* Increase test timeout

* Add better caching for daa window store

* Return to default timeout

* Add ErrPruningProofMissesBlocksBelowPruningPoint

* Add errDAAWindowBlockNotFound

* Force connection loop next iteration on connection manager stop

* Revert to Test64IncomingConnections

* Remove BlockAtDepth from DAGTraversalManager

* numBullies->16

* Set page file size to 8gb

* Increase p2p max message size

* Test64IncomingConnections->Test16IncomingConnections

* Add comment for PruningProofM

* Add comment in `func (c *ConnectionManager) Stop()`

* Rename isAncestorOfPruningPoint->isAncestorOfSelectedTip

* Revert page file to 16gb

* Improve ExpectedHeaderPruningPoint perf

* Fix comment

* Revert "Improve ExpectedHeaderPruningPoint perf"

This reverts commit bca1080e7140c78d510f51bbea858ae280c2f38e.

* Don't test windows
2021-10-26 09:48:27 +03:00
stasatdaglabs
0053ee788d
Use the BlueWork declared in an orphan block's header instead of requesting it explicitly from the peer that sent us the orphan (#1828) 2021-09-13 13:13:03 +03:00
stasatdaglabs
af7e7de247
Add PruningPointProof to the P2P protocol (#1825)
* Add PruningPointProof to externalapi.

* Add BuildPruningPointProof and ValidatePruningPointProof to Consensus.

* Add the pruning point proof to the protocol.

* Add the pruning point proof to the wire package.

* Add PruningPointBlueWork.

* Make go vet happy.

* Properly initialize PruningPointProof in consensus.go.

* Validate pruning point blue work.

* Populate PruningPointBlueWork with the actual blue work of the pruning point.

* Revert "Populate PruningPointBlueWork with the actual blue work of the pruning point."

This reverts commit f2a9829998a8161ca24c259f971bd144084f0488.

* Revert "Validate pruning point blue work."

This reverts commit c6a90c5d2c5137657484d49ae8f33609aeb50235.

* Revert "Properly initialize PruningPointProof in consensus.go."

This reverts commit 9391574bbf711ab14eaf0c7700ae4d4b2e782e9d.

* Revert "Add PruningPointBlueWork."

This reverts commit 48182f652a5ff5048017d2c26640a1ffec79bc60.

* Fix PruningPointProof and MsgPruningPointProof to be two-dimensional.

* Fix wire PruningPointProof to be two-dimensional.
2021-09-05 17:20:15 +03:00