25 Commits

Author SHA1 Message Date
Ori Newman
beee947dda
Fix IBD sync conditions (#2174)
* Fix IBD sync conditions

* Fix syntax

* Fix Sprintf

* Bump version

* On negotiation check only blocks in future of PP

* Only log error and add comment

* Fix comment
2022-11-29 17:18:07 +02:00
Ori Newman
eec6eb9669
Check rule errors when validating blocks with trusted data (#2171) 2022-11-21 23:06:00 +02:00
Ori Newman
9fbfba17b6
Compare blue score with selected tip when checking if a pruning point… (#2169)
* Compare blue score with selected tip when checking if a pruning point proof is needed

* Don't redeclare err

Co-authored-by: Michael Sutton <mikisiton2@gmail.com>
2022-11-20 13:35:00 +02:00
Ori Newman
7d44275eb1
Add found to GetBlock (#2165) 2022-11-16 23:48:05 +02:00
Ori Newman
26c7db251f
Make more checks if status is invalid even if the block exists (#2158)
* Make more checks if status is invalid even if the block exists

* Use HasHeader
2022-10-13 19:22:00 +03:00
Michael Sutton
6a5e7c9e3f
Add IBD error details to the log (#2150) 2022-09-30 13:23:54 +03:00
Ori Newman
0838cc8e32
Update virtual on IBD if nearly synced (#2134)
* Update virtual on IBD if nearly synced

* Don't resolve virtual if updateVirtual
2022-09-09 00:52:08 +03:00
D-Stacks
eb693c4a86
Mempool: Retrive stable state of the mempool. optimze get mempool entries by addresses (#2111)
* fix mempool accessing, rewrite get_mempool_entries_by_addresses

* fix counter, add verbose

* fmt

* addresses as string

* Define error in case utxoEntry is missing.

* fix error variable to string

* stop tests from failing (see in code comment)

* access both pools in the same state via parameters

* get rid of todo message

* fmt - very important!

* perf: scriptpublickey in mempool, no txscript.

* address reveiw

* fmt fix

* mixed up isorphan bool, pass tests now

* do map preallocation in mempoolbyaddresses

* no proallocation for orphanpool sending.

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-07-26 23:06:08 +03:00
Michael Sutton
1c9bb54cc2
Crucial fix for the UTXO difference mechanism (#2114)
* Illustrate the bug through prints

* Change consensus API to a single ResolveVirtual call

* nil changeset is not expected when err=nil

* Fixes a deep bug in the resolve virtual process

* Be more defensive at resolving virtual when adding a block

* When finally resolved, set virtual parents properly

* Return nil changeset when nothing happened

* Make sure the block at the split point is reversed to new chain as well

* bump to version 0.12.4

* Avoid locking consensus twice in the common case of adding block with updateVirtual=true

* check err

* Parents must be picked first before set as virtual parents

* Keep the flag for tracking virtual state, since tip sorting perf is high with many tips

* Improve and clarify resolve virtual tests

* Addressing minor review comments

* Fixed a bug in the reported virtual changeset, and modified the test to verify it

* Addressing review comments
2022-07-15 12:35:20 +03:00
D-Stacks
18d000f625
Logger: change log level for "Couldn't find UTXO entry" to debug (and ignore message on orphan transactions) (#2108)
* Logger: change log level for "Couldn't find UTXO entry" to debug

* ignore error for orphans

* continue also if orphans

* check if blocks exist

* safe rpc mode

* limit window size

* verify block status depending on context

* allow a 2 factor gap in expected mergeset size

Co-authored-by: msutton <mikisiton2@gmail.com>
2022-06-29 15:54:40 +03:00
Michael Sutton
d957a6d93a
Fix UTXO diff child error (#2084)
* Avoid creating the chain iterator if high hash is actually low hash

* Always use iterator in nextPruningPointAndCandidateByBlockHash

* Initial failing test

* Minimal failing test + some comments

* go lint

* Add simpler tests with two different errors

* Missed some error checks

* Minor

* A workaround patch for preventing the missing utxo child diff bug

* Make sure we fully resolve virtual

* Move ResolveVirtualWithMaxParam to test consensus

* Mark virtual not updated and loop in batches

* Refactor: remove VirtualChangeSet from functions return values

* Remove workaround comments

* If block has no body, virtual is still considered updated

* Remove special error ErrReverseUTXODiffsUTXODiffChildNotFound

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-06-15 02:52:14 +03:00
Michael Sutton
016ddfdfce
Use a channel for utxo change events (#2052)
* Use a channel from within consensus in order to raise change events in order -- note that this is only a draft commit for discussion

* Fix compilation

* Check for nil

* Allow nil virtualChangeChan

* Remove redundant comments

* Call notifyVirtualChange instead of notifyUTXOsChanged

* Remove redundant comment

* Add a separate function for initVirtualChangeHandler

* Remove redundant type

* Check for nil in the right place

* Fix integration test

* Add data to virtual changeset and cleanup block added event logic

* Renames

* Comment

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-05-19 14:07:48 +03:00
Michael Sutton
723aebbec9
Do not apply merge depth root heuristic on orphan roots (#2019) 2022-04-13 21:06:08 +03:00
Ori Newman
2b395e34b1
Use separate depth than finality depth for merge set calculations after HF (#2013)
* Use separate than finality depth for merge set calculations after HF

* Add comments and edit error messages

* Fix TestValidateTransactionInContextAndPopulateFee

* Don't disconnect from node if isViolatingBoundedMergeDepth

* Use new merge root for virtual pick parents; apply HF1 daa score split for validation only

* Use `blue work` heuristic to skip irrelevant relay blocks

* Minor

* Make sure virtual's merge depth root is a real block

* For ghostdag data we always use the non-trusted data

* Fix TestBoundedMergeDepth and in IBD use VirtualMergeDepthRoot instead of MergeDepthRoot

* Update HF1DAAScore

* Make sure merge root and finality are called + avoid calculating virtual root twice

* Update block version to 1 after HF

* Update to v0.12.0

Co-authored-by: msutton <mikisiton2@gmail.com>
2022-04-12 00:26:44 +03:00
Ori Newman
99bb21c512
Decrement estimatedHeaderUpperBound from mempool's MaxBlockMass (#2009) 2022-04-06 21:53:00 +03:00
Ori Newman
3f840233d8
Fix migrate resolve virtual percents (#2001)
* Fix migration virtual resolution percents

* Don't sync consensus if pruning point is genesis

* Don't add virtual to pruning point proof

* Remove redundant check
2022-04-03 01:01:49 +03:00
Michael Sutton
95fa045297
New definition for "out of sync" (#1996)
* Changed the definition of should mine and redefined "out of sync"

* Check `IsNearlySynced` only if IBD running since IBD check is cheap
2022-04-01 10:36:59 +03:00
Michael Sutton
c4a034eb43
Optimize the miner-kaspad flow and latency (#1988)
* protobuf for new block template notification structs

* appmessage and wire for new block template notification structs

* Set up the entire handler/call-chain for managing the new-block-template event
2022-03-28 23:41:59 +03:00
Michael Sutton
9fa08442cf
Minor log fixes (#1983) 2022-03-21 11:45:52 +02:00
Michael Sutton
0dd50394ec
Fix a bug in the new p2p v5 IBD chain negotiation (#1981)
* Fix a bug in the case where syncer chain is fully known to syncee

* Extract chain negotiation to a separated method

* Bump version and update the changelog

* Add zoom-in progress validation and some debug logs

* Improved error explanation

* go fmt

* Validate zoom-in progress through a total count
2022-03-20 18:06:55 +02:00
Michael Sutton
2ab8065142
Improve output of non-critical protocol errors to avoid user panic (#1978)
* Improve output of non-critical protocol errors to avoid user panic

* Add log messages at the end of IBD with headers proof

* Found a case where this was falsely triggered due to the wrong equality test
2022-03-15 17:23:29 +02:00
Michael Sutton
25410b86ae
Make sure there are no negative numbers in the progress report (#1977) 2022-03-15 11:58:15 +02:00
Michael Sutton
4e44dd8510
Various P2P V5 IBD fixes (#1976)
* The first message is expected to contain headers and not a "done" message (+comment and error text fixes)

* Dequeue w/o timeout during pp anticone batch processing

* Add a verification step for catching possible new IBD errors

* Fetch missing bodies for both, syncer selected tip past and relay block past

* Make sure the syncer is behaving correctly to avoid out of index errors

* Make sure progress reporter does not exceed 100%

* No orphan roots, so no need to queue the empty list

* Add a log to report utxo fetch failure with err message

* A duplicate blocks should not appear as a warning

* typo
2022-03-14 12:21:32 +02:00
Michael Sutton
c81506220b
Send pruning point anticone in batches (#1973)
* add p2p v5 which is currently identical to v4

* set all internal imports to v5

* set default version to 5

* Send pruning point and its anticone in batches

* go lint

* Fix jsom format

* Use DequeueWithTimeout

* Assert that batch size < route capacity

* oops, this is a flow handler, by definition it needs to be w/o a timeout

* here however, a timeout is required

* Keep IDs of prev messages unmodified

* previous merge operation accidentally erased an important part of this pr

* Extend timeout of simple sync

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-03-13 16:31:34 +02:00
Michael Sutton
e5598c15a7
Fix ibd shared past negotiation to be non quadratic also in the worst-case (#1969)
* add p2p v5 which is currently identical to v4

* set all internal imports to v5

* wip

* set default version to 5

* protobuf gen for new ibd chain locator

* wire for new ibd chain locator types

* new ibd shared block algo -- only basic test passing

* address the case where pruning points disagree, now both IBD tests pass

* protobuf gen for new past diff request message

* wire for new request past diff message

* handle and flow for new request past diff message - logic unimplemented yet

* implement ibd sync past diff of relay and selected tip

* go fmt

* remove unused methods

* missed one err check

* addressing simple comments

* apply the traversal limit logic and sort headers

* rename pastdiff -> anticone

* apply Don't relay blocks in virtual anticone #1970 to v5

* go fmt

* Fixed minor comments

* Limit the number of chain negotiation restarts
2022-03-13 11:27:50 +02:00