508 Commits

Author SHA1 Message Date
D-Stacks
f7fcffe6aa save changes 2022-09-13 22:25:50 +02:00
D-Stacks
de5f5518f5
Merge branch 'dev' into add-tx-index 2022-09-02 10:27:01 +02:00
Ori Newman
9f51330f38
Remove tests from docker files (#2133)
* Remove tests from docker files

* Remove unused interface method
2022-09-01 14:14:37 +03:00
D-Stacks
ca01c1f6d6 fic virtual change nil 2022-08-29 18:43:50 +02:00
D-Stacks
576a24a326 update with dev 2022-08-29 15:51:33 +02:00
Ori Newman
3286a7d010
Call update pruning point if required on resolve virtual (#2129)
* Call UpdatePruningPointIfRequired when resolving virtual

* Don't sanity check pruning point UTXO set if it's genesis

* Add UpdatePruningPointIfRequired on init
2022-08-24 13:32:39 +03:00
Elichai Turkel
20b7ab89f9
Add tests for hash writers (#2120)
Co-authored-by: Michael Sutton <mikisiton2@gmail.com>
2022-08-21 09:08:20 +03:00
Ori Newman
10f1e7e3f4
Replace daglabs's dnsseeder with Wolfie's (#2119)
Co-authored-by: Michael Sutton <mikisiton2@gmail.com>
2022-08-21 03:10:59 +03:00
Ori Newman
d941c73701
Change testnet dnsseeder (#2126)
Co-authored-by: Michael Sutton <mikisiton2@gmail.com>
2022-08-21 02:32:40 +03:00
Michael Sutton
266ec6c270
Calculate pruning point utxo set from acceptance data (#2123)
* Calc new pruning point utxo diff through chain acceptance data

* Fix daa score to chain block
2022-08-17 15:56:53 +03:00
Michael Sutton
715cb3b1ac
Fix a subtle lock sync issue in consensus insert block (#2121)
* Manage the locks more tightly and fix a slight and rare sync issue

* Extract virtualResolveChunk constant
2022-08-02 10:33:39 +03:00
D-Stacks
eb693c4a86
Mempool: Retrive stable state of the mempool. optimze get mempool entries by addresses (#2111)
* fix mempool accessing, rewrite get_mempool_entries_by_addresses

* fix counter, add verbose

* fmt

* addresses as string

* Define error in case utxoEntry is missing.

* fix error variable to string

* stop tests from failing (see in code comment)

* access both pools in the same state via parameters

* get rid of todo message

* fmt - very important!

* perf: scriptpublickey in mempool, no txscript.

* address reveiw

* fmt fix

* mixed up isorphan bool, pass tests now

* do map preallocation in mempoolbyaddresses

* no proallocation for orphanpool sending.

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-07-26 23:06:08 +03:00
Michael Sutton
1c9bb54cc2
Crucial fix for the UTXO difference mechanism (#2114)
* Illustrate the bug through prints

* Change consensus API to a single ResolveVirtual call

* nil changeset is not expected when err=nil

* Fixes a deep bug in the resolve virtual process

* Be more defensive at resolving virtual when adding a block

* When finally resolved, set virtual parents properly

* Return nil changeset when nothing happened

* Make sure the block at the split point is reversed to new chain as well

* bump to version 0.12.4

* Avoid locking consensus twice in the common case of adding block with updateVirtual=true

* check err

* Parents must be picked first before set as virtual parents

* Keep the flag for tracking virtual state, since tip sorting perf is high with many tips

* Improve and clarify resolve virtual tests

* Addressing minor review comments

* Fixed a bug in the reported virtual changeset, and modified the test to verify it

* Addressing review comments
2022-07-15 12:35:20 +03:00
D-Stacks
edb6962e40
Merge branch 'dev' into add-tx-index 2022-06-30 00:40:40 +02:00
D-Stacks
18d000f625
Logger: change log level for "Couldn't find UTXO entry" to debug (and ignore message on orphan transactions) (#2108)
* Logger: change log level for "Couldn't find UTXO entry" to debug

* ignore error for orphans

* continue also if orphans

* check if blocks exist

* safe rpc mode

* limit window size

* verify block status depending on context

* allow a 2 factor gap in expected mergeset size

Co-authored-by: msutton <mikisiton2@gmail.com>
2022-06-29 15:54:40 +03:00
D-Stacks
8bacd2e7bf logger to tracef 2022-06-24 22:10:15 +02:00
D-Stacks
600f39ffc6 fix pruning notification 2022-06-24 22:08:24 +02:00
D-Stacks
09b2c53b3f fix not found sending err, add to kaspactl 2022-06-24 21:11:38 +02:00
D-Stacks
560a1a99b1 linting and deal with pruning point commitment 2022-06-24 17:18:49 +02:00
D-Stacks
b4754d4b23 fix cases of nil pointers... 2022-06-24 13:26:16 +02:00
D-Stacks
e059e880c7 include removal logic in txindex 2022-06-24 11:48:13 +02:00
D-Stacks
463db63e22 add minimal rpc for usecase and testing. 2022-06-24 10:52:14 +02:00
D-Stacks
346bd2676e turn pruning point changes into a consensus event 2022-06-23 00:41:14 +02:00
D-Stacks
02326dccef proper naming and fmt / formatting 2022-06-22 08:54:45 +02:00
D-Stacks
d9b5f0ecf1 remove smaller bugs and unused code. 2022-06-21 23:47:07 +02:00
D-Stacks
540d77ff17 remove leftover from copy pasting and bug splattering 2022-06-21 23:45:44 +02:00
D-Stacks
c3393988c1 mostly remove old code from first round. 2022-06-21 23:32:00 +02:00
D-Stacks
345eaac498 preliminary draft 2022-06-21 23:12:06 +02:00
D-Stacks
3a33c0ffa8 starting checkpoint 2022-06-20 16:32:26 +02:00
D-Stacks
74a4f927e9
RPC: include orphans into mempool entries (#2046)
* RPC: include orphans into mempool entries

* no need for + 1

* give request option to choose mempool pool(s)

* add to wallet, fix bg

* use orphanpool rpc to test for orphans

* fix fmt

* fix crash when quering orphan pool in get_mempool_entries

* pass the tests, fix fromAppMessage bug

* Update config_test.go

don't think this is needed

* needed for tests to pass

* inverse to transactionpoolfilter, cut down code to two ifs.

* fmt

* update test to true false, forgot one includetransactionpool renaming

* update and simplyfiy get_mempool_entry handler

* comment outdated

* i think we usually use make instead of var.

* Fix some leftovers of includeTransactionPool

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-06-15 23:46:44 +03:00
Michael Sutton
d957a6d93a
Fix UTXO diff child error (#2084)
* Avoid creating the chain iterator if high hash is actually low hash

* Always use iterator in nextPruningPointAndCandidateByBlockHash

* Initial failing test

* Minimal failing test + some comments

* go lint

* Add simpler tests with two different errors

* Missed some error checks

* Minor

* A workaround patch for preventing the missing utxo child diff bug

* Make sure we fully resolve virtual

* Move ResolveVirtualWithMaxParam to test consensus

* Mark virtual not updated and loop in batches

* Refactor: remove VirtualChangeSet from functions return values

* Remove workaround comments

* If block has no body, virtual is still considered updated

* Remove special error ErrReverseUTXODiffsUTXODiffChildNotFound

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-06-15 02:52:14 +03:00
Michael Sutton
b2648aa5bd
Fix not in selected chain crash (#2082)
* Avoid creating the chain iterator if high hash is actually low hash

* Always use iterator in nextPruningPointAndCandidateByBlockHash

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-06-15 02:32:47 +03:00
D-Stacks
3908f274ae
[Ready] - RPC & UtxoIndex: keep track of, query and test circulating supply. (#2070)
* start

* pass tests, (cannot get before put!) and clean up.

* add rpc call.

* create and pass tests, fix bugs.  fully implement rpc

* As always fmt

* remover old test

* clean up proto comment

* put the logger back in place.

* revert back to 10 sec limit.

* migration, change utxoChanged removal to whole utxoEntryPair, add methods to update circulating supply, intialize circulating supply from reset.

* Update utxoindex.go

* ad

* rename to max, change comment

* one more total to max

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-06-05 16:42:08 +03:00
Michael Sutton
d50ad0667c
Unite multiple GetBlockAcceptanceData consensus calls to one (#2074)
* Unite multiple `GetBlockAcceptanceData` consensus calls to one

* Variable rename
2022-06-01 12:19:21 +03:00
D-Stacks
052193865e
populate with verbos (#2064)
Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: Michael Sutton <mikisiton2@gmail.com>
2022-05-25 18:23:44 +03:00
D-Stacks
a4d9fa10bf
fix typo (#2060)
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-05-25 10:17:58 +03:00
Svarog
24c94b38be
Move OnBlockAdded event to the channel that was only used by virtualChanged events (#2059)
* Move OnBlockAdded event to the channel that was only used by virtualChanged events

* Don't send notifications for header-only and invalid blocks

* Return block status from block processor and use it for event raising decision

* Use MaybeEnqueue for consensus events

* go lint

* Fix RPC call to actually include tx ids

Co-authored-by: msutton <mikisiton2@gmail.com>
2022-05-24 01:11:01 +03:00
Ori Newman
4dd7113dc5
Increase virtualChangeChan to 100e3 (#2056)
* Increase virtualChangeChan to 100e3
Don't crash when sending UTXO RPC notification to a closed route
Throw error if virtualChangeChan is full

* Use MaybeEnqueue in more places

* Remove comment

* Ignore capacity reached errors on MaybeEnqueue
2022-05-20 19:31:13 +03:00
Michael Sutton
016ddfdfce
Use a channel for utxo change events (#2052)
* Use a channel from within consensus in order to raise change events in order -- note that this is only a draft commit for discussion

* Fix compilation

* Check for nil

* Allow nil virtualChangeChan

* Remove redundant comments

* Call notifyVirtualChange instead of notifyUTXOsChanged

* Remove redundant comment

* Add a separate function for initVirtualChangeHandler

* Remove redundant type

* Check for nil in the right place

* Fix integration test

* Add data to virtual changeset and cleanup block added event logic

* Renames

* Comment

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-05-19 14:07:48 +03:00
Svarog
5d24e2afbc
Allow blank address in NotifyUTXOsChanged to get all updates (#2027)
* Allow blank address in NotifyUTXOsChanged to get all updates

* To see if address is possible to extract: Check for NonStandardTy rather than error

* Don't swallow errors

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-05-18 22:28:33 +03:00
biospb
8735da045f
Block template cache improvement (#2023)
* Block template cache improvement

* Avoid concurrent calls to template builder
* Clear cache on new block event
* Move IsNearlySynced logic to within consensus and cache it for each template 
* Use a single consensus call for building template and checking synced

Co-authored-by: msutton <mikisiton2@gmail.com>
2022-05-06 11:36:07 +03:00
Ori Newman
c839337425
Add finality check to ResolveVirtual (#2041)
* Add finality check to ResolveVirtual

* Warn on finality conflict only if needed
2022-05-06 00:29:48 +03:00
Ori Newman
7390651072
Remove HF1 activation code (#2042)
* Remove HF1 activation code

* Remove test with an overflow

* Fix "max sompi amount is never dust"

* Remove estimatedHeaderUpperBound logic
2022-05-05 20:35:17 +03:00
biospb
1660cf0cf1
Improved staging shard performance (#2034)
using map instead of growing array as shard id can be 1000+ in some cases

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-05-04 21:24:25 +03:00
Ori Newman
35a959b56f
Making a workaround for the UTXO diff child bug (#2020)
* Making a workaround for the UTXO diff child bug

* Fix error message

* Fix error message
2022-04-14 12:55:20 +03:00
D-Stacks
57c6118be8
Adds a "sweep" command to kaspawallet (#2018)
* Add "sweep" command to kaspawallet

* minor linting & layout improvements.

* contain code in "sweep"

* update to sweep.go

* formatting and bugs

* clean up renaming of ExtractTransaction functions

* Nicer formating of display

* address reveiw.

* one more sweeped -> swept

* missed one reveiw comment. (extra mass).

* remove redundant code from split function.

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-04-14 02:16:16 +03:00
Ori Newman
2b395e34b1
Use separate depth than finality depth for merge set calculations after HF (#2013)
* Use separate than finality depth for merge set calculations after HF

* Add comments and edit error messages

* Fix TestValidateTransactionInContextAndPopulateFee

* Don't disconnect from node if isViolatingBoundedMergeDepth

* Use new merge root for virtual pick parents; apply HF1 daa score split for validation only

* Use `blue work` heuristic to skip irrelevant relay blocks

* Minor

* Make sure virtual's merge depth root is a real block

* For ghostdag data we always use the non-trusted data

* Fix TestBoundedMergeDepth and in IBD use VirtualMergeDepthRoot instead of MergeDepthRoot

* Update HF1DAAScore

* Make sure merge root and finality are called + avoid calculating virtual root twice

* Update block version to 1 after HF

* Update to v0.12.0

Co-authored-by: msutton <mikisiton2@gmail.com>
2022-04-12 00:26:44 +03:00
Ori Newman
99bb21c512
Decrement estimatedHeaderUpperBound from mempool's MaxBlockMass (#2009) 2022-04-06 21:53:00 +03:00
Michael Sutton
80879cabe1
Clean up debug log level by moving many frequent logs to trace level (#2004)
* Clean up debug log level and move most/frequent logs to trace level

* Change function call traces to trace log level
2022-04-03 23:25:29 +03:00
Ori Newman
71afc62298
Avoid override and accidental staging of pruning point by index (#2005)
* Avoid override and accidental staging of pruning point by index

* Update pruning point when resolving virtual
2022-04-03 22:52:05 +03:00