476 Commits

Author SHA1 Message Date
D-Stacks
3908f274ae
[Ready] - RPC & UtxoIndex: keep track of, query and test circulating supply. (#2070)
* start

* pass tests, (cannot get before put!) and clean up.

* add rpc call.

* create and pass tests, fix bugs.  fully implement rpc

* As always fmt

* remover old test

* clean up proto comment

* put the logger back in place.

* revert back to 10 sec limit.

* migration, change utxoChanged removal to whole utxoEntryPair, add methods to update circulating supply, intialize circulating supply from reset.

* Update utxoindex.go

* ad

* rename to max, change comment

* one more total to max

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-06-05 16:42:08 +03:00
Michael Sutton
d50ad0667c
Unite multiple GetBlockAcceptanceData consensus calls to one (#2074)
* Unite multiple `GetBlockAcceptanceData` consensus calls to one

* Variable rename
2022-06-01 12:19:21 +03:00
D-Stacks
052193865e
populate with verbos (#2064)
Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: Michael Sutton <mikisiton2@gmail.com>
2022-05-25 18:23:44 +03:00
D-Stacks
a4d9fa10bf
fix typo (#2060)
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-05-25 10:17:58 +03:00
Svarog
24c94b38be
Move OnBlockAdded event to the channel that was only used by virtualChanged events (#2059)
* Move OnBlockAdded event to the channel that was only used by virtualChanged events

* Don't send notifications for header-only and invalid blocks

* Return block status from block processor and use it for event raising decision

* Use MaybeEnqueue for consensus events

* go lint

* Fix RPC call to actually include tx ids

Co-authored-by: msutton <mikisiton2@gmail.com>
2022-05-24 01:11:01 +03:00
Ori Newman
4dd7113dc5
Increase virtualChangeChan to 100e3 (#2056)
* Increase virtualChangeChan to 100e3
Don't crash when sending UTXO RPC notification to a closed route
Throw error if virtualChangeChan is full

* Use MaybeEnqueue in more places

* Remove comment

* Ignore capacity reached errors on MaybeEnqueue
2022-05-20 19:31:13 +03:00
Michael Sutton
016ddfdfce
Use a channel for utxo change events (#2052)
* Use a channel from within consensus in order to raise change events in order -- note that this is only a draft commit for discussion

* Fix compilation

* Check for nil

* Allow nil virtualChangeChan

* Remove redundant comments

* Call notifyVirtualChange instead of notifyUTXOsChanged

* Remove redundant comment

* Add a separate function for initVirtualChangeHandler

* Remove redundant type

* Check for nil in the right place

* Fix integration test

* Add data to virtual changeset and cleanup block added event logic

* Renames

* Comment

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-05-19 14:07:48 +03:00
Svarog
5d24e2afbc
Allow blank address in NotifyUTXOsChanged to get all updates (#2027)
* Allow blank address in NotifyUTXOsChanged to get all updates

* To see if address is possible to extract: Check for NonStandardTy rather than error

* Don't swallow errors

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-05-18 22:28:33 +03:00
biospb
8735da045f
Block template cache improvement (#2023)
* Block template cache improvement

* Avoid concurrent calls to template builder
* Clear cache on new block event
* Move IsNearlySynced logic to within consensus and cache it for each template 
* Use a single consensus call for building template and checking synced

Co-authored-by: msutton <mikisiton2@gmail.com>
2022-05-06 11:36:07 +03:00
Ori Newman
c839337425
Add finality check to ResolveVirtual (#2041)
* Add finality check to ResolveVirtual

* Warn on finality conflict only if needed
2022-05-06 00:29:48 +03:00
Ori Newman
7390651072
Remove HF1 activation code (#2042)
* Remove HF1 activation code

* Remove test with an overflow

* Fix "max sompi amount is never dust"

* Remove estimatedHeaderUpperBound logic
2022-05-05 20:35:17 +03:00
biospb
1660cf0cf1
Improved staging shard performance (#2034)
using map instead of growing array as shard id can be 1000+ in some cases

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-05-04 21:24:25 +03:00
Ori Newman
35a959b56f
Making a workaround for the UTXO diff child bug (#2020)
* Making a workaround for the UTXO diff child bug

* Fix error message

* Fix error message
2022-04-14 12:55:20 +03:00
D-Stacks
57c6118be8
Adds a "sweep" command to kaspawallet (#2018)
* Add "sweep" command to kaspawallet

* minor linting & layout improvements.

* contain code in "sweep"

* update to sweep.go

* formatting and bugs

* clean up renaming of ExtractTransaction functions

* Nicer formating of display

* address reveiw.

* one more sweeped -> swept

* missed one reveiw comment. (extra mass).

* remove redundant code from split function.

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-04-14 02:16:16 +03:00
Ori Newman
2b395e34b1
Use separate depth than finality depth for merge set calculations after HF (#2013)
* Use separate than finality depth for merge set calculations after HF

* Add comments and edit error messages

* Fix TestValidateTransactionInContextAndPopulateFee

* Don't disconnect from node if isViolatingBoundedMergeDepth

* Use new merge root for virtual pick parents; apply HF1 daa score split for validation only

* Use `blue work` heuristic to skip irrelevant relay blocks

* Minor

* Make sure virtual's merge depth root is a real block

* For ghostdag data we always use the non-trusted data

* Fix TestBoundedMergeDepth and in IBD use VirtualMergeDepthRoot instead of MergeDepthRoot

* Update HF1DAAScore

* Make sure merge root and finality are called + avoid calculating virtual root twice

* Update block version to 1 after HF

* Update to v0.12.0

Co-authored-by: msutton <mikisiton2@gmail.com>
2022-04-12 00:26:44 +03:00
Ori Newman
99bb21c512
Decrement estimatedHeaderUpperBound from mempool's MaxBlockMass (#2009) 2022-04-06 21:53:00 +03:00
Michael Sutton
80879cabe1
Clean up debug log level by moving many frequent logs to trace level (#2004)
* Clean up debug log level and move most/frequent logs to trace level

* Change function call traces to trace log level
2022-04-03 23:25:29 +03:00
Ori Newman
71afc62298
Avoid override and accidental staging of pruning point by index (#2005)
* Avoid override and accidental staging of pruning point by index

* Update pruning point when resolving virtual
2022-04-03 22:52:05 +03:00
Michael Sutton
ca5c8549b9
Add DB compaction following the deletion of a DB prefix (#2003) 2022-04-03 19:08:32 +03:00
Michael Sutton
ab73def07a
Cache the pruning point anticone (#2002) 2022-04-03 14:44:42 +03:00
Ori Newman
3f840233d8
Fix migrate resolve virtual percents (#2001)
* Fix migration virtual resolution percents

* Don't sync consensus if pruning point is genesis

* Don't add virtual to pruning point proof

* Remove redundant check
2022-04-03 01:01:49 +03:00
Michael Sutton
21b82d7efc
Block template cache (#1994)
* minor text fix

* Implement a block template cache with template modification/reuse mechanism

* Fix compilation error

* Address review comments

* Added a through TestModifyBlockTemplate test

* Update header timestamp if possible

* Avoid copying the transactions when only the header changed

* go fmt
2022-03-31 16:37:48 +03:00
Ori Newman
63c6d7443b
In blockParentBuilder.BuildParents check if a block is isInFutureOfVi… (#1993)
* In blockParentBuilder.BuildParents check if a block is isInFutureOfVirtualGenesisChildren instead of checking if it has reachability data

* Add comment
2022-03-31 02:43:07 +03:00
Ori Newman
58d627e05a
Unite reachability stores (#1963)
* Unite all reachability stores

* Upgrade script

* Fix tests

* Add UpdateReindexRoot to RebuildReachability

* Use dbTx when deleting reachability stores

* Use ghostdagDataWithoutPrunedBlocks when rebuilding reachability

* Use next tree ancestor wherever possible and avoid finality point search if the block is too close to pruning point

* Address the boundary case where the pruning point becomes the finality point

* some minor fixes

* Remove RebuildReachability and use manual syncing between old and new consensus for migration

* Remove sanity test (it failed when tips where not in the same order)

Co-authored-by: msutton <mikisiton2@gmail.com>
2022-03-27 21:23:03 +03:00
Svarog
639183ba0e
Add support for auto-compound in kaspawallet send (#1951)
* Add GetUTXOsByBalances command to rpc

* Fix wrong commands in GetBalanceByAddress

* Moved calculation of TransactionMass out of TransactionValidator, so t that it can be used in kaspawallet

* Allow CreateUnsignedTransaction to return multiple transactions

* Start working on split

* Implement maybeSplitTransactionInner

* estimateMassIncreaseForSignatures should multiply by the number of inputs

* Implement createSplitTransaction

* Implement mergeTransactions

* Broadcast all transaction, not only 1

* workaround missing UTXOEntry in partially signed transaction

* Bugfix in broadcast loop

* Add underscores in some constants

* Make all nets RelayNonStdTxs: false

* Change estimateMassIncreaseForSignatures to estimateMassAfterSignatures

* Allow situations where merge transaction doesn't have enough funds to pay fees

* Add comments

* A few of renames

* Handle missed errors

* Fix clone of PubKeySignaturePair  to properly clone nil signatures

* Add sanity check to make sure originalTransaction has exactly two outputs

* Re-use change address for splitAddress

* Add one more utxo if the total amount is smaller then what we need to send due to fees

* Fix off-by-1 error in splitTrasnaction

* Add a comment to maybeAutoCompoundTransaction

* Add comment on why we are increasing inputCountPerSplit

* Add comment explaining while originalTransaction has 1 or 2 outputs

* Move oneMoreUTXOForMergeTransaction to split_transaction.go

* Allow to add multiple utxos to pay fee for mergeTransactions, if needed

* calculate split input counts and sizes properly

* Allow multiple transactions inside the create-unsigned-transaction -> sign -> broadcast workflow

* Print the number of transaction which was sent, in case there were multiple

* Rename broadcastConfig.Transaction(File) to Transactions(File)

* Convert alreadySelectedUTXOs to a map

* Fix a typo

* Add comment explaining that we assume all inputs are the same

* Revert over-refactor of rename of config.Transaction -> config.Transactions

* Rename: inputPerSplitCount -> inputsPerSplitCount

* Add comment for splitAndInputPerSplitCounts

* Use createSplitTransaction to calculate the upper bound of mass for split transactions
2022-03-27 20:06:55 +03:00
Michael Sutton
9fa08442cf
Minor log fixes (#1983) 2022-03-21 11:45:52 +02:00
Michael Sutton
2ab8065142
Improve output of non-critical protocol errors to avoid user panic (#1978)
* Improve output of non-critical protocol errors to avoid user panic

* Add log messages at the end of IBD with headers proof

* Found a case where this was falsely triggered due to the wrong equality test
2022-03-15 17:23:29 +02:00
Svarog
1e56a22b32
Ignore not found errors from tp.transactionsOrderedByFeeRate.Remove. (#1974)
* Fix error message referencing wrong function name

* Ignore not found errors from tp.transactionsOrderedByFeeRate.Remove.

Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2022-03-13 18:01:13 +02:00
Ori Newman
7a95f0c7a4
Use nil suggestedLowHash if selected parent pruning point is not in the future of the current one (#1972)
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2022-03-13 17:22:03 +02:00
Michael Sutton
c81506220b
Send pruning point anticone in batches (#1973)
* add p2p v5 which is currently identical to v4

* set all internal imports to v5

* set default version to 5

* Send pruning point and its anticone in batches

* go lint

* Fix jsom format

* Use DequeueWithTimeout

* Assert that batch size < route capacity

* oops, this is a flow handler, by definition it needs to be w/o a timeout

* here however, a timeout is required

* Keep IDs of prev messages unmodified

* previous merge operation accidentally erased an important part of this pr

* Extend timeout of simple sync

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-03-13 16:31:34 +02:00
Michael Sutton
e5598c15a7
Fix ibd shared past negotiation to be non quadratic also in the worst-case (#1969)
* add p2p v5 which is currently identical to v4

* set all internal imports to v5

* wip

* set default version to 5

* protobuf gen for new ibd chain locator

* wire for new ibd chain locator types

* new ibd shared block algo -- only basic test passing

* address the case where pruning points disagree, now both IBD tests pass

* protobuf gen for new past diff request message

* wire for new request past diff message

* handle and flow for new request past diff message - logic unimplemented yet

* implement ibd sync past diff of relay and selected tip

* go fmt

* remove unused methods

* missed one err check

* addressing simple comments

* apply the traversal limit logic and sort headers

* rename pastdiff -> anticone

* apply Don't relay blocks in virtual anticone #1970 to v5

* go fmt

* Fixed minor comments

* Limit the number of chain negotiation restarts
2022-03-13 11:27:50 +02:00
Svarog
433af5e0fe
Make findTransactionIndex return wasFound explicitly + fix crash caused by invalid handling of not found transaction (#1971)
* Make findTransactionIndex return wasFound explicitly + fix crash caused by invalid handling of not found transaction

* Add comment on findTransactionIndex
2022-03-12 11:07:10 +02:00
Ori Newman
061e65be93
Fix argument order for IsAncestorOf in boundedMergeBreakingParents (#1966) 2022-03-09 21:11:00 +02:00
Ori Newman
190e725dd0
Optimize expected header pruning point (#1962)
* Use the correct heuristic to avoid checking for next pruning point movement when not needed
2022-03-07 00:16:29 +02:00
Ori Newman
9f02a24e8b
Add merge set and IsChainBlock to the RPC (#1961)
* Add merge set and IsChainBlock to the RPC

* Fix BlockInfo.Clone()
2022-02-25 16:22:00 +02:00
Ori Newman
1c18a49992
Add cache to block window (#1948)
* Add cache to block window

* Copy the window heap slice with the right capacity

* Use WindowHeapSliceStore

* Use the selected parent window as a basis (and some comments and variable renames)

* Clone slice on newSizedUpHeapFromSlice

* Rename isNotFoundError->currentIsNonTrustedBlock

* Increase windowHeapSliceStore cache size to 2000 and some cosmetic changes
2022-02-20 16:52:36 +02:00
stasatdaglabs
28d0f1ea2e
Set MaxBlockLevels for non-mainnet networks to 250 (#1952)
* Make MaxBlockLevel a DAG params instead of a constant.

* Change the testnet network name to 9.

* Fix TestBlockWindow.

* Set MaxBlockLevels for non-mainnet networks to 250.

* Revert "Fix TestBlockWindow."

This reverts commit 30a7892f53e0bb8d0d24435a68f0561a8efab575.

* Fix TestPruning.
2022-02-20 13:43:42 +02:00
Ori Newman
ce4f5fcc33
Remove BlockWithTrustedData method (#1950)
* Remove BlockWithTrustedData method

* Remove unused methods
2022-02-20 00:04:47 +02:00
Svarog
f452531df0
Fix bookkeeping of chained transactions in mempool (#1946)
* Fix bookkeeping of chained transactions in mempool

* Fix golint error
2022-02-18 13:21:45 +02:00
Ori Newman
69d90fe827
Remove duplicate median time calculation on tx validation (#1943)
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2022-02-06 16:34:05 +02:00
Ori Newman
1cd712a63e
Increase headers proof timeout and add more progress logs (#1939)
* Increase timeout for pruning proof and add some logs

* Show resolving virtual progress as whole percents
2022-02-06 12:42:28 +02:00
stasatdaglabs
27ba9d0374
Split ApplyPruningPointProof to multiple small database transactions (#1937)
* Split ApplyPruningPointProof to multiple small database transactions.

* Increase the timeout duration in TestIBDWithPruning.

* Increase the timeout duration in simple-sync.

* Explain that if ApplyPruningPointProof fails, the database must be discarded.
2022-02-06 10:36:27 +02:00
Michael Sutton
dab1a881fe
Fix for rare consensus bug regarding daa window order (#1934)
* Fix for rare consensus bug: daa window min-time-block was not deterministic when timestamps are equal

* Something is missing

* Extract compare logic to a function with better performance

* typo
2022-01-27 20:29:44 +02:00
Ori Newman
dadacdc0f4
Filter redundant blocks from daa window (#1925)
* Copy blockrelay flows to v4

* Remove duplicate sending of the same DAA blocks

* Advance testnet version

* Renames and add comments

* Add IBD test between v3 and v4

* Fix syncee v4 p2p version

* Check if netsync finished with selected tip
2022-01-09 16:58:51 +02:00
Ori Newman
d2379608ad
P2P upgrade mechanism (#1921)
* Implement upgrade mechanism for p2p

* Remove dependencies from flowcontext to v3

* Add p2p v4

* Add Ready flow

* Remove copy paste code of v3

* Register SendAddresses flow at the top level

* Add option to set protocol version from CLI and add TestAddressExchangeV3V4

* Send ready message on minimal net adapter

* Rename defaultMaxProtocolVersion->maxAcceptableProtocolVersion
2022-01-09 09:59:45 +02:00
Michael Sutton
504ec36612
Remove DNS seeder managed by Elichai Turkel (its offline) (#1918) 2021-12-30 10:10:57 +02:00
Ori Newman
c80b113319
Fix two pruning proof bugs (#1913)
* Don't assume pruning point is at specific level

* Reverse pruning points order in ArePruningPointsViolatingFinality

Co-authored-by: Michael Sutton <mikisiton2@gmail.com>
2021-12-29 21:24:29 +02:00
stasatdaglabs
0bdd19136f
Implement the new monetary policy (#1892)
* Remove unused functions, constants, and variables.

* Rename maxSubsidy to baseSubsidy.

* Remove unused parameters from CalcBlockSubsidy.

* Remove link to old monetary policy.

* If a block's DAA score is smaller than half a year, it should have a base subsidy.

* Fix merge errors.

* Fix more merge errors.

* Add DeflationaryPhaseBaseSubsidy to the params.

* Implement TestCalcDeflationaryPeriodBlockSubsidy.

* Implement calcDeflationaryPeriodBlockSubsidy naively.

* Implement calcDeflationaryPeriodBlockSubsidy not naively.

* Adjust the subsidy based on target block rate.

* Fix deflationaryPhaseDaaScore in TestCalcDeflationaryPeriodBlockSubsidy.

* Explain how secondsPerMonth is calculated.

* Don't adjust the subsidy based on the target block rate.

* Update defaultDeflationaryPhaseDaaScore and add an explanation.

* Use a pre-calculated table for subsidy per month

* Make the generation function fail if base subsidy is changed

* go fmt

* Use test logger for printing + simplify print loop

Co-authored-by: msutton <mikisiton2@gmail.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-12-29 20:49:20 +02:00
Aleoami
f54659ead0
Add kaspadns.kaspacalc.net to seeder list (#1910)
* Update to version v0.11.8

* bugfix: addresses issue #1903, raised kaspawallet bruteforce core limit to 256

* fix contributing.md text

* added dns seeder kaspadns.kaspacalc.net

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-12-25 13:22:44 +02:00
georges-kuenzli
aa43c14fc5
Added seeder[1-4].kaspad.net to seeder list (#1901)
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-12-23 00:42:17 +02:00