4689 Commits

Author SHA1 Message Date
Ori Newman
90d9edb8e5
Bump version to v0.11.15 (#1999)
Co-authored-by: Michael Sutton <mikisiton2@gmail.com>
2022-04-02 22:20:41 +03:00
Ori Newman
b9b360bce4
Remove increase pagefile (#2000)
Co-authored-by: Michael Sutton <mikisiton2@gmail.com>
2022-04-02 22:12:57 +03:00
Ori Newman
27654961f9
Remove v4 p2p version (#1998)
Co-authored-by: Michael Sutton <mikisiton2@gmail.com>
2022-04-02 21:54:40 +03:00
Ori Newman
d45af760d8
Validate GetBlockTemplate extra data (#1997)
* Validate GetBlockTemplate extra data

* Check payload instead of extra data

* Fix error message
2022-04-02 21:25:58 +03:00
Michael Sutton
95fa045297
New definition for "out of sync" (#1996)
* Changed the definition of should mine and redefined "out of sync"

* Check `IsNearlySynced` only if IBD running since IBD check is cheap
2022-04-01 10:36:59 +03:00
Ori Newman
cb65dae63d
Add extra data to GetBlockTemplate request (#1995) 2022-03-31 20:12:33 +03:00
Michael Sutton
21b82d7efc
Block template cache (#1994)
* minor text fix

* Implement a block template cache with template modification/reuse mechanism

* Fix compilation error

* Address review comments

* Added a through TestModifyBlockTemplate test

* Update header timestamp if possible

* Avoid copying the transactions when only the header changed

* go fmt
2022-03-31 16:37:48 +03:00
Ori Newman
63c6d7443b
In blockParentBuilder.BuildParents check if a block is isInFutureOfVi… (#1993)
* In blockParentBuilder.BuildParents check if a block is isInFutureOfVirtualGenesisChildren instead of checking if it has reachability data

* Add comment
2022-03-31 02:43:07 +03:00
Svarog
753f4a2ec1
Add package name to kaspawalletd .proto file (#1991)
Co-authored-by: Michael Sutton <mikisiton2@gmail.com>
2022-03-30 20:56:36 +03:00
Svarog
ed667f7e54
Upgrade to go 1.18 (#1992)
* Upgrade to go 1.18

* Fix stability-test shell script
2022-03-30 20:17:29 +03:00
Michael Sutton
c4a034eb43
Optimize the miner-kaspad flow and latency (#1988)
* protobuf for new block template notification structs

* appmessage and wire for new block template notification structs

* Set up the entire handler/call-chain for managing the new-block-template event
2022-03-28 23:41:59 +03:00
Aleoami
2eca0f0b5f
Add names to nameless routes (#1986)
* add names to nameless routes

* Update router.go

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-03-28 09:41:25 +03:00
Ori Newman
58d627e05a
Unite reachability stores (#1963)
* Unite all reachability stores

* Upgrade script

* Fix tests

* Add UpdateReindexRoot to RebuildReachability

* Use dbTx when deleting reachability stores

* Use ghostdagDataWithoutPrunedBlocks when rebuilding reachability

* Use next tree ancestor wherever possible and avoid finality point search if the block is too close to pruning point

* Address the boundary case where the pruning point becomes the finality point

* some minor fixes

* Remove RebuildReachability and use manual syncing between old and new consensus for migration

* Remove sanity test (it failed when tips where not in the same order)

Co-authored-by: msutton <mikisiton2@gmail.com>
2022-03-27 21:23:03 +03:00
Svarog
639183ba0e
Add support for auto-compound in kaspawallet send (#1951)
* Add GetUTXOsByBalances command to rpc

* Fix wrong commands in GetBalanceByAddress

* Moved calculation of TransactionMass out of TransactionValidator, so t that it can be used in kaspawallet

* Allow CreateUnsignedTransaction to return multiple transactions

* Start working on split

* Implement maybeSplitTransactionInner

* estimateMassIncreaseForSignatures should multiply by the number of inputs

* Implement createSplitTransaction

* Implement mergeTransactions

* Broadcast all transaction, not only 1

* workaround missing UTXOEntry in partially signed transaction

* Bugfix in broadcast loop

* Add underscores in some constants

* Make all nets RelayNonStdTxs: false

* Change estimateMassIncreaseForSignatures to estimateMassAfterSignatures

* Allow situations where merge transaction doesn't have enough funds to pay fees

* Add comments

* A few of renames

* Handle missed errors

* Fix clone of PubKeySignaturePair  to properly clone nil signatures

* Add sanity check to make sure originalTransaction has exactly two outputs

* Re-use change address for splitAddress

* Add one more utxo if the total amount is smaller then what we need to send due to fees

* Fix off-by-1 error in splitTrasnaction

* Add a comment to maybeAutoCompoundTransaction

* Add comment on why we are increasing inputCountPerSplit

* Add comment explaining while originalTransaction has 1 or 2 outputs

* Move oneMoreUTXOForMergeTransaction to split_transaction.go

* Allow to add multiple utxos to pay fee for mergeTransactions, if needed

* calculate split input counts and sizes properly

* Allow multiple transactions inside the create-unsigned-transaction -> sign -> broadcast workflow

* Print the number of transaction which was sent, in case there were multiple

* Rename broadcastConfig.Transaction(File) to Transactions(File)

* Convert alreadySelectedUTXOs to a map

* Fix a typo

* Add comment explaining that we assume all inputs are the same

* Revert over-refactor of rename of config.Transaction -> config.Transactions

* Rename: inputPerSplitCount -> inputsPerSplitCount

* Add comment for splitAndInputPerSplitCounts

* Use createSplitTransaction to calculate the upper bound of mass for split transactions
2022-03-27 20:06:55 +03:00
Michael Sutton
9fa08442cf
Minor log fixes (#1983) v0.11.14 2022-03-21 11:45:52 +02:00
Michael Sutton
0dd50394ec
Fix a bug in the new p2p v5 IBD chain negotiation (#1981)
* Fix a bug in the case where syncer chain is fully known to syncee

* Extract chain negotiation to a separated method

* Bump version and update the changelog

* Add zoom-in progress validation and some debug logs

* Improved error explanation

* go fmt

* Validate zoom-in progress through a total count
v0.11.14-rc1
2022-03-20 18:06:55 +02:00
Michael Sutton
ac8d4e1341
Update changelog with v0.11.13 changes (#1980) v0.11.13 2022-03-16 10:48:11 +02:00
Michael Sutton
2488fbde78
Apply avoiding IBD patch10 logic to p2p v4 (#1979)
* a patch for fixing p2p v4 IBD issues for all side-chains

* Perform side-chain check earlier to avoid IBD start

* A few comments explaining the IBD patch
2022-03-15 19:16:25 +02:00
Michael Sutton
2ab8065142
Improve output of non-critical protocol errors to avoid user panic (#1978)
* Improve output of non-critical protocol errors to avoid user panic

* Add log messages at the end of IBD with headers proof

* Found a case where this was falsely triggered due to the wrong equality test
2022-03-15 17:23:29 +02:00
Michael Sutton
25410b86ae
Make sure there are no negative numbers in the progress report (#1977) 2022-03-15 11:58:15 +02:00
Michael Sutton
4e44dd8510
Various P2P V5 IBD fixes (#1976)
* The first message is expected to contain headers and not a "done" message (+comment and error text fixes)

* Dequeue w/o timeout during pp anticone batch processing

* Add a verification step for catching possible new IBD errors

* Fetch missing bodies for both, syncer selected tip past and relay block past

* Make sure the syncer is behaving correctly to avoid out of index errors

* Make sure progress reporter does not exceed 100%

* No orphan roots, so no need to queue the empty list

* Add a log to report utxo fetch failure with err message

* A duplicate blocks should not appear as a warning

* typo
2022-03-14 12:21:32 +02:00
Svarog
1e56a22b32
Ignore not found errors from tp.transactionsOrderedByFeeRate.Remove. (#1974)
* Fix error message referencing wrong function name

* Ignore not found errors from tp.transactionsOrderedByFeeRate.Remove.

Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2022-03-13 18:01:13 +02:00
Ori Newman
7a95f0c7a4
Use nil suggestedLowHash if selected parent pruning point is not in the future of the current one (#1972)
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2022-03-13 17:22:03 +02:00
Michael Sutton
c81506220b
Send pruning point anticone in batches (#1973)
* add p2p v5 which is currently identical to v4

* set all internal imports to v5

* set default version to 5

* Send pruning point and its anticone in batches

* go lint

* Fix jsom format

* Use DequeueWithTimeout

* Assert that batch size < route capacity

* oops, this is a flow handler, by definition it needs to be w/o a timeout

* here however, a timeout is required

* Keep IDs of prev messages unmodified

* previous merge operation accidentally erased an important part of this pr

* Extend timeout of simple sync

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-03-13 16:31:34 +02:00
Michael Sutton
e5598c15a7
Fix ibd shared past negotiation to be non quadratic also in the worst-case (#1969)
* add p2p v5 which is currently identical to v4

* set all internal imports to v5

* wip

* set default version to 5

* protobuf gen for new ibd chain locator

* wire for new ibd chain locator types

* new ibd shared block algo -- only basic test passing

* address the case where pruning points disagree, now both IBD tests pass

* protobuf gen for new past diff request message

* wire for new request past diff message

* handle and flow for new request past diff message - logic unimplemented yet

* implement ibd sync past diff of relay and selected tip

* go fmt

* remove unused methods

* missed one err check

* addressing simple comments

* apply the traversal limit logic and sort headers

* rename pastdiff -> anticone

* apply Don't relay blocks in virtual anticone #1970 to v5

* go fmt

* Fixed minor comments

* Limit the number of chain negotiation restarts
2022-03-13 11:27:50 +02:00
Svarog
433af5e0fe
Make findTransactionIndex return wasFound explicitly + fix crash caused by invalid handling of not found transaction (#1971)
* Make findTransactionIndex return wasFound explicitly + fix crash caused by invalid handling of not found transaction

* Add comment on findTransactionIndex
2022-03-12 11:07:10 +02:00
Ori Newman
b7be807167
Don't relay blocks in virtual anticone (#1970) 2022-03-11 13:24:45 +02:00
Ori Newman
e687ceeae7
Add version to block template (#1967) 2022-03-11 08:56:36 +02:00
Ori Newman
04e35321aa
Bump to v0.11.13 (#1968) 2022-03-11 08:33:53 +02:00
Ori Newman
061e65be93
Fix argument order for IsAncestorOf in boundedMergeBreakingParents (#1966) 2022-03-09 21:11:00 +02:00
Ori Newman
190e725dd0
Optimize expected header pruning point (#1962)
* Use the correct heuristic to avoid checking for next pruning point movement when not needed
2022-03-07 00:16:29 +02:00
Ori Newman
6449b03034
Ignore transaction invs on IBD (#1960)
* Ignore transaction invs on IBD

* Add IsIBDRunning mock to TestHandleRelayedTransactionsNotFound

Co-authored-by: Ori Newman <>
2022-02-26 22:20:08 +02:00
Ori Newman
9f02a24e8b
Add merge set and IsChainBlock to the RPC (#1961)
* Add merge set and IsChainBlock to the RPC

* Fix BlockInfo.Clone()
2022-02-25 16:22:00 +02:00
Isaac Cook
9b23bbcdb5
kaspactl: string slice deser for GetUtxosByAddresses (#1955)
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-02-24 00:40:01 +02:00
stasatdaglabs
b30f7309a2
Implement a parse sub command in the walllet (#1953)
* Add boilerplate for the `parse` sub command.

* Deserialize the given transaction hax.

* Implement the rest of the wallet parse command.

* Hide transaction inputs behind a `verbose` flag.

* Indicate that we aren't able to extract an address out of a nonstandard transaction.

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-02-20 22:12:23 +02:00
Ori Newman
1c18a49992
Add cache to block window (#1948)
* Add cache to block window

* Copy the window heap slice with the right capacity

* Use WindowHeapSliceStore

* Use the selected parent window as a basis (and some comments and variable renames)

* Clone slice on newSizedUpHeapFromSlice

* Rename isNotFoundError->currentIsNonTrustedBlock

* Increase windowHeapSliceStore cache size to 2000 and some cosmetic changes
v0.11.12-testnet3 v0.11.12-testnet2
2022-02-20 16:52:36 +02:00
stasatdaglabs
28d0f1ea2e
Set MaxBlockLevels for non-mainnet networks to 250 (#1952)
* Make MaxBlockLevel a DAG params instead of a constant.

* Change the testnet network name to 9.

* Fix TestBlockWindow.

* Set MaxBlockLevels for non-mainnet networks to 250.

* Revert "Fix TestBlockWindow."

This reverts commit 30a7892f53e0bb8d0d24435a68f0561a8efab575.

* Fix TestPruning.
2022-02-20 13:43:42 +02:00
Ori Newman
3f7e482291
Add progress indication for virtual resolution (#1949)
* Add progress indication for virtual resolution

* Avoid division by zero
2022-02-20 00:32:41 +02:00
Ori Newman
ce4f5fcc33
Remove BlockWithTrustedData method (#1950)
* Remove BlockWithTrustedData method

* Remove unused methods
2022-02-20 00:04:47 +02:00
Svarog
be3a6604d7
Make kaspawallet store the utxos sorted by amount (#1947)
* Make kaspawallet store the utxos sorted by amount, so that the bigger utxos are spent first - making it less likely a compound will be required

* Start refactor addEntryToUTXOSet

* Add GetUTXOsByBalances command to rpc

* Store list of addresses, updated with the collectAddresses methods
(replacing collectUTXOs methods)

* Fix wrong commands in GetBalanceByAddress

* Rename: refreshExistingUTXOs -> refreshUTXOs

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2022-02-18 15:08:08 +02:00
Svarog
f452531df0
Fix bookkeeping of chained transactions in mempool (#1946)
* Fix bookkeeping of chained transactions in mempool

* Fix golint error
2022-02-18 13:21:45 +02:00
Ori Newman
13a09da848
Bump version to v0.11.12 (#1941)
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2022-02-17 18:27:25 +02:00
Michael Sutton
f58aeb4f9f
In profile dump file - use a time format supporting all file systems (#1945)
* Use a time format without ":" to support all file systems

* go fmt
2022-02-12 22:00:46 +02:00
Ori Newman
82f0a4d74f
Drop support for p2p v3 (#1942)
* Drop support for p2p v3

* Remove redundant aliases

* Remove redundant condition
2022-02-06 17:28:06 +02:00
Ori Newman
69d90fe827
Remove duplicate median time calculation on tx validation (#1943)
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2022-02-06 16:34:05 +02:00
Ori Newman
c85b5d70fd
Add AllowConnectionToDifferentVersions flag to kaspactl (#1940) 2022-02-06 15:43:09 +02:00
Ori Newman
1cd712a63e
Increase headers proof timeout and add more progress logs (#1939)
* Increase timeout for pruning proof and add some logs

* Show resolving virtual progress as whole percents
v0.11.12-testnet1
2022-02-06 12:42:28 +02:00
stasatdaglabs
27ba9d0374
Split ApplyPruningPointProof to multiple small database transactions (#1937)
* Split ApplyPruningPointProof to multiple small database transactions.

* Increase the timeout duration in TestIBDWithPruning.

* Increase the timeout duration in simple-sync.

* Explain that if ApplyPruningPointProof fails, the database must be discarded.
2022-02-06 10:36:27 +02:00
stasatdaglabs
b1229f7908
Display progress of IBD process in Kaspad logs (#1938)
* Report progress percentage when downloading headers in IBD.

* Extract reporting logic to a separate type.

* Report progress for IBD missing block bodies.
2022-02-02 21:24:45 +02:00
Michael Sutton
4a560f25a6
Update changelog and version number for v0.11.11 (#1935) v0.11.11 2022-01-27 21:53:05 +02:00