4347 Commits

Author SHA1 Message Date
Svarog
d5a3a96bde
Use hard-coded sample config instead of assumed path (#1466)
* Use hard-coded sample config instead of assumed path

* Fix bad path to sample-kaspad.conf in TestCreateDefaultConfigFile

Co-authored-by: Elichai Turkel <elichai.turkel@gmail.com>
2021-02-01 15:15:37 +02:00
Elichai Turkel
12c438d389
Fix data races in ConnectionManager and flow tests (#1474)
* Reuse the ticker in ConnectionManager.waitTillNextIteration

* Fix a data race in ConnectionManager by locking the mutex

* Add a mutex to fakeRelayInvsContext in block relay flow test

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-02-01 15:03:31 +02:00
Elichai Turkel
280fa3de46
Prevent infinite ticker leaks in kaspaminer (#1476)
* Prevent infinite tickers leaks in kaspaminer

* Reset ticker in ConnectionManager instead of allocating a new one

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-02-01 14:52:17 +02:00
Elichai Turkel
d281dabdb4
Bump Go version to 1.15 (#1477) 2021-02-01 14:35:11 +02:00
Ori Newman
331042edf1
Add defaultTargetBlocksPerSecond (#1473)
* Add defaultTargetBlocksPerSecond

* Use different default per network
2021-02-01 14:26:45 +02:00
Ori Newman
669a9ab4c3
Ban by IP (#1471)
* Ban by IP

* Fix panic

* Fix error format

* Remove failed addresses

Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2021-02-01 10:51:18 +02:00
stasatdaglabs
65e149b2bb
In kaspaminer, don't crash on submitBlock timeout (#1462)
* In kaspaminer, don't crash on submitBlock timeout.

* Make timeout messages have a log level of Warn.

* Wait for a second after receiving a reject for IBD.

Co-authored-by: Elichai Turkel <elichai.turkel@gmail.com>
2021-01-29 09:10:21 +02:00
stasatdaglabs
7c1495ba65
Force stop gRPC servers after a short timeout (#1463)
* Force stop gRPC servers after a short timeout.

* Use spawn instead of go.
2021-01-28 19:43:04 +02:00
Ori Newman
13ffa5093c
Increase the waiting for error timeout (#1465) 2021-01-28 13:33:37 +02:00
Ori Newman
a9a810a2b2
Add block type to MineJSON (#1464) 2021-01-28 13:22:20 +02:00
Michael Sutton
c9b591f2d3
Final reindex algorithm (#1430)
* Mine JSON

* [Reindex tests] add test_params and validate_mining flag to test_consensus

* Rename file and extend tests

* Ignore local test datasets

* Use spaces over tabs

* Reindex algorithm - full algorithm, initial commit, some tests fail

* Reindex algorithm - a few critical fixes

* Reindex algorithm - move reindex struct and all related operations to new file

* Reindex algorithm - added a validateIntervals method and modified tests to use it (instead of exact comparisons)

* Reindex algorithm - modified reindexIntervals to receive the new child as argument and fixed an important related bug

* Reindex attack tests - move logic to helper function and add stretch test

* Reindex algorithm - variable names and some comments

* Reindex algorithm - minor changes

* Reindex algorithm - minor changes 2

* Reindex algorithm - extended stretch test

* Reindex algorithm - small fix to validate function

* Reindex tests - move tests and add DAG files

* go format fixes

* TestParams doc comment

* Reindex tests - exact comparisons are not needed

* Update to version 0.8.6

* Remove TestParams and use AddUTXOInvalidHeader instead

* Use gzipeed test files

* This unintended change somehow slipped in through branch merges

* Rename test

* Move interval increase/decrease methods to reachability interval file

* Addressing a bunch of minor review comments

* Addressed a few more minor review comments

* Make code of offsetSiblingsBefore and offsetSiblingsAfter symmetric

* Optimize reindex logic in cases where reorg occurs + reorg test

* Do not change reindex root too fast (on reorg)

* Some comments

* A few more comments

* Addressing review comments

* Remove TestNoAttackAlternateReorg and assert chain attack

* Minor

Co-authored-by: Elichai Turkel <elichai.turkel@gmail.com>
Co-authored-by: Mike Zak <feanorr@gmail.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-01-27 17:09:20 +02:00
Ori Newman
8d6e71d490
Add IBD test cases and check for MsgUnexpectedPruningPoint on receivePruningPointBlock as well (#1459)
* Check for MsgUnexpectedPruningPoint on receivePruningPointBlock as well

* Add IBD test cases

* Revert "Check for MsgUnexpectedPruningPoint on receivePruningPointBlock as well"

This reverts commit 6a6d1ea1807e5e4a4207fd13c78a6f066654606d.

* Change log level for two logs

* Remove "testing a situation where the pruning point moved during IBD (before sending the pruning point block)"
2021-01-27 16:42:42 +02:00
Elichai Turkel
2823461fe2
Change AddressKey from string to port+ipv6 address (#1458) 2021-01-27 16:09:32 +02:00
Elichai Turkel
2075c585da
Fix race condition in kaspaminer (#1455)
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-01-27 13:08:35 +02:00
Elichai Turkel
01aee62cb0
Add log and measure to pruning points (#1457) 2021-01-27 11:40:51 +02:00
Ori Newman
a6ee871f7e
Increase maxSelectedParentTimeDiffToAllowMiningInMilliSeconds to one hour (#1456) 2021-01-27 11:04:58 +02:00
Mike Zak
6393a8186a Update to version 0.9.0 2021-01-27 09:22:56 +02:00
stasatdaglabs
3916534a7e
In kaspactl, prettify responses before printing them (#1453)
* In kaspactl, prettify responses before printing them.

* Indent with four spaces instead of a tab.

* Unwrap the response.

* Simplify unwrapping the response.

* Don't unwrap responses.

* Use protojson.MarshalOptions for prettification.
v0.8.6-rc1 v0.8.6
2021-01-27 09:13:48 +02:00
Elichai Turkel
0561347ff1
Remove default dns/grpc seeders (#1454) 2021-01-26 17:40:54 +02:00
Svarog
fb11981da1
Make kaspactl usable by human beings (#1452)
* Add request_types and their help

* Added command parser

* Updated main to use command if it's specified

* Some progress in making everything work

* Fix command parser for pointers to structs

* Cleanup code

* Enhance usage text

* Fixed typo

* Some minor style fixing, and remove temporary code

* Correctly fallthrough in stringToValue unsupported types

Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2021-01-26 14:22:35 +02:00
Ori Newman
1742e76af7
Add TestHandleRelayInvsErrors (#1450) 2021-01-26 13:31:28 +02:00
Svarog
ddfe376388
Don't ban peers that sent a requested duplicate block (#1440)
* Ignore ErrDuplicateBlock for any blocks that were requested

* Don't check for DuplicateBlockErr in processHeaders + improve logs

* Return the check for ErrDuplicateBlock in processHeader

* Fix log message
2021-01-26 08:42:04 +02:00
Elichai Turkel
52da65077a
codecov: Ignore protobuf autogenerated files (#1449) 2021-01-25 16:58:11 +02:00
Elichai Turkel
ed85f09742
Add P2P/RPC name to the gRPC logs (#1448)
* Add P2P/RPC name to the gRPC logs

* Add p2p/rpc name in more logs
2021-01-25 16:41:43 +02:00
stasatdaglabs
819ec9f2a7
Pass fromOutpoint into GetPruningPointUTXOs instead of offset, so it could Seek over the cursor (#1447)
* Implement TestGetPruningPointUTXOs.

* Fix a bad error message.

* Fix TestGetPruningPointUTXOs for testnet.

* Make sure all the UTXOs are returned in TestGetPruningPointUTXOs.

* Implement BenchmarkGetPruningPointUTXOs.

* Pass fromOutpoint into GetPruningPointUTXOs instead of offset, so it could Seek over the cursor.

* Fix weird benchmark timer calls.

* Remove unnecessary collection of outpointAndUTXOEntryPairs from BenchmarkGetPruningPointUTXOs.

* Fix a comment.
2021-01-25 13:38:59 +02:00
Ori Newman
7ea8a72a9e
Add TestReceiveAddressesErrors (#1446)
* Add TestReceiveAddressesErrors

* Change errors to be more descriptive

* Fix checkFlowError
2021-01-24 17:35:20 +02:00
stasatdaglabs
ca04c049ab
When the pruning point moves, update its UTXO set outside of a database transaction (#1444)
* Remove pruningPointUTXOSetStaging and implement UpdatePruningPointUTXOSet.

* Implement StageStartSavingNewPruningPointUTXOSet, HadStartedSavingNewPruningPointUTXOSet, and FinishSavingNewPruningPointUTXOSet.

* Fix a bad return.

* Implement UpdatePruningPointUTXOSetIfRequired.

* Call UpdatePruningPointUTXOSetIfRequired on consensus creation.

* Rename savingNewPruningPointUTXOSetKey to updatingPruningPointUTXOSet.

* Add a log.

* Add calls to runtime.GC() at its start and end.

* Rename a variable.

* Replace calls to runtime.GC to calls to LogMemoryStats.

* Wrap the contents of LogMemoryStats in a log closure.
2021-01-24 14:48:11 +02:00
Ori Newman
9a17198e7d
Remove redundant type check (#1445) 2021-01-24 14:14:03 +02:00
stasatdaglabs
756f40c59a
Sync pruning point UTXO sets incrementally instead of all at once (#1431)
* Replaced the content of MsgIBDRootUTXOSetChunk with pairs of outpoint-utxo entry pairs.

* Rename utxoIter to utxoIterator.

* Add a big stinky TODO on an assert.

* Replace pruningStore staging with a UTXO set iterator.

* Reimplement receiveAndInsertIBDRootUTXOSet.

* Extract OutpointAndUTXOEntryPairsToDomainOutpointAndUTXOEntryPairs into domainconverters.go.

* Pass the outpoint and utxy entry pairs to the pruning store.

* Implement InsertCandidatePruningPointUTXOs.

* Implement ClearCandidatePruningPointUTXOs.

* Implement UpdateCandidatePruningPointMultiset.

* Use the candidate pruning point multiset in updatePruningPoint.

* Implement CandidatePruningPointUTXOIterator.

* Use the pruning point utxo set iterator for StageVirtualUTXOSet.

* Defer ClearCandidatePruningPointUTXOs.

* Implement OverwriteVirtualUTXOSet.

* Implement CommitCandidatePruningPointUTXOSet.

* Implement BeginOverwritingVirtualUTXOSet and FinishOverwritingVirtualUTXOSet.

* Implement overwriteVirtualUTXOSetAndCommitPruningPointUTXOSet.

* Rename ClearCandidatePruningPointUTXOs to ClearCandidatePruningPointData.

* Add missing methods to dbManager.

* Implement PruningPointUTXOs.

* Implement RecoverUTXOIfRequired.

* Delete the utxoserialization package.

* Fix compilation errors in TestValidateAndInsertPruningPoint.

* Switch order of operations in the if statements in PruningPointUTXOs so that Next() wouldn't be unnecessarily called.

* Fix missing pruning point utxo set staging and bad slice length.

* Fix no default multiset in InsertCandidatePruningPointUTXOs.

* Make go vet happy.

* Rename candidateXXX to importedXXX.

* Do some more renaming.

* Rename some more.

* Fix bad MsgIBDRootNotFound logic.

* Fix an error message.

* Simplify receiveIBDRootBlock.

* Fix error message in receiveAndInsertIBDRootUTXOSet.

* Do some more renaming.

* Fix merge errors.

* Fix a bug caused by calling iterator.First() unnecessarily.

* Remove databaseContext from stores and don't use a transaction in ClearXXX functions.

* Simplify receiveAndInsertIBDRootUTXOSet.

* Fix offset count in PruningPointUTXOs().

* Fix readOnlyUTXOIteratorWithDiff.First().

* Split handleRequestIBDRootUTXOSetAndBlockFlow into smaller methods.

* Rename IbdRootNotFound to UnexpectedPruningPoint.

* Rename requestIBDRootHash to requestPruningPointHash.

* Rename IBDRootHash to PruningPointHash.

* Rename RequestIBDRootUTXOSetAndBlock to RequestPruningPointUTXOSetAndBlock.

* Rename IBDRootUTXOSetChunk to PruningPointUTXOSetChunk.

* Rename RequestNextIBDRootUTXOSetChunk to RequestNextPruningPointUTXOSetChunk.

* Rename DoneIBDRootUTXOSetChunks to DonePruningPointUTXOSetChunks.

* Rename remaining references to IBD root.

* Fix an error message.

* Add a check for HadStartedImportingPruningPointUTXOSet in commitVirtualUTXODiff.

* Add a check for HadStartedImportingPruningPointUTXOSet in ImportPruningPointUTXOSetIntoVirtualUTXOSet.

* Move FinishImportingPruningPointUTXOSet closer to HadStartedImportingPruningPointUTXOSet.

* Remove reference to pruningStore in utxoSetIterator.

* Pointerify utxoSetIterator receivers.

* Fix bad insert in CommitImportedPruningPointUTXOSet.

* Rename commitImportedPruningPointUTXOSetAll to applyImportedPruningPointUTXOSet.

* Simplify PruningPointUTXOs.

* Add populateTransactionWithUTXOEntriesFromUTXOSet.

* Fix a TODO comment.

* Rename InsertImportedPruningPointUTXOs to AppendImportedPruningPointUTXOs.

* Extract handleRequestPruningPointUTXOSetAndBlockMessage to a separate method.

* Rename stuff in readOnlyUTXOIteratorWithDiff.First().

* Address toAddIterator in readOnlyUTXOIteratorWithDiff.First().

* Call First() before any full iteration on ReadOnlyUTXOSetIterator.

* Call First() before any full iteration on a database Cursor.

* Put StartImportingPruningPointUTXOSet inside the pruning point transaction.

* Make serializeOutpoint and serializeUTXOEntry free functions in pruningStore.

* Fix readOnlyUTXOIteratorWithDiff.First().

* Fix bad validations in importPruningPoint.

* Remove superfluous call to validateBlockTransactionsAgainstPastUTXO.
2021-01-21 17:24:52 +02:00
Elichai Turkel
6a03d31f98
Remove accidental pointer indirection in dbKey (#1441) 2021-01-21 12:14:52 +02:00
Ori Newman
319ab6cfcd
Always request orphan roots, even when you get an inv of a known orphan (#1436)
Co-authored-by: Svarog <feanorr@gmail.com>
2021-01-20 10:58:45 +02:00
Ori Newman
abef96e3de
Add TestIBDWithPruning (#1425)
* Add TestIBDWithPruning

* Test block count

* Fix a typo

Co-authored-by: Elichai Turkel <elichai.turkel@gmail.com>
Co-authored-by: Mike Zak <feanorr@gmail.com>
2021-01-20 10:07:32 +02:00
stasatdaglabs
2e0bc0f8c4
Increase P2P connections' dial timeouts to 5 seconds (#1437)
Co-authored-by: Elichai Turkel <elichai.turkel@gmail.com>
2021-01-20 09:48:48 +02:00
Ori Newman
acf5423c63
Remove docker test parallelism (#1434)
* Remove docker parallelism

* Remove redundant dash
2021-01-19 17:30:24 +02:00
Ori Newman
effb545d20
Fix wrong condition and add logs (#1435) 2021-01-19 17:20:25 +02:00
Svarog
ad9c213a06
Restructure database to prevent double-slashes in keys, causing bugs in cursors (#1432)
* Add TestValidateAndInsertPruningPointWithSideBlocks

* Optimize infrastracture bucket paths

* Update infrastracture tests

* Refactor the consensus/database layer

* Remove utils/dbkeys

* Use consensus/database in consensus instead of infrastructure

* Fix a bug in dbBucketToDatabaseBucket and MakeBucket combination

Co-authored-by: Elichai Turkel <elichai.turkel@gmail.com>
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2021-01-19 14:19:08 +02:00
Svarog
a4adbabf96
TestBuildBlockErrorCases and remove redundant check of coinbase script length (#1427)
* Write structure of TestBlockBuilderErrorCases

* Remove double verification of script length in serializeCoinbasePayload

* Remove redundant code in TestBuildBlockErrorCases

* Rename povTransactionHash -> povBlockHash

* Convert coinbasePayloadScriptPublicKeyMaxLength to uint8

* Re-use consensus in TestBuildBlockErrorCases
2021-01-19 10:37:51 +02:00
Ori Newman
799eb7515c
Test validateAndInsertPruningPoint (#1420)
* Add TestValidateAndInsertPruningPoint

* Check fake UTXO set and validate that the pruning point changed
2021-01-18 18:17:13 +02:00
Mike Zak
0769705b37 Update to version 0.8.6 2021-01-18 15:17:15 +02:00
Svarog
189e3b6be9
Fix missing utxo notifications (#1428)
* fix missing UTXO notifications (#1426)

* Remove redundant semicolon

Co-authored-by: aspect <anton.yemelyanov@gmail.com>
v0.8.5-rc1
2021-01-18 13:08:16 +02:00
Mike Zak
e8dfbc8367 Merge remote-tracking branch 'origin/master' into v0.8.5-dev 2021-01-18 11:36:25 +02:00
Ori Newman
d70740331a
Remove hashesQueueSet (#1424)
Co-authored-by: Svarog <feanorr@gmail.com>
2021-01-18 09:10:26 +02:00
Svarog
9a81b1328a
Add the Address of node to whom connected in log of send/receiveVersion (#1423)
* Add the Address of node to whom connected in log of send/receiveVersion

* Don't call functions before LogAndMeasureExecutionTime

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-01-17 16:31:48 +02:00
Ori Newman
d4f3a252ff
Add TestIsFinalizedTransaction (#1422)
Co-authored-by: Svarog <feanorr@gmail.com>
2021-01-17 15:47:49 +02:00
Ori Newman
f14527de4c
Give different limit to the RPC server (#1421) 2021-01-17 13:58:42 +02:00
Ori Newman
dd57e6abe6
Fix checkParentHeadersExist and cover pruning_violation_proof_of_work_and_difficulty.go with tests (#1418)
* Fix checkParentHeadersExist and cover pruning_violation_proof_of_work_and_difficulty.go with tests

* Remove unused variable

* Change consensus violation

* Change condition order

* Get rid of irrelevant error codes in extractRejectCode

* Fix wrong test db names

* Fix checkParentHeadersExist
2021-01-17 11:27:04 +02:00
Ori Newman
67be4d82bf
Don't mark bad merkle root as invalid (#1419)
* Don't mark bad merkle root as invalid

* Fix TestBlockStatus

* Move discardAllChanges inside the inner if
2021-01-17 10:40:05 +02:00
Ori Newman
a1381d6768
Add TestCheckParentBlockBodiesExist (#1405)
* Add TestCheckParentBlockBodiesExist

* Use block in pruning point's anticone for the test

* Fix test db name
2021-01-14 13:31:17 +02:00
Ori Newman
10b519a3e2
Add tests to ValidateHeaderInIsolation (#1415)
* Add tests to ValidateHeaderInIsolation

* Fix tests db names

Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2021-01-14 11:06:08 +02:00
Ori Newman
a35f8269ea
Add checkBlockIsNotPruned (#1413)
* Add checkBlockIsNotPruned

* Fix test name and comment

Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2021-01-14 10:56:22 +02:00