12 Commits

Author SHA1 Message Date
Ori Newman
d207888b67
Implement pruned headers node (#1787)
* Pruning headers p2p basic structure

* Remove headers-first

* Fix consensus tests except TestValidateAndInsertPruningPointWithSideBlocks and TestValidateAndInsertImportedPruningPoint

* Add virtual genesis

* Implement PruningPointAndItsAnticoneWithMetaData

* Start fixing TestValidateAndInsertImportedPruningPoint

* Fix TestValidateAndInsertImportedPruningPoint

* Fix BlockWindow

* Update p2p and gRPC

* Fix all tests except TestHandleRelayInvs

* Delete TestHandleRelayInvs parts that cover the old IBD flow

* Fix lint errors

* Add p2p_request_ibd_blocks.go

* Clean code

* Make MsgBlockWithMetaData implement its own representation

* Remove redundant check if highest share block is below the pruning point

* Fix TestCheckLockTimeVerifyConditionedByAbsoluteTimeWithWrongLockTime

* Fix comments, errors ane names

* Fix window size to the real value

* Check reindex root after each block at TestUpdateReindexRoot

* Remove irrelevant check

* Renames and comments

* Remove redundant argument from sendGetBlockLocator

* Don't delete staging on non-recoverable errors

* Renames and comments

* Remove redundant code

* Commit changes inside ResolveVirtual

* Add comment to IsRecoverableError

* Remove blocksWithMetaDataGHOSTDAGDataStore

* Increase windows pagefile

* Move DeleteStagingConsensus outside of defer

* Get rid of mustAccepted in receiveBlockWithMetaData

* Ban on invalid pruning point

* Rename interface_datastructures_daawindowstore.go to interface_datastructures_blocks_with_meta_data_daa_window_store.go

* * Change GetVirtualSelectedParentChainFromBlockResponseMessage and VirtualSelectedParentChainChangedNotificationMessage to show only added block hashes
*  Remove ResolveVirtual
* Use externalapi.ConsensusWrapper inside MiningManager
* Fix pruningmanager.blockwithmetadata

* Set pruning point selected child when importing the pruning point UTXO set

* Change virtual genesis hash

* replace the selected parent with virtual genesis on removePrunedBlocksFromGHOSTDAGData

* Get rid of low hash in block locators

* Remove +1 from everywhere we use difficultyAdjustmentWindowSize and increase the default value by one

* Add comments about consensus wrapper

* Don't use separate staging area when resolving resolveBlockStatus

* Fix netsync stability test

* Fix checkResolveVirtual

* Rename ConsensusWrapper->ConsensusReference

* Get rid of blockHeapNode

* Add comment to defaultDifficultyAdjustmentWindowSize

* Add SelectedChild to DAGTraversalManager

* Remove redundant copy

* Rename blockWindowHeap->calculateBlockWindowHeap

* Move isVirtualGenesisOnlyParent to utils

* Change BlockWithMetaData->BlockWithTrustedData

* Get rid of maxReasonLength

* Split IBD to 100 blocks each time

* Fix a bug in calculateBlockWindowHeap

* Switch to trusted data when encountering virtual genesis in blockWithTrustedData

* Move ConsensusReference to domain

* Update ConsensusReference comment

* Add comment

* Rename shouldNotAddGenesis->skipAddingGenesis
2021-07-26 12:24:07 +03:00
talelbaz
d7f2cf81c0
Change merge set order to topological order (#1654)
* Change mergeSet to be ordered topologically.

* Add special condition for genesis.

* Add check that the coinbase is validated.

* Change names of variables(old: chainHash, blueHash).

* Fix the DAG diagram in the comment above the function.

* Fix variables names.

Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-05-19 14:40:55 +03:00
Svarog
f1451406f7
Add support for multiple staging areas (#1633)
* Add StagingArea struct

* Implemented staging areas in blockStore

* Move blockStagingShard to separate folder

* Apply staging shard to acceptanceDataStore

* Update blockHeaderStore with StagingArea

* Add StagingArea to BlockRelationStore

* Add StagingArea to blockStatusStore

* Add StagingArea to consensusStateStore

* Add StagingArea to daaBlocksStore

* Add StagingArea to finalityStore

* Add StagingArea to ghostdagDataStore

* Add StagingArea to headersSelectedChainStore and headersSelectedTipStore

* Add StagingArea to multisetStore

* Add StagingArea to pruningStore

* Add StagingArea to reachabilityDataStore

* Add StagingArea to utxoDiffStore

* Fix forgotten compilation error

* Update reachability manager and some more things with StagingArea

* Add StagingArea to dagTopologyManager, and some more

* Add StagingArea to GHOSTDAGManager, and some more

* Add StagingArea to difficultyManager, and some more

* Add StagingArea to dagTraversalManager, and some more

* Add StagingArea to headerTipsManager, and some more

* Add StagingArea to constnsusStateManager, pastMedianTimeManager

* Add StagingArea to transactionValidator

* Add StagingArea to finalityManager

* Add StagingArea to mergeDepthManager

* Add StagingArea to pruningManager

* Add StagingArea to rest of ValidateAndInsertBlock

* Add StagingArea to blockValidator

* Add StagingArea to coinbaseManager

* Add StagingArea to syncManager

* Add StagingArea to blockBuilder

* Update consensus with StagingArea

* Add StagingArea to ghostdag2

* Fix remaining compilation errors

* Update names of stagingShards

* Fix forgotten stagingArea passing

* Mark stagingShard.isCommited = true once commited

* Move isStaged to stagingShard, so that it's available without going through store

* Make blockHeaderStore count be avilable from stagingShard

* Fix remaining forgotten stagingArea passing

* commitAllChanges should call dbTx.Commit in the end

* Fix all tests tests in blockValidator

* Fix all tests in consensusStateManager and some more

* Fix all tests in pruningManager

* Add many missing stagingAreas in tests

* Fix many tests

* Fix most of all other tests

* Fix ghostdag_test.go

* Add comment to StagingArea

* Make list of StagingShards an array

* Add comment to StagingShardID

* Make sure all staging shards are pointer-receiver

* Undo bucket rename in block_store

* Typo: isCommited -> isCommitted

* Add comment explaining why stagingArea.shards is an array
2021-03-29 10:34:11 +03:00
Elichai Turkel
68bd8330ac
Log the networks hashrate (#1406)
* Log the hashrate of each block

* Add a test for GetHashrateString

* Move difficulty related functions to its own package

* Convert the validated log in validateAndInsertBlock to a log function

* Add tests for max/min int
2021-01-13 12:51:23 +02:00
Elichai Turkel
6279db2bf1
Reverse ghostdag tie break direction (#1359)
* Remove hash reversal in ghostdag

* Update tests after chaning ghostdag hash direction

Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2021-01-05 19:10:59 +02:00
Elichai Turkel
e509cb1597
Smal performance improvements to BlueWindow (#1343)
* Convert BlockGHOSTDAGData from an interface to a public struct with getters

* Move hashes.Less to externalapi so it can access the hashes directly without copying

* Reduce calls to ghostdagstore.Get in blueWindow

* Simplify the logic in RequiredDifficulty and reuse big.Int

* Remove bigintpool as its no longer used

* Use ChooseSelectedParent in RequiredDifficulty instead of looping over the parents

* Remove comment
2021-01-05 12:13:02 +02:00
Ori Newman
49b6cc6038
Add mutable and immutable header interfaces (#1305)
* Add mutable and immutable header interfaces

* Fix ShouldMine()

* Remove false comment

* Fix Equal signature

* Fix Equal implementation
2020-12-29 13:55:17 +02:00
Svarog
05941a76e7
Make DomainHash and TransactionID read-only structs (#1282)
* Increase size of reachability cache

* Change DomainHash to struct with unexported hashArray

* Fixing compilation errors stemming from new DomainHash structure

* Remove obsolete Read/WriteElement methods in appmessage

* Fix all tests

* Fix all tests

* Add comments

* A few renamings

* go mod tidy
2020-12-24 16:15:23 +02:00
Ori Newman
cb9d7e313d
Implement Clone and Equal for all model types (#1155)
* [NOD-1575] Implement Clone and Equal for all model types

* [NOD-1575] Add assertion for transaction ID equality

* [NOD-1575] Use DomainTransaction.Equal to compare to expected coinbase transaction

* [NOD-1575] Add TestDomainBlockHeader_Clone

* [NOD-1575] Don't clone nil values

* [NOD-1575] Add type assertions

* [NOD-1575] Don't clone nil values

* [NOD-1575] Add missing Equals

* [NOD-1575] Add length checks

* [NOD-1575] Update comment

* [NOD-1575] Check length for TransactionAcceptanceData

* [NOD-1575] Explicitly clone nils where needed

* [NOD-1575] Clone tx id

* [NOD-1575] Flip condition

* Nod 1576 make coverage tests for equal clone inside model externalapi (#1177)

* [NOD-1576] Make coverage tests for equal and clone inside model and externalapi

* Some formatting and naming fixes

* Made transactionToCompare type exported

* Added some tests and made some changes to the tests code

* No changes made

* Some formatting and naming changes made

* Made better test coverage for externalapi clone and equal functions

* Changed expected result for two cases

* Added equal and clone functions tests for ghostdag and utxodiff

* Added tests

* [NOD-1576] Implement reachabilitydata equal/clone unit tests

* [NOD-1576]  Full coverage of reachabilitydata equal/clone unit tests

* Made changes and handling panic to transaction_equal_clone_test.go and formating of utxodiff_equal_clone_test.go

* Added recoverForEqual2 for handling panic to transaction_equal_clone_test.go

* [NOD-1576]  Full coverage of transaction equal unit test

* [NOD-1576] Add expects panic

* [NOD-1576] Allow composites in go vet

* [NOD-1576] Code review fixes (#1223)

* [NOD-1576] Code review fixes

* [NOD-1576] Code review fixes part 2

* [NOD-1576] Fix wrong name

Co-authored-by: karim1king <karimkaspersky@yahoo.com>
Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: Karim <karim1king@users.noreply.github.com>

* Fix merge errors

* Use Equal where possible

* Use Equal where possible

* Use Equal where possible

Co-authored-by: andrey-hash <74914043+andrey-hash@users.noreply.github.com>
Co-authored-by: karim1king <karimkaspersky@yahoo.com>
Co-authored-by: Karim <karim1king@users.noreply.github.com>
2020-12-22 17:38:54 +02:00
Elichai Turkel
c3902ed7a8
Replace blue score with blue work in ghostdag (#1172)
* Replace blueScore with blueWork in ghostDAG SelectedParent selection

* Add blueWork to protopuf ghostdag data

* Auto generate protobuf go code

* Serialize/Deserialize blueWork when converting to protobuf

* pass block header store to ghostdagmanager

* Convert tal's ghostdag2 implementation to blueWork

* Change finality test to check the blueWork instead of blueScore

* Update ghostdag_test to pass blockHeaderStore to ghostdag, and test all networks genesis headers

* Add sanity blueWork check to ghostdag_test
2020-12-06 14:45:21 +02:00
Svarog
33eaf9edac
[NOD-1548] Re-add test difficulty + Make GHOSTDAGData immutable + don't clone in store (#1178)
* [NOD-1548] Readd TestDifficulty

* [NOD-1548] Make GHOSTDAGData immutable + don't clone in store

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2020-12-06 12:35:14 +02:00
Elichai Turkel
21fc2d4219
[NOD-1433] Write specified unit tests for GHOSTDAG (#1010)
commit 3830df34b2a53c361c8a6ca450f7f4a683fe1ae1
Merge: 46dc2e977 17e7819c2
Author: Elichai Turkel <elichai.turkel@gmail.com>
Date:   Tue Dec 1 16:29:51 2020 +0200

    Merge pull request #1170 from kaspanet/tal-ghost-fix

    Fix GhostDAG tests and jsons

commit 17e7819c27153b8527f18d638a3173a6cc4975eb
Author: Elichai Turkel <elichai.turkel@gmail.com>
Date:   Tue Dec 1 16:24:01 2020 +0200

    Remove non-json ghostdag tests

commit 4bebb1d96a8740c5359351c0aea7aca20067d65b
Author: Elichai Turkel <elichai.turkel@gmail.com>
Date:   Tue Dec 1 13:26:06 2020 +0200

    Add a coment above tal's ghostdag2 impl

commit faf21a042e0f40f3842bbe2b4a6aeef48a9bf0fd
Author: Elichai Turkel <elichai.turkel@gmail.com>
Date:   Tue Dec 1 13:20:08 2020 +0200

    fix the interfaces after merge

commit a8b7a25b2e85a47ef2acffa426db25ce0d8c753c
Merge: af91b69b2 f1c6df48c
Author: Elichai Turkel <elichai.turkel@gmail.com>
Date:   Tue Dec 1 13:19:08 2020 +0200

    Merge branch 'v0.8.2-dev' into tal-ghost-fix

commit af91b69b207cf3c3e9eaedf031810153b8a576d3
Author: Elichai Turkel <elichai.turkel@gmail.com>
Date:   Tue Dec 1 13:18:41 2020 +0200

    Fix the non-json tests

commit c56f34b73b16f0b28f02f4dc5ae4607b420afb36
Author: Elichai Turkel <elichai.turkel@gmail.com>
Date:   Tue Dec 1 13:18:17 2020 +0200

    Fix the jsons

commit 46dc2e97736e304fa5398289566c0d281d02cf8e
Author: tal <tal@daglabs.com>
Date:   Mon Nov 30 17:15:20 2020 +0200

    [NOD - 1143] Cosmetics changes.

commit b28e5ce816c8f9ea69720b0c8d2e8fb63fc1d6c3
Author: tal <tal@daglabs.com>
Date:   Mon Nov 30 15:48:08 2020 +0200

    [#1126] Place selectedParent to be first on blueMergeSet.

commit 4b56ed2da94638736d02c038acaa4ab4d8b625ac
Author: tal <tal@daglabs.com>
Date:   Mon Nov 30 14:51:50 2020 +0200

    [#1126] Change pacement between blockRight and blockLeft .

commit b09f31be9301cf51a8f3866166cf38a174c368c9
Merge: e17a98b7b 0db39833f
Author: talelbaz <63008512+talelbaz@users.noreply.github.com>
Date:   Mon Nov 30 14:30:22 2020 +0200

    Merge pull request #1162 from kaspanet/new-jsons

    Update the dag json tests

commit e17a98b7ba4beba77699fb68081a0d23f4feb714
Author: tal <tal@daglabs.com>
Date:   Mon Nov 30 14:08:25 2020 +0200

    [#1126] Use WALK function in tests & cosmetic changes.

commit 0db39833f3afeea8f79a67795d2e532c111b0eda
Author: Elichai Turkel <elichai.turkel@gmail.com>
Date:   Mon Nov 30 12:20:13 2020 +0200

    Update the dag json tests

commit 5a3da43dd4c492120debabca643293ed5269a842
Author: tal <tal@daglabs.com>
Date:   Sun Nov 29 12:03:37 2020 +0200

    [NOD-1433] Remove unneccessry code.

commit a6cde558ac8ff788b548da7ec695460cd680bb32
Author: tal <tal@daglabs.com>
Date:   Mon Nov 23 17:05:56 2020 +0200

    [NOD-1433] Change "Stage" sig function according to the new interface - added error as a return type.

commit 07859b6218f503f13079640d1ab9139f9b63a51b
Author: tal <tal@daglabs.com>
Date:   Mon Nov 23 17:03:26 2020 +0200

    [NOD-1433]  Print formats changed & Cosmetics code changes.

commit e1a851664ea69185a93cd189bb328f4cded07a8e
Author: tal <tal@daglabs.com>
Date:   Sun Nov 15 17:34:59 2020 +0200

    [NOD-1433] Travers the tests dir and run each test.

commit 4c7474edc16ce4553532f893c81f0c795a649b1c
Author: tal <tal@daglabs.com>
Date:   Mon Nov 9 12:44:53 2020 +0200

    [NOD-1433] Travers the tests dir and run each test.

commit 89dd1e61d36af077493f38d90d925e3057eb962a
Author: tal <tal@daglabs.com>
Date:   Mon Nov 9 11:48:36 2020 +0200

    [NOD-1433] Change implementation to adjust genesis's score 0.
    Also, keep changing the test file to fit the new implementation.

commit 6acdcd17def0bb29aca1f65c713d89d1e9960416
Author: tal <tal@daglabs.com>
Date:   Sun Nov 8 17:07:22 2020 +0200

    [NOD-1433] New test was added(Test 6).

commit bf238893170e5b5ebc509d40937b410b22d8a9ff
Author: tal <tal@daglabs.com>
Date:   Sun Nov 8 14:59:36 2020 +0200

    Fix golint errors

commit 79ff990b5f04a5a80ccccd95ec3c6fb745e2ca41
Author: tal <tal@daglabs.com>
Date:   Sun Nov 8 14:47:12 2020 +0200

    added "Optimize imports".

commit 73d0128f639517b27a01d016a14d67d7f1a42570
Author: tal <tal@daglabs.com>
Date:   Sun Nov 8 13:03:22 2020 +0200

    Added an implementation factory.

commit 61ca8b2e7e20f0dbced5424028e6b52ea511d25c
Author: tal <tal@daglabs.com>
Date:   Thu Nov 5 16:03:18 2020 +0200

    1. impl - choose the highest hash.
    2. test - changed the test accordingly.

commit ef0943ca29ad8699b2c50978cf94d28561bb3e10
Author: tal <tal@daglabs.com>
Date:   Thu Oct 29 18:00:45 2020 +0200

    Update Tests

commit 6e5936abff9be604a0bd62329ec2d09be13c1d33
Author: tal <tal@daglabs.com>
Date:   Tue Oct 27 10:22:45 2020 +0200

    Change to the new API

commit 5a70dc48b376994634586e0053a56f775b93f6b9
Author: tal <tal@daglabs.com>
Date:   Mon Oct 26 18:35:31 2020 +0200

    1. Added tests for ori

commit 2b9f78353f671d2243fee36394618a59f3eb8d4c
Author: tal <tal@daglabs.com>
Date:   Mon Oct 26 13:04:37 2020 +0200

    1. Added structure "isolatedTest" {k, test}
    2. Added for loop on the tests.
    3. New test - Test 5.

commit c026d7b7a26348d0f2999a73cb9caefd3ada0ab1
Author: tal <tal@daglabs.com>
Date:   Thu Oct 22 17:35:56 2020 +0300

    Fix bugs in the GHOSTDAG : counters, conntains and isAncestorOf.
    Added more tests.

commit 74493b27d21ed09195eaab164c0f3399312090b5
Author: tal <tal@daglabs.com>
Date:   Thu Oct 22 16:49:27 2020 +0300

    added compare between Hashes

commit f689253463cb46cb5787067aade9c7c6545878bc
Author: tal <tal@daglabs.com>
Date:   Thu Oct 22 11:49:01 2020 +0300

    added compare between Hashes

commit 66be07f6168c8cde92ac68ebbafa45c7ab3484d9
Author: tal <tal@daglabs.com>
Date:   Mon Oct 19 18:42:40 2020 +0300

    First test - pass.

commit 327f34f2dcb5567ef983a7d63e857688e5b8a395
Author: tal <tal@daglabs.com>
Date:   Mon Oct 19 15:20:27 2020 +0300

    Add alternative implementation for ghostdag.
    change all function's signatures (add error type)

commit fd2ea3d84a4f21bed0f1e2a4c6eba549287bef46
Author: tal <tal@daglabs.com>
Date:   Mon Oct 19 11:57:05 2020 +0300

    add alternative implementation for ghostdag
2020-12-01 16:54:13 +02:00