66 Commits

Author SHA1 Message Date
Ori Newman
5566aaf95a
[NOD-1512] Implement utxo deserialization (#1003)
* [NOD-1512] Implement UTXO set deserialization

* [NOD-1512] Remove redundant file

* [NOD-1512] Don't use big endian for serialization

* [NOD-1512] Use Read/Write element

* [NOD-1512] Unexport ReadElement

* [NOD-1512] Fix StageVirtualUTXOSet

* [NOD-1512] Get rid of dagParams in consensusStateManager

* [NOD-1512] Get rid of dagParams in consensusStateManager
2020-11-05 10:59:49 +02:00
Ori Newman
52c73d3a08
[NOD-1511] Implement missingBlockBodyHashes (#1000)
* [NOD-1511] Implement missingBlockBodyHashes

* [NOD-1511] Rename selectedparentiterator.go to blockiterator.go

* [NOD-1511] Fix condition

* [NOD-1511] Simplify missingBlocks logic
2020-11-05 10:11:42 +02:00
Elichai Turkel
6a46cb2be6
[NOD-1503] Pruning Manager (#994)
* Update pruningmanager interface

* Add a ProtoUTXOSet to hashserialization

* Update miningmanager with all the necessary stores and managers

* Implement mining manager

* Prune P.AC not in V.Past

* PruningManager fix all review comments
2020-11-04 10:29:45 +02:00
Ori Newman
7f2ef708a6
[NOD-1506] Implement SetPruningPointUTXOSet (#996)
* [NOD-1506] Implement SetPruningPointUTXOSet

* [NOD-1506] Rename ErrHeaderlessBlockInIBD->ErrMissingBlockHeaderInIBD

* [NOD-1506] Change virtualHeaderHash
2020-11-03 18:24:45 +02:00
Svarog
8dc246a2a7
[NOD-1498] Consensus State Store (#992)
* [NOD-1420] Start working on ConsensusStateManager. Might be redundant due to recent changes

* [NOD-1420] Convert model to externalapi in utxo_algerbra helpers

* [NOD-1420] Add UTXO-diff algebra

* [NOD-1420] Prepare skeleton of calculateAcceptanceDataAndMultiset

* [NOD-1420] Added skeleton for AddBlockToVirtual

* [NOD-1420] Implement PopulateTransactionWithUTXOEntries

* [NOD-1420] Implement restorePastUTXO

* [NOD-1420] Implement finality check

* [NOD-1420] Move handling of tips to consensusStateManager

* [NOD-1420] Implement calculateAcceptanceDataAndMultiset

* [NOD-1420] Start implementing resolveBlockStatus

* [NOD-1420] Implement resolveBlockStatus

* [NOD-1420] Update related fields in end of resolveSingleBlockStatus

* [NOD-1420] Start working on selectVirtualParents

* [NOD-1420] Implemented BlockHeap

* [NOD-1420] Implement selectVirtualParents

* [NOD-1420] Implement updateVirtual

* [NOD-1420] Added comments where they were missing

* [NOD-1420] Place all consensusStateManager functions in correct files

* [NOD-1420] Return the missing outpoints from populateTransactionWithUTXOEntriesFromVirtualOrDiff

* [NOD-1420] Outpoint.ID -> TransactionID

* [NOD-1420] Fix Stringer tests

* [NOD-1420] Copy hash.FromString into utils

* [NOD-1420] SetParents should return an error

* [NOD-1420] Remove all reachabilityManager references from consensusStateManager

* [NOD-1420] Remove VirtualData. Get the info from the stores where needed

* [NOD-1420] Invert parameters to IsAncestorOf

* [NOD-1420] Use model.AcceptanceData

* [NOD-1420] Don't return accumulatedMassBefore in error cases

* [NOD-1420] Don't expect store functions to return nil when the requested data was found - instead add HasXXX functions

* [NOD-1420] addTransactionToMultiset sets isCoinbase properly

* [NOD-1420] expected hash string length is externalapi.DomainHashSize * 2

* [NOD-1420] Rename reachabilityTree -> reachabilityManager + updateReindexRoot if isNextVirtualSelectedParent

* [NOD-1420] ValidateCoinbaseTransaction in csm.verifyAndBuildUTXO

* [NOD-1420] Re-write HAsUTXODiffChild

* [NOD-1420] delete past_utxo.go.bak

* [NOD-1420] Implement validateCoinbaseTransaction in CSM

* [NOD-1420] Imlemented missing functionality in ValidateTransactionAndPopulateWithConsensusData

* [NOD-1420] Moved merge depth logic to MergeDepthManager

* [NOD-1420] Add logs

* [NOD-1498] Implement tips-related methods of consensusStateStore

* [NOD-1498] Implement consensusStateStore virtualDiffParents functionality

* [NOD-1498] Implement ConsensusStateStore UTXO-Set part

* [NOD-1498] Implement rest of consensusStateStore methods

* [NOD-1498] Use io.ReadFull instead of r.Read

* [NOD-1498] Added comments

* [NOD-1498] Move utxo serialization to protobufs

* [NOD-1498] Add comments

* [NOD-1498] Minor fixes in ConsensusStateStore

* [NOD-1498] Use empty bucket key + simplify serializeUTXOEntry
2020-11-03 14:42:26 +02:00
Ori Newman
3f979399b1
[NOD-1478] Implement coinbase manager (#990)
* [NOD-1478] Implement coinbase manager

* [NOD-1478] Add arguments to factory

* [NOD-1478] Remove validation functions from CoinbaseManager

* [NOD-1478] Remove ValidateCoinbaseTransactionInContext

* [NOD-1478] Add consts to constants package

* [NOD-1478] Move scriptPublicKeyMaxLength to constants.go
2020-11-02 16:51:47 +02:00
Svarog
c5707f64dc
[NOD-1420] Implement consensusStateManager (#985)
* [NOD-1420] Start working on ConsensusStateManager. Might be redundant due to recent changes

* [NOD-1420] Convert model to externalapi in utxo_algerbra helpers

* [NOD-1420] Add UTXO-diff algebra

* [NOD-1420] Prepare skeleton of calculateAcceptanceDataAndMultiset

* [NOD-1420] Added skeleton for AddBlockToVirtual

* [NOD-1420] Implement PopulateTransactionWithUTXOEntries

* [NOD-1420] Implement restorePastUTXO

* [NOD-1420] Implement finality check

* [NOD-1420] Move handling of tips to consensusStateManager

* [NOD-1420] Implement calculateAcceptanceDataAndMultiset

* [NOD-1420] Start implementing resolveBlockStatus

* [NOD-1420] Implement resolveBlockStatus

* [NOD-1420] Update related fields in end of resolveSingleBlockStatus

* [NOD-1420] Start working on selectVirtualParents

* [NOD-1420] Implemented BlockHeap

* [NOD-1420] Implement selectVirtualParents

* [NOD-1420] Implement updateVirtual

* [NOD-1420] Added comments where they were missing

* [NOD-1420] Place all consensusStateManager functions in correct files

* [NOD-1420] Return the missing outpoints from populateTransactionWithUTXOEntriesFromVirtualOrDiff

* [NOD-1420] Outpoint.ID -> TransactionID

* [NOD-1420] Fix Stringer tests

* [NOD-1420] Copy hash.FromString into utils

* [NOD-1420] SetParents should return an error

* [NOD-1420] Remove all reachabilityManager references from consensusStateManager

* [NOD-1420] Remove VirtualData. Get the info from the stores where needed

* [NOD-1420] Invert parameters to IsAncestorOf

* [NOD-1420] Use model.AcceptanceData

* [NOD-1420] Don't return accumulatedMassBefore in error cases

* [NOD-1420] Don't expect store functions to return nil when the requested data was found - instead add HasXXX functions

* [NOD-1420] addTransactionToMultiset sets isCoinbase properly

* [NOD-1420] expected hash string length is externalapi.DomainHashSize * 2

* [NOD-1420] Rename reachabilityTree -> reachabilityManager + updateReindexRoot if isNextVirtualSelectedParent

* [NOD-1420] ValidateCoinbaseTransaction in csm.verifyAndBuildUTXO

* [NOD-1420] Re-write HAsUTXODiffChild

* [NOD-1420] delete past_utxo.go.bak

* [NOD-1420] Implement validateCoinbaseTransaction in CSM

* [NOD-1420] Imlemented missing functionality in ValidateTransactionAndPopulateWithConsensusData

* [NOD-1420] Moved merge depth logic to MergeDepthManager

* [NOD-1420] Add logs
2020-11-02 16:18:53 +02:00
Elichai Turkel
87ad9dfc59
[NOD-1423] Refactor the miner and mempool (#981)
* Make TransactionOutputEstimatedSerializedSize public

* Update the mempool interface

* Refactor the mempool to the new design

* refactor txselection and blocktemplatebuilder to the new design

* Update the mining manager

* Update the MiningManager factory

* mempool fix requested changed
2020-11-01 18:27:49 +02:00
Ori Newman
14fbe50636
[NOD-1493] Implement serialization in data stores (#983)
* [NOD-1493] Implement serialization in data stores

* [NOD-1493] Remove redundant functions

* [NOD-1493] Use bluesAnticoneSizesToDBBluesAnticoneSizes inside BlockGHOSTDAGDataToDBBlockGHOSTDAGData
2020-10-29 17:39:35 +02:00
Elichai Turkel
d3ede3a46f
Add new ErrMissingTxOut and ErrInvalidTransactionsInNewBlock errors (#972)
* Add new ErrMissingTxOut error

* Add tests for ruleError wrapping

* Update consensus to use new ErrMissingTxOut type where appropriate

* Add new ErrInvalidTransactionsInNewBlock error

* Add wrapping tests for ErrInvalidTransactionsInNewBlock

* Fix Review suggestions

* Fix broken serialization(add pointer redirection)
2020-10-29 16:59:00 +02:00
stasatdaglabs
01c7c67aed
[NOD-1493] Implement serialization in AcceptanceDataStore, BlockRelationStore, BlockStatusStore, and BlockStore (#982)
* [NOD-1493] Add DbHashToDomainHash and DomainHashToDbHash.

* [NOD-1493] Use DbHashToDomainHash and DomainHashToDbHash.

* [NOD-1493] Begin implementing serializeAcceptanceData.

* [NOD-1493] Extract serialization blockHeader logic to serialization.

* [NOD-1493] Extract serialization acceptance data logic to serialization.

* [NOD-1493] Implement acceptance data serialization/deserialization.

* [NOD-1493] Implement transaction serialization/deserialization.

* [NOD-1493] Implement outpoint serialization/deserialization.

* [NOD-1493] Implement transaction ID serialization/deserialization.

* [NOD-1493] Implement subnetwork ID serialization/deserialization.

* [NOD-1493] Implement block relation serialization/deserialization.

* [NOD-1493] Implement block status serialization/deserialization.

* [NOD-1493] Implement block serialization/deserialization.

* [NOD-1493] Implement serialization/deserialization in BlockRelationStore.

* [NOD-1493] Implement serialization/deserialization in BlockStatusStore.

* [NOD-1493] Implement serialization/deserialization in BlockStore.

* [NOD-1493] Make go vet happy.

* [NOD-1493] Use DomainHashesToDbHashes.
2020-10-29 16:49:05 +02:00
stasatdaglabs
126e2e49bb
[NOD-1493] Implement serialization/deserialization inside BlockHeaderStore (#979)
* [NOD-1492] Rename dbmanager to database.

* [NOD-1492] Write messages.proto for DbBlock and DbTransaction.

* [NOD-1492] Implement serializeHeader.

* [NOD-1492] Implement deserializeHeader.
2020-10-29 11:15:14 +02:00
Ori Newman
eae8bce941
[NOD-1491] Implement block headers store (#976)
* [NOD-1491] Implement block headers store

* [NOD-1491] Don't commit transaction and delete from staging too
2020-10-28 14:34:00 +02:00
stasatdaglabs
97b5b0b875
[NOD-1416] Implement BlockProcessor. (#969)
* [NOD-1416] Add entry/exit logs to all the functions.

* [NOD-1416] Build some scaffolding inside BlockProcessor.

* [NOD-1416] Implement selectParentsForNewBlock.

* [NOD-1416] Implement validateBlock.

* [NOD-1476] Fix merge errors.

* [NOD-1416] Move buildBlock and validateAndInsertBlock to separate files.

* [NOD-1416] Begin implementing buildBlock.

* [NOD-1416] Implement newBlockDifficulty.

* [NOD-1416] Add skeletons for the rest of the buildBlock functions.

* [NOD-1416] Implement newBlockUTXOCommitment.

* [NOD-1416] Implement newBlockAcceptedIDMerkleRoot.

* [NOD-1416] Implement newBlockHashMerkleRoot.

* [NOD-1416] Fix bad function call.

* [NOD-1416] Implement validateHeaderAndProofOfWork and validateBody.

* [NOD-1416] Use ValidateProofOfWorkAndDifficulty.

* [NOD-1416] Finish validateAndInsertBlock.

* [NOD-1416] Implement newBlockHashMerkleRoot.

* [NOD-1416] Implement newBlockAcceptedIDMerkleRoot.

* [NOD-1416] Fix a comment.

* [NOD-1416] Implement newBlockCoinbaseTransaction.

* [NOD-1416] Add VirtualBlockHash.

* [NOD-1416] Add ParentHashes and SelectedParent to VirtualData().

* [NOD-1416] Make go vet happy.

* [NOD-1416] Implement discardAllChanges.

* [NOD-1416] Implement commitAllChanges.

* [NOD-1416] Fix factory.

* [NOD-1416] Make go vet happy.

* [NOD-1416] Format factory.

* [NOD-1416] Pass transactionsWithCoinbase to buildHeader.

* [NOD-1416] Call VirtualData() from buildHeader.

* [NOD-1416] Fix a typo.

* [NOD-1416] Fix in-out-of-context/header-body confusion.

* [NOD-1416] Extract LogAndMeasureExecutionTime.

* [NOD-1416] Add a comment about LogAndMeasureExecutionTime.

* [NOD-1416] Simplify discardAllChanges and commitAllChanges.

* [NOD-1416] If in-context validations fail, discard all changes and store the block with StatusInvalid.

* [NOD-1416] Add a comment above Store.

* [NOD-1416] Use errors.As instead of errors.Is.
2020-10-27 17:24:15 +02:00
Ori Newman
f62183473c
[NOD-1486] Make coinbase mass and size 0 (#970) 2020-10-27 17:14:02 +02:00
Ori Newman
aeb4b96560
[NOD-1451] Implement Validators (#966)
* [NOD-1451] Implement block validator

* [NOD-1451] Implement block validator

* [NOD-1451] Fix merge errors

* [NOD-1451] Implement block validator

* [NOD-1451] Implement checkTransactionInIsolation

* [NOD-1451] Copy txscript to validator

* [NOD-1451] Change txscript to new design

* [NOD-1451] Add checkTransactionInContext

* [NOD-1451] Add checkBlockSize

* [NOD-1451] Add error handling

* [NOD-1451] Implement checkTransactionInContext

* [NOD-1451] Add checkTransactionMass placeholder

* [NOD-1451] Finish validators

* [NOD-1451] Add comments and stringers

* [NOD-1451] Return model.TransactionValidator interface

* [NOD-1451] Premake rule errors for each "code"

* [NOD-1451] Populate transaction mass

* [NOD-1451] Renmae functions

* [NOD-1451] Always use skipPow=false

* [NOD-1451] Renames

* [NOD-1451] Remove redundant types from WriteElement

* [NOD-1451] Fix error message

* [NOD-1451] Add checkTransactionPayload

* [NOD-1451] Add ValidateProofOfWorkAndDifficulty to block validator interface

* [NOD-1451] Move stringers to model

* [NOD-1451] Fix error message
2020-10-26 17:33:39 +02:00