Compare commits

...

32 Commits

Author SHA1 Message Date
Svarog
6db296b98b DownloadHeaders: Instead of using doneChan - close blockHeadersMessageChan. (#1523) 2021-02-11 16:59:31 +02:00
hashdag
05d1671f3a Update README.md 2021-02-11 15:04:28 +02:00
Svarog
ce2758e825 Change Testnet name to kaspad-testnet-2 (#1521)
* Change Testnet name to kaspad-testnet-2

* Fix tests that hardcoded network names
2021-02-11 15:00:21 +02:00
Mike Zak
09545b94b7 Update to version 0.8.8 2021-02-10 18:36:19 +02:00
Elichai Turkel
7c3beb526e Limit stdout log level to info (#1518)
* Rename debuglevel to loglevel

* Limit stdout level to info by default

Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
Co-authored-by: Svarog <feanorr@gmail.com>
2021-02-10 18:35:13 +02:00
Svarog
171deded4e Implement GetBlocks RPC command (#1514)
* Remove BlockHexes from GetBlocks request and response

* Add GetBlocks RPC

* Include the selectedTip's anticone in GetBlocks

* Add Anticone to fakeRelayInvsContext

* Include verbose data only if it was requested + Add comments to HandleGetBlocks

* Allow antiPastHashesBetween to receive unrelated low and high hashes

* Convert to/from protowire GetBlocksResponse with no verbose data correctly

* Removed NextLowHash

* Update GetBlocks in rpc_client

* Validate in consensus.Anticone that blockHash exists

Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2021-02-10 18:27:04 +02:00
stasatdaglabs
94cdc77481 Send peers the hash of the virtual selected parent once connection is established (#1519)
* Send peers the hash of the virtual selected parent once connection is established.

* Add a log to SendVirtualSelectedParentInv.

* Fix TestIBDWithPruning.

* Fix TestIBDWithPruning better and signal from the IBD syncer to the IBD syncee that the DAG is split amongst them.

* Fix TestVirtualSelectedParentChain.

* Add comments.
2021-02-10 18:09:25 +02:00
Svarog
1222a555f2 Prune blocks below pruning point when moving pruning point during IBD (#1513)
* Split deletePastBlocks into sub-routines

* Remove SelectedParentIterator, and refactor SelectedChildIterator to support First and Error

* Implement PruneAllBlocks

* Prune all blocks in the store

* Prune only blocks that are below the pruning point

* Defer call onEnd of LogAndMeasureExecutionTime

* Handle a forgotten error

* Minor style fixes
2021-02-10 16:39:36 +02:00
talelbaz
f13fc35b9e Adds new tests for "BlockAtDepth" function and validate the old tests on DAGTraversal. (#1500)
* Adds tests for the "blockAtDepth" function and verify old other tests.

* Optimization on create the Dag chain.

* Changes according to the review - more detailed error messages, added constants, changed to 3 independent graphs (instead of extending), and changes all the abbreviations.

* Changes according to the review - divide the test into three separate tests and change names to variables.

* Changes according to the review - the order of the function has changed.

* delete double lines

Co-authored-by: tal <tal@daglabs.com>
Co-authored-by: Svarog <feanorr@gmail.com>
2021-02-09 15:28:37 +02:00
Elichai Turkel
2d61a67592 Change some logs (#1511) 2021-02-09 14:00:02 +02:00
stasatdaglabs
3a4fa6e0e1 Add blockVerboseData to blockAddedNotifications (#1508)
* Add blockVerboseData to blockAddedNotifications.

* Run the documentation generator.
2021-02-09 10:30:16 +02:00
Elichai Turkel
2edf6bfd07 Minimize memory usage in tests (#1495)
* Make leveldb cache configurable

* Fix leveldb tests

* Add a preallocate option to all caches and disable in tests

Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2021-02-08 18:37:02 +02:00
Ori Newman
8225f7fb3c Add GetInfo RPC command (#1504)
* Add GetInfo RPC command

* Rename ID to p2p ID
2021-02-08 16:33:21 +02:00
Elichai Turkel
3d0a2a47b2 Move testGHOSTDagSorter to testutils, and build a boilerplate for overriding specific managers (#1486)
* Move testGHOSTDagSorter to testutils

* Allow overriding managers in consensus, starting with ghostdag

* Add test prefix to SetDataDir and SetGHOSTDAGManager

Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2021-02-08 15:24:26 +02:00
Ori Newman
4a354cd538 Validate transactions on BuildBlock (#1491)
* Validate transactions on BuildBlock

* Rename tx -> transactions

* Add transaction validator to block builder constructor and fix TestValidateAndInsertErrors

Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2021-02-08 14:59:43 +02:00
Ori Newman
1a3b16aaa3 Don't change the new reindex root if the blue score of the selected tip is lower than the current reindex root (#1501) 2021-02-08 14:00:53 +02:00
Ori Newman
5b5a7e60af Add aggregated headers processing logs (#1487)
* Add aggregated headers processing logs

* Unite headers and blocks log

Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2021-02-08 10:45:13 +02:00
Ori Newman
d30f05b250 Remove IsPushOnlyScript from mempool validation (#1492)
* Remove IsPushOnlyScript from mempool validation

* Fix TestCheckTransactionStandard
2021-02-08 10:04:19 +02:00
Svarog
6bc7a4eb85 Allow GetMissingBlockBodyHashes return an empty list if the missing blocks were requested before IBD start (#1498)
* Allow GetMissingBlockBodyHashes return an empty list if the missing blocks were requested before IBD start

* Add link to issue in comment about error to be fixed
2021-02-07 16:12:15 +02:00
Ori Newman
608d1f8ef9 Add TestBlueWork (#1488)
* Add TestBlueWork

* Add comments and blue score check

Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2021-02-04 17:12:33 +02:00
stasatdaglabs
a792d4a19e Don't fsync immediately after all writes (#1490) 2021-02-04 16:36:46 +02:00
stasatdaglabs
8941c518fc Remove the no-longer relevant highHashReceived mechanism in syncHeaders. (#1489) 2021-02-04 16:06:20 +02:00
Ori Newman
6f53da18b1 Increase stores cache (#1485)
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2021-02-04 10:06:02 +02:00
stasatdaglabs
44280b9006 Require the --miningaddr parameter in kaspaminer. (#1482)
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-02-04 09:42:02 +02:00
Ori Newman
dbababb978 Limit mempool size to million transactions and remove the least profitable transactions (#1483)
* Limit mempool size to million transactions and remove the least profitable transactions

* Simplify insert

* Fix typo

* Improve findTxIndexInOrderedTransactionsByFeeRate readability
2021-02-03 19:45:39 +02:00
Ori Newman
238950cb98 Add logs (#1484)
* Add logs

* Fix log name
2021-02-03 17:47:59 +02:00
Ori Newman
ee8fa32ff8 Refactor miner and mine when waiting for block to validate (#1481)
* Refactor miner and mine when waiting for block to validate

* Fix -n to work after the refactor.
Change foundBlockChan capacity.
Use lock instead of atomic in the template manager.

* Fix self assignment

* Fix lock

* Fix Dockerfile

* Add comment
2021-02-03 11:53:55 +02:00
Elichai Turkel
e7f9606683 Add dummy go files for test only package, to mitigate golang/go#27333 (#1480)
* Add dummy go files for test only package, to mitigate golang/go#27333

* Stop ignoring errors when producing the coverage

* Add comments explaining the dummy go files

* Make the coverage output non-json
2021-02-02 18:20:15 +02:00
stasatdaglabs
97be133cee Add logs to help debug long virtual parent selection. (#1470)
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-02-01 19:15:50 +02:00
Ori Newman
aeb8e9d2cd Unban address after one day (#1479)
* Unban address after one day

* Unban addresses one by one

Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2021-02-01 18:52:49 +02:00
Ori Newman
b636ae234e Add ban and unban RPC commands (#1478)
* Add ban and unban RPC commands

* Fix names

* Fix commands strings

* Update RPC documentation

* Rename functions

* Simplify return

* Use IP strings in app messages

* Add parse IP error

* Fix wrong condition
2021-02-01 17:34:43 +02:00
Elichai Turkel
a3913dbf80 Update to version 0.9.0 2021-02-01 15:39:39 +02:00
132 changed files with 4538 additions and 1908 deletions

View File

@@ -63,8 +63,7 @@ jobs:
go-version: 1.15
- name: Create coverage file
# Because of https://github.com/golang/go/issues/27333 this seem to "fail" even though nothing is wrong, so ignore the failure
run: go test -json -covermode=atomic -coverpkg=./... -coverprofile coverage.txt ./... || true
run: go test -covermode=atomic -coverpkg=./... -coverprofile coverage.txt ./...
- name: Upload coverage file
run: bash <(curl -s https://codecov.io/bash)

View File

@@ -65,7 +65,7 @@ is used for this project.
## Documentation
The documentation is a work-in-progress.
The [documentation](https://github.com/kaspanet/docs) is a work-in-progress
## License

View File

@@ -22,6 +22,8 @@ import (
"github.com/kaspanet/kaspad/infrastructure/os/winservice"
)
const leveldbCacheSizeMiB = 256
var desiredLimits = &limits.DesiredLimits{
FileLimitWant: 2048,
FileLimitMin: 1024,
@@ -181,5 +183,5 @@ func removeDatabase(cfg *config.Config) error {
func openDB(cfg *config.Config) (database.Database, error) {
dbPath := databasePath(cfg)
log.Infof("Loading database from '%s'", dbPath)
return ldb.NewLevelDB(dbPath)
return ldb.NewLevelDB(dbPath, leveldbCacheSizeMiB)
}

View File

@@ -59,6 +59,7 @@ const (
CmdPruningPointHash
CmdIBDBlockLocator
CmdIBDBlockLocatorHighestHash
CmdIBDBlockLocatorHighestHashNotFound
CmdBlockHeaders
CmdRequestNextPruningPointUTXOSetChunk
CmdDonePruningPointUTXOSetChunks
@@ -123,6 +124,12 @@ const (
CmdNotifyVirtualSelectedParentBlueScoreChangedRequestMessage
CmdNotifyVirtualSelectedParentBlueScoreChangedResponseMessage
CmdVirtualSelectedParentBlueScoreChangedNotificationMessage
CmdBanRequestMessage
CmdBanResponseMessage
CmdUnbanRequestMessage
CmdUnbanResponseMessage
CmdGetInfoRequestMessage
CmdGetInfoResponseMessage
)
// ProtocolMessageCommandToString maps all MessageCommands to their string representation
@@ -156,6 +163,7 @@ var ProtocolMessageCommandToString = map[MessageCommand]string{
CmdPruningPointHash: "PruningPointHash",
CmdIBDBlockLocator: "IBDBlockLocator",
CmdIBDBlockLocatorHighestHash: "IBDBlockLocatorHighestHash",
CmdIBDBlockLocatorHighestHashNotFound: "IBDBlockLocatorHighestHashNotFound",
CmdBlockHeaders: "BlockHeaders",
CmdRequestNextPruningPointUTXOSetChunk: "RequestNextPruningPointUTXOSetChunk",
CmdDonePruningPointUTXOSetChunks: "DonePruningPointUTXOSetChunks",
@@ -220,6 +228,12 @@ var RPCMessageCommandToString = map[MessageCommand]string{
CmdNotifyVirtualSelectedParentBlueScoreChangedRequestMessage: "NotifyVirtualSelectedParentBlueScoreChangedRequest",
CmdNotifyVirtualSelectedParentBlueScoreChangedResponseMessage: "NotifyVirtualSelectedParentBlueScoreChangedResponse",
CmdVirtualSelectedParentBlueScoreChangedNotificationMessage: "VirtualSelectedParentBlueScoreChangedNotification",
CmdBanRequestMessage: "BanRequest",
CmdBanResponseMessage: "BanResponse",
CmdUnbanRequestMessage: "UnbanRequest",
CmdUnbanResponseMessage: "UnbanResponse",
CmdGetInfoRequestMessage: "GetInfoRequestMessage",
CmdGetInfoResponseMessage: "GeInfoResponseMessage",
}
// Message is an interface that describes a kaspa message. A type that

View File

@@ -0,0 +1,16 @@
package appmessage
// MsgIBDBlockLocatorHighestHashNotFound represents a kaspa BlockLocatorHighestHashNotFound message
type MsgIBDBlockLocatorHighestHashNotFound struct {
baseMessage
}
// Command returns the protocol command string for the message
func (msg *MsgIBDBlockLocatorHighestHashNotFound) Command() MessageCommand {
return CmdIBDBlockLocatorHighestHashNotFound
}
// NewMsgIBDBlockLocatorHighestHashNotFound returns a new IBDBlockLocatorHighestHashNotFound message
func NewMsgIBDBlockLocatorHighestHashNotFound() *MsgIBDBlockLocatorHighestHashNotFound {
return &MsgIBDBlockLocatorHighestHashNotFound{}
}

39
app/appmessage/rpc_ban.go Normal file
View File

@@ -0,0 +1,39 @@
package appmessage
// BanRequestMessage is an appmessage corresponding to
// its respective RPC message
type BanRequestMessage struct {
baseMessage
IP string
}
// Command returns the protocol command string for the message
func (msg *BanRequestMessage) Command() MessageCommand {
return CmdBanRequestMessage
}
// NewBanRequestMessage returns an instance of the message
func NewBanRequestMessage(ip string) *BanRequestMessage {
return &BanRequestMessage{
IP: ip,
}
}
// BanResponseMessage is an appmessage corresponding to
// its respective RPC message
type BanResponseMessage struct {
baseMessage
Error *RPCError
}
// Command returns the protocol command string for the message
func (msg *BanResponseMessage) Command() MessageCommand {
return CmdBanResponseMessage
}
// NewBanResponseMessage returns a instance of the message
func NewBanResponseMessage() *BanResponseMessage {
return &BanResponseMessage{}
}

View File

@@ -4,9 +4,9 @@ package appmessage
// its respective RPC message
type GetBlocksRequestMessage struct {
baseMessage
LowHash string
IncludeBlockHexes bool
IncludeBlockVerboseData bool
LowHash string
IncludeBlockVerboseData bool
IncludeTransactionVerboseData bool
}
// Command returns the protocol command string for the message
@@ -15,11 +15,12 @@ func (msg *GetBlocksRequestMessage) Command() MessageCommand {
}
// NewGetBlocksRequestMessage returns a instance of the message
func NewGetBlocksRequestMessage(lowHash string, includeBlockHexes bool, includeBlockVerboseData bool) *GetBlocksRequestMessage {
func NewGetBlocksRequestMessage(lowHash string, includeBlockVerboseData bool,
includeTransactionVerboseData bool) *GetBlocksRequestMessage {
return &GetBlocksRequestMessage{
LowHash: lowHash,
IncludeBlockHexes: includeBlockHexes,
IncludeBlockVerboseData: includeBlockVerboseData,
LowHash: lowHash,
IncludeBlockVerboseData: includeBlockVerboseData,
IncludeTransactionVerboseData: includeTransactionVerboseData,
}
}
@@ -28,7 +29,6 @@ func NewGetBlocksRequestMessage(lowHash string, includeBlockHexes bool, includeB
type GetBlocksResponseMessage struct {
baseMessage
BlockHashes []string
BlockHexes []string
BlockVerboseData []*BlockVerboseData
Error *RPCError
@@ -45,7 +45,6 @@ func NewGetBlocksResponseMessage(blockHashes []string, blockHexes []string,
return &GetBlocksResponseMessage{
BlockHashes: blockHashes,
BlockHexes: blockHexes,
BlockVerboseData: blockVerboseData,
}
}

View File

@@ -0,0 +1,38 @@
package appmessage
// GetInfoRequestMessage is an appmessage corresponding to
// its respective RPC message
type GetInfoRequestMessage struct {
baseMessage
}
// Command returns the protocol command string for the message
func (msg *GetInfoRequestMessage) Command() MessageCommand {
return CmdGetInfoRequestMessage
}
// NewGeInfoRequestMessage returns a instance of the message
func NewGeInfoRequestMessage() *GetInfoRequestMessage {
return &GetInfoRequestMessage{}
}
// GetInfoResponseMessage is an appmessage corresponding to
// its respective RPC message
type GetInfoResponseMessage struct {
baseMessage
P2PID string
Error *RPCError
}
// Command returns the protocol command string for the message
func (msg *GetInfoResponseMessage) Command() MessageCommand {
return CmdGetInfoResponseMessage
}
// NewGetInfoResponseMessage returns a instance of the message
func NewGetInfoResponseMessage(p2pID string) *GetInfoResponseMessage {
return &GetInfoResponseMessage{
P2PID: p2pID,
}
}

View File

@@ -37,7 +37,8 @@ func NewNotifyBlockAddedResponseMessage() *NotifyBlockAddedResponseMessage {
// its respective RPC message
type BlockAddedNotificationMessage struct {
baseMessage
Block *MsgBlock
Block *MsgBlock
BlockVerboseData *BlockVerboseData
}
// Command returns the protocol command string for the message
@@ -46,8 +47,9 @@ func (msg *BlockAddedNotificationMessage) Command() MessageCommand {
}
// NewBlockAddedNotificationMessage returns a instance of the message
func NewBlockAddedNotificationMessage(block *MsgBlock) *BlockAddedNotificationMessage {
func NewBlockAddedNotificationMessage(block *MsgBlock, blockVerboseData *BlockVerboseData) *BlockAddedNotificationMessage {
return &BlockAddedNotificationMessage{
Block: block,
Block: block,
BlockVerboseData: blockVerboseData,
}
}

View File

@@ -0,0 +1,39 @@
package appmessage
// UnbanRequestMessage is an appmessage corresponding to
// its respective RPC message
type UnbanRequestMessage struct {
baseMessage
IP string
}
// Command returns the protocol command string for the message
func (msg *UnbanRequestMessage) Command() MessageCommand {
return CmdUnbanRequestMessage
}
// NewUnbanRequestMessage returns an instance of the message
func NewUnbanRequestMessage(ip string) *UnbanRequestMessage {
return &UnbanRequestMessage{
IP: ip,
}
}
// UnbanResponseMessage is an appmessage corresponding to
// its respective RPC message
type UnbanResponseMessage struct {
baseMessage
Error *RPCError
}
// Command returns the protocol command string for the message
func (msg *UnbanResponseMessage) Command() MessageCommand {
return CmdUnbanResponseMessage
}
// NewUnbanResponseMessage returns a instance of the message
func NewUnbanResponseMessage() *UnbanResponseMessage {
return &UnbanResponseMessage{}
}

View File

@@ -1,7 +1,6 @@
package flowcontext
import (
"github.com/kaspanet/kaspad/app/protocol/blocklogger"
peerpkg "github.com/kaspanet/kaspad/app/protocol/peer"
"github.com/kaspanet/kaspad/domain/consensus/ruleerrors"
"github.com/pkg/errors"
@@ -38,8 +37,6 @@ func (f *FlowContext) OnNewBlock(block *externalapi.DomainBlock,
}
for i, newBlock := range newBlocks {
blocklogger.LogBlock(block)
log.Debugf("OnNewBlock: passing block %s transactions to mining manager", hash)
_, err = f.Domain().MiningManager().HandleNewBlockTransactions(newBlock.Transactions)
if err != nil {

View File

@@ -194,6 +194,9 @@ func (f *FlowContext) GetOrphanRoots(orphan *externalapi.DomainHash) ([]*externa
if !blockInfo.Exists || blockInfo.BlockStatus == externalapi.StatusHeaderOnly {
roots = append(roots, current)
} else {
log.Debugf("Block %s was skipped when checking for orphan roots: "+
"exists: %t, status: %s", current, blockInfo.Exists, blockInfo.BlockStatus)
}
continue
}

View File

@@ -70,8 +70,14 @@ func HandleIBDBlockLocator(context HandleIBDBlockLocatorContext, incomingRoute *
}
if !foundHighestHashInTheSelectedParentChainOfTargetHash {
return protocolerrors.Errorf(true, "no hash was found in the blockLocator "+
log.Warnf("no hash was found in the blockLocator "+
"that was in the selected parent chain of targetHash %s", targetHash)
ibdBlockLocatorHighestHashNotFoundMessage := appmessage.NewMsgIBDBlockLocatorHighestHashNotFound()
err = outgoingRoute.Enqueue(ibdBlockLocatorHighestHashNotFoundMessage)
if err != nil {
return err
}
}
}
}

View File

@@ -28,10 +28,14 @@ func (flow *handleRelayInvsFlow) runIBDIfNotRunning(highHash *externalapi.Domain
log.Debugf("IBD started with peer %s and highHash %s", flow.peer, highHash)
log.Debugf("Syncing headers up to %s", highHash)
err := flow.syncHeaders(highHash)
headersSynced, err := flow.syncHeaders(highHash)
if err != nil {
return err
}
if !headersSynced {
log.Debugf("Aborting IBD because the headers failed to sync")
return nil
}
log.Debugf("Finished syncing headers up to %s", highHash)
log.Debugf("Syncing the current pruning point UTXO set")
@@ -55,47 +59,61 @@ func (flow *handleRelayInvsFlow) runIBDIfNotRunning(highHash *externalapi.Domain
return nil
}
func (flow *handleRelayInvsFlow) syncHeaders(highHash *externalapi.DomainHash) error {
highHashReceived := false
for !highHashReceived {
log.Debugf("Trying to find highest shared chain block with peer %s with high hash %s", flow.peer, highHash)
highestSharedBlockHash, err := flow.findHighestSharedBlockHash(highHash)
if err != nil {
return err
}
log.Debugf("Found highest shared chain block %s with peer %s", highestSharedBlockHash, flow.peer)
err = flow.downloadHeaders(highestSharedBlockHash, highHash)
if err != nil {
return err
}
// We're finished once highHash has been inserted into the DAG
blockInfo, err := flow.Domain().Consensus().GetBlockInfo(highHash)
if err != nil {
return err
}
highHashReceived = blockInfo.Exists
log.Debugf("Headers downloaded from peer %s. Are further headers required: %t", flow.peer, !highHashReceived)
// syncHeaders attempts to sync headers from the peer. This method may fail
// because the peer and us have conflicting pruning points. In that case we
// return (false, nil) so that we may stop IBD gracefully.
func (flow *handleRelayInvsFlow) syncHeaders(highHash *externalapi.DomainHash) (bool, error) {
log.Debugf("Trying to find highest shared chain block with peer %s with high hash %s", flow.peer, highHash)
highestSharedBlockHash, highestSharedBlockFound, err := flow.findHighestSharedBlockHash(highHash)
if err != nil {
return false, err
}
return nil
if !highestSharedBlockFound {
return false, nil
}
log.Debugf("Found highest shared chain block %s with peer %s", highestSharedBlockHash, flow.peer)
err = flow.downloadHeaders(highestSharedBlockHash, highHash)
if err != nil {
return false, err
}
// If the highHash has not been received, the peer is misbehaving
highHashBlockInfo, err := flow.Domain().Consensus().GetBlockInfo(highHash)
if err != nil {
return false, err
}
if !highHashBlockInfo.Exists {
return false, protocolerrors.Errorf(true, "did not receive "+
"highHash header %s from peer %s during header download", highHash, flow.peer)
}
log.Debugf("Headers downloaded from peer %s", flow.peer)
return true, nil
}
func (flow *handleRelayInvsFlow) findHighestSharedBlockHash(targetHash *externalapi.DomainHash) (*externalapi.DomainHash, error) {
// findHighestSharedBlock attempts to find the highest shared block between the peer
// and this node. This method may fail because the peer and us have conflicting pruning
// points. In that case we return (nil, false, nil) so that we may stop IBD gracefully.
func (flow *handleRelayInvsFlow) findHighestSharedBlockHash(
targetHash *externalapi.DomainHash) (*externalapi.DomainHash, bool, error) {
log.Debugf("Sending a blockLocator to %s between pruning point and headers selected tip", flow.peer)
blockLocator, err := flow.Domain().Consensus().CreateFullHeadersSelectedChainBlockLocator()
if err != nil {
return nil, err
return nil, false, err
}
for {
highestHash, err := flow.fetchHighestHash(targetHash, blockLocator)
highestHash, highestHashFound, err := flow.fetchHighestHash(targetHash, blockLocator)
if err != nil {
return nil, err
return nil, false, err
}
if !highestHashFound {
return nil, false, nil
}
highestHashIndex, err := flow.findHighestHashIndex(highestHash, blockLocator)
if err != nil {
return nil, err
return nil, false, err
}
if highestHashIndex == 0 ||
@@ -104,7 +122,7 @@ func (flow *handleRelayInvsFlow) findHighestSharedBlockHash(targetHash *external
// an endless loop, we explicitly stop the loop in such situation.
(len(blockLocator) == 2 && highestHashIndex == 1) {
return highestHash, nil
return highestHash, true, nil
}
locatorHashAboveHighestHash := highestHash
@@ -114,7 +132,7 @@ func (flow *handleRelayInvsFlow) findHighestSharedBlockHash(targetHash *external
blockLocator, err = flow.nextBlockLocator(highestHash, locatorHashAboveHighestHash)
if err != nil {
return nil, err
return nil, false, err
}
}
}
@@ -159,27 +177,35 @@ func (flow *handleRelayInvsFlow) findHighestHashIndex(
return highestHashIndex, nil
}
// fetchHighestHash attempts to fetch the highest hash the peer knows amongst the given
// blockLocator. This method may fail because the peer and us have conflicting pruning
// points. In that case we return (nil, false, nil) so that we may stop IBD gracefully.
func (flow *handleRelayInvsFlow) fetchHighestHash(
targetHash *externalapi.DomainHash, blockLocator externalapi.BlockLocator) (*externalapi.DomainHash, error) {
targetHash *externalapi.DomainHash, blockLocator externalapi.BlockLocator) (*externalapi.DomainHash, bool, error) {
ibdBlockLocatorMessage := appmessage.NewMsgIBDBlockLocator(targetHash, blockLocator)
err := flow.outgoingRoute.Enqueue(ibdBlockLocatorMessage)
if err != nil {
return nil, err
return nil, false, err
}
message, err := flow.dequeueIncomingMessageAndSkipInvs(common.DefaultTimeout)
if err != nil {
return nil, err
return nil, false, err
}
ibdBlockLocatorHighestHashMessage, ok := message.(*appmessage.MsgIBDBlockLocatorHighestHash)
if !ok {
return nil, protocolerrors.Errorf(true, "received unexpected message type. "+
switch message := message.(type) {
case *appmessage.MsgIBDBlockLocatorHighestHash:
highestHash := message.HighestHash
log.Debugf("The highest hash the peer %s knows is %s", flow.peer, highestHash)
return highestHash, true, nil
case *appmessage.MsgIBDBlockLocatorHighestHashNotFound:
log.Debugf("Peer %s does not know any block within our blockLocator. "+
"This should only happen if there's a DAG split deeper than the pruning point.", flow.peer)
return nil, false, nil
default:
return nil, false, protocolerrors.Errorf(true, "received unexpected message type. "+
"expected: %s, got: %s", appmessage.CmdIBDBlockLocatorHighestHash, message.Command())
}
highestHash := ibdBlockLocatorHighestHashMessage.HighestHash
log.Debugf("The highest hash the peer %s knows is %s", flow.peer, highestHash)
return highestHash, nil
}
func (flow *handleRelayInvsFlow) downloadHeaders(highestSharedBlockHash *externalapi.DomainHash,
@@ -195,7 +221,6 @@ func (flow *handleRelayInvsFlow) downloadHeaders(highestSharedBlockHash *externa
// headers
blockHeadersMessageChan := make(chan *appmessage.BlockHeadersMessage, 2)
errChan := make(chan error)
doneChan := make(chan interface{})
spawn("handleRelayInvsFlow-downloadHeaders", func() {
for {
blockHeadersMessage, doneIBD, err := flow.receiveHeaders()
@@ -204,7 +229,7 @@ func (flow *handleRelayInvsFlow) downloadHeaders(highestSharedBlockHash *externa
return
}
if doneIBD {
doneChan <- struct{}{}
close(blockHeadersMessageChan)
return
}
@@ -220,7 +245,10 @@ func (flow *handleRelayInvsFlow) downloadHeaders(highestSharedBlockHash *externa
for {
select {
case blockHeadersMessage := <-blockHeadersMessageChan:
case blockHeadersMessage, ok := <-blockHeadersMessageChan:
if !ok {
return nil
}
for _, header := range blockHeadersMessage.BlockHeaders {
err = flow.processHeader(header)
if err != nil {
@@ -229,8 +257,6 @@ func (flow *handleRelayInvsFlow) downloadHeaders(highestSharedBlockHash *externa
}
case err := <-errChan:
return err
case <-doneChan:
return nil
}
}
}
@@ -468,6 +494,13 @@ func (flow *handleRelayInvsFlow) syncMissingBlockBodies(highHash *externalapi.Do
if err != nil {
return err
}
if len(hashes) == 0 {
// Blocks can be inserted inside the DAG during IBD if those were requested before IBD started.
// In rare cases, all the IBD blocks might be already inserted by the time we reach this point.
// In these cases - GetMissingBlockBodyHashes would return an empty array.
log.Debugf("No missing block body hashes found.")
return nil
}
for offset := 0; offset < len(hashes); offset += ibdBatchSize {
var hashesToRequest []*externalapi.DomainHash

View File

@@ -0,0 +1,28 @@
package blockrelay
import (
"github.com/kaspanet/kaspad/app/appmessage"
peerpkg "github.com/kaspanet/kaspad/app/protocol/peer"
"github.com/kaspanet/kaspad/domain"
"github.com/kaspanet/kaspad/infrastructure/network/netadapter/router"
)
// SendVirtualSelectedParentInvContext is the interface for the context needed for the SendVirtualSelectedParentInv flow.
type SendVirtualSelectedParentInvContext interface {
Domain() domain.Domain
}
// SendVirtualSelectedParentInv sends a peer the selected parent hash of the virtual
func SendVirtualSelectedParentInv(context SendVirtualSelectedParentInvContext,
outgoingRoute *router.Route, peer *peerpkg.Peer) error {
virtualSelectedParent, err := context.Domain().Consensus().GetVirtualSelectedParent()
if err != nil {
return err
}
log.Debugf("Sending virtual selected parent hash %s to peer %s", virtualSelectedParent, peer)
virtualSelectedParentInv := appmessage.NewMsgInvBlock(virtualSelectedParent)
return outgoingRoute.Enqueue(virtualSelectedParentInv)
}

View File

@@ -2,6 +2,10 @@ package testing
import (
"fmt"
"sync"
"testing"
"time"
"github.com/kaspanet/kaspad/app/appmessage"
"github.com/kaspanet/kaspad/app/protocol/flows/blockrelay"
peerpkg "github.com/kaspanet/kaspad/app/protocol/peer"
@@ -18,9 +22,6 @@ import (
"github.com/kaspanet/kaspad/infrastructure/network/netadapter/router"
"github.com/kaspanet/kaspad/util/mstime"
"github.com/pkg/errors"
"sync"
"testing"
"time"
)
var orphanBlock = &externalapi.DomainBlock{
@@ -109,6 +110,10 @@ type fakeRelayInvsContext struct {
rwLock sync.RWMutex
}
func (f *fakeRelayInvsContext) Anticone(blockHash *externalapi.DomainHash) ([]*externalapi.DomainHash, error) {
panic(errors.Errorf("called unimplemented function from test '%s'", f.testName))
}
func (f *fakeRelayInvsContext) BuildBlock(coinbaseData *externalapi.DomainCoinbaseData, transactions []*externalapi.DomainTransaction) (*externalapi.DomainBlock, error) {
panic(errors.Errorf("called unimplemented function from test '%s'", f.testName))
}

View File

@@ -0,0 +1,4 @@
package testing
// Because of a bug in Go coverage fails if you have packages with test files only. See https://github.com/golang/go/issues/27333
// So this is a dummy non-test go file in the package.

View File

@@ -2,6 +2,7 @@ package protocol
import (
"github.com/kaspanet/kaspad/app/protocol/flows/rejects"
"github.com/kaspanet/kaspad/infrastructure/network/connmanager"
"sync/atomic"
"github.com/kaspanet/kaspad/app/appmessage"
@@ -78,7 +79,11 @@ func (m *Manager) handleError(err error, netConnection *netadapter.NetConnection
if !m.context.Config().DisableBanning && protocolErr.ShouldBan {
log.Warnf("Banning %s (reason: %s)", netConnection, protocolErr.Cause)
m.context.ConnectionManager().Ban(netConnection)
err := m.context.ConnectionManager().Ban(netConnection)
if !errors.Is(err, connmanager.ErrCannotBanPermanent) {
panic(err)
}
err = outgoingRoute.Enqueue(appmessage.NewMsgReject(protocolErr.Error()))
if err != nil && !errors.Is(err, routerpkg.ErrRouteClosed) {
panic(err)
@@ -131,11 +136,16 @@ func (m *Manager) registerBlockRelayFlows(router *routerpkg.Router, isStopping *
outgoingRoute := router.OutgoingRoute()
return []*flow{
m.registerOneTimeFlow("SendVirtualSelectedParentInv", router, []appmessage.MessageCommand{},
isStopping, errChan, func(route *routerpkg.Route, peer *peerpkg.Peer) error {
return blockrelay.SendVirtualSelectedParentInv(m.context, outgoingRoute, peer)
}),
m.registerFlow("HandleRelayInvs", router, []appmessage.MessageCommand{
appmessage.CmdInvRelayBlock, appmessage.CmdBlock, appmessage.CmdBlockLocator, appmessage.CmdIBDBlock,
appmessage.CmdDoneHeaders, appmessage.CmdUnexpectedPruningPoint, appmessage.CmdPruningPointUTXOSetChunk,
appmessage.CmdBlockHeaders, appmessage.CmdPruningPointHash, appmessage.CmdIBDBlockLocatorHighestHash,
appmessage.CmdDonePruningPointUTXOSetChunks},
appmessage.CmdIBDBlockLocatorHighestHashNotFound, appmessage.CmdDonePruningPointUTXOSetChunks},
isStopping, errChan, func(incomingRoute *routerpkg.Route, peer *peerpkg.Peer) error {
return blockrelay.HandleRelayInvs(m.context, incomingRoute,
outgoingRoute, peer)

View File

@@ -69,7 +69,12 @@ func (m *Manager) NotifyBlockAddedToDAG(block *externalapi.DomainBlock, blockIns
return err
}
blockAddedNotification := appmessage.NewBlockAddedNotificationMessage(appmessage.DomainBlockToMsgBlock(block))
msgBlock := appmessage.DomainBlockToMsgBlock(block)
blockVerboseData, err := m.context.BuildBlockVerboseData(block.Header, block, false)
if err != nil {
return err
}
blockAddedNotification := appmessage.NewBlockAddedNotificationMessage(msgBlock, blockVerboseData)
return m.context.NotificationManager.NotifyBlockAdded(blockAddedNotification)
}

View File

@@ -38,6 +38,9 @@ var handlers = map[appmessage.MessageCommand]handler{
appmessage.CmdGetUTXOsByAddressesRequestMessage: rpchandlers.HandleGetUTXOsByAddresses,
appmessage.CmdGetVirtualSelectedParentBlueScoreRequestMessage: rpchandlers.HandleGetVirtualSelectedParentBlueScore,
appmessage.CmdNotifyVirtualSelectedParentBlueScoreChangedRequestMessage: rpchandlers.HandleNotifyVirtualSelectedParentBlueScoreChanged,
appmessage.CmdBanRequestMessage: rpchandlers.HandleBan,
appmessage.CmdUnbanRequestMessage: rpchandlers.HandleUnban,
appmessage.CmdGetInfoRequestMessage: rpchandlers.HandleGetInfo,
}
func (m *Manager) routerInitializer(router *router.Router, netConnection *netadapter.NetConnection) {

View File

@@ -4,6 +4,7 @@ import (
"encoding/hex"
"fmt"
"github.com/kaspanet/kaspad/domain/consensus/utils/constants"
"github.com/kaspanet/kaspad/infrastructure/logger"
"github.com/kaspanet/kaspad/util/difficulty"
"math"
"math/big"
@@ -23,8 +24,14 @@ import (
"github.com/kaspanet/kaspad/util/pointers"
)
// BuildBlockVerboseData builds a BlockVerboseData from the given block.
func (ctx *Context) BuildBlockVerboseData(blockHeader externalapi.BlockHeader, includeTransactionVerboseData bool) (*appmessage.BlockVerboseData, error) {
// BuildBlockVerboseData builds a BlockVerboseData from the given blockHeader.
// A block may optionally also be given if it's available in the calling context.
func (ctx *Context) BuildBlockVerboseData(blockHeader externalapi.BlockHeader, block *externalapi.DomainBlock,
includeTransactionVerboseData bool) (*appmessage.BlockVerboseData, error) {
onEnd := logger.LogAndMeasureExecutionTime(log, "BuildBlockVerboseData")
defer onEnd()
hash := consensushashing.HeaderHash(blockHeader)
blockInfo, err := ctx.Domain.Consensus().GetBlockInfo(hash)
@@ -48,9 +55,11 @@ func (ctx *Context) BuildBlockVerboseData(blockHeader externalapi.BlockHeader, i
}
if blockInfo.BlockStatus != externalapi.StatusHeaderOnly {
block, err := ctx.Domain.Consensus().GetBlock(hash)
if err != nil {
return nil, err
if block == nil {
block, err = ctx.Domain.Consensus().GetBlock(hash)
if err != nil {
return nil, err
}
}
txIDs := make([]string, len(block.Transactions))
@@ -100,6 +109,9 @@ func (ctx *Context) BuildTransactionVerboseData(tx *externalapi.DomainTransactio
blockHeader externalapi.BlockHeader, blockHash string) (
*appmessage.TransactionVerboseData, error) {
onEnd := logger.LogAndMeasureExecutionTime(log, "BuildTransactionVerboseData")
defer onEnd()
var payloadHash string
if tx.SubnetworkID != subnetworks.SubnetworkIDNative {
payloadHash = tx.PayloadHash.String()

View File

@@ -0,0 +1,28 @@
package rpchandlers
import (
"github.com/kaspanet/kaspad/app/appmessage"
"github.com/kaspanet/kaspad/app/rpc/rpccontext"
"github.com/kaspanet/kaspad/infrastructure/network/netadapter/router"
"net"
)
// HandleBan handles the respectively named RPC command
func HandleBan(context *rpccontext.Context, _ *router.Router, request appmessage.Message) (appmessage.Message, error) {
banRequest := request.(*appmessage.BanRequestMessage)
ip := net.ParseIP(banRequest.IP)
if ip == nil {
errorMessage := &appmessage.BanResponseMessage{}
errorMessage.Error = appmessage.RPCErrorf("Could not parse IP %s", banRequest.IP)
return errorMessage, nil
}
err := context.ConnectionManager.BanByIP(ip)
if err != nil {
errorMessage := &appmessage.BanResponseMessage{}
errorMessage.Error = appmessage.RPCErrorf("Could not ban IP: %s", err)
return errorMessage, nil
}
response := appmessage.NewBanResponseMessage()
return response, nil
}

View File

@@ -28,7 +28,7 @@ func HandleGetBlock(context *rpccontext.Context, _ *router.Router, request appme
response := appmessage.NewGetBlockResponseMessage()
blockVerboseData, err := context.BuildBlockVerboseData(header, getBlockRequest.IncludeTransactionVerboseData)
blockVerboseData, err := context.BuildBlockVerboseData(header, nil, getBlockRequest.IncludeTransactionVerboseData)
if err != nil {
return nil, err
}

View File

@@ -3,6 +3,8 @@ package rpchandlers
import (
"github.com/kaspanet/kaspad/app/appmessage"
"github.com/kaspanet/kaspad/app/rpc/rpccontext"
"github.com/kaspanet/kaspad/domain/consensus/model/externalapi"
"github.com/kaspanet/kaspad/domain/consensus/utils/hashes"
"github.com/kaspanet/kaspad/infrastructure/network/netadapter/router"
)
@@ -14,7 +16,77 @@ const (
// HandleGetBlocks handles the respectively named RPC command
func HandleGetBlocks(context *rpccontext.Context, _ *router.Router, request appmessage.Message) (appmessage.Message, error) {
response := &appmessage.GetBlocksResponseMessage{}
response.Error = appmessage.RPCErrorf("not implemented")
getBlocksRequest := request.(*appmessage.GetBlocksRequestMessage)
// Validate that user didn't set IncludeTransactionVerboseData without setting IncludeBlockVerboseData
if !getBlocksRequest.IncludeBlockVerboseData && getBlocksRequest.IncludeTransactionVerboseData {
return &appmessage.GetBlocksResponseMessage{
Error: appmessage.RPCErrorf(
"If includeTransactionVerboseData is set, then includeBlockVerboseData must be set as well"),
}, nil
}
// Decode lowHash
// If lowHash is empty - use genesis instead.
lowHash := context.Config.ActiveNetParams.GenesisHash
if getBlocksRequest.LowHash != "" {
var err error
lowHash, err = externalapi.NewDomainHashFromString(getBlocksRequest.LowHash)
if err != nil {
return &appmessage.GetBlocksResponseMessage{
Error: appmessage.RPCErrorf("Could not decode lowHash %s: %s", getBlocksRequest.LowHash, err),
}, nil
}
}
// Get hashes between lowHash and virtualSelectedParent
virtualSelectedParent, err := context.Domain.Consensus().GetVirtualSelectedParent()
if err != nil {
return nil, err
}
blockHashes, err := context.Domain.Consensus().GetHashesBetween(
lowHash, virtualSelectedParent, maxBlocksInGetBlocksResponse)
if err != nil {
return nil, err
}
// If there are no maxBlocksInGetBlocksResponse between lowHash and virtualSelectedParent -
// add virtualSelectedParent's anticone
if len(blockHashes) < maxBlocksInGetBlocksResponse {
virtualSelectedParentAnticone, err := context.Domain.Consensus().Anticone(virtualSelectedParent)
if err != nil {
return nil, err
}
blockHashes = append(blockHashes, virtualSelectedParentAnticone...)
}
// Both GetHashesBetween and Anticone might return more then the allowed number of blocks, so
// trim any extra blocks.
if len(blockHashes) > maxBlocksInGetBlocksResponse {
blockHashes = blockHashes[:maxBlocksInGetBlocksResponse]
}
// Prepare the response
response := &appmessage.GetBlocksResponseMessage{
BlockHashes: hashes.ToStrings(blockHashes),
}
// Retrieve all block data in case BlockVerboseData was requested
if getBlocksRequest.IncludeBlockVerboseData {
response.BlockVerboseData = make([]*appmessage.BlockVerboseData, len(blockHashes))
for i, blockHash := range blockHashes {
blockHeader, err := context.Domain.Consensus().GetBlockHeader(blockHash)
if err != nil {
return nil, err
}
blockVerboseData, err := context.BuildBlockVerboseData(blockHeader, nil,
getBlocksRequest.IncludeTransactionVerboseData)
if err != nil {
return nil, err
}
response.BlockVerboseData[i] = blockVerboseData
}
}
return response, nil
}

View File

@@ -0,0 +1,13 @@
package rpchandlers
import (
"github.com/kaspanet/kaspad/app/appmessage"
"github.com/kaspanet/kaspad/app/rpc/rpccontext"
"github.com/kaspanet/kaspad/infrastructure/network/netadapter/router"
)
// HandleGetInfo handles the respectively named RPC command
func HandleGetInfo(context *rpccontext.Context, _ *router.Router, _ appmessage.Message) (appmessage.Message, error) {
response := appmessage.NewGetInfoResponseMessage(context.NetAdapter.ID().String())
return response, nil
}

View File

@@ -0,0 +1,27 @@
package rpchandlers
import (
"github.com/kaspanet/kaspad/app/appmessage"
"github.com/kaspanet/kaspad/app/rpc/rpccontext"
"github.com/kaspanet/kaspad/infrastructure/network/netadapter/router"
"net"
)
// HandleUnban handles the respectively named RPC command
func HandleUnban(context *rpccontext.Context, _ *router.Router, request appmessage.Message) (appmessage.Message, error) {
unbanRequest := request.(*appmessage.UnbanRequestMessage)
ip := net.ParseIP(unbanRequest.IP)
if ip == nil {
errorMessage := &appmessage.UnbanResponseMessage{}
errorMessage.Error = appmessage.RPCErrorf("Could not parse IP %s", unbanRequest.IP)
return errorMessage, nil
}
err := context.AddressManager.Unban(appmessage.NewNetAddressIPPort(ip, 0, 0))
if err != nil {
errorMessage := &appmessage.UnbanResponseMessage{}
errorMessage.Error = appmessage.RPCErrorf("Could not unban IP: %s", err)
return errorMessage, nil
}
response := appmessage.NewUnbanResponseMessage()
return response, nil
}

View File

@@ -13,6 +13,7 @@ var commandTypes = []reflect.Type{
reflect.TypeOf(protowire.KaspadMessage_GetConnectedPeerInfoRequest{}),
reflect.TypeOf(protowire.KaspadMessage_GetPeerAddressesRequest{}),
reflect.TypeOf(protowire.KaspadMessage_GetCurrentNetworkRequest{}),
reflect.TypeOf(protowire.KaspadMessage_GetInfoRequest{}),
reflect.TypeOf(protowire.KaspadMessage_GetBlockRequest{}),
reflect.TypeOf(protowire.KaspadMessage_GetBlocksRequest{}),
@@ -32,6 +33,9 @@ var commandTypes = []reflect.Type{
reflect.TypeOf(protowire.KaspadMessage_SubmitTransactionRequest{}),
reflect.TypeOf(protowire.KaspadMessage_GetUtxosByAddressesRequest{}),
reflect.TypeOf(protowire.KaspadMessage_BanRequest{}),
reflect.TypeOf(protowire.KaspadMessage_UnbanRequest{}),
}
type commandDescription struct {

View File

@@ -77,6 +77,10 @@ func parseConfig() (*configFlags, error) {
}
}
if cfg.MiningAddr == "" {
return nil, errors.New("--miningaddr is required")
}
initLog(defaultLogFile, defaultErrLogFile)
return cfg, nil

View File

@@ -2,13 +2,13 @@ package main
import (
nativeerrors "errors"
"github.com/kaspanet/kaspad/cmd/kaspaminer/templatemanager"
"github.com/kaspanet/kaspad/domain/consensus/model/pow"
"github.com/kaspanet/kaspad/util/difficulty"
"math/rand"
"sync/atomic"
"time"
"github.com/kaspanet/kaspad/domain/consensus/model/pow"
"github.com/kaspanet/kaspad/domain/consensus/utils/consensushashing"
"github.com/kaspanet/kaspad/domain/consensus/model/externalapi"
@@ -29,11 +29,19 @@ func mineLoop(client *minerClient, numberOfBlocks uint64, targetBlocksPerSecond
rand.Seed(time.Now().UnixNano()) // Seed the global concurrent-safe random source.
errChan := make(chan error)
templateStopChan := make(chan struct{})
doneChan := make(chan struct{})
spawn("mineLoop-internalLoop", func() {
// We don't want to send router.DefaultMaxMessages blocks at once because there's
// a high chance we'll get disconnected from the node, so we make the channel
// capacity router.DefaultMaxMessages/2 (we give some slack for getBlockTemplate
// requests)
foundBlockChan := make(chan *externalapi.DomainBlock, router.DefaultMaxMessages/2)
spawn("templatesLoop", func() {
templatesLoop(client, miningAddr, errChan)
})
spawn("blocksLoop", func() {
const windowSize = 10
var expectedDurationForWindow time.Duration
var windowExpectedEndTime time.Time
@@ -44,16 +52,8 @@ func mineLoop(client *minerClient, numberOfBlocks uint64, targetBlocksPerSecond
}
blockInWindowIndex := 0
for i := uint64(0); numberOfBlocks == 0 || i < numberOfBlocks; i++ {
foundBlock := make(chan *externalapi.DomainBlock)
mineNextBlock(client, miningAddr, foundBlock, mineWhenNotSynced, templateStopChan, errChan)
block := <-foundBlock
templateStopChan <- struct{}{}
err := handleFoundBlock(client, block)
if err != nil {
errChan <- err
}
for {
foundBlockChan <- mineNextBlock(mineWhenNotSynced)
if hasBlockRateTarget {
blockInWindowIndex++
@@ -70,6 +70,17 @@ func mineLoop(client *minerClient, numberOfBlocks uint64, targetBlocksPerSecond
}
}
})
spawn("handleFoundBlock", func() {
for i := uint64(0); numberOfBlocks == 0 || i < numberOfBlocks; i++ {
block := <-foundBlockChan
err := handleFoundBlock(client, block)
if err != nil {
errChan <- err
return
}
}
doneChan <- struct{}{}
})
@@ -99,21 +110,9 @@ func logHashRate() {
})
}
func mineNextBlock(client *minerClient, miningAddr util.Address, foundBlock chan *externalapi.DomainBlock, mineWhenNotSynced bool,
templateStopChan chan struct{}, errChan chan error) {
newTemplateChan := make(chan *appmessage.GetBlockTemplateResponseMessage)
spawn("templatesLoop", func() {
templatesLoop(client, miningAddr, newTemplateChan, errChan, templateStopChan)
})
spawn("solveLoop", func() {
solveLoop(newTemplateChan, foundBlock, mineWhenNotSynced)
})
}
func handleFoundBlock(client *minerClient, block *externalapi.DomainBlock) error {
blockHash := consensushashing.BlockHash(block)
log.Infof("Found block %s with parents %s. Submitting to %s", blockHash, block.Header.ParentHashes(), client.Address())
log.Infof("Submitting block %s to %s", blockHash, client.Address())
rejectReason, err := client.SubmitBlock(block)
if err != nil {
@@ -132,48 +131,73 @@ func handleFoundBlock(client *minerClient, block *externalapi.DomainBlock) error
return nil
}
func solveBlock(block *externalapi.DomainBlock, stopChan chan struct{}, foundBlock chan *externalapi.DomainBlock) {
targetDifficulty := difficulty.CompactToBig(block.Header.Bits())
headerForMining := block.Header.ToMutable()
initialNonce := rand.Uint64() // Use the global concurrent-safe random source.
for i := initialNonce; i != initialNonce-1; i++ {
select {
case <-stopChan:
return
default:
headerForMining.SetNonce(i)
atomic.AddUint64(&hashesTried, 1)
if pow.CheckProofOfWorkWithTarget(headerForMining, targetDifficulty) {
block.Header = headerForMining.ToImmutable()
foundBlock <- block
return
}
func mineNextBlock(mineWhenNotSynced bool) *externalapi.DomainBlock {
nonce := rand.Uint64() // Use the global concurrent-safe random source.
for {
nonce++
// For each nonce we try to build a block from the most up to date
// block template.
// In the rare case where the nonce space is exhausted for a specific
// block, it'll keep looping the nonce until a new block template
// is discovered.
block := getBlockForMining(mineWhenNotSynced)
targetDifficulty := difficulty.CompactToBig(block.Header.Bits())
headerForMining := block.Header.ToMutable()
headerForMining.SetNonce(nonce)
atomic.AddUint64(&hashesTried, 1)
if pow.CheckProofOfWorkWithTarget(headerForMining, targetDifficulty) {
block.Header = headerForMining.ToImmutable()
log.Infof("Found block %s with parents %s", consensushashing.BlockHash(block), block.Header.ParentHashes())
return block
}
}
}
func templatesLoop(client *minerClient, miningAddr util.Address,
newTemplateChan chan *appmessage.GetBlockTemplateResponseMessage, errChan chan error, stopChan chan struct{}) {
func getBlockForMining(mineWhenNotSynced bool) *externalapi.DomainBlock {
tryCount := 0
for {
tryCount++
const sleepTime = 500 * time.Millisecond
shouldLog := (tryCount-1)%10 == 0
template := templatemanager.Get()
if template == nil {
if shouldLog {
log.Info("Waiting for the initial template")
}
time.Sleep(sleepTime)
continue
}
if !template.IsSynced && !mineWhenNotSynced {
if shouldLog {
log.Warnf("Kaspad is not synced. Skipping current block template")
}
time.Sleep(sleepTime)
continue
}
return appmessage.MsgBlockToDomainBlock(template.MsgBlock)
}
}
func templatesLoop(client *minerClient, miningAddr util.Address, errChan chan error) {
getBlockTemplate := func() {
template, err := client.GetBlockTemplate(miningAddr.String())
if nativeerrors.Is(err, router.ErrTimeout) {
log.Warnf("Got timeout while requesting block template from %s: %s", client.Address(), err)
return
} else if err != nil {
}
if err != nil {
errChan <- errors.Errorf("Error getting block template from %s: %s", client.Address(), err)
return
}
newTemplateChan <- template
templatemanager.Set(template)
}
getBlockTemplate()
const tickerTime = 500 * time.Millisecond
ticker := time.NewTicker(tickerTime)
for {
select {
case <-stopChan:
close(newTemplateChan)
return
case <-client.blockAddedNotificationChan:
getBlockTemplate()
ticker.Reset(tickerTime)
@@ -182,30 +206,3 @@ func templatesLoop(client *minerClient, miningAddr util.Address,
}
}
}
func solveLoop(newTemplateChan chan *appmessage.GetBlockTemplateResponseMessage, foundBlock chan *externalapi.DomainBlock,
mineWhenNotSynced bool) {
var stopOldTemplateSolving chan struct{}
for template := range newTemplateChan {
if !template.IsSynced && !mineWhenNotSynced {
log.Warnf("Kaspad is not synced. Skipping current block template")
continue
}
if stopOldTemplateSolving != nil {
close(stopOldTemplateSolving)
}
stopOldTemplateSolving = make(chan struct{})
block := appmessage.MsgBlockToDomainBlock(template.MsgBlock)
stopOldTemplateSolvingCopy := stopOldTemplateSolving
spawn("solveBlock", func() {
solveBlock(block, stopOldTemplateSolvingCopy, foundBlock)
})
}
if stopOldTemplateSolving != nil {
close(stopOldTemplateSolving)
}
}

View File

@@ -0,0 +1,23 @@
package templatemanager
import (
"github.com/kaspanet/kaspad/app/appmessage"
"sync"
)
var currentTemplate *appmessage.GetBlockTemplateResponseMessage
var lock = &sync.Mutex{}
// Get returns the template to work on
func Get() *appmessage.GetBlockTemplateResponseMessage {
lock.Lock()
defer lock.Unlock()
return currentTemplate
}
// Set sets the current template to work on
func Set(template *appmessage.GetBlockTemplateResponseMessage) {
lock.Lock()
defer lock.Unlock()
currentTemplate = template
}

View File

@@ -27,7 +27,7 @@ WORKDIR /app
RUN apk add --no-cache ca-certificates tini
COPY --from=build /go/src/github.com/kaspanet/kaspad/kaspad /app/
COPY --from=build /go/src/github.com/kaspanet/kaspad/sample-kaspad.conf /app/
COPY --from=build /go/src/github.com/kaspanet/kaspad/infrastructure/config/sample-kaspad.conf /app/
USER nobody
ENTRYPOINT [ "/sbin/tini", "--" ]

View File

@@ -403,3 +403,15 @@ func (s *consensus) GetHeadersSelectedTip() (*externalapi.DomainHash, error) {
return s.headersSelectedTipStore.HeadersSelectedTip(s.databaseContext)
}
func (s *consensus) Anticone(blockHash *externalapi.DomainHash) ([]*externalapi.DomainHash, error) {
s.lock.Lock()
defer s.lock.Unlock()
err := s.validateBlockHashExists(blockHash)
if err != nil {
return nil, err
}
return s.dagTraversalManager.Anticone(blockHash)
}

View File

@@ -0,0 +1,6 @@
package consensus
import "github.com/kaspanet/kaspad/domain/consensus/model"
// GHOSTDAGManagerConstructor is the function signature for a constructor of a type implementing model.GHOSTDAGManager
type GHOSTDAGManagerConstructor func(model.DBReader, model.DAGTopologyManager, model.GHOSTDAGDataStore, model.BlockHeaderStore, model.KType) model.GHOSTDAGManager

View File

@@ -19,11 +19,11 @@ type acceptanceDataStore struct {
}
// New instantiates a new AcceptanceDataStore
func New(cacheSize int) model.AcceptanceDataStore {
func New(cacheSize int, preallocate bool) model.AcceptanceDataStore {
return &acceptanceDataStore{
staging: make(map[externalapi.DomainHash]externalapi.AcceptanceData),
toDelete: make(map[externalapi.DomainHash]struct{}),
cache: lrucache.New(cacheSize),
cache: lrucache.New(cacheSize, preallocate),
}
}

View File

@@ -21,11 +21,11 @@ type blockHeaderStore struct {
}
// New instantiates a new BlockHeaderStore
func New(dbContext model.DBReader, cacheSize int) (model.BlockHeaderStore, error) {
func New(dbContext model.DBReader, cacheSize int, preallocate bool) (model.BlockHeaderStore, error) {
blockHeaderStore := &blockHeaderStore{
staging: make(map[externalapi.DomainHash]externalapi.BlockHeader),
toDelete: make(map[externalapi.DomainHash]struct{}),
cache: lrucache.New(cacheSize),
cache: lrucache.New(cacheSize, preallocate),
}
err := blockHeaderStore.initializeCount(dbContext)

View File

@@ -18,10 +18,10 @@ type blockRelationStore struct {
}
// New instantiates a new BlockRelationStore
func New(cacheSize int) model.BlockRelationStore {
func New(cacheSize int, preallocate bool) model.BlockRelationStore {
return &blockRelationStore{
staging: make(map[externalapi.DomainHash]*model.BlockRelations),
cache: lrucache.New(cacheSize),
cache: lrucache.New(cacheSize, preallocate),
}
}

View File

@@ -18,10 +18,10 @@ type blockStatusStore struct {
}
// New instantiates a new BlockStatusStore
func New(cacheSize int) model.BlockStatusStore {
func New(cacheSize int, preallocate bool) model.BlockStatusStore {
return &blockStatusStore{
staging: make(map[externalapi.DomainHash]externalapi.BlockStatus),
cache: lrucache.New(cacheSize),
cache: lrucache.New(cacheSize, preallocate),
}
}

View File

@@ -21,11 +21,11 @@ type blockStore struct {
}
// New instantiates a new BlockStore
func New(dbContext model.DBReader, cacheSize int) (model.BlockStore, error) {
func New(dbContext model.DBReader, cacheSize int, preallocate bool) (model.BlockStore, error) {
blockStore := &blockStore{
staging: make(map[externalapi.DomainHash]*externalapi.DomainBlock),
toDelete: make(map[externalapi.DomainHash]struct{}),
cache: lrucache.New(cacheSize),
cache: lrucache.New(cacheSize, preallocate),
}
err := blockStore.initializeCount(dbContext)
@@ -212,3 +212,34 @@ func (bs *blockStore) serializeBlockCount(count uint64) ([]byte, error) {
dbBlockCount := &serialization.DbBlockCount{Count: count}
return proto.Marshal(dbBlockCount)
}
type allBlockHashesIterator struct {
cursor model.DBCursor
}
func (a allBlockHashesIterator) First() bool {
return a.cursor.First()
}
func (a allBlockHashesIterator) Next() bool {
return a.cursor.Next()
}
func (a allBlockHashesIterator) Get() (*externalapi.DomainHash, error) {
key, err := a.cursor.Key()
if err != nil {
return nil, err
}
blockHashBytes := key.Suffix()
return externalapi.NewDomainHashFromByteSlice(blockHashBytes)
}
func (bs *blockStore) AllBlockHashesIterator(dbContext model.DBReader) (model.BlockIterator, error) {
cursor, err := dbContext.Cursor(bucket)
if err != nil {
return nil, err
}
return &allBlockHashesIterator{cursor: cursor}, nil
}

View File

@@ -19,9 +19,9 @@ type consensusStateStore struct {
}
// New instantiates a new ConsensusStateStore
func New(utxoSetCacheSize int) model.ConsensusStateStore {
func New(utxoSetCacheSize int, preallocate bool) model.ConsensusStateStore {
return &consensusStateStore{
virtualUTXOSetCache: utxolrucache.New(utxoSetCacheSize),
virtualUTXOSetCache: utxolrucache.New(utxoSetCacheSize, preallocate),
}
}

View File

@@ -16,11 +16,11 @@ type finalityStore struct {
}
// New instantiates a new FinalityStore
func New(cacheSize int) model.FinalityStore {
func New(cacheSize int, preallocate bool) model.FinalityStore {
return &finalityStore{
staging: make(map[externalapi.DomainHash]*externalapi.DomainHash),
toDelete: make(map[externalapi.DomainHash]struct{}),
cache: lrucache.New(cacheSize),
cache: lrucache.New(cacheSize, preallocate),
}
}

View File

@@ -18,10 +18,10 @@ type ghostdagDataStore struct {
}
// New instantiates a new GHOSTDAGDataStore
func New(cacheSize int) model.GHOSTDAGDataStore {
func New(cacheSize int, preallocate bool) model.GHOSTDAGDataStore {
return &ghostdagDataStore{
staging: make(map[externalapi.DomainHash]*model.BlockGHOSTDAGData),
cache: lrucache.New(cacheSize),
cache: lrucache.New(cacheSize, preallocate),
}
}

View File

@@ -26,14 +26,14 @@ type headersSelectedChainStore struct {
}
// New instantiates a new HeadersSelectedChainStore
func New(cacheSize int) model.HeadersSelectedChainStore {
func New(cacheSize int, preallocate bool) model.HeadersSelectedChainStore {
return &headersSelectedChainStore{
stagingAddedByHash: make(map[externalapi.DomainHash]uint64),
stagingRemovedByHash: make(map[externalapi.DomainHash]struct{}),
stagingAddedByIndex: make(map[uint64]*externalapi.DomainHash),
stagingRemovedByIndex: make(map[uint64]struct{}),
cacheByIndex: lrucacheuint64tohash.New(cacheSize),
cacheByHash: lrucache.New(cacheSize),
cacheByIndex: lrucacheuint64tohash.New(cacheSize, preallocate),
cacheByHash: lrucache.New(cacheSize, preallocate),
}
}

View File

@@ -19,11 +19,11 @@ type multisetStore struct {
}
// New instantiates a new MultisetStore
func New(cacheSize int) model.MultisetStore {
func New(cacheSize int, preallocate bool) model.MultisetStore {
return &multisetStore{
staging: make(map[externalapi.DomainHash]model.Multiset),
toDelete: make(map[externalapi.DomainHash]struct{}),
cache: lrucache.New(cacheSize),
cache: lrucache.New(cacheSize, preallocate),
}
}

View File

@@ -21,10 +21,10 @@ type reachabilityDataStore struct {
}
// New instantiates a new ReachabilityDataStore
func New(cacheSize int) model.ReachabilityDataStore {
func New(cacheSize int, preallocate bool) model.ReachabilityDataStore {
return &reachabilityDataStore{
reachabilityDataStaging: make(map[externalapi.DomainHash]model.ReachabilityData),
reachabilityDataCache: lrucache.New(cacheSize),
reachabilityDataCache: lrucache.New(cacheSize, preallocate),
}
}

View File

@@ -23,13 +23,13 @@ type utxoDiffStore struct {
}
// New instantiates a new UTXODiffStore
func New(cacheSize int) model.UTXODiffStore {
func New(cacheSize int, preallocate bool) model.UTXODiffStore {
return &utxoDiffStore{
utxoDiffStaging: make(map[externalapi.DomainHash]model.UTXODiff),
utxoDiffChildStaging: make(map[externalapi.DomainHash]*externalapi.DomainHash),
toDelete: make(map[externalapi.DomainHash]struct{}),
utxoDiffCache: lrucache.New(cacheSize),
utxoDiffChildCache: lrucache.New(cacheSize),
utxoDiffCache: lrucache.New(cacheSize, preallocate),
utxoDiffChildCache: lrucache.New(cacheSize, preallocate),
}
}

View File

@@ -1,11 +1,12 @@
package consensus
import (
"github.com/kaspanet/kaspad/domain/consensus/datastructures/headersselectedchainstore"
"io/ioutil"
"os"
"sync"
"github.com/kaspanet/kaspad/domain/consensus/datastructures/headersselectedchainstore"
"github.com/kaspanet/kaspad/domain/consensus/processes/dagtraversalmanager"
"github.com/kaspanet/kaspad/domain/consensus/processes/finalitymanager"
@@ -45,21 +46,37 @@ import (
"github.com/kaspanet/kaspad/infrastructure/db/database/ldb"
)
const (
defaultTestLeveldbCacheSizeMiB = 8
defaultPreallocateCaches = true
defaultTestPreallocateCaches = false
)
// Factory instantiates new Consensuses
type Factory interface {
NewConsensus(dagParams *dagconfig.Params, db infrastructuredatabase.Database, isArchivalNode bool) (
externalapi.Consensus, error)
NewTestConsensus(dagParams *dagconfig.Params, isArchivalNode bool, testName string) (
tc testapi.TestConsensus, teardown func(keepDataDir bool), err error)
NewTestConsensusWithDataDir(dagParams *dagconfig.Params, dataDir string, isArchivalNode bool) (
tc testapi.TestConsensus, teardown func(keepDataDir bool), err error)
SetTestDataDir(dataDir string)
SetTestGHOSTDAGManager(ghostdagConstructor GHOSTDAGManagerConstructor)
SetTestLevelDBCacheSize(cacheSizeMiB int)
SetTestPreAllocateCache(preallocateCaches bool)
}
type factory struct{}
type factory struct {
dataDir string
ghostdagConstructor GHOSTDAGManagerConstructor
cacheSizeMiB *int
preallocateCaches *bool
}
// NewFactory creates a new Consensus factory
func NewFactory() Factory {
return &factory{}
return &factory{
ghostdagConstructor: ghostdagmanager.New,
}
}
// NewConsensus instantiates a new Consensus
@@ -70,27 +87,39 @@ func (f *factory) NewConsensus(dagParams *dagconfig.Params, db infrastructuredat
pruningWindowSizeForCaches := int(dagParams.PruningDepth())
var preallocateCaches bool
if f.preallocateCaches != nil {
preallocateCaches = *f.preallocateCaches
} else {
preallocateCaches = defaultPreallocateCaches
}
// This is used for caches that are used as part of deletePastBlocks that need to traverse until
// the previous pruning point.
pruningWindowSizePlusFinalityDepthForCache := int(dagParams.PruningDepth() + dagParams.FinalityDepth())
// Data Structures
acceptanceDataStore := acceptancedatastore.New(200)
blockStore, err := blockstore.New(dbManager, 200)
acceptanceDataStore := acceptancedatastore.New(200, preallocateCaches)
blockStore, err := blockstore.New(dbManager, 200, preallocateCaches)
if err != nil {
return nil, err
}
blockHeaderStore, err := blockheaderstore.New(dbManager, 10_000)
blockHeaderStore, err := blockheaderstore.New(dbManager, 10_000, preallocateCaches)
if err != nil {
return nil, err
}
blockRelationStore := blockrelationstore.New(pruningWindowSizeForCaches)
blockStatusStore := blockstatusstore.New(200)
multisetStore := multisetstore.New(200)
blockRelationStore := blockrelationstore.New(pruningWindowSizePlusFinalityDepthForCache, preallocateCaches)
blockStatusStore := blockstatusstore.New(pruningWindowSizePlusFinalityDepthForCache, preallocateCaches)
multisetStore := multisetstore.New(200, preallocateCaches)
pruningStore := pruningstore.New()
reachabilityDataStore := reachabilitydatastore.New(pruningWindowSizeForCaches)
utxoDiffStore := utxodiffstore.New(200)
consensusStateStore := consensusstatestore.New(10_000)
ghostdagDataStore := ghostdagdatastore.New(pruningWindowSizeForCaches)
reachabilityDataStore := reachabilitydatastore.New(pruningWindowSizePlusFinalityDepthForCache, preallocateCaches)
utxoDiffStore := utxodiffstore.New(200, preallocateCaches)
consensusStateStore := consensusstatestore.New(10_000, preallocateCaches)
ghostdagDataStore := ghostdagdatastore.New(pruningWindowSizeForCaches, preallocateCaches)
headersSelectedTipStore := headersselectedtipstore.New()
finalityStore := finalitystore.New(200)
headersSelectedChainStore := headersselectedchainstore.New(pruningWindowSizeForCaches)
finalityStore := finalitystore.New(200, preallocateCaches)
headersSelectedChainStore := headersselectedchainstore.New(pruningWindowSizeForCaches, preallocateCaches)
// Processes
reachabilityManager := reachabilitymanager.New(
@@ -102,7 +131,7 @@ func (f *factory) NewConsensus(dagParams *dagconfig.Params, db infrastructuredat
reachabilityManager,
blockRelationStore,
ghostdagDataStore)
ghostdagManager := ghostdagmanager.New(
ghostdagManager := f.ghostdagConstructor(
dbManager,
dagTopologyManager,
ghostdagDataStore,
@@ -113,7 +142,8 @@ func (f *factory) NewConsensus(dagParams *dagconfig.Params, db infrastructuredat
dagTopologyManager,
ghostdagDataStore,
reachabilityDataStore,
ghostdagManager)
ghostdagManager,
consensusStateStore)
pastMedianTimeManager := pastmediantimemanager.New(
dagParams.TimestampDeviationTolerance,
dbManager,
@@ -270,6 +300,8 @@ func (f *factory) NewConsensus(dagParams *dagconfig.Params, db infrastructuredat
coinbaseManager,
consensusStateManager,
ghostdagManager,
transactionValidator,
acceptanceDataStore,
blockRelationStore,
multisetStore,
@@ -375,19 +407,23 @@ func (f *factory) NewConsensus(dagParams *dagconfig.Params, db infrastructuredat
func (f *factory) NewTestConsensus(dagParams *dagconfig.Params, isArchivalNode bool, testName string) (
tc testapi.TestConsensus, teardown func(keepDataDir bool), err error) {
dataDir, err := ioutil.TempDir("", testName)
if err != nil {
return nil, nil, err
datadir := f.dataDir
if datadir == "" {
datadir, err = ioutil.TempDir("", testName)
if err != nil {
return nil, nil, err
}
}
return f.NewTestConsensusWithDataDir(dagParams, dataDir, isArchivalNode)
}
func (f *factory) NewTestConsensusWithDataDir(dagParams *dagconfig.Params, dataDir string, isArchivalNode bool) (
tc testapi.TestConsensus, teardown func(keepDataDir bool), err error) {
db, err := ldb.NewLevelDB(dataDir)
var cacheSizeMiB int
if f.cacheSizeMiB != nil {
cacheSizeMiB = *f.cacheSizeMiB
} else {
cacheSizeMiB = defaultTestLeveldbCacheSizeMiB
}
if f.preallocateCaches == nil {
f.SetTestPreAllocateCache(defaultTestPreallocateCaches)
}
db, err := ldb.NewLevelDB(datadir, cacheSizeMiB)
if err != nil {
return nil, nil, err
}
@@ -397,9 +433,7 @@ func (f *factory) NewTestConsensusWithDataDir(dagParams *dagconfig.Params, dataD
}
consensusAsImplementation := consensusAsInterface.(*consensus)
testConsensusStateManager := consensusstatemanager.NewTestConsensusStateManager(consensusAsImplementation.consensusStateManager)
testTransactionValidator := transactionvalidator.NewTestTransactionValidator(consensusAsImplementation.transactionValidator)
tstConsensus := &testConsensus{
@@ -415,12 +449,26 @@ func (f *factory) NewTestConsensusWithDataDir(dagParams *dagconfig.Params, dataD
teardown = func(keepDataDir bool) {
db.Close()
if !keepDataDir {
err := os.RemoveAll(dataDir)
err := os.RemoveAll(f.dataDir)
if err != nil {
log.Errorf("Error removing data directory for test consensus: %s", err)
}
}
}
return tstConsensus, teardown, nil
}
func (f *factory) SetTestDataDir(dataDir string) {
f.dataDir = dataDir
}
func (f *factory) SetTestGHOSTDAGManager(ghostdagConstructor GHOSTDAGManagerConstructor) {
f.ghostdagConstructor = ghostdagConstructor
}
func (f *factory) SetTestLevelDBCacheSize(cacheSizeMiB int) {
f.cacheSizeMiB = &cacheSizeMiB
}
func (f *factory) SetTestPreAllocateCache(preallocateCaches bool) {
f.preallocateCaches = &preallocateCaches
}

View File

@@ -18,7 +18,7 @@ func TestNewConsensus(t *testing.T) {
return
}
db, err := ldb.NewLevelDB(tmpDir)
db, err := ldb.NewLevelDB(tmpDir, 8)
if err != nil {
t.Fatalf("error in NewLevelDB: %s", err)
}

View File

@@ -5,6 +5,7 @@ import "github.com/kaspanet/kaspad/domain/consensus/model/externalapi"
// BlockHeap represents a heap of block hashes, providing a priority-queue functionality
type BlockHeap interface {
Push(blockHash *externalapi.DomainHash) error
PushSlice(blockHash []*externalapi.DomainHash) error
Pop() *externalapi.DomainHash
Len() int
ToSlice() []*externalapi.DomainHash

View File

@@ -4,6 +4,7 @@ import "github.com/kaspanet/kaspad/domain/consensus/model/externalapi"
// BlockIterator is an iterator over blocks according to some order.
type BlockIterator interface {
First() bool
Next() bool
Get() *externalapi.DomainHash
Get() (*externalapi.DomainHash, error)
}

View File

@@ -29,4 +29,5 @@ type Consensus interface {
GetVirtualSelectedParentChainFromBlock(blockHash *DomainHash) (*SelectedChainPath, error)
IsInSelectedParentChainOf(blockHashA *DomainHash, blockHashB *DomainHash) (bool, error)
GetHeadersSelectedTip() (*DomainHash, error)
Anticone(blockHash *DomainHash) ([]*DomainHash, error)
}

View File

@@ -85,6 +85,16 @@ func (hash *DomainHash) Equal(other *DomainHash) bool {
return hash.hashArray == other.hashArray
}
// Less returns true if hash is less than other
func (hash *DomainHash) Less(other *DomainHash) bool {
return bytes.Compare(hash.hashArray[:], other.hashArray[:]) < 0
}
// LessOrEqual returns true if hash is smaller or equal to other
func (hash *DomainHash) LessOrEqual(other *DomainHash) bool {
return bytes.Compare(hash.hashArray[:], other.hashArray[:]) <= 0
}
// CloneHashes returns a clone of the given hashes slice.
// Note: since DomainHash is a read-only type, the clone is shallow
func CloneHashes(hashes []*DomainHash) []*DomainHash {
@@ -106,8 +116,3 @@ func HashesEqual(a, b []*DomainHash) bool {
}
return true
}
// Less returns true iff hash a is less than hash b
func Less(a, b *DomainHash) bool {
return bytes.Compare(a.hashArray[:], b.hashArray[:]) < 0
}

View File

@@ -313,6 +313,16 @@ func (id *DomainTransactionID) Equal(other *DomainTransactionID) bool {
return (*DomainHash)(id).Equal((*DomainHash)(other))
}
// Less returns true if id is less than other
func (id *DomainTransactionID) Less(other *DomainTransactionID) bool {
return (*DomainHash)(id).Less((*DomainHash)(other))
}
// LessOrEqual returns true if id is smaller or equal to other
func (id *DomainTransactionID) LessOrEqual(other *DomainTransactionID) bool {
return (*DomainHash)(id).LessOrEqual((*DomainHash)(other))
}
// ByteArray returns the bytes in this transactionID represented as a byte array.
// The transactionID bytes are cloned, therefore it is safe to modify the resulting array.
func (id *DomainTransactionID) ByteArray() *[DomainHashSize]byte {

View File

@@ -12,4 +12,5 @@ type BlockStore interface {
Blocks(dbContext DBReader, blockHashes []*externalapi.DomainHash) ([]*externalapi.DomainBlock, error)
Delete(blockHash *externalapi.DomainHash)
Count() uint64
AllBlockHashesIterator(dbContext DBReader) (BlockIterator, error)
}

View File

@@ -10,7 +10,6 @@ type DAGTopologyManager interface {
IsParentOf(blockHashA *externalapi.DomainHash, blockHashB *externalapi.DomainHash) (bool, error)
IsChildOf(blockHashA *externalapi.DomainHash, blockHashB *externalapi.DomainHash) (bool, error)
IsAncestorOf(blockHashA *externalapi.DomainHash, blockHashB *externalapi.DomainHash) (bool, error)
IsDescendantOf(blockHashA *externalapi.DomainHash, blockHashB *externalapi.DomainHash) (bool, error)
IsAncestorOfAny(blockHash *externalapi.DomainHash, potentialDescendants []*externalapi.DomainHash) (bool, error)
IsInSelectedParentChainOf(blockHashA *externalapi.DomainHash, blockHashB *externalapi.DomainHash) (bool, error)
ChildInSelectedParentChainOf(context, highHash *externalapi.DomainHash) (*externalapi.DomainHash, error)

View File

@@ -7,9 +7,10 @@ import "github.com/kaspanet/kaspad/domain/consensus/model/externalapi"
type DAGTraversalManager interface {
BlockAtDepth(highHash *externalapi.DomainHash, depth uint64) (*externalapi.DomainHash, error)
LowestChainBlockAboveOrEqualToBlueScore(highHash *externalapi.DomainHash, blueScore uint64) (*externalapi.DomainHash, error)
SelectedParentIterator(highHash *externalapi.DomainHash) BlockIterator
// SelectedChildIterator should return a BlockIterator that iterates
// from lowHash (exclusive) to highHash (inclusive) over highHash's selected parent chain
SelectedChildIterator(highHash, lowHash *externalapi.DomainHash) (BlockIterator, error)
AnticoneFromContext(context, lowHash *externalapi.DomainHash) ([]*externalapi.DomainHash, error)
Anticone(blockHash *externalapi.DomainHash) ([]*externalapi.DomainHash, error)
BlueWindow(highHash *externalapi.DomainHash, windowSize int) ([]*externalapi.DomainHash, error)
NewDownHeap() BlockHeap
NewUpHeap() BlockHeap

View File

@@ -9,4 +9,5 @@ type PruningManager interface {
ClearImportedPruningPointData() error
AppendImportedPruningPointUTXOs(outpointAndUTXOEntryPairs []*externalapi.OutpointAndUTXOEntryPair) error
UpdatePruningPointUTXOSetIfRequired() error
PruneAllBlocksBelow(pruningPointHash *externalapi.DomainHash) error
}

View File

@@ -1,7 +1,9 @@
package blockbuilder
import (
"github.com/kaspanet/kaspad/domain/consensus/ruleerrors"
"github.com/kaspanet/kaspad/domain/consensus/utils/blockheader"
"github.com/pkg/errors"
"sort"
"github.com/kaspanet/kaspad/domain/consensus/model"
@@ -9,7 +11,6 @@ import (
"github.com/kaspanet/kaspad/domain/consensus/utils/consensushashing"
"github.com/kaspanet/kaspad/domain/consensus/utils/constants"
"github.com/kaspanet/kaspad/domain/consensus/utils/merkle"
"github.com/kaspanet/kaspad/domain/consensus/utils/transactionid"
"github.com/kaspanet/kaspad/infrastructure/logger"
"github.com/kaspanet/kaspad/util/mstime"
)
@@ -22,6 +23,7 @@ type blockBuilder struct {
coinbaseManager model.CoinbaseManager
consensusStateManager model.ConsensusStateManager
ghostdagManager model.GHOSTDAGManager
transactionValidator model.TransactionValidator
acceptanceDataStore model.AcceptanceDataStore
blockRelationStore model.BlockRelationStore
@@ -38,6 +40,7 @@ func New(
coinbaseManager model.CoinbaseManager,
consensusStateManager model.ConsensusStateManager,
ghostdagManager model.GHOSTDAGManager,
transactionValidator model.TransactionValidator,
acceptanceDataStore model.AcceptanceDataStore,
blockRelationStore model.BlockRelationStore,
@@ -52,10 +55,12 @@ func New(
coinbaseManager: coinbaseManager,
consensusStateManager: consensusStateManager,
ghostdagManager: ghostdagManager,
acceptanceDataStore: acceptanceDataStore,
blockRelationStore: blockRelationStore,
multisetStore: multisetStore,
ghostdagDataStore: ghostdagDataStore,
transactionValidator: transactionValidator,
acceptanceDataStore: acceptanceDataStore,
blockRelationStore: blockRelationStore,
multisetStore: multisetStore,
ghostdagDataStore: ghostdagDataStore,
}
}
@@ -73,6 +78,11 @@ func (bb *blockBuilder) BuildBlock(coinbaseData *externalapi.DomainCoinbaseData,
func (bb *blockBuilder) buildBlock(coinbaseData *externalapi.DomainCoinbaseData,
transactions []*externalapi.DomainTransaction) (*externalapi.DomainBlock, error) {
err := bb.validateTransactions(transactions)
if err != nil {
return nil, err
}
coinbase, err := bb.newBlockCoinbaseTransaction(coinbaseData)
if err != nil {
return nil, err
@@ -90,6 +100,53 @@ func (bb *blockBuilder) buildBlock(coinbaseData *externalapi.DomainCoinbaseData,
}, nil
}
func (bb *blockBuilder) validateTransactions(transactions []*externalapi.DomainTransaction) error {
invalidTransactions := make([]ruleerrors.InvalidTransaction, 0)
for _, transaction := range transactions {
err := bb.validateTransaction(transaction)
if err != nil {
if !errors.As(err, &ruleerrors.RuleError{}) {
return err
}
invalidTransactions = append(invalidTransactions,
ruleerrors.InvalidTransaction{Transaction: transaction, Error: err})
}
}
if len(invalidTransactions) > 0 {
return ruleerrors.NewErrInvalidTransactionsInNewBlock(invalidTransactions)
}
return nil
}
func (bb *blockBuilder) validateTransaction(transaction *externalapi.DomainTransaction) error {
originalEntries := make([]externalapi.UTXOEntry, len(transaction.Inputs))
for i, input := range transaction.Inputs {
originalEntries[i] = input.UTXOEntry
input.UTXOEntry = nil
}
defer func() {
for i, input := range transaction.Inputs {
input.UTXOEntry = originalEntries[i]
}
}()
err := bb.consensusStateManager.PopulateTransactionWithUTXOEntries(transaction)
if err != nil {
return err
}
virtualSelectedParentMedianTime, err := bb.pastMedianTimeManager.PastMedianTime(model.VirtualBlockHash)
if err != nil {
return err
}
return bb.transactionValidator.ValidateTransactionInContextAndPopulateMassAndFee(transaction,
model.VirtualBlockHash, virtualSelectedParentMedianTime)
}
func (bb *blockBuilder) newBlockCoinbaseTransaction(
coinbaseData *externalapi.DomainCoinbaseData) (*externalapi.DomainTransaction, error) {
@@ -197,7 +254,7 @@ func (bb *blockBuilder) calculateAcceptedIDMerkleRoot(acceptanceData externalapi
sort.Slice(acceptedTransactions, func(i, j int) bool {
acceptedTransactionIID := consensushashing.TransactionID(acceptedTransactions[i])
acceptedTransactionJID := consensushashing.TransactionID(acceptedTransactions[j])
return transactionid.Less(acceptedTransactionIID, acceptedTransactionJID)
return acceptedTransactionIID.Less(acceptedTransactionJID)
})
return merkle.CalculateIDMerkleRoot(acceptedTransactions), nil

View File

@@ -1,6 +1,8 @@
package blockbuilder_test
import (
"github.com/kaspanet/kaspad/domain/consensus/utils/constants"
"github.com/kaspanet/kaspad/domain/consensus/utils/subnetworks"
"testing"
"github.com/kaspanet/kaspad/domain/consensus/ruleerrors"
@@ -22,12 +24,14 @@ func TestBuildBlockErrorCases(t *testing.T) {
}
defer teardown(false)
tests := []struct {
name string
coinbaseData *externalapi.DomainCoinbaseData
transactions []*externalapi.DomainTransaction
expectedErrorType error
}{
type testData struct {
name string
coinbaseData *externalapi.DomainCoinbaseData
transactions []*externalapi.DomainTransaction
testFunc func(test testData, err error) error
}
tests := []testData{
{
"scriptPublicKey too long",
&externalapi.DomainCoinbaseData{
@@ -38,7 +42,78 @@ func TestBuildBlockErrorCases(t *testing.T) {
ExtraData: nil,
},
nil,
ruleerrors.ErrBadCoinbasePayloadLen,
func(_ testData, err error) error {
if !errors.Is(err, ruleerrors.ErrBadCoinbasePayloadLen) {
return errors.Errorf("Unexpected error: %+v", err)
}
return nil
},
},
{
"missing UTXO transactions",
&externalapi.DomainCoinbaseData{
ScriptPublicKey: &externalapi.ScriptPublicKey{
Script: nil,
Version: 0,
},
ExtraData: nil,
},
[]*externalapi.DomainTransaction{
{
Version: constants.MaxTransactionVersion,
Inputs: []*externalapi.DomainTransactionInput{
{
PreviousOutpoint: externalapi.DomainOutpoint{
TransactionID: externalapi.DomainTransactionID{}, Index: 0},
},
},
Outputs: nil,
LockTime: 0,
SubnetworkID: subnetworks.SubnetworkIDNative,
Gas: 0,
PayloadHash: externalapi.DomainHash{},
Payload: []byte{0},
},
{
Version: constants.MaxTransactionVersion,
Inputs: []*externalapi.DomainTransactionInput{
{
PreviousOutpoint: externalapi.DomainOutpoint{
TransactionID: externalapi.DomainTransactionID{}, Index: 0},
},
},
Outputs: nil,
LockTime: 0,
SubnetworkID: subnetworks.SubnetworkIDNative,
Gas: 0,
PayloadHash: externalapi.DomainHash{},
Payload: []byte{1},
},
},
func(test testData, err error) error {
errInvalidTransactionsInNewBlock := ruleerrors.ErrInvalidTransactionsInNewBlock{}
if !errors.As(err, &errInvalidTransactionsInNewBlock) {
return errors.Errorf("Unexpected error: %+v", err)
}
if len(errInvalidTransactionsInNewBlock.InvalidTransactions) != len(test.transactions) {
return errors.Errorf("Expected %d transaction but got %d",
len(test.transactions), len(errInvalidTransactionsInNewBlock.InvalidTransactions))
}
for i, invalidTx := range errInvalidTransactionsInNewBlock.InvalidTransactions {
if !invalidTx.Transaction.Equal(test.transactions[i]) {
return errors.Errorf("Expected transaction %d to be equal to its corresponding "+
"test transaction", i)
}
if !errors.As(invalidTx.Error, &ruleerrors.ErrMissingTxOut{}) {
return errors.Errorf("Unexpected error for transaction %d: %+v", i, invalidTx.Error)
}
}
return nil
},
},
}
@@ -48,8 +123,11 @@ func TestBuildBlockErrorCases(t *testing.T) {
t.Errorf("%s: No error from BuildBlock", test.name)
return
}
if test.expectedErrorType != nil && !errors.Is(err, test.expectedErrorType) {
t.Errorf("%s: Expected error '%s', but got '%s'", test.name, test.expectedErrorType, err)
err := test.testFunc(test, err)
if err != nil {
t.Errorf("%s: %s", test.name, err)
return
}
}
})

View File

@@ -5,7 +5,6 @@
package blocklogger
import (
"sync"
"time"
"github.com/kaspanet/kaspad/domain/consensus/model/externalapi"
@@ -13,46 +12,55 @@ import (
)
var (
receivedLogBlocks int64
receivedLogTx int64
lastBlockLogTime = mstime.Now()
mtx sync.Mutex
receivedLogBlocks int64
receivedLogHeaders int64
receivedLogTransactions int64
lastBlockLogTime time.Time
)
// LogBlock logs a new block blue score as an information message
// to show progress to the user. In order to prevent spam, it limits logging to
// one message every 10 seconds with duration and totals included.
func LogBlock(block *externalapi.DomainBlock) {
mtx.Lock()
defer mtx.Unlock()
if len(block.Transactions) == 0 {
receivedLogHeaders++
} else {
receivedLogBlocks++
}
receivedLogBlocks++
receivedLogTx += int64(len(block.Transactions))
receivedLogTransactions += int64(len(block.Transactions))
now := mstime.Now()
now := time.Now()
duration := now.Sub(lastBlockLogTime)
if duration < time.Second*10 {
return
}
// Truncate the duration to 10s of milliseconds.
tDuration := duration.Round(10 * time.Millisecond)
truncatedDuration := duration.Round(10 * time.Millisecond)
// Log information about new block blue score.
blockStr := "blocks"
if receivedLogBlocks == 1 {
blockStr = "block"
}
txStr := "transactions"
if receivedLogTx == 1 {
if receivedLogTransactions == 1 {
txStr = "transaction"
}
log.Infof("Processed %d %s in the last %s (%d %s, %s)",
receivedLogBlocks, blockStr, tDuration, receivedLogTx,
headerStr := "headers"
if receivedLogBlocks == 1 {
headerStr = "header"
}
log.Infof("Processed %d %s and %d %s in the last %s (%d %s, %s)",
receivedLogBlocks, blockStr, receivedLogHeaders, headerStr, truncatedDuration, receivedLogTransactions,
txStr, mstime.UnixMilliseconds(block.Header.TimeInMilliseconds()))
receivedLogBlocks = 0
receivedLogTx = 0
receivedLogHeaders = 0
receivedLogTransactions = 0
lastBlockLogTime = now
}

View File

@@ -8,4 +8,4 @@ import (
"github.com/kaspanet/kaspad/infrastructure/logger"
)
var log, _ = logger.Get(logger.SubsystemTags.PROT)
var log, _ = logger.Get(logger.SubsystemTags.BDAG)

View File

@@ -2,6 +2,7 @@ package blockprocessor
import (
"fmt"
"github.com/kaspanet/kaspad/domain/consensus/processes/blockprocessor/blocklogger"
"github.com/kaspanet/kaspad/util/difficulty"
"github.com/kaspanet/kaspad/domain/consensus/model"
@@ -143,6 +144,8 @@ func (bp *blockProcessor) validateAndInsertBlock(block *externalapi.DomainBlock,
return nil, logClosureErr
}
blocklogger.LogBlock(block)
return &externalapi.BlockInsertionResult{
VirtualSelectedParentChainChanges: selectedParentChainChanges,
}, nil
@@ -290,6 +293,9 @@ func (bp *blockProcessor) discardAllChanges() {
}
func (bp *blockProcessor) commitAllChanges() error {
onEnd := logger.LogAndMeasureExecutionTime(log, "commitAllChanges")
defer onEnd()
dbTx, err := bp.databaseContext.Begin()
if err != nil {
return err

View File

@@ -126,7 +126,8 @@ func TestValidateAndInsertErrors(t *testing.T) {
tipHash, emptyCoinbase, tx1 := initData(params)
// Tests all the error case on the function: "checkBlockStatus"(sub-function in function validateBlock)
blockWithStatusInvalid, err := tc.BuildBlock(&emptyCoinbase, []*externalapi.DomainTransaction{tx1, tx1})
blockWithStatusInvalid, _, err := tc.BuildBlockWithParents([]*externalapi.DomainHash{params.GenesisHash},
&emptyCoinbase, []*externalapi.DomainTransaction{tx1, tx1})
if err != nil {
t.Fatalf("AddBlock: %+v", err)
}

View File

@@ -28,6 +28,12 @@ func (bp *blockProcessor) validateAndInsertImportedPruningPoint(newPruningPoint
return err
}
log.Info("Deleting block data for all blocks in blockStore")
err = bp.pruningManager.PruneAllBlocksBelow(newPruningPointHash)
if err != nil {
return err
}
log.Infof("Updating consensus state manager according to the new pruning point %s", newPruningPointHash)
err = bp.consensusStateManager.ImportPruningPoint(newPruningPoint)
if err != nil {

View File

@@ -4,6 +4,7 @@ import (
"github.com/kaspanet/kaspad/domain/consensus/utils/consensushashing"
"github.com/kaspanet/kaspad/domain/consensus/utils/multiset"
"github.com/kaspanet/kaspad/domain/consensus/utils/utxo"
"github.com/kaspanet/kaspad/infrastructure/logger"
"github.com/pkg/errors"
"github.com/kaspanet/kaspad/domain/consensus/model"
@@ -15,8 +16,10 @@ import (
func (csm *consensusStateManager) CalculatePastUTXOAndAcceptanceData(blockHash *externalapi.DomainHash) (
model.UTXODiff, externalapi.AcceptanceData, model.Multiset, error) {
onEnd := logger.LogAndMeasureExecutionTime(log, "CalculatePastUTXOAndAcceptanceData")
defer onEnd()
log.Debugf("CalculatePastUTXOAndAcceptanceData start for block %s", blockHash)
defer log.Debugf("CalculatePastUTXOAndAcceptanceData end for block %s", blockHash)
if blockHash.Equal(csm.genesisHash) {
log.Debugf("Block %s is the genesis. By definition, "+
@@ -35,6 +38,9 @@ func (csm *consensusStateManager) CalculatePastUTXOAndAcceptanceData(blockHash *
if err != nil {
return nil, nil, nil, err
}
log.Debugf("Restored the past UTXO of block %s with selectedParent %s. "+
"Diff toAdd length: %d, toRemove length: %d", blockHash, blockGHOSTDAGData.SelectedParent(),
selectedParentPastUTXO.ToAdd().Len(), selectedParentPastUTXO.ToRemove().Len())
log.Debugf("Applying blue blocks to the selected parent past UTXO of block %s", blockHash)
acceptanceData, utxoDiff, err := csm.applyMergeSetBlocks(blockHash, selectedParentPastUTXO, blockGHOSTDAGData)
@@ -53,8 +59,10 @@ func (csm *consensusStateManager) CalculatePastUTXOAndAcceptanceData(blockHash *
}
func (csm *consensusStateManager) restorePastUTXO(blockHash *externalapi.DomainHash) (model.MutableUTXODiff, error) {
onEnd := logger.LogAndMeasureExecutionTime(log, "restorePastUTXO")
defer onEnd()
log.Debugf("restorePastUTXO start for block %s", blockHash)
defer log.Debugf("restorePastUTXO end for block %s", blockHash)
var err error
@@ -111,7 +119,7 @@ func (csm *consensusStateManager) applyMergeSetBlocks(blockHash *externalapi.Dom
externalapi.AcceptanceData, model.MutableUTXODiff, error) {
log.Debugf("applyMergeSetBlocks start for block %s", blockHash)
defer log.Tracef("applyMergeSetBlocks end for block %s", blockHash)
defer log.Debugf("applyMergeSetBlocks end for block %s", blockHash)
mergeSetHashes := ghostdagData.MergeSet()
log.Debugf("Merge set for block %s is %v", blockHash, mergeSetHashes)
@@ -263,6 +271,9 @@ func (csm *consensusStateManager) checkTransactionMass(
func (csm *consensusStateManager) RestorePastUTXOSetIterator(blockHash *externalapi.DomainHash) (
model.ReadOnlyUTXOSetIterator, error) {
onEnd := logger.LogAndMeasureExecutionTime(log, "RestorePastUTXOSetIterator")
defer onEnd()
blockStatus, err := csm.resolveBlockStatus(blockHash)
if err != nil {
return nil, err
@@ -276,7 +287,7 @@ func (csm *consensusStateManager) RestorePastUTXOSetIterator(blockHash *external
log.Tracef("RestorePastUTXOSetIterator start for block %s", blockHash)
defer log.Tracef("RestorePastUTXOSetIterator end for block %s", blockHash)
log.Tracef("Calculating UTXO diff for block %s", blockHash)
log.Debugf("Calculating UTXO diff for block %s", blockHash)
blockDiff, err := csm.restorePastUTXO(blockHash)
if err != nil {
return nil, err

View File

@@ -61,7 +61,7 @@ func (csm *consensusStateManager) importPruningPoint(newPruningPoint *externalap
}
log.Debugf("The new pruning point UTXO commitment validation passed")
log.Debugf("Staging the the pruning point as the only DAG tip")
log.Debugf("Staging the pruning point as the only DAG tip")
newTips := []*externalapi.DomainHash{newPruningPointHash}
csm.consensusStateStore.StageTips(newTips)
@@ -86,7 +86,7 @@ func (csm *consensusStateManager) importPruningPoint(newPruningPoint *externalap
return err
}
log.Debugf("Staging the new pruning point")
log.Debugf("Staging the new pruning point %s", newPruningPointHash)
csm.pruningStore.StagePruningPoint(newPruningPointHash)
log.Debugf("Populating the pruning point with UTXO entries")

View File

@@ -1,6 +1,7 @@
package consensusstatemanager
import (
"github.com/kaspanet/kaspad/infrastructure/logger"
"github.com/pkg/errors"
"github.com/kaspanet/kaspad/domain/consensus/model"
@@ -9,8 +10,10 @@ import (
)
func (csm *consensusStateManager) pickVirtualParents(tips []*externalapi.DomainHash) ([]*externalapi.DomainHash, error) {
onEnd := logger.LogAndMeasureExecutionTime(log, "pickVirtualParents")
defer onEnd()
log.Debugf("pickVirtualParents start for tips len: %d", len(tips))
defer log.Debugf("pickVirtualParents end for tips len: %d", len(tips))
log.Debugf("Pushing all tips into a DownHeap")
candidatesHeap := csm.dagTraversalManager.NewDownHeap()
@@ -84,8 +87,8 @@ func (csm *consensusStateManager) pickVirtualParents(tips []*externalapi.DomainH
func (csm *consensusStateManager) selectVirtualSelectedParent(
candidatesHeap model.BlockHeap) (*externalapi.DomainHash, error) {
log.Tracef("selectVirtualSelectedParent start")
defer log.Tracef("selectVirtualSelectedParent end")
onEnd := logger.LogAndMeasureExecutionTime(log, "selectVirtualSelectedParent")
defer onEnd()
disqualifiedCandidates := hashset.New()
@@ -153,8 +156,8 @@ func (csm *consensusStateManager) selectVirtualSelectedParent(
func (csm *consensusStateManager) mergeSetIncrease(
candidate *externalapi.DomainHash, selectedVirtualParents hashset.HashSet) (uint64, error) {
log.Tracef("mergeSetIncrease start")
defer log.Tracef("mergeSetIncrease end")
onEnd := logger.LogAndMeasureExecutionTime(log, "mergeSetIncrease")
defer onEnd()
visited := hashset.New()
queue := csm.dagTraversalManager.NewDownHeap()
@@ -204,8 +207,10 @@ func (csm *consensusStateManager) mergeSetIncrease(
func (csm *consensusStateManager) boundedMergeBreakingParents(
parents []*externalapi.DomainHash) (hashset.HashSet, error) {
onEnd := logger.LogAndMeasureExecutionTime(log, "boundedMergeBreakingParents")
defer onEnd()
log.Tracef("boundedMergeBreakingParents start for parents: %s", parents)
defer log.Tracef("boundedMergeBreakingParents end for parents: %s", parents)
log.Debug("Temporarily setting virtual to all parents, so that we can run ghostdag on it")
err := csm.dagTopologyManager.SetParents(model.VirtualBlockHash, parents)

View File

@@ -56,7 +56,7 @@ func (csm *consensusStateManager) resolveBlockStatus(blockHash *externalapi.Doma
csm.blockStatusStore.Stage(unverifiedBlockHash, blockStatus)
selectedParentStatus = blockStatus
log.Debugf("Block %s status resolved to `%s`", unverifiedBlockHash, blockStatus)
log.Debugf("Block %s status resolved to `%s`, finished %d/%d of unverified blocks", unverifiedBlockHash, blockStatus, len(unverifiedBlocks)-i, len(unverifiedBlocks))
}
return blockStatus, nil
@@ -122,7 +122,7 @@ func (csm *consensusStateManager) getUnverifiedChainBlocks(
func (csm *consensusStateManager) resolveSingleBlockStatus(blockHash *externalapi.DomainHash) (externalapi.BlockStatus, error) {
log.Debugf("resolveSingleBlockStatus start for block %s", blockHash)
defer log.Tracef("resolveSingleBlockStatus end for block %s", blockHash)
defer log.Debugf("resolveSingleBlockStatus end for block %s", blockHash)
log.Tracef("Calculating pastUTXO and acceptance data and multiset for block %s", blockHash)
pastUTXODiff, acceptanceData, multiset, err := csm.CalculatePastUTXOAndAcceptanceData(blockHash)

View File

@@ -3,13 +3,16 @@ package consensusstatemanager
import (
"github.com/kaspanet/kaspad/domain/consensus/model"
"github.com/kaspanet/kaspad/domain/consensus/model/externalapi"
"github.com/kaspanet/kaspad/infrastructure/logger"
)
func (csm *consensusStateManager) updateVirtual(newBlockHash *externalapi.DomainHash,
tips []*externalapi.DomainHash) (*externalapi.SelectedChainPath, error) {
onEnd := logger.LogAndMeasureExecutionTime(log, "updateVirtual")
defer onEnd()
log.Debugf("updateVirtual start for block %s", newBlockHash)
defer log.Debugf("updateVirtual end for block %s", newBlockHash)
log.Debugf("Saving a reference to the GHOSTDAG data of the old virtual")
var oldVirtualSelectedParent *externalapi.DomainHash
@@ -44,6 +47,9 @@ func (csm *consensusStateManager) updateVirtual(newBlockHash *externalapi.Domain
if err != nil {
return nil, err
}
log.Debugf("Calculated the past UTXO of the new virtual. "+
"Diff toAdd length: %d, toRemove length: %d",
virtualUTXODiff.ToAdd().Len(), virtualUTXODiff.ToRemove().Len())
log.Debugf("Staging new acceptance data for the virtual block")
csm.acceptanceDataStore.Stage(model.VirtualBlockHash, virtualAcceptanceData)
@@ -73,6 +79,8 @@ func (csm *consensusStateManager) updateVirtual(newBlockHash *externalapi.Domain
if err != nil {
return nil, err
}
log.Debugf("Selected parent chain changes: %d blocks were removed and %d blocks were added",
len(selectedParentChainChanges.Removed), len(selectedParentChainChanges.Added))
}
return selectedParentChainChanges, nil

View File

@@ -5,8 +5,6 @@ import (
"github.com/kaspanet/kaspad/domain/consensus/utils/transactionhelper"
"github.com/kaspanet/kaspad/domain/consensus/utils/transactionid"
"github.com/kaspanet/kaspad/domain/consensus/utils/consensushashing"
"github.com/kaspanet/kaspad/domain/consensus/utils/merkle"
@@ -142,8 +140,7 @@ func calculateAcceptedIDMerkleRoot(multiblockAcceptanceData externalapi.Acceptan
}
}
sort.Slice(acceptedTransactions, func(i, j int) bool {
return transactionid.Less(
consensushashing.TransactionID(acceptedTransactions[i]),
return consensushashing.TransactionID(acceptedTransactions[i]).Less(
consensushashing.TransactionID(acceptedTransactions[j]))
})

View File

@@ -32,8 +32,8 @@ func TestConsensusStateManager_pickVirtualParents(t *testing.T) {
t.Fatalf("Consensus failed building a block: %v", err)
}
blockParents := block.Header.ParentHashes()
sort.Sort(consensus.NewTestGhostDAGSorter(virtualRelations.Parents, tc, t))
sort.Sort(consensus.NewTestGhostDAGSorter(blockParents, tc, t))
sort.Sort(testutils.NewTestGhostDAGSorter(virtualRelations.Parents, tc, t))
sort.Sort(testutils.NewTestGhostDAGSorter(blockParents, tc, t))
if !externalapi.HashesEqual(virtualRelations.Parents, blockParents) {
t.Fatalf("Block relations and BuildBlock return different parents for virtual, %s != %s", virtualRelations.Parents, blockParents)
}
@@ -54,7 +54,7 @@ func TestConsensusStateManager_pickVirtualParents(t *testing.T) {
}
virtualParents := getSortedVirtualParents(tc)
sort.Sort(consensus.NewTestGhostDAGSorter(parents, tc, t))
sort.Sort(testutils.NewTestGhostDAGSorter(parents, tc, t))
// Make sure the first half of the blocks are with highest blueWork
// we use (max+1)/2 because the first "half" is rounded up, so `(dividend + (divisor - 1)) / divisor` = `(max + (2-1))/2` = `(max+1)/2`
@@ -102,7 +102,7 @@ func TestConsensusStateManager_pickVirtualParents(t *testing.T) {
parents = append(parents, block)
}
sort.Sort(consensus.NewTestGhostDAGSorter(parents, tc, t))
sort.Sort(testutils.NewTestGhostDAGSorter(parents, tc, t))
virtualParents = getSortedVirtualParents(tc)
if !externalapi.HashesEqual(virtualParents, parents) {
t.Fatalf("Expected VirtualParents and parents to be equal, instead: %s != %s", virtualParents, parents)

View File

@@ -71,11 +71,6 @@ func (dtm *dagTopologyManager) IsAncestorOf(blockHashA *externalapi.DomainHash,
return dtm.reachabilityManager.IsDAGAncestorOf(blockHashA, blockHashB)
}
// IsDescendantOf returns true if blockHashA is a DAG descendant of blockHashB
func (dtm *dagTopologyManager) IsDescendantOf(blockHashA *externalapi.DomainHash, blockHashB *externalapi.DomainHash) (bool, error) {
return dtm.reachabilityManager.IsDAGAncestorOf(blockHashB, blockHashA)
}
// IsAncestorOfAny returns true if `blockHash` is an ancestor of at least one of `potentialDescendants`
func (dtm *dagTopologyManager) IsAncestorOfAny(blockHash *externalapi.DomainHash, potentialDescendants []*externalapi.DomainHash) (bool, error) {
for _, potentialDescendant := range potentialDescendants {

View File

@@ -5,10 +5,12 @@ import (
"github.com/kaspanet/kaspad/domain/consensus/utils/hashset"
)
// AnticoneFromContext returns blocks in (context.Past ⋂ block.Anticone)
func (dtm *dagTraversalManager) AnticoneFromContext(context, block *externalapi.DomainHash) ([]*externalapi.DomainHash, error) {
func (dtm *dagTraversalManager) Anticone(blockHash *externalapi.DomainHash) ([]*externalapi.DomainHash, error) {
anticone := []*externalapi.DomainHash{}
queue := []*externalapi.DomainHash{context}
queue, err := dtm.consensusStateStore.Tips(dtm.databaseContext)
if err != nil {
return nil, err
}
visited := hashset.New()
for len(queue) > 0 {
@@ -21,7 +23,7 @@ func (dtm *dagTraversalManager) AnticoneFromContext(context, block *externalapi.
visited.Add(current)
currentIsAncestorOfBlock, err := dtm.dagTopologyManager.IsAncestorOf(current, block)
currentIsAncestorOfBlock, err := dtm.dagTopologyManager.IsAncestorOf(current, blockHash)
if err != nil {
return nil, err
}
@@ -30,7 +32,7 @@ func (dtm *dagTraversalManager) AnticoneFromContext(context, block *externalapi.
continue
}
blockIsAncestorOfCurrent, err := dtm.dagTopologyManager.IsAncestorOf(block, current)
blockIsAncestorOfCurrent, err := dtm.dagTopologyManager.IsAncestorOf(blockHash, current)
if err != nil {
return nil, err
}

View File

@@ -109,6 +109,16 @@ func (bh *blockHeap) Push(blockHash *externalapi.DomainHash) error {
return nil
}
func (bh *blockHeap) PushSlice(blockHashes []*externalapi.DomainHash) error {
for _, blockHash := range blockHashes {
err := bh.Push(blockHash)
if err != nil {
return err
}
}
return nil
}
// Len returns the length of this heap
func (bh *blockHeap) Len() int {
return bh.impl.Len()

View File

@@ -1,7 +1,6 @@
package dagtraversalmanager
import (
"fmt"
"github.com/kaspanet/kaspad/domain/consensus/model"
"github.com/kaspanet/kaspad/domain/consensus/model/externalapi"
"github.com/pkg/errors"
@@ -16,29 +15,7 @@ type dagTraversalManager struct {
ghostdagManager model.GHOSTDAGManager
ghostdagDataStore model.GHOSTDAGDataStore
reachabilityDataStore model.ReachabilityDataStore
}
// selectedParentIterator implements the `model.BlockIterator` API
type selectedParentIterator struct {
databaseContext model.DBReader
ghostdagDataStore model.GHOSTDAGDataStore
current *externalapi.DomainHash
}
func (spi *selectedParentIterator) Next() bool {
if spi.current == nil {
return false
}
ghostdagData, err := spi.ghostdagDataStore.Get(spi.databaseContext, spi.current)
if err != nil {
panic(fmt.Sprintf("ghostdagDataStore is missing ghostdagData for: %v. '%s' ", spi.current, err))
}
spi.current = ghostdagData.SelectedParent()
return spi.current != nil
}
func (spi *selectedParentIterator) Get() *externalapi.DomainHash {
return spi.current
consensusStateStore model.ConsensusStateStore
}
// New instantiates a new DAGTraversalManager
@@ -47,23 +24,15 @@ func New(
dagTopologyManager model.DAGTopologyManager,
ghostdagDataStore model.GHOSTDAGDataStore,
reachabilityDataStore model.ReachabilityDataStore,
ghostdagManager model.GHOSTDAGManager) model.DAGTraversalManager {
ghostdagManager model.GHOSTDAGManager,
conssensusStateStore model.ConsensusStateStore) model.DAGTraversalManager {
return &dagTraversalManager{
databaseContext: databaseContext,
dagTopologyManager: dagTopologyManager,
ghostdagDataStore: ghostdagDataStore,
reachabilityDataStore: reachabilityDataStore,
ghostdagManager: ghostdagManager,
}
}
// SelectedParentIterator creates an iterator over the selected
// parent chain of the given highHash
func (dtm *dagTraversalManager) SelectedParentIterator(highHash *externalapi.DomainHash) model.BlockIterator {
return &selectedParentIterator{
databaseContext: dtm.databaseContext,
ghostdagDataStore: dtm.ghostdagDataStore,
current: highHash,
consensusStateStore: conssensusStateStore,
}
}

View File

@@ -3,11 +3,200 @@ package dagtraversalmanager_test
import (
"github.com/kaspanet/kaspad/domain/consensus"
"github.com/kaspanet/kaspad/domain/consensus/model/externalapi"
"github.com/kaspanet/kaspad/domain/consensus/model/testapi"
"github.com/kaspanet/kaspad/domain/consensus/utils/testutils"
"github.com/kaspanet/kaspad/domain/dagconfig"
"testing"
)
const commonChainSize = 5
const depth uint64 = 2
//TestBlockAtDepthOnChainDag compares the result of BlockAtDepth to the result of looping over the SelectedChain on a single chain DAG.
func TestBlockAtDepthOnChainDag(t *testing.T) {
testutils.ForAllNets(t, true, func(t *testing.T, params *dagconfig.Params) {
factory := consensus.NewFactory()
tc, tearDown, err := factory.NewTestConsensus(params, false,
"TestBlockAtDepthOnChainDag")
if err != nil {
t.Fatalf("Failed creating a NewTestConsensus: %s", err)
}
defer tearDown(false)
highHash, err := createAChainDAG(params.GenesisHash, tc)
if err != nil {
t.Fatalf("Failed creating a Chain DAG In BlockAtDepthTEST: %+v", err)
}
currentBlockHash := highHash
currentBlockData, err := tc.GHOSTDAGDataStore().Get(tc.DatabaseContext(), currentBlockHash)
if err != nil {
t.Fatalf("Failed getting GHOSTDAGData for block with hash %s: %+v", currentBlockHash.String(), err)
}
for i := uint64(0); i <= depth; i++ {
if currentBlockData.SelectedParent() == nil {
break
}
currentBlockHash = currentBlockData.SelectedParent()
currentBlockData, err = tc.GHOSTDAGDataStore().Get(tc.DatabaseContext(), currentBlockHash)
if err != nil {
t.Fatalf("Failed getting GHOSTDAGData for block with hash %s: %+v", currentBlockHash.String(), err)
}
}
expectedBlockHash := currentBlockHash
actualBlockHash, err := tc.DAGTraversalManager().BlockAtDepth(highHash, depth)
if err != nil {
t.Fatalf("Failed on BlockAtDepth: %+v", err)
}
if !actualBlockHash.Equal(expectedBlockHash) {
t.Fatalf("Expected block %s but got %s", expectedBlockHash, actualBlockHash)
}
})
}
func createAChainDAG(genesisHash *externalapi.DomainHash, tc testapi.TestConsensus) (*externalapi.DomainHash, error) {
block := genesisHash
var err error
for i := 0; i < commonChainSize; i++ {
block, _, err = tc.AddBlock([]*externalapi.DomainHash{block}, nil, nil)
if err != nil {
return nil, err
}
}
return block, nil
}
// TestBlockAtDepthOnDAGWhereTwoBlocksHaveSameSelectedParent compares the results of BlockAtDepth
// of 2 children that have the same selectedParent.
func TestBlockAtDepthOnDAGWhereTwoBlocksHaveSameSelectedParent(t *testing.T) {
testutils.ForAllNets(t, true, func(t *testing.T, params *dagconfig.Params) {
factory := consensus.NewFactory()
tc, tearDown, err := factory.NewTestConsensus(params, false,
"TestBlockAtDepthOnDAGWhereTwoBlocksHaveSameSelectedParent")
if err != nil {
t.Fatalf("Failed creating a NewTestConsensus: %s", err)
}
defer tearDown(false)
firstChild, secondChild, err := createADAGTwoChildrenWithSameSelectedParent(params.GenesisHash, tc)
if err != nil {
t.Fatalf("Failed creating a DAG where two blocks have same selected parent: %+v", err)
}
actualBlockHash, err := tc.DAGTraversalManager().BlockAtDepth(firstChild, depth)
if err != nil {
t.Fatalf("Failed at BlockAtDepth: %+v", err)
}
expectedSameHash, err := tc.DAGTraversalManager().BlockAtDepth(secondChild, depth)
if err != nil {
t.Fatalf("Failed in BlockAtDepth: %+v", err)
}
if !actualBlockHash.Equal(expectedSameHash) {
t.Fatalf("Expected block %s but got %s", expectedSameHash, actualBlockHash)
}
})
}
func createADAGTwoChildrenWithSameSelectedParent(genesisHash *externalapi.DomainHash,
tc testapi.TestConsensus) (*externalapi.DomainHash, *externalapi.DomainHash, error) {
block := genesisHash
var err error
for i := 0; i < commonChainSize; i++ {
block, _, err = tc.AddBlock([]*externalapi.DomainHash{block}, nil, nil)
if err != nil {
return nil, nil, err
}
}
firstChild, _, err := tc.AddBlock([]*externalapi.DomainHash{block}, nil, nil)
if err != nil {
return nil, nil, err
}
secondChild, _, err := tc.AddBlock([]*externalapi.DomainHash{block}, nil, nil)
if err != nil {
return nil, nil, err
}
return firstChild, secondChild, nil
}
// TestBlockAtDepthOnDAGWithTwoDifferentChains compares results of BlockAtDepth on two different chains,
// on the same DAG, and validates they merge at the correct point.
func TestBlockAtDepthOnDAGWithTwoDifferentChains(t *testing.T) {
testutils.ForAllNets(t, true, func(t *testing.T, params *dagconfig.Params) {
factory := consensus.NewFactory()
tc, tearDown, err := factory.NewTestConsensus(params, false,
"TestBlockAtDepthOnDAGWithTwoDifferentChains")
if err != nil {
t.Fatalf("Failed creating a NewTestConsensus: %s", err)
}
defer tearDown(false)
const sizeOfTheFirstChildSubChainDAG = 3
const sizeOfTheSecondChildSubChainDAG = 2
firstChild, secondChild, err := createADAGWithTwoDifferentChains(params.GenesisHash, tc, sizeOfTheFirstChildSubChainDAG,
sizeOfTheSecondChildSubChainDAG)
if err != nil {
t.Fatalf("Failed creating a DAG with two different chains in BlockAtDepthTEST: %+v", err)
}
actualBlockHash, err := tc.DAGTraversalManager().BlockAtDepth(firstChild, sizeOfTheFirstChildSubChainDAG)
if err != nil {
t.Fatalf("Failed in BlockAtDepth: %+v", err)
}
expectedSameHash, err := tc.DAGTraversalManager().BlockAtDepth(secondChild, sizeOfTheSecondChildSubChainDAG)
if err != nil {
t.Fatalf("Failed in BlockAtDepth: %+v", err)
}
if !actualBlockHash.Equal(expectedSameHash) {
t.Fatalf("Expected block %s but got %s", expectedSameHash, actualBlockHash)
}
expectedDiffHash, err := tc.DAGTraversalManager().BlockAtDepth(secondChild, sizeOfTheSecondChildSubChainDAG-1)
if err != nil {
t.Fatalf("Failed in BlockAtDepth: %+v", err)
}
if actualBlockHash.Equal(expectedDiffHash) {
t.Fatalf("Expected to a differente block")
}
})
}
func createADAGWithTwoDifferentChains(genesisHash *externalapi.DomainHash, tc testapi.TestConsensus,
sizeOfTheFirstChildSubChainDAG int, sizeOfTheSecondChildSubChainDAG int) (*externalapi.DomainHash, *externalapi.DomainHash, error) {
block := genesisHash
var err error
for i := 0; i < commonChainSize; i++ {
block, _, err = tc.AddBlock([]*externalapi.DomainHash{block}, nil, nil)
if err != nil {
return nil, nil, err
}
}
firstChainTipHash, _, err := tc.AddBlock([]*externalapi.DomainHash{block}, nil, nil)
if err != nil {
return nil, nil, err
}
secondChainTipHash, _, err := tc.AddBlock([]*externalapi.DomainHash{block}, nil, nil)
if err != nil {
return nil, nil, err
}
for i := 0; i < sizeOfTheFirstChildSubChainDAG; i++ {
firstChainTipHash, _, err = tc.AddBlock([]*externalapi.DomainHash{firstChainTipHash}, nil, nil)
if err != nil {
return nil, nil, err
}
}
for i := 0; i < sizeOfTheSecondChildSubChainDAG; i++ {
secondChainTipHash, _, err = tc.AddBlock([]*externalapi.DomainHash{secondChainTipHash}, nil, nil)
if err != nil {
return nil, nil, err
}
}
return firstChainTipHash, secondChainTipHash, nil
}
func TestLowestChainBlockAboveOrEqualToBlueScore(t *testing.T) {
testutils.ForAllNets(t, true, func(t *testing.T, params *dagconfig.Params) {
params.FinalityDuration = 10 * params.TargetTimePerBlock
@@ -18,7 +207,6 @@ func TestLowestChainBlockAboveOrEqualToBlueScore(t *testing.T) {
t.Fatalf("NewTestConsensus: %s", err)
}
defer tearDown(false)
checkExpectedBlock := func(highHash *externalapi.DomainHash, blueScore uint64, expected *externalapi.DomainHash) {
blockHash, err := tc.DAGTraversalManager().LowestChainBlockAboveOrEqualToBlueScore(highHash, blueScore)
if err != nil {

View File

@@ -11,20 +11,34 @@ type selectedChildIterator struct {
dagTopologyManager model.DAGTopologyManager
reachabilityDataStore model.ReachabilityDataStore
highHash *externalapi.DomainHash
highHash, lowHash *externalapi.DomainHash
current *externalapi.DomainHash
err error
}
func (s *selectedChildIterator) First() bool {
s.current = s.lowHash
return s.Next()
}
func (s *selectedChildIterator) Next() bool {
if s.err != nil {
return true
}
data, err := s.reachabilityDataStore.ReachabilityData(s.databaseContext, s.current)
if err != nil {
panic(err)
s.current = nil
s.err = err
return true
}
for _, child := range data.Children() {
isChildInSelectedParentChainOfHighHash, err := s.dagTopologyManager.IsInSelectedParentChainOf(child, s.highHash)
if err != nil {
panic(err)
s.current = nil
s.err = err
return true
}
if isChildInSelectedParentChainOfHighHash {
@@ -35,10 +49,12 @@ func (s *selectedChildIterator) Next() bool {
return false
}
func (s *selectedChildIterator) Get() *externalapi.DomainHash {
return s.current
func (s *selectedChildIterator) Get() (*externalapi.DomainHash, error) {
return s.current, s.err
}
// SelectedChildIterator returns a BlockIterator that iterates from lowHash (exclusive) to highHash (inclusive) over
// highHash's selected parent chain
func (dtm *dagTraversalManager) SelectedChildIterator(highHash, lowHash *externalapi.DomainHash) (model.BlockIterator, error) {
isLowHashInSelectedParentChainOfHighHash, err := dtm.dagTopologyManager.IsInSelectedParentChainOf(lowHash, highHash)
if err != nil {
@@ -53,6 +69,7 @@ func (dtm *dagTraversalManager) SelectedChildIterator(highHash, lowHash *externa
dagTopologyManager: dtm.dagTopologyManager,
reachabilityDataStore: dtm.reachabilityDataStore,
highHash: highHash,
lowHash: lowHash,
current: lowHash,
}, nil
}

View File

@@ -19,7 +19,7 @@ func TestBlueBlockWindow(t *testing.T) {
id string //id is a virtual entity that is used only for tests so we can define relations between blocks without knowing their hash
expectedWindowWithGenesisPadding []string
}{
"kaspa-mainnet": {
dagconfig.MainnetParams.Name: {
{
parents: []string{"A"},
id: "B",
@@ -91,7 +91,7 @@ func TestBlueBlockWindow(t *testing.T) {
expectedWindowWithGenesisPadding: []string{"N", "M", "L", "K", "J", "I", "F", "C", "D", "H"},
},
},
"kaspa-testnet": {
dagconfig.TestnetParams.Name: {
{
parents: []string{"A"},
id: "B",
@@ -163,7 +163,7 @@ func TestBlueBlockWindow(t *testing.T) {
expectedWindowWithGenesisPadding: []string{"N", "M", "L", "K", "J", "I", "F", "H", "C", "D"},
},
},
"kaspa-devnet": {
dagconfig.DevnetParams.Name: {
{
parents: []string{"A"},
id: "B",
@@ -235,7 +235,7 @@ func TestBlueBlockWindow(t *testing.T) {
expectedWindowWithGenesisPadding: []string{"N", "M", "L", "K", "J", "I", "F", "H", "C", "D"},
},
},
"kaspa-simnet": {
dagconfig.SimnetParams.Name: {
{
parents: []string{"A"},
id: "B",
@@ -344,7 +344,7 @@ func TestBlueBlockWindow(t *testing.T) {
if err != nil {
t.Fatalf("BlueWindow: %s", err)
}
sort.Sort(consensus.NewTestGhostDAGSorter(window, tc, t))
sort.Sort(testutils.NewTestGhostDAGSorter(window, tc, t))
if err := checkWindowIDs(window, blockData.expectedWindowWithGenesisPadding, idByBlockMap); err != nil {
t.Errorf("Unexpected values for window for block %s: %s", blockData.id, err)
}

View File

@@ -1,10 +1,11 @@
package difficultymanager_test
import (
"github.com/kaspanet/kaspad/util/difficulty"
"testing"
"time"
"github.com/kaspanet/kaspad/util/difficulty"
"github.com/kaspanet/kaspad/util/mstime"
"github.com/kaspanet/kaspad/domain/consensus/utils/consensushashing"
@@ -132,9 +133,9 @@ func TestDifficulty(t *testing.T) {
var expectedBits uint32
switch params.Name {
case "kaspa-testnet", "kaspa-devnet":
case dagconfig.TestnetParams.Name, dagconfig.DevnetParams.Name:
expectedBits = uint32(0x1e7f83df)
case "kaspa-mainnet":
case dagconfig.MainnetParams.Name:
expectedBits = uint32(0x207f83df)
}

View File

@@ -60,7 +60,7 @@ func (gm *ghostdagManager) Less(blockHashA *externalapi.DomainHash, ghostdagData
case 1:
return false
case 0:
return externalapi.Less(blockHashA, blockHashB)
return blockHashA.Less(blockHashB)
default:
panic("big.Int.Cmp is defined to always return -1/1/0 and nothing else")
}

View File

@@ -2,15 +2,17 @@ package ghostdagmanager_test
import (
"encoding/json"
"github.com/kaspanet/kaspad/domain/consensus/utils/blockheader"
"github.com/kaspanet/kaspad/domain/consensus/utils/constants"
"github.com/kaspanet/kaspad/util/difficulty"
"math"
"math/big"
"os"
"path/filepath"
"reflect"
"testing"
"github.com/kaspanet/kaspad/domain/consensus/utils/blockheader"
"github.com/kaspanet/kaspad/domain/consensus/utils/constants"
"github.com/kaspanet/kaspad/util/difficulty"
"github.com/kaspanet/kaspad/domain/consensus/model"
"github.com/kaspanet/kaspad/domain/consensus/model/externalapi"
"github.com/kaspanet/kaspad/domain/consensus/processes/ghostdag2"
@@ -177,6 +179,101 @@ func TestGHOSTDAG(t *testing.T) {
})
}
// TestBlueWork tests if GHOSTDAG picks as selected parent the parent
// with the most blue work, even if its blue score is not the greatest.
// To do that it creates one chain of 3 blocks over genesis, and another
// chain of 2 blocks with more blue work than the 3 blocks chain, and
// checks that a block that points to both chain tips will have the
// 2 blocks chain tip as its selected parent.
func TestBlueWork(t *testing.T) {
dagTopology := &DAGTopologyManagerImpl{
parentsMap: make(map[externalapi.DomainHash][]*externalapi.DomainHash),
}
ghostdagDataStore := &GHOSTDAGDataStoreImpl{
dagMap: make(map[externalapi.DomainHash]*model.BlockGHOSTDAGData),
}
blockHeadersStore := &blockHeadersStore{
dagMap: make(map[externalapi.DomainHash]externalapi.BlockHeader),
}
fakeGenesisHash := externalapi.NewDomainHashFromByteArray(&[externalapi.DomainHashSize]byte{0})
longestChainBlock1Hash := externalapi.NewDomainHashFromByteArray(&[externalapi.DomainHashSize]byte{1})
longestChainBlock2Hash := externalapi.NewDomainHashFromByteArray(&[externalapi.DomainHashSize]byte{2})
longestChainBlock3Hash := externalapi.NewDomainHashFromByteArray(&[externalapi.DomainHashSize]byte{3})
heaviestChainBlock1Hash := externalapi.NewDomainHashFromByteArray(&[externalapi.DomainHashSize]byte{4})
heaviestChainBlock2Hash := externalapi.NewDomainHashFromByteArray(&[externalapi.DomainHashSize]byte{5})
tipHash := externalapi.NewDomainHashFromByteArray(&[externalapi.DomainHashSize]byte{6})
lowDifficultyHeader := blockheader.NewImmutableBlockHeader(
0,
nil,
&externalapi.DomainHash{},
&externalapi.DomainHash{},
&externalapi.DomainHash{},
0,
0,
0,
)
dagTopology.parentsMap[*fakeGenesisHash] = nil
ghostdagDataStore.dagMap[*fakeGenesisHash] = model.NewBlockGHOSTDAGData(0, new(big.Int), nil, nil, nil, nil)
blockHeadersStore.dagMap[*fakeGenesisHash] = lowDifficultyHeader
dagTopology.parentsMap[*longestChainBlock1Hash] = []*externalapi.DomainHash{fakeGenesisHash}
blockHeadersStore.dagMap[*longestChainBlock1Hash] = lowDifficultyHeader
dagTopology.parentsMap[*longestChainBlock2Hash] = []*externalapi.DomainHash{longestChainBlock1Hash}
blockHeadersStore.dagMap[*longestChainBlock2Hash] = lowDifficultyHeader
dagTopology.parentsMap[*longestChainBlock3Hash] = []*externalapi.DomainHash{longestChainBlock2Hash}
blockHeadersStore.dagMap[*longestChainBlock3Hash] = lowDifficultyHeader
dagTopology.parentsMap[*heaviestChainBlock1Hash] = []*externalapi.DomainHash{fakeGenesisHash}
blockHeadersStore.dagMap[*heaviestChainBlock1Hash] = blockheader.NewImmutableBlockHeader(
0,
nil,
&externalapi.DomainHash{},
&externalapi.DomainHash{},
&externalapi.DomainHash{},
0,
math.MaxUint32, // Put a very high difficulty so the chain that contains this block will have a very high blue work
0,
)
dagTopology.parentsMap[*heaviestChainBlock2Hash] = []*externalapi.DomainHash{heaviestChainBlock1Hash}
blockHeadersStore.dagMap[*heaviestChainBlock2Hash] = lowDifficultyHeader
dagTopology.parentsMap[*tipHash] = []*externalapi.DomainHash{heaviestChainBlock2Hash, longestChainBlock3Hash}
blockHeadersStore.dagMap[*tipHash] = lowDifficultyHeader
manager := ghostdagmanager.New(nil, dagTopology, ghostdagDataStore, blockHeadersStore, 18)
blocksForGHOSTDAG := []*externalapi.DomainHash{
longestChainBlock1Hash,
longestChainBlock2Hash,
longestChainBlock3Hash,
heaviestChainBlock1Hash,
heaviestChainBlock2Hash,
tipHash,
}
for _, blockHash := range blocksForGHOSTDAG {
err := manager.GHOSTDAG(blockHash)
if err != nil {
t.Fatalf("GHOSTDAG: %+v", err)
}
}
if ghostdagDataStore.dagMap[*longestChainBlock3Hash].BlueScore() <= ghostdagDataStore.dagMap[*heaviestChainBlock2Hash].BlueScore() {
t.Fatalf("Expected longestChainBlock3Hash to have greater blue score than heaviestChainBlock2Hash")
}
if !ghostdagDataStore.dagMap[*tipHash].SelectedParent().Equal(heaviestChainBlock2Hash) {
t.Fatalf("Expected the block with the most blue work to be the selected parent of the tip")
}
}
func hashesToStrings(arr []*externalapi.DomainHash) []string {
var strArr = make([]string, len(arr))
for i, hash := range arr {
@@ -290,10 +387,6 @@ func (dt *DAGTopologyManagerImpl) IsAncestorOf(hashBlockA *externalapi.DomainHas
}
func (dt *DAGTopologyManagerImpl) IsDescendantOf(blockHashA *externalapi.DomainHash, blockHashB *externalapi.DomainHash) (bool, error) {
panic("unimplemented")
}
func (dt *DAGTopologyManagerImpl) IsAncestorOfAny(blockHash *externalapi.DomainHash, potentialDescendants []*externalapi.DomainHash) (bool, error) {
panic("unimplemented")
}

View File

@@ -27,16 +27,16 @@ type testJSON struct {
func TestPruning(t *testing.T) {
expectedPruningPointByNet := map[string]map[string]string{
"chain-for-test-pruning.json": {
"kaspa-mainnet": "1582",
"kaspa-simnet": "1582",
"kaspa-devnet": "1582",
"kaspa-testnet": "1582",
dagconfig.MainnetParams.Name: "1582",
dagconfig.TestnetParams.Name: "1582",
dagconfig.DevnetParams.Name: "1582",
dagconfig.SimnetParams.Name: "1582",
},
"dag-for-test-pruning.json": {
"kaspa-mainnet": "503",
"kaspa-simnet": "502",
"kaspa-devnet": "503",
"kaspa-testnet": "503",
dagconfig.MainnetParams.Name: "503",
dagconfig.TestnetParams.Name: "503",
dagconfig.DevnetParams.Name: "503",
dagconfig.SimnetParams.Name: "502",
},
}

View File

@@ -151,8 +151,11 @@ func (pm *pruningManager) UpdatePruningPointByVirtual() error {
newPruningPoint := currentPruningPoint
newPruningPointGHOSTDAGData := currentPruningPointGHOSTDAGData
for iterator.Next() {
selectedChild := iterator.Get()
for ok := iterator.First(); ok; ok = iterator.Next() {
selectedChild, err := iterator.Get()
if err != nil {
return err
}
selectedChildGHOSTDAGData, err := pm.ghostdagDataStore.Get(pm.databaseContext, selectedChild)
if err != nil {
return err
@@ -221,47 +224,47 @@ func (pm *pruningManager) deletePastBlocks(pruningPoint *externalapi.DomainHash)
onEnd := logger.LogAndMeasureExecutionTime(log, "pruningManager.deletePastBlocks")
defer onEnd()
// Go over all P.Past and P.AC that's not in V.Past
// Go over all pruningPoint.Past and pruningPoint.Anticone that's not in virtual.Past
queue := pm.dagTraversalManager.NewDownHeap()
// Find P.AC that's not in V.Past
dagTips, err := pm.consensusStateStore.Tips(pm.databaseContext)
if err != nil {
return err
}
newTips := make([]*externalapi.DomainHash, 0, len(dagTips))
virtualParents, err := pm.dagTopologyManager.Parents(model.VirtualBlockHash)
if err != nil {
return err
}
for _, tip := range dagTips {
isInPruningFutureOrInVirtualPast, err := pm.isInPruningFutureOrInVirtualPast(tip, pruningPoint, virtualParents)
if err != nil {
return err
}
if !isInPruningFutureOrInVirtualPast {
// Add them to the queue so they and their past will be pruned
err := queue.Push(tip)
if err != nil {
return err
}
} else {
newTips = append(newTips, tip)
}
// Start queue with all tips that are below the pruning point (and on the way remove them from list of tips)
prunedTips, err := pm.pruneTips(pruningPoint, virtualParents)
if err != nil {
return err
}
pm.consensusStateStore.StageTips(newTips)
// Add P.Parents
err = queue.PushSlice(prunedTips)
if err != nil {
return err
}
// Add pruningPoint.Parents to queue
parents, err := pm.dagTopologyManager.Parents(pruningPoint)
if err != nil {
return err
}
for _, parent := range parents {
err = queue.Push(parent)
if err != nil {
return err
}
err = queue.PushSlice(parents)
if err != nil {
return err
}
err = pm.deleteBlocksDownward(queue)
if err != nil {
return err
}
err = pm.pruneVirtualDiffParents(pruningPoint, virtualParents)
if err != nil {
return err
}
return nil
}
func (pm *pruningManager) deleteBlocksDownward(queue model.BlockHeap) error {
visited := map[externalapi.DomainHash]struct{}{}
// Prune everything in the queue including its past
for queue.Len() > 0 {
@@ -280,16 +283,16 @@ func (pm *pruningManager) deletePastBlocks(pruningPoint *externalapi.DomainHash)
if err != nil {
return err
}
for _, parent := range parents {
err = queue.Push(parent)
if err != nil {
return err
}
err = queue.PushSlice(parents)
if err != nil {
return err
}
}
}
return nil
}
// Delete virtual diff parents that are in PruningPoint's Anticone and not in Virtual's Past
func (pm *pruningManager) pruneVirtualDiffParents(pruningPoint *externalapi.DomainHash, virtualParents []*externalapi.DomainHash) error {
virtualDiffParents, err := pm.consensusStateStore.VirtualDiffParents(pm.databaseContext)
if err != nil {
return err
@@ -309,6 +312,31 @@ func (pm *pruningManager) deletePastBlocks(pruningPoint *externalapi.DomainHash)
return nil
}
func (pm *pruningManager) pruneTips(pruningPoint *externalapi.DomainHash, virtualParents []*externalapi.DomainHash) (
prunedTips []*externalapi.DomainHash, err error) {
// Find P.AC that's not in V.Past
dagTips, err := pm.consensusStateStore.Tips(pm.databaseContext)
if err != nil {
return nil, err
}
newTips := make([]*externalapi.DomainHash, 0, len(dagTips))
for _, tip := range dagTips {
isInPruningFutureOrInVirtualPast, err := pm.isInPruningFutureOrInVirtualPast(tip, pruningPoint, virtualParents)
if err != nil {
return nil, err
}
if !isInPruningFutureOrInVirtualPast {
prunedTips = append(prunedTips, tip)
} else {
newTips = append(newTips, tip)
}
}
pm.consensusStateStore.StageTips(newTips)
return prunedTips, nil
}
func (pm *pruningManager) savePruningPoint(pruningPointHash *externalapi.DomainHash) error {
onEnd := logger.LogAndMeasureExecutionTime(log, "pruningManager.savePruningPoint")
defer onEnd()
@@ -551,3 +579,33 @@ func (pm *pruningManager) updatePruningPointUTXOSet() error {
log.Debugf("Finishing updating the pruning point UTXO set")
return pm.pruningStore.FinishUpdatingPruningPointUTXOSet(pm.databaseContext)
}
func (pm *pruningManager) PruneAllBlocksBelow(pruningPointHash *externalapi.DomainHash) error {
onEnd := logger.LogAndMeasureExecutionTime(log, "PruneAllBlocksBelow")
defer onEnd()
iterator, err := pm.blocksStore.AllBlockHashesIterator(pm.databaseContext)
if err != nil {
return err
}
for ok := iterator.First(); ok; ok = iterator.Next() {
blockHash, err := iterator.Get()
if err != nil {
return err
}
isInPastOfPruningPoint, err := pm.dagTopologyManager.IsAncestorOf(pruningPointHash, blockHash)
if err != nil {
return err
}
if !isInPastOfPruningPoint {
continue
}
_, err = pm.deleteBlock(blockHash)
if err != nil {
return err
}
}
return nil
}

View File

@@ -337,30 +337,36 @@ func (rt *reachabilityManager) updateReindexRoot(selectedTip *externalapi.Domain
rc := newReindexContext(rt)
// Iterate from reindexRootAncestor towards newReindexRoot
for {
chosenChild, err := rt.FindNextAncestor(selectedTip, reindexRootAncestor)
if err != nil {
return err
if !newReindexRoot.Equal(reindexRootAncestor) {
log.Debugf("Concentrating the intervals towards the new reindex root")
// Iterate from reindexRootAncestor towards newReindexRoot
for {
chosenChild, err := rt.FindNextAncestor(selectedTip, reindexRootAncestor)
if err != nil {
return err
}
isFinalReindexRoot := chosenChild.Equal(newReindexRoot)
// Concentrate interval from current ancestor to its chosen child
err = rc.concentrateInterval(reindexRootAncestor, chosenChild, isFinalReindexRoot)
if err != nil {
return err
}
if isFinalReindexRoot {
break
}
reindexRootAncestor = chosenChild
}
isFinalReindexRoot := chosenChild.Equal(newReindexRoot)
// Concentrate interval from current ancestor to it's chosen child
err = rc.concentrateInterval(reindexRootAncestor, chosenChild, isFinalReindexRoot)
if err != nil {
return err
}
if isFinalReindexRoot {
break
}
reindexRootAncestor = chosenChild
} else {
log.Debugf("newReindexRoot is the same as reindexRootAncestor. Skipping concentration...")
}
// Update reindex root data store
rt.stageReindexRoot(newReindexRoot)
log.Debugf("Updated the reindex root to %s", newReindexRoot)
return nil
}
@@ -393,8 +399,12 @@ func (rt *reachabilityManager) findNextReindexRoot(currentReindexRoot, selectedT
// We have reindex root out of selected tip chain, however we switch chains only after a sufficient
// threshold of reindexSlack score in order to address possible alternating reorg attacks.
// The reindexSlack constant is used as an heuristic for a large enough constant on the one hand, but
// one which will not harm performance on the other hand - given the available slack at the chain split point
if selectedTipGHOSTDAGData.BlueScore()-currentRootGHOSTDAGData.BlueScore() < rt.reindexSlack {
// one which will not harm performance on the other hand - given the available slack at the chain split point.
//
// Note: In some cases the blue score selected tip can be lower than the current reindex root blue score.
// If that's the case we keep the reindex root unchanged.
if selectedTipGHOSTDAGData.BlueScore() < currentRootGHOSTDAGData.BlueScore() ||
selectedTipGHOSTDAGData.BlueScore()-currentRootGHOSTDAGData.BlueScore() < rt.reindexSlack {
// Return current - this indicates no change
return currentReindexRoot, currentReindexRoot, nil
}
@@ -422,6 +432,11 @@ func (rt *reachabilityManager) findNextReindexRoot(currentReindexRoot, selectedT
return nil, nil, err
}
if selectedTipGHOSTDAGData.BlueScore() < chosenChildGHOSTDAGData.BlueScore() {
return nil, nil, errors.Errorf("chosen child %s has blue score greater "+
"than %s although it's in its selected parent chain", chosenChild, selectedTip)
}
if selectedTipGHOSTDAGData.BlueScore()-chosenChildGHOSTDAGData.BlueScore() < rt.reindexWindow {
break
}

View File

@@ -12,6 +12,26 @@ import (
func (sm *syncManager) antiPastHashesBetween(lowHash, highHash *externalapi.DomainHash,
maxBlueScoreDifference uint64) ([]*externalapi.DomainHash, error) {
// If lowHash is not in the selectedParentChain of highHash - SelectedChildIterator will fail.
// Therefore, we traverse down lowHash's selectedParentChain until we reach a block that is in
// highHash's selectedParentChain.
// We keep originalLowHash to filter out blocks in it's past later down the road
originalLowHash := lowHash
for {
isInSelectedParentChain, err := sm.dagTopologyManager.IsInSelectedParentChainOf(lowHash, highHash)
if err != nil {
return nil, err
}
if isInSelectedParentChain {
break
}
lowBlockGHOSTDAGData, err := sm.ghostdagDataStore.Get(sm.databaseContext, lowHash)
if err != nil {
return nil, err
}
lowHash = lowBlockGHOSTDAGData.SelectedParent()
}
lowBlockGHOSTDAGData, err := sm.ghostdagDataStore.Get(sm.databaseContext, lowHash)
if err != nil {
return nil, err
@@ -39,8 +59,11 @@ func (sm *syncManager) antiPastHashesBetween(lowHash, highHash *externalapi.Doma
if err != nil {
return nil, err
}
for iterator.Next() {
highHash = iterator.Get()
for ok := iterator.First(); ok; ok = iterator.Next() {
highHash, err = iterator.Get()
if err != nil {
return nil, err
}
highBlockGHOSTDAGData, err = sm.ghostdagDataStore.Get(sm.databaseContext, highHash)
if err != nil {
return nil, err
@@ -80,10 +103,17 @@ func (sm *syncManager) antiPastHashesBetween(lowHash, highHash *externalapi.Doma
if isCurrentAncestorOfLowHash {
continue
}
err = hashesUpHeap.Push(current)
// Push current to hashesUpHeap if it's not in the past of originalLowHash
isInPastOfOriginalLowHash, err := sm.dagTopologyManager.IsAncestorOf(current, originalLowHash)
if err != nil {
return nil, err
}
if !isInPastOfOriginalLowHash {
err = hashesUpHeap.Push(current)
if err != nil {
return nil, err
}
}
parents, err := sm.dagTopologyManager.Parents(current)
if err != nil {
return nil, err
@@ -112,8 +142,11 @@ func (sm *syncManager) missingBlockBodyHashes(highHash *externalapi.DomainHash)
lowHash := pruningPoint
foundHeaderOnlyBlock := false
for selectedChildIterator.Next() {
selectedChild := selectedChildIterator.Get()
for ok := selectedChildIterator.First(); ok; ok = selectedChildIterator.Next() {
selectedChild, err := selectedChildIterator.Get()
if err != nil {
return nil, err
}
hasBlock, err := sm.blockStore.HasBlock(sm.databaseContext, selectedChild)
if err != nil {
return nil, err
@@ -126,8 +159,14 @@ func (sm *syncManager) missingBlockBodyHashes(highHash *externalapi.DomainHash)
lowHash = selectedChild
}
if !foundHeaderOnlyBlock {
// TODO: Once block children are fixed, this error
// should be returned instead of simply logged
if lowHash == highHash {
// Blocks can be inserted inside the DAG during IBD if those were requested before IBD started.
// In rare cases, all the IBD blocks might be already inserted by the time we reach this point.
// In these cases - return an empty list of blocks to sync
return []*externalapi.DomainHash{}, nil
}
// TODO: Once block children are fixed (https://github.com/kaspanet/kaspad/issues/1499),
// this error should be returned rather the logged
log.Errorf("no header-only blocks between %s and %s",
lowHash, highHash)
}

View File

@@ -1,17 +1,18 @@
package consensus
import (
"testing"
"github.com/kaspanet/kaspad/domain/consensus/utils/testutils"
"github.com/kaspanet/kaspad/domain/dagconfig"
"testing"
)
func TestPruningDepth(t *testing.T) {
expectedResult := map[string]uint64{
"kaspa-mainnet": 244838,
"kaspa-testnet": 244838,
"kaspa-devnet": 244838,
"kaspa-simnet": 192038,
dagconfig.MainnetParams.Name: 244838,
dagconfig.TestnetParams.Name: 244838,
dagconfig.DevnetParams.Name: 244838,
dagconfig.SimnetParams.Name: 192038,
}
testutils.ForAllNets(t, true, func(t *testing.T, params *dagconfig.Params) {
expected, found := expectedResult[params.Name]

View File

@@ -263,11 +263,11 @@ func NewErrMissingParents(missingParentHashes []*externalapi.DomainHash) error {
// InvalidTransaction is a struct containing an invalid transaction, and the error explaining why it's invalid.
type InvalidTransaction struct {
Transaction *externalapi.DomainTransaction
err error
Error error
}
func (invalid InvalidTransaction) String() string {
return fmt.Sprintf("(%v: %s)", consensushashing.TransactionID(invalid.Transaction), invalid.err)
return fmt.Sprintf("(%v: %s)", consensushashing.TransactionID(invalid.Transaction), invalid.Error)
}
// ErrInvalidTransactionsInNewBlock indicates that some transactions in a new block are invalid

View File

@@ -56,8 +56,8 @@ func TestNewErrInvalidTransactionsInNewBlock(t *testing.T) {
if len(inner.InvalidTransactions) != 1 {
t.Fatalf("TestNewErrInvalidTransactionsInNewBlock: Expected len(inner.MissingOutpoints) 1, found: %d", len(inner.InvalidTransactions))
}
if inner.InvalidTransactions[0].err != ErrNoTxInputs {
t.Fatalf("TestNewErrInvalidTransactionsInNewBlock: Expected ErrNoTxInputs. found: %v", inner.InvalidTransactions[0].err)
if inner.InvalidTransactions[0].Error != ErrNoTxInputs {
t.Fatalf("TestNewErrInvalidTransactionsInNewBlock: Expected ErrNoTxInputs. found: %v", inner.InvalidTransactions[0].Error)
}
if inner.InvalidTransactions[0].Transaction.Fee != 1337 {
t.Fatalf("TestNewErrInvalidTransactionsInNewBlock: Expected 1337. found: %v", inner.InvalidTransactions[0].Transaction.Fee)

View File

@@ -12,9 +12,15 @@ type LRUCache struct {
}
// New creates a new LRUCache
func New(capacity int) *LRUCache {
func New(capacity int, preallocate bool) *LRUCache {
var cache map[externalapi.DomainHash]interface{}
if preallocate {
cache = make(map[externalapi.DomainHash]interface{}, capacity+1)
} else {
cache = make(map[externalapi.DomainHash]interface{})
}
return &LRUCache{
cache: make(map[externalapi.DomainHash]interface{}, capacity+1),
cache: cache,
capacity: capacity,
}
}

View File

@@ -10,9 +10,15 @@ type LRUCache struct {
}
// New creates a new LRUCache
func New(capacity int) *LRUCache {
func New(capacity int, preallocate bool) *LRUCache {
var cache map[uint64]*externalapi.DomainHash
if preallocate {
cache = make(map[uint64]*externalapi.DomainHash, capacity+1)
} else {
cache = make(map[uint64]*externalapi.DomainHash)
}
return &LRUCache{
cache: make(map[uint64]*externalapi.DomainHash, capacity+1),
cache: cache,
capacity: capacity,
}
}

View File

@@ -1,4 +1,4 @@
package consensus
package testutils
import (
"github.com/kaspanet/kaspad/domain/consensus/model/externalapi"

View File

@@ -1,10 +0,0 @@
package transactionid
import (
"github.com/kaspanet/kaspad/domain/consensus/model/externalapi"
)
// Less returns true iff transaction ID a is less than hash b
func Less(a, b *externalapi.DomainTransactionID) bool {
return externalapi.Less((*externalapi.DomainHash)(a), (*externalapi.DomainHash)(b))
}

View File

@@ -192,7 +192,7 @@ const (
// or to be in compressed format as a 32 byte string prefixed with 0x02 or 0x03 to signal oddness.
ErrPubKeyFormat
// ErrCleanStack is returned when after evalution, the stack
// ErrCleanStack is returned when after evaluation, the stack
// contains more than one element.
ErrCleanStack

View File

@@ -84,17 +84,6 @@ func isPushOnly(pops []parsedOpcode) bool {
return true
}
// IsPushOnlyScript returns whether or not the passed script only pushes data.
//
// False will be returned when the script does not parse.
func IsPushOnlyScript(script []byte) (bool, error) {
pops, err := parseScript(script)
if err != nil {
return false, err
}
return isPushOnly(pops), nil
}
// parseScriptTemplate is the same as parseScript but allows the passing of the
// template list for testing purposes. When there are parse errors, it returns
// the list of parsed opcodes up to the point of failure along with the error.

View File

@@ -3723,6 +3723,17 @@ func TestPushedData(t *testing.T) {
}
}
// isPushOnlyScript returns whether or not the passed script only pushes data.
//
// False will be returned when the script does not parse.
func isPushOnlyScript(script []byte) (bool, error) {
pops, err := parseScript(script)
if err != nil {
return false, err
}
return isPushOnly(pops), nil
}
// TestHasCanonicalPush ensures the canonicalPush function works as expected.
func TestHasCanonicalPush(t *testing.T) {
t.Parallel()
@@ -3734,8 +3745,8 @@ func TestHasCanonicalPush(t *testing.T) {
err)
continue
}
if result, _ := IsPushOnlyScript(script); !result {
t.Errorf("IsPushOnlyScript: test #%d failed: %x\n", i,
if result, _ := isPushOnlyScript(script); !result {
t.Errorf("isPushOnlyScript: test #%d failed: %x\n", i,
script)
continue
}
@@ -3760,8 +3771,8 @@ func TestHasCanonicalPush(t *testing.T) {
t.Errorf("StandardPushesTests test #%d unexpected error: %v\n", i, err)
continue
}
if result, _ := IsPushOnlyScript(script); !result {
t.Errorf("StandardPushesTests IsPushOnlyScript test #%d failed: %x\n", i, script)
if result, _ := isPushOnlyScript(script); !result {
t.Errorf("StandardPushesTests isPushOnlyScript test #%d failed: %x\n", i, script)
continue
}
pops, err := parseScript(script)
@@ -3889,9 +3900,9 @@ func TestHasCanonicalPushes(t *testing.T) {
}
}
// TestIsPushOnlyScript ensures the IsPushOnlyScript function returns the
// TestIsPushOnly ensures the isPushOnly function returns the
// expected results.
func TestIsPushOnlyScript(t *testing.T) {
func TestIsPushOnly(t *testing.T) {
t.Parallel()
tests := []struct {
@@ -3922,19 +3933,19 @@ func TestIsPushOnlyScript(t *testing.T) {
}
for _, test := range tests {
isPushOnly, err := IsPushOnlyScript(test.script)
isPushOnly, err := isPushOnlyScript(test.script)
if isPushOnly != test.expectedResult {
t.Errorf("IsPushOnlyScript (%s) wrong result\ngot: %v\nwant: "+
t.Errorf("isPushOnlyScript (%s) wrong result\ngot: %v\nwant: "+
"%v", test.name, isPushOnly, test.expectedResult)
}
if test.shouldFail && err == nil {
t.Errorf("IsPushOnlyScript (%s) expected an error but got <nil>", test.name)
t.Errorf("isPushOnlyScript (%s) expected an error but got <nil>", test.name)
}
if !test.shouldFail && err != nil {
t.Errorf("IsPushOnlyScript (%s) expected no error but got: %v", test.name, err)
t.Errorf("isPushOnlyScript (%s) expected no error but got: %v", test.name, err)
}
}
}

View File

@@ -12,9 +12,15 @@ type LRUCache struct {
}
// New creates a new LRUCache
func New(capacity int) *LRUCache {
func New(capacity int, preallocate bool) *LRUCache {
var cache map[externalapi.DomainOutpoint]externalapi.UTXOEntry
if preallocate {
cache = make(map[externalapi.DomainOutpoint]externalapi.UTXOEntry, capacity+1)
} else {
cache = make(map[externalapi.DomainOutpoint]externalapi.UTXOEntry)
}
return &LRUCache{
cache: make(map[externalapi.DomainOutpoint]externalapi.UTXOEntry, capacity+1),
cache: cache,
capacity: capacity,
}
}

View File

@@ -256,7 +256,7 @@ var MainnetParams = Params{
// TestnetParams defines the network parameters for the test Kaspa network.
var TestnetParams = Params{
K: defaultGHOSTDAGK,
Name: "kaspa-testnet",
Name: "kaspa-testnet-2",
Net: appmessage.Testnet,
RPCPort: "16210",
DefaultPort: "16211",

View File

@@ -7,6 +7,7 @@ package mempool
import (
"container/list"
"fmt"
"sort"
"sync"
"time"
@@ -78,6 +79,8 @@ type mempool struct {
mempoolUTXOSet *mempoolUTXOSet
consensus consensusexternalapi.Consensus
orderedTransactionsByFeeRate []*consensusexternalapi.DomainTransaction
// nextExpireScan is the time after which the orphan pool will be
// scanned in order to evict orphans. This is NOT a hard deadline as
// the scan will only run when an orphan is added to the pool as opposed
@@ -378,16 +381,18 @@ func (mp *mempool) removeBlockTransactionsFromPool(txs []*consensusexternalapi.D
for _, tx := range txs[transactionhelper.CoinbaseTransactionIndex+1:] {
txID := consensushashing.TransactionID(tx)
if _, exists := mp.fetchTxDesc(txID); !exists {
// We use the mempool transaction, because it has populated fee and mass
mempoolTx, exists := mp.fetchTxDesc(txID)
if !exists {
continue
}
err := mp.cleanTransactionFromSets(tx)
err := mp.cleanTransactionFromSets(mempoolTx.DomainTransaction)
if err != nil {
return err
}
mp.updateBlockTransactionChainedTransactions(tx)
mp.updateBlockTransactionChainedTransactions(mempoolTx.DomainTransaction)
}
return nil
}
@@ -434,7 +439,7 @@ func (mp *mempool) cleanTransactionFromSets(tx *consensusexternalapi.DomainTrans
delete(mp.pool, *txID)
delete(mp.chainedTransactions, *txID)
return nil
return mp.removeTransactionFromOrderedTransactionsByFeeRate(tx)
}
// updateBlockTransactionChainedTransactions processes the dependencies of a
@@ -504,7 +509,7 @@ func (mp *mempool) removeDoubleSpends(tx *consensusexternalapi.DomainTransaction
// helper for maybeAcceptTransaction.
//
// This function MUST be called with the mempool lock held (for writes).
func (mp *mempool) addTransaction(tx *consensusexternalapi.DomainTransaction, mass uint64, fee uint64, parentsInPool []consensusexternalapi.DomainOutpoint) (*txDescriptor, error) {
func (mp *mempool) addTransaction(tx *consensusexternalapi.DomainTransaction, parentsInPool []consensusexternalapi.DomainOutpoint) (*txDescriptor, error) {
// Add the transaction to the pool and mark the referenced outpoints
// as spent by the pool.
txDescriptor := &txDescriptor{
@@ -527,9 +532,77 @@ func (mp *mempool) addTransaction(tx *consensusexternalapi.DomainTransaction, ma
return nil, err
}
err = mp.addTransactionToOrderedTransactionsByFeeRate(tx)
if err != nil {
return nil, err
}
return txDescriptor, nil
}
func (mp *mempool) findTxIndexInOrderedTransactionsByFeeRate(tx *consensusexternalapi.DomainTransaction) (int, error) {
if tx.Fee == 0 || tx.Mass == 0 {
return 0, errors.Errorf("findTxIndexInOrderedTransactionsByFeeRate expects a transaction with " +
"populated fee and mass")
}
txID := consensushashing.TransactionID(tx)
txFeeRate := float64(tx.Fee) / float64(tx.Mass)
return sort.Search(len(mp.orderedTransactionsByFeeRate), func(i int) bool {
elementFeeRate := float64(mp.orderedTransactionsByFeeRate[i].Fee) / float64(mp.orderedTransactionsByFeeRate[i].Mass)
if elementFeeRate > txFeeRate {
return true
}
if elementFeeRate == txFeeRate && txID.LessOrEqual(consensushashing.TransactionID(mp.orderedTransactionsByFeeRate[i])) {
return true
}
return false
}), nil
}
func (mp *mempool) addTransactionToOrderedTransactionsByFeeRate(tx *consensusexternalapi.DomainTransaction) error {
index, err := mp.findTxIndexInOrderedTransactionsByFeeRate(tx)
if err != nil {
return err
}
mp.orderedTransactionsByFeeRate = append(mp.orderedTransactionsByFeeRate[:index],
append([]*consensusexternalapi.DomainTransaction{tx}, mp.orderedTransactionsByFeeRate[index:]...)...)
return nil
}
func (mp *mempool) removeTransactionFromOrderedTransactionsByFeeRate(tx *consensusexternalapi.DomainTransaction) error {
index, err := mp.findTxIndexInOrderedTransactionsByFeeRate(tx)
if err != nil {
return err
}
txID := consensushashing.TransactionID(tx)
if !consensushashing.TransactionID(mp.orderedTransactionsByFeeRate[index]).Equal(txID) {
return errors.Errorf("Couldn't find %s in mp.orderedTransactionsByFeeRate", txID)
}
mp.orderedTransactionsByFeeRate = append(mp.orderedTransactionsByFeeRate[:index], mp.orderedTransactionsByFeeRate[index+1:]...)
return nil
}
func (mp *mempool) enforceTransactionLimit() error {
const limit = 1_000_000
if len(mp.pool)+len(mp.chainedTransactions) > limit {
// mp.orderedTransactionsByFeeRate[0] is the least profitable transaction
txToRemove := mp.orderedTransactionsByFeeRate[0]
log.Debugf("Mempool size is over the limit of %d transactions. Removing %s",
limit,
consensushashing.TransactionID(txToRemove),
)
return mp.removeTransactionAndItsChainedTransactions(txToRemove)
}
return nil
}
// checkPoolDoubleSpend checks whether or not the passed transaction is
// attempting to spend coins already spent by other transactions in the pool.
// Note it does not check for double spends against transactions already in the
@@ -673,7 +746,7 @@ func (mp *mempool) maybeAcceptTransaction(tx *consensusexternalapi.DomainTransac
return nil, nil, txRuleError(RejectInsufficientFee, str)
}
// Add to transaction pool.
txDesc, err := mp.addTransaction(tx, tx.Mass, tx.Fee, parentsInPool)
txDesc, err := mp.addTransaction(tx, parentsInPool)
if err != nil {
return nil, nil, err
}
@@ -681,6 +754,11 @@ func (mp *mempool) maybeAcceptTransaction(tx *consensusexternalapi.DomainTransac
log.Debugf("Accepted transaction %s (pool size: %d)", txID,
len(mp.pool))
err = mp.enforceTransactionLimit()
if err != nil {
return nil, nil, err
}
return nil, txDesc, nil
}

View File

@@ -222,19 +222,6 @@ func checkTransactionStandard(tx *consensusexternalapi.DomainTransaction, policy
maxStandardSigScriptSize)
return txRuleError(RejectNonstandard, str)
}
// Each transaction input signature script must only contain
// opcodes which push data onto the stack.
isPushOnly, err := txscript.IsPushOnlyScript(txIn.SignatureScript)
if err != nil {
str := fmt.Sprintf("transaction input %d: IsPushOnlyScript: %t. Error %s", i, isPushOnly, err)
return txRuleError(RejectNonstandard, str)
}
if !isPushOnly {
str := fmt.Sprintf("transaction input %d: signature "+
"script is not push only", i)
return txRuleError(RejectNonstandard, str)
}
}
// None of the output public key scripts can be a non-standard script or

View File

@@ -235,18 +235,6 @@ func TestCheckTransactionStandard(t *testing.T) {
isStandard: false,
code: RejectNonstandard,
},
{
name: "Signature script that does more than push data",
tx: consensusexternalapi.DomainTransaction{Version: 0, Inputs: []*consensusexternalapi.DomainTransactionInput{{
PreviousOutpoint: dummyPrevOut,
SignatureScript: []byte{
txscript.OpCheckSigVerify},
Sequence: constants.MaxTxInSequenceNum,
}}, Outputs: []*consensusexternalapi.DomainTransactionOutput{&dummyTxOut}},
height: 300000,
isStandard: false,
code: RejectNonstandard,
},
{
name: "Valid but non standard public key script",
tx: consensusexternalapi.DomainTransaction{Version: 0, Inputs: []*consensusexternalapi.DomainTransactionInput{&dummyTxIn}, Outputs: []*consensusexternalapi.DomainTransactionOutput{{

View File

@@ -102,7 +102,7 @@ type Flags struct {
ProxyPass string `long:"proxypass" default-mask:"-" description:"Password for proxy server"`
DbType string `long:"dbtype" description:"Database backend to use for the Block DAG"`
Profile string `long:"profile" description:"Enable HTTP profiling on given port -- NOTE port must be between 1024 and 65536"`
DebugLevel string `short:"d" long:"debuglevel" description:"Logging level for all subsystems {trace, debug, info, warn, error, critical} -- You may also specify <subsystem>=<level>,<subsystem2>=<level>,... to set the log level for individual subsystems -- Use show to list available subsystems"`
LogLevel string `short:"d" long:"loglevel" description:"Logging level for all subsystems {trace, debug, info, warn, error, critical} -- You may also specify <subsystem>=<level>,<subsystem2>=<level>,... to set the log level for individual subsystems -- Use show to list available subsystems"`
Upnp bool `long:"upnp" description:"Use UPnP to map our listening port outside of NAT"`
MinRelayTxFee float64 `long:"minrelaytxfee" description:"The minimum transaction fee in KAS/kB to be considered a non-zero fee."`
MaxOrphanTxs int `long:"maxorphantx" description:"Max number of orphan transactions to keep in memory"`
@@ -166,7 +166,7 @@ func newConfigParser(cfgFlags *Flags, options flags.Options) *flags.Parser {
func defaultFlags() *Flags {
return &Flags{
ConfigFile: defaultConfigFile,
DebugLevel: defaultLogLevel,
LogLevel: defaultLogLevel,
TargetOutboundPeers: defaultTargetOutboundPeers,
MaxInboundPeers: defaultMaxInboundPeers,
BanDuration: defaultBanDuration,
@@ -326,7 +326,7 @@ func LoadConfig() (*Config, error) {
cfg.LogDir = filepath.Join(cfg.LogDir, cfg.NetParams().Name)
// Special show command to list supported subsystems and exit.
if cfg.DebugLevel == "show" {
if cfg.LogLevel == "show" {
fmt.Println("Supported subsystems", logger.SupportedSubsystems())
os.Exit(0)
}
@@ -336,7 +336,7 @@ func LoadConfig() (*Config, error) {
logger.InitLog(filepath.Join(cfg.LogDir, defaultLogFilename), filepath.Join(cfg.LogDir, defaultErrLogFilename))
// Parse, validate, and set debug log level(s).
if err := logger.ParseAndSetDebugLevels(cfg.DebugLevel); err != nil {
if err := logger.ParseAndSetLogLevels(cfg.LogLevel); err != nil {
err := errors.Errorf("%s: %s", funcName, err.Error())
fmt.Fprintln(os.Stderr, err)
fmt.Fprintln(os.Stderr, usageMessage)

Some files were not shown because too many files have changed in this diff Show More