kaspad/app/appmessage/p2p_msgblock.go
stasatdaglabs 4c915f12b7
[NOD-1319] Reimplement kaspad's RPC in gRPC (#914)
* [NOD-1319] Create the protorpc package.

* [NOD-1319] Use a general ClientStream in closeSend.

* [NOD-1319] Decouple p2pServer from gRPCServer.

* [NOD-1319] Begin implementing rpcServer.

* [NOD-1319] Move grpcStream to grpc_connection.go.

* [NOD-1319] Fold the rpc messages.proto into a common message.proto.

* [NOD-1319] Remove code duplication in MessageStream.

* [NOD-1319] Rename methods in netadapter.

* [NOD-1319] Rename message_xxx to p2p_xxx.

* [NOD-1319] Section off p2p messages from rpc messages in messages.proto.

* [NOD-1319] Split toPayload to a p2p part and and rpc part.

* [NOD-1319] Rename msgxxx.go to p2p_msgxx.go in the appmessage package.

* [NOD-1319] Implement GetCurrentVersionRequestMessage and GetCurrentVersionResponseMessage.

* [NOD-1319] Implement toAppMessage and fromAppMessage for getCurrentNetwork

* [NOD-1319] Make a temporary workaround so that tests pass.

* [NOD-1319] Begin implementing the rpc manager.

* [NOD-1319] Implement an initial routerInitializer for rpc.

* [NOD-1319] Rename the spawn in routerInitializer.

* [NOD-1319] Implement an RPC context.

* [NOD-1319] Move the actual handlers to a separate package.

* [NOD-1319] Use the correct value for the GetCurrentNetwork response.

* [NOD-1319] Fix some names.

* [NOD-1319] Begin implementing a cli rpc client.

* [NOD-1319] Implement connecting to the RPC server.

* [NOD-1319] Make sure that connecting to the server and sending/receiving messages works.

* [NOD-1319] Make kaspactl2 speak in json strings.

* [NOD-1319] Finish implementing kaspactl2.

* [NOD-1319] Remove debug messages.

* [NOD-1319] Properly handle errors in rpc.go.

* [NOD-1319] Move the grpc client to a separate package.

* [NOD-1319] Extract Post out of PostString.

* [NOD-1319] Implement PostAppMessage.

* [NOD-1319] Stub out submitBlock.

* [NOD-1319] Stub out getBlockTemplate.

* [NOD-1319] Combine request and reponse files.

* [NOD-1319] Implement submitBlock.

* [NOD-1319] Implement returning errors from RPC.

* [NOD-1319] Begin implementing getBlockTemplate.

* [NOD-1319] Add missing field in GetBlockTemplateRequestMessage.

* [NOD-1319] Implement a minimal getBlockTemplate.

* [NOD-1319] Add getBlockTemplate stuff to grpc.

* [NOD-1319] Implement the rest of getBlockTemplate.

* [NOD-1319] Add block/transaction added handlers to the protocol manager.

* [NOD-1319] Implement NotifyTransactionAddedToMempool.

* [NOD-1319] Implement NotifyBlockAddedToDAG.

* [NOD-1319] Connect block/transaction added handlers.

* [NOD-1319] Add notifyBlockAdded.

* [NOD-1319] Add a notification system.

* [NOD-1319] Improve the notification system.

* [NOD-1319] Add a block added listener stub.

* [NOD-1319] Add BlockAddedNotificationMessage.

* [NOD-1319] Finish implementing HandleNotifyBlockAdded.

* [NOD-1319] Println instead of Print in kaspactl2.

* [NOD-1319] Remove unused flags in kaspactl2.

* [NOD-1319] Make kaspaminer work with the new RPC.

* [NOD-1319] Fix a bad log.

* [NOD-1319] Make kaspaminer work.

* [NOD-1319] Disconnect the old RPC.

* [NOD-1319] Move grpcclient.go.

* [NOD-1319] Begin generalizing the rpcClient.

* [NOD-1319] Move errors to the side of the payload.

* [NOD-1319] Add errors to appmessage.

* [NOD-1319] Move AttachRouter to grpcclient.

* [NOD-1319] Fix kaspaminer not handling responses.

* [NOD-1319] Properly handle blockAddedNotifications.

* [NOD-1319] Move errors into individual response objects.

* [NOD-1319] Begin replacing the RPC client in the integration tests.

* [NOD-1319] Implement GetPeerAddresses.

* [NOD-1319] Implement GetPeerAddresses.

* [NOD-1319] Fix setOnBlockAddedHandler.

* [NOD-1319] Remove the old kaspactl.

* [NOD-1319] Move ConvertGetBlockTemplateResultToBlock to the mining package.

* [NOD-1319] Implement getSelectedTipHash.

* [NOD-1319] Simplify testRPCRouter.

* [NOD-1319] Write stubs for the required test RPC commands.

* [NOD-1319] Implement a minimal getMempoolEntry.

* [NOD-1319] Implement a minimal getMempoolEntry.

* [NOD-1319] Implement getConnectedPeerInfo.

* [NOD-1319] Delete the old RPC.

* [NOD-1319] Fix a fromAppMessage.

* [NOD-1319] Implement connectToPeer.

* [NOD-1319] Fix a bug in registerForBlockAddedNotifications.

* [NOD-1319] Fix a deadlock in closing notification listeners.

* [NOD-1319] Fix merge errors.

* [NOD-1319] Fix an import.

* [NOD-1319] Properly handle errors in grpcclient.

* [NOD-1319] Fix TestIBD.

* [NOD-1319] Prevent kaspaminer from running when not connected.

* [NOD-1319] Implement sendRawTransaction.

* [NOD-1319] Implement sendRawTransaction in the client.

* [NOD-1319] Extract a general RPC client from the integration test RPC client.

* [NOD-1319] Use the general RPC client for the miner.

* [NOD-1319] Move the rpcclient package out of netadapter.

* [NOD-1319] Normalize onBlockAdded.

* [NOD-1319] Begin implementing notifyChainChanged.

* [NOD-1319] Implement the model for notifyChainChanged.

* [NOD-1319] Implement conversions for notifyChainChanged.

* [NOD-1319] Implement HandleNotifyChainChanged.

* [NOD-1319] Normalize notifications.

* [NOD-1319] Implement RegisterForChainChangedNotifications.

* [NOD-1319] Begin connecting blockdag's chain-changed notification with the RPC.

* [NOD-1319] Finish implementing notifyChainChanged.

* [NOD-1319] Implement getBlockHex.

* [NOD-1319] Rename getBlockHex to getBlock.

* [NOD-1319] Implement the verbose model for getBlock.

* [NOD-1319] Implement buildBlockVerboseData.

* [NOD-1319] Implement buildTransactionVerboseData.

* [NOD-1319] Move verboseData stuff to verbosedata.go.

* [NOD-1319] Add includeTransactionVerboseData.

* [NOD-1319] Begin implementing getSubnetwork.

* [NOD-1319] Finish implementing getSubnetwork.

* [NOD-1319] Begin implementing getChainFromBlock.

* [NOD-1319] Finish implementing getChainFromBlock.

* [NOD-1319] Begin implementing getBlocks.

* [NOD-1319] Finish implementing getBlocks.

* [NOD-1319] Fix bad responses in HandleNotifyChainChanged.

* [NOD-1319] Fix bugs in verbosedata.go.

* [NOD-1319] Fix more bugs in verbosedata.go.

* [NOD-1319] Make go vet happy.

* [NOD-1319] Extract handleBlockDAGNotifications to a method.

* [NOD-1319] Add a newline.

* [NOD-1319] Use peers instead of connections to check if connected.

* [NOD-1319] Add a comment.

* [NOD-1319] Lock the dag lock in getBlock.

* [NOD-1319] Rename netAdapter.connections to p2pConnections.

* [NOD-1319] In protowire, rename wireXXX to protoXXX.

* [NOD-1319] Rename PostString to PostJSON.

* [NOD-1319] Disallow empty transactions in SendRawTransaction.

* [NOD-1319] Disallow empty blocks in SubmitBlocks.

* [NOD-1319] Add SetLogger.

* [NOD-1319] Fix an error message.

* [NOD-1319] Fix an error message.

* [NOD-1319] Rename testTimeout to rpcTimeout.

* [NOD-1319] Rename SendRawTransaction to SubmitTransaction.

* [NOD-1319] Rename ConnectToPeer to AddPeer.

* [NOD-1319] Add missing longPollID to request.

* [NOD-1319] Rename ChainChangedChainBlock to ChainBlock.

* [NOD-1319] Rename Vin and Vout.

* [NOD-1319] Implement RPCErrorf.

* [NOD-1319] Fix RPCErrorf's comment.

* [NOD-1319] Remove unused flags in kaspaminer.
2020-09-07 14:35:40 +03:00

251 lines
8.5 KiB
Go

// Copyright (c) 2013-2016 The btcsuite developers
// Use of this source code is governed by an ISC
// license that can be found in the LICENSE file.
package appmessage
import (
"bytes"
"fmt"
"io"
"github.com/kaspanet/kaspad/util/subnetworkid"
"github.com/kaspanet/kaspad/util/daghash"
)
// defaultTransactionAlloc is the default size used for the backing array
// for transactions. The transaction array will dynamically grow as needed, but
// this figure is intended to provide enough space for the number of
// transactions in the vast majority of blocks without needing to grow the
// backing array multiple times.
const defaultTransactionAlloc = 2048
// MaxMassPerBlock is the maximum total transaction mass a block may have.
const MaxMassPerBlock = 10000000
// MaxMassPerTx is the maximum total mass a transaction may have.
const MaxMassPerTx = MaxMassPerBlock / 2
// MaxTxPerBlock is the maximum number of transactions that could
// possibly fit into a block.
const MaxTxPerBlock = (MaxMassPerBlock / minTxPayload) + 1
// TxLoc holds locator data for the offset and length of where a transaction is
// located within a MsgBlock data buffer.
type TxLoc struct {
TxStart int
TxLen int
}
// MsgBlock implements the Message interface and represents a kaspa
// block message. It is used to deliver block and transaction information in
// response to a getdata message (MsgGetData) for a given block hash.
type MsgBlock struct {
baseMessage
Header BlockHeader
Transactions []*MsgTx
}
// AddTransaction adds a transaction to the message.
func (msg *MsgBlock) AddTransaction(tx *MsgTx) {
msg.Transactions = append(msg.Transactions, tx)
}
// ClearTransactions removes all transactions from the message.
func (msg *MsgBlock) ClearTransactions() {
msg.Transactions = make([]*MsgTx, 0, defaultTransactionAlloc)
}
// KaspaDecode decodes r using the kaspa protocol encoding into the receiver.
// This is part of the Message interface implementation.
// See Deserialize for decoding blocks stored to disk, such as in a database, as
// opposed to decoding blocks from the appmessage.
func (msg *MsgBlock) KaspaDecode(r io.Reader, pver uint32) error {
err := readBlockHeader(r, pver, &msg.Header)
if err != nil {
return err
}
txCount, err := ReadVarInt(r)
if err != nil {
return err
}
// Prevent more transactions than could possibly fit into a block.
// It would be possible to cause memory exhaustion and panics without
// a sane upper bound on this count.
if txCount > MaxTxPerBlock {
str := fmt.Sprintf("too many transactions to fit into a block "+
"[count %d, max %d]", txCount, MaxTxPerBlock)
return messageError("MsgBlock.KaspaDecode", str)
}
msg.Transactions = make([]*MsgTx, 0, txCount)
for i := uint64(0); i < txCount; i++ {
tx := MsgTx{}
err := tx.KaspaDecode(r, pver)
if err != nil {
return err
}
msg.Transactions = append(msg.Transactions, &tx)
}
return nil
}
// Deserialize decodes a block from r into the receiver using a format that is
// suitable for long-term storage such as a database while respecting the
// Version field in the block. This function differs from KaspaDecode in that
// KaspaDecode decodes from the kaspa appmessage protocol as it was sent across the
// network. The appmessage encoding can technically differ depending on the protocol
// version and doesn't even really need to match the format of a stored block at
// all. As of the time this comment was written, the encoded block is the same
// in both instances, but there is a distinct difference and separating the two
// allows the API to be flexible enough to deal with changes.
func (msg *MsgBlock) Deserialize(r io.Reader) error {
// At the current time, there is no difference between the appmessage encoding
// at protocol version 0 and the stable long-term storage format. As
// a result, make use of KaspaDecode.
return msg.KaspaDecode(r, 0)
}
// DeserializeTxLoc decodes r in the same manner Deserialize does, but it takes
// a byte buffer instead of a generic reader and returns a slice containing the
// start and length of each transaction within the raw data that is being
// deserialized.
func (msg *MsgBlock) DeserializeTxLoc(r *bytes.Buffer) ([]TxLoc, error) {
fullLen := r.Len()
// At the current time, there is no difference between the appmessage encoding
// at protocol version 0 and the stable long-term storage format. As
// a result, make use of existing appmessage protocol functions.
err := readBlockHeader(r, 0, &msg.Header)
if err != nil {
return nil, err
}
txCount, err := ReadVarInt(r)
if err != nil {
return nil, err
}
// Prevent more transactions than could possibly fit into a block.
// It would be possible to cause memory exhaustion and panics without
// a sane upper bound on this count.
if txCount > MaxTxPerBlock {
str := fmt.Sprintf("too many transactions to fit into a block "+
"[count %d, max %d]", txCount, MaxTxPerBlock)
return nil, messageError("MsgBlock.DeserializeTxLoc", str)
}
// Deserialize each transaction while keeping track of its location
// within the byte stream.
msg.Transactions = make([]*MsgTx, 0, txCount)
txLocs := make([]TxLoc, txCount)
for i := uint64(0); i < txCount; i++ {
txLocs[i].TxStart = fullLen - r.Len()
tx := MsgTx{}
err := tx.Deserialize(r)
if err != nil {
return nil, err
}
msg.Transactions = append(msg.Transactions, &tx)
txLocs[i].TxLen = (fullLen - r.Len()) - txLocs[i].TxStart
}
return txLocs, nil
}
// KaspaEncode encodes the receiver to w using the kaspa protocol encoding.
// This is part of the Message interface implementation.
// See Serialize for encoding blocks to be stored to disk, such as in a
// database, as opposed to encoding blocks for the appmessage.
func (msg *MsgBlock) KaspaEncode(w io.Writer, pver uint32) error {
err := writeBlockHeader(w, pver, &msg.Header)
if err != nil {
return err
}
err = WriteVarInt(w, uint64(len(msg.Transactions)))
if err != nil {
return err
}
for _, tx := range msg.Transactions {
err = tx.KaspaEncode(w, pver)
if err != nil {
return err
}
}
return nil
}
// Serialize encodes the block to w using a format that suitable for long-term
// storage such as a database while respecting the Version field in the block.
// This function differs from KaspaEncode in that KaspaEncode encodes the block to
// the kaspa appmessage protocol in order to be sent across the network. The appmessage
// encoding can technically differ depending on the protocol version and doesn't
// even really need to match the format of a stored block at all. As of the
// time this comment was written, the encoded block is the same in both
// instances, but there is a distinct difference and separating the two allows
// the API to be flexible enough to deal with changes.
func (msg *MsgBlock) Serialize(w io.Writer) error {
// At the current time, there is no difference between the appmessage encoding
// at protocol version 0 and the stable long-term storage format. As
// a result, make use of KaspaEncode.
return msg.KaspaEncode(w, 0)
}
// SerializeSize returns the number of bytes it would take to serialize the
// block.
func (msg *MsgBlock) SerializeSize() int {
// Block header bytes + Serialized varint size for the number of
// transactions.
n := msg.Header.SerializeSize() + VarIntSerializeSize(uint64(len(msg.Transactions)))
for _, tx := range msg.Transactions {
n += tx.SerializeSize()
}
return n
}
// Command returns the protocol command string for the message. This is part
// of the Message interface implementation.
func (msg *MsgBlock) Command() MessageCommand {
return CmdBlock
}
// MaxPayloadLength returns the maximum length the payload can be for the
// receiver. This is part of the Message interface implementation.
func (msg *MsgBlock) MaxPayloadLength(pver uint32) uint32 {
return MaxMessagePayload
}
// BlockHash computes the block identifier hash for this block.
func (msg *MsgBlock) BlockHash() *daghash.Hash {
return msg.Header.BlockHash()
}
// ConvertToPartial clears out all the payloads of the subnetworks that are
// incompatible with the given subnetwork ID.
// Note: this operation modifies the block in place.
func (msg *MsgBlock) ConvertToPartial(subnetworkID *subnetworkid.SubnetworkID) {
for _, tx := range msg.Transactions {
if !tx.SubnetworkID.IsEqual(subnetworkID) {
tx.Payload = []byte{}
}
}
}
// NewMsgBlock returns a new kaspa block message that conforms to the
// Message interface. See MsgBlock for details.
func NewMsgBlock(blockHeader *BlockHeader) *MsgBlock {
return &MsgBlock{
Header: *blockHeader,
Transactions: make([]*MsgTx, 0, defaultTransactionAlloc),
}
}