220 Commits

Author SHA1 Message Date
Ori Newman
765dd170e4
Optimizations and header size reduce hardfork (#1853)
* Modify DefaultTimeout to 120 seconds

A temporary workaround for nodes having trouble to sync (currently the download of pruning point related data during IBD takes more than 30 seconds)

* Cache existence in reachability store

* Cache block level in the header

* Fix IBD indication on submit block

* Add hardForkOmitGenesisFromParentsDAAScore logic

* Fix NumThreads bug in the wallet

* Get rid of ParentsAtLevel header method

* Fix a bug in BuildPruningPointProof

* Increase race detector timeout

* Add cache to BuildPruningPointProof

* Add comments and temp comment out go vet

* Fix ParentsAtLevel

* Dont fill empty parents

* Change HardForkOmitGenesisFromParentsDAAScore in fast netsync test

* Add --allow-submit-block-when-not-synced in stability tests

* Fix TestPruning

* Return fast tests

* Fix off by one error on kaspawallet

* Fetch only one block with trusted data at a time

* Update fork DAA score

* Don't ban for unexpected message type

* Fix tests

Co-authored-by: Michael Sutton <mikisiton2@gmail.com>
Co-authored-by: Ori Newman <>
2021-11-22 09:00:39 +02:00
Ori Newman
ad8bdbed21
Update changelog (#1845)
Co-authored-by: Kaspa Profiler <>
2021-11-09 00:32:56 +02:00
Elichai Turkel
7cdceb6df0
Cache the miner state (#1844)
* Implement a MinerState to cache the matrix and friends

* Modify the miner and related code to use the new MinerCache

* Change MinerState to State

* Make go lint happy

Co-authored-by: Ori Newman <orinewman1@gmail.com>
Co-authored-by: Kaspa Profiler <>
2021-11-09 00:12:30 +02:00
Ori Newman
0a1daae319
Allow mainnet flag and raise wallet fee (#1838)
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2021-11-07 10:04:27 +02:00
stasatdaglabs
3dbc42b4f7
Implement the new block subsidy function (#1830)
* Replace the old blockSubsidy parameters with the new ones.

* Return subsidyGenesisReward if blockHash is the genesis hash.

* Traverse a block's past for the subsidy calculation.

* Partially implement SubsidyStore.

* Refer to SubsidyStore from CoinbaseManager.

* Wrap calcBlockSubsidy in getBlockSubsidy, which first checks the database.

* Fix finalityStore not calling GenerateShardingID.

* Implement calculateAveragePastSubsidy.

* Implement calculateMergeSetSubsidySum.

* Implement calculateSubsidyRandomVariable.

* Implement calcBlockSubsidy.

* Add a TODO about floats.

* Update the calcBlockSubsidy TODO.

* Use binary.LittleEndian in calculateSubsidyRandomVariable.

* Fix bad range in calculateSubsidyRandomVariable.

* Replace float64 with big.Rat everywhere except for subsidyRandomVariable.

* Fix a nil dereference.

* Use a random walk to approximate the normal distribution.

* In order to avoid unsupported fractional results from powInt64, flip the numerator and the denominator manually.

* Set standardDeviation to 0.25, MaxSompi to 10_000_000_000 * SompiPerKaspa and defaultSubsidyGenesisReward to 1_000.

* Set the standard deviation to 0.2.

* Use a binomial distribution instead of trying to estimate the normal distribution.

* Change some values around.

* Clamp the block subsidy.

* Remove the fake duplicate constants in the util package.

* Reduce MaxSompi to only 100m Kaspa to avoid hitting the uint64 ceiling.

* Lower MaxSompi further to avoid new and exciting ways for the uint64 ceiling to be hit.

* Remove debug logs.

* Fix a couple of failing tests.

* Fix TestBlockWindow.

* Fix limitTransactionCount sometimes crashing on index-out-of-bounds.

* In TrustedDataDataDAABlock, replace BlockHeader with DomainBlock

* In calculateAveragePastSubsidy, use blockWindow instead of doing a BFS manually.

* Remove the reference to DAGTopologyManager in coinbaseManager.

* Add subsidy to the coinbase payload.

* Get rid of the subsidy store and extract subsidies out of coinbase transactions.

* Keep a blockWindow amount of blocks under the virtual for IBD purposes.

* Manually remove the virtual genesis from the merge set.

* Fix simnet genesis.

* Fix TestPruning.

* Fix TestCheckBlockIsNotPruned.

* Fix TestBlockWindow.

* Fix TestCalculateSignatureHashSchnorr.

* Fix TestCalculateSignatureHashECDSA.

* Fix serializing the wrong value into the coinbase payload.

* Rename coinbaseOutputForBlueBlock to coinbaseOutputAndSubsidyForBlueBlock.

* Add a TODO about optimizing trusted data DAA window blocks.

* Expand on a comment in TestCheckBlockIsNotPruned.

* In calcBlockSubsidy, divide the big.Int numerator by the big.Int denominator instead of converting to float64.

* Clarify a comment.

* Rename SubsidyMinGenesisReward to MinSubsidy.

* Properly handle trusted data blocks in calculateMergeSetSubsidySum.

* Use the first two bytes of the selected parent's hash for randomness instead of math/rand.

* Restore maxSompi to what it used to be.

* Fix TestPruning.

* Fix TestAmountCreation.

* Fix TestBlockWindow.

* Fix TestAmountUnitConversions.

* Increase the timeout in many-tips to 30 minutes.

* Check coinbase subsidy for every block

* Re-rename functions

* Use shift instead of powInt64 to determine subsidyRandom

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-10-30 10:16:47 +03:00
stasatdaglabs
837dac68b5
Update block headers to include multiple levels of parent blocks (#1822)
* Replace the old parents in the block header with BlockLevelParents.

* Begin fixing compilation errors.

* Implement database serialization for block level parents.

* Implement p2p serialization for block level parents.

* Implement rpc serialization for block level parents.

* Add DirectParents() to the block header interface.

* Use DirectParents() instead of Parents() in some places.

* Revert test_block_builder.go.

* Add block level parents to hash serialization.

* Use the zero hash for genesis finality points.

* Fix failing tests.

* Fix a variable name.

* Update headerEstimatedSerializedSize.

* Add comments in blocklevelparents.go.

* Fix the rpc-stability stability test.

* Change the field number for `parents` fields in p2p.proto and rpc.proto.

* Remove MsgBlockHeader::NumParentBlocks.
2021-08-24 12:06:39 +03:00
Svarog
369a3bac09
Limit block mass instead of merge set limit + Introduce SigOpCount to TransactionInput (#1790)
* Update constants

* Add to transaction SigOpCount

* Update mass calculation, and move it from InContext to InIsolation

* Update block validation accordingly

* Add SigOpCount validation during TransactionInContext

* Remove checking of mass vs maxMassAcceptedByBlock from consensusStateManager

* Update mining manager with latest changes

* Add SigOpCount to MsgTx.Copy()

* Fix initTestTransactionAcceptanceDataForClone

* Fix all tests in transaction_equal_clone_test.go

* Fix TestBlockMass

* Fix tests in transactionvalidator package

* Add SigOpCount to sighash

* Fix TestPruningDepth

* Fix problems in libkaspawalelt

* Fix integration tests

* Fix CalculateSignatureHash tests

* Remove remaining places talking about block size

* Add sanity check to checkBlockMass to make sure all transactions have their mass filled

* always add own sigOpCount to sigHash

* Update protowire/rpc.md

* Start working on removing any remaining reference to block/tx size

* Update rpc transaction verbose data to include mass rather then size

* Convert verboseData and block size check to mass

* Remove remaining usages of tx size in mempool

* Move transactionEstimatedSerializedSize to transactionvalidator

* Add PopulateMass to fakeRelayInvsContext

* Move PopulateMass to beggining of ValidateAndInsertTransaction + fix in it

* Assign mass a new number for backward-compatibility
2021-07-14 14:21:57 +03:00
Constantine Bitensky
e3a6d9e49a
Added RPC connection server version checking, fixes https://github.com/kaspanet/kaspad/issues/1047 (#1783)
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-07-06 15:10:35 +03:00
Svarog
798c5fab7d
Add allowOrphans to rpcclient.SubmitTransaction (#1765) 2021-06-24 15:23:10 +03:00
Svarog
9bedf84740
kaspawallet: Add timeout to connect to daemon, and print friendly error (#1756)
message if it's not available
2021-06-16 15:05:19 +03:00
cbitensky
d1df97c4c5
added --password and --yes cmdline parameters (#1735)
* added --password and --yes cmdline parameters

* Renamed:
confPassword -> cmdLinePassword
yes -> forceOverride

Added password in ImportMnemonics

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-06-01 15:36:18 +03:00
Svarog
4f4a8934e7
Add option to specify blockHash in EstimateNetworkHashesPerSecond (#1731)
* Add BlockHash optional parameter to EstimateNetworkBlockHashesPerSecond

* Allow to pass '-' for optional values in kaspactl

* Solve a division-by-zero in estimateNetworkHashesPerSecond

* Add BlockHash to toAppMessage/fromAppMessage functions

* Rename: topHash -> StartHash

* Return proper error message if provided startHash doesn't deserialize into a hash
2021-05-27 14:59:29 +03:00
Svarog
16ba2bd312
Export DefaultPath + Add logging to kaspawallet daemon (#1730)
* Export DefaultPath + Add logging to kaspawallet daemon

* Export purpose and CoinType instead of defaultPath

* Move TODO to correct place
2021-05-26 18:51:11 +03:00
Svarog
6613faee2d
Invert coin-type and purpose in default derivation path (#1728) 2021-05-20 17:43:07 +03:00
Ori Newman
4658f9d05c
Implement BIP 39 and HD wallet features (#1705)
* Naive bip39 with address reuse

* Avoid address reuse in libkaspawallet

* Add wallet daemon

* Use daemon everywhere

* Add forceOverride

* Make CreateUnsignedTransaction endpoint receive amount in sompis

* Collect close UTXOs

* Filter out non-spendable UTXOs from selectUTXOs

* Use different paths for multisig and non multisig

* Fix tests to use non zero path

* Fix multisig cosigner index detection

* Add comments

* Fix dump_unencrypted_data.go according to bip39 and bip32

* Fix wrong derivation path for multisig on wallet creation

* Remove IsSynced endpoint and add validation if wallet is synced for the relevant endpoints

* Rename server address to daemon address

* Fix capacity for extendedPublicKeys

* Use ReadBytes instead of ReadLine

* Add validation when importing

* Increment before using index value, and use it as is

* Save keys file exactly where needed

* Use %+v printErrorAndExit

* Remove redundant consts

* Rnemae collectCloseUTXOs and collectFarUTXOs

* Move typedefs around

* Add comment to addressesToQuery

* Update collectUTXOsFromRecentAddresses comment about locks

* Split collectUTXOs to small functions

* Add sanity check

* Add addEntryToUTXOSet function

* Change validateIsSynced to isSynced

* Simplify createKeyPairsFromFunction logic

* Rename .Sync() to .Save()

* Fix typo

* Create bip39BitSize const

* Add consts to purposes

* Add multisig check for 'send'

* Rename updatedPSTxBytes to partiallySignedTransaction

* Change collectUTXOsFromFarAddresses's comment

* Use setters for last used indexes

* Don't use the pstx acronym

* Fix SetPath

* Remove spaces when reading lines

* Fix walletserver to daemonaddress

* Fix isUTXOSpendable to use DAA score

Co-authored-by: Svarog <feanorr@gmail.com>
2021-05-19 10:03:23 +03:00
Ori Newman
268906a7ce Add VirtualDaaScore to GetBlockDagInfo (#1719)
Co-authored-by: Svarog <feanorr@gmail.com>

(cherry picked from commit 36c56f73bf7b41528c81013d5d0d3bc7d7f6a206)
2021-05-18 17:07:07 +03:00
Ori Newman
fa16c30cf3
Implement bip32 (#1676)
* Implement bip32

* Unite private and public extended keys

* Change variable names

* Change test name and add comment

* Rename var name

* Rename ckd.go to child_key_derivation.go

* Rename ser32 -> serializeUint32

* Add PrivateKey method

* Rename Path -> DeriveFromPath

* Add comment to validateChecksum

* Remove redundant condition from parsePath

* Rename Fingerprint->ParentFingerprint

* Merge hardened and non-hardened paths in calcI

* Change fingerPrintFromPoint to method

* Move hash160 to hash.go

* Fix a bug in calcI

* Simplify doubleSha256

* Remove slice end bound

* Split long line

* Change KaspaMainnetPrivate/public to represent kprv/kpub

* Add comments

* Fix comment

* Copy base58 library to kaspad

* Add versions for all networks

* Change versions to hex

* Add comments

Co-authored-by: Svarog <feanorr@gmail.com>
Co-authored-by: Elichai Turkel <elichai.turkel@gmail.com>
2021-04-28 15:27:16 +03:00
stasatdaglabs
1f56a68a28
Add an RPC command: EstimateNetworkHashesPerSecond (#1686)
* Implement EstimateNetworkHashesPerSecond.

* Fix failing tests.

* Add request/response messages to the .proto files.

* Add the EstimateNetworkHashesPerSecond RPC command.

* Add the EstimateNetworkHashesPerSecond RPC client function.

* Add the EstimateNetworkHashesPerSecond RPC command to kaspactl.

* Disallow windowSize lesser than 2.

* Fix wrong scale (milliseconds instead of seconds).

* Handle windowHashes being 0.
2021-04-22 15:18:21 +03:00
Svarog
28bfc0fb9c
Move pow package from model to utils (#1681) 2021-04-19 15:35:36 +03:00
Elichai Turkel
83beae4463
Add consensus.Config as a wrapper for dagParams (#1680)
* Add a new consensus.Config wrapper to dagParams

* Update all tests to use consensus.Config
2021-04-19 09:07:34 +03:00
Svarog
a8a7e3dd9b
Add windows to the CI + fix errors when testing on Windows (#1674)
* Add windows to the CI

* Cast syscall.Stdin into an integer

* DataDir -> AppDir in service_windows.go

* Rename mempool-limits package to something non-main

* Close database after re-assigining to it

* Up rpcTimout to 10 seconds
2021-04-12 14:53:34 +03:00
Svarog
3f193e9219
Use a separate folder for every network under ~/.kaspawallet (#1673) 2021-04-11 18:07:21 +03:00
Ori Newman
d2cccd2829
Add ECDSA support to the wallet (#1664)
* Add ECDSA support to the wallet

* Fix genkeypair

* Fix typo and rename var
2021-04-06 17:25:09 +03:00
Ori Newman
6dd3d4a9e7
Add dump unencrypted data sub command to the wallet (#1661) 2021-04-06 12:29:13 +03:00
stasatdaglabs
73b36f12f0
Implement importing private keys into the wallet (#1655)
* Implement importing private keys into the wallet.

* Fix bad --import default.

* Fix typo in --import annotation.

* Make go lint happy.

* Make go lint happier.

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-04-05 18:10:33 +03:00
Ori Newman
6afc06ce58
Replace p2pkh with p2pk (#1650)
* Replace p2pkh with p2pk

* Fix tests

* Fix comments and variable names

* Add README.md for genkeypair

* Rename pubkey->publicKey

* Rename p2pkh to p2pk

* Use util.PublicKeySize where needed

* Remove redundant pointer

* Fix comment

* Rename pubKey->publicKey
2021-04-05 14:35:34 +03:00
stasatdaglabs
d01a213f3d
Add a show-address subcommand to kaspawallet (#1653)
* Add a show-address subcommand to kaspawallet.

* Update the description of the key-file command line parameter.
2021-04-05 14:22:03 +03:00
stasatdaglabs
7ad8ce521c
Implement reconnection logic within the RPC client (#1643)
* Add a reconnect mechanism to RPCClient.

* Fix Reconnect().

* Connect the internal reconnection logic to the miner reconnection logic.

* Rename shouldReconnect to isClosed.

* Move safe reconnection logic from the miner to rpcclient.

* Remove sleep from HandleSubmitBlock.

* Properly handle client errors and only disconnect if we're already connected.

* Make go lint happy.

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-04-05 13:57:28 +03:00
Ori Newman
86ba80a091
Improve wallet functionality (#1636)
* Add basic wallet library

* Add CLI

* Add multisig support

* Add persistence to wallet

* Add tests

* go mod tidy

* Fix lint errors

* Fix wallet send command

* Always use the password as byte slice

* Remove redundant empty string

* Use different salt per private key

* Don't sign a signed transaction

* Add comment

* Remove old wallet

* Change directory permissions

* Use NormalizeRPCServerAddress

* Fix compilation errors
2021-03-31 15:58:22 +03:00
Ori Newman
af10b59181
Use go-secp256k1 v0.0.5 (#1640) 2021-03-30 18:01:56 +03:00
stasatdaglabs
c5b0394bbc
In RPC, use RPCTransactions and RPCBlocks instead of TransactionMessages and BlockMessages (#1609)
* Replace BlockMessage with RpcBlock in rpc.proto.

* Convert everything in kaspad to use RPCBlocks and fix tests.

* Fix compilation errors in stability tests and the miner.

* Update TransactionVerboseData in rpc.proto.

* Update TransactionVerboseData in the rest of kaspad.

* Make golint happy.

* Include RpcTransactionVerboseData in RpcTransaction instead of the other way around.

* Regenerate rpc.pb.go after merge.

* Update appmessage types.

* Update appmessage request and response types.

* Reimplement conversion functions between appmessage.RPCTransaction and protowire.RpcTransaction.

* Extract RpcBlockHeader toAppMessage/fromAppMessage out of RpcBlock.

* Fix compilation errors in getBlock, getBlocks, and submitBlock.

* Fix compilation errors in getMempoolEntry.

* Fix compilation errors in notifyBlockAdded.

* Update verbosedata.go.

* Fix compilation errors in getBlock and getBlocks.

* Fix compilation errors in getBlocks tests.

* Fix conversions between getBlocks message types.

* Fix integration tests.

* Fix a comment.

* Add selectedParent to the verbose block response.
2021-03-30 17:43:02 +03:00
Svarog
cebcab7f5c
Implement BIP-143-like sighash (#1598)
* Move CalculateSignatureHash to consensushashing

* Added CalcSignatureHash_BIP143 with all parameters except the re-used hashes

* Add handling of outputHash

* Add sequencesHash to the mix

* Add previousOutputsHash to the mix

* Replace legacy CalculateSigHash with new one, and re-wire to all non-test code

* Add missing types to WriteElement

* Fix tests in txscript

* Fix tests in rest of code

* Add missing comments

* Add SubnetworkID and Gas to sigHash

* Add TestCalculateSignatureHash

* Invert condition in SigHashSingle getOutputsHash

* Explicitly define that payloadHash for native transactions is 0

* added benchmark to CalculateSignatureHash

* Reformat call for signAndCheck

* Change SigHashes to be true bit-fields

* Add check for transaction version

* Write length of byte array in WriteElement

* hashOutpoint should get outpoint, not txIn

* Use inputIndex instead of i to determine SigHashType

* Use correct transaction version + fix some typos

* Fix hashes in test

* Reformat an overly-long line

* Replace checkHashTypeEncoding with caalls to hashType.IsStandardSigHashType

* Convert hashType to uint8

* Add comment
2021-03-17 15:17:38 +02:00
Elichai Turkel
caf251b7a8
Replace the HomeDir flag with a AppDir flag (#1615) 2021-03-17 12:48:38 +02:00
Ori Newman
d9449a32b8
Use DAA score where needed (#1602)
* Replace blue score with DAA score in UTXO entries

* Use DAA score for coinbase maturity

* Use DAA score for sequence lock

* Fix calcBlockSubsidy to use DAA score

* Don't pay to blocks that are not included in the DAA added blocks, and bestow red blocks reward to the merging block

* Fix TestGetPruningPointUTXOs

* Fix TestTransactionAcceptance

* Fix TestChainedTransactions

* Fix TestVirtualDiff

* Fix TestBlockWindow

* Fix TestPruning

* Use NewFromSlice instead of manually creating the hash set

* Add assert

* Add comment

* Remove redundant call to UpdateDAADataAndReturnDifficultyBits

* Add RequiredDifficulty, rename UpdateDAADataAndReturnDifficultyBits to StageDAADataAndReturnRequiredDifficulty and add comments

* Make buildUTXOInvalidHeader get bits as an argument

* Fix comments
2021-03-15 13:48:40 +02:00
Ori Newman
53781eed4d
Remove payload hash (#1583)
* Remove payload hash

* Fix tests
2021-03-08 15:15:03 +02:00
Elichai Turkel
837fa65735
kaspaminer: User tickers and regulate each block individually (#1580)
* User tickers and regulate each block individually

* Add comments, logs and rename variables

* Fix review comments
2021-03-08 10:51:35 +02:00
Svarog
df19bdfaf3
Use EmitUnpopulated so that kaspactl prints all fields, even the default ones (#1561)
Co-authored-by: Elichai Turkel <elichai.turkel@gmail.com>
2021-03-01 14:47:34 +02:00
Elichai Turkel
1f69f9eed9
Cleanup the logger and make it asynchronous (#1524)
* Remove Subsystems map and replace with RegisterSubSystem

* Clean up the logger

* Fix LOGFLAGS and make LongFile work correctly

* Parallelize the logger backend

* More logger cleanup

* Initialize and close the logger backend wherever it's needed

* Move the location where the backend is closed, also print the log if it panics while writing

* Add TestMain to reachability manager tests to preserve the same log level

* Fix review comments

Co-authored-by: Svarog <feanorr@gmail.com>
2021-03-01 14:04:40 +02:00
Ori Newman
201646282b
Fix the target block rate to create less bursty mining (#1554)
Co-authored-by: Svarog <feanorr@gmail.com>
2021-02-24 16:35:38 +02:00
Ori Newman
581a12db96
Add RPC reconnection to the miner (#1552)
* Add RPC reconnection to the miner

* Fix wrapf

* Change logs
2021-02-24 10:25:13 +02:00
Svarog
8fbea5d239
Increase the sleep time in kaspaminer when the node is not synced (#1544)
Co-authored-by: stasatdaglabs <39559713+stasatdaglabs@users.noreply.github.com>
2021-02-21 11:46:06 +02:00
Svarog
5fa06fe7d7
Upgrade everything to go1.16 (#1539)
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-02-21 09:17:21 +02:00
Elichai Turkel
19878aa062
Make templateManager hold a DomainBlock and isSynced bool instead of a GetBlockTemplateResponseMessage (#1538) 2021-02-18 00:59:11 +02:00
Ori Newman
8225f7fb3c
Add GetInfo RPC command (#1504)
* Add GetInfo RPC command

* Rename ID to p2p ID
2021-02-08 16:33:21 +02:00
stasatdaglabs
44280b9006
Require the --miningaddr parameter in kaspaminer. (#1482)
Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-02-04 09:42:02 +02:00
Ori Newman
ee8fa32ff8
Refactor miner and mine when waiting for block to validate (#1481)
* Refactor miner and mine when waiting for block to validate

* Fix -n to work after the refactor.
Change foundBlockChan capacity.
Use lock instead of atomic in the template manager.

* Fix self assignment

* Fix lock

* Fix Dockerfile

* Add comment
2021-02-03 11:53:55 +02:00
Ori Newman
b636ae234e
Add ban and unban RPC commands (#1478)
* Add ban and unban RPC commands

* Fix names

* Fix commands strings

* Update RPC documentation

* Rename functions

* Simplify return

* Use IP strings in app messages

* Add parse IP error

* Fix wrong condition
2021-02-01 17:34:43 +02:00
Elichai Turkel
280fa3de46
Prevent infinite ticker leaks in kaspaminer (#1476)
* Prevent infinite tickers leaks in kaspaminer

* Reset ticker in ConnectionManager instead of allocating a new one

Co-authored-by: Ori Newman <orinewman1@gmail.com>
2021-02-01 14:52:17 +02:00
Elichai Turkel
d281dabdb4
Bump Go version to 1.15 (#1477) 2021-02-01 14:35:11 +02:00
Ori Newman
331042edf1
Add defaultTargetBlocksPerSecond (#1473)
* Add defaultTargetBlocksPerSecond

* Use different default per network
2021-02-01 14:26:45 +02:00