3449 Commits

Author SHA1 Message Date
Ori Newman
7fc2430ab1 [NOD-304] Implement get transactions by address for api server (#395)
* [NOD-303] Implement get transactions by address for API server

* [NOD-304] Implement get transactions by address

* [NOD-304] Implement get transactions by address

* [NOD-304] Use structs for where if possible

* [NOD-304] Auto increment IDs

* [NOD-304] Make defaultGetTransactionsLimit constant

* [NOD-304] Delete db directory

* [NOD-304] change db var to query

* [NOD-304] Extract route handle function from addRoutes

* [NOD-304] Order transactions by ID

* [NOD-304] Add error for passing arrays to GET
2019-09-08 10:52:03 +03:00
Ori Newman
cf9af0fb5d [NOD-320] Make txgen explicitly skip mempool transactions (#396) 2019-09-05 16:14:55 +03:00
Ori Newman
db6d6293c7 [NOD-319] Add query params to api server route handler (#394) 2019-09-04 17:34:36 +03:00
Ori Newman
ae25ec2e6b [NOD-303] Implement get transaction by id for api server (#391)
* [NOD-303] Implement get transaction by id for api server

* [NOD-303] Make routeParamTxID a constant

* [NOD-303] Change database is not current error.

* [NOD-303] Add ID to TransactionInput and TransactionOutput models

* [NOD-303] Change transactions_outputs table name to transaction_outputs and transactions_inputs to transaction_inputs

* [NOD-303] Add json annotations to transaction response types

* [NOD-303] Split server package

* [NOD-303] Add GetTransactionByHashHandler

* [NOD-303] Add comments to exported functions and variables

* [NOD-303] Put response types in a separate file

* [NOD-303] Rename functions
2019-09-03 15:54:59 +03:00
Svarog
7521545682 [NOD-311] Removed JSONRPCfyer (#392) 2019-09-03 11:24:25 +03:00
Ori Newman
169e96e851 [NOD-310] Implement REST server in API server (#389)
* [NOD-310] Implement REST server in API server

* [NOD-310] MetaData -> Metadata

* [NOD-310] Make custom context methods instead of custom request functions

* [NOD-310] change "Request ID" prefix to "RID" and convert to apiServerContext with newAPIServerContext everywhere
2019-09-01 17:03:43 +03:00
Ori Newman
893b8a88c8 [NOD-312] Change defaultMinRelayTxFee to 1 satoshi per byte (#390) 2019-09-01 15:21:24 +03:00
Svarog
c60711ab15 [NOD-302] Accept ConnectionEstablished when Pending is expected in TestRetryPermanent (#388) 2019-08-29 15:27:03 +03:00
Ori Newman
1b00e01030 [NOD-301] Don't sync with peer if the rendezvous point is below finality (#387)
* [NOD-301] Don't sync with peer if the rendezvous point is below finality

* [NOD-301] Add block hash and peer address for the warn message

* [NOD-301] Fix perrLog.Warnf arguments order
2019-08-29 10:47:05 +03:00
Ori Newman
f0c80905eb [NOD-300] If node has invalid ancestor set the according status in blockindex (#386)
* [NOD-300] If node has invalid ancestor set the according status in blockindex

* [NOD-300] Test that status is also updated for grand child of an invalid block

* [NOD-300] change make(blockSet) to newSet()
2019-08-28 17:44:33 +03:00
stasatdaglabs
b07a118431 [NOD-292] When writing block to database - also create record in block index (#376)
* [NOD-292] In accept.go, made dbStoreBlock and flushToDB occur within the same transaction.

* [NOD-292] Implemented processing blocks that were not validated on BTCD start.

* [NOD-292] Fixed processing logic on init. Added a test for it.

* [NOD-292] Fixed some comments.

* [NOD-292] Made unlocks deferred in a couple of places.

* [NOD-292] Made unprocessed block reprocess via ProcessBlock rather than maybeAcceptBlock.

* [NOD-292] Fixed grammar in comment. Added an explanation to TestAcceptingInInit.

* [NOD-292] Split flushToDB into two versions.

* [NOD-292] Fixed a bad assignment.

* [NOD-292] Fixed bad spacing.
2019-08-28 14:52:57 +03:00
Ori Newman
0ae06cd277 [NOD-297] fix onchainchanged on rpcclient (#384)
* [NOD-297] Fix onChainChanged on rpcclient

* [NOD-285] create gorm models for db (#378)

* [NOD-285] Map API-Server database using GORM

* [NOD-285] Add accepting block to transactions and blocks models, and remove accepting block model

* [NOD-285] Define model relations

* [NOD-285] Fix many to many for Transaction and Block models

* [NOD-285] Remove redundant main file

* [NOD-296] Send SyncMgr.SubmitBlock errors as rpc errors (#381)

* [NOD-296] Send SyncMgr.SubmitBlock errors as rpc errors

* [NOD-296] Add error message prefix

* [NOD-298] Add comments to gorm models (#382)

* [NOD-294] Fix golint in deploy.sh and fix all lint warnings (#380)

* [NOD-294] Fix golint in deploy.sh and fixed all lint errors

* [NOD-294] Fix typos in comments

* [NOD-294] Convert VirtualForTest into alias of *virtualBlock

* [NOD-294] Fixed some more typos in comments

* [NOD-295] Limit the length of GetData to 50 (#383)

* [NOD-295] Fixed bad break condition in addInvsToGetDataMessageFromQueue.

* [NOD-295] Fixed the fix for bad break condition in addInvsToGetDataMessageFromQueue.

* [NOD-295] Made the check for max invs refer to invsNum instead of MaxInvPerGetDataMsg.

* [NOD-297] Fix onChainChanged on rpcclient

* [NOD-286] Implement API-Server base structure (#379)

* [NOD-286] Implement API-Server base structure

* [NOD-286] Add rpc user and password as command line arguments

* [NOD-286] Make log directory a CLI argument

* [NOD-286] Add db login details as CLI arguments

* [NOD-297] Fix onChainChanged on rpcclient and server

* [NOD-297] Fix variables and functions names

* [NOD-297] Fix AcceptedTxIds -> AcceptedTxIDs
2019-08-28 12:52:07 +03:00
Ori Newman
ed9165f533 [NOD-286] Implement API-Server base structure (#379)
* [NOD-286] Implement API-Server base structure

* [NOD-286] Add rpc user and password as command line arguments

* [NOD-286] Make log directory a CLI argument

* [NOD-286] Add db login details as CLI arguments
2019-08-27 16:19:01 +03:00
stasatdaglabs
c73113a12e [NOD-295] Limit the length of GetData to 50 (#383)
* [NOD-295] Fixed bad break condition in addInvsToGetDataMessageFromQueue.

* [NOD-295] Fixed the fix for bad break condition in addInvsToGetDataMessageFromQueue.

* [NOD-295] Made the check for max invs refer to invsNum instead of MaxInvPerGetDataMsg.
2019-08-27 13:09:36 +03:00
Svarog
480b2ca07c [NOD-294] Fix golint in deploy.sh and fix all lint warnings (#380)
* [NOD-294] Fix golint in deploy.sh and fixed all lint errors

* [NOD-294] Fix typos in comments

* [NOD-294] Convert VirtualForTest into alias of *virtualBlock

* [NOD-294] Fixed some more typos in comments
2019-08-27 12:00:23 +03:00
Ori Newman
c72b914050 [NOD-298] Add comments to gorm models (#382) 2019-08-27 11:48:43 +03:00
Ori Newman
5cf7f01d3f [NOD-296] Send SyncMgr.SubmitBlock errors as rpc errors (#381)
* [NOD-296] Send SyncMgr.SubmitBlock errors as rpc errors

* [NOD-296] Add error message prefix
2019-08-27 11:25:45 +03:00
Ori Newman
552a5917c2 [NOD-285] create gorm models for db (#378)
* [NOD-285] Map API-Server database using GORM

* [NOD-285] Add accepting block to transactions and blocks models, and remove accepting block model

* [NOD-285] Define model relations

* [NOD-285] Fix many to many for Transaction and Block models

* [NOD-285] Remove redundant main file
2019-08-27 11:25:07 +03:00
stasatdaglabs
5c14719f14 [NOD-295] Capped amount of invs in a getData message. (#377) 2019-08-26 15:12:55 +03:00
Svarog
d2353a189a [NOD-291] Remove database check from dag.BlockExists (#375) 2019-08-25 15:00:16 +03:00
Ori Newman
4fcd705ae3 [NOD-290] Remove block_id from transactions table (#374)
* [NOD-290] Remove block_id from transactions table

* [NOD-290] Remove block_id foreign key from transactions table
2019-08-25 10:23:03 +03:00
Ori Newman
744c17b4c8 [NOD-280] Create database for API server (#373)
* [NOD-280] Create database for API server

* [NOD-280] Rename public_key_script to pk_script

* [NOD-280] Change indexes names

* [NOD-280] Add accepting block to blocks and transactions table and remove accepting_blocks table

* [NOD-280] Add readme

* [NOD-280] Change VARCHAR(32) to CHAR(64)

* [NOD-280] Rename location_in_block to index
2019-08-22 17:21:09 +03:00
stasatdaglabs
e2eca24b33 [NOD-282] Fixed Telegram messages not being sent. (#371) 2019-08-22 13:41:48 +03:00
stasatdaglabs
36d5ac189f [NOD-283] Fixed a crash in notifyChainChanged. (#372) 2019-08-22 13:23:30 +03:00
stasatdaglabs
1a569c7bd7 [NOD-270] Implement NotifyChainUpdates api call (#368)
* [NOD-270] Added notifyChainChanges and related commands.

* [NOD-270] Added NTChainChanged to blockdag.

* [NOD-270] Implemented collection and sending of ChainChanged notifications.

* [NOD-270] Fixed an improperly named test.

* [NOD-270] Added a test: TestChainChangedNotification.

* [NOD-270] Fixed a couple copy+paste errors.

* [NOD-270] Added a couple of comments for TestChainChangedNotification.

* [NOD-270] Fixed formatting error.

* [NOD-270] Fixed a comment.

* [NOD-270] Uncoupled chain updates inside blockdag from the concept of a notification.

* [NOD-270] Removed intermediary ChainUpdates object from ChainChangedNotificationData.
2019-08-21 12:58:32 +03:00
Ori Newman
6bb53eaae3 [NOD-256] add error log (#369)
* [NOD-256] Add error log

* [NOD-256] Add error log

* [NOD-256] Fix typo and comment

* [NOD-256] Remove btclog dir

* [NOD-256] Format project

* [NOD-256] Add error log files

* [NOD-256] Add an option to add a log file to write into to an existing backend logger

* [NOD-256] Get rid of redundant logs initialization

* [NOD-256] rename initLogRotators to initLog

* [NOD-256] Get rid ExampleSignTxOutput and convert ExampleBlockDAG_ProcessBlock to a regular test

* [NOD-256] Show error message if os.Exiting from initLog
2019-08-21 11:26:21 +03:00
Svarog
747a9bb944 [NOD-278] Added default for MinRelayTxFee (#367) 2019-08-19 17:52:58 +03:00
Ori Newman
d2daf334a5 [NOD-241] Implement lower resolution peer rendezvous point discovery (#353)
* [NOD-241] Implement lower resolution peer rendezvous point discovery

* [NOD-241] Implement lower resolution peer rendezvous point discovery

* [NOD-241] Find exact rendezvous point

* [NOD-241] Find exact rendezvous point

* [NOD-241] Fix tests

* [NOD-241] Remove hash stop from MsgBlockLocator and add tests to MsgBlockLocator and MsgGetBlockLocator

* [NOD-241] Change everywhere startHash to hashStart and change comments

* [NOD-241] Fix locateBlockNodes to stop at hashStop

* [NOD-241] Formatted locatorSummary.

* [NOD-241] Fix node reversal

* [NOD-241] Fix hash start and hash stop order, and don't include startNode in dag.blockLocator

* [NOD-241] rename locateBlockNodes -> getBlueBlocksBetween and add a comment to it

* [NOD-241] change hash start to start hash and hash stop to stop hash

* [NOD-241] Move block locator stuff to a different file

* [NOD-241] Rename msggetblocks.go to msggetblockinvs.go

* [NOD-241] Format project

* [NOD-241] Rename rpcserverSyncManager.LocateHeaders to GetBlueBlocksHeadersBetween

* [NOD-241] Move the logic of finding the highest shared block to OnBlockLocator

* [NOD-241] Rename chainHeight -> nextChainHeight

* [NOD-241] Fix typo in comment
2019-08-19 15:35:13 +03:00
stasatdaglabs
70737e4e94 [NOD-264] Implement tx-selection algorithm (#358)
* [NOD-264] Implemented calcTxSelectionValue.

* [NOD-264] Fixed bad subnetworkID in calcTxSelectionValue.

* [NOD-264] Implemented sorting the txDescs by value.

* [NOD-264] Got rid of txPrioItem.

* [NOD-264] Moved transaction selection to a separate file.

* [NOD-264] Renamed the result object to txsForBlockTemplate.

* [NOD-264] Implemented tx selection.

* [NOD-264] Fixed trying to get the gas limit for built-in subnetworks.

* [NOD-264] Wrote comments where appropriate.

* [NOD-264] Moved calcTxSelectionValue to the mining package. (Non-mining nodes shouldn't be forced to calc selection value for every transaction)

* [NOD-264] Wrote a test for selectTxs.

* [NOD-264] Fixed a comment.

* [NOD-264] Fixed misunderstood test.

* [NOD-264] Added zero fee check. Added a couple more tests.

* [NOD-264] Added probabilistic tests. Fixed a couple of bugs in tx selection.

* [NOD-264] Fixed tests with missing fees.

* [NOD-264] Added a test over a range of txs with different gas/mass.

* [NOD-264] Added expected probability to the rest of the test cases.

* [NOD-264] Tightened bounds in probability test.

* [NOD-264] Fixed values in probabily test.

* [NOD-264] Added a comments for alpha and rebalanceThreshold.

* [NOD-264] Fixed a couple of comments, renamed result to txsForBlockTemplate.

* [NOD-264] Removed an irrelevant comment. Changed Tracef to Warnf in some logs.

* [NOD-264] Renamed selectionValue -> txValue.

* [NOD-264] Moved rebalancing to the start of the tx selection loop.

* [NOD-264] Added overflow check for gasUsage.

* [NOD-264] Renamed blockSigOps and blockMass to totalSigOps and totalMass.

* [NOD-264] Removed the need to pass usedCount to reblanaceCandidates. Also relaxed bounds in a test.

* [NOD-264] Split selectTxs into smaller functions. Also relaxed bounds in a test some more.

* [NOD-264] Added a comment for findTx.

* [NOD-264] Ordered candidateTxs by subnetwork instead of txValue.

* [NOD-264] Disallowed zero tx fees in mempool and config. Renamed iterateCandidateTxs to populateTemplateFromCandidates.

* [NOD-264] Changed isFinalizedTransaction log level from Warn to Debug.

* [NOD-264] Removed references to SigOps in txSelection.

* [NOD-264] Removed SigOps validation. Validating mass should suffice.

* [NOD-264] Renamed wasUsed to isMarkedForDeletion.

* [NOD-264] Renamed markCandidateTxUsed to markCandidateTxForDeletion.

* [NOD-264] Made some probabilistic tests less likely to fail when they shouldn't.

* [NOD-264] Added a message warning people about probabilistic tests.

* [NOD-264] Rephrased a comment about rebalanceThreshold.

* [NOD-264] Removed IsCoinBase, CheckTransactionInputsAndCalulateFee, and ValidateTransactionScripts from txSelection.

* [NOD-264] Removed a condition that is no longer relevant.

* [NOD-264] "which's" -> "whose"

* [NOD-264] Removed wasteful preallocations.

* [NOD-264] Fixed a comment referring to "used" transactions.
2019-08-19 12:08:48 +03:00
stasatdaglabs
5f49115cac [NOD-269] Implement GetChainFromBlock api-call (#364)
* [NOD-269] Added a skeleton for getChainFromBlock.

* [NOD-269] Made startHash and includeBlocks optional.

* [NOD-269] Implemented chainBlock collection.

* [NOD-269] Extracted GetBlockVerboseResult building to its own method.

* [NOD-269] Implemented the IncludeBlocks part of GetChainFromBlock.

* [NOD-269] Added a comment for NewGetChainFromBlockCmd.

* [NOD-269] Made IsInSelectedPathChain return an error.

* [NOD-269] Fixed a very wrong comment.

* [NOD-269] Made SelectedPathChain allocate only the required amount of space.

* [NOD-269] Renamed pathChain to parentChain.

* [NOD-269] Split handleGetChainFromBlock to separate functions.

* [NOD-269] Fixed some grammar.
2019-08-18 13:31:54 +03:00
stasatdaglabs
534cb2bf5b [NOD-272] In CheckTransactionSanity, multiply mass limit check by massPerTxByte. (#365) 2019-08-15 15:05:26 +03:00
stasatdaglabs
187c525667 [NOD-234] In getBlockTemplate check if node is current (#362)
* [NOD-234] Added an IsCurrent check to handleGetBlockTemplate.

* [NOD-234] Removed IsCurrent check from handleGetBlockTemplateRequest. Added an explanation for why we're checking the chainHeight.

* [NOD-234] Added ShouldMineOnGenesis to the IsCurrent check.

* [NOD-234] Flipped && operands to fail fast.
2019-08-14 15:09:35 +03:00
stasatdaglabs
6032727965 [NOD-268] Implement selectedParentChain-related structures in btcjson. (#363) 2019-08-13 11:15:51 +03:00
stasatdaglabs
bb3f23b6dc [NOD-240] Get rid of all references for wallet related RPC commands (#361)
* [NOD-240] Removed references to the wallet in rpcserver.go.

* [NOD-240] Began removing btcwalletxxx.go.

* [NOD-240] Got rid of rpcclient/wallet.go and walletsvrcmds.go.

* [NOD-240] Moved GetBestBlockResult to dagsvrresults.go.

* [NOD-240] Finished removing walletsvrXXX.go.

* [NOD-240] Removed wallet stuff from btcctl.

* [NOD-240] Removed a few last things that I've missed.
2019-08-12 09:54:07 +03:00
stasatdaglabs
e5485ac5e6
[NOD-262] Renamed all instances of GetBlocks to GetBlockInvs. (#359) 2019-08-11 12:25:29 +03:00
stasatdaglabs
594a209f83 [NOD-263] Rename all instances of hashStop to stopHash. (#360) 2019-08-11 11:20:21 +03:00
Svarog
9981ce7adb [NOD-255] When adding orphans during netsync process - report only to debug log, unless number of orphans > k*2 (#357)
* [NOD-255] When orphan blocks arrive from netsync - don't write log unless we are in Debug

* [NOD-255] If there are more than K*2 orphans in pool - report as a potential problem anyway

* [NOD-255] Update comment to explain the K*2 figure
2019-08-07 11:30:32 +03:00
Svarog
49ac97c7db [NOD-265] Return an empty array for searchRawTransactions when no txs were found (#356) 2019-08-06 17:54:56 +03:00
stasatdaglabs
bfdf7a2cf2 [NOD-237] Implement transaction mass (#355)
* [NOD-237] Implemented transaction mass.

* [NOD-237] Added transaction mass validation to the mempool.

* [NOD-237] Made blockMaxMassMax not rely on MaxBlockPayload.

* [NOD-237] Added comments describing the new constants in validate.go.

* [NOD-237] Changed the default blockmaxmass to 10,000,000.

* [NOD-237] Fixed a comment that erroneously didn't refer to mass.

* [NOD-237] Added comments to ValidateTxMass and CalcTxMass.

* [NOD-237] Renamed "size" to "byte". Made validateBlockMass exit early if validation fails. Fixed unit names in comments. In CalcTxMass, moved summing of mass to the bottom of the function.

* [NOD-237] Instead of ErrMassTooHigh, renamed ErrBlockTooBig and ErrTxTooBig. Replaced wire.MaxBlockPayload with MaxMassPerBlock.

* [NOD-237] Fixed sanity checks related to block size in commands.

* [NOD-237] To use up less memory during testing, made the mass in the "too big" test come from pkScripts rather than input bytes.

* [NOD-237] Added an overflow check to validateBlockMass.
2019-08-05 16:04:24 +03:00
Ori Newman
54b681460d [NOD-244] Don't validate subnetwork registry transactions when they are being registered (#354) 2019-08-01 12:40:01 +03:00
stasatdaglabs
2147d16c1f [NOD-220] When handling an INV message, made it skip tx invs that are currently being requested. (#352) 2019-08-01 12:31:40 +03:00
Ori Newman
7c1cb47bd0 [NOD-249] Change WaitGroup to use channels (#350)
* [NOD-248] Implement waitgroup to enable waiting while adding

* [NOD-248] fix waitGroup.done() error message

* [NOD-248] atomically read wg.counter

* [NOD-248] return lowPriorityMutex

* [NOD-249] Add tests to waitgroup

* [NOD-249] Change waitgroup to use channels

* [NOD-249] Format project

* [NOD-249] Add comments and logs to waitGroup, and remove timeouts from
prioritymutex_test.go

* [NOD-249] Fix comments
2019-07-28 18:23:26 +03:00
stasatdaglabs
6acfa18d7c [NOD-220] Fixed handleSearchRawTransactions trying to get confirmations of mempool transactions, which made txgen crash on start. (#351) 2019-07-28 13:02:16 +03:00
Svarog
f0a675162c [NOD-253] Use spawn instead of go (#349) 2019-07-22 12:06:13 +03:00
Svarog
7a4deb6f18 [NOD-251] Use bluest parent anywhere validating difficulty, instead of selectedTip (#348) 2019-07-17 14:46:58 +03:00
Svarog
96842353de [NOD-250] Calculate UTXOCommitment when loading DAG + add UTXOCommitment to getBlockDagInfo (#347) 2019-07-16 16:53:28 +03:00
Svarog
5ce8875ce0 [NOD-243] Optimize validateGasLimit (#346)
* [NOD-243] Optimize validateGasLimit to only remember the current subnetwork gasUsage

* [NOD-243] Fix a typo

* [NOD-243] Rephrased comment
2019-07-16 11:07:58 +03:00
Ori Newman
812819e92f [NOD-248] Implement waitgroup to enable waiting while adding (#345)
* [NOD-248] Implement waitgroup to enable waiting while adding

* [NOD-248] fix waitGroup.done() error message

* [NOD-248] atomically read wg.counter

* [NOD-248] return lowPriorityMutex
2019-07-14 18:50:09 +03:00
Ori Newman
5cb536643e [NOD-236] Add secondary address to txgen (#341)
* [NOD-236] Add secondary address to txgen

* [NOD-236] Add description to secondary address cli argument

* [NOD-236] Remove unnecessary empty line
2019-07-14 11:26:32 +03:00
Ori Newman
4c6b8969d3 [NOD-245] Increase MaxInvPerMsg and MaxBlocksPerMsg to 65536 (#343)
* [NOD-245] Increase MaxInvPerMsg and MaxBlocksPerMsg to 65536

* [NOD-245] Fix MaxInvPerMsg to 1 << 16
2019-07-14 10:47:41 +03:00