3463 Commits

Author SHA1 Message Date
Ori Newman
a036618b44 [NOD-324] Properly handle GORM errors in API server (#409)
* [NOD-324] Properly handle GORM errors in API server

* [NOD-324] Handle RecordNotFound error in GetBlockByHashHandler

* [NOD-324] Make a separate function for NewErrorFromDBErrors
2019-09-18 14:09:07 +03:00
Ori Newman
2429b623fc [NOD-327] Add --migrate cli flag to API server (#407)
* [NOD-327] Add --migrate cli flag to API server

* [NOD-327] Change log messages

* [NOD-327] Remove `required` flag from API server RPC CLI arguments

* [NOD-327] Add database version in migrations logs
2019-09-18 13:51:20 +03:00
Ori Newman
f4850b9e7a [NOD-330] Use BTCD logs for gorm (#410) 2019-09-18 11:47:54 +03:00
Ori Newman
e81ac5f19e [NOD-307] Implement get blocks for api server (#405)
* [NOD-307] Implement API-Server GET /blocks

* [NOD-307] Implement API-Server GET /blocks

* [NOD-307] Add comments to exported constants

* [NOD-307] Flatten GET query values and check that 'order' value is valid

* [NOD-307] Validate order values in GetBlocksHandler

* [NOD-307] Add convertQueryParamToInt function
2019-09-16 16:53:57 +03:00
Ori Newman
31ccedf136 [NOD-325] Enable separate error messages for logging and client (#406)
* [NOD-325] Enable separate error messages for logging and client

* [NOD-325] Add json annotation to clientError
2019-09-16 13:26:05 +03:00
Svarog
502b510ccd [NOD-322] Minor api-server refactoring. (#401)
* [NOD-322] Make database.DB a function

* [NOD-322] Move context to be the first parameter in all functions

* [NOD-322] Set db to nil on database.Close()

* [NOD-322] Tidy go.mod/go.sum

* [NOD-322] Use http package const + message for StatusInternalServerError
2019-09-15 12:32:12 +03:00
stasatdaglabs
369031f963 [NOD-326] Replaced the UTXOs table with TransactionOutputs.isSpent (#404)
* [NOD-326] Replaced UTXO table with TransactionOutput.IsSpent.

* [NOD-326] Fixed merge errors.
2019-09-15 12:04:51 +03:00
Ori Newman
a789680db1 [NOD-314] change pkscript to scriptpubkey (#400)
* [NOD-314] Change everywhere PkScript to ScriptPubKey

* [NOD-314] Change everywhere PkScript to ScriptPubKey

* [NOD-314] Rename pkPops -> scriptPubKeyPops
2019-09-15 11:09:36 +03:00
Svarog
90bda69931 [NOD-323] Move rpc-client to separate package (#402) 2019-09-15 10:21:42 +03:00
Svarog
9647cb3e08 [NOD-318] Upgrade everything to Go1.13 (#393) 2019-09-09 15:57:31 +03:00
Ori Newman
79c9060909 [NOD-308] Implement API-Server GET /fee-estimates (#399) 2019-09-09 11:21:56 +03:00
Svarog
20206789e0 [NOD-299] Add waitgroup to wait for all spawns to complete before calling teardown (#385)
* [NOD-299] Add waitgroup to wait for all `spawn`s to complete before calling teardown

* [NOD-299] Restore spawn on teardown + mark spawn done in the correct thread
2019-09-09 11:02:31 +03:00
Ori Newman
1ddae35277 [NOD-305] Implement API-Server GET /utxos/{address} (#398)
* [NOD-305] Implement API-Server GET /utxos/{address}

* [NOD-305] Add accepting block blue score to the resulted utxo
2019-09-08 16:58:28 +03:00
Ori Newman
75a8c6459a [NOD-306] Implement API-Server GET /block/{hash} (#397)
* [NOD-306] Implement API-Server GET /block/{hash}

* [NOD-306] Validate that hash string is a valid hex

* [NOD-306] Unite invalid hash errors
2019-09-08 16:09:09 +03:00
Ori Newman
7fc2430ab1 [NOD-304] Implement get transactions by address for api server (#395)
* [NOD-303] Implement get transactions by address for API server

* [NOD-304] Implement get transactions by address

* [NOD-304] Implement get transactions by address

* [NOD-304] Use structs for where if possible

* [NOD-304] Auto increment IDs

* [NOD-304] Make defaultGetTransactionsLimit constant

* [NOD-304] Delete db directory

* [NOD-304] change db var to query

* [NOD-304] Extract route handle function from addRoutes

* [NOD-304] Order transactions by ID

* [NOD-304] Add error for passing arrays to GET
2019-09-08 10:52:03 +03:00
Ori Newman
cf9af0fb5d [NOD-320] Make txgen explicitly skip mempool transactions (#396) 2019-09-05 16:14:55 +03:00
Ori Newman
db6d6293c7 [NOD-319] Add query params to api server route handler (#394) 2019-09-04 17:34:36 +03:00
Ori Newman
ae25ec2e6b [NOD-303] Implement get transaction by id for api server (#391)
* [NOD-303] Implement get transaction by id for api server

* [NOD-303] Make routeParamTxID a constant

* [NOD-303] Change database is not current error.

* [NOD-303] Add ID to TransactionInput and TransactionOutput models

* [NOD-303] Change transactions_outputs table name to transaction_outputs and transactions_inputs to transaction_inputs

* [NOD-303] Add json annotations to transaction response types

* [NOD-303] Split server package

* [NOD-303] Add GetTransactionByHashHandler

* [NOD-303] Add comments to exported functions and variables

* [NOD-303] Put response types in a separate file

* [NOD-303] Rename functions
2019-09-03 15:54:59 +03:00
Svarog
7521545682 [NOD-311] Removed JSONRPCfyer (#392) 2019-09-03 11:24:25 +03:00
Ori Newman
169e96e851 [NOD-310] Implement REST server in API server (#389)
* [NOD-310] Implement REST server in API server

* [NOD-310] MetaData -> Metadata

* [NOD-310] Make custom context methods instead of custom request functions

* [NOD-310] change "Request ID" prefix to "RID" and convert to apiServerContext with newAPIServerContext everywhere
2019-09-01 17:03:43 +03:00
Ori Newman
893b8a88c8 [NOD-312] Change defaultMinRelayTxFee to 1 satoshi per byte (#390) 2019-09-01 15:21:24 +03:00
Svarog
c60711ab15 [NOD-302] Accept ConnectionEstablished when Pending is expected in TestRetryPermanent (#388) 2019-08-29 15:27:03 +03:00
Ori Newman
1b00e01030 [NOD-301] Don't sync with peer if the rendezvous point is below finality (#387)
* [NOD-301] Don't sync with peer if the rendezvous point is below finality

* [NOD-301] Add block hash and peer address for the warn message

* [NOD-301] Fix perrLog.Warnf arguments order
2019-08-29 10:47:05 +03:00
Ori Newman
f0c80905eb [NOD-300] If node has invalid ancestor set the according status in blockindex (#386)
* [NOD-300] If node has invalid ancestor set the according status in blockindex

* [NOD-300] Test that status is also updated for grand child of an invalid block

* [NOD-300] change make(blockSet) to newSet()
2019-08-28 17:44:33 +03:00
stasatdaglabs
b07a118431 [NOD-292] When writing block to database - also create record in block index (#376)
* [NOD-292] In accept.go, made dbStoreBlock and flushToDB occur within the same transaction.

* [NOD-292] Implemented processing blocks that were not validated on BTCD start.

* [NOD-292] Fixed processing logic on init. Added a test for it.

* [NOD-292] Fixed some comments.

* [NOD-292] Made unlocks deferred in a couple of places.

* [NOD-292] Made unprocessed block reprocess via ProcessBlock rather than maybeAcceptBlock.

* [NOD-292] Fixed grammar in comment. Added an explanation to TestAcceptingInInit.

* [NOD-292] Split flushToDB into two versions.

* [NOD-292] Fixed a bad assignment.

* [NOD-292] Fixed bad spacing.
2019-08-28 14:52:57 +03:00
Ori Newman
0ae06cd277 [NOD-297] fix onchainchanged on rpcclient (#384)
* [NOD-297] Fix onChainChanged on rpcclient

* [NOD-285] create gorm models for db (#378)

* [NOD-285] Map API-Server database using GORM

* [NOD-285] Add accepting block to transactions and blocks models, and remove accepting block model

* [NOD-285] Define model relations

* [NOD-285] Fix many to many for Transaction and Block models

* [NOD-285] Remove redundant main file

* [NOD-296] Send SyncMgr.SubmitBlock errors as rpc errors (#381)

* [NOD-296] Send SyncMgr.SubmitBlock errors as rpc errors

* [NOD-296] Add error message prefix

* [NOD-298] Add comments to gorm models (#382)

* [NOD-294] Fix golint in deploy.sh and fix all lint warnings (#380)

* [NOD-294] Fix golint in deploy.sh and fixed all lint errors

* [NOD-294] Fix typos in comments

* [NOD-294] Convert VirtualForTest into alias of *virtualBlock

* [NOD-294] Fixed some more typos in comments

* [NOD-295] Limit the length of GetData to 50 (#383)

* [NOD-295] Fixed bad break condition in addInvsToGetDataMessageFromQueue.

* [NOD-295] Fixed the fix for bad break condition in addInvsToGetDataMessageFromQueue.

* [NOD-295] Made the check for max invs refer to invsNum instead of MaxInvPerGetDataMsg.

* [NOD-297] Fix onChainChanged on rpcclient

* [NOD-286] Implement API-Server base structure (#379)

* [NOD-286] Implement API-Server base structure

* [NOD-286] Add rpc user and password as command line arguments

* [NOD-286] Make log directory a CLI argument

* [NOD-286] Add db login details as CLI arguments

* [NOD-297] Fix onChainChanged on rpcclient and server

* [NOD-297] Fix variables and functions names

* [NOD-297] Fix AcceptedTxIds -> AcceptedTxIDs
2019-08-28 12:52:07 +03:00
Ori Newman
ed9165f533 [NOD-286] Implement API-Server base structure (#379)
* [NOD-286] Implement API-Server base structure

* [NOD-286] Add rpc user and password as command line arguments

* [NOD-286] Make log directory a CLI argument

* [NOD-286] Add db login details as CLI arguments
2019-08-27 16:19:01 +03:00
stasatdaglabs
c73113a12e [NOD-295] Limit the length of GetData to 50 (#383)
* [NOD-295] Fixed bad break condition in addInvsToGetDataMessageFromQueue.

* [NOD-295] Fixed the fix for bad break condition in addInvsToGetDataMessageFromQueue.

* [NOD-295] Made the check for max invs refer to invsNum instead of MaxInvPerGetDataMsg.
2019-08-27 13:09:36 +03:00
Svarog
480b2ca07c [NOD-294] Fix golint in deploy.sh and fix all lint warnings (#380)
* [NOD-294] Fix golint in deploy.sh and fixed all lint errors

* [NOD-294] Fix typos in comments

* [NOD-294] Convert VirtualForTest into alias of *virtualBlock

* [NOD-294] Fixed some more typos in comments
2019-08-27 12:00:23 +03:00
Ori Newman
c72b914050 [NOD-298] Add comments to gorm models (#382) 2019-08-27 11:48:43 +03:00
Ori Newman
5cf7f01d3f [NOD-296] Send SyncMgr.SubmitBlock errors as rpc errors (#381)
* [NOD-296] Send SyncMgr.SubmitBlock errors as rpc errors

* [NOD-296] Add error message prefix
2019-08-27 11:25:45 +03:00
Ori Newman
552a5917c2 [NOD-285] create gorm models for db (#378)
* [NOD-285] Map API-Server database using GORM

* [NOD-285] Add accepting block to transactions and blocks models, and remove accepting block model

* [NOD-285] Define model relations

* [NOD-285] Fix many to many for Transaction and Block models

* [NOD-285] Remove redundant main file
2019-08-27 11:25:07 +03:00
stasatdaglabs
5c14719f14 [NOD-295] Capped amount of invs in a getData message. (#377) 2019-08-26 15:12:55 +03:00
Svarog
d2353a189a [NOD-291] Remove database check from dag.BlockExists (#375) 2019-08-25 15:00:16 +03:00
Ori Newman
4fcd705ae3 [NOD-290] Remove block_id from transactions table (#374)
* [NOD-290] Remove block_id from transactions table

* [NOD-290] Remove block_id foreign key from transactions table
2019-08-25 10:23:03 +03:00
Ori Newman
744c17b4c8 [NOD-280] Create database for API server (#373)
* [NOD-280] Create database for API server

* [NOD-280] Rename public_key_script to pk_script

* [NOD-280] Change indexes names

* [NOD-280] Add accepting block to blocks and transactions table and remove accepting_blocks table

* [NOD-280] Add readme

* [NOD-280] Change VARCHAR(32) to CHAR(64)

* [NOD-280] Rename location_in_block to index
2019-08-22 17:21:09 +03:00
stasatdaglabs
e2eca24b33 [NOD-282] Fixed Telegram messages not being sent. (#371) 2019-08-22 13:41:48 +03:00
stasatdaglabs
36d5ac189f [NOD-283] Fixed a crash in notifyChainChanged. (#372) 2019-08-22 13:23:30 +03:00
stasatdaglabs
1a569c7bd7 [NOD-270] Implement NotifyChainUpdates api call (#368)
* [NOD-270] Added notifyChainChanges and related commands.

* [NOD-270] Added NTChainChanged to blockdag.

* [NOD-270] Implemented collection and sending of ChainChanged notifications.

* [NOD-270] Fixed an improperly named test.

* [NOD-270] Added a test: TestChainChangedNotification.

* [NOD-270] Fixed a couple copy+paste errors.

* [NOD-270] Added a couple of comments for TestChainChangedNotification.

* [NOD-270] Fixed formatting error.

* [NOD-270] Fixed a comment.

* [NOD-270] Uncoupled chain updates inside blockdag from the concept of a notification.

* [NOD-270] Removed intermediary ChainUpdates object from ChainChangedNotificationData.
2019-08-21 12:58:32 +03:00
Ori Newman
6bb53eaae3 [NOD-256] add error log (#369)
* [NOD-256] Add error log

* [NOD-256] Add error log

* [NOD-256] Fix typo and comment

* [NOD-256] Remove btclog dir

* [NOD-256] Format project

* [NOD-256] Add error log files

* [NOD-256] Add an option to add a log file to write into to an existing backend logger

* [NOD-256] Get rid of redundant logs initialization

* [NOD-256] rename initLogRotators to initLog

* [NOD-256] Get rid ExampleSignTxOutput and convert ExampleBlockDAG_ProcessBlock to a regular test

* [NOD-256] Show error message if os.Exiting from initLog
2019-08-21 11:26:21 +03:00
Svarog
747a9bb944 [NOD-278] Added default for MinRelayTxFee (#367) 2019-08-19 17:52:58 +03:00
Ori Newman
d2daf334a5 [NOD-241] Implement lower resolution peer rendezvous point discovery (#353)
* [NOD-241] Implement lower resolution peer rendezvous point discovery

* [NOD-241] Implement lower resolution peer rendezvous point discovery

* [NOD-241] Find exact rendezvous point

* [NOD-241] Find exact rendezvous point

* [NOD-241] Fix tests

* [NOD-241] Remove hash stop from MsgBlockLocator and add tests to MsgBlockLocator and MsgGetBlockLocator

* [NOD-241] Change everywhere startHash to hashStart and change comments

* [NOD-241] Fix locateBlockNodes to stop at hashStop

* [NOD-241] Formatted locatorSummary.

* [NOD-241] Fix node reversal

* [NOD-241] Fix hash start and hash stop order, and don't include startNode in dag.blockLocator

* [NOD-241] rename locateBlockNodes -> getBlueBlocksBetween and add a comment to it

* [NOD-241] change hash start to start hash and hash stop to stop hash

* [NOD-241] Move block locator stuff to a different file

* [NOD-241] Rename msggetblocks.go to msggetblockinvs.go

* [NOD-241] Format project

* [NOD-241] Rename rpcserverSyncManager.LocateHeaders to GetBlueBlocksHeadersBetween

* [NOD-241] Move the logic of finding the highest shared block to OnBlockLocator

* [NOD-241] Rename chainHeight -> nextChainHeight

* [NOD-241] Fix typo in comment
2019-08-19 15:35:13 +03:00
stasatdaglabs
70737e4e94 [NOD-264] Implement tx-selection algorithm (#358)
* [NOD-264] Implemented calcTxSelectionValue.

* [NOD-264] Fixed bad subnetworkID in calcTxSelectionValue.

* [NOD-264] Implemented sorting the txDescs by value.

* [NOD-264] Got rid of txPrioItem.

* [NOD-264] Moved transaction selection to a separate file.

* [NOD-264] Renamed the result object to txsForBlockTemplate.

* [NOD-264] Implemented tx selection.

* [NOD-264] Fixed trying to get the gas limit for built-in subnetworks.

* [NOD-264] Wrote comments where appropriate.

* [NOD-264] Moved calcTxSelectionValue to the mining package. (Non-mining nodes shouldn't be forced to calc selection value for every transaction)

* [NOD-264] Wrote a test for selectTxs.

* [NOD-264] Fixed a comment.

* [NOD-264] Fixed misunderstood test.

* [NOD-264] Added zero fee check. Added a couple more tests.

* [NOD-264] Added probabilistic tests. Fixed a couple of bugs in tx selection.

* [NOD-264] Fixed tests with missing fees.

* [NOD-264] Added a test over a range of txs with different gas/mass.

* [NOD-264] Added expected probability to the rest of the test cases.

* [NOD-264] Tightened bounds in probability test.

* [NOD-264] Fixed values in probabily test.

* [NOD-264] Added a comments for alpha and rebalanceThreshold.

* [NOD-264] Fixed a couple of comments, renamed result to txsForBlockTemplate.

* [NOD-264] Removed an irrelevant comment. Changed Tracef to Warnf in some logs.

* [NOD-264] Renamed selectionValue -> txValue.

* [NOD-264] Moved rebalancing to the start of the tx selection loop.

* [NOD-264] Added overflow check for gasUsage.

* [NOD-264] Renamed blockSigOps and blockMass to totalSigOps and totalMass.

* [NOD-264] Removed the need to pass usedCount to reblanaceCandidates. Also relaxed bounds in a test.

* [NOD-264] Split selectTxs into smaller functions. Also relaxed bounds in a test some more.

* [NOD-264] Added a comment for findTx.

* [NOD-264] Ordered candidateTxs by subnetwork instead of txValue.

* [NOD-264] Disallowed zero tx fees in mempool and config. Renamed iterateCandidateTxs to populateTemplateFromCandidates.

* [NOD-264] Changed isFinalizedTransaction log level from Warn to Debug.

* [NOD-264] Removed references to SigOps in txSelection.

* [NOD-264] Removed SigOps validation. Validating mass should suffice.

* [NOD-264] Renamed wasUsed to isMarkedForDeletion.

* [NOD-264] Renamed markCandidateTxUsed to markCandidateTxForDeletion.

* [NOD-264] Made some probabilistic tests less likely to fail when they shouldn't.

* [NOD-264] Added a message warning people about probabilistic tests.

* [NOD-264] Rephrased a comment about rebalanceThreshold.

* [NOD-264] Removed IsCoinBase, CheckTransactionInputsAndCalulateFee, and ValidateTransactionScripts from txSelection.

* [NOD-264] Removed a condition that is no longer relevant.

* [NOD-264] "which's" -> "whose"

* [NOD-264] Removed wasteful preallocations.

* [NOD-264] Fixed a comment referring to "used" transactions.
2019-08-19 12:08:48 +03:00
stasatdaglabs
5f49115cac [NOD-269] Implement GetChainFromBlock api-call (#364)
* [NOD-269] Added a skeleton for getChainFromBlock.

* [NOD-269] Made startHash and includeBlocks optional.

* [NOD-269] Implemented chainBlock collection.

* [NOD-269] Extracted GetBlockVerboseResult building to its own method.

* [NOD-269] Implemented the IncludeBlocks part of GetChainFromBlock.

* [NOD-269] Added a comment for NewGetChainFromBlockCmd.

* [NOD-269] Made IsInSelectedPathChain return an error.

* [NOD-269] Fixed a very wrong comment.

* [NOD-269] Made SelectedPathChain allocate only the required amount of space.

* [NOD-269] Renamed pathChain to parentChain.

* [NOD-269] Split handleGetChainFromBlock to separate functions.

* [NOD-269] Fixed some grammar.
2019-08-18 13:31:54 +03:00
stasatdaglabs
534cb2bf5b [NOD-272] In CheckTransactionSanity, multiply mass limit check by massPerTxByte. (#365) 2019-08-15 15:05:26 +03:00
stasatdaglabs
187c525667 [NOD-234] In getBlockTemplate check if node is current (#362)
* [NOD-234] Added an IsCurrent check to handleGetBlockTemplate.

* [NOD-234] Removed IsCurrent check from handleGetBlockTemplateRequest. Added an explanation for why we're checking the chainHeight.

* [NOD-234] Added ShouldMineOnGenesis to the IsCurrent check.

* [NOD-234] Flipped && operands to fail fast.
2019-08-14 15:09:35 +03:00
stasatdaglabs
6032727965 [NOD-268] Implement selectedParentChain-related structures in btcjson. (#363) 2019-08-13 11:15:51 +03:00
stasatdaglabs
bb3f23b6dc [NOD-240] Get rid of all references for wallet related RPC commands (#361)
* [NOD-240] Removed references to the wallet in rpcserver.go.

* [NOD-240] Began removing btcwalletxxx.go.

* [NOD-240] Got rid of rpcclient/wallet.go and walletsvrcmds.go.

* [NOD-240] Moved GetBestBlockResult to dagsvrresults.go.

* [NOD-240] Finished removing walletsvrXXX.go.

* [NOD-240] Removed wallet stuff from btcctl.

* [NOD-240] Removed a few last things that I've missed.
2019-08-12 09:54:07 +03:00
stasatdaglabs
e5485ac5e6
[NOD-262] Renamed all instances of GetBlocks to GetBlockInvs. (#359) 2019-08-11 12:25:29 +03:00
stasatdaglabs
594a209f83 [NOD-263] Rename all instances of hashStop to stopHash. (#360) 2019-08-11 11:20:21 +03:00