* [NOD-46] Immediately associate received address to their subnetwork
* [NOD-46] fix comment
* [NOD-46] Disconnect peer if it sends an MsgAddr with no subnetwork
* [NOD-46] Disconnect peer if it sends an MsgAddr with incorrect subnetwork id
* [NOD-46] add parenthesis to condition
* [NOD-46] change order of conditions
* [DEV-363] Added testnet seeders
* [DEV-363] Prepare BTCD to work with DNS seeder
* [DEV-363] Finished BTCD part of DNS seeder
* [DEV-363] Reverted protocol changes
* [DEV-363] Reverted protocol changes
* [DEV-363] Reverted protocol changes and fixed tests
* [DEV-363] Added tests and refactored
* [DEV-363] Small refactoring of p2p changes
* [DEV-363] Update NeedMoreAddresses function to take into account full peers + small refactoring
* [DEV-363] Removed IsDNSSeederPeer flag
* [DEV-363] Fixed comment
* [DEV-378] Added feeAccumulator structures
* [DEV-378] Added logic to create and store fee data when validating a block
* [DEV-378] Renamed feeAccumulator to compactFeeData, and all related entities accordingly
* [DEV-378] Converted MsgTx.TxHash() to pointer to hash and not hash
* [DEV-378] Restructured parameters to buildFeeTransaction and related entities
* [DEV-378] Finished the code that calculates fees for blocks
* [DEV-378] Fix TxIndex after changing the structure of AcceptedTxsData
* [DEV-378] For genesis block: Return empty AcceptedTxsData instead of nil
* [DEV-378] Off-by-one error
* [DEV-378] Length of compactFeeData should be determined by specific method, not
* [DEV-378] Multiple bugfixes in tx fee calculation
* [DEV-378] Calculate fee even if fastAdd, to save feeData
* [DEV-378] use IsEqual instead of == when comparing TxHash
* [DEV-378] txindex: if including block is the new block - don't fetch id from DB
* [DEV-378] Fixed a few typos and made some vars consts
* [DEV-378] Re-organized fee functions, removed redundant functions and constants, and revised a few comments
* [DEV-378] Recovered fmt string changes lost in merge
* [DEV-378] Renamed acceptedTxsData and related types and vars to txsAcceptanceData
* [DEV-378] Some comment fixes
* [DEV-378] Remove redundant .ToString()
* [DEV-376] Changed any instance of %v in format strings with a more specific format token
* [DEV-376] Fixed some more wrong formatting strings + removed redundant
cast
* [DEV-376] Added fmt.Sprintf where it was missing
* [DEV-376] use %s for util.Amount, to invoke .String()
* [DEV-376] Some more fixes in format strings
* [DEV-376] fixed mruinvmap_test to expect the correct behaviour
* [DEV-364] Add fee transactions validation
* [DEV-364] make NextBlockFeeTransactions for creating block templates
* [DEV-364] apply coinbase rules to fee transaction is some cases
* [DEV-364] Add comments
* [DEV-364] put getTXO as separate function
* [DEV-364] Make getParentsFeeData a separate function
* [DEV-364] fix calculateFees
* [DEV-364] force maximum sequence for fee transactions
* [DEV-364] add TestValidateFeeTransactions
* [DEV-364] change fee transaction to be one tx per block rather than one tx for each blue
* [DEV-364] fix tests
* [DEV-364] Use constants instead of inline numbers
* [DEV-336] Split addresses in address manager by subnetwork id
* [DEV-336] Load DAG with subnetwork from the config file
* [DEV-336] Remove redundant checks in updateAddrNew and updateAddrTried
* [DEV-361] Create type TxID as alias to daghash.Hash. Use it for transaction IDs
* [DEV-361] Fixed missed renames
* [DEV-361] Removed usage of zeroHash
* [DEV-361] Fixed more missed renames
* [DEV-332] Added MerkleProof to MsgBlock and rejected full-node/partial-block type misbehaviors.
* [DEV-332] Fixed merge issues.
* [DEV-332] Got rid of MerkleProof. Turns out we no longer need it.
* [DEV-332] Got rid of NTBlockDisconnected, as no one was ever triggering it. (It was part of reorg)
* [DEV-332] Implemented clearing out the payloads of transactions of outgoing blocks for partial nodes.
* [DEV-332] Extracted ConvertToPartial to its own method. Added a test. Added a condition for converting to a partial block.
* [DEV-332] Fixed bad ConvertToPartial condition.
* [DEV-348] Removed a couple of unused methods.
* [DEV-348] Implemented validating incoming transactions for bad partial transactions.
* [DEV-348] Added a (incomplete) filter for propogation of transactions.
* [DEV-348] Implemented filtering inventory by subnetwork.
* [DEV-348] Fixed broken tests.
* [DEV-348] Added test for non-zero payload partial transactions.
* [DEV-348] Added a comment for Config.SubnetworkID.
* [DEV-348] Fixed formatting.
* [DEV-348] Renamed isRemoteTransactionFull to shouldTxBeFull.
* [DEV-348] Added a check for invalid transaction in maybeAcceptTransaction. Added handling for native networks.
* [DEV-348] Fixed formatting.
* [DEV-348] Fixed a bug in transaction validation.
* [DEV-348] Rephrased a comment.
* [DEV-348] Extracted subnetwork compatibility to a method. Wrote a test for it.
* [DEV-348] Removed an unnecessary check over the native subnetwork.
* [DEV-333] Added subnetwork to local peer version message.
* [DEV-333] Fixed broken references.
* [DEV-333] Added serialization/deserialization of the version message.
* [DEV-333] Added rejection for peers with wrong subnetwork.
* [DEV-333] Fixed bad comment.
* [DEV-333] Renamed ChainParams to DAGParams.
* [DEV-333] Fixed partial nodes disconnecting from full nodes.
* [DEV-333] chain -> DAG
* [DEV-333] Possibly fixed outbound msgVersion rules.
* [DEV-333] Combined the two incompatible subnetwork cases into one if.
* [DEV-333] Reformatted the condition.
* [DEV-345] Validate that gas and payload are 0 when required by sub-network
* [DEV-345] Remove check for txOut.Value < 0, since txOut.Value is a uint64
* [DEV-345] Added tests for CheckTransactionSanity
* [DEV-345] Remove checks for Gas and Payload validity in wire.MsgTx.Decode
* [DEV-345] Verify that payload in Gas sub-network is always 8 bytes (uint64).
* [DEV-345] Renamed tstCheck{Script/Rule}Error to check{Script/Rule}Error
* [DEV-345] Improved formatting
* [DEV-343] Made sub-network registry use subNetworkIDs.
* [DEV-343] Removed an unnecessary clone.
* [DEV-343] Renamed buildSubNetworkID to txToSubNetworkID. Broke out of a loop when it was known that no further processing is required. Handled error cases from dbGetNetwork separately from the "not-found" case/
* [DEV-343] Added an error case in GasLimit() for where the sub-network is nil.
* [DEV-343] Fixed return nil instead of err. Used a better way to check whether we should continue checking accepted transactions.
* [DEV-307] Implemented adding pending registry transactions.
* [DEV-307] Implemented a skeleton for the sub-network registry algorithm.
* [DEV-307] Implemented the serialization/deserialization logic for pending sub-network transactions.
* [DEV-307] Implemented marking sub-network registry transactions as successfully registered.
* [DEV-307] Implemented clearing pending sub-network entries and writing registered sub-networks.
* [DEV-307] Added comments for all the dagio functions.
* [DEV-307] Fixed a couple of bugs and added a test for checking serialization/deserialization of pending sub-network transactions.
* [DEV-307] Implemented getting a registered sub-network's gas limit.
* [DEV-307] Updated sub-network announcement transaction validation rules.
* [DEV-307] Specified what is validated.
* [DEV-307] Added initialization for lastSubNetworkID.
* [DEV-307] Renamed extractValidSubNetworkRegistryTxs to validateAndExtractSubNetworkRegistryTxs to better reflect what it does.
* Dev 303 implement block finality (#139)
* [DEV-303] Implement block finality
* [DEV-303] Add finality tests
* [DEV-303] Make finality tests use maybeAcceptBlock
* [DEV-303] Only check finality rules if we are fastAdd mode
* [DEV-303] replace hasBiggerFinalityScoreThan checks with .finalityScore()
* [DEV-307] Forgot to actually create the sub-network buckets.
* [DEV-307] Wrote a full-flow test for sub-network registry and fixed a couple of bugs that it had uncovered.
* [DEV-307] Took firstSubNetworkID out of dagconfig.Params. It's unnecessary there.
* [DEV-307] Added persistance for dag.lastSubNetworkID.
* [DEV-307] Moved sub-network stuff to their own files. Got rid of firstSubNetworkID. Replaced SubNetworkReservedLast with SubNetworkUnreservedFirst. Added ErrSubNetworkRegistry.
* [DEV-307] Added a newline at the end of dag_test.go.
* [DEV-307] Renamed previousFinalityPoint to initialFinalityPoint.
* [DEV-307] Moved sub-network IO stuff to subnetworks.go and subnetworks_test.go.
* [DEV-307] Merged dbPutRegisteredSubNetworkTx and dbRegisterSubNetwork. Fixed a too-big allocation and a block double-processing bug.
* [DEV-307] Simplified the serialized representation of a slice of transactions.
* [DEV-307] Fixed comments.
* [DEV-331] Disallowed node to run in reserved sub-networks
* [DEV-331] Added test to make sure that constant are not modified without modifying help text
* [DEV-134] Implement Continuous Integration
Squashed commit:
[5e41d830] Dev 223 fix txindex (#100)
* [DEV-201] In handleGetBlockDAGInfo calculate difficulty by the tip with the lowest bits
* [DEV-202] Move VirtualBlock.GetUTXOEntry to BlockDAG
* [DEV-203] Move VirtualBlock.SelectedTip() to BlockDAG
* [DEV-203] Move VirtualBlock.SelectedTip() to BlockDAG
* [DEV-204] Unexport VirtualBlock() and add CalcMedianTime method for DAG
* [DEV-204] add explanation about difficulty in CurrentBits() comment
* [DEV-204] unexport VirtualBlock type
* [DEV-223] make applyUTXOChanges return pastUTXOResults
* [DEV-223] add bluestxdata for current block as well
* [DEV-223] re-design tx index
* [DEV-223] edit txindex comments
* [DEV-223] rename BluesTxData -> AcceptedTxData, and return from applyUTXOChanges only transactions that got accepted
* [DEV-223] add unit test for txindex
* [DEV-223] fix comments and unite blueTransaction and AcceptedTxData to one type
* [DEV-223] use bucket cursor for dbFetchFirstTxRegion
* [DEV-223] use the same cursor instance for dbFetchFirstTxRegion
* [DEV-223] write in dbFetchFirstTxRegion's comment that it returns the first block region
* [DEV-223] rename type BlueBlockTransaction to TxWithBlockHash
* [DEV-223] add named returned value for applyUTXOChanges
[4c95e293] [DEV-134] Made golint ignore the vendor directory.
[21736dbc] [DEV-134] Renamed ExampleBlockChain_ProcessBlock to ExampleBlockDAG_ProcessBlock to satisfy go vet.
[beea6486] [DEV-134] Removed pushing the built docker to a remove repository. That's unnecessary at this stage.
[bee911ed] [DEV-134] Made all precompilation checks run on everything instead of only the root dir.
[585f92ae] [DEV-134] Added "github.com/pkg/errors" to dep.
[5f02f570] [DEV-134] -vendor-only is written with only one hyphen.
[3eee7f95] [DEV-134] go vet instead of go tool vet.
[0c2d4343] [DEV-134] Split all the pre-compile checks to separate lines to be able to tell which of them is failing.
[780519c8] [DEV-134] Ran gofmt on everything.
[8247146b] Dev 223 fix txindex (#100)
* [DEV-201] In handleGetBlockDAGInfo calculate difficulty by the tip with the lowest bits
* [DEV-202] Move VirtualBlock.GetUTXOEntry to BlockDAG
* [DEV-203] Move VirtualBlock.SelectedTip() to BlockDAG
* [DEV-203] Move VirtualBlock.SelectedTip() to BlockDAG
* [DEV-204] Unexport VirtualBlock() and add CalcMedianTime method for DAG
* [DEV-204] add explanation about difficulty in CurrentBits() comment
* [DEV-204] unexport VirtualBlock type
* [DEV-223] make applyUTXOChanges return pastUTXOResults
* [DEV-223] add bluestxdata for current block as well
* [DEV-223] re-design tx index
* [DEV-223] edit txindex comments
* [DEV-223] rename BluesTxData -> AcceptedTxData, and return from applyUTXOChanges only transactions that got accepted
* [DEV-223] add unit test for txindex
* [DEV-223] fix comments and unite blueTransaction and AcceptedTxData to one type
* [DEV-223] use bucket cursor for dbFetchFirstTxRegion
* [DEV-223] use the same cursor instance for dbFetchFirstTxRegion
* [DEV-223] write in dbFetchFirstTxRegion's comment that it returns the first block region
* [DEV-223] rename type BlueBlockTransaction to TxWithBlockHash
* [DEV-223] add named returned value for applyUTXOChanges
[bff68aa3] [DEV-134] Gave executable permission to deploy.sh
[638a99d9] [DEV-134] Added jenkinsfile and deploy script.
* [DEV-134] Added a robust testing script.
* [DEV-134] Fixed a bash-ism.
* [DEV-134] Disabled testing with coverage for now.
* [DEV-134] Disabled golint and removed removing debug symbols.
* [DEV-134] Disabled aligncheck.
* [DEV-134] Disabled structcheck and varcheck.
* [DEV-134] Added "don't inline functions" to compiler flags for testing.
* [DEV-134] Made build fail if gofmt prints out anything.
* [DEV-134] Fixed misleading comment.
* [DEV-134] Added comments to test.sh.
* [DEV-134] Renamed tm to measure_runtime and removed do_ prefixes from functions.
* [DEV-134] Fixed gofmt line in build script.
* [DEV-134] Fixed gofmt some more.
* [DEV-134] Fixed gofmt not actually failing due to logical or.
* [DEV-222] Changed type of TxOut.Value and util.Amount to uin64
* [DEV-222] Replaced math.MaxUint64 with 0
* [DEV-222] Fixed comment to reflect uint64 instead of int64
* [DEV-222] Fixed overflow comment
* [DEV-62] add phantom constructs to blocknode
* [DEV-62] add phantom constructs to blocknode
* [DEV-72] write blues()
* [DEV-72] write blues()
* [DEV-72] write blues()
* [DEV-62] add comments to new phantom constructs in blocknode
* Fixed dbIndexConnectBlock. (#33)
* Fixed dbIndexConnectBlock.
* Removed redundant check in storeFilter.
* Created a new method to BlockHeader: IsGenesis.
* [DEV-71] Implement BlockHeap (#35)
* [DEV-71] Implemented BlockHeap.
* [DEV-71] Removed irrelevant comment.
* [DEV-71] Renamed variables in Pop() and split Less() to multiple lines.
* [DEV-72] write blues()
* [DEV-72] write blues()
* [DEV-72] write blues()
* [DEV-72] write blues tests
* [DEV-72] write blues tests
* [DEV-72] remove relevant past
* [DEV-72] write blues tests
* [DEV-72] write blues tests
* [DEV-72] write blues tests
* [DEV-72] write functions to order blockSet by hash and write blue tests
* [DEV-72] add secret mining and censorship attack tests
* [DEV-72] remove prints
* [DEV-72] remove K from dagconfig.Params
* [DEV-72] remove K from dagconfig.Params
* [DEV-72] change blueScore to uint64
* [DEV-72] block V was missing, so renamed w -> v, x -> w etc
* [DEV-72] use node.String instead of %v
* [DEV-72] block V was missing, so renamed w -> v, x -> w etc
* [DEV-72] add K to dagconfig.Params, and add expected reds to all phantom tests
* [DEV-72] set K=10 and add comments to phantom and phantom tests
* [DEV-72] fix formatting and add comments to TestPhantom
* [DEV-72] fix grammar
* [DEV-77] Added tests for ScriptFreeList
* [DEV-77] Fixed formatting in BlockHeader .SelectedPrevBlock and IsGenesis
* [DEV-77] Restructured returnScriptBuffers to be more idiomatic and testable
* [DEV-77] Added tests for binaryFreeList
* [DEV-77] Added test-cases to TestVarIntWireErrors to check for errors when writing the varint length
* [DEV-77] Added MsgSendHeaders and MsgFeeFilter test-cases for TestMessage
* [DEV-77] Added test for BlockHeader.IsGenesis()
* [DEV-77] using binaryFreeListMaxItems instead of freeListMaxItems in TestBinaryFreeList
* [DEV-77] Fixed error message copy-paste typo
This modifies the utxoset in the database and related UtxoViewpoint to
store and work with unspent transaction outputs on a per-output basis
instead of at a transaction level. This was inspired by similar recent
changes in Bitcoin Core.
The primary motivation is to simplify the code, pave the way for a
utxo cache, and generally focus on optimizing runtime performance.
The tradeoff is that this approach does somewhat increase the size of
the serialized utxoset since it means that the transaction hash is
duplicated for each output as a part of the key and some additional
details such as whether the containing transaction is a coinbase and the
block height it was a part of are duplicated in each output.
However, in practice, the size difference isn't all that large, disk
space is relatively cheap, certainly cheaper than memory, and it is much
more important to provide more efficient runtime operation since that is
the ultimate purpose of the daemon.
While performing this conversion, it also simplifies the code to remove
the transaction version information from the utxoset as well as the
spend journal. The logic for only serializing it under certain
circumstances is complicated and it isn't actually used anywhere aside
from the gettxout RPC where it also isn't used by anything important
either. Consequently, this also removes the version field of the
gettxout RPC result.
The utxos in the database are automatically migrated to the new format
with this commit and it is possible to interrupt and resume the
migration process.
Finally, it also updates the tests for the new format and adds a new
function to the tests to convert the old test data to the new format for
convenience. The data has already been converted and updated in the
commit.
An overview of the changes are as follows:
- Remove transaction version from both spent and unspent output entries
- Update utxo serialization format to exclude the version
- Modify the spend journal serialization format
- The old version field is now reserved and always stores zero and
ignores it when reading
- This allows old entries to be used by new code without having to
migrate the entire spend journal
- Remove version field from gettxout RPC result
- Convert UtxoEntry to represent a specific utxo instead of a
transaction with all remaining utxos
- Optimize for memory usage with an eye towards a utxo cache
- Combine details such as whether the txout was contained in a
coinbase, is spent, and is modified into a single packed field of
bit flags
- Align entry fields to eliminate extra padding since ultimately
there will be a lot of these in memory
- Introduce a free list for serializing an outpoint to the database
key format to significantly reduce pressure on the GC
- Update all related functions that previously dealt with transaction
hashes to accept outpoints instead
- Update all callers accordingly
- Only add individually requested outputs from the mempool when
constructing a mempool view
- Modify the spend journal to always store the block height and coinbase
information with every spent txout
- Introduce code to handle fetching the missing information from
another utxo from the same transaction in the event an old style
entry is encountered
- Make use of a database cursor with seek to do this much more
efficiently than testing every possible output
- Always decompress data loaded from the database now that a utxo entry
only consists of a specific output
- Introduce upgrade code to migrate the utxo set to the new format
- Store versions of the utxoset and spend journal buckets
- Allow migration process to be interrupted and resumed
- Update all tests to expect the correct encodings, remove tests that no
longer apply, and add new ones for the new expected behavior
- Convert old tests for the legacy utxo format deserialization code to
test the new function that is used during upgrade
- Update the utxostore test data and add function that was used to
convert it
- Introduce a few new functions on UtxoViewpoint
- AddTxOut for adding an individual txout versus all of them
- addTxOut to handle the common code between the new AddTxOut and
existing AddTxOuts
- RemoveEntry for removing an individual txout
- fetchEntryByHash for fetching any remaining utxo for a given
transaction hash
