Docs: new page on timestamps, updated other timestamp docs

bigchaindb/util.py

@@ -127,11 +127,11 @@ def deserialize(data):
 
 
 def timestamp():
-    """Calculate a UTC timestamp with second precision.
+    """The Unix time, rounded to the nearest second.
+       See https://en.wikipedia.org/wiki/Unix_time
 
     Returns:
-        str: UTC timestamp.
-
+        str: the Unix time
     """
     return str(round(time.time()))
 

docs/source/topic-guides/index.rst

@@ -15,4 +15,5 @@ Topic guides give background and explain concepts at a high level.
    assets
    smart-contracts
    models
+   timestamps
    node-cluster-fed

docs/source/topic-guides/models.md

@@ -8,6 +8,11 @@ _Transactions_ are used to register, issue, create or transfer things (e.g. asse
 Below we often refer to cryptographic hashes, keys and signatures. The details of those are covered in [the section on cryptography](../appendices/cryptography.html).
 
 
+## Some Words of Caution
+
+BigchainDB is still in the early stages of development. The data models described below may change substantially before BigchainDB reaches a production-ready state (i.e. version 1.0 and higher).
+
+
 ## Transaction Concepts
 
 Transactions are the most basic kind of record stored by BigchainDB. There are two kinds: creation transactions and transfer transactions.
@@ -38,13 +43,6 @@ When a node is asked to check the validity of a transaction, it must do several
 The full details of transaction validation can be found in the code for `validate_transaction()` in the `BaseConsensusRules` class of [`consensus.py`](https://github.com/bigchaindb/bigchaindb/blob/master/bigchaindb/consensus.py) (unless other validation rules are being used by a federation, in which case those should be consulted instead).
 
 
-## Some Words of Caution
-
-BigchainDB is still in the early stages of development. The data models described below may change substantially before BigchainDB reaches a production-ready state (i.e. version 1.0 and higher).
-
-Also, note that timestamps come from clients and nodes. Unless you have some reason to believe that some timestamps are correct or meaningful, we advise you to ignore them (i.e. don't make any decisions based on them). (You might trust a timestamp, for example, if it came from a trusted timestamping service and it is embedded in the transaction data `payload` along with the signature from the timestamping service. You might trust node timestamps if you know all the nodes are running NTP servers.)
-
-
 ## The Transaction Model
 
 ```json
@@ -76,7 +74,7 @@ Here's some explanation of the contents of a transaction:
     - `conditions`: List of conditions. Each _condition_ is a _crypto-condition_ that needs to be fulfilled by a transfer transaction in order to transfer ownership to new owners.
     See [Conditions and Fulfillments](#conditions-and-fulfillments) below.
     - `operation`: String representation of the operation being performed (currently either "CREATE" or "TRANSFER"). It determines how the transaction should be validated.
-    - `timestamp`: Time of creation of the transaction in UTC. It's provided by the client.
+    - `timestamp`: The Unix time when the transaction was created. It's provided by the client. See [the section on timestamps](timestamps.html).
     - `data`:
         - `uuid`: UUID version 4 (random) converted to a string of hex digits in standard form.
         - `payload`: Can be any JSON document. It may be empty in the case of a transfer transaction.
@@ -238,7 +236,7 @@ If there is only one _current owner_, the fulfillment will be a simple signature
 
 - `id`: The hash of the serialized `block` (i.e. the `timestamp`, `transactions`, `node_pubkey`, and `voters`). This is also a database primary key; that's how we ensure that all blocks are unique.
 - `block`:
-    - `timestamp`: Timestamp when the block was created. It's provided by the node that created the block.
+    - `timestamp`: The Unix time when the block was created. It's provided by the node that created the block. See [the section on timestamps](timestamps.html).
     - `transactions`: A list of the transactions included in the block.
     - `node_pubkey`: The public key of the node that create the block.
     - `voters`: A list of public keys of federation nodes. Since the size of the 
@@ -260,7 +258,7 @@ Each node must generate a vote for each block, to be appended to that block's `v
         "previous_block": "<id of the block previous to this one>",
         "is_block_valid": "<true|false>",
         "invalid_reason": "<None|DOUBLE_SPEND|TRANSACTIONS_HASH_MISMATCH|NODES_PUBKEYS_MISMATCH",
-        "timestamp": "<timestamp of the voting action>"
+        "timestamp": "<Unix time when the vote was generated, provided by the voting node>"
     },
     "signature": "<signature of vote>"
 }

docs/source/topic-guides/timestamps.md

@@ -0,0 +1,64 @@
+# Timestamps in BigchainDB
+
+Each transaction, block and vote has an associated timestamp.
+
+Interpreting those timestamps is tricky. If there's one message to take away from this section, it is this: you need to look at multiple timestamps from multiple sources to get some idea of when something happened.
+
+
+## Timestamp Sources
+
+A transaction's timestamp is provided by the client which created and submitted the transaction to a BigchainDB node. A block's timestamp is provided by the BigchainDB node which created the block. A vote's timestamp is provided by the BigchainDB node which created the vote.
+
+When a BigchainDB client or node needs a timestamp, it calls a BigchainDB utility function named `timestamp()`. There's a detailed explanation of how that function works below, but the bottom line is that it gets the Unix time from the system clock, rounded to the nearest second.
+
+Unix time is defined as defined as the number of seconds that have elapsed since 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970 (i.e. since the Unix Epoch), _not counting leap seconds_. There are more details in [the Wikipedia article about Unix time](https://en.wikipedia.org/wiki/Unix_time).
+
+
+## How to Interpret Timestamps
+
+If a client or node's system clock is wrong, then its timestamps will be wrong. That's not the end of the world, though, because you can look at many timestamps to get an idea of when something happened. For example, a transaction in a decided-valid block has many associated timestamps:
+
+* its own timestamp
+* the timestamps of the other transactions in the block; there could be as many as 999 of those
+* the timestamp of the block
+* the timestamps of all the votes on the block
+
+Those timestamps come from many sources, so you can look at all of them to get some statistical sense of when the transaction "actually happened." The timestamp of the block should always be after the timestamp of the transaction, and the timestamp of the votes should always be after the timestamp of the block.
+
+
+## Syncing with Standard Clocks
+
+A client or node can run an NTP daemon to keep its system clock in sync with standard clocks. There's more information about that in the section titled [Sync Your System Clock](../nodes/setup-run-node.html#sync-your-system-clock).
+
+
+## Converting Timestamps to UTC
+
+It's not always possible to convert a Unix time to a UTC time, because Unix time doesn't have leap seconds, but UTC does. That means that there are some Unix times which correspond to two different UTC times, and there are other Unix times which don't correspond to any UTC time. Be suspicious of any function which claims to convert Unix time to UTC time. It's possible, but to be correct, it would have to give two answers for some Unix times, and errors for other Unix times.
+
+Leap seconds are rare, so it's usually not a problem to convert a Unix time to a UTC time; just check the converter that you're using to make sure that it's got some sophistication.
+
+
+## Including Trusted Timestamps
+
+If you want to create a transaction payload with a trusted timestamp, you can.
+
+One way to do that would be to send a payload to a trusted timestamping service. They will send back a timestamp, a signature, and their public key. They should also explain how you can verify the signature. You can then include the original payload, the timestamp, the signature, and the service's public key in your transaction. That way, anyone can verify that the original payload was signed by the trusted timestamping service.
+
+
+## How the timestamp() Function Works
+
+BigchainDB has a utility function named `timestamp()` which amounts to:
+```python
+timestamp() = str(round(time.time()))
+```
+
+In other words, it calls the `time()` function in Python's `time` module, rounds that to the nearest integer, and converts the result to a string.
+
+It rounds the output of `time.time()` to the nearest second because, according to [the Python documentation for `time.time()`](https://docs.python.org/3.4/library/time.html#time.time), "...not all systems provide time with a better precision than 1 second."
+
+How does `time.time()` work? If you look in the C source code, it calls `floattime()` and `floattime()` calls [clock_gettime()](https://www.cs.rutgers.edu/~pxk/416/notes/c-tutorials/gettime.html), if it's available.
+```text
+ret = clock_gettime(CLOCK_REALTIME, &tp);
+```
+
+(If `clock_gettime()` is not available or returns with a nonzero exit code, it falls back to `_PyTime_gettimeofday_info()`, which is a wrapper around [gettimeofday()](http://man7.org/linux/man-pages/man2/gettimeofday.2.html), an older function that was marked obsolete in POSIX.1-2008.)