600 Commits

Author SHA1 Message Date
Xiang Li
4a64373225 raft: add flow control for progress
Each progress has a inflighs sliding window. When the progress
is in replicate state, inflights will control the sending speed
of the leader.

The leader can have at most maxInflight number of inflight
messages for each replicate progress. Receving a appResp moves
forward the sliding window. Heartbeat response free one
slot if the window is full.
2015-03-20 20:04:33 -07:00
Xiang Li
09a86cb9b9 Merge pull request #2553 from xiang90/raft-design
raft: add progress state machine graph
2015-03-20 19:57:51 -07:00
Xiang Li
86622537a1 raft: add progress state machine graph 2015-03-20 15:28:50 -07:00
Xiang Li
44d9209990 Merge pull request #2548 from xiang90/raft-design
raft: add our very first design.md
2015-03-20 09:07:44 -07:00
Yicheng Qin
6e557c58c7 Merge pull request #2532 from yichengq/342
raft: print out data and time in log
2015-03-20 08:03:23 -07:00
Xiang Li
59d8089295 raft: add our very first design.md 2015-03-19 21:00:47 -07:00
Xiang Li
2adb58f9de raft: move progress to progress.go 2015-03-19 10:05:04 -07:00
Xiang Li
7571b2cde2 raft: limit the size of msgApp
limit the max size of entries sent per message.
Lower the cost at probing state as we limit the size per message;
lower the penalty when aggressively decrease to a too low next.
2015-03-18 15:59:30 -07:00
Yicheng Qin
0634cf2cfe raft: print out data and time in log
Keep the default log setting consistent with other packages.
2015-03-18 15:49:06 -07:00
Yicheng Qin
7e7bc76038 Merge pull request #2514 from yichengq/340
raft: introduce progress states
2015-03-18 09:40:30 -07:00
Yicheng Qin
67194c0b22 raft: introduce progress states 2015-03-18 08:16:32 -07:00
Xiang Li
d17f3a4452 Merge pull request #2519 from bdarnell/multinode-commit
raft: Use the correct commit index when advancing in MultiNode.
2015-03-17 10:31:53 -07:00
Ben Darnell
cd1ff78ff3 raft: Elaborate a little more about committed entries in commitReady. 2015-03-17 13:22:36 -04:00
funkygao
0b912c0faf raft: fix godoc about starting a node 2015-03-17 17:35:18 +08:00
Ben Darnell
271d911c32 raft: Use the correct commit index when advancing in MultiNode.
This fixes an issue when restoring from a snapshot and brings
MultiNode closer to Node.
2015-03-16 18:40:51 -04:00
Ben Darnell
5e19adcf70 raft: correctly pass arguments to Logger.Panicf() 2015-03-12 16:15:43 -04:00
Iago López Galeiras
e698192e4a rafttest: fix build error
raftLogger is not exported so we can't access it from here. Go back to
using log.
2015-03-12 11:47:13 +01:00
Xiang Li
39731724ff Merge pull request #2485 from yichengq/337
raft: fall back to bad path when unreachable
2015-03-11 14:16:39 -07:00
Yicheng Qin
be0bf2a2bd raft: fall back to bad path when unreachable 2015-03-11 13:21:23 -07:00
Xiang Li
c643967a41 raft: reply with the commit index when receives a smaller append message
Follower should not reject the append message with a smaller index than its commit
index. Or it will trigger the leader's resending logic, which might have a high cost.
2015-03-10 22:32:36 -07:00
Xiang Li
a2be25cba4 Merge pull request #2460 from xiang90/raft-logger
raft: introduce logger interface
2015-03-09 08:00:21 -07:00
Xiang Li
97579e2e1d raft: introduce logger interface 2015-03-08 21:36:32 -07:00
Xiang Li
7fe608532a raft: do not reset vote if term is not changed
raft MUST keep the voting information for the same term. reset
should not reset vote if term is not changed.
2015-03-07 22:31:20 -08:00
Ben Darnell
725c411346 Add ReportUnreachable and ReportSnapshot to MultiNode.
Add ReportSnapshot requirement to doc.go.
2015-03-05 12:39:52 -05:00
Xiang Li
6b9b695167 Merge pull request #2435 from bdarnell/multinode
raft: Introduce MultiNode.
2015-03-04 21:27:20 -08:00
Ben Darnell
c824c867ec raft: more doc updates.
Including parallelism of persist and send, cancellation of
ConfChanges, and the risks of two-node clusters.
2015-03-04 15:48:35 -05:00
Ben Darnell
4e74d81bbb raft: Introduce MultiNode.
MultiNode is an alternative to raft.Node that is more efficient
when a node may participate in many consensus groups. It is currently
used in the CockroachDB project; this commit merges the
github.com/cockroachdb/etcd fork back into the mainline.
2015-03-04 15:30:21 -05:00
Ben Darnell
250970cc23 raft: Expand doc.go
Includes more details on the required caller behavior and the safety of
membership changes.

Closes #2397
2015-03-04 13:18:02 -05:00
Yicheng Qin
b4b9b9118a rafthttp: report MsgSnap status 2015-03-02 09:38:11 -08:00
Yicheng Qin
09f181f585 raft: log unreachable remote node 2015-03-01 16:47:49 -08:00
Yicheng Qin
fbd5c81139 raft: remove shadowing of variables from test 2015-02-28 12:09:33 -08:00
Xiang Li
9b4d52ee73 raft: do not resend snapshot if not necessary
raft relies on the link layer to report the status of the sent snapshot.
If the snapshot is still sending, the replication to that remote peer will
be paused. If the snapshot finish sending, the replication will begin
optimistically after electionTimeout. If the snapshot fails, raft will
try to resend it.
2015-02-28 11:41:58 -08:00
Xiang Li
2185ac5ac8 raft: cleanup unreachable 2015-02-28 11:35:16 -08:00
Xiang Li
2af33fd494 raft: add reportUnreachable 2015-02-28 10:45:22 -08:00
Xiang Li
cbef6ab152 raft: clean up storage 2015-02-28 10:09:07 -08:00
Xiang Li
5ede18be74 raft: separate compact and createsnap in memory storage 2015-02-28 10:08:30 -08:00
Ben Darnell
b53dc0826e Only use the EntryFormatter for normal entries.
ConfChange entries also have a Data field but the application-supplied
formatter won't know what to do with them.
2015-02-20 13:51:14 -05:00
Barak Michener
92dca0af0f *: remove shadowing of variables from etcd and add travis test
We've been bitten by this enough times that I wrote a tool so that
it never happens again.
2015-02-17 16:31:42 -05:00
Xiang Li
fa66055f66 rafttest: drop isPaused 2015-02-09 18:52:34 -08:00
Xiang Li
085b608de9 rafttest: support node pause 2015-02-09 16:26:43 -08:00
Xiang Li
279b216f9a raftest: wait for network sending 2015-02-09 15:52:16 -08:00
Xiang Li
65cd0051fe rafttest: add network delay 2015-02-06 15:01:07 -08:00
Xiang Li
d423946fa4 rafttest: add network drop 2015-02-06 10:50:55 -08:00
Xiang Li
83edf0d862 rafttest: separate network interface and network 2015-02-03 22:50:27 -08:00
Xiang Li
b147a6328d raftest: add restart and related simple test 2015-02-03 10:08:52 -08:00
Xiang Li
d65af21b73 raft: add raft test suite 2015-02-01 14:53:22 -08:00
Xiang Li
bff2ccaa22 Merge pull request #2170 from xiang90/remove_log
raft: remove default verbose logging
2015-01-27 15:58:53 -08:00
Xiang Li
553379e82b raft: remove default verbose logging 2015-01-27 15:57:44 -08:00
Ben Darnell
33d2400063 raft: Send any waiting appends after receiving MsgAppResp.
This addresses a problem that comes up in the cockroach tests,
in which the order of messages may lead to deadlocks (due to
the fact that we don't have regular heartbeat timers in most
of our tests).
2015-01-27 17:43:29 -05:00
Xiang Li
276c9540b4 etcdserver: support raft.status 2015-01-26 16:39:33 -08:00