413 Commits

Author SHA1 Message Date
Xiang Li
746c66b466 raft: fix start term 2014-11-26 21:21:13 -08:00
Yicheng Qin
0f070f3d2d raft: no need to save dummy entry into stable storage 2014-11-26 14:04:56 -08:00
Xiang Li
66252c7d62 raft: move all unstable stuff into one struct for future cleanup 2014-11-26 13:36:17 -08:00
Yicheng Qin
ab2a40ea37 Merge branch 'log_interface'
Conflicts:
	raft/log.go
2014-11-26 12:16:02 -08:00
Xiang Li
732cfa1ad6 raft: remove the applysnap from Storage interface 2014-11-26 11:28:51 -08:00
Xiang Li
e23f9e76d1 raft: do not applysnapshot in raft 2014-11-26 10:59:13 -08:00
Xiang Li
39e6631447 raft: always write dummy entry to storage 2014-11-25 23:27:40 -08:00
Xiang Li
8de98d4903 raft: clean up 2014-11-25 16:21:50 -08:00
Xiang Li
9bd1786fe4 raft: memory storage does not append out of date entries 2014-11-25 15:18:40 -08:00
Xiang Li
9df0e7715d raft: do not panic on out of date compaction 2014-11-25 15:14:39 -08:00
Xiang Li
01cbcce8ba etcdserver: do not applySnapshot twice 2014-11-25 14:53:49 -08:00
Yicheng Qin
7e6e305c4f Merge branch 'log_interface'
Conflicts:
	raft/raft.go
2014-11-25 14:22:11 -08:00
Yicheng Qin
4b43824be9 raft: not compact log if the compact index < first index of the log
It should ignore the compact operation instead of panic because the case that
the log is restored from snapshot before executing compact is reasonable.
2014-11-25 11:51:20 -08:00
Yicheng Qin
8aa89dba3d raft: make if checking match the error in storage.Term 2014-11-25 00:52:13 -08:00
Yicheng Qin
8ee1bf31d6 raft: use IsEmptySnap to check the empty snapshot 2014-11-25 00:37:21 -08:00
Yicheng Qin
e466126510 raft: set snapshot to nil when it is saved 2014-11-25 00:30:22 -08:00
Yicheng Qin
e17bcd8932 raft: remove wont-fix TODO in ApplyConfChange 2014-11-25 00:10:44 -08:00
Yicheng Qin
85d0e2f130 raft: remove unused raftLog.isOutOfAppliedBounds 2014-11-25 00:07:55 -08:00
Yicheng Qin
1e0f87df8c raft: stricter checking in raftLog.slice 2014-11-25 00:05:00 -08:00
Yicheng Qin
1d01c8aa2d raft: remove unused raftLog.at function 2014-11-24 23:52:28 -08:00
Yicheng Qin
2c06a1d815 raft: not set applied when restore log from snapshot
applied is only updated by application level through Advance.
2014-11-24 23:37:47 -08:00
Yicheng Qin
0d200baf72 raft: refine raftLog.term 2014-11-24 23:27:57 -08:00
Yicheng Qin
7fcaca6d18 raft: simplify raftLog.lastIndex 2014-11-24 23:08:51 -08:00
Yicheng Qin
8670f4012b raft: remove useless line in raftLog.append 2014-11-24 22:42:55 -08:00
Yicheng Qin
239c8dd479 raft: add comment to newLog 2014-11-24 21:47:12 -08:00
Xiang Li
9455119968 raft: always check leader changes in node run loop 2014-11-24 19:07:10 -08:00
Xiang Li
65ad1f6ffd raft: attach Index to Entry in all tests 2014-11-24 17:13:47 -08:00
Xiang Li
10ebf1a335 raft: fix memoryStorage append 2014-11-24 16:36:59 -08:00
Xiang Li
2876c652ab raft: fix for go vet 2014-11-24 15:00:38 -08:00
Xiang Li
62a8df304a raft: fix error message in TestLogRestore 2014-11-24 11:10:02 -08:00
Xiang Li
e8afdcfe0a raft: refactor testUnstableEnts 2014-11-24 10:40:38 -08:00
Xiang Li
3dd4c458ca raft: refactor term in log.go 2014-11-24 10:13:56 -08:00
Xiang Li
94190286ff raft: add comment for append in unstableEntries in log.go 2014-11-24 09:05:40 -08:00
Xiang Li
0a46c70f5d raft: use empty slice in unstableEntries in log.go 2014-11-24 09:04:45 -08:00
Xiang Li
bc0e72acb9 raft: clean up panic in log.go 2014-11-24 09:01:25 -08:00
Xiang Li
f3cef87c69 raft: remove extra empty line in log.go 2014-11-24 08:43:34 -08:00
Xiang Li
bdbafe2cf3 raft: use max in log.slice 2014-11-24 08:36:15 -08:00
Ben Darnell
9ddd8ee539 Rename Storage.HardState back to InitialState and include ConfState.
This fixes integration/migration_test.go (and highlights the fact that
we need some more raft-level testing of restoring from snapshots).
2014-11-21 17:22:20 -05:00
Ben Darnell
03c8881e35 Fix TestSlowNodeRestore 2014-11-21 16:40:41 -05:00
Ben Darnell
0d680d0e6b Merge remote-tracking branch 'coreos/master' into merge
* coreos/master:
  rafthttp: fix import
  raft: should not decrease match and next when handling out of order msgAppResp
  Fix migration to allow snapshots to have the right IDs
  add snapshotted integration test
  fix test import loop
  fix import loop, add set to types, and fix comments
  etcdserver: autodetect v0.4 WALs and upgrade them to v0.5 automatically
  wal: add a bench for write entry
  rafthttp: add streaming server and client
  dep: use vendored imports in codegangsta/cli
  dep: bump golang.org/x/net/context

Conflicts:
	etcdserver/server.go
	etcdserver/server_test.go
	migrate/snapshot.go
2014-11-21 15:40:11 -05:00
Xiang Li
063c5c77a0 raft: should not decrease match and next when handling out of order msgAppResp 2014-11-20 17:58:23 -08:00
Brian Waldon
9a728a127a dep: bump golang.org/x/net/context
Move from code.google.com/p/go.net/context to
golang.org/x/net/context before bumping to latest.
2014-11-20 10:19:12 -08:00
Ben Darnell
b29240baf0 Merge remote-tracking branch 'coreos/master' into merge
* coreos/master:
  scripts: build-docker tag and use ENTRYPOINT
  scripts: build-release add etcd-migrate
  create .godir
  raft: optimistically increase the next if the follower is already matched
  raft: add handleHeartbeat handleHeartbeat commits to the commit index in the message. It never decreases the commit index of the raft state machine.
  rafthttp: send takes raft message instead of bytes
  *: add rafthttp pkg into test list
  raft: include commitIndex in heartbeat
  rafthttp: move server stats in raftHandler to etcdserver
  *: etcdhttp.raftHandler -> rafthttp.RaftHandler
  etcdserver: rename sender.go -> sendhub.go
  *: etcdserver.sender -> rafthttp.Sender

Conflicts:
	raft/log.go
	raft/raft_paper_test.go
2014-11-19 17:05:16 -05:00
Ben Darnell
355ee4f393 raft: Integrate snapshots into the raft.Storage interface.
Compaction is now treated as an implementation detail of Storage
implementations; Node.Compact() and related functionality have been
removed. Ready.Snapshot is now used only for incoming snapshots.

A return value has been added to ApplyConfChange to allow applications
to track the node information that must be stored in the snapshot.

raftpb.Snapshot has been split into Snapshot and SnapshotMetadata, to
allow the full snapshot data to be read from disk only when needed.

raft.Storage has new methods Snapshot, ApplySnapshot, HardState, and
SetHardState. The Snapshot and HardState parameters have been removed
from RestartNode() and will now be loaded from Storage instead.
The only remaining difference between StartNode and RestartNode is that
the former bootstraps an initial list of Peers.
2014-11-19 16:40:26 -05:00
Xiang Li
b50f331558 Merge pull request #1744 from xiang90/next
raft: optimistically increase the next if the follower is already matched
2014-11-19 13:21:11 -08:00
Xiang Li
4c1fd07311 raft: optimistically increase the next if the follower is already matched
This is useful since we want to pipeline the appendEntry requests. Without
enabling optimistic increasing, the second pipelining appendEntry request
will include the entries the first one has already sent out. We decrease
the next directly to match if the leader receives a rejection for a matched
follower. This happens if one pipelining request get lost and following ones
arrives at the follower.
2014-11-18 13:41:38 -08:00
Ben Darnell
46ee58c6f0 raft: Rename ErrSnapshotRequired To ErrCompacted. 2014-11-18 13:15:10 -05:00
Xiang Li
bd4cfa2a07 raft: add handleHeartbeat
handleHeartbeat commits to the commit index in the message. It never decreases the
commit index of the raft state machine.
2014-11-18 08:34:06 -08:00
Xiang Li
b93d87f17f raft: include commitIndex in heartbeat 2014-11-17 16:19:28 -08:00
Ben Darnell
300c5a2001 Merge remote-tracking branch 'coreos/master' into log-storage-interface
* coreos/master: (21 commits)
  etcdserver: refactor ValidateClusterAndAssignIDs
  integration: add integration test for remove member
  integration: add test for member restart
  version: bump to alpha.3
  etcdserver: add buffer to the sender queue
  *: gracefully stop etcdserver
  Fix up migration tool, add snapshot migration
  etcd4: migration from v0.4 -> v0.5
  etcdserver: export Member.StoreKey
  etcdserver: recover cluster when receiving newer snapshot
  etcdserver: check and select committed entries to apply
  etcdserver: recover from snapshot before applying requests
  raft: not set applied when restored from snapshot
  sender: support elegant stop
  etcdserver: add StopNotify
  etcdserver: fix TestDoProposalStopped test
  etcdserver: minor cleanup
  etcdserver: validate new node is not registered before in best effort
  etcdserver: fix server.Stop()
  *: print out configuration when necessary
  ...

Conflicts:
	etcdserver/server.go
	etcdserver/server_test.go
	raft/log.go
2014-11-17 18:28:24 -05:00