313 Commits

Author SHA1 Message Date
Yicheng Qin
9c8f5c9535 Merge pull request #1891 from yichengq/257
etcdserver: init state before run loop correctly
2014-12-08 16:38:33 -08:00
Yicheng Qin
13814c9d7d etcdserver: init state before run loop correctly 2014-12-08 16:13:16 -08:00
Yicheng Qin
7e06d85651 etcdserver: apply entries when it is not empty
Or it updates appliedi wrongly.
2014-12-08 15:56:38 -08:00
Yicheng Qin
71f3b80fbe etcdserver: check recovery error when new server 2014-12-08 14:55:23 -08:00
Yicheng Qin
8c338ffcc7 etcdserver: correct the log about recovering from snapshot 2014-12-08 14:51:42 -08:00
Yicheng Qin
771ff4589d etcdserver: not add self into sendhub when new server 2014-12-05 00:18:40 -08:00
Yicheng Qin
1d1c2ff834 Merge pull request #1841 from yichengq/246
etcdserver: close storage when stop
2014-12-04 15:36:24 -08:00
Yicheng Qin
a7bc03b42b etcdserver: close storage when stop 2014-12-04 15:16:22 -08:00
Xiang Li
88e2fab572 Merge pull request #1859 from xiang90/pause_test
*: add pauseMember test
2014-12-04 15:11:59 -08:00
Veres Lajos
3de2ab2c04 *: typofixes
https://github.com/vlajos/misspell_fixer
2014-12-04 22:51:19 +00:00
Xiang Li
151f043414 *: add pauseMember test 2014-12-04 14:22:43 -08:00
Xiang Li
7beac083ff Merge pull request #1810 from xiang90/purge
*: support purging old wal/snap files
2014-12-01 12:05:05 -08:00
Xiang Li
d3db010190 *: support purging old wal/snap files 2014-12-01 11:50:17 -08:00
Xiang Li
bc5acd3c42 etcdserver: log snapshot event 2014-11-30 12:10:20 -08:00
Xiang Li
e23f9e76d1 raft: do not applysnapshot in raft 2014-11-26 10:59:13 -08:00
Xiang Li
9df0e7715d raft: do not panic on out of date compaction 2014-11-25 15:14:39 -08:00
Xiang Li
01cbcce8ba etcdserver: do not applySnapshot twice 2014-11-25 14:53:49 -08:00
Xiang Li
74d8c7f457 etcdserver: cleanup main loop 2014-11-25 14:38:18 -08:00
Yicheng Qin
a13d5a70ff etcdserver: save snapshot before entries 2014-11-25 12:39:15 -08:00
Yicheng Qin
54e1237271 etcdserver: panic when snapshot on raft storage
Snapshot on raft storage should always succeed. If there is an error, it must
be internal fault and needs stack info to debug.
2014-11-24 21:22:49 -08:00
Yicheng Qin
1b038da18a etcdserver: init snapi when init appliedi 2014-11-24 21:19:30 -08:00
Yicheng Qin
bd9e93eeea etcdserver: remove finished TODO for raftStorage.Compact 2014-11-24 21:10:53 -08:00
Yicheng Qin
185d37c333 etcdserver: not load dummy entry from the wal 2014-11-24 20:51:04 -08:00
Xiang Li
d69e4dbe6d etcdserver: initial index to 1 2014-11-24 14:57:08 -08:00
Xiang Li
453133977d etcdserver: save snapshot only if the index is greater than previous snap index 2014-11-24 14:47:59 -08:00
Xiang Li
08f156a1de etcdserver: remove extra empty line in snapshot func 2014-11-24 10:27:18 -08:00
Ben Darnell
0d680d0e6b Merge remote-tracking branch 'coreos/master' into merge
* coreos/master:
  rafthttp: fix import
  raft: should not decrease match and next when handling out of order msgAppResp
  Fix migration to allow snapshots to have the right IDs
  add snapshotted integration test
  fix test import loop
  fix import loop, add set to types, and fix comments
  etcdserver: autodetect v0.4 WALs and upgrade them to v0.5 automatically
  wal: add a bench for write entry
  rafthttp: add streaming server and client
  dep: use vendored imports in codegangsta/cli
  dep: bump golang.org/x/net/context

Conflicts:
	etcdserver/server.go
	etcdserver/server_test.go
	migrate/snapshot.go
2014-11-21 15:40:11 -05:00
Ben Darnell
30690d15d9 Re-enable a few tests I had missed.
Fix integration test for the change to log entry zero.

Increase test timeouts since integration tests often take
longer than 10s for me.
2014-11-21 15:27:17 -05:00
Brian Waldon
c0fb1c8a00 Merge pull request #1755 from bcwaldon/golang.org-deps
Switch to golang.org/x/net/context
2014-11-20 16:26:14 -08:00
Barak Michener
59a0c64e9f fix import loop, add set to types, and fix comments 2014-11-20 15:38:08 -05:00
Barak Michener
78ea3335bf etcdserver: autodetect v0.4 WALs and upgrade them to v0.5 automatically 2014-11-20 15:38:08 -05:00
Yicheng Qin
9d53b94546 rafthttp: add streaming server and client 2014-11-20 11:34:50 -08:00
Brian Waldon
9a728a127a dep: bump golang.org/x/net/context
Move from code.google.com/p/go.net/context to
golang.org/x/net/context before bumping to latest.
2014-11-20 10:19:12 -08:00
Ben Darnell
b29240baf0 Merge remote-tracking branch 'coreos/master' into merge
* coreos/master:
  scripts: build-docker tag and use ENTRYPOINT
  scripts: build-release add etcd-migrate
  create .godir
  raft: optimistically increase the next if the follower is already matched
  raft: add handleHeartbeat handleHeartbeat commits to the commit index in the message. It never decreases the commit index of the raft state machine.
  rafthttp: send takes raft message instead of bytes
  *: add rafthttp pkg into test list
  raft: include commitIndex in heartbeat
  rafthttp: move server stats in raftHandler to etcdserver
  *: etcdhttp.raftHandler -> rafthttp.RaftHandler
  etcdserver: rename sender.go -> sendhub.go
  *: etcdserver.sender -> rafthttp.Sender

Conflicts:
	raft/log.go
	raft/raft_paper_test.go
2014-11-19 17:05:16 -05:00
Ben Darnell
355ee4f393 raft: Integrate snapshots into the raft.Storage interface.
Compaction is now treated as an implementation detail of Storage
implementations; Node.Compact() and related functionality have been
removed. Ready.Snapshot is now used only for incoming snapshots.

A return value has been added to ApplyConfChange to allow applications
to track the node information that must be stored in the snapshot.

raftpb.Snapshot has been split into Snapshot and SnapshotMetadata, to
allow the full snapshot data to be read from disk only when needed.

raft.Storage has new methods Snapshot, ApplySnapshot, HardState, and
SetHardState. The Snapshot and HardState parameters have been removed
from RestartNode() and will now be loaded from Storage instead.
The only remaining difference between StartNode and RestartNode is that
the former bootstraps an initial list of Peers.
2014-11-19 16:40:26 -05:00
Yicheng Qin
a2c568a144 Merge pull request #1669 from yichengq/215
*: add rafthttp as a separate package
2014-11-17 16:14:59 -08:00
Yicheng Qin
f24e214ee5 rafthttp: move server stats in raftHandler to etcdserver 2014-11-17 16:02:20 -08:00
Yicheng Qin
5dc5f8145c *: etcdhttp.raftHandler -> rafthttp.RaftHandler 2014-11-17 15:52:24 -08:00
Ben Darnell
300c5a2001 Merge remote-tracking branch 'coreos/master' into log-storage-interface
* coreos/master: (21 commits)
  etcdserver: refactor ValidateClusterAndAssignIDs
  integration: add integration test for remove member
  integration: add test for member restart
  version: bump to alpha.3
  etcdserver: add buffer to the sender queue
  *: gracefully stop etcdserver
  Fix up migration tool, add snapshot migration
  etcd4: migration from v0.4 -> v0.5
  etcdserver: export Member.StoreKey
  etcdserver: recover cluster when receiving newer snapshot
  etcdserver: check and select committed entries to apply
  etcdserver: recover from snapshot before applying requests
  raft: not set applied when restored from snapshot
  sender: support elegant stop
  etcdserver: add StopNotify
  etcdserver: fix TestDoProposalStopped test
  etcdserver: minor cleanup
  etcdserver: validate new node is not registered before in best effort
  etcdserver: fix server.Stop()
  *: print out configuration when necessary
  ...

Conflicts:
	etcdserver/server.go
	etcdserver/server_test.go
	raft/log.go
2014-11-17 18:28:24 -05:00
Xiang Li
e04e4632b3 Merge pull request #1736 from xiang90/verify
etcdserver: refactor ValidateClusterAndAssignIDs
2014-11-17 14:34:59 -08:00
Xiang Li
0541f0afa0 etcdserver: refactor ValidateClusterAndAssignIDs 2014-11-17 14:23:37 -08:00
Xiang Li
c26de66262 integration: add integration test for remove member 2014-11-17 13:28:09 -08:00
Xiang Li
8bf71d796e *: gracefully stop etcdserver 2014-11-14 14:12:24 -08:00
Barak Michener
192f200d9e Fix up migration tool, add snapshot migration
Fixes all updates since bcwaldon sketched the original, with cleanup and
into an acutal working state. The commit log follows:

fix pb reference and remove unused file post rebase

unbreak the migrate folder

correctly detect node IDs

fix snapshotting

Fix previous broken snapshot

Add raft log entries to the translation; fix test for all timezones. (Still in progress, but passing)

Fix etcd:join and etcd:remove

print more data when dumping the log

Cleanup based on yichengq's comments

more comments

Fix the commited index based on the snapshot, if one exists

detect nodeIDs from snapshot

add initial tool documentation and match the semantics in the build script and main

formalize migration doc

rename function and clarify docs

fix nil pointer

fix the record conversion test

add migration to test suite and fix govet
2014-11-14 16:46:08 -05:00
Yicheng Qin
77433ff6da etcdserver: recover cluster when receiving newer snapshot 2014-11-14 12:11:21 -08:00
Yicheng Qin
dfaa7290c4 etcdserver: check and select committed entries to apply 2014-11-14 12:11:16 -08:00
Yicheng Qin
f6a7f96967 etcdserver: recover from snapshot before applying requests 2014-11-14 12:08:39 -08:00
Xiang Li
e66bda957b Merge pull request #1714 from xiang90/stop
StopNotify
2014-11-13 15:16:52 -08:00
Xiang Li
6a1fe00615 Merge pull request #1704 from xiang90/print_config
*: print out configuration when necessary
2014-11-13 14:35:50 -08:00
Yicheng Qin
11f392bdc8 Merge pull request #1708 from yichengq/223
etcdserver: validate new node is not registered before in best effort
2014-11-13 14:30:40 -08:00