593 Commits

Author SHA1 Message Date
Yicheng Qin
84fbf7aab5 *: etcdserver.sender -> rafthttp.Sender 2014-11-17 15:35:10 -08:00
Ben Darnell
300c5a2001 Merge remote-tracking branch 'coreos/master' into log-storage-interface
* coreos/master: (21 commits)
  etcdserver: refactor ValidateClusterAndAssignIDs
  integration: add integration test for remove member
  integration: add test for member restart
  version: bump to alpha.3
  etcdserver: add buffer to the sender queue
  *: gracefully stop etcdserver
  Fix up migration tool, add snapshot migration
  etcd4: migration from v0.4 -> v0.5
  etcdserver: export Member.StoreKey
  etcdserver: recover cluster when receiving newer snapshot
  etcdserver: check and select committed entries to apply
  etcdserver: recover from snapshot before applying requests
  raft: not set applied when restored from snapshot
  sender: support elegant stop
  etcdserver: add StopNotify
  etcdserver: fix TestDoProposalStopped test
  etcdserver: minor cleanup
  etcdserver: validate new node is not registered before in best effort
  etcdserver: fix server.Stop()
  *: print out configuration when necessary
  ...

Conflicts:
	etcdserver/server.go
	etcdserver/server_test.go
	raft/log.go
2014-11-17 18:28:24 -05:00
Xiang Li
e04e4632b3 Merge pull request #1736 from xiang90/verify
etcdserver: refactor ValidateClusterAndAssignIDs
2014-11-17 14:34:59 -08:00
Xiang Li
0541f0afa0 etcdserver: refactor ValidateClusterAndAssignIDs 2014-11-17 14:23:37 -08:00
Ben Darnell
64d9bcabf1 Add Storage.Term() method and hide the first entry from other methods.
The first entry in the log is a dummy which is used for matchTerm
but may not have an actual payload. This change permits Storage
implementations to treat this term value specially instead of
storing it as a dummy Entry.

Storage.FirstIndex() no longer includes the term-only entry.

This reverses a recent decision to create entry zero as initially
unstable; Storage implementations are now required to make
Term(0) == 0 and the first unstable entry is now index 1.
stableTo(0) is no longer allowed.
2014-11-17 16:54:12 -05:00
Xiang Li
c26de66262 integration: add integration test for remove member 2014-11-17 13:28:09 -08:00
Xiang Li
7c4b84a6cd etcdserver: add buffer to the sender queue 2014-11-14 15:18:16 -08:00
Xiang Li
8bf71d796e *: gracefully stop etcdserver 2014-11-14 14:12:24 -08:00
Barak Michener
192f200d9e Fix up migration tool, add snapshot migration
Fixes all updates since bcwaldon sketched the original, with cleanup and
into an acutal working state. The commit log follows:

fix pb reference and remove unused file post rebase

unbreak the migrate folder

correctly detect node IDs

fix snapshotting

Fix previous broken snapshot

Add raft log entries to the translation; fix test for all timezones. (Still in progress, but passing)

Fix etcd:join and etcd:remove

print more data when dumping the log

Cleanup based on yichengq's comments

more comments

Fix the commited index based on the snapshot, if one exists

detect nodeIDs from snapshot

add initial tool documentation and match the semantics in the build script and main

formalize migration doc

rename function and clarify docs

fix nil pointer

fix the record conversion test

add migration to test suite and fix govet
2014-11-14 16:46:08 -05:00
Brian Waldon
5ea1f2d96f etcd4: migration from v0.4 -> v0.5 2014-11-14 15:57:26 -05:00
Brian Waldon
c36abeabd1 etcdserver: export Member.StoreKey 2014-11-14 15:57:26 -05:00
Yicheng Qin
b6887e4a83 Merge pull request #1719 from yichengq/228
etcdserver: recover snapshot before applying committed entries
2014-11-14 12:18:41 -08:00
Yicheng Qin
77433ff6da etcdserver: recover cluster when receiving newer snapshot 2014-11-14 12:11:21 -08:00
Yicheng Qin
dfaa7290c4 etcdserver: check and select committed entries to apply 2014-11-14 12:11:16 -08:00
Yicheng Qin
f6a7f96967 etcdserver: recover from snapshot before applying requests 2014-11-14 12:08:39 -08:00
Yicheng Qin
12dba7d413 sender: support elegant stop 2014-11-13 17:44:36 -08:00
Xiang Li
e66bda957b Merge pull request #1714 from xiang90/stop
StopNotify
2014-11-13 15:16:52 -08:00
Xiang Li
6a1fe00615 Merge pull request #1704 from xiang90/print_config
*: print out configuration when necessary
2014-11-13 14:35:50 -08:00
Yicheng Qin
11f392bdc8 Merge pull request #1708 from yichengq/223
etcdserver: validate new node is not registered before in best effort
2014-11-13 14:30:40 -08:00
Xiang Li
b5d480f17a etcdserver: add StopNotify 2014-11-13 14:16:48 -08:00
Xiang Li
978d0f1ca1 etcdserver: fix TestDoProposalStopped test
We start etcd server in this test without the cluster. Sometimes it panics when
accessing the cluster. Most of the time it does not panic, since we can stop the
server fast enough before applying the first configuration change entry.
2014-11-13 14:08:59 -08:00
Xiang Li
fb344bc33f etcdserver: minor cleanup 2014-11-13 14:01:56 -08:00
Xiang Li
813ff6ba48 Merge pull request #1713 from xiang90/stop
etcdserver: fix server.Stop()
2014-11-13 13:58:07 -08:00
Yicheng Qin
ac907d746b etcdserver: validate new node is not registered before in best effort 2014-11-13 13:56:11 -08:00
Xiang Li
30dfdb0ea9 etcdserver: fix server.Stop()
Stop should be idempotent. It should simply send a stop signal to the server.
It is the server's responsibility to stop the go-routines and related components.
2014-11-13 13:47:12 -08:00
Ben Darnell
39eddd8565 Merge remote-tracking branch 'coreos/master' into log-storage-interface
* coreos/master:
  etcdserver: add sender tests
  raft: Only call stableTo when we have ready entries or a snapshot.
  etcdserver: add ID() function to the Server interface.
  sender: use RoundTripper instead of Client in sender
2014-11-13 15:50:08 -05:00
Yicheng Qin
9716def94b Merge pull request #1700 from yichengq/222
etcdserver: add sender tests
2014-11-13 12:37:29 -08:00
Yicheng Qin
d89bf9f215 etcdserver: add sender tests 2014-11-13 12:29:25 -08:00
Xiang Li
d6f40acc86 etcdserver: add ID() function to the Server interface. 2014-11-13 11:37:06 -08:00
Ben Darnell
b29c512f50 Merge remote-tracking branch 'coreos/master' into log-storage-interface
* coreos/master: (27 commits)
  pkg/wait: move wait to pkg/wait
  etcdserver: do not add/remove/update local member to/from sender hub
  etcdserver: not record attributes when add member
  raft: add a test for proposeConfChange
  raft: block Stop() on n.done, support idempotency
  raft: add a test for node proposal
  integration: add increase cluster size test
  integration: remove unnecessary t.Testing argument
  raft: stop the node synchronously
  integration: fix test to propagate NewServer errors
  etcdserver: move peer URLs check to config
  etcdserver: ensure initial-advertise-peer-urls match initial-cluster
  raft: add a test for node.Tick
  raft: add comment string for TestNodeStart
  etcdserver: use member instead of node at etcd level
  raft: nodes return sorted ids
  raft: update unstable when calling stableTo with 0
  *: support updating advertise-peer-url Users might want to update the peerurl of the etcd member in several cases. For example, if the IP address of the physical machine etcd running on is changed, user need to update the adversite-pee-rurl accordingly. This commit makes etcd support updating the advertise-peer-url of its members.
  transport: create a tls listener only if the tlsInfo is not empty and the scheme is HTTPS
  etcdserver: use member pointer for all tests
  ...

Conflicts:
	etcdserver/server.go
	raft/log.go
	raft/log_test.go
	raft/node.go
2014-11-13 14:21:09 -05:00
Xiang Li
92096dfdc3 *: print out configuration when necessary 2014-11-13 10:46:42 -08:00
Xiang Li
0d18a0f381 pkg/wait: move wait to pkg/wait 2014-11-13 09:11:53 -08:00
Yicheng Qin
23b5bc0dfe sender: use RoundTripper instead of Client in sender 2014-11-12 21:42:08 -08:00
Yicheng Qin
1e1535e6f9 Merge pull request #1620 from yichengq/204
etcdserver: not record attributes when add member
2014-11-12 21:33:53 -08:00
Xiang Li
ba915ad5a8 etcdserver: do not add/remove/update local member to/from sender hub 2014-11-12 20:45:21 -08:00
Yicheng Qin
0c2b45ddc6 etcdserver: not record attributes when add member
There is no need to set attributes value when adding member because new
member will publish the information whenever it starts.
2014-11-12 17:48:15 -08:00
Ben Darnell
54b07d7974 Remove raft.loadEnts and the ents parameter to raft.RestartNode.
The initial entries are now provided via the Storage interface.
2014-11-12 18:31:19 -05:00
Ben Darnell
147fd614ce The initial term=0 log entry is now initially unstable.
This entry is now persisted through the normal flow instead of appearing
in the stored log at creation time.  This is how things worked before
the Storage interface was introduced. (see coreos/etcd#1689)
2014-11-12 18:24:16 -05:00
Jonathan Boulle
1197c1f965 etcdserver: move peer URLs check to config 2014-11-12 13:12:49 -08:00
Jonathan Boulle
3f358b6d5d etcdserver: ensure initial-advertise-peer-urls match initial-cluster
This adds a check to setupCluster to ensure that the list of URLs
specified in `initial-advertise-peer-urls` matches those configured in
`initial-cluster` for this node. Also updates the documentation to
clarify this and address some changes in wording.
2014-11-12 12:54:35 -08:00
Xiang Li
0aa8258d29 etcdserver: use member instead of node at etcd level 2014-11-12 10:45:35 -08:00
Xiang Li
5967794009 *: support updating advertise-peer-url
Users might want to update the peerurl of the etcd member in several cases.
For example, if the IP address of the physical machine etcd running on is
changed, user need to update the adversite-pee-rurl accordingly.
This commit makes etcd support updating the advertise-peer-url of its members.
2014-11-11 12:07:03 -08:00
Xiang Li
67a0de4bbc etcdserver: use member pointer for all tests 2014-11-11 11:20:56 -08:00
Xiang Li
e4931e0c47 etcdserver: remove unnecessary newTestMemberp 2014-11-11 11:09:33 -08:00
Xiang Li
077e144e8a etcdserver: move newTestMember* to member_test.go 2014-11-11 11:02:50 -08:00
Ben Darnell
25b6590547 raft: introduce log storage interface.
This change splits the raftLog.entries array into an in-memory
"unstable" list and a pluggable interface for retrieving entries that
have been persisted to disk. An in-memory implementation of this
interface is provided which behaves the same as the old version;
in a future commit etcdserver could replace the MemoryStorage with
one backed by the WAL.
2014-11-10 17:40:39 -05:00
Jonathan Boulle
41757e7f78 etcdserver: collapse shared readWAL logic 2014-11-08 17:07:05 -08:00
Yicheng Qin
4b9c3a9102 etcdserver: not get cluster info from self peer urls
Self peer urls have not started to serve at the time that it tries to
get cluster info, so it is useless to get cluster info from self peer
urls.
2014-11-08 13:52:48 -08:00
Yicheng Qin
014ef0f52d etcdserver: fix data race in cluster
The data race happens when etcd updates member attributes and fetches
member info in http handler at the same time.
2014-11-07 16:13:07 -08:00
Yicheng Qin
46cbfbc630 etcdserver: not add sender when it has existed 2014-11-07 14:05:00 -08:00