385 Commits

Author SHA1 Message Date
Brian Waldon
c0fb1c8a00 Merge pull request #1755 from bcwaldon/golang.org-deps
Switch to golang.org/x/net/context
2014-11-20 16:26:14 -08:00
Barak Michener
59a0c64e9f fix import loop, add set to types, and fix comments 2014-11-20 15:38:08 -05:00
Barak Michener
78ea3335bf etcdserver: autodetect v0.4 WALs and upgrade them to v0.5 automatically 2014-11-20 15:38:08 -05:00
Yicheng Qin
9d53b94546 rafthttp: add streaming server and client 2014-11-20 11:34:50 -08:00
Brian Waldon
9a728a127a dep: bump golang.org/x/net/context
Move from code.google.com/p/go.net/context to
golang.org/x/net/context before bumping to latest.
2014-11-20 10:19:12 -08:00
Ben Darnell
b29240baf0 Merge remote-tracking branch 'coreos/master' into merge
* coreos/master:
  scripts: build-docker tag and use ENTRYPOINT
  scripts: build-release add etcd-migrate
  create .godir
  raft: optimistically increase the next if the follower is already matched
  raft: add handleHeartbeat handleHeartbeat commits to the commit index in the message. It never decreases the commit index of the raft state machine.
  rafthttp: send takes raft message instead of bytes
  *: add rafthttp pkg into test list
  raft: include commitIndex in heartbeat
  rafthttp: move server stats in raftHandler to etcdserver
  *: etcdhttp.raftHandler -> rafthttp.RaftHandler
  etcdserver: rename sender.go -> sendhub.go
  *: etcdserver.sender -> rafthttp.Sender

Conflicts:
	raft/log.go
	raft/raft_paper_test.go
2014-11-19 17:05:16 -05:00
Ben Darnell
355ee4f393 raft: Integrate snapshots into the raft.Storage interface.
Compaction is now treated as an implementation detail of Storage
implementations; Node.Compact() and related functionality have been
removed. Ready.Snapshot is now used only for incoming snapshots.

A return value has been added to ApplyConfChange to allow applications
to track the node information that must be stored in the snapshot.

raftpb.Snapshot has been split into Snapshot and SnapshotMetadata, to
allow the full snapshot data to be read from disk only when needed.

raft.Storage has new methods Snapshot, ApplySnapshot, HardState, and
SetHardState. The Snapshot and HardState parameters have been removed
from RestartNode() and will now be loaded from Storage instead.
The only remaining difference between StartNode and RestartNode is that
the former bootstraps an initial list of Peers.
2014-11-19 16:40:26 -05:00
Yicheng Qin
a2c568a144 Merge pull request #1669 from yichengq/215
*: add rafthttp as a separate package
2014-11-17 16:14:59 -08:00
Yicheng Qin
f24e214ee5 rafthttp: move server stats in raftHandler to etcdserver 2014-11-17 16:02:20 -08:00
Yicheng Qin
5dc5f8145c *: etcdhttp.raftHandler -> rafthttp.RaftHandler 2014-11-17 15:52:24 -08:00
Ben Darnell
300c5a2001 Merge remote-tracking branch 'coreos/master' into log-storage-interface
* coreos/master: (21 commits)
  etcdserver: refactor ValidateClusterAndAssignIDs
  integration: add integration test for remove member
  integration: add test for member restart
  version: bump to alpha.3
  etcdserver: add buffer to the sender queue
  *: gracefully stop etcdserver
  Fix up migration tool, add snapshot migration
  etcd4: migration from v0.4 -> v0.5
  etcdserver: export Member.StoreKey
  etcdserver: recover cluster when receiving newer snapshot
  etcdserver: check and select committed entries to apply
  etcdserver: recover from snapshot before applying requests
  raft: not set applied when restored from snapshot
  sender: support elegant stop
  etcdserver: add StopNotify
  etcdserver: fix TestDoProposalStopped test
  etcdserver: minor cleanup
  etcdserver: validate new node is not registered before in best effort
  etcdserver: fix server.Stop()
  *: print out configuration when necessary
  ...

Conflicts:
	etcdserver/server.go
	etcdserver/server_test.go
	raft/log.go
2014-11-17 18:28:24 -05:00
Xiang Li
e04e4632b3 Merge pull request #1736 from xiang90/verify
etcdserver: refactor ValidateClusterAndAssignIDs
2014-11-17 14:34:59 -08:00
Xiang Li
0541f0afa0 etcdserver: refactor ValidateClusterAndAssignIDs 2014-11-17 14:23:37 -08:00
Xiang Li
c26de66262 integration: add integration test for remove member 2014-11-17 13:28:09 -08:00
Xiang Li
8bf71d796e *: gracefully stop etcdserver 2014-11-14 14:12:24 -08:00
Barak Michener
192f200d9e Fix up migration tool, add snapshot migration
Fixes all updates since bcwaldon sketched the original, with cleanup and
into an acutal working state. The commit log follows:

fix pb reference and remove unused file post rebase

unbreak the migrate folder

correctly detect node IDs

fix snapshotting

Fix previous broken snapshot

Add raft log entries to the translation; fix test for all timezones. (Still in progress, but passing)

Fix etcd:join and etcd:remove

print more data when dumping the log

Cleanup based on yichengq's comments

more comments

Fix the commited index based on the snapshot, if one exists

detect nodeIDs from snapshot

add initial tool documentation and match the semantics in the build script and main

formalize migration doc

rename function and clarify docs

fix nil pointer

fix the record conversion test

add migration to test suite and fix govet
2014-11-14 16:46:08 -05:00
Yicheng Qin
77433ff6da etcdserver: recover cluster when receiving newer snapshot 2014-11-14 12:11:21 -08:00
Yicheng Qin
dfaa7290c4 etcdserver: check and select committed entries to apply 2014-11-14 12:11:16 -08:00
Yicheng Qin
f6a7f96967 etcdserver: recover from snapshot before applying requests 2014-11-14 12:08:39 -08:00
Xiang Li
e66bda957b Merge pull request #1714 from xiang90/stop
StopNotify
2014-11-13 15:16:52 -08:00
Xiang Li
6a1fe00615 Merge pull request #1704 from xiang90/print_config
*: print out configuration when necessary
2014-11-13 14:35:50 -08:00
Yicheng Qin
11f392bdc8 Merge pull request #1708 from yichengq/223
etcdserver: validate new node is not registered before in best effort
2014-11-13 14:30:40 -08:00
Xiang Li
b5d480f17a etcdserver: add StopNotify 2014-11-13 14:16:48 -08:00
Xiang Li
fb344bc33f etcdserver: minor cleanup 2014-11-13 14:01:56 -08:00
Yicheng Qin
ac907d746b etcdserver: validate new node is not registered before in best effort 2014-11-13 13:56:11 -08:00
Xiang Li
30dfdb0ea9 etcdserver: fix server.Stop()
Stop should be idempotent. It should simply send a stop signal to the server.
It is the server's responsibility to stop the go-routines and related components.
2014-11-13 13:47:12 -08:00
Ben Darnell
39eddd8565 Merge remote-tracking branch 'coreos/master' into log-storage-interface
* coreos/master:
  etcdserver: add sender tests
  raft: Only call stableTo when we have ready entries or a snapshot.
  etcdserver: add ID() function to the Server interface.
  sender: use RoundTripper instead of Client in sender
2014-11-13 15:50:08 -05:00
Xiang Li
d6f40acc86 etcdserver: add ID() function to the Server interface. 2014-11-13 11:37:06 -08:00
Ben Darnell
b29c512f50 Merge remote-tracking branch 'coreos/master' into log-storage-interface
* coreos/master: (27 commits)
  pkg/wait: move wait to pkg/wait
  etcdserver: do not add/remove/update local member to/from sender hub
  etcdserver: not record attributes when add member
  raft: add a test for proposeConfChange
  raft: block Stop() on n.done, support idempotency
  raft: add a test for node proposal
  integration: add increase cluster size test
  integration: remove unnecessary t.Testing argument
  raft: stop the node synchronously
  integration: fix test to propagate NewServer errors
  etcdserver: move peer URLs check to config
  etcdserver: ensure initial-advertise-peer-urls match initial-cluster
  raft: add a test for node.Tick
  raft: add comment string for TestNodeStart
  etcdserver: use member instead of node at etcd level
  raft: nodes return sorted ids
  raft: update unstable when calling stableTo with 0
  *: support updating advertise-peer-url Users might want to update the peerurl of the etcd member in several cases. For example, if the IP address of the physical machine etcd running on is changed, user need to update the adversite-pee-rurl accordingly. This commit makes etcd support updating the advertise-peer-url of its members.
  transport: create a tls listener only if the tlsInfo is not empty and the scheme is HTTPS
  etcdserver: use member pointer for all tests
  ...

Conflicts:
	etcdserver/server.go
	raft/log.go
	raft/log_test.go
	raft/node.go
2014-11-13 14:21:09 -05:00
Xiang Li
92096dfdc3 *: print out configuration when necessary 2014-11-13 10:46:42 -08:00
Xiang Li
0d18a0f381 pkg/wait: move wait to pkg/wait 2014-11-13 09:11:53 -08:00
Xiang Li
ba915ad5a8 etcdserver: do not add/remove/update local member to/from sender hub 2014-11-12 20:45:21 -08:00
Ben Darnell
54b07d7974 Remove raft.loadEnts and the ents parameter to raft.RestartNode.
The initial entries are now provided via the Storage interface.
2014-11-12 18:31:19 -05:00
Xiang Li
0aa8258d29 etcdserver: use member instead of node at etcd level 2014-11-12 10:45:35 -08:00
Xiang Li
5967794009 *: support updating advertise-peer-url
Users might want to update the peerurl of the etcd member in several cases.
For example, if the IP address of the physical machine etcd running on is
changed, user need to update the adversite-pee-rurl accordingly.
This commit makes etcd support updating the advertise-peer-url of its members.
2014-11-11 12:07:03 -08:00
Ben Darnell
25b6590547 raft: introduce log storage interface.
This change splits the raftLog.entries array into an in-memory
"unstable" list and a pluggable interface for retrieving entries that
have been persisted to disk. An in-memory implementation of this
interface is provided which behaves the same as the old version;
in a future commit etcdserver could replace the MemoryStorage with
one backed by the WAL.
2014-11-10 17:40:39 -05:00
Jonathan Boulle
41757e7f78 etcdserver: collapse shared readWAL logic 2014-11-08 17:07:05 -08:00
Yicheng Qin
4b9c3a9102 etcdserver: not get cluster info from self peer urls
Self peer urls have not started to serve at the time that it tries to
get cluster info, so it is useless to get cluster info from self peer
urls.
2014-11-08 13:52:48 -08:00
Yicheng Qin
014ef0f52d etcdserver: fix data race in cluster
The data race happens when etcd updates member attributes and fetches
member info in http handler at the same time.
2014-11-07 16:13:07 -08:00
Jonathan Boulle
ca06fd0060 etcdserver: log cluster when adding/removing node 2014-11-07 13:36:41 -08:00
Jonathan Boulle
958ade86a5 etcdserver: log message after loading peers from snapshot 2014-11-07 13:34:43 -08:00
Jonathan Boulle
285cd404e3 etcdserver: print peerURLs when adding member 2014-11-07 12:00:41 -08:00
Jonathan Boulle
5055863e09 etcdserver: add docstrings for confchanges 2014-11-07 10:19:55 -08:00
Xiang Li
bf47fe7cac Merge pull request #1647 from xiangli-cmu/force_cluster
etcdserver: force new cluster
2014-11-07 10:15:53 -08:00
Yicheng Qin
9d19429993 Merge pull request #1609 from yichengq/202
etcdserver: refactor sender
2014-11-07 10:12:02 -08:00
Xiang Li
0a9c6164af etcdserver: add support for force cluster 2014-11-07 08:49:01 -08:00
Jonathan Boulle
8f1885a398 discovery: add command line flag for discovery-proxy 2014-11-06 16:35:24 -08:00
Xiang Li
c5e6053fcd Merge pull request #1638 from xiangli-cmu/better_logging
etcdserver: better logging for clusterFromPeerURLs
2014-11-06 14:33:53 -08:00
Xiang Li
eb0d80767e etcdserver: better logging for clusterFromPeerURLs 2014-11-06 14:28:07 -08:00
Yicheng Qin
457b30e585 etcdserver: add/remove sender in sendhub explicitly 2014-11-06 14:04:14 -08:00