6545 Commits

Author SHA1 Message Date
Anthony Romano
8346a7c052 Merge pull request #4094 from heyitsanthony/send-merged-done-nowait
etcdserver: respect done channel when sleeping for snapshot backoff
2015-12-29 20:18:08 -08:00
Anthony Romano
942b5570bd Merge pull request #4096 from heyitsanthony/serialize-applier-snapmerge
etcdserver: serialize snapshot merger with applier
2015-12-29 19:37:11 -08:00
Anthony Romano
4cd86ae1ef etcdserver: serialize snapshot merger with applier
Avoids inconsistent snapshotting by only attempting to
create a snapshot after an apply completes.

Fixes #4061
2015-12-29 18:38:39 -08:00
Gyu-Ho Lee
04ac8969a1 Merge pull request #3986 from mqliang/defer
refactor store/store.go
2015-12-29 14:42:11 -07:00
Xiang Li
4b355bd81e Merge pull request #4093 from xiang90/rm_proto
doc: remove proto in rfc, link to proto file in codebase
2015-12-29 13:25:46 -08:00
Anthony Romano
c7c3bda8b9 etcdserver: respect done channel when sleeping for snapshot backoff 2015-12-29 13:23:41 -08:00
Xiang Li
9d9680121c etcdserver/etcdserverpb: make rpc.proto updated 2015-12-29 12:58:14 -08:00
Xiang Li
39ed73c290 doc: remove proto in rfc, link to proto file in codebase 2015-12-29 12:53:27 -08:00
Xiang Li
d9d7137ea3 Merge pull request #4092 from xiang90/api
*: update api proto
2015-12-29 12:46:12 -08:00
Xiang Li
1a0201a31a *: update api proto 2015-12-29 12:31:53 -08:00
Xiang Li
d62edfb464 Merge pull request #4090 from xiang90/writable
etcdserver: always check if the data dir is writable before starting
2015-12-29 11:40:41 -08:00
Xiang Li
150e646b05 etcdserver: always check if the data dir is writable before starting etcd 2015-12-29 11:29:01 -08:00
Xiang Li
f1761798e9 Merge pull request #4089 from xiang90/fix
etcdserver: fix creating member dir
2015-12-28 23:37:50 -08:00
Anthony Romano
0f2675d9b6 Merge pull request #4060 from heyitsanthony/fix-etcdctl-err
etcdctl: return exitcode 2 if can't connect to any peer urls
2015-12-28 23:28:29 -08:00
Xiang Li
aef55342d1 etcdsever: avoid creating member dir before finishing validate bootstrap
This commit fixes the issue of creating member dir before validating
the configuration. When member dir exists, it indicates the local etcd
process is a valid etcd member. So we should only create member dir
after we finish configuration validation, joining validation or
discovery validation.
2015-12-28 23:20:27 -08:00
Xiang Li
eae52a3138 Revert "etcdserver: always remove member directory when bootstrap fails"
This reverts commit a7e443d6412ec405948571d1f758ab7f1c078bf9.
2015-12-28 23:16:08 -08:00
Gyu-Ho Lee
cd42c9139e Merge pull request #4087 from gyuho/delete_discovery_check
etcdserver: always remove member directory when bootstrap fails
2015-12-29 00:04:37 -07:00
Anthony Romano
4477ef636e etcdctl: return exitcode 2 if can't connect to any peers 2015-12-28 23:04:34 -08:00
Gyu-Ho Lee
a7e443d641 etcdserver: always remove member directory when bootstrap fails
This removes member directory when bootstrap fails including joining existing
cluster and forming a new cluster. This fixes https://github.com/coreos/etcd/issues/3827.
2015-12-28 22:56:03 -08:00
Gyu-Ho Lee
c3655cbfd9 Merge pull request #4084 from gyuho/revisioin_document
storage: add/fix revision/generation description
2015-12-28 20:45:36 -07:00
Gyu-Ho Lee
5284c1c3bb storage: add/fix revision/generation description
This adds some struct descriptions to revision/generation structs and corrects
some spelling errors.
2015-12-28 19:32:03 -08:00
Xiang Li
f3b3557951 Merge pull request #4085 from gyuho/doc_iana
Documentation: clarification on official etcd ports
2015-12-28 18:13:25 -08:00
Gyu-Ho Lee
7c587e0855 Documentation: clarification on official etcd ports
This is for https://github.com/coreos/etcd/issues/3480.
This adds a notice that etcd officially use 2379 and 2380 for
client and server ports.
2015-12-28 16:29:19 -08:00
Gyu-Ho Lee
6c5dc28d0f Merge pull request #4043 from gyuho/storage_range_all_unsynced
storage: range all unsynced at once
2015-12-28 14:45:40 -07:00
Gyu-Ho Lee
0d7fb820c7 Merge pull request #4082 from gyuho/storage_test_cleanup
storage: clean up test variable names, minor typos in comments
2015-12-28 13:02:17 -07:00
Gyu-Ho Lee
d64eb94580 storage: clean up test variable names, minor typos in comments
This just changes variable name to be more consistent: `N` rather than `Size`.
And fix some minor grammatical errors.
2015-12-28 11:49:57 -08:00
Gyu-Ho Lee
ecc3e15a46 storage: delete RangeHistory
This has been replaced by operations inside `syncWatchings`.
2015-12-28 11:37:32 -08:00
Gyu-Ho Lee
78b0b8a4a0 storage: range all unsynced at once
This is for https://github.com/coreos/etcd/issues/3848.
It replaces RangeHistory method for more efficient event
sending.
2015-12-28 11:37:26 -08:00
Anthony Romano
570687a509 Merge pull request #4081 from heyitsanthony/benchmark-multi-ep
tools/benchmark: support connecting to several endpoints
2015-12-28 10:31:35 -08:00
Anthony Romano
8e728afa62 tools/benchmark: support connecting to several endpoints
--endpoints is comma separated but gRPC blocks forever on comma
separated lists. Instead, round-robin select endpoints when
creating new connections.
2015-12-28 10:22:33 -08:00
Xiang Li
d07a9cd893 Merge pull request #4059 from xiang90/snap_log
rafthttp: better snapshot sending logging
2015-12-28 10:05:35 -08:00
Gyu-Ho Lee
466b33445f Merge pull request #4079 from gyuho/store_shadow
store: fix govet shadow on expiration variable
2015-12-28 11:03:57 -07:00
Gyu-Ho Lee
b072f0b048 store: fix expiration var shadowing, change test function names
Found at https://travis-ci.org/coreos/etcd/jobs/99087279#L298.
And changes test function names to make them clear.
2015-12-28 08:50:34 -08:00
Xiang Li
aec356e416 Merge pull request #4064 from xiang90/reduce_fysnc
backend: do not commit unless there is a pending change
2015-12-28 07:31:50 -08:00
Jonathan Boulle
1238187b72 Merge pull request #4078 from jonboulle/master
docs: clarify CAS/CAD do not work on directories
2015-12-27 22:35:51 +01:00
Jonathan Boulle
216c6674ed docs: clarify CAS/CAD do not work on directories
As noted in #4075. There are numerous style issues with this document
but I don't want to go down that rabbit hole so this is an attempt at a
minimally invasive clarification.
2015-12-27 22:35:08 +01:00
Gyu-Ho Lee
729b530c48 Merge pull request #4071 from gyuho/store_event_node
store: clean up event.go, node.go and add tests
2015-12-25 21:42:50 -07:00
Gyu-Ho Lee
64e182c69e store: clean up event.go, node.go and add tests
Updates IsCreated logic on event.go. Cleans up node.go
and adds tests to it.
2015-12-25 13:25:12 -08:00
Jonathan Boulle
70bcde89bc Merge pull request #4073 from gyuho/remove_seed
storage: remove unnecessary math/rand seed
2015-12-25 19:14:35 +01:00
Gyu-Ho Lee
df0c2e6842 storage: remove unnecessary math/rand seed
As @jonboulle pointed out at
https://github.com/coreos/etcd/pull/4070/files#r48441847:

> math/rand is unrelated to crypto/rand; the latter reads from /dev/urandom and
> is relying on the kernel's PRNG. Just remove the seed entirely.
2015-12-25 09:55:11 -08:00
Jonathan Boulle
dac56faf61 Merge pull request #4030 from mitake/endpoint-selection
client: add a mechanism for various endpoint selection mode
2015-12-24 12:55:16 +01:00
Jonathan Boulle
ff319add53 Merge pull request #4066 from gyuho/tip_shadow
pkg/fileutil: fix error var shadow
2015-12-24 10:24:27 +01:00
Gyu-Ho Lee
8d368c4dba pkg/fileutil: fix error var shadow
Go tip complains about error variable shadowing at
https://travis-ci.org/coreos/etcd/jobs/98636879#L291-L292.
2015-12-23 23:56:26 -08:00
Xiang Li
22b3b3e07a Merge pull request #4038 from AkihiroSuda/etcd-4007
pkg/fileutil: skip TestIsDirWriteable when running as root
2015-12-23 22:23:57 -08:00
Akihiro Suda
058f1449d6 pkg/fileutil: skip TestIsDirWriteable when running as root 2015-12-24 14:52:40 +09:00
Xiang Li
8bc59b66d1 backend: do not commit unless there is a pending change
Reduce the nubmer of fsync etcd issues when the cluster is
idle.
2015-12-23 18:58:37 -08:00
Hitoshi Mitake
a46ffc60e5 client: add a mechanism for various endpoint selection mode
Current etcd client library chooses a default destination node from
every member of a cluster in a random manner. However, requests of
write and read (for consistent results) need to be forwarded to the
leader node as the nature of Raft algorithm. If the chosen node is a
follower, additional network traffic will be caused by the forwarding
from follower to leader.

Mainly for reducing the forward traffic, this commit adds a new
mechanism for various endpoint selection mode to the client library
which can be configured with client.Config.SelectionMode.

Currently, two modes are provided:
 - EndpointSelectionRandom: default, same to existing behavior (pick
   a node in a random manner)
 - EndpointSelectionPrioritizeLeader: prioritize leader, for the above
   purpose

I evaluated the effectiveness of the EndpointSelectionPrioritizeLeader
with 4 t1.micro instances of AWS (3 nodes for etcd cluster and 1 node
for etcd client). Client executes this simple benchmark
(https://github.com/mitake/etcd-things/tree/master/prioritize-leader-bench),
just writes 10000 keys. When SelectionMode == EndpointSelectionRandom
(default), the benchmark needed 1 min and 32.102 sec to finish. When
SelectionMode == EndpointSelectionPrioritizeLeader, the benchmark
needed 1 min 4.760 sec.
2015-12-24 11:02:40 +09:00
Xiang Li
72e115ee6e Merge pull request #4062 from xiang90/fix_snap
*: fix snapshot sending cycle
2015-12-23 17:10:10 -08:00
Anthony Romano
3c17e45bcb Merge pull request #4063 from heyitsanthony/fix-shouldstop
etcdserver: stop if removed along with multiple conf changes
2015-12-23 16:46:18 -08:00
Anthony Romano
d7ad721ede etcdserver: stop if removed along with multiple conf changes
shouldstop would get clobbered when several conf changes are in an apply
2015-12-23 16:29:21 -08:00