411 Commits

Author SHA1 Message Date
Xiang Li
f0dbd0b856 Merge pull request #4646 from xiang90/starvation
etcdserver: detect raft stravation
2016-02-29 19:58:17 -08:00
Anthony Romano
1d6ebdd35c pkg/timeutil: removal
Overkill of a package for three lines of code.
2016-02-29 17:07:24 -08:00
Xiang Li
d6520303c6 etcdserver: detect raft starvation caused by contention 2016-02-29 17:06:57 -08:00
Xiang Li
d265fe000c *: support time based auto compaction.
Fix https://github.com/coreos/etcd/issues/3906.

We will have extensive doc to talk about what is compaction
and what is auto compaction soon.
2016-02-25 16:02:03 -08:00
Anthony Romano
c5b51946eb *: exported godoc fixups 2016-02-21 20:36:44 -08:00
Anthony Romano
616f395920 etcdserver: use fifo scheduler for applier 2016-02-11 19:21:30 -08:00
Caleb Champlin
82778ed478 Add refresh parameter to allow TTL refreshes without firing watch/wait responses 2016-02-08 10:37:37 -07:00
Anthony Romano
fe7cfe4d3d rafthttp: plumb local peer URLs through transport 2016-02-05 16:44:41 -08:00
Xiang Li
e44e753e66 pkg/idutil: reduce conflict rate from 1% to 0.005%
Perviously, we only use 8bits from member identification
in id generation. The conflict rate is A(256,3)/256^3, which
is around 1%. Now we use 16bites to reduce the rate to 0.005%.

We can attach the full member id into id generation if needed...
2016-02-04 13:25:18 -08:00
Anthony Romano
9ae8d85049 integration: decrease timeout for isMemberBootstrapped
Spending seconds(!) when it would fail anyway.

integration/TestV3 (before): 100.670
integration/TestV3 (after): 29.571
2016-02-02 14:34:58 -08:00
Anthony Romano
20461ab11a *: fix many typos 2016-01-31 21:42:39 -08:00
Heungsub Lee
c4a0159601 documentation: fix typo "a etcd" -> "an etcd"
"a" is not a correct article for "etcd".
2016-01-28 19:19:32 +09:00
Anthony Romano
bd02d668c8 etcdserver: don't try to apply empty message list
If all messages have been applied, don't apply an empty messages list;
otherwise appliedi will update to 0 and etcd will panic.

Fixes #4278
2016-01-26 11:56:37 -08:00
Anthony Romano
12f6b8e72d etcdserver: complete stopWithDelay on server shutdown
Was causing goroutine leaks on my machine.
2016-01-25 19:45:29 -08:00
Xiang Li
ef6320e638 etcdserver: make cluster checking interval shorter 2016-01-25 08:16:05 +08:00
Xiang Li
1aa312fcce *: lease forwarding should resue transport 2016-01-25 06:56:07 +08:00
Anthony Romano
2e157530a0 etcdhttp, lease, v3api: forward keepalives to leader
keepalives don't go through raft so let follower peers announce
keepalives to the leader through the peer http handler
2016-01-22 12:40:40 -08:00
Anthony Romano
9113a27bde lease: grant consistent lease IDs
When raft broadcasts a Grant to all nodes, all nodes must
agree on the same lease ID. Otherwise, attaching a key to
a lease will fail since the lease ID is node-dependent.
2016-01-22 09:43:39 -08:00
Hitoshi Mitake
588f655b4e etcdmain: add an option for pprof
This commit adds a new option for activating profiling based on pprof
in etcd process.
 - -enable-pprof: boolean type option which activates profiling

For example, if a client URL is http://localhost:12379, users and
developers access heap profiler with this URL:
http://localhost:12379/debug/pprof/heap
2016-01-13 16:12:26 +09:00
Xiang Li
6b1d9fb7ce *: stop lessor when etcdserver is stopped 2016-01-12 10:06:11 -08:00
Xiang Li
f5753f2f51 *: support lease Attach
Now we can attach keys to leases. And revoking the lease removes all
the attached keys of that lease.
2016-01-09 11:01:58 -08:00
Xiang Li
2566699a48 *: revoke expired leases 2016-01-08 13:37:58 -08:00
Gyu-Ho Lee
f76166a041 *: fix minor typos 2016-01-08 00:21:19 -08:00
Xiang Li
d9ca929a33 *: add support for lease create and revoke
Basic support for lease operations like create and revoke.
We still need to:
1. attach keys to leases in KV implmentation if lease field is set
2. leader periodically removes expired leases
3. leader serves keepAlive requests and follower forwards keepAlive
requests to leader.
2016-01-07 16:39:39 -08:00
Xiang Li
1714290f4e storage: support recovering from backend
We want the KV to support recovering from backend to avoid
additional pointer swap. Or we have to do coordination between
etcdserver and API layer, since API layer might have access to
kv pointer and use a closed kv.
2016-01-06 21:16:55 -08:00
Xiang Li
5dd3f91903 *: make backend outside kv
KV and lease will share the same backend. Thus we need to make
backend outside KV.
2016-01-05 19:55:29 -08:00
Anthony Romano
8346a7c052 Merge pull request #4094 from heyitsanthony/send-merged-done-nowait
etcdserver: respect done channel when sleeping for snapshot backoff
2015-12-29 20:18:08 -08:00
Anthony Romano
942b5570bd Merge pull request #4096 from heyitsanthony/serialize-applier-snapmerge
etcdserver: serialize snapshot merger with applier
2015-12-29 19:37:11 -08:00
Anthony Romano
4cd86ae1ef etcdserver: serialize snapshot merger with applier
Avoids inconsistent snapshotting by only attempting to
create a snapshot after an apply completes.

Fixes #4061
2015-12-29 18:38:39 -08:00
Anthony Romano
c7c3bda8b9 etcdserver: respect done channel when sleeping for snapshot backoff 2015-12-29 13:23:41 -08:00
Xiang Li
150e646b05 etcdserver: always check if the data dir is writable before starting etcd 2015-12-29 11:29:01 -08:00
Xiang Li
aef55342d1 etcdsever: avoid creating member dir before finishing validate bootstrap
This commit fixes the issue of creating member dir before validating
the configuration. When member dir exists, it indicates the local etcd
process is a valid etcd member. So we should only create member dir
after we finish configuration validation, joining validation or
discovery validation.
2015-12-28 23:20:27 -08:00
Xiang Li
eae52a3138 Revert "etcdserver: always remove member directory when bootstrap fails"
This reverts commit a7e443d6412ec405948571d1f758ab7f1c078bf9.
2015-12-28 23:16:08 -08:00
Gyu-Ho Lee
a7e443d641 etcdserver: always remove member directory when bootstrap fails
This removes member directory when bootstrap fails including joining existing
cluster and forming a new cluster. This fixes https://github.com/coreos/etcd/issues/3827.
2015-12-28 22:56:03 -08:00
Xiang Li
72e115ee6e Merge pull request #4062 from xiang90/fix_snap
*: fix snapshot sending cycle
2015-12-23 17:10:10 -08:00
Anthony Romano
d7ad721ede etcdserver: stop if removed along with multiple conf changes
shouldstop would get clobbered when several conf changes are in an apply
2015-12-23 16:29:21 -08:00
Xiang Li
4be152bb4f rework 2015-12-23 16:21:16 -08:00
Xiang Li
9a51d40940 fix comment 2015-12-23 14:10:39 -08:00
Xiang Li
ab31ba0d29 *: fix snapshot sending cycle 2015-12-23 13:58:57 -08:00
Xiang Li
d6d12b4d86 etcdserver: move unti out of server.go
etcdserver file is messy enough. Let's make it be less messy.
2015-12-22 15:17:14 -08:00
Anthony Romano
aca0c466ed etcdserver: asynchronously notify applier when raft writes finish
The raft loop would block on the applier's done channel after
persisting the raft messages; the latency could cause dropped network
messages. Instead, asynchronously notify the applier with a buffered
channel when the raft writes complete.
2015-12-22 14:15:14 -08:00
Anthony Romano
9d05a0d959 etcdserver: apply v3 database updates outside server event loop
raft's applyc writes block on the server loop's database IO since
the next applyc read must wait on the db operation to finish.
Instead, stream applyc to a run queue outside the server loop.
2015-12-22 14:15:09 -08:00
Anthony Romano
7e00325fe9 etcdserver: refactor server.go select loop
splits out the apply case into smaller functions
2015-12-22 12:13:39 -08:00
Gyu-Ho Lee
40b11038f2 etcdserver: fixes shadowed variables for go tool vet
Fix for https://github.com/coreos/etcd/issues/3954.
2015-12-12 09:13:12 -08:00
Xiang Li
c4cbaf5c2a etcdsever: swap kv pointer atomically 2015-12-10 17:29:36 -08:00
Xiang Li
23bd60ccce *: rewrite snapshot sending 2015-12-08 18:21:21 -08:00
Xiang Li
a07e4bb6e2 etcdserver: handle incoming v3 snapshot correctly
1. we should update all kv reference (including the
on in snapStore).

2. we should first restore a new KV and then close
the old one asynchronously.
2015-11-18 16:07:41 -08:00
Xiang Li
ff36b9d9bc Merge pull request #3700 from xiang90/metrics_hi
Replace Summary with Histogram for all metrics
2015-11-10 10:06:45 -08:00
Yicheng Qin
0874c44cdc etcdserver: fix snapshot index in creation log line
The snapshot is created at appliedi instead of snapi.
2015-11-05 14:02:09 -08:00
Yicheng Qin
3d15526c35 Merge pull request #3796 from yichengq/fix-get-version
etcdserver: not reuse connections for peer transport
2015-11-04 11:39:14 -08:00