1153 Commits

Author SHA1 Message Date
Xiang Li
6296054ff6 etcdhttp: version endpoint also returns cluster version. 2015-05-13 15:48:10 -07:00
Yicheng Qin
75ee7f4aa1 Merge pull request #2821 from yichengq/private-cluster
etcdserver: stop exposing Cluster struct
2015-05-13 10:26:48 -07:00
Xiang Li
2690535f8a Merge pull request #2820 from xiang90/cap
version capability checking
2015-05-13 10:16:49 -07:00
Xiang Li
d3b1d5c008 etcdhttp: support capability checking
etcdhttp will check the cluster version and update its
capability version periodically.

Any new handler's after 2.0 needs to wrap by capability handler
to ensure it is not accessable until rolling upgrade finished.
2015-05-13 10:11:35 -07:00
Yicheng Qin
a6a649f1c3 etcdserver: stop exposing Cluster struct
After this PR, only cluster's interface Cluster is exposed, which makes
code much cleaner. And it avoids external packages to rely on cluster
struct in the future.
2015-05-13 10:01:25 -07:00
Xiang Li
f2905f2828 etcdserver: remove unnecessary around detect datadir
The log is super unhelpful. When I have a 2.1.0 etcd, it prints out
`2.0.1 vaild dir`. I have no idea why the data dir of a 2.1.0 etcd is
2.0.1.
2015-05-12 22:06:42 -07:00
Yicheng Qin
032db5e396 *: extract types.Cluster from etcdserver.Cluster
The PR extracts types.Cluster from etcdserver.Cluster. types.Cluster
is used for flag parsing and etcdserver config.

There is no need to expose etcdserver.Cluster public, which contains
lots of etcdserver internal details and methods. This is the first step
for it.
2015-05-12 14:53:11 -07:00
Xiang Li
e866314b94 etcdserver: support update cluster version through raft
1. Persist the cluster version change through raft. When the member is restarted, it can recover
the previous known decided cluster version.

2. When there is a new leader, it is forced to do a version checking immediately. This helps to
update the first cluster version fast.
2015-05-12 11:44:34 -07:00
Xiang Li
94ffd72c7e etcdserver: rename StoreAdminPrefix to StoreClusterPrefix
We store cluster related key in StoreAdminPrefix for some
historical reason. The previous API is called admin. But now,
the admin name is gone and `cluster` is a more clear and correct
name.
2015-04-29 12:05:51 -07:00
Xiang Li
6699107f61 *: add cluster version and cluster version detection.
Cluster version is the min major.minor of all members in
the etcd cluster. Cluster version is set to the min version
that a etcd member is compatible with when first bootstrapp.

During a rolling upgrades, the cluster version will be updated
automatically.

For example:

```
Cluster [a:1, b:1 ,c:1] -> clusterVersion 1

update a -> 2, b -> 2

after a detection

Cluster [a:2, b:2 ,c:1] -> clusterVersion 1, since c is still 1

update c -> 2

after a detection

Cluster [a:2, b:2 ,c:2] -> clusterVersion 2
```

The API/raft component can utilize clusterVersion to determine if
it can accept a client request or a raft RPC.

We choose polling rather than pushing since we want to use the same
logic for cluster version detection and (TODO) cluster version checking.

Before a member actually joins a etcd cluster, it should check the version
of the cluster. Push does not work since the other members cannot push
version info to it before it actually joins. Moreover, we do not want our
raft RPC system (which is doing the heartbeat pushing) to coordinate cluster version.
2015-04-29 11:31:59 -07:00
Yicheng Qin
1c1cccd236 rafthttp: stop etcd if it is found removed when stream dial
The original process is stopping etcd only when pipeline message finds itself
has been removed. After this PR, stream dial has this functionality too.
It helps fast etcd stop, which doesn't need to wait for stream break to
fall back to pipeline, and wait for election timeout to send out message
to detect self removal.
2015-04-27 15:10:00 -07:00
Yicheng Qin
ebecee34e0 Merge pull request #2701 from yichengq/rafthttp-anon
rafthttp: add remotes
2015-04-24 13:04:37 -07:00
Yicheng Qin
9f19b5660f rafthttp: add AddRemote
Add remotes to rafthttp, who help newly joined members catch up the
progress of the cluster. It supports basic message sending to remote, and
has no stream connection for simplicity. remotes will not be used
after the latest peers have been added into rafthttp.
2015-04-24 11:49:23 -07:00
xiaost
cab1e9a723 etcdserver: skip noop entry in apply 2015-04-24 12:15:51 +08:00
Barak Michener
fa74e702d8 security: Improve the security api as per the suggestions list in #2384
Subcommits:

decouple root and security enable/disable

create root role

prefix matching

godep: bump go-etcd to include credentials

add godep for speakeasy and auth entry parsing

appropriate errors for security enable/disable

WIP adding to etcd/client all the security client methods

add guest access

minor ui return tweaks

revert client changes

respond to comments, log more security operations

fix major ensure() bug, add better UX

block recursive access

fix some boneheaded mistakes

fix integration test

last comments

fix up security_api.md

philips nits

fix docs
2015-04-23 16:11:38 -04:00
Yicheng Qin
1d96de459a etcdserver: init server stats before passing it as argument
It is more reasonable to init the variable before passing it as an
argument.

It fixes a bug that etcdserver may panic on server stats when processing
a message from rafthttp streamReader before server stats is initialized
in server.Start().
2015-04-22 08:28:08 -07:00
Xiang Li
5ad559b503 *: serve json version on both client and peer url 2015-04-20 16:23:51 -07:00
Yicheng Qin
1811701427 Revert "etcdserver: fix cluster fallback recovery"
This reverts commit cff005777a40bcf3a5bea3e87387273afe054ce1.

Conflicts:
	etcdserver/server.go
2015-04-19 11:34:33 -07:00
Yicheng Qin
88224f6f4e Revert "etcdserver: not apply stale conf change in cluster and transport"
This reverts commit 40197f06987aac9c3a539e9022ad1f1e573326e7.
2015-04-19 11:08:03 -07:00
Xiang Li
98f8dfbc9d etcdserver: prevExist=true + condition is compareAndSwap
PrevExist indicates the key should exist. Condition compares with
an existing key. So PrevExist+condition = CompareAndSwap not Update.
2015-04-14 23:44:06 -07:00
xiaost
eab2c2224a etcdserver: fix minor bug in EtcdServer.send
it seems to nothing serious.
after deleted peers, the log may output:
"etcdserver: send message to unknown receiver %s"
2015-04-13 20:35:58 +08:00
Yicheng Qin
2141308524 Merge pull request #2631 from yichengq/metrics-fd
etcdserver: metrics and monitor number of file descriptor
2015-04-08 11:28:58 -07:00
Yicheng Qin
7a7e1f7a7c etcdserver: metrics and monitor number of file descriptor
It exposes the metrics of file descriptor limit and file descriptor used.
Moreover, it prints out warning when more than 80% of fd limit has been used.

```
2015/04/08 01:26:19 etcdserver: 80% of the file descriptor limit is open
[open = 969, limit = 1024]
```
2015-04-08 11:17:48 -07:00
Alex Crawford
d9ad6aa2a9 *: update to use IANA-assigned ports 2015-04-06 13:49:43 -07:00
Xiang Li
471aa1aa89 Merge pull request #2622 from xiang90/fix_watcher
store: fix watcher removal
2015-04-03 10:39:03 -07:00
Xiang Li
999917010d store: fix watcher removal 2015-04-03 10:13:43 -07:00
Yicheng Qin
9e5743c816 etcdserver: stop raft node goroutine before stop server
Stop raftNode goroutine before stopping server goroutine, so
server.Stop does stop all underlying stuffs elegantly now. This fixes
the problem that previous-round lock on WAL may not be released when
etcd is restarted.
2015-04-01 11:20:51 -07:00
Xiang Li
77a04cda0c Merge pull request #2597 from xiang90/wal-repair
wal: fix the unexpectedEOF error in the last wal.
2015-03-30 13:49:05 -07:00
Xiang Li
253f7c4ae1 Merge pull request #2522 from xiang90/user_pw
etcdserver/etcdhttp: do not return back the password of a user
2015-03-30 13:42:41 -07:00
Xiang Li
0b9a318e68 etcdserver: make the wal repairing logic clear 2015-03-29 21:10:28 -07:00
Xiang Li
1231f82f22 etcdserver: save snapshot into wal first 2015-03-29 14:23:05 -07:00
Xiang Li
8b4eed29e5 wal: fix the unexpectedEOF error in the last wal.
It is safe to repair the unexpectedEOF error in the last wal. raft
will not send out message before the entry successfully comitted
into wal. Thus we can safely truncate the last entry in the wal
to repair.
2015-03-28 21:08:14 -07:00
Yicheng Qin
60efd4d96e Revert "etcdhttp: add internalVersion"
This reverts commit a77bf97c147a93e5b78898b427d081ba11b8f118.

Conflicts:
	version/version.go
2015-03-27 16:53:55 -07:00
Yicheng Qin
dd92a2b484 Merge pull request #2556 from yichengq/fix-apply-conf
etcdserver: not apply stale conf change
2015-03-27 14:00:30 -07:00
Kelsey Hightower
538d624cfa etcdserver: add stats.LatencyStats and stats.CountsStats types 2015-03-27 13:42:44 -07:00
Yicheng Qin
40197f0698 etcdserver: not apply stale conf change in cluster and transport 2015-03-27 12:53:34 -07:00
Xiang Li
e3817adb5b etcdserver: loose member validation for joining existing cluster 2015-03-25 13:59:22 -07:00
Xiang Li
05e240b892 *: update protobuf 2015-03-25 10:14:35 -07:00
Yicheng Qin
5e0077cc0c etcdserver: print out extra files in data dir instead of erroring 2015-03-24 18:56:22 -07:00
Xiang Li
866a9d4e41 Merge pull request #2568 from xiang90/raftnode
raft: make node configurable
2015-03-24 11:18:22 -07:00
Yicheng Qin
ea78f5d1aa Merge pull request #2552 from yichengq/fix-2396
etcdserver: check -initial-cluster in join case
2015-03-23 22:46:38 -07:00
Yicheng Qin
abcd828114 etcdserver: add join-existing check 2015-03-23 22:31:20 -07:00
Xiang Li
abddef0f28 raft: make node configurable 2015-03-23 21:20:49 -07:00
Kelsey Hightower
4611c3b2d7 netutil: add BasicAuth function
etcd ships it's own BasicAuth function and no longer requires
Go 1.4 to build.
2015-03-20 17:32:33 -07:00
Xiang Li
9d28f94005 etcdserver/etcdhttp: do not return back the password of a user 2015-03-16 22:35:01 -07:00
Xiang Li
f3e4dbf967 etcdserver/etcdhttp: write the http error to response writer 2015-03-16 15:24:19 -07:00
Xiang Li
bba7f75562 Merge pull request #2517 from yichengq/fix-sec2
security: fix var shadowing in CreateOrUpdateUser
2015-03-16 15:08:55 -07:00
Yicheng Qin
8335a5407b security: fix var shadowing in CreateOrUpdateUser 2015-03-16 14:59:05 -07:00
Yicheng Qin
d7780cf293 security: fix var shadowing in CreateOrUpdate 2015-03-16 14:55:04 -07:00
Barak Michener
001efa0639 security: Implement RBAC security for etcd
stub out security

further wip

Last stub before CRUD for roles

Complete role merging

start tests

add Godep for golang.org/x/crypto/bcrypt

first round of comments

add tests, remove root addition (will be added back as part of creation)

Add security checks for /v2/machines and /v2/keys

Allow non-root to determine if security is enabled, get machine list.

Responding to comments, remove multiple verbs (like /v2/security/user/foo/password)

add some prefixes to the logging
2015-03-16 16:23:11 -04:00