15596 Commits

Author SHA1 Message Date
Sahdev P. Zala
434f7e83f0 pkg: check file stats
modify file util.
2020-06-20 16:29:47 -04:00
Gyuho Lee
91b1a9182a
Merge pull request #11977 from jpbetz/automated-cherry-pick-of-#11946-release-3.4
Automated cherry pick of #11946
2020-06-05 12:46:33 -07:00
David Crawshaw
78f67988aa
etcdserver, et al: add --unsafe-no-fsync flag
This makes it possible to run an etcd node for testing and development
without placing lots of load on the file system.

Fixes #11930.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
2020-06-04 20:19:28 -07:00
Gyuho Lee
54ba958911 version: 3.4.9
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
v3.4.9
2020-05-20 16:28:29 -07:00
Gyuho Lee
609e844f86
Merge pull request #11811 from wswcfan/automated-cherry-pick-of-#11735-origin-release-3.4
Automated cherry pick of #11735 on release-3.4
2020-05-20 15:57:48 -07:00
tangcong
166b4473fa wal: add TestValidSnapshotEntriesAfterPurgeWal testcase
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-20 11:38:06 -07:00
tangcong
ed231df7c0 wal: fix crc mismatch crash bug 2020-05-20 11:37:04 -07:00
Gyuho Lee
cfe37de6c0 rafthttp: log snapshot download duration
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-20 11:37:01 -07:00
Gyuho Lee
0de2b1f860 version: 3.4.8
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
v3.4.8
2020-05-18 11:40:23 -07:00
Gyuho Lee
a668adba78 rafthttp: improve snapshot send logging
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-18 11:39:24 -07:00
Gyuho Lee
9bad82fee5 *: make sure snapshot save downloads SHA256 checksum
ref. https://github.com/etcd-io/etcd/pull/11896

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-17 17:38:42 -07:00
Gyuho Lee
f1ea03a7c8 etcdserver/api/snap: exclude orphaned defragmentation files in snapNames
ref. https://github.com/etcd-io/etcd/pull/11900

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-17 14:21:02 -07:00
Ted Yu
4079deadb4 etcdserver: continue releasing snap db in case of error
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-17 14:15:26 -07:00
Viacheslav Biriukov
87fc3c9e57 etcdserver,wal: fix inconsistencies in WAL and snapshot
etcdserver/*, wal/*: changes to snapshots and wal logic
etcdserver/*: changes to snapshots and wal logic to fix #10219
etcdserver/*, wal/*: add Sync method
etcdserver/*, wal/*: find valid snapshots by cross checking snap files and wal snap entries
etcdserver/*, wal/*:Add comments, clean up error messages and tests
etcdserver/*, wal/*: Remove orphaned .snap.db files during Release

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-15 08:40:09 -07:00
Gary Belvin
e048e166ab
cherry pick of #11564 (#11880)
* clientv3: fix grpc-go(v1.27.0) incompatible changes to balancer/resolver.

* vendor: upgrade gRPC Go to v1.24.0

Picking up some performance improvements and bug fixes.

https://github.com/grpc/grpc-go/releases/tag/v1.24.0

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

* vendor: update gRPC Go to v1.26.0 (#11522)

* GO111MODULE=on go mod vendor

* GO111MODULE=on go mod vendor go 1.14

Bump travis 2

Co-authored-by: EDDYCJY <313687982@qq.com>
Co-authored-by: Gyuho Lee <leegyuho@amazon.com>
Co-authored-by: Yuchen Zhou <yczhou@google.com>
2020-05-13 10:12:58 -07:00
Gyuho Lee
2333c727a2
Merge pull request #11855 from tangcong/automated-cherry-pick-of-#11817-origin-release-3.4
Automated cherry pick of #11817 on release-3.4
2020-05-07 20:00:26 -07:00
tangcong
aa75e90ac8 mvcc: fix deadlock bug 2020-05-08 09:56:23 +08:00
shawwang
f18976f4b8 auth: optimize lock scope for CheckPassword
to improve authentication performance in concurrent scenarios when enable auth and using authentication based password
2020-04-25 18:36:18 +08:00
Jingyi Hu
f1eca4e1fa
Merge pull request #11752 from tangcong/automated-cherry-pick-of-#11652-#11670-#11710-origin-release-3.4
Automated cherry pick of #11652 #11670 #11710
2020-04-10 23:21:45 +08:00
tangcong
b733b22712 auth: ensure RoleGrantPermission is compatible with older versions 2020-04-09 09:33:40 +08:00
tangcong
eb80716532 etcdserver: print warn log when failed to apply request 2020-04-09 09:33:40 +08:00
tangcong
e2abd97659 auth: cleanup saveConsistentIndex in NewAuthStore 2020-04-09 09:33:40 +08:00
tangcong
716821b9b5 auth: print warning log when error is ErrAuthOldRevision 2020-04-09 09:33:40 +08:00
shawwang
63116ffdb4 auth: add new metric 'etcd_debugging_auth_revision' 2020-04-09 09:33:40 +08:00
shawwang
b3d54def77 tools/etcd-dump-db: add auth decoder, optimize print format 2020-04-09 09:33:40 +08:00
tangcong
347c8dac3b *: fix auth revision corruption bug 2020-04-09 09:33:36 +08:00
Sahdev Zala
e2ae6013a4
Merge pull request #11757 from jingyih/automated-cherry-pick-of-#11754-upstream-release-3.4
Automated cherry pick of #11754 on release-3.4
2020-04-06 11:09:26 -04:00
Changxin Miao
9c8554573f etcdserver: watch stream got closed once one request is not permitted (#11708) 2020-04-06 07:06:57 -07:00
Gyuho Lee
e694b7bb08 version: 3.4.7
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
v3.4.7
2020-04-01 10:46:54 -07:00
Gyuho Lee
e99399d0dc wal: add "etcd_wal_writes_bytes_total"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-04-01 09:30:09 -07:00
Gyuho Lee
b68f8ff31d pkg/ioutil: add "FlushN"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-04-01 09:29:59 -07:00
Gyuho Lee
857dffa386
Merge pull request #11734 from jingyih/automated-cherry-pick-of-#11330-upstream-release-3.4
Cherry pick of #11330 on release-3.4
2020-03-31 14:58:59 -07:00
jingyih
5f17aa2c8b test: auto detect branch when finding merge base 2020-03-31 10:59:44 -07:00
shenjiangc
89b10cf967 mvcc/kvstore:when the number key-value is greater than one million, compact take too long and blocks other requests 2020-03-30 08:21:38 -07:00
Gyuho Lee
bdc9bc1d81 version: 3.4.6
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
v3.4.6
2020-03-29 12:47:27 -07:00
tangcong
b0bdaaa449 lease: fix memory leak in LeaseGrant when node is follower 2020-03-29 12:47:14 -07:00
Gyuho Lee
e784ba73c2 version: 3.4.5
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
v3.4.5
2020-03-18 17:24:42 -07:00
Gyuho Lee
35dc623a98 words: whitelist "racey"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-18 17:24:15 -07:00
Gyuho Lee
130342152a Revert "version: 3.4.5"
This reverts commit 0dc5d577fc08b3666481fdf9617035280c6f5e3f.
2020-03-18 17:17:19 -07:00
Gyuho Lee
fc93fbf9de words: whitelist "hasleader"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-18 17:17:04 -07:00
Gyuho Lee
0dc5d577fc version: 3.4.5
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-18 17:17:04 -07:00
Gyuho Lee
e63db56cc9 etcdserver/api/v3rpc: handle api version metadata, add metrics
ref.
https://github.com/etcd-io/etcd/pull/11687

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-18 17:17:04 -07:00
Gyuho Lee
1471e12108 clientv3: embed api version in metadata
ref.
https://github.com/etcd-io/etcd/pull/11687

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

clientv3: fix racy writes to context key

=== RUN   TestWatchOverlapContextCancel

==================

WARNING: DATA RACE

Write at 0x00c42110dd40 by goroutine 99:

  runtime.mapassign()

      /usr/local/go/src/runtime/hashmap.go:485 +0x0

  github.com/coreos/etcd/clientv3.metadataSet()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/ctx.go:61 +0x8c

  github.com/coreos/etcd/clientv3.withVersion()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/ctx.go:47 +0x137

  github.com/coreos/etcd/clientv3.newStreamClientInterceptor.func1()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/client.go:309 +0x81

  google.golang.org/grpc.NewClientStream()

      /go/src/github.com/coreos/etcd/gopath/src/google.golang.org/grpc/stream.go:101 +0x10e

  github.com/coreos/etcd/etcdserver/etcdserverpb.(*watchClient).Watch()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/etcdserver/etcdserverpb/rpc.pb.go:3193 +0xe9

  github.com/coreos/etcd/clientv3.(*watchGrpcStream).openWatchClient()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:788 +0x143

  github.com/coreos/etcd/clientv3.(*watchGrpcStream).newWatchClient()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:700 +0x5c3

  github.com/coreos/etcd/clientv3.(*watchGrpcStream).run()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:431 +0x12b

Previous read at 0x00c42110dd40 by goroutine 130:

  reflect.maplen()

      /usr/local/go/src/runtime/hashmap.go:1165 +0x0

  reflect.Value.MapKeys()

      /usr/local/go/src/reflect/value.go:1090 +0x43b

  fmt.(*pp).printValue()

      /usr/local/go/src/fmt/print.go:741 +0x1885

  fmt.(*pp).printArg()

      /usr/local/go/src/fmt/print.go:682 +0x1b1

  fmt.(*pp).doPrintf()

      /usr/local/go/src/fmt/print.go:998 +0x1cad

  fmt.Sprintf()

      /usr/local/go/src/fmt/print.go:196 +0x77

  github.com/coreos/etcd/clientv3.streamKeyFromCtx()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:825 +0xc8

  github.com/coreos/etcd/clientv3.(*watcher).Watch()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:265 +0x426

  github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel.func1()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:959 +0x23e

Goroutine 99 (running) created at:

  github.com/coreos/etcd/clientv3.(*watcher).newWatcherGrpcStream()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:236 +0x59d

  github.com/coreos/etcd/clientv3.(*watcher).Watch()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:278 +0xbb6

  github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel.func1()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:959 +0x23e

Goroutine 130 (running) created at:

  github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:979 +0x76d

  github.com/coreos/etcd/clientv3/integration.TestWatchOverlapContextCancel()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:922 +0x44

  testing.tRunner()

      /usr/local/go/src/testing/testing.go:657 +0x107

==================

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-18 17:17:00 -07:00
Gyuho Lee
0b9cfa8677 etcdserver/api/etcdhttp: log server-side /health checks
ref.
https://github.com/etcd-io/etcd/pull/11704

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-18 16:29:24 -07:00
Sam Batschelet
b66c53ff5f proxy/grpcproxy: add return on error for metrics handler
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2020-03-16 12:06:01 -04:00
Sahdev Zala
8f6c3f4d09
Merge pull request #11664 from jingyih/automated-cherry-pick-of-#11638-upstream-release-3.4
Automated cherry pick of #11638 on release-3.4
2020-03-11 19:26:11 -04:00
jingyih
379d01a0d2 etcdctl: fix member add command
Use members information from member add response, which is
guaranteed to be up to date.
2020-02-29 07:18:11 -08:00
Gyuho Lee
c65a9e2dd1 version: 3.4.4
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
v3.4.4
2020-02-24 13:14:02 -08:00
Jingyi Hu
7862f6ed2c
Merge pull request #11644 from jingyih/automated-cherry-pick-of-#11640-upstream-release-3.4
Automated cherry pick of #11640 on release-3.4
2020-02-23 14:42:34 +08:00
Rafael Fernández López
257319fb18 etcdserver: fix quorum calculation when promoting a learner member
When promoting a learner member we should not count already a voting
member, but take only into account the number of existing voting
members and their current status (started, unstarted) when taking the
decision whether a learner member can be promoted.

Before this change, it was impossible to grow from a quorum N to a N+1
through promoting a learning member.

Fixes: #11633
2020-02-21 23:14:55 -08:00