959 Commits

Author SHA1 Message Date
Bogdan Kanivets
2d99b341ad [backport 3.4] clientv3/integration: Reduce flakines of TestGetTokenWithoutAuth
backport from branch-3.5:
https://github.com/etcd-io/etcd/pull/12200/

Signed-off-by: Bogdan Kanivets <bkanivets@apple.com>
2022-06-27 11:31:16 -07:00
Benjamin Wang
1abf085cfb fix all the pipeline failues for release 3.4
Items resolved:
1. fix the vet error: possible misuse of reflect.SliceHeader;
2. fix the vet error: call to (*T).Fatal from a non-test goroutine;
3. bump package golang.org/x/crypto, net and sys;
4. bump boltdb from 1.3.3 to 1.3.6;
5. remove the vendor directory;
6. remove go 1.12.17 and 1.15.15, add go 1.16.15 into pipeline;
7. bump go version to 1.16 in go.mod;
8. fix the issue: compile: version go1.16.15 does not match go tool version go1.17.11,
   refer to https://github.com/actions/setup-go/issues/107;
9. fix data race on compactMainRev and watcherGauge;
10. fix test failure for TestLeasingTxnOwnerGet in cluster_proxy mode.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-22 05:28:45 +08:00
Bogdan Kanivets
ceed023f7c tests: fixing dependencies that brake tests in go.1.15.15
- retry_interceptor_test causes:
clientv3/naming/grpc.go:25:2: module google.golang.org/grpc@latest found (v1.46.0),
but does not contain package google.golang.org/grpc/naming
https://github.com/etcd-io/etcd/issues/12124
2022-05-30 12:08:47 -07:00
Hitoshi Mitake
757a8e8f5b *: implement a retry logic for auth old revision in the client 2022-04-29 23:46:24 +09:00
Ashish Ranjan
9bbdeb4a64 client/v3: refresh the token when ErrUserEmpty is received while retrying
To fix a bug in the retry logic caused when the auth token is cleared after receiving `ErrInvalidAuthToken` from the server and the subsequent call to `getToken` also fails due to some reason (eg. context deadline exceeded).
This leaves the client without a token and the retry will continue to fail with `ErrUserEmpty` unless the token is refreshed.
2022-04-29 23:43:36 +09:00
Sam Batschelet
501d8f01ea [release-3.4]: ClientV3: Ordering: Fix TestEndpointSwitchResolvesViolation test
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-06-23 21:26:55 -04:00
CFC4N
40b71074e8
clientv3: get AuthToken automatically when clientConn is ready.
fixes: #11954
2020-09-30 17:14:22 +08:00
jingyih
f44aaf8248 integration: add flag WatchProgressNotifyInterval in integration test 2020-09-09 12:39:42 -07:00
jingyih
6fcab5af9f clientv3: remove excessive watch cancel logging 2020-08-13 13:51:21 +08:00
Ted Yu
e800c62eca clientv3: log warning in case of error sending request 2020-07-30 22:54:35 +08:00
cfc4n
490c6139ac auth: return incorrect result 'ErrUserNotFound' when client request without username or username was empty.
Fiexs https://github.com/etcd-io/etcd/issues/12004 .
2020-06-25 19:48:36 +08:00
Jack Kleeman
36452a1c1d clientv3: cancel watches proactively on client context cancellation
Currently, watch cancel requests are only sent to the server after a
message comes through on a watch where the client has cancelled. This
means that cancelled watches that don't receive any new messages are
never cancelled; they persist for the lifetime of the client stream.
This has negative connotations for locking applications where a watch
may observe a key which might never change again after cancellation,
leading to many accumulating watches on the server.

By cancelling proactively, in most cases we simply move the cancel
request to happen earlier, and additionally we solve the case where the
cancel request would never be sent.

Fixes #9416
Heavy inspiration drawn from the solutions proposed there.
2020-06-23 19:50:21 +08:00
Gyuho Lee
9bad82fee5 *: make sure snapshot save downloads SHA256 checksum
ref. https://github.com/etcd-io/etcd/pull/11896

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-17 17:38:42 -07:00
Gary Belvin
e048e166ab
cherry pick of #11564 (#11880)
* clientv3: fix grpc-go(v1.27.0) incompatible changes to balancer/resolver.

* vendor: upgrade gRPC Go to v1.24.0

Picking up some performance improvements and bug fixes.

https://github.com/grpc/grpc-go/releases/tag/v1.24.0

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

* vendor: update gRPC Go to v1.26.0 (#11522)

* GO111MODULE=on go mod vendor

* GO111MODULE=on go mod vendor go 1.14

Bump travis 2

Co-authored-by: EDDYCJY <313687982@qq.com>
Co-authored-by: Gyuho Lee <leegyuho@amazon.com>
Co-authored-by: Yuchen Zhou <yczhou@google.com>
2020-05-13 10:12:58 -07:00
Gyuho Lee
1471e12108 clientv3: embed api version in metadata
ref.
https://github.com/etcd-io/etcd/pull/11687

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

clientv3: fix racy writes to context key

=== RUN   TestWatchOverlapContextCancel

==================

WARNING: DATA RACE

Write at 0x00c42110dd40 by goroutine 99:

  runtime.mapassign()

      /usr/local/go/src/runtime/hashmap.go:485 +0x0

  github.com/coreos/etcd/clientv3.metadataSet()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/ctx.go:61 +0x8c

  github.com/coreos/etcd/clientv3.withVersion()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/ctx.go:47 +0x137

  github.com/coreos/etcd/clientv3.newStreamClientInterceptor.func1()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/client.go:309 +0x81

  google.golang.org/grpc.NewClientStream()

      /go/src/github.com/coreos/etcd/gopath/src/google.golang.org/grpc/stream.go:101 +0x10e

  github.com/coreos/etcd/etcdserver/etcdserverpb.(*watchClient).Watch()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/etcdserver/etcdserverpb/rpc.pb.go:3193 +0xe9

  github.com/coreos/etcd/clientv3.(*watchGrpcStream).openWatchClient()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:788 +0x143

  github.com/coreos/etcd/clientv3.(*watchGrpcStream).newWatchClient()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:700 +0x5c3

  github.com/coreos/etcd/clientv3.(*watchGrpcStream).run()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:431 +0x12b

Previous read at 0x00c42110dd40 by goroutine 130:

  reflect.maplen()

      /usr/local/go/src/runtime/hashmap.go:1165 +0x0

  reflect.Value.MapKeys()

      /usr/local/go/src/reflect/value.go:1090 +0x43b

  fmt.(*pp).printValue()

      /usr/local/go/src/fmt/print.go:741 +0x1885

  fmt.(*pp).printArg()

      /usr/local/go/src/fmt/print.go:682 +0x1b1

  fmt.(*pp).doPrintf()

      /usr/local/go/src/fmt/print.go:998 +0x1cad

  fmt.Sprintf()

      /usr/local/go/src/fmt/print.go:196 +0x77

  github.com/coreos/etcd/clientv3.streamKeyFromCtx()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:825 +0xc8

  github.com/coreos/etcd/clientv3.(*watcher).Watch()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:265 +0x426

  github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel.func1()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:959 +0x23e

Goroutine 99 (running) created at:

  github.com/coreos/etcd/clientv3.(*watcher).newWatcherGrpcStream()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:236 +0x59d

  github.com/coreos/etcd/clientv3.(*watcher).Watch()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:278 +0xbb6

  github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel.func1()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:959 +0x23e

Goroutine 130 (running) created at:

  github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:979 +0x76d

  github.com/coreos/etcd/clientv3/integration.TestWatchOverlapContextCancel()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:922 +0x44

  testing.tRunner()

      /usr/local/go/src/testing/testing.go:657 +0x107

==================

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-18 17:17:00 -07:00
Gyuho Lee
4388404f56 clientv3: fix retry/streamer error message
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-10-31 10:09:18 -07:00
Gyuho Lee
2c36cab87d
Merge pull request #11223 from YoyinZyc/automated-cherry-pick-of-#11179-origin-release-3.4
Automated cherry pick of #11179
2019-10-09 13:28:04 -07:00
yoyinzyc
daa432cfa7 etcdserver: add put request steps. mvcc: add put request steps; add trace to KV.Write() as input parameter. 2019-10-09 12:40:12 -07:00
Joe Betz
0781c0327d
clientv3: Replace endpoint.ParseHostPort with net.SplitHostPort to fix IPv6 client endpoints 2019-10-08 18:27:03 -07:00
Joe Betz
c454344f14
clientv3: Set authority used in cert checks to host of endpoint 2019-10-08 15:35:27 -07:00
Guangming Wang
67cc70926d integration: fix bug in for loop, make it break properly 2019-09-17 13:30:12 -07:00
vimalk78
83bf125d93 clientv3: add nil checks in Close()
Added nil checks in Close() for Watcher and Lease fields
Added test case
2019-08-25 23:40:05 -07:00
Jingyi Hu
e00224f87e integration: fix TestKVPutError
Give backend quota enough overhead.
2019-08-22 13:33:19 -07:00
Gyuho Lee
1691eec2db clientv3/integration: fix "mvcc.NewStore" call
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-15 13:31:46 -07:00
Joe Betz
1e213b7ab6 *: Add experimental-compaction-batch-limit flag
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-15 13:31:39 -07:00
Gyuho Lee
2a8d09b83b clientv3: use Endpoints(), fix context creation
If overwritten, the previous context should be canceled first.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-13 12:43:49 -07:00
Zeming YU
a6a9a71b6a integration: fix a data race about err
don't share `err` between goroutines
2019-08-06 16:15:27 -07:00
Gyuho Lee
ab1db0dfd8 clientv3: deprecate "grpc.ErrClientConnClosing"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-05 14:38:27 -07:00
Gyuho Lee
3468505e38 clientv3: document "WithBlock" dial option
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-04 23:53:02 -07:00
Gyuho Lee
ba42e65b59 clientv3/integration: give more time for balancer resolution
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-26 09:19:22 -07:00
Gyuho Lee
8c7c6ec0c1 clientv3/balancer: refactor
refactor + remove unused

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-26 09:19:19 -07:00
Gyuho Lee
3dc00ab615 clientv3: move auth token credential to "credentials" package
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-26 09:17:44 -07:00
Gyuho Lee
db61ee106c clientv3/credentials: set dial target "Authority" with target address
Overwrite authority when it's IP.

When user dials with "grpc.WithDialer", "grpc.DialContext" "cc.parsedTarget"
update only happens once. This is problematic, because when TLS is enabled,
retries happen through "grpc.WithDialer" with static "cc.parsedTarget" from
the initial dial call.
If the server authenticates by IP addresses, we want to set a new endpoint as
a new authority. Otherwise
"transport: authentication handshake failed: x509: certificate is valid for 127.0.0.1, 192.168.121.180, not 192.168.223.156"
when the new dial target is "192.168.121.180" whose certificate host name is also "192.168.121.180"
but client tries to authenticate with previously set "cc.parsedTarget" field "192.168.223.156"

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-26 09:17:40 -07:00
Gyuho Lee
db7231accc clientv3: use new "credentials" package
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-26 08:56:33 -07:00
Gyuho Lee
324c876742 clientv3/credential: implement grpc/credentials.Bundle
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-26 08:56:05 -07:00
Gyuho Lee
12ab2ee3c4 clientv3: do not use pointer to TransportCredentials interface
Interface in Go is already reference

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-26 08:52:31 -07:00
Sahdev P. Zala
1cef112a79 etcdserver: do not allow creating empty role
Like user, we should not allow creating empty role.

Related #10905
2019-07-24 17:41:24 -04:00
Tobias Schottdorf
b9c051e7a7 raftpb: clean up naming in ConfChange 2019-07-23 10:40:03 +02:00
lzhfromustc
8194aa3f03
Fixed a missing block bug
Description: w.mu is locked at line 385 and unlocked at line 396. Among 5 return statements in this function, 4 are below line 396 but there is 1 return at line 387. 
Fix: Add w.mu.Unlock() before that return at line 387.
2019-06-28 11:27:13 -07:00
Gyuho Lee
1caaa9ed4a test: test update for Go 1.12.5 and related changes
Update to Go 1.12.5 testing. Remove deprecated unused and gosimple
pacakges, and mask staticcheck 1006. Also, fix unconvert errors related
to unnecessary type conversions and following staticcheck errors:
- remove redundant return statements
- use for range instead of for select
- use time.Since instead of time.Now().Sub
- omit comparison to bool constant
- replace T.Fatal and T.Fatalf in tests with T.Error and T.Fatalf respectively because the goroutine calls T.Fatal must be called in the same goroutine as the test
- fix error strings that should not be capitalized
- use sort.Strings(...) instead of sort.Sort(sort.StringSlice(...))
- use he status code of Canceled instead of grpc.ErrClientConnClosing which is deprecated
- use use status.Errorf instead of grpc.Errorf which is deprecated

Related #10528 #10438
2019-06-05 17:02:05 -04:00
Xiang Li
ea0f919cdc
Merge pull request #10775 from jingyih/integration_member_promote_failed_cases
clientv3/integration: add member promote failure test cases
2019-06-02 20:39:00 -07:00
Jingyi Hu
ec17872c98 integration: add member promote failure test cases
Add TestMemberPromoteMemberNotLearner and
TestMemberPromoteMemberNotExist.
2019-05-31 10:52:54 -07:00
Hitoshi Mitake
5a67dd788d *: support creating a user without password
This commit adds a feature for creating a user without password. The
purpose of the feature is reducing attack surface by configuring bad
passwords (CN based auth will be allowed for the user).

The feature can be used with `--no-password` of `etcdctl user add`
command.

Fix https://github.com/coreos/etcd/issues/9590
2019-05-30 21:59:30 +09:00
Xiang Li
23a89b0f09
Merge pull request #10755 from wrfly/patch-3
fix #10754
2019-05-28 20:50:52 -07:00
Shaun
3754767dbc clientv3: return getToken error when retryAuth 2019-05-29 10:15:22 +08:00
Jingyi Hu
6bf609b96d integration: update TestMemberPromote test
Update TestMemberPromote to include both learner not-ready and learner
ready test cases.

Removed unit test TestPromoteMember, it requires underlying raft node to
be started and running. The member promote is covered by the integration
test.
2019-05-28 18:50:13 -07:00
Jingyi Hu
f8ad8ae4ad etcdserver: use etcdserver ErrLearnerNotReady
If learner is not ready to be promoted, use etcdserver.ErrLearnerNotReady
instead of using membership.ErrLearnerNotReady.
2019-05-28 18:50:13 -07:00
宇慕
dfe296ac3c etcdserver: add mayPromote check 2019-05-28 18:47:03 -07:00
Jingyi Hu
7a4d233bab clientv3/integration: better way to deflake test
Use ReadyNotify instead of time.Sleep to wait for server ready.
2019-05-28 18:47:03 -07:00
Jingyi Hu
aa4cda2f5c etcdserver: allow 1 learner in cluster
Hard-coded the maximum number of learners to 1.
2019-05-28 18:47:03 -07:00