1001 Commits

Author SHA1 Message Date
Siyuan Zhang
c3af9427ed add tests to test tx delete consistency.
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-01-12 09:22:55 -08:00
Chao Chen
2c8e2e933f [3.4] backport #16790: upgrade grpc to 1.58.3
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-11-22 14:58:32 -08:00
Chao Chen
f549da33de backport https://github.com/etcd-io/etcd/pull/12709 and https://github.com/etcd-io/etcd/pull/12801 to resolve gogo unmarshal errors
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-11-22 11:14:50 -08:00
Chao Chen
1665b8ea4d release-3.4: upgrade gRPC-go to v1.52.0
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-11-22 09:41:43 -08:00
Chao Chen
5c95ef3e31 [3.4] backport etcd-io#13359 Fix http2 authority header in single endpoint scenario
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-11-21 23:34:55 -08:00
Chao Chen
97a6f957b2 Fix unit test unreferenced mu in clientv3/client_test.go
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-31 10:24:08 -07:00
Yi-Shu Tai
055b6d7b19 client: call .Endpoints() in dial() in client/v3/client.go instead of accessing cfg.Endpoints directly
0cdd558361/client/v3/client.go (L299) accesses
endpoints without acquiring lock. Fix it to call Endpoints()

Fix #13201

Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-31 09:39:20 -07:00
Chao Chen
3a0dd2d6a4 [3.4] backport #12706 clientv3: PS: Replace balancer with upstream grpc solution
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-30 09:26:40 -07:00
Chao Chen
83da5ff575 [3.4] Backport #12671 clientv3: Replace balancer with upstream grpc solution
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-30 09:26:40 -07:00
James Blair
678b00adf9
Backport clientv3: fix errors init client RejectOldCluster.
Signed-off-by: James Blair <mail@jamesblair.net>
2023-10-30 19:30:10 +13:00
yoyinzyc
03f79003d4 clientV3: simplify grpc dialer usage. Remove workaround #11184 after bumping grpc to 1.26.0.
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-27 12:10:54 -07:00
CFC4N
3181b44d76 clientv3: get AuthToken gracefully without extra connection.
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-25 12:51:30 -07:00
CFC4N
0cf5a72c2a credentials: let GetRequestMetadata() return nil when authToken isn't initialized.
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-25 12:51:30 -07:00
Chao Chen
e61f1d886b [3.4] backport 12675: Cleanup grpc clientv3/naming API
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-24 09:17:32 -07:00
Chao Chen
7c4696a7e8 [3.4] backport 12669: Implement Endpoint Watch and new Resolver
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-24 08:57:33 -07:00
Chao Chen
15d6a11e14 [3.4] backport 12667: Implement Update method for EndpointManager
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-24 08:10:00 -07:00
Piotr Tabor
6c0e4d97f1 Introduce grpc-1.30+ compatible client/v3/naming API.
This is not yet implementation, just API and tests to be filled
with implementation in next CLs,
tracked by: https://github.com/etcd-io/etcd/issues/12652

We propose here 3 packages:
 - clientv3/naming/endpoints ->
    That is abstraction layer over etcd that allows to write, read &
    watch Endpoints information. It's independent from GRPC API. It hides
    the storage details.

 - clientv3/naming/endpoints/internal ->
    That contains the grpc's compatible Update class to preserve the
    internal JSON mashalling format.

 - clientv3/naming/resolver ->
   That implements the GRPC resolver API, such that etcd can be
   used for connection.Dial in grpc.

Please see the grpc_naming.md document changes & grpcproxy/cluster.go
new integration, to see how the new abstractions work.

Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-19 12:59:24 -07:00
Benjamin Wang
05d7c10043
Merge pull request #16240 from liangyuanpeng/backport_13577_3.4
[3.4] Backport #13577 to 3.4
2023-07-14 16:09:26 +01:00
Benjamin Wang
81755115bf
Merge pull request #16202 from kensou97/release-3.4
[3.4] clientv3: create keepAliveCtxCloser goroutine only if ctx can be canc…
2023-07-14 13:46:54 +01:00
Lan Liang
47cef60ac4 Backport #13577
Disable auth gracefully without impacting existing watchers.

Signed-off-by: Lan Liang <gcslyp@gmail.com>
2023-07-14 12:48:22 +08:00
zhangwenkang
30426eb755 clientv3: create keepAliveCtxCloser goroutine only if ctx can be canceled
Signed-off-by: zhangwenkang <zwenkang@vmware.com>
2023-07-10 23:08:23 +08:00
Wenjia
44e59b18a6 Update clientv3/snapshot/v3_snapshot.go
Co-authored-by: Benjamin Wang <wachao@vmware.com>
Signed-off-by: Wenjia Zhang <wenjiazhang@google.com>
2023-07-06 23:46:17 -07:00
Wenjia Zhang
1130bf7f84 Adding optional revision bump and mark compacted to snapshot restore
Signed-off-by: Wenjia Zhang <wenjiazhang@google.com>
2023-07-06 23:08:38 -07:00
Marek Siarkowicz
6de105e89b refactor: Use proper variable names for urls
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-04-03 15:35:10 +02:00
Hitoshi Mitake
c8f890cde1 Revert "*: handle auth invalid token and old revision errors in watch"
This reverts commit 0c6e466024ea2030380b13e3e2248b0b8fb879ca.

Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com>
2023-03-21 22:13:17 +09:00
James Blair
a91bacf567
Formatted source code for go 1.19.6.
Signed-off-by: James Blair <mail@jamesblair.net>
2023-02-20 12:44:14 +13:00
Benjamin Wang
ed529ab0e5 clientv3: correct the nextRev on receving progress notification response
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-02-10 16:47:56 +08:00
Benjamin Wang
5413ce46dc bump go version to 1.17.3
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-19 18:34:04 +08:00
Benjamin Wang
8ca42a7ae4 clientv3: do not refresh token when using TLS CommonName based authentication
When users use the TLS CommonName based authentication, the
authTokenBundle is always nil. But it's possible for the clients
to get `rpctypes.ErrAuthOldRevision` response when the clients
concurrently modify auth data (e.g, addUser, deleteUser etc.).
In this case, there is no need to refresh the token; instead the
clients just need to retry the operations (e.g. Put, Delete etc).

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-17 08:32:35 +08:00
Benjamin Wang
b058374fbd
Merge pull request #14594 from ZoeShaw101/fix-watch-test-issue-3.4
Backport #14591 to 3.4.
2022-10-17 05:25:50 +08:00
王霄霄
dcebdf7958 Backport #14591 to 3.4.
Signed-off-by: 王霄霄 1141195807@qq.com
Signed-off-by: 王霄霄 <1141195807@qq.com>
2022-10-16 21:18:53 +08:00
Hisanobu Tomari
7b7fbbf8b8 client/v3: Add backoff before retry when watch stream returns unavailable
The client retries connection without backoff when the server is gone
after the watch stream is established. This results in high CPU usage
in the client process. This change introduces backoff when the stream is
failed and unavailable.

Signed-off-by: Hisanobu Tomari <posco.grubb@gmail.com>
2022-10-13 05:26:31 +09:00
Kafuu Chino
ed10ca13f4 *: avoid closing a watch with ID 0 incorrectly
Signed-off-by: Kafuu Chino <KafuuChinoQ@gmail.com>

add test

1

1

1
2022-10-10 19:54:58 +08:00
Hitoshi Mitake
0c6e466024 *: handle auth invalid token and old revision errors in watch
Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com>
2022-10-04 22:49:06 +09:00
Bogdan Kanivets
6f483a649e clientv3/balancer: fixed flaky TestRoundRobinBalancedResolvableFailoverFromServerFail
- ignore "transport is closing" error during connections warmup after stopping one peer.

Signed-off-by: Bogdan Kanivets <bkanivets@apple.com>
2022-07-26 08:06:59 -07:00
Marek Siarkowicz
8f4735dfd4 server: Require either cluster version v3.6 or --experimental-enable-lease-checkpoint-persist to persist lease remainingTTL
To avoid inconsistant behavior during cluster upgrade we are feature
gating persistance behind cluster version. This should ensure that
all cluster members are upgraded to v3.6 before changing behavior.

To allow backporting this fix to v3.5 we are also introducing flag
--experimental-enable-lease-checkpoint-persist that will allow for
smooth upgrade in v3.5 clusters with this feature enabled.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-07-22 10:28:29 +02:00
Benjamin Wang
3ea12d352e
Merge pull request #14241 from vivekpatani/release-3.4
clientv3: fix isOptsWithFromKey/isOptsWithPrefix
2022-07-20 18:51:24 +08:00
Chris Ayoub
36a76e8531 clientv3: filter learners members during autosync
This change is to ensure that all members returned during the client's
AutoSync are started and are not learners, which are not valid
etcd members to make requests to.

Signed-off-by: Chris Ayoub <cayoub@hubspot.com>
2022-07-20 00:04:03 -04:00
vivekpatani
4fef7fcb90 clientv3: fix isOptsWithFromKey/isOptsWithPrefix
- Addressing: https://github.com/etcd-io/etcd/issues/13332
- Backporting: https://github.com/etcd-io/etcd/pull/13334

Signed-off-by: vivekpatani <9080894+vivekpatani@users.noreply.github.com>
2022-07-19 17:20:56 -07:00
Bogdan Kanivets
185f203528 clientv3/balance: fixed flaky balancer tests
- added verification step to indirectly verify that all peers are in balancer subconn list

Signed-off-by: Bogdan Kanivets <bkanivets@apple.com>
2022-07-11 14:43:58 -07:00
Benjamin Wang
6958ee8ff2 Skip WatchRequestProgress test in grpc-proxy mode.
We shouldn't fail the grpc-server (completely) by a not implemented RPC.
Failing whole server by remote request is anti-pattern and security
risk.

Refer to https://github.com/etcd-io/etcd/runs/7034342964?check_suite_focus=true#step:5:2284

```
=== RUN   TestWatchRequestProgress/1-watcher
panic: not implemented
goroutine 83024 [running]:
go.etcd.io/etcd/proxy/grpcproxy.(*watchProxyStream).recvLoop(0xc009232f00, 0x4a73e1, 0xc00e2406e0)
	/home/runner/work/etcd/etcd/proxy/grpcproxy/watch.go:265 +0xbf2
go.etcd.io/etcd/proxy/grpcproxy.(*watchProxy).Watch.func1(0xc0038a3bc0, 0xc009232f00)
	/home/runner/work/etcd/etcd/proxy/grpcproxy/watch.go:125 +0x70
created by go.etcd.io/etcd/proxy/grpcproxy.(*watchProxy).Watch
	/home/runner/work/etcd/etcd/proxy/grpcproxy/watch.go:123 +0x73b
FAIL	go.etcd.io/etcd/clientv3/integration	222.813s
FAIL
```

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-29 05:12:43 +08:00
Benjamin Wang
1c9fa07cd7 Fix deadlock in 'go test -tags cluster_proxy -v ./integration/... ./clientv3/...'
Cherry pick https://github.com/etcd-io/etcd/pull/12319 to 3.4.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-28 13:44:47 +08:00
Bogdan Kanivets
2d99b341ad [backport 3.4] clientv3/integration: Reduce flakines of TestGetTokenWithoutAuth
backport from branch-3.5:
https://github.com/etcd-io/etcd/pull/12200/

Signed-off-by: Bogdan Kanivets <bkanivets@apple.com>
2022-06-27 11:31:16 -07:00
Benjamin Wang
1abf085cfb fix all the pipeline failues for release 3.4
Items resolved:
1. fix the vet error: possible misuse of reflect.SliceHeader;
2. fix the vet error: call to (*T).Fatal from a non-test goroutine;
3. bump package golang.org/x/crypto, net and sys;
4. bump boltdb from 1.3.3 to 1.3.6;
5. remove the vendor directory;
6. remove go 1.12.17 and 1.15.15, add go 1.16.15 into pipeline;
7. bump go version to 1.16 in go.mod;
8. fix the issue: compile: version go1.16.15 does not match go tool version go1.17.11,
   refer to https://github.com/actions/setup-go/issues/107;
9. fix data race on compactMainRev and watcherGauge;
10. fix test failure for TestLeasingTxnOwnerGet in cluster_proxy mode.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-22 05:28:45 +08:00
Bogdan Kanivets
ceed023f7c tests: fixing dependencies that brake tests in go.1.15.15
- retry_interceptor_test causes:
clientv3/naming/grpc.go:25:2: module google.golang.org/grpc@latest found (v1.46.0),
but does not contain package google.golang.org/grpc/naming
https://github.com/etcd-io/etcd/issues/12124
2022-05-30 12:08:47 -07:00
Hitoshi Mitake
757a8e8f5b *: implement a retry logic for auth old revision in the client 2022-04-29 23:46:24 +09:00
Ashish Ranjan
9bbdeb4a64 client/v3: refresh the token when ErrUserEmpty is received while retrying
To fix a bug in the retry logic caused when the auth token is cleared after receiving `ErrInvalidAuthToken` from the server and the subsequent call to `getToken` also fails due to some reason (eg. context deadline exceeded).
This leaves the client without a token and the retry will continue to fail with `ErrUserEmpty` unless the token is refreshed.
2022-04-29 23:43:36 +09:00
Sam Batschelet
501d8f01ea [release-3.4]: ClientV3: Ordering: Fix TestEndpointSwitchResolvesViolation test
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-06-23 21:26:55 -04:00
CFC4N
40b71074e8
clientv3: get AuthToken automatically when clientConn is ready.
fixes: #11954
2020-09-30 17:14:22 +08:00
jingyih
f44aaf8248 integration: add flag WatchProgressNotifyInterval in integration test 2020-09-09 12:39:42 -07:00