339 Commits

Author SHA1 Message Date
Benjamin Wang
a1018dbddf
Merge pull request #14604 from ahrtr/double_barrier
clientv3: fix the implementation of double barrier
2022-10-31 08:13:12 +08:00
jianfei.zhang
d75d1e575a fix: close conn
Signed-off-by: jianfei.zhang <jianfei.zhang@daocloud.io>
2022-10-27 11:09:14 +08:00
Benjamin Wang
62167d1f1f clientv3: fix the design & implementation of double barrier
Check the client count before creating the ephemeral key, do not
create the key if there are already too many clients. Check the
count after creating the key again, if the total kvs is bigger
than the expected count, then check the rev of the current key,
and take action accordingly based on its rev. If its rev is in
the first ${count}, then it's valid client, otherwise, it should
fail.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-10-20 16:24:20 +08:00
happlins
8a587447d2 clientv3: fix errors init client RejectOldCluster
Signed-off-by: happlins <happlins@foxmail.com>
2022-10-13 17:24:24 +08:00
Hisanobu Tomari
428fb9623a client/v3: Add backoff before retry when watch stream returns unavailable
The client retries connection without backoff when the server is gone
after the watch stream is established. This results in high CPU usage
in the client process. This change introduces backoff when the stream is
failed and unavailable.

Signed-off-by: Hisanobu Tomari <posco.grubb@gmail.com>
2022-10-08 06:57:30 +09:00
Chao Chen
8d057ea2b8 common tests framework: cluster client creation fail with invalid auth
Signed-off-by: Chao Chen <chaochn@amazon.com>
2022-09-29 23:09:32 -07:00
Kafuu Chino
f1d4935e91 *: avoid closing a watch with ID 0 incorrectly
Signed-off-by: Kafuu Chino <KafuuChinoQ@gmail.com>

add test
2022-09-26 20:30:33 +08:00
Benjamin Wang
7f10dccbaf Bump go 1.19: update all the dependencies and go.sum files
1. run ./scripts/fix.sh;
2. cd tools/mod; gofmt -w . & go mod tidy;

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-22 08:47:46 +08:00
Benjamin Wang
cd0b1d0c66 Bump go 1.19: upgrade go version to 1.19 in all go.mod files
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-22 08:47:46 +08:00
Benjamin Wang
2441a24cee
Merge pull request #14493 from demoManito/style/format-import-order
etcd: format import order
2022-09-21 06:03:31 +08:00
demoManito
f67ec10779 etcd: format import order
golang CodeReviewComments:
https://github.com/golang/go/wiki/CodeReviewComments#imports

Signed-off-by: demoManito <1430482733@qq.com>
2022-09-20 18:41:39 +08:00
Hitoshi Mitake
2dcfa83094 *: handle auth invalid token and old revision errors in watch
Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com>
2022-09-17 21:51:36 +09:00
Babis Kaidos
0eb642c4d9 Mockserver can handle requests to the leases API
This should allow clients using the leases API to perform UTs.

Fixes #14426

Signed-off-by: Babis Kaidos <babis_k@outlook.com>
2022-09-06 18:48:51 +03:00
Justin Kolberg
295044fba2
clientv3: close streams after use in lessor keepAliveOnce method
Streams are now closed after being used in the lessor `keepAliveOnce` method.
This prevents the "failed to receive lease keepalive request from gRPC stream"
message from being logged by the server after the context is cancelled by the
client.

Signed-off-by: Justin Kolberg <amd.prophet@gmail.com>
2022-08-17 18:02:13 -07:00
Chao Chen
d4778e78c8 more logging around auto sync
Signed-off-by: Chao Chen <chaochn@amazon.com>
2022-08-10 18:00:06 -07:00
QianChenglong
8e1e118fea clientv3: fix --insecure-skip-tls-verify not working when not specify --cert-path and --key-path
Fixes 14315

Signed-off-by: QianChenglong <qian_cheng_long@163.com>
2022-08-08 16:24:12 +08:00
Jille Timmermans
ac3f913dad Improve error message for incorrect values of ETCD_CLIENT_DEBUG
Signed-off-by: Jille Timmermans <jille@quis.cx>
2022-07-13 18:55:51 +02:00
Jille Timmermans
e3fd58df46 clientv3: Fix parsing of ETCD_CLIENT_DEBUG
It checked `err == nil` rather than `err != nil`.

Signed-off-by: Jille Timmermans <jille@quis.cx>
2022-07-10 11:21:15 +02:00
Sahdev Zala
8637c54bcc Client: fix check for WithPrefix op
Make sure that WithPrefix correctly set the flag, and add test.
Also, add test for WithFromKey.

fixes #14056

Signed-off-by: Sahdev Zala <spzala@us.ibm.com>
2022-06-29 17:52:05 -04:00
chavacava
0887d9acf1 fix #14161
Signed-off-by: chavacava <salvadorcavadini+github@gmail.com>
2022-06-25 10:09:05 +00:00
Piotr Tabor
88cd0fe695 Refresh minor dependencies.
Signed-off-by: Piotr Tabor <ptab@google.com>
2022-06-17 10:23:03 +02:00
Piotr Tabor
e7a84b69c8 Update zap to 1.21.
Signed-off-by: Piotr Tabor <ptab@google.com>
2022-06-17 10:05:25 +02:00
Piotr Tabor
17c1dcd614 Update grpc to 1.47 and go-cmp to 0.5.8.
Signed-off-by: Piotr Tabor <ptab@google.com>
2022-06-17 08:57:22 +02:00
Marek Siarkowicz
d58f971895 tests: Use zaptest.NewLogger in places we missed 2022-05-06 12:23:20 +02:00
Sean Wei
6d133aaf6a doc: Fix Markdown code block 2022-04-25 15:06:38 +08:00
ahrtr
0dae4b3b1e rollback the opentelemetry bumpping to recover the pipeline failures 2022-04-14 16:13:28 +08:00
Will Beason
eab1e0c5d5 go.mod: upgrade opentelemetry deps
Downstream users of etcd experience build issues when using dependencies
which require more recent (incompatible) versions of opentelemetry. This
commit upgrades the dependencies so that downstream users stop
experiencing these issues.
2022-04-13 07:14:10 -07:00
Danielle Lancashire
e324cc1cbe cv3/mirror: Fetch the most recent prefix revision
When a user sets up a Mirror with a restricted user that doesn't have
access to the `foo` path, we will fail to get the most recent revision
due to permissions issues.

With this change, when a prefix is provided we will get the initial
revision from the prefix rather than /foo. This allows restricted users
to setup sync.
2022-04-11 13:42:03 +00:00
Sahdev Zala
0c9a4e0f93
Merge pull request #13837 from chrisayoub/main
clientv3: filter learner members during autosync
2022-04-09 19:00:55 -04:00
Donal Hunt
6e1afa9677
Update client/v3/snapshot/v3_snapshot.go
Co-authored-by: Marek Siarkowicz <marek.siarkowicz@protonmail.com>
2022-04-08 18:44:50 +01:00
Donal Hunt
4c8ef011e0 *: drop use of humanize.Time() in favour of time.Duration.String()
humanize.Time() drops precision resulting in some events reporting they took
"now" time to complete. Using time.Duration.String() results in accurate
duration being reported.

Fixes #13905
2022-04-07 23:24:35 +01:00
Manuel Rüger
f0f77fc14e go.mod: Bump prometheus/client_golang to v1.12.1
Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2022-04-06 19:03:24 +02:00
Piotr Tabor
f85cd0296f
Merge pull request #13872 from ptabor/20220402-osx-unit-test-pass
Fix TestauthTokenBundleOnOverwrite on OsX:
2022-04-02 20:03:38 +02:00
Piotr Tabor
3b589fb3b2 Fix TestauthTokenBundleOnOverwrite on OsX:
```
% (cd client/v3 && env go test -short -timeout=3m --race ./...)
--- FAIL: TestAuthTokenBundleNoOverwrite (0.00s)
    client_test.go:210: listen unix /var/folders/t1/3m8z9xz93t9c3vpt7zyzjm6w00374n/T/TestAuthTokenBundleNoOverwrite3197524989/001/etcd-auth-test:0: bind: invalid argument
FAIL
FAIL	go.etcd.io/etcd/client/v3	4.270s
```

The reason was that the path exceeded 108 chars (that is too much for socket).
In the mitigation we first change chroot (working directory) to the tempDir... such the path is 'local'.
2022-04-02 16:12:02 +02:00
Marek Siarkowicz
63346bfead server: Use default logging configuration instead of zap production one
This fixes problem where logs json changes format of timestamp.
2022-04-01 10:23:42 +02:00
Chris Ayoub
125f3c3f9a clientv3: filter learners members during autosync
This change is to ensure that all members returned during the client's
AutoSync are started and are not learners, which are not valid
etcd members to make requests to.
2022-03-29 13:38:21 -04:00
ahrtr
49e9a14580 migrate unit test to cover the logic of converting ConfigSpec to Config 2022-03-24 07:24:22 +08:00
ahrtr
f978da4f4f move the newClientCfg into clientv3 package so as to be reused by both etcdctl and v3discovery 2022-03-24 06:18:25 +08:00
ahrtr
1a3822f2c3 Rename ClientConfig to ConfigSpec
The ClientConfig is a fully declarive configuration, so it makes more
sense to rename it to ConfigSpec. It can also mitigate the confusion
between Config and ClientConfig.
2022-03-13 05:41:49 +08:00
ahrtr
3dcbbf62d9 Move clientconfig into clientv3 so that it can be reused by both etcdctl and v3 discovery 2022-03-12 06:38:41 +08:00
Marek Siarkowicz
fb55910500 version: bump up to 3.6.0-alpha.0 2022-02-28 13:20:27 +01:00
Marek Siarkowicz
d0c1c3a1fd client: Alias downgrade action enum 2022-02-21 17:58:20 +01:00
ahrtr
8681888012 fix typo, renamed ErrGPRCNotSupportedForLearner to ErrGRPCNotSupportedForLearner 2022-02-21 14:46:58 +08:00
ahrtr
15568f4c00 add protection code for Range when the sortTarget is an invalid value 2022-01-18 07:46:37 +08:00
Piotr Tabor
fdd98477ef Update dep: require gopkg.in/yaml.v2 v2.2.8 -> v2.4.0 due to: CVE-2019-11254 2022-01-17 16:52:58 +01:00
Jille Timmermans
3710083dc3 retry_interceptor.go: Log the method that failed
When getting warnings from the retry_interceptors that RPCs failed, they
provide very little context as to what exactly failed.

I used this patch to try debugging
https://groups.google.com/g/etcd-dev/c/3hxxr9ktdpM.
2022-01-16 16:46:50 +01:00
Sahdev Zala
96a9fd0a1e
Merge pull request #13574 from cunnie/defer_cancel
Golang Client docs: defer `cancel()`, avoid erroring
2022-01-03 20:03:22 -05:00
Yap Sok Ann
17fd2e7282 Disable auth gracefully without impacting existing watchers
This attempts to fix a special case of the problem described in #12385,
where trying to do `clientv3.Watch` with an expired token would result
in `ErrGRPCPermissionDenied`, due to the failing authorization check in
`isWatchPermitted`. Furthermore, the client can't auto recover, since
`shouldRefreshToken` rightly returns false for the permission denied
error.

In this case, we would like to have a runbook to dynamically disable
auth, without causing any disruption. Doing so would immediately expire
all existing tokens, which would then cause the behavior described
above. This means existing watchers would still work for a period of
time after disabling auth, until they have to reconnect, e.g. due to a
rolling restart of server nodes.

This commit adds a client-side fix and a server-side fix, either of
which is sufficient to get the added test case to pass. Note that it is
an e2e test case instead of an integration one, as the reconnect only
happens if the server node is stopped via SIGINT or SIGTERM.

A generic fix for the problem described in #12385 would be better, as
that shall also fix this special case. However, the fix would likely be
a lot more involved, as some untangling of authn/authz is required.
2021-12-31 14:39:46 +07:00
Brian Cunnie
5620a9c227
Golang Client docs: defer cancel(), avoid erroring
In the sample code demonstrating how to specify a client request
timeout, the `cancel()` is called immediately after the Put, but it
should be deferred instead, giving the Put enough time to complete.

In the canonical Golang context
[docs](https://pkg.go.dev/context#WithTimeout), the sample code sets a
`defer cancel()` immediately after context creation, and with this
commit we adhere to that convention.

fixes:
```json
{
  "level": "warn",
  "ts": "2021-12-29T09:56:42.439-0800",
  "logger": "etcd-client",
  "caller": "v3@v3.5.1/retry_interceptor.go:62",
  "msg": "retrying of unary invoker failed",
  "target": "etcd-endpoints://0xc000213340/localhost:2379",
  "attempt": 0,
  "error": "rpc error: code = Canceled desc = context canceled"
}
```
2021-12-29 14:10:36 -08:00
Linhai
d1194977eb fix potential goroutine leaks in TestTxnPanics 2021-12-15 01:22:56 -05:00