17fd2e7282
This attempts to fix a special case of the problem described in #12385, where trying to do `clientv3.Watch` with an expired token would result in `ErrGRPCPermissionDenied`, due to the failing authorization check in `isWatchPermitted`. Furthermore, the client can't auto recover, since `shouldRefreshToken` rightly returns false for the permission denied error. In this case, we would like to have a runbook to dynamically disable auth, without causing any disruption. Doing so would immediately expire all existing tokens, which would then cause the behavior described above. This means existing watchers would still work for a period of time after disabling auth, until they have to reconnect, e.g. due to a rolling restart of server nodes. This commit adds a client-side fix and a server-side fix, either of which is sufficient to get the added test case to pass. Note that it is an e2e test case instead of an integration one, as the reconnect only happens if the server node is stopped via SIGINT or SIGTERM. A generic fix for the problem described in #12385 would be better, as that shall also fix this special case. However, the fix would likely be a lot more involved, as some untangling of authn/authz is required.
etcd/client/v3
etcd/clientv3 is the official Go etcd client for v3.
Install
go get go.etcd.io/etcd/client/v3
Get started
Create client using clientv3.New:
import clientv3 "go.etcd.io/etcd/client/v3"
func main() {
cli, err := clientv3.New(clientv3.Config{
Endpoints: []string{"localhost:2379", "localhost:22379", "localhost:32379"},
DialTimeout: 5 * time.Second,
})
if err != nil {
// handle error!
}
defer cli.Close()
}
etcd v3 uses gRPC for remote procedure calls. And clientv3 uses
grpc-go to connect to etcd. Make sure to close the client after using it.
If the client is not closed, the connection will have leaky goroutines. To specify client request timeout,
pass context.WithTimeout to APIs:
ctx, cancel := context.WithTimeout(context.Background(), timeout)
resp, err := cli.Put(ctx, "sample_key", "sample_value")
cancel()
if err != nil {
// handle error!
}
// use the response
For full compatibility, it is recommended to install released versions of clients using go modules.
Error Handling
etcd client returns 2 types of errors:
- context error: canceled or deadline exceeded.
- gRPC error: see api/v3rpc/rpctypes.
Here is the example code to handle client errors:
resp, err := cli.Put(ctx, "", "")
if err != nil {
switch err {
case context.Canceled:
log.Fatalf("ctx is canceled by another routine: %v", err)
case context.DeadlineExceeded:
log.Fatalf("ctx is attached with a deadline is exceeded: %v", err)
case rpctypes.ErrEmptyKey:
log.Fatalf("client-side error: %v", err)
default:
log.Fatalf("bad cluster endpoints, which are not etcd servers: %v", err)
}
}
Metrics
The etcd client optionally exposes RPC metrics through go-grpc-prometheus. See the examples.
Namespacing
The namespace package provides clientv3 interface wrappers to transparently isolate client requests to a user-defined prefix.
Request size limit
Client request size limit is configurable via clientv3.Config.MaxCallSendMsgSize and MaxCallRecvMsgSize in bytes. If none given, client request send limit defaults to 2 MiB including gRPC overhead bytes. And receive limit defaults to math.MaxInt32.