mirror of
https://github.com/etcd-io/etcd.git
synced 2024-09-27 06:25:44 +00:00
grpcproxy: Disable fast fail on lease grant call to cluster
Problem Observed ---------------- When there is no etcd process behind the proxy, clients repeat resending lease grant requests without delay. This behavior can cause abnormal resource consumption on CPU/RAM and network. Problem Detail -------------- `LeaseGrant()` uses a bare protobuf client to forward requests. However, it doesn't use `grpc.FailFast(false)`, which means the method returns an `Unavailable` error immediately when no etcd process is available. In clientv3, `Unavailable` errors are not considered the "Halt" error, and library retries the request without delay. Both clients and the proxy consume much CPU cycles to process retry requests. Resolution ---------- Add `grpc.FailFast(false))` to `LeaseGrant()` of the `leaseProxy`. This makes the proxy not to return immediately when no etcd process is available. Clients will simply timeout requests instead.
This commit is contained in:
@@ -73,7 +73,7 @@ func NewLeaseProxy(c *clientv3.Client) (pb.LeaseServer, <-chan struct{}) {
|
||||
}
|
||||
|
||||
func (lp *leaseProxy) LeaseGrant(ctx context.Context, cr *pb.LeaseGrantRequest) (*pb.LeaseGrantResponse, error) {
|
||||
rp, err := lp.leaseClient.LeaseGrant(ctx, cr)
|
||||
rp, err := lp.leaseClient.LeaseGrant(ctx, cr, grpc.FailFast(false))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user