The gRPC server supports to use GracefulStop to drain all the inflight RPCs,
including streaming RPCs.
When we use non-cmux mode to start gRPC server (non-TLS or
TLS+gRPC-only), we always invoke GracefulStop to drain requests.
For cmux mode (gRPC.ServeHTTP), since the connection is maintained by http
server, gRPC server is unable to send GOAWAY control frame to client.
So, it's always force close all the connections and doesn't drain
requests by default.
In gRPC v1.61.0 version, it introduces new experimental feature
`WaitForHandlers` to block gRPC.Stop() until all the RPCs finish. This patch
is to use `WaitForHandlers` for cmux mode's graceful shutdown.
This patch also introduces `v3rpcBeforeSnapshot` failpoint. That's used
to verify cmux mode's graceful shutdown behaviour.
For TestAuthGracefulDisable (tests/common) case, increased timeout from
10s to 15s because we try to graceful shutdown after connection closed
and it takes more time than before.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
