Would retry a few times before returning a not primary error that the client should never see. Instead, use proper timeouts and then return a request timeout error on failure. Fixes #6922
