Progress notifications requested using ProgressRequest were sent
directly using the ctrlStream, which means that they could race
against watch responses in the watchStream.
This would especially happen when the stream was not synced - e.g. if
you requested a progress notification on a freshly created unsynced
watcher, the notification would typically arrive indicating a revision
for which not all watch responses had been sent.
This changes the behaviour so that v3rpc always goes through the watch
stream, using a new RequestProgressAll function that closely matches
the behaviour of the v3rpc code - i.e.
1. Generate a message with WatchId -1, indicating the revision for
*all* watchers in the stream
2. Guarantee that a response is (eventually) sent
The latter might require us to defer the response until all watchers
are synced, which is likely as it should be. Note that we do *not*
guarantee that the number of progress notifications matches the number
of requests, only that eventually at least one gets sent.
Signed-off-by: Benjamin Wang <wachao@vmware.com>
Difference in load configuration for watch delay tests show how huge the
impact is. Even with random write scheduler grpc under http
server can only handle 500 KB with 2 seconds delay. On the other hand,
separate grpc server easily hits 10, 100 or even 1000 MB within 100 miliseconds.
Priority write scheduler that was used in most previous releases
is far worse than random one.
Tests configured to only 5 MB to avoid flakes and taking too long to fill
etcd.
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
There are two goroutines accessing the `gs` grpc server var. Before
insecure `gs` server start, the `gs` can be changed to secure server and
then the client will fail to connect to etcd with insecure request. It
is data-race. We should use argument for reference in the new goroutine.
fix: #15495
Signed-off-by: Wei Fu <fuweid89@gmail.com>
(cherry picked from commit a9988e2625eede1af81d189b5f2ecf7d4af3edf1)
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Historic capnslog timestamps are in microsecond resolution. We need to match that when we migrate to the zap logger.
Signed-off-by: James Blair <mail@jamesblair.net>
In order to fix https://github.com/etcd-io/etcd/issues/12385,
PR https://github.com/etcd-io/etcd/pull/14322 introduced a change
in which the client side may retry based on the error message
returned from server side.
This is not good, as it's too fragile and it's also changed the
protocol between client and server. Please see the discussion
in https://github.com/kubernetes/kubernetes/pull/114403
Note: The issue https://github.com/etcd-io/etcd/issues/12385 only
happens when auth is enabled, and client side reuse the same client
to watch.
So we decided to rollback the change on 3.5, reasons:
1.K8s doesn't enable auth at all. It has no any impact on K8s.
2.It's very easy for client application to workaround the issue.
The client just needs to create a new client each time before watching.
Signed-off-by: Benjamin Wang <wachao@vmware.com>
This changes the builds to always add -trimpath which removes specific
build time paths from the binary (like current directories etc).
Improves build reproducability to make the final binary independent from
the specific build path.
Lastly, when stripping debug symbols, also add -w to strip DWARF symbols
as well which aren't needed in that case either.
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Check the client count before creating the ephemeral key, do not
create the key if there are already too many clients. Check the
count after creating the key again, if the total kvs is bigger
than the expected count, then check the rev of the current key,
and take action accordingly based on its rev. If its rev is in
the first ${count}, then it's valid client, otherwise, it should
fail.
Signed-off-by: Benjamin Wang <wachao@vmware.com>
