17187 Commits

Author SHA1 Message Date
Justin Kolberg
eab0b999a8
clientv3: close streams after use in lessor keepAliveOnce method
Streams are now closed after being used in the lessor `keepAliveOnce` method.
This prevents the "failed to receive lease keepalive request from gRPC stream"
message from being logged by the server after the context is cancelled by the
client.

Signed-off-by: Justin Kolberg <amd.prophet@gmail.com>
2022-08-18 09:54:12 -07:00
Benjamin Wang
9e95685d0a
Merge pull request #14312 from ahrtr/3.5_bump_otl
[3.5] etcdserver: bump OpenTelemetry to 1.0.1 and gRPC to 1.41.0
2022-08-09 04:03:21 +08:00
Benjamin Wang
8fdca41cd8 Change default sampling rate from 100% to 0%
Refer to https://github.com/etcd-io/etcd/pull/14318

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-08-07 07:19:30 +08:00
Benjamin Wang
8c5f110b59 Fix the failure in TestEndpointSwitchResolvesViolation
Refer to a0bdfc4fc9

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-08-07 07:17:27 +08:00
Benjamin Wang
2751c61f24 update all related dependencies
Upgrade grpc to 1.41.0;
Run ./script/fix.sh to fix all related issue.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-08-07 07:17:27 +08:00
Benjamin Wang
5a86ae2c33 move setupTracing into a separate file config_tracing.go
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-08-07 07:17:27 +08:00
Benjamin Wang
2d7e49002c etcdserver: bump OpenTelemetry to 1.0.1
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-08-07 07:16:08 +08:00
Benjamin Wang
6145831683
Merge pull request #14318 from damemi/3.5-tracing-sample
Change default sampling rate from 100% to 0%
2022-08-07 07:14:35 +08:00
Mike Dame
4c013c91e9
Change default sampling rate from 100% to 0%
This changes the default parent-based trace sampling rate from
100% to 0%. Due to the high QPS etcd can handle, having 100% trace
sampling leads to very high resource usage. Defaulting to 0% means
that only already-sampled traces will be sampled in etcd.

Fixes #14310

Signed-off-by: Mike Dame <mikedame@google.com>
2022-08-05 15:00:40 +00:00
Marek Siarkowicz
9d7e10863e
Merge pull request #14227 from mitake/perm-cache-lock-3.5
server/auth: protect rangePermCache with a RW lock
2022-07-20 10:36:00 +02:00
Hitoshi Mitake
e15c005fef server/auth: protect rangePermCache with a RW lock
Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com>
2022-07-19 15:56:12 +09:00
Benjamin Wang
3237289fff
Merge pull request #14222 from Jille/backport-14203
[3.5] clientv3: Fix parsing of ETCD_CLIENT_DEBUG
2022-07-15 08:27:07 +08:00
Jille Timmermans
cbedaf90fe Improve error message for incorrect values of ETCD_CLIENT_DEBUG
Signed-off-by: Jille Timmermans <jille@quis.cx>
2022-07-14 09:43:54 +02:00
Benjamin Wang
fb71790611
Merge pull request #14219 from ahrtr/3.5_backport_maxstream
[3.5] Support configuring `MaxConcurrentStreams` for http2
2022-07-13 16:57:48 +08:00
Benjamin Wang
ff447b4a35 add e2e test cases to cover the maxConcurrentStreams
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-07-13 14:43:44 +08:00
Benjamin Wang
437f3778d0 Add flag --max-concurrent-streams to set the max concurrent stream each client can open at a time
Also refer to https://github.com/etcd-io/etcd/pull/14169#discussion_r917154243

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-07-13 14:18:15 +08:00
Benjamin Wang
40d1a43176 add the uint32Value data type
The golang buildin package `flag` doesn't support `uint32` data
type, so we need to support it via the `flag.Var`.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-07-13 13:57:22 +08:00
Benjamin Wang
57c1d92e20
Merge pull request #14187 from spzala/automated-cherry-pick-of-#14182-upstream-release-3.5
Automated cherry pick of #14182
2022-07-03 19:02:10 +08:00
Sahdev Zala
4df61af2df Client: fix check for WithPrefix op
Make sure that WithPrefix correctly set the flag, and add test.
Also, add test for WithFromKey.

fixes #14056

Signed-off-by: Sahdev Zala <spzala@us.ibm.com>
2022-07-02 23:33:26 -04:00
Marek Siarkowicz
c9f7473173
Merge pull request #14132 from ahrtr/auth_bundle
[3.5] client/v3: do not overwrite authTokenBundle on dial
2022-06-20 10:46:39 +02:00
Benjamin Wang
df632abd8a client/v3: do not overwrite authTokenBundle on dial
Cherry pick the PR https://github.com/etcd-io/etcd/pull/12992
to 3.5, so please refer to the original PR for more detailed info.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-18 06:59:55 +08:00
Benjamin Wang
4443e14dcd
Merge pull request #14127 from ahrtr/threshold_3.5
[3.5] Restrict the max size of each WAL entry to the remaining size of the WAL file
2022-06-17 15:03:35 +08:00
Benjamin Wang
621cd7b9e5 restrict the max size of each WAL entry to the remaining size of the file
Currently the max size of each WAL entry is hard coded as 10MB. If users
set a value > 10MB for the flag --max-request-bytes, then etcd may run
into a situation that it successfully processes a big request, but fails
to decode it when replaying the WAL file on startup.

On the other hand, we can't just remove the limitation, because if a
WAL entry is somehow corrupted, and its recByte is a huge value, then
etcd may run out of memory. So the solution is to restrict the max size
of each WAL entry as a dynamic value, which is the remaining size of
the WAL file.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-17 09:01:29 +08:00
Benjamin Wang
db0b67e8a0 Add FileReader and FileBufReader utilities
The FileReader interface is the wrapper of io.Reader. It provides
the fs.FileInfo as well. The FileBufReader struct is the wrapper of
bufio.Reader, it also provides fs.FileInfo.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-17 09:00:43 +08:00
Marek Siarkowicz
0be65da6cc
Merge pull request #14087 from ahrtr/lease_revoke_race
[3.5] Backport two lease related bug fixes to 3.5
2022-06-06 16:58:04 +02:00
Benjamin Wang
acb1ee993a Backport two lease related bug fixes to 3.5
The first bug fix is to resolve the race condition between goroutine
and channel on the same leases to be revoked. It's a classic mistake
in using Golang channel + goroutine. Please refer to
https://go.dev/doc/effective_go#channels

The second bug fix is to resolve the issue that etcd lessor may
continue to schedule checkpoint after stepping down the leader role.
2022-06-04 14:01:08 +08:00
Marek Siarkowicz
73876b176f
Merge pull request #14050 from serathius/avoid-clone-v3.5
[release-3.5] scripts: Avoid additional repo clone
2022-05-18 13:48:51 +02:00
Marek Siarkowicz
6aa934e546 scripts: Detect staged files before building release 2022-05-18 13:11:10 +02:00
Marek Siarkowicz
c05b9b13a8 scripts: Avoid additional repo clone
This PR removes additional clone when building artifacts.

When releasing v3.5.4 this clone was main cause of issues and
confusion about what release script is doing.

release.sh script already clones repo in /tmp/ directory, so clonning
before build is not needed. As precautions for bug in script leaving
/tmp/ clone in bad state  I moved "Verify the latest commit has the
version tag" and added "Verify the clean working tree" to be always run
before build.
2022-05-18 10:19:35 +02:00
Marek Siarkowicz
2e76dfb657
Merge pull request #14043 from serathius/test-release-3.5-v2
[release-3.5] Test release scripts
2022-05-16 14:03:01 +02:00
Marek Siarkowicz
c4b0a569ba Make DRY_RUN explicit 2022-05-16 13:10:05 +02:00
Marek Siarkowicz
c76a010b48 scripts: Add tests for release scripts 2022-05-16 13:09:46 +02:00
Piotr Tabor
b57881a164
Merge pull request #13205 from cfz/cherry-pick-#13172
[backport 3.5]: server/auth: enable tokenProvider if recoved store enables auth
2022-05-06 13:05:50 +02:00
cfz
cceb25d758
server/auth: enable tokenProvider if recoved store enables auth
we found a lease leak issue:
if a new member(by member add) is recovered by snapshot, and then
become leader, the lease will never expire afterwards. leader will
log the revoke failure caused by "invalid auth token", since the
token provider is not functional, and drops all generated token
from upper layer, which in this case, is the lease revoking
routine.
2022-05-06 12:24:28 +08:00
Piotr Tabor
8453b10e58
Merge pull request #13996 from cmurphy/update-crypto-3.5
Update golang.org/x/crypto to latest
2022-05-05 10:35:47 +02:00
Colleen Murphy
5c44c3022b Update golang.org/x/crypto to latest
Update crypto to address CVE-2022-27191.

The CVE fix is added in 0.0.0-20220315160706-3147a52a75dd but this
change updates to latest.
2022-04-28 09:27:02 -07:00
Marek Siarkowicz
08407ff760 version: bump up to 3.5.4 v3.5.4 tests/v3.5.4 etcdctl/v3.5.4 etcdutl/v3.5.4 server/v3.5.4 client/v3.5.4 client/v2.305.4 client/pkg/v3.5.4 raft/v3.5.4 pkg/v3.5.4 api/v3.5.4 2022-04-24 12:44:36 +02:00
Marek Siarkowicz
c3c908e39a
Merge pull request #13946 from ahrtr/move_cindex_on_apply_fail_353
[3.5] Update consitent_index when applying fails
2022-04-21 16:21:24 +02:00
ahrtr
5c68f2e510 Update conssitent_index when applying fails
When clients have no permission to perform whatever operation, then
the applying may fail. We should also move consistent_index forward
in this case, otherwise the consitent_index may smaller than the
snapshot index.
2022-04-20 22:17:49 +08:00
Piotr Tabor
b872757492
Merge pull request #13950 from liggitt/revert-srv-dot-3.5
Revert #13714
2022-04-19 12:23:02 +02:00
Jordan Liggitt
081b4e2a0f Add unit test for canonical SRV records 2022-04-15 15:33:31 -04:00
Jordan Liggitt
76564778a9 Revert "trim the suffix dot from the srv.Target for etcd-client DNS lookup"
This reverts commit 4f51cc1d9a6d31ae5622fe50c90f03962a7ee3a4.
2022-04-15 15:33:16 -04:00
Marek Siarkowicz
0452feec71 version: bump up to 3.5.3 v3.5.3 tests/v3.5.3 etcdctl/v3.5.3 etcdutl/v3.5.3 server/v3.5.3 client/v3.5.3 client/v2.305.3 client/pkg/v3.5.3 raft/v3.5.3 pkg/v3.5.3 api/v3.5.3 2022-04-13 17:17:51 +02:00
Marek Siarkowicz
842cb4b4fc
Merge pull request #13938 from endocrimes/dani/backport
[backport]  PR 13923 to release-3.5
2022-04-13 15:15:59 +02:00
Danielle Lancashire
50978d5b25 clientv3: disable mirror auth test with proxy 2022-04-13 12:41:24 +00:00
Danielle Lancashire
5d44f2242f cv3/mirror: Fetch the most recent prefix revision
When a user sets up a Mirror with a restricted user that doesn't have
access to the `foo` path, we will fail to get the most recent revision
due to permissions issues.

With this change, when a prefix is provided we will get the initial
revision from the prefix rather than /foo. This allows restricted users
to setup sync.
2022-04-13 12:39:06 +00:00
Marek Siarkowicz
cd750e4542
Merge pull request #13862 from mrueg/update-baseimage
[release-3.5] Dockerfile*: Switch baseimage to k8s hosted one
2022-04-12 11:55:46 +02:00
Marek Siarkowicz
003a310489
Merge pull request #13933 from ahrtr/fix_snapshot_recover_cindex_3.5
[3.5]Set backend to cindex before recovering the lessor in applySnapshot
2022-04-12 10:46:55 +02:00
Manuel Rüger
6095cf810a Dockerfile-release.*: Update base image snapshot
Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2022-04-12 10:39:55 +02:00
Marek Siarkowicz
719082e4fc
Merge pull request #13932 from ahrtr/lease_renew_linearizable_3.5
[3.5] Support linearizable renew lease
2022-04-12 10:03:20 +02:00