15808 Commits

Author SHA1 Message Date
Bogdan Kanivets
6f483a649e clientv3/balancer: fixed flaky TestRoundRobinBalancedResolvableFailoverFromServerFail
- ignore "transport is closing" error during connections warmup after stopping one peer.

Signed-off-by: Bogdan Kanivets <bkanivets@apple.com>
2022-07-26 08:06:59 -07:00
Benjamin Wang
2c778eebf7
Merge pull request #14269 from ahrtr/3.4_resend_readindex
[3.4] etcdserver: resend ReadIndex request on empty apply request
2022-07-25 16:53:06 +08:00
Benjamin Wang
f53db9b246 etcdserver: resend ReadIndex request on empty apply request
Backport https://github.com/etcd-io/etcd/pull/12795 to 3.4

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-07-25 09:21:31 +08:00
Benjamin Wang
e2b36f8879
Merge pull request #14253 from serathius/checkpoints-fix-3.4
[3.4] Checkpoints fix 3.4
2022-07-22 16:56:17 +08:00
Benjamin Wang
de2e8ccc78
Merge pull request #14258 from ahrtr/3.4_postphone_read_index
[3.4] raft: postpone MsgReadIndex until first commit in the term
2022-07-22 16:46:32 +08:00
Marek Siarkowicz
783e99cbfe Fix lease checkpointing tests by forcing a snapshot
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-07-22 10:28:44 +02:00
Marek Siarkowicz
8f4735dfd4 server: Require either cluster version v3.6 or --experimental-enable-lease-checkpoint-persist to persist lease remainingTTL
To avoid inconsistant behavior during cluster upgrade we are feature
gating persistance behind cluster version. This should ensure that
all cluster members are upgraded to v3.6 before changing behavior.

To allow backporting this fix to v3.5 we are also introducing flag
--experimental-enable-lease-checkpoint-persist that will allow for
smooth upgrade in v3.5 clusters with this feature enabled.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-07-22 10:28:29 +02:00
Benjamin Wang
9c9148c4cd raft: postpone MsgReadIndex until first commit in the term
Backport https://github.com/etcd-io/etcd/pull/12762 to 3.4

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-07-22 13:56:27 +08:00
Benjamin Wang
f18d074866
Merge pull request #14254 from ramses/backport-13435
[3.4] Backport: non mutating requests pass through quotaKVServer when NOSPACE
2022-07-22 09:27:00 +08:00
Benjamin Wang
aca5cd1717
Merge pull request #14246 from vivekpatani/release-3.4
[3.4] etcdserver,pkg: remove temp files in snap dir when etcdserver starting
2022-07-22 09:14:23 +08:00
vivekpatani
e4deb09c9e etcdserver,pkg: remove temp files in snap dir when etcdserver starting
- Backporting: https://github.com/etcd-io/etcd/pull/12846
- Reference: https://github.com/etcd-io/etcd/issues/14232

Signed-off-by: vivekpatani <9080894+vivekpatani@users.noreply.github.com>
2022-07-21 15:50:27 -07:00
Chao Chen
96f69dee47 Backport: non mutating requests pass through quotaKVServer when NOSPACE
This is a backport of https://github.com/etcd-io/etcd/pull/13435 and is
part of the work for 3.4.20
https://github.com/etcd-io/etcd/issues/14232.

The original change had a second commit that modifies a changelog file.
The 3.4 branch does not include any changelog file, so that part was not
cherry-picked.

Local Testing:

- `make build`
- `make test`

Both succeed.

Signed-off-by: Ramsés Morales <ramses@gmail.com>
2022-07-21 15:06:09 -07:00
Michał Jasionowski
8d83691d53 etcdserver,integration: Store remaining TTL on checkpoint
To extend lease checkpointing mechanism to cases when the whole etcd
cluster is restarted.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-07-21 17:35:21 +02:00
Michał Jasionowski
a30aba8fc2 lease,integration: add checkpoint scheduling after leader change
Current checkpointing mechanism is buggy. New checkpoints for any lease
are scheduled only until the first leader change. Added fix for that
and a test that will check it.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-07-21 17:35:15 +02:00
Benjamin Wang
7ee7029c08
Merge pull request #14251 from ahrtr/3.4_maxstream
[3.4] Support configuring MaxConcurrentStreams for http2
2022-07-21 17:43:15 +08:00
Benjamin Wang
6071b1c523 Support configuring MaxConcurrentStreams for http2
Backport https://github.com/etcd-io/etcd/pull/14219 to 3.4

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-07-21 14:25:29 +08:00
Benjamin Wang
40ccb8b454
Merge pull request #14240 from chaochn47/cherry-pick-12335
[3.4] etcdserver: add more detailed traces on linearized reading
2022-07-21 03:32:02 +08:00
Chao Chen
864006b72d print out applied index as uint64
Signed-off-by: Chao Chen <chaochn@amazon.com>
2022-07-20 12:07:51 -07:00
Pierre Zemb
3f9fba9112 etcdserver: add more detailed traces on linearized reading
To improve debuggability of `agreement among raft nodes before
linearized reading`, we added some tracing inside
`linearizableReadLoop`.

This will allow us to know the timing of `s.r.ReadIndex` vs
`s.applyWait.Wait(rs.Index)`.

Signed-off-by: Chao Chen <chaochn@amazon.com>
2022-07-20 12:07:51 -07:00
Benjamin Wang
fc76e90cf2
Merge pull request #14230 from mitake/perm-cache-lock-3.4
server/auth: protect rangePermCache with a RW lock
2022-07-20 18:51:54 +08:00
Benjamin Wang
3ea12d352e
Merge pull request #14241 from vivekpatani/release-3.4
clientv3: fix isOptsWithFromKey/isOptsWithPrefix
2022-07-20 18:51:24 +08:00
Benjamin Wang
6313502fb4
Merge pull request #14239 from chaochn47/backport-13676
backport 3.5: #13676 load all leases from backend
2022-07-20 18:50:46 +08:00
Benjamin Wang
b0e1aaef69
Merge pull request #14236 from chrisayoub/release-3.4
[release-3.4] clientv3: filter learners members during autosync
2022-07-20 12:49:31 +08:00
Chris Ayoub
36a76e8531 clientv3: filter learners members during autosync
This change is to ensure that all members returned during the client's
AutoSync are started and are not learners, which are not valid
etcd members to make requests to.

Signed-off-by: Chris Ayoub <cayoub@hubspot.com>
2022-07-20 00:04:03 -04:00
vivekpatani
4fef7fcb90 clientv3: fix isOptsWithFromKey/isOptsWithPrefix
- Addressing: https://github.com/etcd-io/etcd/issues/13332
- Backporting: https://github.com/etcd-io/etcd/pull/13334

Signed-off-by: vivekpatani <9080894+vivekpatani@users.noreply.github.com>
2022-07-19 17:20:56 -07:00
Chao Chen
fd51434b54 backport 3.5: #13676 load all leases from backend
Signed-off-by: Chao Chen <chaochn@amazon.com>
2022-07-19 16:08:01 -07:00
Benjamin Wang
d58a0c0434
Merge pull request #14177 from ahrtr/3.4_lease_renew_linearizable
[3.4] Support linearizable renew lease for 3.4
2022-07-19 16:39:00 +08:00
Hitoshi Mitake
ecd91da40d server/auth: protect rangePermCache with a RW lock
Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com>
2022-07-19 15:51:48 +09:00
Benjamin Wang
07d2b1d626 support linearizable renew lease for 3.4
Cherry pick https://github.com/etcd-io/etcd/pull/13932 to 3.4.

When etcdserver receives a LeaseRenew request, it may be still in
progress of processing the LeaseGrantRequest on exact the same
leaseID. Accordingly it may return a TTL=0 to client due to the
leaseID not found error. So the leader should wait for the appliedID
to be available before processing client requests.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-07-19 13:34:55 +08:00
Benjamin Wang
4636a5fab4 Bump version to 3.4.19
Signed-off-by: Benjamin Wang <wachao@vmware.com>
v3.4.19
2022-07-12 16:18:45 +08:00
Benjamin Wang
06561ae4bf
Merge pull request #14210 from ahrtr/fix_release_script
[3.4] Fix pipeline failure for release test
2022-07-12 16:06:33 +08:00
Benjamin Wang
be0ce4f15b fix pipeline failure for release test
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-07-12 08:31:59 +08:00
Benjamin Wang
d3dfc9b796
Merge pull request #14204 from lavacat/release-3.4-balancer-tests
clientv3/balance: fixed flaky balancer tests
2022-07-12 06:14:35 +08:00
Bogdan Kanivets
185f203528 clientv3/balance: fixed flaky balancer tests
- added verification step to indirectly verify that all peers are in balancer subconn list

Signed-off-by: Bogdan Kanivets <bkanivets@apple.com>
2022-07-11 14:43:58 -07:00
Benjamin Wang
7de53273dd
Merge pull request #14205 from ahrtr/3.4_release_script
[3.4] Update release scripts for release-3.4
2022-07-11 20:06:06 +08:00
Benjamin Wang
6cc9416ae5 backport release test to 3.4
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-07-11 19:47:08 +08:00
Benjamin Wang
e6b3d97712 Update release scripts for release-3.4
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-07-11 16:06:32 +08:00
Marek Siarkowicz
852ac37bc0
Merge pull request #14200 from ahrtr/3.4_pipeline_race
set RACE as true for linux-amd64-unit and linux-amd64-grpcproxy
2022-07-08 10:23:21 +02:00
Benjamin Wang
8c1c5fefdb set RACE as true for linux-amd64-unit and linux-amd64-grpcproxy
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-07-08 08:37:31 +08:00
Marek Siarkowicz
0c6063fa82
Merge pull request #14192 from ahrtr/3.4_bump_yaml
[3.4] Bump gopkg.in/yaml.v2 v2.2.2 -> v2.4.0 due to: CVE-2019-11254
2022-07-05 14:32:09 +02:00
Benjamin Wang
860dc149b2 Bump gopkg.in/yaml.v2 v2.2.8 -> v2.4.0 due to: CVE-2019-11254
Cherry pick https://github.com/etcd-io/etcd/pull/13616 to 3.4.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-07-05 06:26:06 +08:00
Marek Siarkowicz
f0256eeec9
Merge pull request #14179 from lavacat/release-3.4-crypto
[backport 3.4] Update golang.org/x/crypto to latest
2022-07-04 11:57:58 +02:00
Bogdan Kanivets
576a798bf9 [backport 3.4] Update golang.org/x/crypto to latest
Update crypto to address CVE-2022-27191.

The CVE fix is added in 0.0.0-20220315160706-3147a52a75dd but this
change updates to latest.

Backport of https://github.com/etcd-io/etcd/pull/13996

Signed-off-by: Bogdan Kanivets <bkanivets@apple.com>
2022-06-30 23:08:13 -07:00
Benjamin Wang
bae61786fc
Merge pull request #14183 from ahrtr/3.4_pipeline_issues_20220630
[3.4] Fix pipeline failures in 3.4
2022-07-01 05:36:29 +08:00
Benjamin Wang
8160e9ebe2 disable test cases on certificate-based authentication which isn't supported by gRPC proxy.
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-30 14:11:54 +08:00
Benjamin Wang
5b3f269159 replace all 3.4 certificates and keys with the files from 3.5
Fix the following error in integration pipeline,
```
=== RUN   TestTLSReloadCopy
    v3_grpc_test.go:1754: tls: failed to find any PEM data in key input
    v3_grpc_test.go:1754: tls: private key does not match public key
    v3_grpc_test.go:1754: tls: private key does not match public key
    v3_grpc_test.go:1754: tls: private key does not match public key
```

Refer to https://github.com/etcd-io/etcd/runs/7123775361?check_suite_focus=true

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-30 13:21:48 +08:00
Benjamin Wang
bb9113097a fix test failure in TestCtlV3WatchClientTLS
Also refer to the following commit in 3.5,
093282f5ea

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-30 10:19:03 +08:00
Benjamin Wang
f169e5dcba
Merge pull request #14151 from ahrtr/3.4_skip_TestWatchRequestProgress_proxy
[3.4] Skip WatchRequestProgress test in grpc-proxy mode.
2022-06-29 05:40:05 +08:00
Benjamin Wang
6958ee8ff2 Skip WatchRequestProgress test in grpc-proxy mode.
We shouldn't fail the grpc-server (completely) by a not implemented RPC.
Failing whole server by remote request is anti-pattern and security
risk.

Refer to https://github.com/etcd-io/etcd/runs/7034342964?check_suite_focus=true#step:5:2284

```
=== RUN   TestWatchRequestProgress/1-watcher
panic: not implemented
goroutine 83024 [running]:
go.etcd.io/etcd/proxy/grpcproxy.(*watchProxyStream).recvLoop(0xc009232f00, 0x4a73e1, 0xc00e2406e0)
	/home/runner/work/etcd/etcd/proxy/grpcproxy/watch.go:265 +0xbf2
go.etcd.io/etcd/proxy/grpcproxy.(*watchProxy).Watch.func1(0xc0038a3bc0, 0xc009232f00)
	/home/runner/work/etcd/etcd/proxy/grpcproxy/watch.go:125 +0x70
created by go.etcd.io/etcd/proxy/grpcproxy.(*watchProxy).Watch
	/home/runner/work/etcd/etcd/proxy/grpcproxy/watch.go:123 +0x73b
FAIL	go.etcd.io/etcd/clientv3/integration	222.813s
FAIL
```

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-29 05:12:43 +08:00
Marek Siarkowicz
f1c59dcfac
Merge pull request #14170 from ahrtr/3.4_proxy_fix_20220628
Fix deadlock in 'go test -tags cluster_proxy -v ./integration/... ./client'
2022-06-28 17:56:44 +02:00