748 Commits

Author SHA1 Message Date
Marek Siarkowicz
eed94f6f94 server: Run health check tests in subtests
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2023-11-27 09:31:00 -08:00
Marek Siarkowicz
2f6c84e91d server: Rename test case expect fields
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2023-11-27 09:31:00 -08:00
Marek Siarkowicz
c6784a7e82 server: Use named struct initialization in healthcheck test
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2023-11-27 09:31:00 -08:00
sharath sivakumar
f26074ae56 CVE-2023-47108: Backport go.opentelemetry.io/otel@v1.20.0 and go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc@v0.46.0
Signed-off-by: sharath sivakumar <sharath.sivakumar@mollie.com>
2023-11-15 14:09:07 +01:00
Marek Siarkowicz
0223ca52b8 version: bump up to 3.5.10 2023-10-27 12:33:25 +02:00
Benjamin Wang
88beb6ca47 bump bbolt to 1.3.8 for etcd 3.5
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-10-26 16:32:07 +01:00
Benjamin Wang
3521aaad9a
Merge pull request #16790 from chaochn47/3.5-grpc-update
[3.5] Upgrade gRPC-go to 1.58.3
2023-10-19 10:21:36 +01:00
Marek Siarkowicz
31b7c58fd5
Merge pull request #16637 from serathius/check-cluster-id-release-3.5
[release-3.5] etcdserver: add cluster id check for hashKVHandler
2023-10-17 20:58:21 +02:00
Chao Chen
1aa4aa8a96 3.5: upgrade gRPC-go to 1.58.3
The last step with gRPC update behavior changes auditing to resolve CVE #16740 in 3.5

This PR backports #14922, #16338, #16587, #16630, #16636 and #16739 to release-3.5.

Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-17 11:07:08 -07:00
caojiamingalan
04cfb4c660 etcdserver: add cluster id check for hashKVHandler
Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com>
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-10-17 13:27:47 +02:00
Chao Chen
6f4fa5a27a [release-3.5]: upgrade gRPC-go to v1.52.0
backport https://github.com/etcd-io/etcd/pull/14834 and https://github.com/etcd-io/etcd/pull/16324

Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-16 21:43:00 -07:00
Chao Chen
db16069588 backport #14125 to release-3.5: Update to grpc-1.47 (and fix the connection-string format)
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-12 09:46:49 -07:00
Marek Siarkowicz
404eeaa74e Return to default write scheduler since golang.org/x/net@v0.11.0 started using round robin
Introduction of round robin 120fc906b3
Added in v0.10.0 https://github.com/golang/net/compare/v0.10.0...v0.11.0

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-10-12 15:19:17 +02:00
Allen Ray
24ee8e491f bump golang.org/x/net to 0.17.0
Part of https://github.com/etcd-io/etcd/issues/16740

Signed-off-by: Allen Ray <alray@redhat.com>
2023-10-11 10:43:51 -04:00
James Blair
164175dfe3
Backport adding unit test for socket options.
Co-authored-by: Shawn Gerrard <shawn.gerrard@gmail.com>
Signed-off-by: James Blair <mail@jamesblair.net>
2023-08-18 19:14:57 +12:00
James Blair
2a7bffc63e
Backport export reuse-port and reuse-address
so that they cabn be configured via config file.

Co-authored-by: Shawn Gerrard <shawn.gerrard@gmail.com>
Signed-off-by: James Blair <mail@jamesblair.net>
2023-08-18 12:05:19 +12:00
James Blair
f62a894ae7
Fix goword failure in rafthttp/transport.go.
Signed-off-by: James Blair <mail@jamesblair.net>
2023-08-11 21:47:30 +12:00
James Blair
9c7c8c6b3f
Backport update to golang 1.20 minor release.
Signed-off-by: James Blair <mail@jamesblair.net>
2023-08-11 21:16:01 +12:00
caojiamingalan
eb9bfaa983 Follow up https://github.com/etcd-io/etcd/pull/16068#discussion_r1263667496
Add a UnsafeReadScheduledCompact and UnsafeReadFinishedCompact

Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com>
2023-07-18 10:54:16 -05:00
caojiamingalan
6ac9d94d67 etcdserver: backport check scheduledCompactKeyName and finishedCompactKeyName before writing hash to release-3.5.
Fix #15919.
Check ScheduledCompactKeyName and FinishedCompactKeyName
before writing hash to hashstore.
If they do not match, then it means this compaction has once been interrupted and its hash value is invalid. In such cases, we won't write the hash values to the hashstore, and avoids the incorrect corruption alarm.

Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com>
2023-07-14 19:22:38 -05:00
Lan Liang
960cd26bef Backport #13577
Disable auth gracefully without impacting existing watchers.

Signed-off-by: Lan Liang <gcslyp@gmail.com>
2023-07-14 12:50:21 +08:00
Thomas Jungblut
4425ef572e Adding optional revision bump and mark compacted to snapshot restore
Signed-off-by: Allen Ray <alray@redhat.com>
Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
2023-07-03 12:57:12 +02:00
Benjamin Wang
f565a94844
Merge pull request #16019 from tjungblu/putauthshort_3.5
[3.5] Early exit auth check on lease puts
2023-06-21 11:04:51 +01:00
Benjamin Wang
306c60a083
Merge pull request #16048 from kkkkun/cp-14457-to-3.5
[3.5] etcdserver: fix corruption check when server has just been compacted
2023-06-19 09:33:57 +01:00
Thomas Jungblut
423f951409 Add first unit test for authApplierV3
This contains a slight refactoring to expose enough information
to write meaningful tests for auth applier v3.

Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
2023-06-16 09:42:09 +02:00
Thomas Jungblut
b2fb75d147 Early exit auth check on lease puts
Mitigates etcd-io#15993 by not checking each key individually for permission
when auth is entirely disabled or admin user is calling the method.

Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
2023-06-16 09:14:41 +02:00
kkkkun
8cffdbafba etcdserver: fix corruption check when server has just been compacted
Signed-off-by: kkkkun <scuzk373x@gmail.com>
2023-06-11 22:27:02 +08:00
scuzk373x@gmai.com
d3d530c562 add compact hash check to help
Signed-off-by: scuzk373x@gmai.com <zhuanwajiang@pinduoduo.com>
2023-06-08 14:26:11 +08:00
yellowzf
ecfed91e50 grpcproxy: fix memberlist results not update when proxy node down
If start grpc proxy with --resolver-prefix, memberlist will return all alive proxy nodes, when one grpc proxy node is down, it is expected to not return the down node, but it is still return

Signed-off-by: yellowzf <zzhf3311@163.com>
2023-05-16 11:35:31 +08:00
Marek Siarkowicz
bdbbde998b version: bump up to 3.5.9 2023-05-11 13:39:43 +02:00
Hitoshi Mitake
d1b1aa9dbe etcdserver: protect lease timetilive with auth
Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com>
Co-authored-by: Benjamin Wang <wachao@vmware.com>
2023-05-08 22:45:38 +09:00
sharathsivakumar
c0f2954e9f
server: backport 15743, improved description of --initial-cluster-state flag
Signed-off-by: sharathsivakumar <mailssr9@gmail.com>
2023-04-25 08:02:39 +02:00
Marek Siarkowicz
217d183e5a version: bump up to 3.5.8 2023-04-13 12:08:47 +02:00
Marek Siarkowicz
9d2cda4e44
Merge pull request #15676 from ahrtr/jwt_panic_3.5_20230410
[3.5] etcdserver: verify field 'username' and 'revision' present when decoding a JWT token
2023-04-11 12:21:50 +02:00
Benjamin Wang
cd019255ba etcdserver: Guarantee order of requested progress notifications
Progress notifications requested using ProgressRequest were sent
directly using the ctrlStream, which means that they could race
against watch responses in the watchStream.

This would especially happen when the stream was not synced - e.g. if
you requested a progress notification on a freshly created unsynced
watcher, the notification would typically arrive indicating a revision
for which not all watch responses had been sent.

This changes the behaviour so that v3rpc always goes through the watch
stream, using a new RequestProgressAll function that closely matches
the behaviour of the v3rpc code - i.e.

1. Generate a message with WatchId -1, indicating the revision for
   *all* watchers in the stream

2. Guarantee that a response is (eventually) sent

The latter might require us to defer the response until all watchers
are synced, which is likely as it should be. Note that we do *not*
guarantee that the number of progress notifications matches the number
of requests, only that eventually at least one gets sent.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-04-11 09:51:48 +08:00
Benjamin Wang
643e6e1993 etcdserver: verify field 'username' and 'revision' present when decoding a JWT token
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-04-10 08:21:43 +08:00
Benjamin Wang
5872b80ed5
Merge pull request #15512 from engow/automated-cherry-pick-of-#15432-origin-release-3.5
[3.5] server/auth: fix auth panic bug when user changes password
2023-04-08 06:22:37 +08:00
Marek Siarkowicz
4501fd88c7
Merge pull request #15619 from mitake/3.5-backport-15294
[3.5] backport 15294
2023-04-07 17:47:23 +02:00
Benjamin Wang
b31caa3f12 set zap logging to wsproxy
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-04-07 13:33:39 +08:00
Benjamin Wang
e6c2e380a9 security: remove password after authenticating the user
fix https://nvd.nist.gov/vuln/detail/CVE-2021-28235

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-04-06 20:12:02 +09:00
Hitoshi Mitake
e6eeca6885 server/auth: disallow creating empty permission ranges
Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com>
Co-authored-by: Benjamin Wang <wachao@vmware.com>
2023-04-03 22:55:55 +09:00
Marek Siarkowicz
073c530989 server: Fix defer function closure escape
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-30 16:01:29 +02:00
Marek Siarkowicz
c0421c7330 server: Add --listen-client-http-urls flag to allow running grpc server separate from http server
Difference in load configuration for watch delay tests show how huge the
impact is. Even with random write scheduler grpc under http
server can only handle 500 KB with 2 seconds delay. On the other hand,
separate grpc server easily hits 10, 100 or even 1000 MB within 100 miliseconds.

Priority write scheduler that was used in most previous releases
is far worse than random one.

Tests configured to only 5 MB to avoid flakes and taking too long to fill
etcd.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-30 15:53:11 +02:00
Marek Siarkowicz
2d5f48a7ef server: Pick one address that all grpc gateways connect to
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-30 15:11:59 +02:00
Marek Siarkowicz
a9e0a04c9a server: Extract resolveUrl helper function
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-30 15:11:59 +02:00
Marek Siarkowicz
245067b15d server: Separate client listener grouping from serving
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-30 15:11:59 +02:00
Marek Siarkowicz
63576a25f5 refactor: Use proper variable names for urls
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-30 15:11:58 +02:00
engow
f7ac9dfcd6 sever/auth: fix addUserWithNoOption of store_test
Signed-off-by: engow <engow@hotmail.com>
2023-03-30 19:13:33 +08:00
tangcong
dcb1bf6078 server/auth: fix auth panic bug when user changes password
Signed-off-by: tangcong <tangcong506@foxmail.com>
Signed-off-by: engow <engow@hotmail.com>
2023-03-30 19:13:33 +08:00
Wei Fu
9e974792f9 server/embed: fix data race when start insecure grpc
There are two goroutines accessing the `gs` grpc server var. Before
insecure `gs` server start, the `gs` can be changed to secure server and
then the client will fail to connect to etcd with insecure request. It
is data-race. We should use argument for reference in the new goroutine.

fix: #15495

Signed-off-by: Wei Fu <fuweid89@gmail.com>
(cherry picked from commit a9988e2625eede1af81d189b5f2ecf7d4af3edf1)
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-03-20 20:41:49 +08:00