89 Commits

Author SHA1 Message Date
AlexStocks
0263597ba8 Introduce compaction sleep interval flag
This is a backporting cherry-pick of the following commits:

- add flagsline

Signed-off-by: Jalin Wang <JalinWang@outlook.com>

- etcdserver: rename defaultCompactionSleepInterval var (#18495)

etcdserver: rename `minimumBatchInterval`  to `defaultCompactionSleepInterval` and `defaultCompactBatchLimit` to `defaultCompactionBatchLimit`

Signed-off-by: Jalin Wang <JalinWang@outlook.com>
(cherry picked from commit 2c53be7c5d91e9b618342d63d2d06c3b265abee4)

- test: add CompactionSleepInterval in FakeStore's config

After setting the ComparionSleepInterval, we can use time.Ticker
instead of time.After to optimize the scheduleComparison(),
otherwise it will fail in the 'TestStoreCompact(t)' test.

Signed-off-by: guozhao <guozhao@360.cn>
(cherry picked from commit fab8474ef8370a089fe0174e28c197b5b93898df)

- add sleep interval

(cherry picked from commit 184b0e5d4964f1115590acec50fa5e584a2f7770)
Signed-off-by: Jalin Wang <JalinWang@outlook.com>
2024-08-30 10:04:47 +08:00
James Blair
d6479db0ef
Merge pull request #18289 from jmhbnz/backport-enforce-listen-metrics-tls-info
[3.5] Backport fix(server/embed) enforce non-empty client TLS if scheme is https/unixs
2024-07-16 15:10:55 +12:00
Benjamin Wang
12854ec00d Differentiate the warning message for rejected client and peer connections
Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
2024-07-14 08:50:54 +01:00
James Blair
c6b0b55847
Backport: fix(server/embed): enforce non-empty client TLS if scheme is https/unixs
Backports a657f06, 22f20a8, 497f1a4 and 3e86af6 from #18186.

Also backports required test util elements of 4c77726 from #17661.

Signed-off-by: James Blair <mail@jamesblair.net>
2024-07-09 21:09:23 +12:00
lhy1024
f8befd94f6 Support multiple values for allowed client and peer TLS identities(#18015)
Signed-off-by: lhy1024 <admin@liudos.us>
2024-06-13 14:32:24 +08:00
Seena Fallah
8bc3271af4
config: support AllowedCN and AllowedHostname through config file
Allow setting AllowedCN and AllowedHostname tls fields through config file for peer transport security.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2024-05-23 23:37:11 +05:30
Thomas Jungblut
750bc0b1e4 gRPC health server sets serving status to NOT_SERVING on defrag
gRPC health server sets serving status to NOT_SERVING on defrag
Backport from 3.6 in #16278

Co-authored-by: Chao Chen <chaochn@amazon.com>
Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
2024-04-30 15:09:51 +02:00
Siyuan Zhang
464df4a979 add experimental-snapshot-catchup-entries flag.
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-04-15 17:04:44 -07:00
Chao Chen
a1aaa51cf1 add uds test cases into e2e TestAuthority
Signed-off-by: Chao Chen <chaochn@amazon.com>
2024-02-19 13:26:15 +00:00
Allen Ray
3d64877dc2 [3.5] Update to go1.21
Signed-off-by: Allen Ray <alray@redhat.com>
2024-02-02 14:25:53 -05:00
Benjamin Wang
926d0227bd print error log when creating peer listener failed
Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
2024-01-24 13:50:38 +00:00
Marek Siarkowicz
d4e86108e3
Merge pull request #17000 from siyuanfoundation/livez-bp-3.5-step1
[3.5] Backport healthcheck code cleanup
2023-11-27 19:59:40 +01:00
Marek Siarkowicz
46e394242f server: Split metrics and health code
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2023-11-27 09:31:00 -08:00
Marek Siarkowicz
8ab1c0f25b server: Cover V3 health with tests
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2023-11-27 09:31:00 -08:00
James Blair
5a564d56d7
Backport server: Have tracingExporter own resources it initialises.
Signed-off-by: James Blair <mail@jamesblair.net>
2023-11-26 10:22:10 +13:00
James Blair
3a54851f48
Backport server: Add sampling rate to distributed tracing.
ExperimentalDistributedTracingSamplingRatePerMillion is the
number of samples to collect per million spans.
Defaults to 0.

Signed-off-by: James Blair <mail@jamesblair.net>
2023-11-26 10:22:10 +13:00
Chao Chen
db16069588 backport #14125 to release-3.5: Update to grpc-1.47 (and fix the connection-string format)
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-12 09:46:49 -07:00
Marek Siarkowicz
404eeaa74e Return to default write scheduler since golang.org/x/net@v0.11.0 started using round robin
Introduction of round robin 120fc906b3
Added in v0.10.0 https://github.com/golang/net/compare/v0.10.0...v0.11.0

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-10-12 15:19:17 +02:00
James Blair
164175dfe3
Backport adding unit test for socket options.
Co-authored-by: Shawn Gerrard <shawn.gerrard@gmail.com>
Signed-off-by: James Blair <mail@jamesblair.net>
2023-08-18 19:14:57 +12:00
James Blair
2a7bffc63e
Backport export reuse-port and reuse-address
so that they cabn be configured via config file.

Co-authored-by: Shawn Gerrard <shawn.gerrard@gmail.com>
Signed-off-by: James Blair <mail@jamesblair.net>
2023-08-18 12:05:19 +12:00
Benjamin Wang
b31caa3f12 set zap logging to wsproxy
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-04-07 13:33:39 +08:00
Marek Siarkowicz
073c530989 server: Fix defer function closure escape
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-30 16:01:29 +02:00
Marek Siarkowicz
c0421c7330 server: Add --listen-client-http-urls flag to allow running grpc server separate from http server
Difference in load configuration for watch delay tests show how huge the
impact is. Even with random write scheduler grpc under http
server can only handle 500 KB with 2 seconds delay. On the other hand,
separate grpc server easily hits 10, 100 or even 1000 MB within 100 miliseconds.

Priority write scheduler that was used in most previous releases
is far worse than random one.

Tests configured to only 5 MB to avoid flakes and taking too long to fill
etcd.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-30 15:53:11 +02:00
Marek Siarkowicz
2d5f48a7ef server: Pick one address that all grpc gateways connect to
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-30 15:11:59 +02:00
Marek Siarkowicz
a9e0a04c9a server: Extract resolveUrl helper function
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-30 15:11:59 +02:00
Marek Siarkowicz
245067b15d server: Separate client listener grouping from serving
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-30 15:11:59 +02:00
Marek Siarkowicz
63576a25f5 refactor: Use proper variable names for urls
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-30 15:11:58 +02:00
Wei Fu
9e974792f9 server/embed: fix data race when start insecure grpc
There are two goroutines accessing the `gs` grpc server var. Before
insecure `gs` server start, the `gs` can be changed to secure server and
then the client will fail to connect to etcd with insecure request. It
is data-race. We should use argument for reference in the new goroutine.

fix: #15495

Signed-off-by: Wei Fu <fuweid89@gmail.com>
(cherry picked from commit a9988e2625eede1af81d189b5f2ecf7d4af3edf1)
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-03-20 20:41:49 +08:00
Benjamin Wang
11ca1d356a
Merge pull request #15483 from jmhbnz/release-3.5-backport
[3.5] Backport tls 1.3 support
2023-03-16 06:06:45 +08:00
James Blair
358bcf3fb6
Backport tls 1.3 support.
Signed-off-by: James Blair <mail@jamesblair.net>
2023-03-15 14:10:14 +13:00
Marek Siarkowicz
55bfad950c server: Switch back to random scheduler to improve resilience to watch starvation
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-14 09:16:46 +01:00
Benjamin Wang
ec6f0a74ba
Merge pull request #14500 from dusk125/release-3.5
Release-3.5: server/etcdmain: add configurable cipher list to gRPC proxy listener
2022-10-16 06:35:17 +08:00
Benjamin Wang
f6c4c84da3 etcdserver: added more debug log for the purgeFile goroutine
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-10-12 19:28:32 +08:00
Sahdev Zala
db55011d7c server/etcdmain: add configurable cipher list to gRPC proxy listener
Signed-off-by: Allen Ray <alray@redhat.com>
2022-09-29 11:41:57 -04:00
Marek Siarkowicz
5660bf0e7f server: Make corrtuption check optional and period configurable
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-09-07 15:11:56 +02:00
Marek Siarkowicz
4a75e3d52d server: Refactor compaction checker
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-09-07 15:11:55 +02:00
Marek Siarkowicz
35cbdf3961 server: Extract corruption detection to dedicated struct
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-09-07 15:11:55 +02:00
Benjamin Wang
9ea5b1ba22 Refactor the keepAliveListener and keepAliveConn
Only `net.TCPConn` supports `SetKeepAlive` and `SetKeepAlivePeriod`
by default, so if you want to warp multiple layers of net.Listener,
the `keepaliveListener` should be the one which is closest to the
original `net.Listener` implementation, namely `TCPListener`.

Also refer to https://github.com/etcd-io/etcd/pull/14356

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-08-20 15:03:15 +08:00
Benjamin Wang
8fdca41cd8 Change default sampling rate from 100% to 0%
Refer to https://github.com/etcd-io/etcd/pull/14318

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-08-07 07:19:30 +08:00
Benjamin Wang
5a86ae2c33 move setupTracing into a separate file config_tracing.go
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-08-07 07:17:27 +08:00
Mike Dame
4c013c91e9
Change default sampling rate from 100% to 0%
This changes the default parent-based trace sampling rate from
100% to 0%. Due to the high QPS etcd can handle, having 100% trace
sampling leads to very high resource usage. Defaulting to 0% means
that only already-sampled traces will be sampled in etcd.

Fixes #14310

Signed-off-by: Mike Dame <mikedame@google.com>
2022-08-05 15:00:40 +00:00
Benjamin Wang
437f3778d0 Add flag --max-concurrent-streams to set the max concurrent stream each client can open at a time
Also refer to https://github.com/etcd-io/etcd/pull/14169#discussion_r917154243

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-07-13 14:18:15 +08:00
Marek Siarkowicz
d00e89db2e server: Require either cluster version v3.6 or --experimental-enable-lease-checkpoint-persist to persist lease remainingTTL
To avoid inconsistant behavior during cluster upgrade we are feature
gating persistance behind cluster version. This should ensure that
all cluster members are upgraded to v3.6 before changing behavior.

To allow backporting this fix to v3.5 we are also introducing flag
--experimental-enable-lease-checkpoint-persist that will allow for
smooth upgrade in v3.5 clusters with this feature enabled.
2021-12-02 16:54:10 +01:00
Marek Siarkowicz
58d2b12a50 client: Add grpc authority header integration tests 2021-09-30 12:15:32 +02:00
tangcong
627d91c89d fix self-signed-cert-validity parameter cannot be specified in the config file 2021-07-30 07:53:43 +08:00
spacewander
53e04b2615 embed: unlimit the recv msg size of grpc-gateway
Ensure the client which access etcd via grpc-gateway won't
be limited by the MaxCallRecvMsgSize. Here we choose the same
default value of etcdcli as grpc-gateway's MaxCallRecvMsgSize.

Fix https://github.com/etcd-io/etcd/issues/12576
2021-06-11 07:53:49 +08:00
Mr-Mu
98bbc0107f TestLogRotation add log output relative path
TestLogRotation add test log output relative path test
2021-06-09 14:12:07 -04:00
Mr-Mu
fed3bc3084 [Fix]滚动日志路径解析不正确
例如  rotate:test.log 路径解析不正确
2021-06-09 14:12:07 -04:00
Piotr Tabor
f82b5cb776
Bring back original NewZapCoreLoggerBuilder api. (#12973)
The funcion signature has been changed in:
eafbc8c57efc716644c328f63677ca7eadeebdfe .
Instead we should have added new method `NewZapLoggerBuilder()`.
2021-05-16 10:26:10 -07:00
Piotr Tabor
85341e08f2
Merge pull request #12968 from serathius/logger-simplify
server: Simplify passing logger setup by passing only logger
2021-05-15 15:58:00 +02:00