60 Commits

Author SHA1 Message Date
AlexStocks
0263597ba8 Introduce compaction sleep interval flag
This is a backporting cherry-pick of the following commits:

- add flagsline

Signed-off-by: Jalin Wang <JalinWang@outlook.com>

- etcdserver: rename defaultCompactionSleepInterval var (#18495)

etcdserver: rename `minimumBatchInterval`  to `defaultCompactionSleepInterval` and `defaultCompactBatchLimit` to `defaultCompactionBatchLimit`

Signed-off-by: Jalin Wang <JalinWang@outlook.com>
(cherry picked from commit 2c53be7c5d91e9b618342d63d2d06c3b265abee4)

- test: add CompactionSleepInterval in FakeStore's config

After setting the ComparionSleepInterval, we can use time.Ticker
instead of time.After to optimize the scheduleComparison(),
otherwise it will fail in the 'TestStoreCompact(t)' test.

Signed-off-by: guozhao <guozhao@360.cn>
(cherry picked from commit fab8474ef8370a089fe0174e28c197b5b93898df)

- add sleep interval

(cherry picked from commit 184b0e5d4964f1115590acec50fa5e584a2f7770)
Signed-off-by: Jalin Wang <JalinWang@outlook.com>
2024-08-30 10:04:47 +08:00
James Blair
c6b0b55847
Backport: fix(server/embed): enforce non-empty client TLS if scheme is https/unixs
Backports a657f06, 22f20a8, 497f1a4 and 3e86af6 from #18186.

Also backports required test util elements of 4c77726 from #17661.

Signed-off-by: James Blair <mail@jamesblair.net>
2024-07-09 21:09:23 +12:00
lhy1024
f8befd94f6 Support multiple values for allowed client and peer TLS identities(#18015)
Signed-off-by: lhy1024 <admin@liudos.us>
2024-06-13 14:32:24 +08:00
Thomas Jungblut
750bc0b1e4 gRPC health server sets serving status to NOT_SERVING on defrag
gRPC health server sets serving status to NOT_SERVING on defrag
Backport from 3.6 in #16278

Co-authored-by: Chao Chen <chaochn@amazon.com>
Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
2024-04-30 15:09:51 +02:00
Siyuan Zhang
c391d69b08 test: add checking snapshot be sent in mix_version_test.
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-04-18 19:08:05 +00:00
Siyuan Zhang
464df4a979 add experimental-snapshot-catchup-entries flag.
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-04-15 17:04:44 -07:00
shihuixing
31f097ee9e added dial-keepalive-time、dial-keepalive-timeout、permit-without-stream arguments to the grpc-proxy
Signed-off-by: shihuixing <shihuixing@jd.com>
2024-02-18 20:44:25 +08:00
Ishan Tyagi
0ac8e50a1a Add a unit tests and missing flags in etcd help.
Signed-off-by: ishan16696 <ishan.tyagi@sap.com>
2024-01-05 14:46:42 +05:30
Ishan Tyagi
f87048a92b Add missing flag in etcd help.
Signed-off-by: ishan16696 <ishan.tyagi@sap.com>
2024-01-02 18:08:20 +05:30
Marek Siarkowicz
d4e86108e3
Merge pull request #17000 from siyuanfoundation/livez-bp-3.5-step1
[3.5] Backport healthcheck code cleanup
2023-11-27 19:59:40 +01:00
Marek Siarkowicz
46e394242f server: Split metrics and health code
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2023-11-27 09:31:00 -08:00
James Blair
3a54851f48
Backport server: Add sampling rate to distributed tracing.
ExperimentalDistributedTracingSamplingRatePerMillion is the
number of samples to collect per million spans.
Defaults to 0.

Signed-off-by: James Blair <mail@jamesblair.net>
2023-11-26 10:22:10 +13:00
scuzk373x@gmai.com
d3d530c562 add compact hash check to help
Signed-off-by: scuzk373x@gmai.com <zhuanwajiang@pinduoduo.com>
2023-06-08 14:26:11 +08:00
sharathsivakumar
c0f2954e9f
server: backport 15743, improved description of --initial-cluster-state flag
Signed-off-by: sharathsivakumar <mailssr9@gmail.com>
2023-04-25 08:02:39 +02:00
Marek Siarkowicz
c0421c7330 server: Add --listen-client-http-urls flag to allow running grpc server separate from http server
Difference in load configuration for watch delay tests show how huge the
impact is. Even with random write scheduler grpc under http
server can only handle 500 KB with 2 seconds delay. On the other hand,
separate grpc server easily hits 10, 100 or even 1000 MB within 100 miliseconds.

Priority write scheduler that was used in most previous releases
is far worse than random one.

Tests configured to only 5 MB to avoid flakes and taking too long to fill
etcd.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-30 15:53:11 +02:00
Marek Siarkowicz
63576a25f5 refactor: Use proper variable names for urls
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-30 15:11:58 +02:00
James Blair
358bcf3fb6
Backport tls 1.3 support.
Signed-off-by: James Blair <mail@jamesblair.net>
2023-03-15 14:10:14 +13:00
Aleksandr Razumov
c91978077b client/pkg/fileutil: add missing logger to {Create,Touch}DirAll
Also populate it to every invocation.

Signed-off-by: WangXiaoxiao <1141195807@qq.com>
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-11-17 14:08:30 +01:00
Sahdev Zala
db55011d7c server/etcdmain: add configurable cipher list to gRPC proxy listener
Signed-off-by: Allen Ray <alray@redhat.com>
2022-09-29 11:41:57 -04:00
Marek Siarkowicz
5660bf0e7f server: Make corrtuption check optional and period configurable
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-09-07 15:11:56 +02:00
Vitalii Levitskii
67e4c59e01 Backport of pull/14354 to 3.5.5
Signed-off-by: Vitalii Levitskii <vitalii@uber.com>
2022-08-29 15:58:17 +03:00
Benjamin Wang
437f3778d0 Add flag --max-concurrent-streams to set the max concurrent stream each client can open at a time
Also refer to https://github.com/etcd-io/etcd/pull/14169#discussion_r917154243

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-07-13 14:18:15 +08:00
Marek Siarkowicz
a060b42e47 server: Use default logging configuration instead of zap production one
This fixes problem where logs json changes format of timestamp.
2022-04-01 12:23:44 +02:00
Marek Siarkowicz
d00e89db2e server: Require either cluster version v3.6 or --experimental-enable-lease-checkpoint-persist to persist lease remainingTTL
To avoid inconsistant behavior during cluster upgrade we are feature
gating persistance behind cluster version. This should ensure that
all cluster members are upgraded to v3.6 before changing behavior.

To allow backporting this fix to v3.5 we are also introducing flag
--experimental-enable-lease-checkpoint-persist that will allow for
smooth upgrade in v3.5 clusters with this feature enabled.
2021-12-02 16:54:10 +01:00
Piotr Tabor
4af7fc393c Expose clientv3.CreateDefaultZapLoggerConfig
Need raised in
f3f4259d3f.

BTW: Alligned the implementation of grpclogs with what we use in embed server, so reduced the client code dependencies.
2021-05-26 12:52:46 +02:00
Gyuho Lee
e2d67f2e3b
Merge pull request #12956 from gyuho/rename-to-main
*: rename "master" branch references to "main" in source code
2021-05-13 08:26:33 -07:00
Gyuho Lee
77c8033739 server: rename "master" branch references
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-12 10:37:35 -07:00
Piotr Tabor
ead81df948 Disallow -v2-deprecation>'not-yet' combined with --enable-v2 2021-05-12 18:09:34 +02:00
Piotr Tabor
7c508741b3 Adding --v2-deprecation flag. 2021-05-12 18:09:34 +02:00
Piotr Tabor
1929aa0a36
Merge pull request #12948 from serathius/logger
client: Allow setting zap logger in config
2021-05-12 13:49:48 +02:00
Marek Siarkowicz
06afe87b34 client: Allow setting zap logger in config 2021-05-12 12:15:20 +02:00
Piotr Tabor
e0a8484c8f
Merge pull request #12941 from serathius/defrag
etcdserver: Implement running defrag if freeable space will exceed provided threshold (on boot)
2021-05-12 09:26:56 +02:00
Gyuho Lee
70f7e0d70f
Merge pull request #12929 from gyuho/github-worker-arm64-support 2021-05-11 10:29:53 -07:00
Marek Siarkowicz
efc8505739 etcdserver: Implement running defrag if freeable space will exceed privided threshold 2021-05-11 14:00:29 +02:00
Piotr Tabor
269f22c837 Deprecate V2 API: --enable-v2 and v2v3
Flags `--experimental-enable-v2v3` and '-enable-v2' will raise a warning in 3.5,
in 3.6 they are schedule for decomissioning, such that v2store can stop be written in 3.7.

Deprecation plan in: https://github.com/etcd-io/etcd/issues/12913
2021-05-10 16:19:52 +02:00
Lili Cosic
1a718a958e Add initial Tracing with OpenTelemetry 2021-05-10 10:44:40 +02:00
Gyuho Lee
b9c000fb8f server/etcdmain: officially supports "arm64"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-10 00:34:10 -07:00
Piotr Tabor
f5717016cf
Merge pull request #12797 from lzhfromustc/3_23
server/etcdmain and tests: Fix goroutine leaks
2021-05-10 08:29:17 +02:00
Sam Batschelet
6decbe15db
Merge pull request #12774 from hexfusion/add-log-rotate
server: add support for log rotation
2021-05-07 12:18:10 -04:00
Sam Batschelet
82b2d5c67d server: add support for log rotation
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-05-07 08:39:51 -04:00
Gyuho Lee
344c9f3930
Merge pull request #12896 from wilsonwang371/profiling-txn2
server: make applier use ReadTx() in Txn() instead of ConcurrentReadTx()
2021-05-06 01:59:14 -07:00
Wilson Wang
98083ea914 server: add experimental flag for using shared buffer in transacton write 2021-05-04 11:59:08 -07:00
Piotr Tabor
bc8d3f6639 Clientv3 (retry) logs should use the configured logger.
clientv3 logs (especially tests) were poluted with unattributed to testing.T log lines:

```
{"level":"warn","ts":"2021-04-29T12:42:11.055+0200","logger":"etcd-client","caller":"v3/retry_interceptor.go:64","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0000fafc0/#initially=[unix://localhost:m10]","attempt":0,"error":"rpc error: code = ResourceExhausted desc = etcdserver: mvcc: database space exceeded"}
```

The reasons were 2 fold:
  - Interceptors were copying logger before "WithLogger" could modify it.
  - We were not propagating the loggers in a few testing contexts.
2021-04-29 12:57:09 +02:00
Piotr Tabor
b47c5fcc12 Address review comments a.d. logging. 2021-04-15 17:54:37 +02:00
Piotr Tabor
d72f7ef5cc Give control to Embedded servers whether they override global loggers
So far each instance of embed server was overriding the grpc loggers and zap.global loggers.
It's counter intutitive that last created Embedded server was 'wining' and more-over it was breaking grpc expectation to change it "only" before the grpc stack is being used.

This PR introduces explicit call: `embed.Config::SetupGlobalLoggers()`, that changes the loggers where requested. The call is used by etcd main binary.

The immediate benefit from this change is reduction of  test flakiness, as there were flakes due to not a proper logger being used across tests.
2021-04-14 12:47:38 +02:00
Piotr Tabor
3bb7acc8cf Migrate dependencies pkg/foo -> client/pkg/foo 2021-04-07 00:38:47 +02:00
lzhfromustc
3de6b382df server&tests: Fix goroutine leaks 2021-03-23 12:52:59 -04:00
Piotr Tabor
54189f2f60 Enable --pre-vote=true by default in 3.5. 2021-03-12 16:23:23 +01:00
Gyuho Lee
94a371acd7
Merge pull request #12750 from ptabor/20210306-mlock
--experimental-memory-mlock support
2021-03-09 09:13:40 -08:00
Gyuho Lee
6fd85af641
Merge pull request #12702 from hexfusion/add-so
*: add support for socket options
2021-03-09 09:02:24 -08:00