Mirroristas/etcd
2
0
Fork 0
mirror of https://github.com/etcd-io/etcd.git synced 2024-09-27 06:25:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
17,246 Commits 15 Branches 491 Tags 148 MiB
Commit Graph

55 Commits

Author SHA1 Message Date
Marek Siarkowicz
5660bf0e7f server: Make corrtuption check optional and period configurable 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:56 +02:00
Marek Siarkowicz
4a75e3d52d server: Refactor compaction checker 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Marek Siarkowicz
35cbdf3961 server: Extract corruption detection to dedicated struct 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-07 15:11:55 +02:00
Benjamin Wang
9ea5b1ba22 Refactor the keepAliveListener and keepAliveConn 
Only `net.TCPConn` supports `SetKeepAlive` and `SetKeepAlivePeriod`
by default, so if you want to warp multiple layers of net.Listener,
the `keepaliveListener` should be the one which is closest to the
original `net.Listener` implementation, namely `TCPListener`.

Also refer to https://github.com/etcd-io/etcd/pull/14356

Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-08-20 15:03:15 +08:00
Benjamin Wang
8fdca41cd8 Change default sampling rate from 100% to 0% 
Refer to https://github.com/etcd-io/etcd/pull/14318

Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-08-07 07:19:30 +08:00
Benjamin Wang
5a86ae2c33 move setupTracing into a separate file config_tracing.go 
Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-08-07 07:17:27 +08:00
Mike Dame
4c013c91e9
Change default sampling rate from 100% to 0% 
This changes the default parent-based trace sampling rate from
100% to 0%. Due to the high QPS etcd can handle, having 100% trace
sampling leads to very high resource usage. Defaulting to 0% means
that only already-sampled traces will be sampled in etcd.

Fixes #14310

Signed-off-by: Mike Dame <mikedame@google.com>
 2022-08-05 15:00:40 +00:00
Benjamin Wang
437f3778d0 Add flag --max-concurrent-streams to set the max concurrent stream each client can open at a time 
Also refer to https://github.com/etcd-io/etcd/pull/14169#discussion_r917154243

Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-07-13 14:18:15 +08:00
Marek Siarkowicz
d00e89db2e server: Require either cluster version v3.6 or --experimental-enable-lease-checkpoint-persist to persist lease remainingTTL 
To avoid inconsistant behavior during cluster upgrade we are feature
gating persistance behind cluster version. This should ensure that
all cluster members are upgraded to v3.6 before changing behavior.

To allow backporting this fix to v3.5 we are also introducing flag
--experimental-enable-lease-checkpoint-persist that will allow for
smooth upgrade in v3.5 clusters with this feature enabled.
 2021-12-02 16:54:10 +01:00
Marek Siarkowicz
58d2b12a50 client: Add grpc authority header integration tests 2021-09-30 12:15:32 +02:00
tangcong
627d91c89d fix self-signed-cert-validity parameter cannot be specified in the config file 2021-07-30 07:53:43 +08:00
spacewander
53e04b2615 embed: unlimit the recv msg size of grpc-gateway 
Ensure the client which access etcd via grpc-gateway won't
be limited by the MaxCallRecvMsgSize. Here we choose the same
default value of etcdcli as grpc-gateway's MaxCallRecvMsgSize.

Fix https://github.com/etcd-io/etcd/issues/12576
 2021-06-11 07:53:49 +08:00
Mr-Mu
98bbc0107f TestLogRotation add log output relative path 
TestLogRotation add test log output relative path test
 2021-06-09 14:12:07 -04:00
Mr-Mu
fed3bc3084 [Fix]滚动日志路径解析不正确 
例如  rotate:test.log 路径解析不正确
 2021-06-09 14:12:07 -04:00
Piotr Tabor
f82b5cb776
Bring back original NewZapCoreLoggerBuilder api. (#12973) 
The funcion signature has been changed in:
eafbc8c57efc716644c328f63677ca7eadeebdfe .
Instead we should have added new method `NewZapLoggerBuilder()`.
 2021-05-16 10:26:10 -07:00
Piotr Tabor
85341e08f2
Merge pull request #12968 from serathius/logger-simplify 
server: Simplify passing logger setup by passing only logger
 2021-05-15 15:58:00 +02:00
Marek Siarkowicz
41ed74824e server: Simplify passing logger setup by passing only logger 2021-05-14 13:14:48 +02:00
Piotr Tabor
ead81df948 Disallow -v2-deprecation>'not-yet' combined with --enable-v2 2021-05-12 18:09:34 +02:00
Piotr Tabor
7c508741b3 Adding --v2-deprecation flag. 2021-05-12 18:09:34 +02:00
Piotr Tabor
e0a8484c8f
Merge pull request #12941 from serathius/defrag 
etcdserver: Implement running defrag if freeable space will exceed provided threshold (on boot)
 2021-05-12 09:26:56 +02:00
Marek Siarkowicz
efc8505739 etcdserver: Implement running defrag if freeable space will exceed privided threshold 2021-05-11 14:00:29 +02:00
Piotr Tabor
269f22c837 Deprecate V2 API: --enable-v2 and v2v3 
Flags `--experimental-enable-v2v3` and '-enable-v2' will raise a warning in 3.5,
in 3.6 they are schedule for decomissioning, such that v2store can stop be written in 3.7.

Deprecation plan in: https://github.com/etcd-io/etcd/issues/12913
 2021-05-10 16:19:52 +02:00
Lili Cosic
1a718a958e Add initial Tracing with OpenTelemetry 2021-05-10 10:44:40 +02:00
varvara dvortsova
c3303d94a4 embed: Added a check for an nil value of the request in ServeHTTP method. Fixed #12821 2021-05-09 20:45:43 +03:00
Sam Batschelet
6decbe15db
Merge pull request #12774 from hexfusion/add-log-rotate 
server: add support for log rotation
 2021-05-07 12:18:10 -04:00
Sam Batschelet
82b2d5c67d server: add support for log rotation 
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
 2021-05-07 08:39:51 -04:00
Gyuho Lee
344c9f3930
Merge pull request #12896 from wilsonwang371/profiling-txn2 
server: make applier use ReadTx() in Txn() instead of ConcurrentReadTx()
 2021-05-06 01:59:14 -07:00
Wilson Wang
98083ea914 server: add experimental flag for using shared buffer in transacton write 2021-05-04 11:59:08 -07:00
Piotr Tabor
f53b70facb Embed: In case KVStoreHash verification fails, close the backend. 
In case of failed verification, the server used to keep opened backend
(so the file was locked on OS level).
 2021-04-29 11:51:25 +02:00
Piotr Tabor
c4b13a5c83 Integrate verification framework 
Verification framework is integrated with:
  - integration tests (by default)
  - `ETCD_VERIFY=all etcdctl snapshot restore` command
  - etcd shutdown when running with `ETCD_VERIFY=all` env.
 2021-04-28 07:56:16 +02:00
Piotr Tabor
b47c5fcc12 Address review comments a.d. logging. 2021-04-15 17:54:37 +02:00
Piotr Tabor
d72f7ef5cc Give control to Embedded servers whether they override global loggers 
So far each instance of embed server was overriding the grpc loggers and zap.global loggers.
It's counter intutitive that last created Embedded server was 'wining' and more-over it was breaking grpc expectation to change it "only" before the grpc stack is being used.

This PR introduces explicit call: `embed.Config::SetupGlobalLoggers()`, that changes the loggers where requested. The call is used by etcd main binary.

The immediate benefit from this change is reduction of  test flakiness, as there were flakes due to not a proper logger being used across tests.
 2021-04-14 12:47:38 +02:00
Piotr Tabor
e776efbb2a
Merge pull request #12828 from ptabor/20210404-embed-etcd 
embed: etcd.Close() is closing Errc() channel as well.
 2021-04-08 01:20:07 +02:00
Piotr Tabor
5da9cac193 embed: etcd.Close() is closing Errc() channel as well. 
Inspired by https://github.com/etcd-io/etcd/pull/9612 by purpleidea@.
 2021-04-08 01:19:13 +02:00
Piotr Tabor
3bb7acc8cf Migrate dependencies pkg/foo -> client/pkg/foo 2021-04-07 00:38:47 +02:00
Piotr Tabor
8469108548
Merge pull request #12782 from ptabor/20210316-fixes 
Integration: Test flakiness fixes
 2021-03-19 20:10:11 +01:00
Piotr Tabor
725a8c5e02 Enable configuring delegated zap-logging for embed server. 2021-03-17 08:17:36 +01:00
Piotr Tabor
18382aa234 Fix 2 sources of leaked memory: embed server HTTP & v3_snapshot.leasser. 2021-03-16 22:20:00 +01:00
Piotr Tabor
54189f2f60 Enable --pre-vote=true by default in 3.5. 2021-03-12 16:23:23 +01:00
Piotr Tabor
fd7fed1511 Move config (ServerConfig) out of etcdserver package. 
Motivation:
  - ServerConfig is part of 'embed' public API, while etcdserver is more 'internal'
  - EtcdServer is already too big and config is pretty wide-spread leaf
if we were to split etcdserver (e.g. into pre & post-apply part).
 2021-03-11 20:56:22 +01:00
Gyuho Lee
94a371acd7
Merge pull request #12750 from ptabor/20210306-mlock 
--experimental-memory-mlock support
 2021-03-09 09:13:40 -08:00
Gyuho Lee
6fd85af641
Merge pull request #12702 from hexfusion/add-so 
*: add support for socket options
 2021-03-09 09:02:24 -08:00
Sam Batschelet
5b49fb41c8 fixup: add ListenerOptions 
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
 2021-03-08 11:27:03 -05:00
Piotr Tabor
a46a358577 --experimental-memory-mlock support 
The flag protects etcd memory from being swapped out to disk.
This can happen in memory constrained systems where mmaped bbolt
area is natural condidate for swapping out.

This flag should provide better tail latency on the cost of higher RSS
ram usage. If the experiment is successful, the logic should get moved
into bbolt layer, where we can protect specific bbolt instances
(e.g. avoid protecting both during defragmentation).
 2021-03-07 12:32:57 +01:00
Ben Meier
3d44f5bf80
*: added client-{client,key}-file parameters for supporting separate client and server certs when communicating between peers 
In some environments, the CA is not able to sign certificates with both
'client auth' and 'server auth' extended usage parameters and so an operator
needs to be able to set a seperate client certificate to use when making
requests which is different to the certificate used for accepting requests.
This applies to both proxy and etcd member mode and is available as both a CLI
 flag and config file field for peer TLS.

Signed-off-by: Ben Meier <ben.meier@oracle.com>
 2021-02-28 14:37:56 +00:00
Piotr Tabor
a7f340216d Reformat code according to 'gotip' rules. 
In practices adds annotations in the new syntax:
```
+//go:build !linux
 // +build !linux
```

Fixes failing gotip PASSES='fmt' check:
https://travis-ci.com/github/etcd-io/etcd/jobs/486453806
 2021-02-26 10:14:46 +01:00
Sam Batschelet
49078c683b *: add support for socket options 
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
 2021-02-19 13:31:23 -05:00
Brad Davidson
603d975599 Fix cluster peer HTTP SRV discovery 
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
 2021-02-03 03:08:13 -08:00
Piotr Tabor
958f6f9878
Merge pull request #12481 from kalexmills/fix-defer-log 
fix: pass argument url in defer to avoid loopclosure
 2021-01-31 23:20:32 +01:00
Piotr Tabor
8ccd4e1146 Fix flaky tests reported due to data race on grpc logging registration. 
Example:
```
    ==================
    WARNING: DATA RACE
    Write at 0x000002178320 by goroutine 575:
      google.golang.org/grpc/grpclog.SetLoggerV2()
          /home/ptab/private/golang/pkg/mod/google.golang.org/grpc@v1.29.1/grpclog/loggerv2.go:70 +0x444
      go.etcd.io/etcd/server/v3/embed.(*Config).setupLogging.func1.1()
          /home/ptab/corp/etcd/server/embed/config_logging.go:119 +0x345
      sync.(*Once).doSlow()
          /usr/lib/google-golang/src/sync/once.go:66 +0x109
      sync.(*Once).Do()
          /usr/lib/google-golang/src/sync/once.go:57 +0x68
      go.etcd.io/etcd/server/v3/embed.(*Config).setupLogging.func1()
          /home/ptab/corp/etcd/server/embed/config_logging.go:109 +0x3b1
      go.etcd.io/etcd/server/v3/embed.(*Config).setupLogging()
          /home/ptab/corp/etcd/server/embed/config_logging.go:174 +0x6af
      go.etcd.io/etcd/server/v3/embed.(*Config).Validate()
          /home/ptab/corp/etcd/server/embed/config.go:553 +0x55
      go.etcd.io/etcd/server/v3/embed.StartEtcd()
          /home/ptab/corp/etcd/server/embed/etcd.go:93 +0x84
      go.etcd.io/etcd/tests/v3/integration.TestKVWithEmptyValue()
          /home/ptab/corp/etcd/tests/integration/v3_kv_test.go:33 +0x18c
      testing.tRunner()
          /usr/lib/google-golang/src/testing/testing.go:1123 +0x202

    Previous read at 0x000002178320 by goroutine 956:
      [failed to restore the stack]

    Goroutine 575 (running) created at:
      testing.(*T).Run()
          /usr/lib/google-golang/src/testing/testing.go:1168 +0x5bb
      testing.runTests.func1()
          /usr/lib/google-golang/src/testing/testing.go:1441 +0xa6
      testing.tRunner()
          /usr/lib/google-golang/src/testing/testing.go:1123 +0x202
      testing.runTests()
          /usr/lib/google-golang/src/testing/testing.go:1439 +0x612
      testing.(*M).Run()
          /usr/lib/google-golang/src/testing/testing.go:1347 +0x3c4
      go.etcd.io/etcd/pkg/v3/testutil.MustTestMainWithLeakDetection()
          /home/ptab/corp/etcd/pkg/testutil/leak.go:150 +0x38
      go.etcd.io/etcd/tests/v3/integration.TestMain()
          /home/ptab/corp/etcd/tests/integration/main_test.go:14 +0x272
      main.main()
          _testmain.go:349 +0x269

    Goroutine 956 (finished) created at:
      google.golang.org/grpc/internal/transport.newHTTP2Server()
          /home/ptab/private/golang/pkg/mod/google.golang.org/grpc@v1.29.1/internal/transport/http2_server.go:288 +0x18a4
      google.golang.org/grpc/internal/transport.NewServerTransport()
          /home/ptab/private/golang/pkg/mod/google.golang.org/grpc@v1.29.1/internal/transport/transport.go:534 +0x2f5
      google.golang.org/grpc.(*Server).newHTTP2Transport()
          /home/ptab/private/golang/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:726 +0x2ca
      google.golang.org/grpc.(*Server).handleRawConn()
          /home/ptab/private/golang/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:693 +0x60f
      google.golang.org/grpc.(*Server).Serve.func3()
          /home/ptab/private/golang/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:663 +0x4c
    ==================
    ...
    {"level":"info","ts":"2021-01-09T22:21:04.550+0100","caller":"embed/etcd.go:330","msg":"closed etcd server","name":"default","data-dir":"/tmp/etcd-017337431","advertise-peer-urls":["http://localhost:2380"],"advertise-client-urls":["http://localhost:2379"]}
    --- FAIL: TestKVWithEmptyValue (1.08s)
        v3_kv_test.go:62: my-namespace/foobar = data
        v3_kv_test.go:62: my-namespace/foobar1 = data
        v3_kv_test.go:62: namespace/foobar1 = data
        v3_kv_test.go:72: foobar = data
        v3_kv_test.go:72: foobar1 = data
        v3_kv_test.go:87: delete keys:2
        testing.go:1038: race detected during execution of test
```
 2021-01-11 10:06:31 +01:00