48 Commits

Author SHA1 Message Date
Piotr Tabor
a1fe0b8ea3
Merge pull request #14116 from ptabor/20220613-embed-errors
Embed server should log errors (and not get stuck)
2022-06-15 22:37:40 +02:00
Piotr Tabor
fcc8fce4d2 Expand logging in case of embed server not being able to successfully start.
So far the errors were directed to Etcd.Errc (channel) that is not being consumed in practice.

Signed-off-by: Piotr Tabor <ptab@google.com>
2022-06-15 13:50:17 +02:00
Marek Siarkowicz
7c35dadc25 server: Extract corruption detection to dedicated struct
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:24 +02:00
Piotr Tabor
651de5a057 Rename EtcdServer.Id with EtcdServer.MemberId.
It was misleading and error prone vs. ClusterId.
2022-05-20 14:32:04 +02:00
Marek Siarkowicz
26f42e7a9e server: Apply review comments and split basic handler 2022-05-05 09:52:14 +02:00
Marek Siarkowicz
722ec487df server: Split metrics and health code 2022-05-05 09:52:14 +02:00
Marek Siarkowicz
600ee13ac0 server: Cover V3 health with tests 2022-05-05 09:52:14 +02:00
ahrtr
3dcbbf62d9 Move clientconfig into clientv3 so that it can be reused by both etcdctl and v3 discovery 2022-03-12 06:38:41 +08:00
Piotr Tabor
088807c08e
Merge pull request #13565 from ahrtr/remove_peer_serve_client_requests
Updated servePeers to remove the grpc server
2022-03-01 16:24:42 +01:00
ahrtr
2f36e0c62b Change discovery url to endpoints
Currently the discovery url is just one endpoint. But actually it
should be the same as the etcdctl, which means that it should be
a list of endpoints. When one endpoint is down, the clientv3 can
fail over to the next endpoint automatically.
2022-02-24 09:11:41 +08:00
ahrtr
ebc86d12c0 support v3 discovery to bootstrap a new etcd cluster 2022-02-21 23:22:49 +08:00
ahrtr
a879ccf152 updated servePeers to remvoe the grpc server 2022-01-27 16:22:01 +08:00
ahrtr
1713dc67b5 etcd server shouldn't wait for the ready notification infinitely on startup 2022-01-27 16:19:20 +08:00
Marek Siarkowicz
ee5ef42c5c server: --enable-v2 and --enable-v2v3 is decomissioned 2022-01-14 13:19:30 +01:00
Marek Siarkowicz
7d10899d7f server: Require either cluster version v3.6 or --experimental-enable-lease-checkpoint-persist to persist lease remainingTTL
To avoid inconsistant behavior during cluster upgrade we are feature
gating persistance behind cluster version. This should ensure that
all cluster members are upgraded to v3.6 before changing behavior.

To allow backporting this fix to v3.5 we are also introducing flag
--experimental-enable-lease-checkpoint-persist that will allow for
smooth upgrade in v3.5 clusters with this feature enabled.
2021-12-02 12:26:47 +01:00
Sam Batschelet
63a1cc3fe4 add --experimental-max-learner flag
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-11-09 09:52:00 -05:00
Eng Zer Jun
2a151c8982
*: move from io/ioutil to io and os packages
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-10-28 00:05:28 +08:00
Marek Siarkowicz
90932324b1 client: Add grpc authority header integration tests 2021-09-29 12:42:16 +02:00
Sam Batschelet
a4a82cc982
Merge pull request #13248 from lilic/add-sampling-rate
server: Add sampling rate to distributed tracing
2021-08-30 08:31:00 -04:00
Lili Cosic
810f489017 server: Add sampling rate to distributed tracing
ExperimentalDistributedTracingSamplingRatePerMillion is the
number of samples to collect per million spans.
Defaults to 0.
2021-08-30 13:55:35 +02:00
Marek Siarkowicz
83a325ac46 server: Move all functions needed for storage bootstrap to storage package
This is prerequestite to move storage bootstrap, splitted to separate PR
to make it easier to review.
2021-08-03 13:09:15 +02:00
Alexey Roytman
2a26f7ae4c
etcdserver: configure "expensive" requests duration
When a unary request takes more than predefined duration, this request
is defined as "expensive" and a warning is printed. The expensive request
duration is hard-coded to 300 ms. It can be not enough for example
for transactions with a lot of operations. The warnings just blow up
the log files and reduce throughput.

This fix allows user to configure the "expensive" request duration.

Signed-off-by: Alexey Roytman <roytman@il.ibm.com>
2021-07-27 08:33:44 +03:00
AlexStocks
184b0e5d49 add sleep interval 2021-05-24 16:22:00 +08:00
Piotr Tabor
85341e08f2
Merge pull request #12968 from serathius/logger-simplify
server: Simplify passing logger setup by passing only logger
2021-05-15 15:58:00 +02:00
Marek Siarkowicz
41ed74824e server: Simplify passing logger setup by passing only logger 2021-05-14 13:14:48 +02:00
Piotr Tabor
ead81df948 Disallow -v2-deprecation>'not-yet' combined with --enable-v2 2021-05-12 18:09:34 +02:00
Piotr Tabor
e0a8484c8f
Merge pull request #12941 from serathius/defrag
etcdserver: Implement running defrag if freeable space will exceed provided threshold (on boot)
2021-05-12 09:26:56 +02:00
Marek Siarkowicz
efc8505739 etcdserver: Implement running defrag if freeable space will exceed privided threshold 2021-05-11 14:00:29 +02:00
Piotr Tabor
269f22c837 Deprecate V2 API: --enable-v2 and v2v3
Flags `--experimental-enable-v2v3` and '-enable-v2' will raise a warning in 3.5,
in 3.6 they are schedule for decomissioning, such that v2store can stop be written in 3.7.

Deprecation plan in: https://github.com/etcd-io/etcd/issues/12913
2021-05-10 16:19:52 +02:00
Lili Cosic
1a718a958e Add initial Tracing with OpenTelemetry 2021-05-10 10:44:40 +02:00
Gyuho Lee
344c9f3930
Merge pull request #12896 from wilsonwang371/profiling-txn2
server: make applier use ReadTx() in Txn() instead of ConcurrentReadTx()
2021-05-06 01:59:14 -07:00
Wilson Wang
98083ea914 server: add experimental flag for using shared buffer in transacton write 2021-05-04 11:59:08 -07:00
Piotr Tabor
f53b70facb Embed: In case KVStoreHash verification fails, close the backend.
In case of failed verification, the server used to keep opened backend
(so the file was locked on OS level).
2021-04-29 11:51:25 +02:00
Piotr Tabor
c4b13a5c83 Integrate verification framework
Verification framework is integrated with:
  - integration tests (by default)
  - `ETCD_VERIFY=all etcdctl snapshot restore` command
  - etcd shutdown when running with `ETCD_VERIFY=all` env.
2021-04-28 07:56:16 +02:00
Piotr Tabor
b47c5fcc12 Address review comments a.d. logging. 2021-04-15 17:54:37 +02:00
Piotr Tabor
e776efbb2a
Merge pull request #12828 from ptabor/20210404-embed-etcd
embed: etcd.Close() is closing Errc() channel as well.
2021-04-08 01:20:07 +02:00
Piotr Tabor
5da9cac193 embed: etcd.Close() is closing Errc() channel as well.
Inspired by https://github.com/etcd-io/etcd/pull/9612 by purpleidea@.
2021-04-08 01:19:13 +02:00
Piotr Tabor
3bb7acc8cf Migrate dependencies pkg/foo -> client/pkg/foo 2021-04-07 00:38:47 +02:00
Piotr Tabor
18382aa234 Fix 2 sources of leaked memory: embed server HTTP & v3_snapshot.leasser. 2021-03-16 22:20:00 +01:00
Piotr Tabor
fd7fed1511 Move config (ServerConfig) out of etcdserver package.
Motivation:
  - ServerConfig is part of 'embed' public API, while etcdserver is more 'internal'
  - EtcdServer is already too big and config is pretty wide-spread leaf
if we were to split etcdserver (e.g. into pre & post-apply part).
2021-03-11 20:56:22 +01:00
Gyuho Lee
94a371acd7
Merge pull request #12750 from ptabor/20210306-mlock
--experimental-memory-mlock support
2021-03-09 09:13:40 -08:00
Sam Batschelet
5b49fb41c8 fixup: add ListenerOptions
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-03-08 11:27:03 -05:00
Piotr Tabor
a46a358577 --experimental-memory-mlock support
The flag protects etcd memory from being swapped out to disk.
This can happen in memory constrained systems where mmaped bbolt
area is natural condidate for swapping out.

This flag should provide better tail latency on the cost of higher RSS
ram usage. If the experiment is successful, the logic should get moved
into bbolt layer, where we can protect specific bbolt instances
(e.g. avoid protecting both during defragmentation).
2021-03-07 12:32:57 +01:00
Sam Batschelet
49078c683b *: add support for socket options
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-02-19 13:31:23 -05:00
K. Alex Mills
3f6e0ec94b fix: pass argument url in defer to avoid loopclosure
Because of the well-known range loop closure issue, the value of u may
have changed by the time the anonymous function mentioned in the defer
is run. To address this, the simplest fix is to pass the url used in the
loop as an argument to the function run in defer.
2020-11-19 15:29:26 -06:00
Ankur Gargi
c1c681adc3 server: Added config parameter experimental-warning-apply-duration 2020-11-17 17:33:19 -05:00
Piotr Tabor
aaf423e962 server: Update imports.
find -name '*.go' | xargs sed -i --follow-symlinks 's|etcd/v3/|etcd/server/v3/|g'
2020-10-26 13:02:32 +01:00
Piotr Tabor
4a5e9d1261 server: Move server files to 'server' directory.
26  git mv mvcc wal auth etcdserver etcdmain proxy embed/ lease/ server
   36  git mv go.mod go.sum server
2020-10-26 12:57:19 +01:00