18844 Commits

Author SHA1 Message Date
Marek Siarkowicz
2d638e0fd2
Merge pull request #14559 from serathius/etcdctl_api
Remove mention of ETCDCTL_API environment variable as it was removed from etcdctl
2022-10-08 14:37:25 +02:00
Hisanobu Tomari
428fb9623a client/v3: Add backoff before retry when watch stream returns unavailable
The client retries connection without backoff when the server is gone
after the watch stream is established. This results in high CPU usage
in the client process. This change introduces backoff when the stream is
failed and unavailable.

Signed-off-by: Hisanobu Tomari <posco.grubb@gmail.com>
2022-10-08 06:57:30 +09:00
Marek Siarkowicz
510f26e34c Remove mention of ETCDCTL_API environment variable as it was removed on main branch
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-10-07 21:22:06 +02:00
Nathan VanBenschoten
419ee8a9c6 raft: panic on self-addressed messages
These are nonsensical and a network implementation is not required
to handle them correctly, so panic instead of sending them out.

Signed-off-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
2022-10-06 20:25:07 -04:00
Marek Siarkowicz
6a0bbf3462
Merge pull request #14552 from fuweid/deflaking-14477
integration: deflaking TestMaintenanceSnapshotCancel/Timeout
2022-10-06 09:52:14 +02:00
Wei Fu
b15f6bf978 integration: deflaking TestMaintenanceSnapshotCancel/Timeout
Since http2 spec defines the receive windows's size and max size of
frame in the stream, the underlayer - gRPC client can pre-read data
from server even if the application layer hasn't read it yet.

And the initialized cluster has 20KiB snapshot, which can be pre-read
by underlayer. We should increase the snapshot's size, just in case
that the io.Copy won't return the canceled or timeout error.

Fixes: #14477

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2022-10-06 11:03:51 +08:00
Benjamin Wang
b81e20b7ad
Merge pull request #14551 from ahrtr/update_3.6_changelog_goversion
Update changelog 3.6 to clearly requiring applications which depend on etcd 3.6 packages must be built with go version >= 1.18
2022-10-06 07:02:05 +08:00
Benjamin Wang
ad4e256175 Update changelog 3.6 to clearly requiring applications which depend on etcd 3.6 packages must be built with go version >= 1.18
etcdserver starts to using generic in https://github.com/etcd-io/etcd/pull/14515,
so any applications which depend on package `go.etcd.io/etcd/server/v3' must be
built with go version >= 1.18.

Most likely we may depends on go 1.18 new features in other packages, i.e. client, pkg,
so we just simplify the changelog message to require applications depending on any
etcd 3.6 pacakges must be built with go version >= 1.18, although only etcdserver uses
go generic for now.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-10-05 22:56:40 +08:00
Benjamin Wang
f3e2d7b0bb
Merge pull request #14549 from mitake/changelog-backport-14322
update changelogs of 3.4 and 3.5 for handling auth invalid token and old revision errors in watch
2022-10-05 08:06:56 +08:00
Hitoshi Mitake
b13b19871f update changelogs of 3.4 and 3.5 for handling auth invalid token and old revision errors in watch
Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com>
2022-10-05 08:12:54 +09:00
Nathan VanBenschoten
c50e728518 raft: simplify auto-leave joint config on entry application logic
This commit simplifies the logic added in 37c7e4d to auto-leave joint
configurations. It does so by making the following adjustments to the
code:

- remove the `oldApplied <= r.pendingConfIndex` condition. This does
  not seem necessary. When a node first attempts to auto-leave a joint
  config, it will bump `r.pendingConfIndex` when proposing. In cases
  where `oldApplied >= r.pendingConfIndex`, the proposal must have
  already been applied. Reviewers should double check this.
- use raft.Step instead of custom proposal code. This code was already
  present in stepLeader, so there was no reason to duplicate it. This
  would have avoided bugs like the one we fixed in #14538.
- use `confChangeToMsg` to generate message, to centralize the creation
  of all `MsgProp{EntryConfChange}` messages.

Signed-off-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
2022-10-03 02:11:56 -04:00
Benjamin Wang
a932fb58f2
Merge pull request #14539 from nvanbenschoten/nvanbenschoten/advanceHardState
raft: update prevHardSt on Ready accept, not advance
2022-09-30 16:55:45 +08:00
Benjamin Wang
c102c07081
Merge pull request #14538 from nvanbenschoten/nvanbenschoten/broadcastOnLeave
raft: broadcast MsgApp on auto-leave joint config proposal
2022-09-30 15:43:26 +08:00
Benjamin Wang
7fff4c4241
Merge pull request #14331 from chaochn47/auth_test_framework_update
common tests framework: cluster client creation could fail with invalid auth
2022-09-30 15:38:47 +08:00
Benjamin Wang
62d979be5b
Merge pull request #14536 from ahrtr/update_hash_rev
etcdserver: get peer's hash using the same revision as the value used by leader
2022-09-30 15:09:05 +08:00
Chao Chen
8d057ea2b8 common tests framework: cluster client creation fail with invalid auth
Signed-off-by: Chao Chen <chaochn@amazon.com>
2022-09-29 23:09:32 -07:00
Benjamin Wang
1ccdb3762d Test: fix all corruption detection related unit test cases
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-30 06:06:41 +08:00
Benjamin Wang
d116d02e04 etcdserver: update corrupt hash detection's logic
get peer's hash using the same revision as the value used by leader

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-30 06:03:50 +08:00
Nathan VanBenschoten
bd34388721 raft: broadcast MsgApp on auto-leave joint config proposal
This commit ensures that the raft leader eagerly broadcasts a MsgApp to
each follower when initiating an automatic transition out of a joint
configuration. This had been missed previously, which could lead to
delayed completion of an auto-transition.

Signed-off-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
2022-09-29 12:33:20 -04:00
Nathan VanBenschoten
9a03632043 raft: update prevHardSt on Ready accept, not advance
This commit updates the `RawNode`'s `prevHardSt` to the new HardState in
`acceptReady` instead of on `Advance`. This aligns the handling of
`prevHardSt` with that of `prevSoftSt` (and other fields like `msgs`)
and simplifies the logic in `Advance`.

Signed-off-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
2022-09-29 12:33:04 -04:00
Benjamin Wang
cd9764a99f
Merge pull request #14534 from demoManito/feat/golangci
golangci: add import order check
2022-09-29 15:26:00 +08:00
demoManito
7020e2fd2b fix lint
Signed-off-by: demoManito <1430482733@qq.com>
2022-09-29 13:57:55 +08:00
demoManito
ae118a17f0 fix goimorts local-prefixes
Signed-off-by: demoManito <1430482733@qq.com>
2022-09-29 13:55:47 +08:00
demoManito
0a08fa2b43 golangci: add import order check
Signed-off-by: demoManito <1430482733@qq.com>
2022-09-29 11:30:04 +08:00
Marek Siarkowicz
d379e6221e
Merge pull request #14535 from demoManito/feat/golangci-unconvert
golangci: add unconvert check
2022-09-28 14:04:04 +02:00
demoManito
93b9dc6583 golangci: add unconvert check
Signed-off-by: demoManito <1430482733@qq.com>
2022-09-28 19:13:58 +08:00
Marek Siarkowicz
6828e53b0a
Merge pull request #14531 from ahrtr/changelog_3.4_memberId_zero
Update changelog-3.4 to cover the issue: memberID equals zero in corruption alarm
2022-09-28 09:30:45 +02:00
Benjamin Wang
ea17d9d6bf update changelog-3.4 to cover the issue: memberID equals zero in corruption alarm
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-28 11:05:53 +08:00
Benjamin Wang
abef537a90
Merge pull request #14515 from spongecaptain/btree-generics
upate:use google/btree in the genric way
2022-09-27 16:44:13 +08:00
wathenjiang
319db38b0a update: add benchmark test
benchmark result:
(1) master branch
$ go test -bench='BenchmarkIndexPut$' -count=5
goos: darwin
goarch: amd64
pkg: go.etcd.io/etcd/server/v3/storage/mvcc
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkIndexPut-12             1000000              2591 ns/op
BenchmarkIndexPut-12             1000000              2531 ns/op
BenchmarkIndexPut-12             1000000              2536 ns/op
BenchmarkIndexPut-12             1000000              2546 ns/op
BenchmarkIndexPut-12             1000000              2538 ns/op
PASS
ok      go.etcd.io/etcd/server/v3/storage/mvcc  167.439s

$ go test -bench='BenchmarkIndexGet$' -count=5
goos: darwin
goarch: amd64
pkg: go.etcd.io/etcd/server/v3/storage/mvcc
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkIndexGet-12             1000000              2021 ns/op
BenchmarkIndexGet-12             1000000              2029 ns/op
BenchmarkIndexGet-12             1000000              2044 ns/op
BenchmarkIndexGet-12             1000000              1973 ns/op
BenchmarkIndexGet-12             1000000              2027 ns/op
PASS
ok      go.etcd.io/etcd/server/v3/storage/mvcc  177.815s

(2) google/btree in the generic way
$ go test -bench='BenchmarkIndexPut$' -count=5
goos: darwin
goarch: amd64
pkg: go.etcd.io/etcd/server/v3/storage/mvcc
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkIndexPut-12             1000000              2477 ns/op
BenchmarkIndexPut-12             1000000              2380 ns/op
BenchmarkIndexPut-12             1000000              2360 ns/op
BenchmarkIndexPut-12             1000000              2396 ns/op
BenchmarkIndexPut-12             1000000              2382 ns/op
PASS
ok      go.etcd.io/etcd/server/v3/storage/mvcc  165.841s

$ go test -bench='BenchmarkIndexGet$' -count=5
goos: darwin
goarch: amd64
pkg: go.etcd.io/etcd/server/v3/storage/mvcc
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkIndexGet-12             1000000              1985 ns/op
BenchmarkIndexGet-12             1000000              1914 ns/op
BenchmarkIndexGet-12             1000000              1900 ns/op
BenchmarkIndexGet-12             1000000              1905 ns/op
BenchmarkIndexGet-12             1000000              1894 ns/op
PASS
ok      go.etcd.io/etcd/server/v3/storage/mvcc  177.573s

Signed-off-by: wathenjiang <wathenjiang@tencent.com>
2022-09-27 14:33:02 +08:00
Spongecaptain
c53dfc7c5b upate:use google/btree in the genric way
Signed-off-by: wathenjiang <wathenjiang@tencent.com>
2022-09-27 10:16:15 +08:00
Benjamin Wang
1c20ed2cc5
Merge pull request #14521 from lovehhf/remove_pick_peer_url
membership: Remove PickPeerURL Method
2022-09-27 02:10:35 +08:00
Benjamin Wang
9cb5764859
Merge pull request #14296 from kafuu-chino/fix
Fixed unexpected cancelled watch with WatchID=0.
2022-09-27 02:08:23 +08:00
Hongfei Huang
f6d808736c membership: Remove PickPeerURL Method
PickPeerURL only used by unit test

Signed-off-by: Hongfei Huang <853885165@qq.com>
2022-09-26 23:21:10 +08:00
Kafuu Chino
f1d4935e91 *: avoid closing a watch with ID 0 incorrectly
Signed-off-by: Kafuu Chino <KafuuChinoQ@gmail.com>

add test
2022-09-26 20:30:33 +08:00
Benjamin Wang
434c7c4309
Merge pull request #14506 from SimFG/main
wal: Fix the `walWriteBytes` metric
2022-09-26 09:36:07 +08:00
Jeremy Leach
cc1e245368 etcdserver: fix corruption check when server has just been compacted
When a key-value store corruption check happens immediately after a
compaction, the revision at which the key-value store hash is computed,
is the compacted revision itself.
In that case, the hash computation logic was incorrect because it
returned an ErrCompacted error; this error should instead be returned when
the revision at which the key-value store is hashed, is strictly lower
than the compacted revision.

Fixes #14325

Signed-off-by: Jeremy Leach <44558776+jbml@users.noreply.github.com>
2022-09-24 22:20:26 +10:00
Marek Siarkowicz
4513671b69
Merge pull request #14512 from joycebrum/main
Enable the OpenSSF Scorecard Github Action
2022-09-23 19:02:17 +02:00
Joyce Brum
c6dd01f337 add scorecard badge to README file
Signed-off-by: Joyce Brum <joycebrumu.u@gmail.com>
2022-09-23 13:59:17 -03:00
Joyce Brum
336bae2d4e github: enable the scorecard github action
Signed-off-by: Joyce Brum <joycebrumu.u@gmail.com>
2022-09-23 13:57:22 -03:00
SimFG
5702765729 wal: Fix the walWriteBytes metric
Signed-off-by: SimFG <1142838399@qq.com>
2022-09-22 19:23:06 +08:00
Benjamin Wang
54f9483e72
Merge pull request #14504 from ahrtr/update_changelog_go1.19
Update changelog-3.6 to reflect the bumping of go 1.19
2022-09-22 17:06:46 +08:00
Benjamin Wang
ea3d78faae
Merge pull request #14505 from ahrtr/revert_14400
etcdserve: revert the etcdserver side change for the data loss in one node cluster
2022-09-22 17:06:10 +08:00
Benjamin Wang
9097e61b40 etcdserve: revert the etcdserver side change for the data loss on one node cluster
Since the raft side change has been merged, so we need to revert the etcdserver
side change.
Refer to
  https://github.com/etcd-io/etcd/pull/14413
  https://github.com/etcd-io/etcd/pull/14400

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-22 15:19:20 +08:00
Benjamin Wang
da7e640473 Update changelog-3.6 to reflect the bumping of go 1.19
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-22 15:07:52 +08:00
Benjamin Wang
997260a832
Merge pull request #14463 from ahrtr/bump_go_1.19
Bump golang version to 1.19.1
2022-09-22 09:25:58 +08:00
Benjamin Wang
dd7d30017c Bump go 1.19: revert the change to pkg/adt/interval_tree.go
Some comments in the file are formatted automatically into ugly style,
because the hierarchical structure is missing. After removing the
leading numbers in the comments, `go fmt` will not format the comments
anymore.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-22 08:47:46 +08:00
Benjamin Wang
7f10dccbaf Bump go 1.19: update all the dependencies and go.sum files
1. run ./scripts/fix.sh;
2. cd tools/mod; gofmt -w . & go mod tidy;

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-22 08:47:46 +08:00
Benjamin Wang
cb5f7276c3 Bump go 1.19: upgrade go version to 1.19.1 in the pipeline
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-22 08:47:46 +08:00
Benjamin Wang
cd0b1d0c66 Bump go 1.19: upgrade go version to 1.19 in all go.mod files
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-22 08:47:46 +08:00