18891 Commits

Author SHA1 Message Date
Benjamin Wang
5503a9484a test: cleanup go.mod and go.sum files
Executed commands below,
1. Removed go.etcd.io/raft/v3 => ../raft;
2. go get go.etcd.io/raft/v3@eaa6808e1f7ab2247c13778250f70520b0527ff1;
3. go mod tidy

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 13:36:05 +08:00
Benjamin Wang
7ce21519fb test: update test to use the new raft module go.etcd.io/raft/v3
Just replaced all go.etcd.io/etcd/raft/v3 with go.etcd.io/raft/v3
under directory test.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 13:32:47 +08:00
Benjamin Wang
ff337cc993 tools/etcd-dump-logs: generate go.mod and go.sum for the tool
Add go.mod and go.sum to fix issue below,
```
$ go build
../../server/etcdserver/api/snap/snapshotter.go:33:2: missing go.sum entry for module providing package go.etcd.io/raft/v3 (imported by go.etcd.io/etcd/server/v3/etcdserver/api/snap); to add:
	go get go.etcd.io/etcd/server/v3/etcdserver/api/snap@v3.6.0-alpha.0
../../server/storage/wal/walpb/record.pb.go:14:2: missing go.sum entry for module providing package go.etcd.io/raft/v3/raftpb (imported by go.etcd.io/etcd/v3/tools/etcd-dump-logs); to add:
	go get go.etcd.io/etcd/v3/tools/etcd-dump-logs
```

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 13:27:38 +08:00
Benjamin Wang
bb6e892763 tools/etcd-dump-logs: update tool to use the new raft module go.etcd.io/raft/v3
Just replaced all go.etcd.io/etcd/raft/v3 with go.etcd.io/raft/v3
under tools/etcd-dump-logs.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 13:12:19 +08:00
Benjamin Wang
f54752b977 etcdutl: cleanup go.mod and go.sum files
Executed commands below,
1. Removed go.etcd.io/raft/v3 => ../raft;
2. go get go.etcd.io/raft/v3@eaa6808e1f7ab2247c13778250f70520b0527ff1
3. go mod tidy

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 13:06:59 +08:00
Benjamin Wang
f5b5d12dd3 etcdutl: updated etcdutl to use the new raft module go.etcd.io/raft/v3
Just replaced all go.etcd.io/etcd/raft/v3 with go.etcd.io/raft/v3
under directory etcdutl.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 13:02:31 +08:00
Benjamin Wang
faff80a2b3 etcdserve: format the source code
gofmt -w ./server

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 13:00:59 +08:00
Benjamin Wang
87e1ca2b9a etcdserver: cleanup go.mod and go.sum files
Executed commands below,
1. Removed go.etcd.io/raft/v3 => ../raft;
2. go get go.etcd.io/raft/v3@eaa6808e1f7ab2247c13778250f70520b0527ff1
3. go mod tidy

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 12:59:23 +08:00
Benjamin Wang
e9aa275b36 etcdserver: update etcdserver to use the new raft module go.etcd.io/raft/v3
Just replaced all go.etcd.io/etcd/raft/v3 with go.etcd.io/raft/v3
under directory server.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 09:33:45 +08:00
Marek Siarkowicz
571c65748e
Merge pull request #14868 from Bhargav-InfraCloud/append-build-flags
Read Go Build Flags from env `GO_BUILD_FLAGS` (#14396)
2022-12-01 09:05:46 +01:00
Bhargav Ravuri
dadd7358f4 Makefile: Read GO_BUILD_FLAGS from env and prepend
Build flags read from env GO_BUILD_FLAGS are prepended to
flags in Makefile and sent to scripts/build.sh for building
executables (etcd, etcdctl & etcdutl).

Signed-off-by: Bhargav Ravuri <bhargav.ravuri@infracloud.io>
2022-12-01 09:43:26 +05:30
Benjamin Wang
898415dc90
Merge pull request #14871 from ahrtr/bump_x_20221130
Bump golang.org/x imports to address a bunch of CVEs
2022-11-30 18:17:58 +08:00
Benjamin Wang
3f86db5e53 bump golang.org/x imports to address CVEs
Reference: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-32149

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-30 05:03:43 +08:00
Marek Siarkowicz
61e9ec9833
Merge pull request #14865 from ahrtr/fix_unit_test_20221129
test: fix unit test Instability
2022-11-29 10:56:49 +01:00
Benjamin Wang
fae9435b66 test: fix unit test Instability
When two members in a 5 member cluster are corrupted, and they
have different hashes, etcd will raise alarm for both members,
but the order isn't guaranteed. But if the two corrupted members
have the same hash, then the order is guaranteed. The leader
always raise alarm in the same order as the member list.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-29 06:51:50 +08:00
Benjamin Wang
319651b784
Merge pull request #14862 from ahrtr/add_log_open_wal_failure_20221128
add more debug info for opening WAL files failure
2022-11-29 06:10:16 +08:00
Benjamin Wang
cf171fdd1d
Merge pull request #14828 from ahrtr/identify_corrupted_member_20221123
Identify corrupted member depending on quorum
2022-11-29 06:08:25 +08:00
Benjamin Wang
686846ec65 add more debug info for opening WAL files failure
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-28 18:58:01 +08:00
Marek Siarkowicz
6a156bd555
Merge pull request #14859 from serathius/gofail-cleanup
tests: Cleanup gofail
2022-11-27 21:12:43 +01:00
Marek Siarkowicz
1503f46fd5
Merge pull request #14861 from fuweid/deflake-transport-timeout-case
Deflake transport timeout case
2022-11-27 21:06:28 +01:00
Marek Siarkowicz
dd4d69ca91 tests: Cleanup gofail
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-11-27 20:35:39 +01:00
Wei Fu
cd9ade5403 client/pkg/transport: fix typo in _test.go
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2022-11-27 22:10:06 +08:00
Wei Fu
8a88660262 client/pkg/transport: deflake TestWriteReadTimeoutListener
There is data race on `stop` channel. After verify write-timeout successfully,
the case won't wait for `blocker` to receive close signal from `stop` channel.
If the new `blocker`, which is to read-timeout verifier, get dial's result
immediately, the new `blocker` might fetch the message from `stop` channel
before old one and then close the connection, which causes that the
`conn.Read` returns `EOF` when it reads data.

How to reproduce this in linux devbox?

Use `taskset` to limit the test process in one-cpu.

```bash
cd ./client/pkg/transport
go test -c -o /tmp/test --race=true ./
taskset -c 0 /tmp/test -test.run TestWriteReadTimeoutListener -test.v -test.cpu 4 -test.count=10000 -test.failfast
```

To fix this, suggest to use seperate `stop` channel to prevent from data
race.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2022-11-27 22:06:11 +08:00
Benjamin Wang
d545d603e9 test: update both unit test and e2e/integration test for CompactHashCheck
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-26 20:13:20 +08:00
Benjamin Wang
6049af072c etcdserver: intentionally set memberID as 0 when can't identify the corrupted member
If quorum doesn't exist, we don't know which members data are
corrupted. In such situation, we intentionally set the memberID
as 0, it means it affects the whole cluster.
It's align with what we did for 3.4 and 3.5 in
https://github.com/etcd-io/etcd/issues/14849

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-26 19:35:38 +08:00
Benjamin Wang
e95e82f0b9 etcdserver: added a summary for the CompactHashCheck method
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-26 19:35:38 +08:00
Benjamin Wang
85fc09d09b etcdserver: resolve review comments in PR 14828
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-26 19:35:38 +08:00
Benjamin Wang
7b19ee6396 test: add integration test to cover the multiple member corruption case
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-26 19:35:38 +08:00
Benjamin Wang
a3197102e9 test: rollback the change in PR pull/14824
The change did in https://github.com/etcd-io/etcd/pull/14824 fixed
the test instead of the product code. It isn't correct. After we
fixed the product code in this PR, we can revert the change in
that PR.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-26 19:35:38 +08:00
Benjamin Wang
8b98fee9ce etcdserver: detect corrupted member based on quorum
When the leader detects data inconsistency by comparing hashes,
currently it assumes that the follower is the corrupted member.
It isn't correct, the leader might be the corrupted member as well.

We should depend on quorum to identify the corrupted member.
For example, for 3 member cluster, if 2 members have the same hash,
the the member with different hash is the corrupted one. For 5 member
cluster, if 3 members have the same same, the corrupted member is one
of the left two members; it's also possible that both the left members
are corrupted.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-26 19:35:38 +08:00
Marek Siarkowicz
cdb9b8b2a0
Merge pull request #14858 from ahrtr/fix_release_failure_20221126
fix release pipeline failure
2022-11-26 12:32:14 +01:00
Benjamin Wang
72b9d1d31b fix release pipeline failure
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-26 18:40:54 +08:00
Marek Siarkowicz
08cb83ab87
Merge pull request #14856 from ahrtr/bump_testify_20221126
bump github.com/stretchr/testify from v1.7.2 to v1.8.1
2022-11-26 11:11:32 +01:00
Benjamin Wang
285e44378f bump github.com/stretchr/testify from v1.7.2 to v1.8.1
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-26 10:51:32 +08:00
Benjamin Wang
488e5413d7
Merge pull request #14812 from ahrtr/add_hash_revision_etcdctl_20221121
Display hash_revision for `etcdctl endpoint hash`
2022-11-25 19:36:27 +08:00
Benjamin Wang
9a7f9609d6 etcdctl: changed 'revision' to 'hash_revision' in hashkv table output
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-25 18:59:49 +08:00
Benjamin Wang
f2d765d247 etcdctl: update the examples for endpoint hashkv command in readme
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-25 18:49:16 +08:00
Benjamin Wang
cd15507c65 test: enhance case TestEndpointHashKV to check both hash and hashRevision
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-25 18:49:16 +08:00
Benjamin Wang
3b50c60dd7 changelog: cover the change of adding HashRevision into HashKVResponse
Two chanages:
1. Add field `HashRevision` into `HashKVResponse`;
2. Display the new field when executing `etcdctl endpoint hash`.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-25 18:49:16 +08:00
Benjamin Wang
48b8210044 etcdctl: display HashRevision for 127.0.0.1:2379, 1084519789 command
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-25 18:49:16 +08:00
Piotr Tabor
590911dfe3
Merge pull request #14847 from mkumatag/fix_expect
fix TestSignal test
2022-11-25 10:39:25 +01:00
Benjamin Wang
bad80dc2de
Merge pull request #14854 from ahrtr/changelog_memberid_20221125
changelog: add items for both 3.4 and 3.5 changelog to cover the case of removing memberid from corrupt alarm
2022-11-25 17:01:18 +08:00
Benjamin Wang
8e3b87b61f changelog: add items for both 3.4 and 3.5 changelog to cover the case of removing memberid from corrupt alarm
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-25 16:28:38 +08:00
Manjunath Kumatagi
0bc46ec92c
fix TestSignal test
Signed-off-by: Manjunath A Kumatagi <mkumatag@in.ibm.com>
2022-11-25 12:50:02 +05:30
Marek Siarkowicz
f18ac02e0f
Merge pull request #14834 from fuweid/bump-grpc-1.51
bump grpc to v1.51.0 from v1.47.0
2022-11-24 09:07:56 +01:00
Benjamin Wang
b6a3594710
Merge pull request #14840 from Bhargav-InfraCloud/copyright-check-in-test-files
Shell func go_srcs_in_module to list go src+test files (#14827)
2022-11-24 06:03:21 +08:00
Bhargav Ravuri
dbfe42bbd2 comments: fix comments as per goword in go _test pkg files
Comments fixed as per goword in go _test package files that
shell function go_srcs_in_module lists as per changes on #14827

Helps in #14827

Signed-off-by: Bhargav Ravuri <bhargav.ravuri@infracloud.io>
2022-11-24 00:03:00 +05:30
Bhargav Ravuri
18463081ad scripts: go_srcs_in_module to list test packages files
The shell func go_srcs_in_module will now list
1. go src files
2. go test files belong to same packages
3. go test files that belong to _test packages

Fixes #14827

Signed-off-by: Bhargav Ravuri <bhargav.ravuri@infracloud.io>
2022-11-23 23:50:21 +05:30
Bhargav Ravuri
2feec4fe68 comments: fix comments as per goword in go test files
Comments fixed as per goword in go test files that shell
function go_srcs_in_module lists as per changes on #14827

Helps in #14827

Signed-off-by: Bhargav Ravuri <bhargav.ravuri@infracloud.io>
2022-11-23 23:05:42 +05:30
Bhargav Ravuri
cc77eb1011 scripts: go_srcs_in_module to list go source + test files
The shell function go_srcs_in_module will list go test files
along with go src files in the specified module. This helps
in identifying the copyright header misses in test files.

Fixes #14827

Signed-off-by: Bhargav Ravuri <bhargav.ravuri@infracloud.io>
2022-11-23 21:52:09 +05:30