214 Commits

Author SHA1 Message Date
Wei Fu
3419230eea tests/integration: Update TestLeasingDeleteRangeContendTxn
The TestLeasingDeleteRangeContendTxn is trying to test for RangeDelete when
the target resources are being updated.  When the `txnLeasing` wants a
server-side transaction, it needs to ensure all the keys mod revision should
be leass than what it saw. If the compare fails, it will repeat to apply the
server-side transaction until it is sucessful. I believe the test-case is
trying to verify how the `txnLeasing` handles the race issue.

Before the patch #15401, the resource-updating goroutine keeps updating until
the RangeDelete finishes. The testcase is flaky because two goroutines are
sharing one `ctx` and grpc-go client won't wait for the response if `ctx`
has been canceled.

For example,

| DelLease Goroutine   | PutLease Goroutine         | ETCD Server                    | Key/0 Status |
| --                   | ---                        | --                             | --           |
| deleted              |                            |                                | version = 0  |
|                      | send update(key/0=123) req | received update(key/0=123) req | version = 0  |
| cancel               |                            |                                | version = 0  |
|                      | exit because of cancel     |                                | version = 0  |
| get key/0 by putkv   |                            |                                | version = 0  |
|                      |                            | applied update(key/0=123)      | version = 1  |
| get key/0 by raw-cli |                            |                                | version = 1  |

So `raw-cli` gets `[key/0=123]` while the `putkv` gets `[]`. If `putkv`
applies two update reqs to ETCD server and the last one is canceled
before apply, the error will be like:

```
expected [key:"key/0" version:2 value:"123" ], got [key:"key/0" version:1 value:"123" ]
```

The resource-updating goroutine should not share the ctx with RangeDelete here.
And I also revert current main branch because the resource-update goroutine
only updates 8 times and might exit before `RangeDelete`. In this case,
the `txnLeasing` is not handling the race issue.

Fixes: #15352

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-03-07 23:11:41 +08:00
Thomas Jungblut
63964ec781 Fixing flaky TestLeasingDeleteRangeContendTxn
Fixes etcd-io#15352.
Depending on the goroutine scheduling, the expected count of 8 might not
have been reached yet. This ensures the routine won't stop earlier than
that.

Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
2023-03-03 11:38:22 +01:00
Benjamin Wang
87e271701b test: enhance the test case TestV3WatchProgressOnMemberRestart
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-02-10 20:09:26 +08:00
Benjamin Wang
36fc3cae65 clientv3: correct the nextRev on receving progress notification response
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-02-10 09:09:19 +08:00
Tero Saarni
588b98d085 Add TLSv1.3 support.
Added optional TLS min/max protocol version and command line switches to set
versions for the etcd server.

If max version is not explicitly set by the user, let Go select the max
version which is currently TLSv1.3. Previously max version was set to TLSv1.2.

Signed-off-by: Tero Saarni <tero.saarni@est.tech>
2023-01-30 16:16:53 +02:00
Piotr Tabor
9abc895122 Goimports: Apply automated fixing to test files as well.
Signed-off-by: Piotr Tabor <ptab@google.com>
2022-12-29 13:04:45 +01:00
Piotr Tabor
6f899a7b40
Merge pull request #15052 from ptabor/20221228-goimports-fix
./scripts/fix.sh: Takes care of goimports across the whole project.
2022-12-29 11:31:22 +01:00
Piotr Tabor
9e1abbab6e Fix goimports in all existing files. Execution of ./scripts/fix.sh
Signed-off-by: Piotr Tabor <ptab@google.com>
2022-12-29 09:41:31 +01:00
Wei Fu
4d0b91947e chore: delete // +build buildtag by go fix
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2022-12-29 14:17:05 +08:00
Ramil Mirhasanov
f0153222f1 clientv3/naming/endpoints: fix endpoints prefix bug
fixes bug with multiple endpoints with same prefix

Signed-off-by: Ramil Mirhasanov <ramil600@yahoo.com>
2022-12-22 13:36:16 +03:00
Piotr Tabor
6fc0d96b42
Merge pull request #14993 from ramil600/add-log
clientv3/concurrency: add logger to session, add unit test
2022-12-19 10:30:38 +01:00
Ramil Mirhasanov
3c582fecb0 clientv3/concurrency: add logger to session, add unit test
Signed-off-by: Ramil Mirhasanov <ramil600@yahoo.com>
2022-12-16 11:11:35 +03:00
Wei Fu
e58c73cc18 maintenance: add test to verify content of Snapshot
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2022-12-16 15:53:39 +08:00
Benjamin Wang
3c51c42417 test: fix nil pointer panic in testMutexLock
Refer to: https://github.com/etcd-io/etcd/actions/runs/3671847902/jobs/6207463700

```
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xedc388]

goroutine 5253 [running]:
go.etcd.io/etcd/client/v3/concurrency.(*Session).Client(...)
	/home/runner/work/etcd/etcd/client/v3/concurrency/session.go:76
go.etcd.io/etcd/client/v3/concurrency.(*Mutex).tryAcquire(0xc000133140, {0x18a8668, 0xc000050158})
	/home/runner/work/etcd/etcd/client/v3/concurrency/mutex.go:111 +0x88
go.etcd.io/etcd/client/v3/concurrency.(*Mutex).Lock(0xc000133140, {0x18a8668, 0xc000050158})
	/home/runner/work/etcd/etcd/client/v3/concurrency/mutex.go:74 +0x68
go.etcd.io/etcd/tests/v3/integration/clientv3/experimental/recipes_test.testMutexLock.func1()
	/home/runner/work/etcd/etcd/tests/integration/clientv3/experimental/recipes/v3_lock_test.go:65 +0x285
created by go.etcd.io/etcd/tests/v3/integration/clientv3/experimental/recipes_test.testMutexLock
	/home/runner/work/etcd/etcd/tests/integration/clientv3/experimental/recipes/v3_lock_test.go:59 +0xda
FAIL	go.etcd.io/etcd/tests/v3/integration/clientv3/experimental/recipes	7.070s
FAIL
```

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-12 10:18:45 +08:00
Benjamin Wang
7b19ee6396 test: add integration test to cover the multiple member corruption case
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-26 19:35:38 +08:00
Benjamin Wang
a3197102e9 test: rollback the change in PR pull/14824
The change did in https://github.com/etcd-io/etcd/pull/14824 fixed
the test instead of the product code. It isn't correct. After we
fixed the product code in this PR, we can revert the change in
that PR.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-26 19:35:38 +08:00
Bhargav Ravuri
dbfe42bbd2 comments: fix comments as per goword in go _test pkg files
Comments fixed as per goword in go _test package files that
shell function go_srcs_in_module lists as per changes on #14827

Helps in #14827

Signed-off-by: Bhargav Ravuri <bhargav.ravuri@infracloud.io>
2022-11-24 00:03:00 +05:30
Bhargav Ravuri
2feec4fe68 comments: fix comments as per goword in go test files
Comments fixed as per goword in go test files that shell
function go_srcs_in_module lists as per changes on #14827

Helps in #14827

Signed-off-by: Bhargav Ravuri <bhargav.ravuri@infracloud.io>
2022-11-23 23:05:42 +05:30
Wei Fu
0b30e83b1d tests/integration: deflake Corruption cases
If the corrupted member has been elected as leader, the memberID in alert
response won't be the corrupted one. It will be a smaller follower ID since
the raftCluster.Members always sorts by ID. We should check the leader
ID and decide to use which memberID.

Fixes: #14823

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2022-11-22 22:19:55 +08:00
Sasha Melentyev
c3b6cbdb73 all: goimports -w .
Signed-off-by: Sasha Melentyev <sasha@melentyev.io>
2022-11-17 19:07:04 +03:00
Sasha Melentyev
2c9c209eb6 all: Changing Printf and friends to Print if there is no formatting
Signed-off-by: Sasha Melentyev <sasha@melentyev.io>
2022-11-15 22:11:23 +03:00
Sasha Melentyev
855aa4f7a7 all: Use ReplaceAll instead of Replace with -1 pos
Signed-off-by: Sasha Melentyev <sasha@melentyev.io>
2022-11-15 00:06:09 +03:00
chenyahui
5b8c6b548f etcdclient: check mutex state in Unlock method of concurrency.Mutex
Check the values of myKey and myRev first in Unlock method to prevent calling Unlock without Lock. Because this may cause the value of pfx to be deleted by mistake.

Signed-off-by: chenyahui <cyhone@qq.com>
2022-11-08 22:24:52 +08:00
Benjamin Wang
62167d1f1f clientv3: fix the design & implementation of double barrier
Check the client count before creating the ephemeral key, do not
create the key if there are already too many clients. Check the
count after creating the key again, if the total kvs is bigger
than the expected count, then check the rev of the current key,
and take action accordingly based on its rev. If its rev is in
the first ${count}, then it's valid client, otherwise, it should
fail.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-10-20 16:24:20 +08:00
Marek Siarkowicz
07ca384753 tests: Move MustAbsPath function to testutils
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-10-17 13:37:14 +02:00
王霄霄
2751ec6479 integration: check Watch response error not nil to avoid runtime panic.
Fixes issue: #14259

Signed-off-by: 王霄霄 <1141195807@qq.com>
2022-10-16 11:41:11 +08:00
Thomas Jungblut
20dccfa484 Test port allocating using net.Listen
To avoid port clashes during integration tests, we let the OS give us a
free port.

Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
2022-10-14 09:40:04 +02:00
Benjamin Wang
a5fb57b485
Merge pull request #14586 from serathius/pattern
tests: Use bash like patterns in TestAuthority instead of string fo…
2022-10-14 06:11:36 +08:00
Marek Siarkowicz
fdce1b38fb tests: Use bash like patterns in TestAuthority instead of string formating
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-10-13 16:45:27 +02:00
Benjamin Wang
b48641e5f2
Merge pull request #14348 from VinozzZ/add-integration-test-for-tracing
embed: add integration test for distributed tracing
2022-10-13 02:11:59 +08:00
Wei Fu
b15f6bf978 integration: deflaking TestMaintenanceSnapshotCancel/Timeout
Since http2 spec defines the receive windows's size and max size of
frame in the stream, the underlayer - gRPC client can pre-read data
from server even if the application layer hasn't read it yet.

And the initialized cluster has 20KiB snapshot, which can be pre-read
by underlayer. We should increase the snapshot's size, just in case
that the io.Copy won't return the canceled or timeout error.

Fixes: #14477

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2022-10-06 11:03:51 +08:00
Chao Chen
8d057ea2b8 common tests framework: cluster client creation fail with invalid auth
Signed-off-by: Chao Chen <chaochn@amazon.com>
2022-09-29 23:09:32 -07:00
Kafuu Chino
f1d4935e91 *: avoid closing a watch with ID 0 incorrectly
Signed-off-by: Kafuu Chino <KafuuChinoQ@gmail.com>

add test
2022-09-26 20:30:33 +08:00
Benjamin Wang
7f10dccbaf Bump go 1.19: update all the dependencies and go.sum files
1. run ./scripts/fix.sh;
2. cd tools/mod; gofmt -w . & go mod tidy;

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-22 08:47:46 +08:00
Benjamin Wang
2441a24cee
Merge pull request #14493 from demoManito/style/format-import-order
etcd: format import order
2022-09-21 06:03:31 +08:00
demoManito
f67ec10779 etcd: format import order
golang CodeReviewComments:
https://github.com/golang/go/wiki/CodeReviewComments#imports

Signed-off-by: demoManito <1430482733@qq.com>
2022-09-20 18:41:39 +08:00
Benjamin Wang
159ed15afc
Merge pull request #14479 from demoManito/fix/declaring-empty-slice
etcd: modify declaring empty slices
2022-09-20 05:22:59 +08:00
Hitoshi Mitake
c793f18238
Merge pull request #14322 from mitake/watch-auth-err
*: handle auth invalid token and old revision errors in watch
2022-09-19 23:33:49 +09:00
Hitoshi Mitake
c5614520d5 tests: a test case for watch with auth token expiration
Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com>
2022-09-17 21:51:36 +09:00
demoManito
5b26fc0101 fix test
Signed-off-by: demoManito <1430482733@qq.com>
2022-09-16 17:08:29 +08:00
demoManito
72cf0cc04a etcd: modify declaring empty slices
declare an empty slice to var s []int replace  s :=[]int{}, https://github.com/golang/go/wiki/CodeReviewComments#declaring-empty-slices

Signed-off-by: demoManito <1430482733@qq.com>
2022-09-16 14:41:14 +08:00
Benjamin Wang
e8ad008067 integration_test: fix the flaky test TestV3AuthRestartMembe
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-14 14:30:37 +08:00
Clark
3f3149b15f tests: fix failed integration test cases after enabling --strict-reconfig-check by default
Signed-off-by: Clark <fwyongxing@gmail.com>
2022-09-03 15:56:51 +08:00
Clark
3beb25489d tests: enable --strict-reconfig-check by default and expose it to common framework
Signed-off-by: Clark <fwyongxing@gmail.com>
2022-08-31 22:21:23 +08:00
Hitoshi Mitake
9c8326bb50
Merge pull request #14358 from vivekpatani/main
server/auth: refresh cache on each NewAuthStore
2022-08-27 11:13:59 +09:00
Thomas Jungblut
e1e92e9c0d Add ErrTimeoutDueToLeaderFail as expected error
This fixes ci flakes where timeouts are expected to happen in network
partitions.
issue: #14333

Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
2022-08-24 13:56:45 +02:00
vivekpatani
ae608da7e6 server,test: refresh cache on each NewAuthStore
- permissions were incorrectly loaded on restarts.
- https://github.com/etcd-io/etcd/issues/14355

Signed-off-by: vivekpatani <9080894+vivekpatani@users.noreply.github.com>
2022-08-23 20:11:47 -07:00
Yingrong Zhao
ea2f299ba0 embed: add integration test for distributed tracing
To verify distributed tracing feature is correctly setup, this PR adds
an integration test for this feature.
In the process of writing the test, I discovered a goroutine leak due to
the TraceProvider not being closed. This PR fixs this issue as well.

Signed-off-by: Yingrong Zhao <yingrong.zhao@gmail.com>
2022-08-15 11:19:10 -04:00
Clark
0b9f43410a tests: e2e and integration share TickDuration constant
Signed-off-by: Clark <fwyongxing@gmail.com>
2022-08-14 18:49:46 +08:00
Benjamin Wang
68c0cf834a
Merge pull request #14288 from VladSaioc/bugfix-goleak-watchtest
Fixed goroutine leak in tests/integration/clientv3/watch_test.go
2022-08-01 06:55:57 +08:00