87 Commits

Author SHA1 Message Date
Benjamin Wang
68c0cf834a
Merge pull request #14288 from VladSaioc/bugfix-goleak-watchtest
Fixed goroutine leak in tests/integration/clientv3/watch_test.go
2022-08-01 06:55:57 +08:00
VladSaioc
cbfddfe63f Fixed goroutine leak in tests/integration/clientv3/watch_test.go
Signed-off-by: VladSaioc <vladsaioc10@gmail.com>
2022-07-31 23:09:50 +02:00
VladSaioc
3a775201c0 Fixed goroutine leak
Signed-off-by: VladSaioc <vladsaioc10@gmail.com>
2022-07-31 22:19:59 +02:00
Benjamin Wang
8038e876d3 replace ioutil with os package
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-17 10:11:30 +08:00
Marek Siarkowicz
9612fc1194 tests: Unify TestCompactionHash and extend it to also Delete keys and Defrag
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:19 +02:00
Marek Siarkowicz
1581eef888 tests: Add integration tests for compact hash
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:19 +02:00
Chao Chen
e004c91d36 migrate e2e & integration role_test to common 2022-05-10 10:17:16 -07:00
eval-exec
755e2e7755 fix data race in testWatchOverlapContextCancel #14003 2022-05-01 21:12:38 +08:00
Danielle Lancashire
6cb7372e6d clientv3: disable mirror auth test with proxy 2022-04-12 13:42:13 +00:00
Danielle Lancashire
e324cc1cbe cv3/mirror: Fetch the most recent prefix revision
When a user sets up a Mirror with a restricted user that doesn't have
access to the `foo` path, we will fail to get the most recent revision
due to permissions issues.

With this change, when a prefix is provided we will get the initial
revision from the prefix rather than /foo. This allows restricted users
to setup sync.
2022-04-11 13:42:03 +00:00
Marek Siarkowicz
73fc864247 tests: Pass logger to backend 2022-04-05 15:53:38 +02:00
Piotr Tabor
ed1bc447c7 Flakes: Additional logging and timeouts to understand common flakes. 2022-04-03 14:48:36 +02:00
Piotr Tabor
68f2cb8c77 Fix ExampleAuth from integration/clientv3/examples (on OsX)
The code now ensures that each of the test is running in its own directory as opposed to shared os.tempdir.
```
$  (cd tests && env go test -timeout=15m --race go.etcd.io/etcd/tests/v3/integration/clientv3/examples -run ExampleAuth)
2022/04/03 10:24:59 Running tests (examples): ...
2022/04/03 10:24:59 the function can be called only in the test context. Was integration.BeforeTest() called ?
2022/04/03 10:24:59 2022-04-03T10:24:59.462+0200	INFO	m0	LISTEN GRPC	{"member": "m0", "grpcAddr": "localhost:m0", "m.Name": "m0"}
```
2022-04-03 14:16:45 +02:00
Piotr Tabor
d57f8dba62 Deflaking: Make WaitLeader (and WaitMembersForLeader) aggressively (30s) wait for leader being established.
Nearly none of the tests was checking the value... just assuming WaitLeader success.

```
    maintenance_test.go:277: Waiting for leader...
    logger.go:130: 2022-04-03T08:01:09.914+0200	INFO	m0	cluster version differs from storage version.	{"member": "m0", "cluster-version": "3.6.0", "storage-version": "3.5.0"}
    logger.go:130: 2022-04-03T08:01:09.915+0200	WARN	m0	leader failed to send out heartbeat on time; took too long, leader is overloaded likely from slow disk	{"member": "m0", "to": "2acc3d3b521981", "heartbeat-interval": "10ms", "expected-duration": "20ms", "exceeded-duration": "103.756219ms"}
    logger.go:130: 2022-04-03T08:01:09.916+0200	INFO	m0	updated storage version	{"member": "m0", "new-storage-version": "3.6.0"}
    ...
    logger.go:130: 2022-04-03T08:01:09.926+0200	INFO	grpc	[[roundrobin] roundrobinPicker: Build called with info: {map[0xc002630ac0:{{unix:localhost:m0 localhost <nil> 0 <nil>}} 0xc002630af0:{{unix:localhost:m1 localhost <nil> 0 <nil>}} 0xc002630b20:{{unix:localhost:m2 localhost <nil> 0 <nil>}}]}]
    logger.go:130: 2022-04-03T08:01:09.926+0200	WARN	m0	apply request took too long	{"member": "m0", "took": "114.661766ms", "expected-duration": "100ms", "prefix": "", "request": "header:<ID:12658633312866157316 > cluster_version_set:<ver:\"3.6.0\" > ", "response": ""}
    logger.go:130: 2022-04-03T08:01:09.927+0200	INFO	m0	cluster version is updated	{"member": "m0", "cluster-version": "3.6"}
    logger.go:130: 2022-04-03T08:01:09.955+0200	INFO	m2.raft	9f96af25a04e2ec3 [logterm: 2, index: 8, vote: 9903a56eaf96afac] ignored MsgVote from 2acc3d3b521981 [logterm: 2, index: 8] at term 2: lease is not expired (remaining ticks: 10)	{"member": "m2"}
    logger.go:130: 2022-04-03T08:01:09.955+0200	INFO	m0.raft	9903a56eaf96afac [logterm: 2, index: 8, vote: 9903a56eaf96afac] ignored MsgVote from 2acc3d3b521981 [logterm: 2, index: 8] at term 2: lease is not expired (remaining ticks: 5)	{"member": "m0"}
    logger.go:130: 2022-04-03T08:01:09.955+0200	INFO	m0.raft	9903a56eaf96afac [term: 2] received a MsgAppResp message with higher term from 2acc3d3b521981 [term: 3]	{"member": "m0"}
    logger.go:130: 2022-04-03T08:01:09.955+0200	INFO	m0.raft	9903a56eaf96afac became follower at term 3	{"member": "m0"}
    logger.go:130: 2022-04-03T08:01:09.955+0200	INFO	m0.raft	raft.node: 9903a56eaf96afac lost leader 9903a56eaf96afac at term 3	{"member": "m0"}
    maintenance_test.go:279: Leader established.
```

Tmp
2022-04-03 12:23:09 +02:00
Piotr Tabor
8cd8a1ea10 Flakes in integration/clientv3/examples/...
The tests sometimes flaked due to already existing socket-files.
Now each execution works in a tempoarary directory.
2022-04-02 16:16:25 +02:00
Marek Siarkowicz
a1f24af310
Merge pull request #13804 from endocrimes/dani/watch-fix
integration/client/watch.TestWatchCancelRunning: remove duplicate setup
2022-03-16 15:43:35 +01:00
Danielle Lancashire
3bbbef54c1 integration/clientv3/exp/recipes: Fix lock test
TestMutexTryLock(SingleNode|MultiNode) were being skipped as they had duplicate calls to integration2.BeginTest. These duplicate calls registered a second handler for leak detection which failed and skipped the test as the wrappers had already started a cluster.

part of #13698
2022-03-15 17:42:47 +00:00
Danielle Lancashire
b50239f9a4 integration/client/watch: remove duplicate setup
This test was being skipped as it has a duplicate call to
integration2.BeginTest. This duplicate call registered a second handler
for leak detection which failed and skipped the test as the wrapper
(runWatchTest) has already started a cluster.
2022-03-15 16:41:14 +00:00
Marek Siarkowicz
a60abee0d4 tests: Migrate kv delete tests to common framework. 2022-03-02 11:37:24 +01:00
Marek Siarkowicz
bd9f366f40 tests: Migrate TestKVRange integration test to TestKVGet 2022-03-01 10:43:31 +01:00
Marek Siarkowicz
744dd077cd tests: Create common framework for e2e and integration tests and migrate TestKVPut test 2022-02-23 16:47:02 +01:00
Marek Siarkowicz
3788523a3e tests: Move revision tests out of concurrency 2022-01-24 15:33:30 +01:00
Marek Siarkowicz
264fdbb6a1 test: Remove separation between V2 and V3 cluster in integration tests 2022-01-20 14:10:09 +01:00
Marek Siarkowicz
5bcbf77980 tests: Move client to member struct 2022-01-20 14:10:09 +01:00
Marek Siarkowicz
eac6d71352 tests: Add integration test for revision monotonic under failure injection 2022-01-12 11:51:12 +01:00
Sam Batschelet
63a1cc3fe4 add --experimental-max-learner flag
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-11-09 09:52:00 -05:00
Piotr Tabor
9c28e07588
Merge pull request #13447 from Juneezee/deprecate-ioutil
*: move from io/ioutil to io and os packages
2021-10-29 23:18:36 +02:00
Sam Batschelet
a0bdfc4fc9 tests/integration/clientv3: allow time for endpoints to update
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-10-27 17:26:30 -04:00
Eng Zer Jun
2a151c8982
*: move from io/ioutil to io and os packages
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-10-28 00:05:28 +08:00
Marek Siarkowicz
ca9b720c1d tests: Move integration setup to separa framework package 2021-10-20 10:34:35 +02:00
Marek Siarkowicz
5364758181 tests: Cleanup member interface by exposing Bridge directly 2021-09-29 12:42:16 +02:00
Marek Siarkowicz
451eb5d711 tests: Make using bridge optional 2021-09-29 12:42:14 +02:00
Marek Siarkowicz
f2dd5d80a1 tests: Rename grpcAddr to grpcURL to imply that it includes schema 2021-09-28 16:45:06 +02:00
Eduardo Patrocinio
87f1dc7e40 Fix a few typos 2021-09-03 16:09:09 -04:00
Marek Siarkowicz
5e40a8b00c server: Create storage package and move mvcc files to it 2021-07-12 15:37:21 +02:00
Marek Siarkowicz
e21cf4ef02 tkest: Prevent TestMaintenanceSnapshotWithVersionVersion & TestSaveSnapshotVersion from flaking
When running 100 times in row those tests flaked around 10-20%. Based on
some experimentation 10 keys was enough to ensure that wal snapshot is
created and prevented any flakes.
2021-07-09 13:38:40 +02:00
Marek Siarkowicz
e2740b4afa server,etcdutl: Preserve etcd version in backend allowing etcdutl to read it from snapshot 2021-06-25 14:06:56 +02:00
Marek Siarkowicz
e1b1d93548 *: Snapshot returns local etcd version
Co-authored-by: Lili Cosic <cosiclili@gmail.com>
2021-06-14 16:36:50 +02:00
Piotr Tabor
f15e0b8237 integration.BeforeTest can be run without leak-detection. 2021-05-27 20:55:11 +02:00
Pavan BG
d9c5e1f0a2 *: Replace internal testutil AssertEqual function in favour of github.com/stretchr/testify/assert.Equals
To maintain uniformity in testing methodology, assert.Equals has been used everywhere while replacing testutil.AssertEqual

Fixes #13015
2021-05-20 19:55:45 +05:30
Piotr Tabor
3dd577ec08 Simplify client's logger configuration in tests. 2021-05-18 13:03:16 +02:00
Marek Siarkowicz
06afe87b34 client: Allow setting zap logger in config 2021-05-12 12:15:20 +02:00
Piotr Tabor
a78d072b9a Simplify KVstore dependency on cindex. 2021-05-04 18:21:24 +02:00
Piotr Tabor
3acf6e35b2 Finish migration zap.NewExample -> zaptest.NewLogger(t) in integration tests 2021-04-29 22:32:26 +02:00
Piotr Tabor
ffea1537d4 ClientV3 tests use integration.NewClient that configures proper logger. 2021-04-29 18:18:34 +02:00
Piotr Tabor
9a4b2bdccc Errors: context cancelled or context deadline exceeded are exposed as codes.Canceled, codes.DeadlineExceeded instead of 'codes.Unknown' 2021-04-22 14:35:24 +02:00
Piotr Tabor
b1c04ce043 Applying consistency fix: ClusterVersionSet (and co) might get no applied on v2store
ClusterVersionSet, ClusterMemberAttrSet, DowngradeInfoSet functions are
writing both to V2store and backend. Prior this CL there were
in a branch not executed if shouldApplyV3 was false,
e.g. during restore when Backend is up-to-date (has high
consistency-index) while v2store requires replay from WAL log.

The most serious consequence of this bug was that v2store after restore
could have different index (revision) than the same exact store before restore,
so potentially different content between replicas.

Also this change is supressing double-applying of Membership
(ClusterConfig) changes on Backend (store v3) - that lackilly are not
part of MVCC/KeyValue store, so they didn't caused Revisions to be
bumped.

Inspired by jingyih@ comment:
https://github.com/etcd-io/etcd/pull/12820#issuecomment-815299406
2021-04-12 09:43:48 +02:00
Piotr Tabor
e776efbb2a
Merge pull request #12828 from ptabor/20210404-embed-etcd
embed: etcd.Close() is closing Errc() channel as well.
2021-04-08 01:20:07 +02:00
Piotr Tabor
5da9cac193 embed: etcd.Close() is closing Errc() channel as well.
Inspired by https://github.com/etcd-io/etcd/pull/9612 by purpleidea@.
2021-04-08 01:19:13 +02:00
Piotr Tabor
3bb7acc8cf Migrate dependencies pkg/foo -> client/pkg/foo 2021-04-07 00:38:47 +02:00