1103 Commits

Author SHA1 Message Date
Marek Siarkowicz
b3bb996880
Merge pull request #15420 from ahrtr/dependency_20230307
Bump some dependencies
2023-03-09 13:03:56 +01:00
James Blair
5b3497555f
Updated go to 1.19.7.
Mitigates CVE-2023-24532.

Signed-off-by: James Blair <mail@jamesblair.net>
2023-03-08 21:39:31 +13:00
Wei Fu
3419230eea tests/integration: Update TestLeasingDeleteRangeContendTxn
The TestLeasingDeleteRangeContendTxn is trying to test for RangeDelete when
the target resources are being updated.  When the `txnLeasing` wants a
server-side transaction, it needs to ensure all the keys mod revision should
be leass than what it saw. If the compare fails, it will repeat to apply the
server-side transaction until it is sucessful. I believe the test-case is
trying to verify how the `txnLeasing` handles the race issue.

Before the patch #15401, the resource-updating goroutine keeps updating until
the RangeDelete finishes. The testcase is flaky because two goroutines are
sharing one `ctx` and grpc-go client won't wait for the response if `ctx`
has been canceled.

For example,

| DelLease Goroutine   | PutLease Goroutine         | ETCD Server                    | Key/0 Status |
| --                   | ---                        | --                             | --           |
| deleted              |                            |                                | version = 0  |
|                      | send update(key/0=123) req | received update(key/0=123) req | version = 0  |
| cancel               |                            |                                | version = 0  |
|                      | exit because of cancel     |                                | version = 0  |
| get key/0 by putkv   |                            |                                | version = 0  |
|                      |                            | applied update(key/0=123)      | version = 1  |
| get key/0 by raw-cli |                            |                                | version = 1  |

So `raw-cli` gets `[key/0=123]` while the `putkv` gets `[]`. If `putkv`
applies two update reqs to ETCD server and the last one is canceled
before apply, the error will be like:

```
expected [key:"key/0" version:2 value:"123" ], got [key:"key/0" version:1 value:"123" ]
```

The resource-updating goroutine should not share the ctx with RangeDelete here.
And I also revert current main branch because the resource-update goroutine
only updates 8 times and might exit before `RangeDelete`. In this case,
the `txnLeasing` is not handling the race issue.

Fixes: #15352

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-03-07 23:11:41 +08:00
Benjamin Wang
7ed1219861 dependency: bump golang.org/x/net to v0.8.0
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-03-07 06:52:07 +08:00
Benjamin Wang
b53d63219a dependency: bump github.com/cheggaaa/pb/v3 to v3.1.2
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-03-07 06:17:01 +08:00
James Blair
275e10bcf7
Return default snapshot count to 10,000.
The huge (100k+) value was justified when storev2 was being dumped completely with every snapshot.

With storev2 being decomissioned we can checkpoint more frequently for faster recovery.

Signed-off-by: James Blair <mail@jamesblair.net>
2023-03-06 20:21:03 +13:00
Chao Chen
c002e096b5 migration auth tests to common #5
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-03-03 12:26:36 -08:00
Marek Siarkowicz
85bd9d7719
Merge pull request #15401 from tjungblu/racefix_15352
Fixing flaky TestLeasingDeleteRangeContendTxn
2023-03-03 12:23:25 +01:00
Thomas Jungblut
63964ec781 Fixing flaky TestLeasingDeleteRangeContendTxn
Fixes etcd-io#15352.
Depending on the goroutine scheduling, the expected count of 8 might not
have been reached yet. This ensures the routine won't stop earlier than
that.

Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
2023-03-03 11:38:22 +01:00
Piotr Tabor
d358e35f3e
Merge pull request #15379 from chaochn47/auth_test_split_6
migrate auth tests to common #6
2023-03-03 10:55:28 +01:00
Benjamin Wang
56423eacd2 dependency: bump go.opentelemetry.io/otel to v1.14.0
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-03-03 15:44:47 +08:00
Chao Chen
d798816baf migrate auth tests to common #6
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-02-28 10:32:29 -08:00
Benjamin Wang
5f872589cd dependency: bump golang.org/x/sync to v0.1.0
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-02-28 06:56:16 +08:00
Benjamin Wang
6af2c9dbfa dependency: bump golang.org/x/crypto to v0.6.0
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-02-28 06:51:55 +08:00
Benjamin Wang
f333148da9 dependency: bump golang.org/x/time to v0.3.0
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-02-28 06:49:39 +08:00
Benjamin Wang
d8b8127084 dependency: bump github.com/stretchr/testify from 1.8.1 to 1.8.2
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-02-28 06:44:33 +08:00
Benjamin Wang
da4bf0f76f
Merge pull request #15261 from ahrtr/memberlist_20230208
clientv3: support serializable `MemberList` operation
2023-02-28 06:26:38 +08:00
Marek Siarkowicz
d475cf81a0 tests: Rename linearizability tests to robustness
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-02-26 14:36:18 +01:00
Benjamin Wang
1f0d361848 clientv3: support serializable MemberList operation
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-02-25 07:23:39 +08:00
Chao Chen
b93745a862 migrate to auth test #4
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-02-23 10:46:31 -08:00
Benjamin Wang
de9c0f8fea dependency: bump github.com/golang-jwt/jwt/v4 to v4.5.0
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-02-21 05:29:53 +08:00
xin.li
b17b9c1428 chore: Use http constants to replace numbers as parameters
Signed-off-by: xin.li <xin.li@daocloud.io>
2023-02-20 11:53:41 +08:00
WangXiaoxiao
8e56e2f5ce add mix version for snapshot e2e case
Signed-off-by: WangXiaoxiao <1141195807@qq.com>
2023-02-18 18:01:48 +08:00
Benjamin Wang
a7e94c4843 security: bump golang.org/x/net to v0.7.0 to address CVE GO-2023-1571
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-02-17 11:29:47 +08:00
Marek Siarkowicz
c24ba4d3e8
Merge pull request #15320 from chaochn47/migrate_auth_tests_to_common
migrate 6 auth test cases to common #3
2023-02-16 11:40:34 +01:00
Marek Siarkowicz
e603d92727
Merge pull request #15325 from serathius/linearizability-refactor-report
tests: Refactor file structure
2023-02-16 11:23:56 +01:00
James Blair
ee6781bf6f Bump to go 1.19.6
go 1.19.6 (released 2023-02-14) includes important security and bug fixes.

Signed-off-by: James Blair <mail@jamesblair.net>
2023-02-16 17:12:59 +08:00
Chao Chen
c36ea3fdcf migrate auto tests to common #3
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-02-15 11:08:01 -08:00
Marek Siarkowicz
a36951ad18 tests: Extract report related functions and clean up main file
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-02-15 16:13:43 +01:00
Marek Siarkowicz
116a3150c0
Merge pull request #15282 from serathius/linearizability-report-watch
test: Report watch histories
2023-02-15 16:00:10 +01:00
Marek Siarkowicz
edfc963302 tests: Detect duplicate watch event
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-02-15 13:59:03 +01:00
Marek Siarkowicz
0cd5c9ca37 tests: Refactor reporting results
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-02-15 13:02:42 +01:00
Marek Siarkowicz
d99b1dbdaf tests: Move results reporting to top and add reporting watch histories
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-02-15 13:02:42 +01:00
Marek Siarkowicz
58d74e2b73 test: Refactor TestLinearizability function
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-02-15 13:02:42 +01:00
Marek Siarkowicz
a64263cf49 tests: Refactor persisting single member dir
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-02-15 13:02:42 +01:00
Marek Siarkowicz
d0e5c44f67 tests: Refactor getting test results directory
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-02-15 13:02:42 +01:00
Marek Siarkowicz
2afaddd5b7 tests: Refactor getting longest history
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-02-15 13:02:42 +01:00
Marek Siarkowicz
a806d155eb
Merge pull request #15323 from lavacat/main-mono-clock
tests: use monotonic clock for linearizability test history
2023-02-15 11:38:44 +01:00
Marek Siarkowicz
25eb025fc0
Merge pull request #15293 from serathius/linearizability-bookmark
Codify kubernetes using response header revision as bookmark
2023-02-15 11:36:03 +01:00
Benjamin Wang
d0c3112d5a dependency: bump go.opentelemetry.io/otel from 1.11.2 to 1.13.0
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-02-15 18:33:00 +08:00
Bogdan Kanivets
d2ccb12f36 tests: use monotonic clock for linearizability test history
Problem: time.Now() uses wall clock reading. See https://github.com/golang/go/blob/master/src/time/time.go#L17
"later time-telling operations use the wall clock reading, but later time-measuring operations, specifically comparisons and subtractions, use the monotonic clock reading."
This can cause 'Return' to be before 'Call' and wrong order of operations from different clients.

Solution: use same base time for all clients and only use 'time-measuring' operations to record timestamps for history.
Signed-off-by: Bogdan Kanivets <bkanivets@apple.com>
2023-02-15 01:37:21 -08:00
Marek Siarkowicz
b55dbe88ca tests: Codify kubernetes using response header revision as bookmark
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-02-15 10:26:34 +01:00
Marek Siarkowicz
04ba936ea6 tests: Implement multi operation Txn
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-02-14 17:57:52 +01:00
Marek Siarkowicz
b4f9bb45fc tests: Allow multiple operations with same mod revisions, but forbid those operations to be spread between multiple responses
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-02-14 14:36:51 +01:00
Marek Siarkowicz
6c439aa9e2
Merge pull request #15283 from serathius/linearizability-blackhole-refactor
tests: Refactor blackholing and add separate BlackholePeerUntilSnapsh…
2023-02-14 11:56:37 +01:00
Benjamin Wang
1d8fe11ad9 dependency: bump golang.org/x/net from 0.5.0 to 0.6.0
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-02-14 17:40:47 +08:00
Benjamin Wang
cc47f7bc7c dependency: bump gopkg.in/natefinch/lumberjack.v2 from 2.0.0 to 2.2.1
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-02-14 17:40:47 +08:00
Marek Siarkowicz
e569217a6d tests: Refactor blackholing
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-02-14 09:38:10 +01:00
Marek Siarkowicz
ed30d5415a
Merge pull request #15284 from serathius/funtional-remove
tests: Remove functional testing as they were replaced by linearizabi…
2023-02-13 12:32:04 +01:00
Benjamin Wang
b4dfa9d34f test: correct the logic of comparing revision in linearizablity/watch.go
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-02-13 19:22:31 +08:00