Mirroristas/etcd
2
0
Fork 0
mirror of https://github.com/etcd-io/etcd.git synced 2024-09-27 06:25:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
21,717 Commits 15 Branches 491 Tags 148 MiB
Commit Graph

57 Commits

Author SHA1 Message Date
Marek Siarkowicz
f194f4723c Reduce number of concurrent clients to 8 and compactions to avoid flakes 
Not hitting minimal QPS is expected to be caused by introduction of
compation. Let's avoid it for high throughput test cases.

Reducing number of clients to avoid linearization timeouts.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2024-08-15 13:57:06 +02:00
Faeka Ansari
e97bc39060 Reduce client concurrency for high traffic robustness tests 
Signed-off-by: Faeka Ansari <faeka6@gmail.com>
 2024-06-29 15:00:04 +05:30
Madhav Jivrajani
5c2422ba05 tests/robustness: fix access of ChoiceWeight 
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
 2024-06-18 13:44:40 +05:30
Marek Siarkowicz
c70e0e4f55
Merge pull request #18181 from serathius/robustness-compact-lazyfs 
Avoid sending Compact request when LazyFS is enabled
 2024-06-18 09:26:41 +02:00
Marek Siarkowicz
2deefb081b
Merge pull request #18060 from siyuanfoundation/robust 
robustness: change mixedVersionOption to use ChoiceWeight.
 2024-06-18 08:49:47 +02:00
Marek Siarkowicz
2e04ee77b6 Avoid sending Compact request when LazyFS is enabled 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2024-06-18 08:36:24 +02:00
Siyuan Zhang
fff58bb809 robustness: change mixedVersionOption to use ChoiceWeight. 
Signed-off-by: Siyuan Zhang <sizhang@google.com>
 2024-06-17 15:53:47 -07:00
Marek Siarkowicz
5e42ed9b22 Reproduce issue #17529 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2024-06-15 19:40:23 +02:00
Marek Siarkowicz
5959110f4a Implement Compaction support in robustness test 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2024-06-07 10:33:57 +02:00
Wei Fu
fc1863086c tests/robustness: unlock Delete/LeaseRevoke ops 
We should return token to that bucket if `nonUniqueWriteLimiter.Take()`
return true. After unlock Delete/LeaseRevoke ops, the model should be
updated for replay function. There are two updates for `toWatchEvents`.

1. When leaveRevokes op has deleted few keys, we should generate
   `delete-operation` events based on alphabetical order of deleted
   keys.
2. When putWithLease op hits non-exist lease, we should ignore that
   update event.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
 2024-06-06 20:40:08 +08:00
Marek Siarkowicz
aaa9f15f23 Increase robustness test request timeout to 200ms 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2024-05-22 18:00:15 +02:00
Marek Siarkowicz
c4e3b61a1c Record operation from failpoint injection 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2024-05-01 19:20:22 +02:00
Marek Siarkowicz
062a0ea057
Merge pull request #17825 from serathius/robustness-qps 
Don't require minimal for failpoint injection period
 2024-04-22 19:03:42 +02:00
Marek Siarkowicz
fa9e9504ad Handle watch responses with error 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2024-04-21 20:49:49 +02:00
Marek Siarkowicz
f285330d46 Don't require minimal for failpoint injection period 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2024-04-20 10:34:51 +02:00
Marek Siarkowicz
452445e2d8
Merge pull request #17781 from serathius/robustness-read-limit 
Remove limit from read requests after a failed write
 2024-04-14 12:05:23 +02:00
Marek Siarkowicz
cadfc407e9 Remove limit from read requests after a failed write 
Limit can cause multiple request due to pagination.
For reads after a failed write we would like to return to normal write
request as soon as possible.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2024-04-12 15:01:17 +02:00
Marek Siarkowicz
f8de338ab2 Add explicit checks for assumptions in robustness test validation 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2024-04-12 14:18:22 +02:00
Marek Siarkowicz
718d5ba2b4 Calculate request success rate to provide signal to performance debugging 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2024-04-11 09:36:17 +02:00
Ivan Valdes
0976398964
tests/robustness: address golangci var-naming issues 
Signed-off-by: Ivan Valdes <ivan@vald.es>
 2024-03-25 16:27:05 -07:00
Madhav Jivrajani
0b27570368 tests/robustness: use WithRequireLeader in Kubernetes traffic 
Kubernetes uses WithRequireLeader to modify the context passed
to the Watch() and RequestProgress() calls in order to ensure
that a leader is present in the cluster before serving the request.

This commit mimics that behaviour in the Kubernetes traffic.

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
 2024-02-22 15:01:33 +05:30
Madhav Jivrajani
9aad6700d5 tests/robustness: add robustness test for watch with PrevKV() 
Kubernetes relies on the PrevKV() option in the watches it opens
against etcd. This commit adds a robustness test to validate the
same.

A watch response returned with PrevKV() is valid if:
The value in current event's prevKV matches the previous
event's value of the same key if this is not a create event.

There are cases where there can be a prevKV for a create event
as well, for example if a watch is opened after the key is creatd.
Since we don't simulate for that, we don't check for that.

Further, this adjusts revision numbers such that we can successfully create
a new replay. Needed now since we will have unit tests with
and without PrevKV co-existing and we requite creation of a
new replay everytime we validate PrevKV.

We also regenerate test data with so that prevKV exists in it

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
 2024-02-13 22:55:57 +05:30
Jongwoo Han
08d799c4cc
Correct typo from 'Kuberntes' to 'Kubernetes' 
Signed-off-by: Jongwoo Han <jongwooo.han@gmail.com>
 2023-12-20 18:09:31 +09:00
Wei Fu
07effc4d0a *: fix revive linter 
Remove old revive_pass in the bash scripts and migirate the revive.toml
into golangci linter_settings.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
 2023-09-24 14:21:11 +08:00
Wei Fu
aa97484166 *: enable goimports in verify-lint 
Signed-off-by: Wei Fu <fuweid89@gmail.com>
 2023-09-21 21:14:09 +08:00
Wei Fu
5e3910d96c *: fix govet-shadow lint 
Signed-off-by: Wei Fu <fuweid89@gmail.com>
 2023-09-19 20:24:01 +08:00
Wei Fu
3f6a5c0bb1 *: enable larger runner 
Use ubuntu-latest-8-cores larger runner to support lazyfs in robustness
CI.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
 2023-08-18 22:21:00 +08:00
Marek Siarkowicz
a2bd589cdb tests/robustness: Reduce minimal QPS to eliminate flakes 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-08-05 19:27:10 +02:00
Marek Siarkowicz
eb32d9cccc tests: Add support for lazyfs 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-07-27 13:42:38 +02:00
Marek Siarkowicz
8fca6ebdb2 tests/robustness: Prevent to many concurrent non-unique writes which are causing linearization to timeout 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-07-03 14:39:23 +02:00
Marek Siarkowicz
11da84a1d1 tests/robustness: Implement loading client reports 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-06-28 15:35:17 +02:00
Marek Siarkowicz
09c462e2ea tests/robustness: Add resumable validation and improve reliable validation 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-06-26 12:12:39 +02:00
Marek Siarkowicz
26cd2bc017 tests/robustness: Store whole watch operations 
Want to keep watch requests to properly validate reliability of watch
stream.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-06-24 18:15:50 +02:00
Marek Siarkowicz
c3720fac3d tests/robustness: Implement Kubernetes pagination 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-06-23 09:44:22 +02:00
Marek Siarkowicz
f985890ac3
Merge pull request #16113 from serathius/robustness-range 
tests/robustness: Implement proper range requests
 2023-06-22 20:50:27 +02:00
Marek Siarkowicz
f7831e2607 tests/robustness: Move key into dedicated options to allow to implement proper ranges 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-06-22 18:27:54 +02:00
Marek Siarkowicz
3f09a51c60 tests/robustness: Make watch event independent from etcd operation 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-06-22 15:43:27 +02:00
Marek Siarkowicz
0d2d383f0a tests/robustness: Fix multi txn key prefix 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-06-22 14:27:31 +02:00
Marek Siarkowicz
9fc438cb6b tests/robustness: Add List and StaleList requests to etcd traffic 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-06-21 18:03:41 +02:00
Marek Siarkowicz
519617cfd0 tests/robustness: Tune Kubernetes tests to reduce number of delete requests 
Having too many delete requests is bad as they are not unique requests, so
linearization is more prone to timeout on them.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-06-20 09:45:23 +02:00
Marek Siarkowicz
9c659eb4e0
Merge pull request #16072 from serathius/robustness-stale-read 
Validate stale read
 2023-06-19 18:22:08 +02:00
Marek Siarkowicz
1663600bec tests/robustness: Validate stale get requests by replaying etcd state 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-06-19 14:17:38 +02:00
Marek Siarkowicz
09b9f889e7 tests/robustness: Refactor etcd traffic client 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-06-19 12:08:17 +02:00
Marek Siarkowicz
96987d8b5e tests/robustness: Implement stale reads without validation 
For now we just validate stale read revision, but not response content.
Reason is that etcd model only stores latest version of keys, and no
history like real etcd.

Validating stale read contents needs to be done outside of model
as storing whole history is just to costly for linearization validation.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-06-16 21:17:37 +02:00
Marek Siarkowicz
6979318108 tests/robustness: Make Range a proper request type to allow setting Range.Revision != 0 for stale reads 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-06-14 13:55:45 +02:00
Marek Siarkowicz
974655e02c tests/robustness: Rename operations const to separate from RequestType 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-06-14 13:24:14 +02:00
Marek Siarkowicz
a268a67e45 tests/robustness: Move get to list of randomized operations 2023-06-13 21:09:05 +02:00
Marek Siarkowicz
16bf0f6641 tests/robustness: Use traffic.RecordingClient in watch 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-05-25 22:17:23 +02:00
Marek Siarkowicz
4872b679a5 tests/robustness: Expect revions to be unique for Kubernetes Traffic 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-05-23 15:51:10 +02:00
Marek Siarkowicz
6429f47631 tests/robustness: Validate all etcd watches opened to etcd 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-05-16 10:28:01 +02:00