26 Commits

Author SHA1 Message Date
Marek Siarkowicz
2c56e8edc1
Merge pull request #18107 from serathius/e2e-error-log
Improve e2e error reporting
2024-06-07 13:58:59 +02:00
Marek Siarkowicz
5959110f4a Implement Compaction support in robustness test
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-06-07 10:33:57 +02:00
Marek Siarkowicz
3c5684967f Improve e2e error reporting
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
Co-authored-by: James Blair <mail@jamesblair.net>
Co-authored-by: chao <chaochn@amazon.com>
2024-06-07 10:24:52 +02:00
Marek Siarkowicz
b8eeaacbcb Ignore connection reset error when triggering a failpoint
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-06-05 17:58:46 +02:00
Marek Siarkowicz
1d367fbae6
Merge pull request #17923 from siyuanfoundation/robust
Add randomness in robustness cluster process version to test mixed version scenarios.
2024-05-22 14:07:22 +02:00
Siyuan Zhang
0f94c2ca4f robustness: add mix version scenario with fixed leader.
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-05-21 17:42:12 +00:00
Siyuan Zhang
b54d7552a7 robustness: add mix version option in exploratoryScenarios.
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-05-21 16:57:53 +00:00
Marek Siarkowicz
3fb36d9ae2 Allow gofail trigger to fail as long as the member stops running
This is required for compaction based failpoint, to allow the traffic
send compaction request causing etcd to crash before failpoint executes
the trigger.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-05-21 18:46:35 +02:00
Marek Siarkowicz
d8bb19327b Prevent picking a failpoint that waiting till snapshot that doesn't support lower snapshot catchup entries but allow reproducing issue #15271
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-05-13 12:08:42 +02:00
Marek Siarkowicz
b8ffc5e8c0
Merge pull request #17967 from serathius/robustness-update-readme
Update the robustness README and fix the #14370 reproduction case
2024-05-09 10:05:27 +02:00
Marek Siarkowicz
be9758e2bc Update the robustness README and fix the #14370 reproduction case
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-05-08 11:31:28 +02:00
Marek Siarkowicz
c4e3b61a1c Record operation from failpoint injection
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-05-01 19:20:22 +02:00
Marek Siarkowicz
f285330d46 Don't require minimal for failpoint injection period
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-20 10:34:51 +02:00
Marek Siarkowicz
3a23994fbf Make no failpoint error more readable
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-07 15:13:59 +02:00
Ivan Valdes
0976398964
tests/robustness: address golangci var-naming issues
Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-03-25 16:27:05 -07:00
Marek Siarkowicz
3471ef133d Add an e2e test and robustness failpoint around recovering from snapshot backend
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-01-04 15:25:24 +01:00
Marek Siarkowicz
5175652a8e Abort if failpoint injecton failed
If one of nodes is unhealthy the test would never finish as watchers
would never reach max revision.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-12-03 17:26:51 +01:00
ZhouJianMS
55516234d3 exclude sleep failpoint from 1 node scenario
Signed-off-by: ZhouJianMS <zhoujian@microsoft.com>
2023-11-13 16:19:44 +08:00
ZhouJianMS
d208985aec error handling for gofailpoint
Signed-off-by: ZhouJianMS <zhoujian@microsoft.com>
2023-11-03 19:25:17 +08:00
ZhouJianMS
827dc18682 Add IO stall failpoint in raft loop
Signed-off-by: ZhouJianMS <zhoujian@microsoft.com>
2023-11-03 16:42:33 +08:00
Marek Siarkowicz
45fb4565e3
Merge pull request #16786 from serathius/robustness-drop-packet
Implement random packet dropping
2023-10-19 08:44:23 +02:00
Marek Siarkowicz
aa28a69ce0 Implement random packet dropping
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-10-18 10:14:43 +02:00
Wei Fu
aea1cd0077 feat: enable unparam lint
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-10-17 21:24:13 +08:00
Marek Siarkowicz
7e8bb15ccb Add member replace failpoint to robustness tests
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-10-17 11:17:49 +02:00
Marek Siarkowicz
0d83a72cf5 Split failpoints file
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-10-17 09:51:43 +02:00
Marek Siarkowicz
d6e376b6c6 Move failpoints to separate package
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-10-16 20:57:31 +02:00