Siyuan Zhang
b54d7552a7
robustness: add mix version option in exploratoryScenarios.
...
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-05-21 16:57:53 +00:00
Siyuan Zhang
cde6cd006d
e2e: add flag to pass specific binary path for last release.
...
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-05-21 16:57:48 +00:00
Akshay Nanavare
8eb91d0e15
add error constants in validate pkg
...
Signed-off-by: Akshay Nanavare <nakshay303@gmail.com>
2024-05-14 20:19:08 +05:30
Marek Siarkowicz
d8bb19327b
Prevent picking a failpoint that waiting till snapshot that doesn't support lower snapshot catchup entries but allow reproducing issue #15271
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-05-13 12:08:42 +02:00
James Blair
2e70fc7d1d
Automate labels for tests pull requests.
...
Signed-off-by: James Blair <mail@jamesblair.net>
2024-05-12 22:24:21 +12:00
Marek Siarkowicz
dc35b6493f
Merge pull request #17966 from serathius/robustness-relax
...
Relax assumptions about all client request persisted in WAL to only require first and last request to be persisted
2024-05-09 17:51:27 +02:00
Marek Siarkowicz
e1244f19d6
Merge pull request #17918 from serathius/robustness-serializable-validation-test
...
Add tests to serializable operations validation
2024-05-09 11:07:49 +02:00
Marek Siarkowicz
b8ffc5e8c0
Merge pull request #17967 from serathius/robustness-update-readme
...
Update the robustness README and fix the #14370 reproduction case
2024-05-09 10:05:27 +02:00
Marek Siarkowicz
c4ff2c20bd
Merge pull request #17965 from serathius/makefile-cache
...
Fix caching by not depending on PHONY target in non-PHONY target
2024-05-08 16:47:45 +02:00
Marek Siarkowicz
f5c0e785a7
Fix caching by not depending on PHONY target in non-PHONY target
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-05-08 14:29:28 +02:00
Marek Siarkowicz
b883f839f1
Add tests to serializable operations validation
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-05-08 12:29:55 +02:00
Marek Siarkowicz
bb398a0e6c
Merge pull request #17889 from serathius/robustness-operations-failpoints
...
Robustness operations failpoints
2024-05-08 11:37:22 +02:00
Marek Siarkowicz
be9758e2bc
Update the robustness README and fix the #14370 reproduction case
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-05-08 11:31:28 +02:00
Marek Siarkowicz
7181c7532f
Relax assumptions about all client request persisted in WAL to only require first and last request to be persisted
...
This assumption is not true during durability issues like #14370 .
In reality we want to avoid situations where WAL is was truncated, for
that it's enough that we ensure that first and last operations are
present.
Found it when running `make test-robustness-issue14370` and instead of
getting `Model is not linearizable` I got that assumptions were broken.
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-05-08 10:40:38 +02:00
Siyuan Zhang
dd79332cf6
robustness: add 2 more log lines when persistClientReports
...
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-05-06 09:53:39 -07:00
Marek Siarkowicz
c4e3b61a1c
Record operation from failpoint injection
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-05-01 19:20:22 +02:00
Marek Siarkowicz
1e7dd97e3b
Add LeaseRevoke request to WAL parsing
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-27 12:04:27 +02:00
Marek Siarkowicz
b36d9b2156
Merge pull request #17731 from serathius/robustness-wal-validate-watch
...
Robustness wal validate watch
2024-04-26 08:37:33 +02:00
Marek Siarkowicz
2de719dea4
Use WAL persisted request to validate watch
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-25 21:11:37 +02:00
Marek Siarkowicz
9027014adb
Merge pull request #17827 from siyuanfoundation/flaky
...
robustness: Add option to not overwrite results dir.
2024-04-24 22:48:02 +02:00
Siyuan Zhang
3c3b76cea1
robustness: not overwrite results dir by giving each dir a unique name.
...
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-04-24 12:36:55 -07:00
Marek Siarkowicz
96d619459b
Merge pull request #17734 from MadhavJivrajani/toolchain-directive
...
tests: set GOTOOLCHAIN var for report validation
2024-04-24 09:25:07 +02:00
Madhav Jivrajani
856847d89b
tests: set GOTOOLCHAIN var for report validation
...
Set GOTOOLCHAIN directive in order to successfully run tests
from root. Else, go will try and download a family of releases
(of the form 1.x), which are not published binaries.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
2024-04-23 14:44:51 +05:30
Marek Siarkowicz
9fcde37447
Persist member data with lazyfs enabled
...
Discovered turning off LazyFS before creating the report might result in
empty server directory. This PR moves cluster shutdown to defer executed
after we generate report and copies the data from lazyfs directory.
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-23 10:19:05 +02:00
Marek Siarkowicz
062a0ea057
Merge pull request #17825 from serathius/robustness-qps
...
Don't require minimal for failpoint injection period
2024-04-22 19:03:42 +02:00
Marek Siarkowicz
fa9e9504ad
Handle watch responses with error
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-21 20:49:49 +02:00
Marek Siarkowicz
a097a3b39d
Merge pull request #17810 from serathius/robustness-revisions-between-progress
...
Validate revisions between progress notify
2024-04-21 20:04:25 +02:00
Marek Siarkowicz
f285330d46
Don't require minimal for failpoint injection period
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-20 10:34:51 +02:00
Marek Siarkowicz
964680c8d0
Validate delivery of events between progress notifies
...
Simplifying bookmarkable to just validate revision order between events
and progress notifies.
Use reliable to validate if events are missing, but still report
broken resumable if first event after revision is missing. It's easier
to have one place that validates event slices.
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-19 10:42:54 +02:00
Marek Siarkowicz
5a8c8b703b
Merge pull request #17807 from serathius/robustness-resumable-revision-zero
...
Resumable handles watch with revision zero
2024-04-16 19:41:53 +02:00
Marek Siarkowicz
dc187ce6e8
Validate bookmarkable checks the last event before progress notify
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-16 09:17:40 +02:00
Marek Siarkowicz
94a47a7cbd
Resumable handles watch with revision zero
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-15 20:23:51 +02:00
Marek Siarkowicz
042e7d1a0c
Add filter validation to ensure watch only includes events within selector
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-15 20:05:08 +02:00
Marek Siarkowicz
a95a307698
Add tests to watch validation
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-14 21:38:03 +02:00
Marek Siarkowicz
569693be8d
Utilize WAL to patch operation history
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-14 12:09:38 +02:00
Marek Siarkowicz
452445e2d8
Merge pull request #17781 from serathius/robustness-read-limit
...
Remove limit from read requests after a failed write
2024-04-14 12:05:23 +02:00
Marek Siarkowicz
2e6eebef85
Merge pull request #17759 from serathius/robustness-assumptions
...
Add explicit checks for assumptions in robustness test validation
2024-04-13 00:19:25 +02:00
Marek Siarkowicz
d0bf8ddca4
Improve description for Kubernetes CAS operations
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-12 16:18:31 +02:00
Marek Siarkowicz
cadfc407e9
Remove limit from read requests after a failed write
...
Limit can cause multiple request due to pagination.
For reads after a failed write we would like to return to normal write
request as soon as possible.
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-12 15:01:17 +02:00
Marek Siarkowicz
f8de338ab2
Add explicit checks for assumptions in robustness test validation
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-12 14:18:22 +02:00
Marek Siarkowicz
bfbfee0afa
Merge pull request #17768 from serathius/robustness-success-rate
...
[Robustness] Collect failed read operations to calculate request success rate
2024-04-12 09:46:20 +02:00
Marek Siarkowicz
718d5ba2b4
Calculate request success rate to provide signal to performance debugging
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-11 09:36:17 +02:00
Marek Siarkowicz
ae7f79fd63
Refactor append from appendFailed and appendSuccesfull
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-11 09:36:17 +02:00
Marek Siarkowicz
65130c6d21
Refactor merge succesfull and failed operation in history
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-10 21:11:46 +02:00
Marek Siarkowicz
229275d46e
Refactor appendSuccesful and appendFailed methods to match
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-10 10:33:19 +02:00
Marek Siarkowicz
41ac7e33a1
Don't cache test-robustness-reports
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-09 15:59:58 +02:00
Marek Siarkowicz
6cb4c3f90d
Document re-evaluating existing robustness test reports
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-08 16:58:12 +02:00
Marek Siarkowicz
3a23994fbf
Make no failpoint error more readable
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-07 15:13:59 +02:00
Marek Siarkowicz
e2bb8c698f
Limit a timeout in testing robustness validation
...
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-06 12:28:57 +02:00
Marek Siarkowicz
09769c4be7
Merge pull request #17642 from fuweid/fix-17506
...
*: LeaseTimeToLive returns error if leader changed
2024-04-02 14:55:41 +02:00