113 Commits

Author SHA1 Message Date
fanmin shi
ab2b58a80f functional-tester: add lease stresser
Add lease stresser to test lease code under stress and etcd failures

resolve #6380
2016-10-18 14:20:26 -07:00
fanmin shi
ed75d93625 Merge pull request #6666 from fanminshi/function-tester-refractor
functional-tester: move checker logic to cluster
2016-10-18 11:44:37 -07:00
fanmin shi
7d86d1050e functional-tester: move checker logic to cluster
I move the checker logic from tester to cluster so that stressers and checkers can be initialized at the same time.
this is useful because some checker depends on stressers.
2016-10-18 11:17:40 -07:00
Hitoshi Mitake
b2b03d9926 functional-tester: a new option -failure-wrapper for enabling/disabling external fault injector
This commit adds a new option -failure-wrapper to etcd-tester. The
option receives a path of script that is used for enabling/disabling
external fault injectors. The script is called with an option "enable"
when it needs to be enabled (when failure.Inject() is called) and
called with "disabled" in an opposite case (when failure.Recover() is
called).
2016-10-14 11:31:28 +09:00
Hitoshi Mitake
7d48855630 functional-tester: decouple failures from tester
This commit adds a new option --failures to etcd-tester. The option
receives a comma-delimited argument like this:
"default,failpoints". The given arguments are interpreted as names of
failures and they are injected to an etcd cluster. Available failures
are default (default scenario in etcd-tester) and failpoints. If no
args are passed to the option (--failures=""), no failures are
injected during testing.
2016-09-28 11:30:53 +09:00
Hitoshi Mitake
0e493c11c2 functional-tester: decouple stresser from tester
This commit decouples stresser from the tester of
functional-tester. For doing it, this commit adds a new option
--stresser to etcd-tester. The option accepts two types of stresser:
"default" and "nop". If the option is "default", etcd-tester stresses
its etcd cluster with the existing stresser. If the option is "nop",
etcd-tester does nothing for stressing.

Partially fixes https://github.com/coreos/etcd/issues/6446
2016-09-24 01:04:57 +09:00
Anthony Romano
d4eff5381c etcd-tester: split out consistency checking code from tester 2016-09-20 10:26:58 -07:00
fanmin shi
8a63071463 etcd-tester: fix peer-port parsing bug with localhost url
The following format "http://localhost:1234" causes existing port parser to fail. Add new logic to parse the host name first then extract port.

Fixes #6409
2016-09-12 13:29:52 -07:00
Anthony Romano
55ba3d95fb etcd-tester: support per-agent client/peer/failpoint ports 2016-09-08 16:15:18 -07:00
Gyu-Ho Lee
0bf2ef3c1b etcd-tester: return error for mismatch rev/hash 2016-08-27 22:14:42 -07:00
Xiang Li
017aac88a8 etcd-tester: do not restart stresser on error 2016-08-27 20:47:45 -07:00
Gyu-Ho Lee
0c37df7265 etcd-tester: fix compact rev counting 2016-08-22 22:58:44 -07:00
Anthony Romano
47b6449934 functional-tester: put large keys
For testing writes that must span multiple pages.
2016-08-17 09:51:44 -07:00
Gyu-Ho Lee
c6cf015e26 etcd-tester: fix tester for 5-node cluster
1. fix failure case counting
2. match ErrClientConnClosing in stresser
3. longer timeout for set-health-key
4. fixed range for range/delete stresser
5. remove Limit in RangeRequest
2016-08-07 21:15:01 -07:00
Anthony Romano
978ae9de29 functional-tester: share limiter among stresser
Otherwise, adding more members stresses the cluster with more ops.
2016-08-07 19:15:00 -07:00
Xiang Li
7678b84f2c Merge pull request #6123 from xiang90/fix_limiter
tools/functional-tester: fix limiter
2016-08-07 16:20:17 -07:00
Xiang Li
f6a1585902 functional-tester: reduce rate to 3000 2016-08-07 14:34:01 -07:00
Xiang Li
107a07563f tools/functional-tester: fix limiter 2016-08-07 14:28:16 -07:00
Xiang Li
69204397ee tools/functional-tester: better logging 2016-08-07 14:21:44 -07:00
Gyu-Ho Lee
854a215329 etcd-tester: delete/range with limit, clean up 2016-08-05 11:21:36 -07:00
Gyu-Ho Lee
755eee0d30 etcd-tester: remove unnecessary arg from stresser 2016-08-01 15:35:31 -07:00
Gyu-Ho Lee
969bcd282b etcd-tester: stress with range, delete 2016-07-18 15:17:08 -07:00
Anthony Romano
0419d3ecf7 etcd-tester: add FailFast(false) to grpc calls 2016-07-14 19:16:41 -07:00
Gyu-Ho Lee
ceb9fe4822 etcd-tester: stop stress before compact, fix races
fix race condition between stresser cancel, start
2016-07-14 12:16:42 -07:00
Gyu-Ho Lee
8e59fb749c etcd-tester: increase default qps, fix cleanup 2016-07-14 11:20:16 -07:00
Gyu-Ho Lee
7c39f41e7c etcd-tester: add rate limiter to stresser 2016-07-07 21:55:12 -07:00
Anthony Romano
a5f043c85b etcd-tester: add failpoint cases
Fixes #5754
2016-07-01 15:31:49 -07:00
Gyu-Ho Lee
0d44947c11 etcd-tester: fix slow leader with injectLatency 2016-06-30 13:41:27 -07:00
Gyu-Ho Lee
dc49016987 etcd-tester: handle error in RevHash 2016-06-29 19:31:45 -07:00
Anthony Romano
5f459a64ce etcd-tester: refactor cluster member handling 2016-06-29 11:25:33 -07:00
Anthony Romano
402df5bd03 etcd-tester: refactor failure code to reduce code duplication 2016-06-29 11:03:34 -07:00
Anthony Romano
63f78bf7c8 etcd-tester: refactor round loop 2016-06-29 11:03:34 -07:00
Gyu-Ho Lee
15f2fd0726 etcd-tester: match ErrTimeout in stresser
Fix https://github.com/coreos/etcd/issues/5804.
2016-06-28 19:20:28 -07:00
Gyu-Ho Lee
ac68f70843 etcd-tester: match ErrTimeoutDueToLeaderFail
stresser in followers should retry when failure is injected to
their leader.
2016-06-27 20:48:06 -07:00
Gyu-Ho Lee
a1c7a7df5e *: use capnslog for grpclog 2016-06-20 20:35:03 -07:00
Anthony Romano
9feb3d0e51 etcd-tester: fix goword warnings 2016-06-17 13:37:35 -07:00
Gyu-Ho Lee
84953365a2 etcd-tester: cleanup in compact error, log level 2016-06-13 14:54:53 -07:00
Gyu-Ho Lee
5631acdb8f etcd-tester: do not exit for compact timeout
Temporary fix for https://github.com/coreos/etcd/issues/5606.
2016-06-10 09:44:45 -07:00
Gyu-Ho Lee
350673f1f8 etcd-tester: retry for 'etcdserver: not capable'
Fix https://github.com/coreos/etcd/issues/5573.

Currently stresser starts at the same time as cluster start.
If the stresser got launched too fast/early, all stressers
exit from the error 'etcdserver: not capable', which
means the cluster is not ready yet. This adds additional
error checking, so stresser can retry.
2016-06-07 21:56:04 -07:00
Gyu-Ho Lee
20fc3e968f Merge pull request #5465 from gyuho/compact1
etcd-tester: log more for compact errors
2016-05-27 16:16:04 -07:00
Anthony Romano
fc7da09d67 *: add missing godoc package descriptions
Fixes #4074
2016-05-27 15:15:26 -07:00
Gyu-Ho Lee
04039eb006 etcd-tester: more logs for compact operations 2016-05-27 09:55:13 -07:00
Gyu-Ho Lee
3ed5d28e2e etcd-tester: fix, clean up multiple things (#5462)
* etcd-tester: more logging, fix typo

* etcd-tester: fix prevCompactRev scope

Fix https://github.com/coreos/etcd/issues/5440.

* etcd-tester: move utils to bottom, clean up logs

And remove stresser operation inside defrag

* etcd-tester: separate update revision call

* etcd-tester: fix cleanup when case is -1
2016-05-26 11:37:49 -07:00
Gyu-Ho Lee
4c7af825c7 etcd-tester: timeout per number of compact entries
Fix https://github.com/coreos/etcd/issues/5440.
2016-05-25 18:37:13 -07:00
Gyu-Ho Lee
3d523e34b1 tools: update LICENSE header 2016-05-12 20:50:17 -07:00
Gyu-Ho Lee
cc7dd9b729 etcd-tester: refactor 2016-05-05 10:55:42 -07:00
Gyu-Ho Lee
de7c18909f etcd-tester: match more grpc errors
To prevent stressers from returning from failure injections
2016-04-27 09:34:05 -07:00
Gyu-Ho Lee
151d0d3831 etcd-tester: flag consistency-check 2016-04-22 22:22:12 -07:00
Gyu-Ho Lee
6a0664d701 etcd-tester: wait more for slow network recovery
For https://github.com/coreos/etcd/issues/5121.
2016-04-22 15:24:47 -07:00
Gyu-Ho Lee
0feb88cee1 etcd-tester: change var success->failed
Previous success overwrites the later failure.
Make it simpler by changing the variable to 'failed'.
2016-04-22 09:27:37 -07:00