mirror of
https://github.com/etcd-io/etcd.git
synced 2024-09-27 06:25:44 +00:00
6 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Pavel Kalinnikov
|
0a0f0ae719 |
raft/rafttest: add test for replication pausing
This commit adds a data-driven test which simulates conditions under which Raft messages flow to a particular node is throttled while in StateReplicate. The test demonstrates that MsgApp messages with non-empty Entries may "leak" to a paused stream every time there is successful heartbeat exchange. Signed-off-by: Pavel Kalinnikov <pavel@cockroachlabs.com> |
||
|
Nathan VanBenschoten
|
b757e1bc87 |
raft: create new probe_and_replicate.txt interactive test
This commit creates a new probe_and_replicate.txt interactive test. The test creates a complete Raft log configuration and demonstrates how a leader probes and replicates to each of its followers. The log configuration constructed is identical to the one present in Figure 7 of the raft paper (https://raft.github.io/raft.pdf), which looks like: ``` 1 2 3 4 5 6 7 8 9 10 11 12 n1: [1][1][1][4][4][5][5][6][6][6] n2: [1][1][1][4][4][5][5][6][6] n3: [1][1][1][4] n4: [1][1][1][4][4][5][5][6][6][6][6] n5: [1][1][1][4][4][5][5][6][7][7][7][7] n6: [1][1][1][4][4][4][4] n7: [1][1][1][2][2][2][3][3][3][3][3] ``` Once in this state, we then elect node 1 as the leader and stabilize the entire raft group. This demonstrates how a newly elected leader probes for matching indexes, overwrites conflicting entries, and catches up all followers. This will be useful to demonstrate the impact of more efficient probing behavior. |
||
|
Piotr Tabor
|
e62417297d |
*: Rename of imports of raft (as its now a module)
% find -name '*.go' -o -name '*.md' -o -name '*.sh' | xargs sed -i --follow-symlinks 's|etcd/v3/raft|etcd/raft/v3|g' |
||
|
Piotr Tabor
|
3f449a8548 |
*: 'go test --short ./...' runs all UNIT tests.
Marked all 'integrational, e2e' as skipped in the --short mode. Thanks to this we will be able to significantly simplify ./test script. The run currently takes ~23s. With (follow up) move of ~clientv3/snapshot to integration tests (as part of modularization), we can expect this to fall to 5-10s. ``` % time go test --short ./... --count=1 ok go.etcd.io/etcd/v3 0.098s ? go.etcd.io/etcd/v3/Documentation/learning/lock/client [no test files] ? go.etcd.io/etcd/v3/Documentation/learning/lock/storage [no test files] ok go.etcd.io/etcd/v3/auth 0.724s ? go.etcd.io/etcd/v3/auth/authpb [no test files] ok go.etcd.io/etcd/v3/client 0.166s ok go.etcd.io/etcd/v3/client/integration 0.166s ok go.etcd.io/etcd/v3/clientv3 3.219s ok go.etcd.io/etcd/v3/clientv3/balancer 1.102s ? go.etcd.io/etcd/v3/clientv3/balancer/connectivity [no test files] ? go.etcd.io/etcd/v3/clientv3/balancer/picker [no test files] ? go.etcd.io/etcd/v3/clientv3/balancer/resolver/endpoint [no test files] ok go.etcd.io/etcd/v3/clientv3/clientv3util 0.096s [no tests to run] ok go.etcd.io/etcd/v3/clientv3/concurrency 3.323s ? go.etcd.io/etcd/v3/clientv3/credentials [no test files] ok go.etcd.io/etcd/v3/clientv3/integration 0.131s ? go.etcd.io/etcd/v3/clientv3/leasing [no test files] ? go.etcd.io/etcd/v3/clientv3/mirror [no test files] ok go.etcd.io/etcd/v3/clientv3/namespace 0.041s ok go.etcd.io/etcd/v3/clientv3/naming 0.115s ok go.etcd.io/etcd/v3/clientv3/ordering 0.121s ok go.etcd.io/etcd/v3/clientv3/snapshot 19.325s ok go.etcd.io/etcd/v3/clientv3/yaml 0.090s ok go.etcd.io/etcd/v3/contrib/raftexample 7.572s ? go.etcd.io/etcd/v3/contrib/recipes [no test files] ok go.etcd.io/etcd/v3/embed 0.282s ok go.etcd.io/etcd/v3/etcdctl 0.054s ? go.etcd.io/etcd/v3/etcdctl/ctlv2 [no test files] ok go.etcd.io/etcd/v3/etcdctl/ctlv2/command 0.117s ? go.etcd.io/etcd/v3/etcdctl/ctlv3 [no test files] ok go.etcd.io/etcd/v3/etcdctl/ctlv3/command 0.070s ok go.etcd.io/etcd/v3/etcdmain 0.172s ok go.etcd.io/etcd/v3/etcdserver 1.698s ? go.etcd.io/etcd/v3/etcdserver/api [no test files] ok go.etcd.io/etcd/v3/etcdserver/api/etcdhttp 0.075s ok go.etcd.io/etcd/v3/etcdserver/api/membership 0.104s ? go.etcd.io/etcd/v3/etcdserver/api/membership/membershippb [no test files] ok go.etcd.io/etcd/v3/etcdserver/api/rafthttp 0.181s ok go.etcd.io/etcd/v3/etcdserver/api/snap 0.078s ? go.etcd.io/etcd/v3/etcdserver/api/snap/snappb [no test files] ok go.etcd.io/etcd/v3/etcdserver/api/v2auth 0.142s ok go.etcd.io/etcd/v3/etcdserver/api/v2discovery 0.035s ok go.etcd.io/etcd/v3/etcdserver/api/v2error 0.043s ok go.etcd.io/etcd/v3/etcdserver/api/v2http 0.070s ok go.etcd.io/etcd/v3/etcdserver/api/v2http/httptypes 0.031s ? go.etcd.io/etcd/v3/etcdserver/api/v2stats [no test files] ok go.etcd.io/etcd/v3/etcdserver/api/v2store 0.645s ok go.etcd.io/etcd/v3/etcdserver/api/v2v3 0.218s ? go.etcd.io/etcd/v3/etcdserver/api/v3alarm [no test files] ? go.etcd.io/etcd/v3/etcdserver/api/v3client [no test files] ok go.etcd.io/etcd/v3/etcdserver/api/v3compactor 1.765s ? go.etcd.io/etcd/v3/etcdserver/api/v3election [no test files] ? go.etcd.io/etcd/v3/etcdserver/api/v3election/v3electionpb [no test files] ? go.etcd.io/etcd/v3/etcdserver/api/v3election/v3electionpb/gw [no test files] ? go.etcd.io/etcd/v3/etcdserver/api/v3lock [no test files] ? go.etcd.io/etcd/v3/etcdserver/api/v3lock/v3lockpb [no test files] ? go.etcd.io/etcd/v3/etcdserver/api/v3lock/v3lockpb/gw [no test files] ok go.etcd.io/etcd/v3/etcdserver/api/v3rpc 0.091s ok go.etcd.io/etcd/v3/etcdserver/api/v3rpc/rpctypes 0.012s ok go.etcd.io/etcd/v3/etcdserver/cindex 0.054s ok go.etcd.io/etcd/v3/etcdserver/etcdserverpb 0.039s ? go.etcd.io/etcd/v3/etcdserver/etcdserverpb/gw [no test files] ok go.etcd.io/etcd/v3/functional/agent 0.094s ? go.etcd.io/etcd/v3/functional/cmd/etcd-agent [no test files] ? go.etcd.io/etcd/v3/functional/cmd/etcd-proxy [no test files] ? go.etcd.io/etcd/v3/functional/cmd/etcd-runner [no test files] ? go.etcd.io/etcd/v3/functional/cmd/etcd-tester [no test files] ok go.etcd.io/etcd/v3/functional/rpcpb 0.060s ? go.etcd.io/etcd/v3/functional/runner [no test files] ok go.etcd.io/etcd/v3/functional/tester 0.079s ok go.etcd.io/etcd/v3/integration 0.684s ok go.etcd.io/etcd/v3/integration/embed 0.101s ok go.etcd.io/etcd/v3/lease 3.455s ok go.etcd.io/etcd/v3/lease/leasehttp 2.185s ? go.etcd.io/etcd/v3/lease/leasepb [no test files] ok go.etcd.io/etcd/v3/mvcc 7.246s ok go.etcd.io/etcd/v3/mvcc/backend 0.354s ? go.etcd.io/etcd/v3/mvcc/mvccpb [no test files] ok go.etcd.io/etcd/v3/pkg/adt 0.025s ? go.etcd.io/etcd/v3/pkg/contention [no test files] ? go.etcd.io/etcd/v3/pkg/cpuutil [no test files] ok go.etcd.io/etcd/v3/pkg/crc 0.008s ? go.etcd.io/etcd/v3/pkg/debugutil [no test files] ok go.etcd.io/etcd/v3/pkg/expect 0.015s ok go.etcd.io/etcd/v3/pkg/fileutil 0.268s ok go.etcd.io/etcd/v3/pkg/flags 0.021s ok go.etcd.io/etcd/v3/pkg/httputil 0.020s ok go.etcd.io/etcd/v3/pkg/idutil 0.008s ok go.etcd.io/etcd/v3/pkg/ioutil 0.025s ok go.etcd.io/etcd/v3/pkg/logutil 0.047s ? go.etcd.io/etcd/v3/pkg/mock/mockserver [no test files] ? go.etcd.io/etcd/v3/pkg/mock/mockstorage [no test files] ? go.etcd.io/etcd/v3/pkg/mock/mockstore [no test files] ? go.etcd.io/etcd/v3/pkg/mock/mockwait [no test files] ok go.etcd.io/etcd/v3/pkg/netutil 1.024s ok go.etcd.io/etcd/v3/pkg/osutil 0.021s ok go.etcd.io/etcd/v3/pkg/pathutil 0.008s ok go.etcd.io/etcd/v3/pkg/pbutil 0.008s ok go.etcd.io/etcd/v3/pkg/proxy 4.081s ok go.etcd.io/etcd/v3/pkg/report 0.008s ? go.etcd.io/etcd/v3/pkg/runtime [no test files] ok go.etcd.io/etcd/v3/pkg/schedule 0.009s ok go.etcd.io/etcd/v3/pkg/srv 0.019s ok go.etcd.io/etcd/v3/pkg/stringutil 0.008s ? go.etcd.io/etcd/v3/pkg/systemd [no test files] ok go.etcd.io/etcd/v3/pkg/testutil 0.023s ok go.etcd.io/etcd/v3/pkg/tlsutil 3.965s ok go.etcd.io/etcd/v3/pkg/traceutil 0.034s ok go.etcd.io/etcd/v3/pkg/transport 0.532s ok go.etcd.io/etcd/v3/pkg/types 0.028s ok go.etcd.io/etcd/v3/pkg/wait 0.023s ok go.etcd.io/etcd/v3/proxy/grpcproxy 0.101s ? go.etcd.io/etcd/v3/proxy/grpcproxy/adapter [no test files] ? go.etcd.io/etcd/v3/proxy/grpcproxy/cache [no test files] ok go.etcd.io/etcd/v3/proxy/httpproxy 0.044s ok go.etcd.io/etcd/v3/proxy/tcpproxy 0.047s ok go.etcd.io/etcd/v3/raft 0.312s ok go.etcd.io/etcd/v3/raft/confchange 0.183s ok go.etcd.io/etcd/v3/raft/quorum 0.316s ok go.etcd.io/etcd/v3/raft/raftpb 0.024s ok go.etcd.io/etcd/v3/raft/rafttest 0.640s ok go.etcd.io/etcd/v3/raft/tracker 0.026s ok go.etcd.io/etcd/v3/tests/e2e 0.077s ? go.etcd.io/etcd/v3/tools/benchmark [no test files] ? go.etcd.io/etcd/v3/tools/benchmark/cmd [no test files] ? go.etcd.io/etcd/v3/tools/etcd-dump-db [no test files] ok go.etcd.io/etcd/v3/tools/etcd-dump-logs 0.088s ? go.etcd.io/etcd/v3/tools/etcd-dump-metrics [no test files] ? go.etcd.io/etcd/v3/tools/local-tester/bridge [no test files] ? go.etcd.io/etcd/v3/version [no test files] ok go.etcd.io/etcd/v3/wal 1.517s ? go.etcd.io/etcd/v3/wal/walpb [no test files] go test --short ./... --count=1 76.12s user 12.57s system 375% cpu 23.635 total ``` |
||
|
Brandon Philips
|
96cce208c2 |
go.mod: use go.etcd.io/etcd/v3 versioning
This change makes the etcd package compatible with the existing Go ecosystem for module versioning. Used this tool to update package imports: https://github.com/KSubedi/gomove |
||
|
Tobias Schottdorf
|
e8090e57a2 |
raft/rafttest: introduce datadriven testing
It has often been tedious to test the interactions between multi-member Raft groups, especially when many steps were required to reach a certain scenario. Often, this boilerplate was as boring as it is hard to write and hard to maintain, making it attractive to resort to shortcuts whenever possible, which in turn tended to undercut how meaningful and maintainable the tests ended up being - that is, if the tests were even written, which sometimes they weren't. This change introduces a datadriven framework specifically for testing deterministically the interaction between multiple members of a raft group with the goal of reducing the friction for writing these tests to near zero. In the near term, this will be used to add thorough testing for joint consensus (which is already available today, but wildly undertested), but just converting an existing test into this framework has shown that the concise representation and built-in inspection of log messages highlights unexpected behavior much more readily than the previous unit tests did (the test in question is `snapshot_succeed_via_app_resp`; the reader is invited to compare the old and new version of it). The main building block is `InteractionEnv`, which holds on to the state of the whole system and exposes various relevant methods for manipulating it, including but not limited to adding nodes, delivering and dropping messages, and proposing configuration changes. All of this is extensible so that in the future I hope to use it to explore the phenomena discussed in https://github.com/etcd-io/etcd/issues/7625#issuecomment-488798263 which requires injecting appropriate "crash points" in the Ready handling loop. Discussions of the "what if X happened in state Y" can quickly be made concrete by "scripting up an interaction test". Additionally, this framework is intentionally not kept internal to the raft package.. Though this is in its infancy, a goal is that it should be possible for a suite of interaction tests to allow applications to validate that their Storage implementation behaves accordingly, simply by running a raft-provided interaction suite against their Storage. |