1098 Commits

Author SHA1 Message Date
Tobias Grieger
67c3522893 raft: directly update leader in advance
This makes the alternative option of implementing the leader's self-ack
of entry append the default.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 09:01:42 +02:00
Tobias Grieger
894e5cb685 move ctx param to the front
to appease linter

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 09:01:42 +02:00
Tobias Grieger
f62b9d5e19 remove TestNodeReadIndex
This is tested directly at the level of `RawNode` in
`TestRawNodeReadIndex`. `*node` is a thin wrapper around `RawNode` so
this is sufficient.

The reason to remove the test is that it now incurs data races
since it's not possible to adjust the `readStates` and `step`
fields while the node is running, and there is no primitive
to synchronize with its goroutine. This could all be fixed
but it's not worth it.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 09:01:40 +02:00
Tobias Grieger
f7dcb9ec2a TestInteraction
Reviewed the diff in detail.
The changes here were benign, just the extra raft cycle.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
f7b0a6ad33 TestRawNodeBoundedLogGrowthWithPartition
Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
02efe5135d TestRawNodeStart
Now also sees the extra Ready cycle.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
79bf3b0df4 TestRawNodeJointAutoLeave
This now needs an additional Ready cycle to apply the previous conf change,
so the finalizing conf change does too.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
fbe4d40086 TestLeaderTransferIgnoreProposal
Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
182e1a371d TestReadOnlyWithLearner
Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
b462fd15c2 TestMsgAppRespWaitReset
Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
ff837f3a0b TestProposal
Don't check on `committed` but `lastIndex` instead.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
15abe294e7 TestDueling{Pre,}Candidates
Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
d6f3e88a52 TestSingleNodeCommit
Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
99adcaa299 TestLearnerLogReplication
Needed to `(*raft).advance` on `n1` so that it would actually commit
the entry.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
7060d75527 TestLeaderOnlyCommitsLogFromCurrentTerm
Leader only acks to itself on `(*raft).advance` so we have to
make this test a bit more like the real thing.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
9ff144ef75 TestProgressLeader
This was expecting the progress of the leader to be updated as a
result of MsgProp but it is now happening in `(*raft).advance`.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
bd46776f03 Commit + apply all in nextEnts
This fixes essentially all tests using this, since now they don't have
to do anything special about the extra cycle introduced for single
nodes.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
f10579d3b5 TestLeaderAcknowledgeCommit
This needed to call `(*raft).advance` so that the leader would
self-ack the entries.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
b2dba1c86c TestNodeAdvance
Switched this to baking the conf changes into the initial state
to have fewer cycles to walk through in the test.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
0d9a6061c3 TestNodeReadIndex
Needs to ignore the injected MsgAppResp.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
14a76d755f TestNodeStart
This now sees the extra append-then-commit cycle.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
873cdf3fa6 TestNodeProposeWaitDropped
The test just needs to ignore the MsgAppResp.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
1a81b27bed TestNodePropose{,Config}
This test now observes the `MsgAppResp` injected in `(*raft).advance`.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
87a9b80d7b TestNodeProposeAddDuplicateNode
This needed to apply entries from CommittedEntries, not Entries.
Previously the test got away with it because the two slices were
equal. Now it was hanging because when it proposed the second
conf change the first one hadn't applied yet, and so it got dropped,
and the test would hang.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
931fec3b6d TestCommitPagination
Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
1f39a8fe79 raft: teach readyWithTimeout to log received Readys
Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
36860f863f TestLeaderAcknowledgeCommit
Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
dad8208a4d raft: avoid panics during *node tests
`StartNode` runs a naked goroutine, so it's impossible to test against
it in a way that will reliably produce contained test failures when
assertions are hit on the `(*node).run` goroutine.

This commit introduces a harness that we can use in tests to wrap
this goroutine and allow it to defer errors to `*testing.T`.

Note that tests of `Node` still need to be architected carefully
since it's easy to produce a deadlock in them should things not
go exactly as planned.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
169f4c3cc7 raft: don't emit unstable CommittedEntries
See https://github.com/etcd-io/etcd/issues/14370.

When run in a single-voter configuration, prior to this PR
raft would emit `HardState`s that would emit a proposed `Entry`
simultaneously in `CommittedEntries` and `Entries`.

To be correct, this requires users of the raft library to enforce an
ordering between appending to the log and notifying the client about
`CommittedEntries` also present in `Entries`. This was easy to miss.

Walk back this behavior to arrive at a simpler contract: what's
emitted in `CommittedEntries` is truly committed, i.e. present
in stable storage on a quorum of voters.

This in turn pessimizes the single-voter case: rather than fully
handling an `Entry` in just one `Ready`, now two are required,
and in particular one has to do extra work to save on allocations.

We accept this as a good tradeoff, since raft primarily serves
multi-voter configurations.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
21be9fa337 raft: add single_node InteractionEnv test case
Show-cases the current behavior and changes made in future commits for [^1].

The test demonstrates that a single-voter raft instance will emit an
entry as committed while it still needs to be appended to the log.

[^1]: https://github.com/etcd-io/etcd/issues/14370

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
5e3314da42 raft: add BenchmarkRawNode
This is a speed-of-light benchmark that uses an in-memory single-voter
RawNode to sequentially propose and process entries.

As a bonus, it also measures the number of calls to the underlying
Storage. Calls to the Storage are cheap since the benchmark is in-
memory, but in a real-world implementation, especially one that doesn't
cache results, additional calls to the Storage interface can translate
to a heavy hit as they might involve additional I/O.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
3ad363d070 raft: always mark leader as RecentActive
RecentActive is now initialized to true in `becomeLeader`. Both
configuration changes and CheckQuorum make sure not to break this,
so we now now that the leader is always RecentActive.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
demoManito
72cf0cc04a etcd: modify declaring empty slices
declare an empty slice to var s []int replace  s :=[]int{}, https://github.com/golang/go/wiki/CodeReviewComments#declaring-empty-slices

Signed-off-by: demoManito <1430482733@qq.com>
2022-09-16 14:41:14 +08:00
Abirdcfly
858afba44e
chore: remove duplicate word in comments
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-08-30 20:00:26 +08:00
Piotr Tabor
77773c7798
Merge pull request #14374 from Comolli/main
Reduce redundant code and memory allocation.
2022-08-29 21:15:06 +02:00
Abirdcfly
08a9d1da07
chore: remove duplicate word in comments
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-08-27 13:39:48 +08:00
comolli
a5515171c3 Reduce redundant code and memory allocation.
Signed-off-by: comolli <liangweicheng2@gmail.com>
2022-08-24 09:15:34 +08:00
Marek Siarkowicz
5fab045242
Merge pull request #14273 from RobertIndie/fix-test
tests: fix the logic of testNonleaderElectionTimeoutRandomized in raft_paper_test.go
2022-07-26 09:22:57 +02:00
Zike Yang
b77f56ac25 tests: fix the logic of testNonleaderElectionTimeoutRandomized in raft_paper_test.go
In the test `testNonleaderElectionTimeoutRandomized`, the possible values of election timeout randomized out should be 10-19(inclusive), but the current test only tests the possible values as 11-19(inclusive), which is incorrect here. We need to expand the scope of the tests to ensure robustness, and also to make the logic clearer.

Signed-off-by: Zike Yang <zike@apache.org>
2022-07-25 23:16:02 +08:00
Abirdcfly
37b920e9f4 delete unreachable code
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-07-21 12:54:29 +08:00
Piotr Tabor
e7a84b69c8 Update zap to 1.21.
Signed-off-by: Piotr Tabor <ptab@google.com>
2022-06-17 10:05:25 +02:00
Piotr Tabor
17c1dcd614 Update grpc to 1.47 and go-cmp to 0.5.8.
Signed-off-by: Piotr Tabor <ptab@google.com>
2022-06-17 08:57:22 +02:00
ahrtr
0dae4b3b1e rollback the opentelemetry bumpping to recover the pipeline failures 2022-04-14 16:13:28 +08:00
Will Beason
eab1e0c5d5 go.mod: upgrade opentelemetry deps
Downstream users of etcd experience build issues when using dependencies
which require more recent (incompatible) versions of opentelemetry. This
commit upgrades the dependencies so that downstream users stop
experiencing these issues.
2022-04-13 07:14:10 -07:00
howz97
f9c9bfa44c fix comment in raft.go 2022-04-02 14:27:33 +08:00
Marek Siarkowicz
fb55910500 version: bump up to 3.6.0-alpha.0 2022-02-28 13:20:27 +01:00
Piotr Tabor
271daa204d
Merge pull request #13614 from ptabor/20220217-update-yaml
Update dep: gopkg.in/yaml.v2 v2.2.8 -> v2.4.0 due to: CVE-2019-11254
2022-01-17 20:07:33 +01:00
Piotr Tabor
fdd98477ef Update dep: require gopkg.in/yaml.v2 v2.2.8 -> v2.4.0 due to: CVE-2019-11254 2022-01-17 16:52:58 +01:00
Piotr Tabor
22f142a9f5
Merge pull request #13603 from AdamKorcz/fuzz3
raft: fix out-of-bounds in maybeAppend
2022-01-17 16:11:46 +01:00
AdamKorcz
7ef4fe3288 raft: fix out-of-bounds in maybeAppend 2022-01-17 10:01:46 +00:00