18450 Commits

Author SHA1 Message Date
Tobias Grieger
36860f863f TestLeaderAcknowledgeCommit
Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
dad8208a4d raft: avoid panics during *node tests
`StartNode` runs a naked goroutine, so it's impossible to test against
it in a way that will reliably produce contained test failures when
assertions are hit on the `(*node).run` goroutine.

This commit introduces a harness that we can use in tests to wrap
this goroutine and allow it to defer errors to `*testing.T`.

Note that tests of `Node` still need to be architected carefully
since it's easy to produce a deadlock in them should things not
go exactly as planned.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
169f4c3cc7 raft: don't emit unstable CommittedEntries
See https://github.com/etcd-io/etcd/issues/14370.

When run in a single-voter configuration, prior to this PR
raft would emit `HardState`s that would emit a proposed `Entry`
simultaneously in `CommittedEntries` and `Entries`.

To be correct, this requires users of the raft library to enforce an
ordering between appending to the log and notifying the client about
`CommittedEntries` also present in `Entries`. This was easy to miss.

Walk back this behavior to arrive at a simpler contract: what's
emitted in `CommittedEntries` is truly committed, i.e. present
in stable storage on a quorum of voters.

This in turn pessimizes the single-voter case: rather than fully
handling an `Entry` in just one `Ready`, now two are required,
and in particular one has to do extra work to save on allocations.

We accept this as a good tradeoff, since raft primarily serves
multi-voter configurations.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
21be9fa337 raft: add single_node InteractionEnv test case
Show-cases the current behavior and changes made in future commits for [^1].

The test demonstrates that a single-voter raft instance will emit an
entry as committed while it still needs to be appended to the log.

[^1]: https://github.com/etcd-io/etcd/issues/14370

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
5e3314da42 raft: add BenchmarkRawNode
This is a speed-of-light benchmark that uses an in-memory single-voter
RawNode to sequentially propose and process entries.

As a bonus, it also measures the number of calls to the underlying
Storage. Calls to the Storage are cheap since the benchmark is in-
memory, but in a real-world implementation, especially one that doesn't
cache results, additional calls to the Storage interface can translate
to a heavy hit as they might involve additional I/O.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Tobias Grieger
3ad363d070 raft: always mark leader as RecentActive
RecentActive is now initialized to true in `becomeLeader`. Both
configuration changes and CheckQuorum make sure not to break this,
so we now now that the leader is always RecentActive.

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
2022-09-20 08:59:37 +02:00
Benjamin Wang
c113f93d79
Merge pull request #14482 from fuweid/testing-curl-maxstream
e2e: make maxstream test stable
2022-09-20 12:30:10 +08:00
Wei Fu
16884373b9 e2e: submitConcurrentWatch runs with ExecuteWithTimeout
Use testutils.ExecuteWithTimeout to make sure that the all active
streams are ready in time.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2022-09-20 11:03:51 +08:00
Wei Fu
ec847337d7 e2e: make maxstream test stable
submitConcurrentWatch use sleep 3s to wait for all the watch connections
ready. When the number of connections increases, like 1000, the 3s is not
enough and the test case becomes flaky.

In this commit, spawn curl process and check the ouput line with
`created":true}}` to make sure that the connection has been initialized
and ready to receive the events. It is reliable to test the following
range request.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2022-09-20 10:41:59 +08:00
Benjamin Wang
5344085338
Merge pull request #14491 from ahrtr/bump_jwt_4.4.2
etcd: Bump golang-jwt/jwt/ version to 4.4.2
2022-09-20 10:18:44 +08:00
Sahdev Zala
ad4a465383
Merge pull request #14492 from cmurphy/bump-golang-x-net
*: Update golang.org/x/net to latest
2022-09-19 22:11:43 -04:00
Colleen Murphy
7ea2a3d7cb *: Update golang.org/x/net to latest
Update golang.org/x/net to address CVE-2022-27664.

Signed-off-by: Colleen Murphy <colleen.murphy@suse.com>
2022-09-19 16:01:45 -07:00
Benjamin Wang
159ed15afc
Merge pull request #14479 from demoManito/fix/declaring-empty-slice
etcd: modify declaring empty slices
2022-09-20 05:22:59 +08:00
Benjamin Wang
09db6ec1d7 etcd: Bump golang-jwt/jwt/ version to 4.4.2
github.com/golang-jwt/jwt adds go mod support startig from 4.0.0,
and it's backwards-compatible with existing v3.x.y tags.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-20 04:06:47 +08:00
Benjamin Wang
f12374b7b5
Merge pull request #14483 from spzala/featuregraduation
Doc: Add feature graduation and deprecation policies
2022-09-20 03:09:45 +08:00
Sahdev Zala
9f27d459b0 Doc: Add feature graduation and deprecation policies
Related https://github.com/etcd-io/etcd/pull/14428

Signed-off-by: Sahdev Zala <spzala@us.ibm.com>
2022-09-19 11:13:14 -04:00
Hitoshi Mitake
c793f18238
Merge pull request #14322 from mitake/watch-auth-err
*: handle auth invalid token and old revision errors in watch
2022-09-19 23:33:49 +09:00
Marek Siarkowicz
e0647b9d42
Merge pull request #14481 from serathius/verify-fmt
makefile: Split fmt into separate verify commands
2022-09-19 12:38:30 +02:00
Sahdev Zala
8650ae83f7
Merge pull request #14485 from ahrtr/fix_build_tools
Build: remove the invalid quote character ' for flag -ldflags
2022-09-17 23:52:36 -04:00
Benjamin Wang
35bc65d3ec
Merge pull request #14484 from ahrtr/remove_go_version_from_glangci
CI: remove hardcode go version in .golangci.yaml
2022-09-18 06:42:14 +08:00
Sahdev Zala
5e61b1c59b
Merge pull request #14486 from ahrtr/fix_flaky_TestKVDelete
Test: increase the TestKVDelete's timeout to 15s
2022-09-17 18:28:36 -04:00
Benjamin Wang
d657f2f66b Test: increase the TestKVDelete's timeout to 15s
Sometimes it just needs more time to execute the test case,
so increase the timeout from 10s to 15s.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-18 04:26:30 +08:00
Benjamin Wang
536743042b Build: remove the invalid quote character ' for flag -ldflags
It isn't valid to start with quote character ' for flag -ldflags.
Go cmd older than 1.19 just ignores the error. Starting from go 1.19,
Go cmd will fail with error message something like below,
```
stderr: invalid value "'-X=go.etcd.io/etcd/api/v3/version.GitSHA=01250c9'" for flag -ldflags: parameter may not start with quote character '
```

Actually we don't have such quote character ' when building
etcd/etcdctl/etcdutl.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-18 04:19:40 +08:00
Benjamin Wang
4f907e950e CI: remove hardcode go version in .golangci.yaml
The default Go version used by golinter is coming from the go.mod
file, fallback on the env var `GOVERSION`. So no need to configure
the go version in .golangci.yaml.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-18 04:04:35 +08:00
Hitoshi Mitake
c5614520d5 tests: a test case for watch with auth token expiration
Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com>
2022-09-17 21:51:36 +09:00
Hitoshi Mitake
2dcfa83094 *: handle auth invalid token and old revision errors in watch
Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com>
2022-09-17 21:51:36 +09:00
Marek Siarkowicz
b8347edf3b makefile: Split fmt into separate verify commands
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-09-16 22:37:10 +02:00
Sahdev Zala
cdf4228673
Merge pull request #14308 from dusk125/main
server/etcdmain: add configurable cipher list to gRPC proxy listener
2022-09-16 12:15:08 -04:00
Sahdev Zala
72912fbc20
Merge pull request #14470 from serathius/verify-update
makefile: Make static analysis commands easier to use
2022-09-16 12:02:37 -04:00
Benjamin Wang
c8fef9b7bd
Merge pull request #14480 from ahrtr/clelanup_changelog_3.6
Changelog: cleanup 3.6 changelog to remove all bug fixes
2022-09-16 19:48:05 +08:00
Benjamin Wang
fe24de857a Changelog: cleanup 3.6 changelog to remove all bug fixes
The rules should be:
1. Each patch release should only include change against previous
   patch, such as 3.5.4 vs 3.5.5.
2. For the first release (e.g. 3.6.0) for each minor version, we
   should only describe new features or changes which are not
   included in any releases in previous minor version.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-16 19:18:42 +08:00
Marek Siarkowicz
fd0283d045
Merge pull request #14472 from serathius/changelog-3.5.5
Update changelog for v3.5.5 release
2022-09-16 12:38:05 +02:00
Marek Siarkowicz
f655f667f2 Update changelog for v3.5.5 release
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-09-16 12:04:54 +02:00
demoManito
5b26fc0101 fix test
Signed-off-by: demoManito <1430482733@qq.com>
2022-09-16 17:08:29 +08:00
Benjamin Wang
d6f76e1aa3
Merge pull request #14471 from demoManito/cleanup/remove-assert
pkg: replace the deprecated Expect with ExpectWithContext in pkg/expect/expect_test.go
2022-09-16 15:28:57 +08:00
demoManito
72cf0cc04a etcd: modify declaring empty slices
declare an empty slice to var s []int replace  s :=[]int{}, https://github.com/golang/go/wiki/CodeReviewComments#declaring-empty-slices

Signed-off-by: demoManito <1430482733@qq.com>
2022-09-16 14:41:14 +08:00
demoManito
a281bb0494 pkg: replace the deprecated Expect with ExpectWithContext in pkg/expect/expect_test.go
Signed-off-by: demoManito <1430482733@qq.com>
2022-09-16 14:39:14 +08:00
Benjamin Wang
b7ba0542f6
Merge pull request #14422 from kkkkun/remove-redundant-code
remove redundant log messsages
2022-09-16 12:18:43 +08:00
kkkkun
c4582aaaee remove redundant log messages
Signed-off-by: kkkkun <scuzk373x@gmail.com>
2022-09-16 11:45:43 +08:00
Sahdev Zala
2fa73023a0
Merge pull request #14428 from spzala/featuresguide
Add features dev and support guidelines doc
2022-09-15 10:22:08 -04:00
Marek Siarkowicz
9678697027
Merge pull request #14475 from serathius/fix-flake2
really fix measure-test-flakiness
2022-09-15 16:15:55 +02:00
Marek Siarkowicz
54bd105805 really fix measure-test-flakiness
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-09-15 16:15:25 +02:00
Marek Siarkowicz
f6ee4c3b60
Merge pull request #14473 from serathius/flake
Fix measuring test flakiness
2022-09-15 16:01:25 +02:00
Marek Siarkowicz
e270967ab9 Fix measuring test flakiness 2022-09-15 16:00:53 +02:00
Benjamin Wang
a7136933dd
Merge pull request #14469 from ahrtr/update_changelog_3.4
Changelog: correct the 3.4 changelog item
2022-09-15 18:37:05 +08:00
Marek Siarkowicz
7bb114d171 makefile: Make static analysis commands easier to use
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-09-15 11:54:14 +02:00
Benjamin Wang
102ec3ccd3 Changelog: correct the 3.4 changelog item
There are duplicated items for https://github.com/etcd-io/etcd/pull/14441.
Since it's a etcdctl side change, so we should keep the item under etcdctl,
and remove the duplicated item under "etcd server"

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-15 15:51:19 +08:00
Marek Siarkowicz
f7bae0da4e
Merge pull request #14468 from ahrtr/update_changelog_3.4.21
Update changelog for 3.4.21
2022-09-15 09:44:10 +02:00
Marek Siarkowicz
ceefbce140
Merge pull request #14464 from lovehhf/remove_unused_revisions
mvcc: Remove unused revisions and change comment rev to modified
2022-09-15 08:58:27 +02:00
Benjamin Wang
3e4715e3a3 Changelog: Update changelog for 3.4.21
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-15 09:31:25 +08:00