432 Commits

Author SHA1 Message Date
Geeta Gharpure
e5b7dde17e Add a method to export membership info to v2 store from RaftCluster
Signed-off-by: Geeta Gharpure <geetagh@amazon.com>
2023-07-28 16:55:41 +00:00
lan.tian
0f975acf2f
update typo in raft.go
Signed-off-by: lan.tian <lance5890@163.com>
2023-07-24 15:48:55 +08:00
iuriatan
abbfc2964a Fix goword issue
Fix `make verify` issues after updating golangci-lint

Signed-off-by: iuriatan <iuriatan@gmail.com>
2023-07-14 16:46:26 -03:00
Benjamin Wang
f4444e8fb3
Merge pull request #16154 from CaojiamingAlan/uber_applier_test
add tests for uber applier
2023-07-06 20:00:11 +01:00
Benjamin Wang
e887e5291a
Merge pull request #16067 from geetasg/pr1
Adding test for updateClusterVersionV3
2023-07-06 08:36:19 +01:00
caojiamingalan
eff9517a90 etcdserver: add cluster id check for hashKVHandler
Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com>
2023-07-05 14:09:40 -05:00
caojiamingalan
ffe73f9a15 add tests for uber applier
Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com>
2023-06-30 22:03:29 -05:00
Benjamin Wang
22f9dac7b1
Merge pull request #15708 from chaochn47/confchange_raft_node_notifies_apply
raft node notifies configure when confChanged
2023-06-28 10:03:50 +01:00
Benjamin Wang
bda68d8d06
Merge pull request #16074 from geetasg/pr2
Enable test to verify membership recovery from backend
2023-06-27 20:24:51 +01:00
Chao Chen
6cdc9ae4fe server/etcdserver/raft.go:
1. rename confChangeCh to raftAdvancedC
2. rename waitApply to confChanged
3. add comments and test assertion

Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-06-26 22:42:44 -07:00
Benjamin Wang
ad3b6ee4c6 etcdserver: wait for raft is notified on confChange before responding to client
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-06-26 13:40:51 -07:00
Marek Siarkowicz
1967b8e5e6
Merge pull request #16086 from CaojiamingAlan/applier_test
etcdserver: add tests for apply_auth.go
2023-06-26 12:20:05 +02:00
caojiamingalan
11aa59c42d etcdserver: add tests for apply_auth.go
Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com>
2023-06-23 10:58:30 -05:00
Benjamin Wang
c8247731a2
Merge pull request #16099 from chaochn47/enable_failpoint_in_integration_test
Enable failpoint in integration test
2023-06-23 13:15:08 +01:00
Chao Chen
6d79b86219 Enable failpoint by default in integration tests
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-06-21 23:13:46 -07:00
Geeta Gharpure
550aa152a7 Verify consistent index is latest at the time of snapshot
Signed-off-by: Geeta Gharpure <geetagh@amazon.com>
2023-06-19 16:00:04 +00:00
Marek Siarkowicz
cb3730a30f
Merge pull request #16005 from tjungblu/putauthshort
Early exit auth check on lease puts
2023-06-16 08:33:57 +02:00
Thomas Jungblut
84a9af17cc Add first unit test for authApplierV3
This contains a slight refactoring to expose enough information
to write meaningful tests for auth applier v3.

Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
2023-06-15 15:57:44 +02:00
Geeta Gharpure
e9fa3d30d7 Enable test to verify membership recovery from backend
Signed-off-by: Geeta Gharpure <geetagh@amazon.com>
2023-06-13 18:33:03 +00:00
Geeta Gharpure
2b81483103 Adding test for version update function used in 3.6
Signed-off-by: Geeta Gharpure <geetagh@amazon.com>
2023-06-13 17:38:04 +00:00
caojiamingalan
b9e30bf878 etcdserver: add e2e test to reproduce the incorrect hash issue when resuming scheduled compaction.
check ScheduledCompactKeyName and FinishedCompactKeyName
before writing hash to hashstore. If they do not match, then it means this compaction has once been interrupted and its hash value is invalid. In such cases, we won't write the hash values to the hashstore, and avoids the incorrect corruption alarm.

Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com>
2023-06-07 19:54:09 -05:00
Thomas Jungblut
dfbe2038f3 Early exit auth check on lease puts
Mitigates #15993 by not checking each key individually for permission
when auth is entirely disabled or admin user is calling the method.

Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
2023-06-06 10:23:46 +02:00
Benjamin Wang
3413f2e08d
Merge pull request #15908 from cuishuang/main
*: use strings.Builder instead of bytes.Buffer
2023-05-26 09:41:01 +08:00
Chao Chen
9e1e378e9e update code comments
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-05-24 12:54:27 -07:00
cui fliter
0c919dc212 use the more efficient strings.Builder
Signed-off-by: cui fliter <imcusg@gmail.com>
2023-05-19 10:44:58 +08:00
Benjamin Wang
05b663fbe8
Merge pull request #15828 from chaochn47/add_leadership_transfer_coverage
tests/e2e: add graceful shutdown test
2023-05-11 07:39:25 +08:00
Chao Chen
f31d0eafb9 tests/e2e: add graceful shutdown test
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-05-09 17:08:53 -07:00
Hitoshi Mitake
a0cffb6a1f server/etcdserver: swap order of return types
Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com>
2023-05-08 22:53:04 +09:00
Hitoshi Mitake
49b59cc8e5
Merge pull request #15656 from mitake/lease-timetolive-auth
protect LeaseTimeToLive with RBAC
2023-05-02 23:02:29 +09:00
Hitoshi Mitake
975854f07f etcdserver: protect lease timetilive with auth
Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com>
Co-authored-by: Benjamin Wang <wachao@vmware.com>
2023-04-26 20:35:20 +09:00
Wei Fu
1ba577e499 server/etcdserver: togRPCError for maintenance API
It's to deflake TestAuthMemberRemove.

When the client has multiple endpoints, the client might send a request
with valid token to the follower member which hasn't received token
replicated log yet. The member will reject the request.

For instance, the maintenance.Status API will return "auth: invalid auth
token". But the client doesn't identify the error. The client won't retry to
refresh auth token. The maintenance.Status should togRPCError before return
so that the client can reflesh token. It's align with existing API.

Since the maintenance client always creates one connection to target
member, the member will have the token after refresh auth.

Maybe we can introduce a sync to wait for member is ready with token,
instead of refreshing.

Fixes: #15758

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-04-22 18:35:53 +08:00
Marcondes Viana
9d14ae43c2 fix review: remove if on error check
Signed-off-by: Marcondes Viana <marju10@gmail.com>
2023-04-18 10:43:13 -03:00
Marcondes Viana
ecc7441ba1 fix review: use assert lib
Signed-off-by: Marcondes Viana <marju10@gmail.com>
2023-04-18 10:02:03 -03:00
Marcondes Viana
3654552fe8 fix review: move setup and vars
Signed-off-by: Marcondes Viana <marju10@gmail.com>
2023-04-17 08:19:10 -03:00
Marcondes Viana
7a7e09d3a5 fix review: group calls
Signed-off-by: Marcondes Viana <marju10@gmail.com>
2023-04-16 19:59:47 -03:00
Marcondes Viana
41b1d36bd9 fix review
Signed-off-by: Marcondes Viana <marju10@gmail.com>
2023-04-16 17:44:46 -03:00
Marcondes Viana
a1a2f43f2f fix review
Signed-off-by: Marcondes Viana <marju10@gmail.com>
2023-04-16 17:41:59 -03:00
Marcondes Viana
8fb839eac1 Update server/etcdserver/txn/txn_test.go
Signed-off-by: Marcondes Viana <marju10@gmail.com>
Co-authored-by: Marek Siarkowicz <marek.siarkowicz@protonmail.com>
2023-04-16 17:09:02 -03:00
Marcondes Viana
4c63611768 tests: cover txn.CheckTxnAuth logic with unit tests
Signed-off-by: Marcondes Viana <marju10@gmail.com>
2023-04-16 13:24:12 -03:00
Benjamin Wang
dae1d70189 test: workaround the breaking change in jonboulle/clockwork
See
- https://github.com/jonboulle/clockwork/pull/55
- https://github.com/jonboulle/clockwork/blob/v0.3.0/clockwork.go#L42

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-04-11 12:01:09 +08:00
Benjamin Wang
8b1cd036ff security: remove password after authenticating the user
fix https://nvd.nist.gov/vuln/detail/CVE-2021-28235

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-04-06 17:11:54 +08:00
Peter Wortmann
74feb229c7 etcdserver: Guarantee order of requested progress notifications
Progress notifications requested using ProgressRequest were sent
directly using the ctrlStream, which means that they could race
against watch responses in the watchStream.

This would especially happen when the stream was not synced - e.g. if
you requested a progress notification on a freshly created unsynced
watcher, the notification would typically arrive indicating a revision
for which not all watch responses had been sent.

This changes the behaviour so that v3rpc always goes through the watch
stream, using a new RequestProgressAll function that closely matches
the behaviour of the v3rpc code - i.e.

1. Generate a message with WatchId -1, indicating the revision for
   *all* watchers in the stream

2. Guarantee that a response is (eventually) sent

The latter might require us to defer the response until all watchers
are synced, which is likely as it should be. Note that we do *not*
guarantee that the number of progress notifications matches the number
of requests, only that eventually at least one gets sent.

Signed-off-by: Peter Wortmann <peter.wortmann@skao.int>
2023-04-05 11:54:10 +01:00
Chao Chen
caed563e08 fix flaking auth member remove test
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-04-03 17:41:08 -07:00
Wei Fu
22bdc91302 server/etcdserver: add log for terminating monitors
Adding log for terminating monitors is to make the debug easier.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-03-11 15:07:17 +08:00
James Blair
275e10bcf7
Return default snapshot count to 10,000.
The huge (100k+) value was justified when storev2 was being dumped completely with every snapshot.

With storev2 being decomissioned we can checkpoint more frequently for faster recovery.

Signed-off-by: James Blair <mail@jamesblair.net>
2023-03-06 20:21:03 +13:00
xakdwch
c767f429f0 rafthttp: replace inline code with existing function
The isMsgApp function implements the judgment of the MsgApp message,
use the isMsgApp function instead.

Signed-off-by: xakdwch <xakdwch5@gmail.com>
2023-03-03 09:50:14 +08:00
xin.li
b17b9c1428 chore: Use http constants to replace numbers as parameters
Signed-off-by: xin.li <xin.li@daocloud.io>
2023-02-20 11:53:41 +08:00
caojiamingalan
e2b8e86624 etcdserver: correct the old name of notifyc in comments
The old name(raftDone) of the channel(notifyc) which indicates the apply has been
completed is left unchanged in the comments, resulting in confusion when reading
the source code.

Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com>
2023-01-27 10:02:22 +08:00
Sasha Melentyev
0f4d7a7483 Cleanup
Signed-off-by: Sasha Melentyev <sasha@melentyev.io>
2023-01-21 15:57:48 +03:00
guozhao
de8d6b3792 etcdserver: use time.Ticker instead of time.After
Using time.After will create a new Timer in each cycle, In these cases
, it is better to use time.Ticker.

Signed-off-by: guozhao <guozhao@360.cn>
2023-01-17 16:58:13 +08:00