Mirroristas/etcd
2
0
Fork 0
mirror of https://github.com/etcd-io/etcd.git synced 2024-09-27 06:25:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
20,994 Commits 15 Branches 491 Tags 148 MiB
Commit Graph

519 Commits

Author SHA1 Message Date
Benjamin Wang
8b9909e20d
Merge pull request #17585 from ivanvc/address-api-var-naming-lint-rule 
api: address golangci var-naming issues
 2024-03-16 11:23:46 +00:00
Ivan Valdes
2190b0f1c6
api: address golangci var-naming issues 
Signed-off-by: Ivan Valdes <ivan@vald.es>
 2024-03-15 20:37:53 -07:00
Chao Chen
405862e807 Fix event loss after compaction 
Signed-off-by: Chao Chen <chaochn@amazon.com>
 2024-03-15 14:22:37 -07:00
Marek Siarkowicz
6103504d4b Fix progress notification for watch that doesn't get any events 
When implementing the fix for progress notifications
(https://github.com/etcd-io/etcd/pull/15237) we made a incorrect
assumption that that unsynched watches will always get at least one event.

Unsynched watches include not only slow watchers, but also newly created
watches that requested current or older revision. In case that non of the events
match watch filter, those newly created watches might become synched
without any event going through.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2024-03-11 12:11:09 +01:00
Dat Tien Nguyen
46d59a2e68
etcdserver: skip when detect a removed peer 
Signed-off-by: Dat Tien Nguyen <phongtomfapp@gmail.com>
 2024-03-04 21:22:19 +07:00
Siyuan Zhang
3565a822de Add VerifyTxConsistency to backend. 
Signed-off-by: Siyuan Zhang <sizhang@google.com>

Update server/storage/backend/verify.go

Co-authored-by: Benjamin Wang <benjamin.wang@broadcom.com>

Update server/storage/backend/verify.go

Co-authored-by: Benjamin Wang <benjamin.wang@broadcom.com>
 2024-02-22 11:31:16 -08:00
Wei Fu
32ee8b877a etcdserver: drain leaky goroutines before test completed 
If pending changes aren't committed before test completed, it might cause
data race when we don't drain all the background goroutines.

```bash
$ cd server
$ go test -race -v -run TestApplyRepeat ./etcdserver
...
panic: Log in goroutine after TestApplyRepeat has completed: 2024-02-03T17:06:13.262+0800       DEBUG   bbolt   Committing transaction 2

goroutine 81 [running]:
testing.(*common).logDepth(0xc000502820, {0xc0001b0460, 0x41}, 0x3)
        /usr/local/go/src/testing/testing.go:1022 +0x6d4
testing.(*common).log(...)
        /usr/local/go/src/testing/testing.go:1004
testing.(*common).Logf(0xc000502820, {0x1421ad7, 0x2}, {0xc000603520, 0x1, 0x1})
        /usr/local/go/src/testing/testing.go:1055 +0xa5
go.uber.org/zap/zaptest.testingWriter.Write({{0x15f1f90?, 0xc000502820?}, 0xda?}, {0xc000119800, 0x42, 0x400})
        /home/fuwei/go/pkg/mod/go.uber.org/zap@v1.26.0/zaptest/logger.go:130 +0x11e
go.uber.org/zap/zapcore.(*ioCore).Write(0xc0000b55c0, {0xff, {0xc1679e614f9fd7a4, 0x73a3657, 0x1cc2400}, {0x1422b2d, 0x5}, {0xc0001a0330, 0x18}, {0x0, ...}, ...}, ...)
        /home/fuwei/go/pkg/mod/go.uber.org/zap@v1.26.0/zapcore/core.go:99 +0x193
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc000115930, {0x0, 0x0, 0x0})
        /home/fuwei/go/pkg/mod/go.uber.org/zap@v1.26.0/zapcore/entry.go:253 +0x2f0
go.uber.org/zap.(*SugaredLogger).log(0xc0001960f8, 0xff, {0x1437885, 0x19}, {0xc0006034e0, 0x1, 0x1}, {0x0, 0x0, 0x0})
        /home/fuwei/go/pkg/mod/go.uber.org/zap@v1.26.0/sugar.go:316 +0x130
go.uber.org/zap.(*SugaredLogger).Debugf(...)
        /home/fuwei/go/pkg/mod/go.uber.org/zap@v1.26.0/sugar.go:171
go.etcd.io/bbolt.(*Tx).Commit(0xc0001aa9a0)
        /home/fuwei/go/pkg/mod/go.etcd.io/bbolt@v1.4.0-alpha.0/tx.go:173 +0x206
go.etcd.io/etcd/server/v3/storage/backend.(*batchTx).commit(0xc00019b180, 0x0)
        /home/fuwei/go/src/go.etcd.io/etcd/server/storage/backend/batch_tx.go:269 +0xdf
go.etcd.io/etcd/server/v3/storage/backend.(*batchTxBuffered).unsafeCommit(0xc00019b180, 0x0)
        /home/fuwei/go/src/go.etcd.io/etcd/server/storage/backend/batch_tx.go:378 +0x425
go.etcd.io/etcd/server/v3/storage/backend.(*batchTxBuffered).commit(0xc00019b180, 0x80?)
        /home/fuwei/go/src/go.etcd.io/etcd/server/storage/backend/batch_tx.go:355 +0x78
go.etcd.io/etcd/server/v3/storage/backend.(*batchTxBuffered).Commit(0xc00019b180)
        /home/fuwei/go/src/go.etcd.io/etcd/server/storage/backend/batch_tx.go:342 +0x35
go.etcd.io/etcd/server/v3/storage/backend.(*backend).run(0xc000478180)
        /home/fuwei/go/src/go.etcd.io/etcd/server/storage/backend/backend.go:426 +0x2c7
created by go.etcd.io/etcd/server/v3/storage/backend.newBackend in goroutine 80
        /home/fuwei/go/src/go.etcd.io/etcd/server/storage/backend/backend.go:227 +0xbfd
FAIL    go.etcd.io/etcd/server/v3/etcdserver    0.129s
FAIL
```

This patch also drains goroutines related to raftNode and watch store.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
 2024-02-03 18:58:17 +08:00
Ishan Tyagi
16a5e1da71 Added a error log when learner is not sync with etcd leader. 
Signed-off-by: ishan16696 <ishan.tyagi@sap.com>
 2024-01-30 15:42:11 +05:30
Marek Siarkowicz
40f22e9319
Merge pull request #17176 from YaoC/fix-learner-metric 
server: fix learner metric incorrect issue
 2024-01-12 17:37:46 +01:00
YaoC
f7ab7adf29 server: fix learner metric incorrect issue 
Signed-off-by: YaoC <chengyao09@hotmail.com>
 2024-01-12 09:36:33 +00:00
Sharath Sivakumar
28f4c6fef6
dependency: bump github.com/grpc-ecosystem/grpc-gateway/v2 from 2.18.1 to 2.19.0 
Signed-off-by: Sharath Sivakumar <mailssr9@gmail.com>
 2024-01-09 16:26:47 +01:00
Marek Siarkowicz
a2eb17c809
Merge pull request #17199 from serathius/dont-flock 
Don't flock snapshot files
 2024-01-08 15:03:29 +01:00
Marek Siarkowicz
3471ef133d Add an e2e test and robustness failpoint around recovering from snapshot backend 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2024-01-04 15:25:24 +01:00
Marek Siarkowicz
7f8346b3f2 Don't flock snapshot files 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2024-01-04 14:53:44 +01:00
Marek Siarkowicz
d39d86a214 Improve logs around recovering from snapshot backend 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-12-20 16:26:27 +01:00
Marek Siarkowicz
1e8d66ef95 Add beforeOpenSnapshotBackend failpoint 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-12-20 15:36:54 +01:00
Benjamin Wang
67f17166bf Safeguard lease operations by double checking the leadership 
1. ignore old leader's leases revoking request
2. double check current member's leadership before perform lease renew request
3. etcdserve: ensure current member's leadership before performing lease checkpoint request

Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
 2023-12-15 17:53:36 +00:00
Benjamin Wang
36b2523669 added some log messages for better diagnosis 
Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
 2023-12-13 18:43:22 +00:00
Neil Shen
9f82390ae9 server: refine TestProcessIgnoreMismatchMessage 
Signed-off-by: Neil Shen <overvenus@gmail.com>
 2023-12-07 20:32:43 +08:00
Neil Shen
fb769c4306 server: ignore raft messages if member id mismatch 
Ignore Raft messages when the `To` field mismatches the local member ID.
In cases where incorrect Raft messages are dispatched, potentially due
to a malfunctioning switch, this proactive check prevents panics,
such as "tocommit is out of range".

Signed-off-by: Neil Shen <overvenus@gmail.com>
 2023-12-07 11:57:45 +08:00
Ivan Valdes
8578e07117
server: disable redirects in peer communication 
Disable following redirects from peer HTTP communication on the client's side.
Etcd server may run into SSRF (Server-side request forgery) when adding a new
member. If users provide a malicious peer URL, the existing etcd members may be
redirected to another unexpected internal URL when getting the new member's
version.

Signed-off-by: Ivan Valdes <ivan@vald.es>
 2023-12-04 13:53:28 -08:00
Marek Siarkowicz
bc697bc26e Revert "Switch to validating v3 when v2 and v3 are synchronized" 
This reverts commit 4fe46f92030e4381e6f9bf95adbb22a08282d297.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-12-03 18:12:09 +01:00
Marek Siarkowicz
03d551243b
Merge pull request #17015 from serathius/extract-membership-applier 
Extract membership applier
 2023-11-27 19:59:21 +01:00
Marek Siarkowicz
62b772c321
Merge pull request #17021 from serathius/test-applyconfstate 
Test ApplyConfState after restart
 2023-11-27 13:19:50 +01:00
Marek Siarkowicz
fbdf65f101 Test v3 storage configuration validation 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-11-27 12:00:47 +01:00
Benjamin Wang
e1d79097d5
Merge pull request #17017 from serathius/switch-validation-v3 
Switch to validating v3 when v2 and v3 are synchronized
 2023-11-27 10:50:14 +00:00
Marek Siarkowicz
e192a05193 Test ApplyConfState after restart 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-11-26 17:40:28 +01:00
Benjamin Wang
a97052acf4 remove unused method and functions 
Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
 2023-11-25 16:16:52 +00:00
Marek Siarkowicz
4fe46f9203 Switch to validating v3 when v2 and v3 are synchronized 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-11-24 17:46:33 +01:00
Marek Siarkowicz
2ad21558ac Remove shouldApplyV3 from the v3 applier 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-11-24 16:13:25 +01:00
Marek Siarkowicz
d22c00ccee Extract membership applier 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-11-24 15:57:15 +01:00
Marek Siarkowicz
7fdb33065d Move duplicated shouldApplyV3 logic up into apply method 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-11-24 10:21:14 +01:00
Marek Siarkowicz
093666f450 Cleanup v2 applier 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-11-23 15:41:13 +01:00
Marek Siarkowicz
c72ff1e69c Remove syncing the v2 store TTLs 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-11-23 14:55:01 +01:00
Marek Siarkowicz
ed3375e076 Remove v2 apply logic 
v2 store is no longer available in v3.6.
We can remove apply logic for it as they will never be used.

Only v2 PUT is neeeded as it applies to v3 storage and etcd v3.5 uses it for setting member
attributes and cluster version.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-11-23 14:13:07 +01:00
Marek Siarkowicz
28d9564962
Merge pull request #16994 from serathius/tests-v2-cluster-version 
Add tests for setting cluster version using v2 request
 2023-11-22 21:11:18 +01:00
Marek Siarkowicz
2f30760b37 Add tests for setting cluster version using v2 request 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-11-22 18:05:19 +01:00
Marek Siarkowicz
29dd025b84 Stop using v2 requests in server tests 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-11-22 14:41:53 +01:00
Benjamin Wang
fd0882b67e
Merge pull request #16984 from siyuanfoundation/lin-read 
etcdserver: add linearizable_read check to readyz.
 2023-11-21 19:51:12 +00:00
Siyuan Zhang
12b640523a etcdserver: add linearizable_read check to readyz. 
Signed-off-by: Siyuan Zhang <sizhang@google.com>
 2023-11-21 11:13:20 -08:00
Benjamin Wang
ec6147cd04
Merge pull request #16967 from serathius/remove-v2-proposals 
Remove v2 proposals code
 2023-11-21 15:35:51 +00:00
James Blair
3b37afec7b
Don't follow redirects when checking peer urls. 
It's possible that etcd server may run into SSRF situation when adding a new member. If users provide a malicious peer URL, the existing etcd members may be redirected to other unexpected internal URL when getting the new member's version.

Signed-off-by: James Blair <mail@jamesblair.net>
 2023-11-21 10:25:20 +13:00
Marek Siarkowicz
dd7a4d28a8 Remove code used to make v2 proposals 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-11-19 22:39:33 +01:00
Marek Siarkowicz
b4fd31f254 Remove code for setting cluster version via V2 API 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-11-19 15:28:52 +01:00
Siyuan Zhang
3897103b77 etcdserver: add metric counters for livez/readyz health checks. 
Signed-off-by: Siyuan Zhang <sizhang@google.com>
 2023-11-14 09:26:00 -08:00
Tessa Pham
2b7e1c6f82 fix scripts 
Signed-off-by: Tessa Pham <hpham111@bloomberg.net>
 2023-11-08 00:27:13 -06:00
Chao Chen
8a6c1335e2 v3rpc: run health notifier to listen on online defrag state change 
Signed-off-by: Chao Chen <chaochn@amazon.com>
 2023-10-28 17:49:24 -07:00
Chao Chen
ea035471ce online defrag notifies gRPC health server to expose NOT_SERVING status 
Signed-off-by: Chao Chen <chaochn@amazon.com>
 2023-10-25 08:58:33 -07:00
Benjamin Wang
34382006db test: implement method ForgetLeader for struct nodeRecorder 
Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2023-10-24 10:19:13 +01:00
Chao Chen
1324f03254 add existing http health check handler e2e test 
Signed-off-by: Chao Chen <chaochn@amazon.com>
 2023-10-18 12:42:23 -07:00