Mirroristas/etcd
2
0
Fork 0
mirror of https://github.com/etcd-io/etcd.git synced 2024-09-27 06:25:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
19,257 Commits 15 Branches 491 Tags 148 MiB
Commit Graph

1173 Commits

Author SHA1 Message Date
Benjamin Wang
b48641e5f2
Merge pull request #14348 from VinozzZ/add-integration-test-for-tracing 
embed: add integration test for distributed tracing
 2022-10-13 02:11:59 +08:00
Benjamin Wang
5746d6eb86 etcdserver: added more debug log for the purgeFile goroutine 
Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-10-12 17:32:33 +08:00
Benjamin Wang
1ccdb3762d Test: fix all corruption detection related unit test cases 
Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-09-30 06:06:41 +08:00
Benjamin Wang
d116d02e04 etcdserver: update corrupt hash detection's logic 
get peer's hash using the same revision as the value used by leader

Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-09-30 06:03:50 +08:00
Benjamin Wang
abef537a90
Merge pull request #14515 from spongecaptain/btree-generics 
upate:use google/btree in the genric way
 2022-09-27 16:44:13 +08:00
wathenjiang
319db38b0a update: add benchmark test 
benchmark result:
(1) master branch
$ go test -bench='BenchmarkIndexPut$' -count=5
goos: darwin
goarch: amd64
pkg: go.etcd.io/etcd/server/v3/storage/mvcc
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkIndexPut-12             1000000              2591 ns/op
BenchmarkIndexPut-12             1000000              2531 ns/op
BenchmarkIndexPut-12             1000000              2536 ns/op
BenchmarkIndexPut-12             1000000              2546 ns/op
BenchmarkIndexPut-12             1000000              2538 ns/op
PASS
ok      go.etcd.io/etcd/server/v3/storage/mvcc  167.439s

$ go test -bench='BenchmarkIndexGet$' -count=5
goos: darwin
goarch: amd64
pkg: go.etcd.io/etcd/server/v3/storage/mvcc
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkIndexGet-12             1000000              2021 ns/op
BenchmarkIndexGet-12             1000000              2029 ns/op
BenchmarkIndexGet-12             1000000              2044 ns/op
BenchmarkIndexGet-12             1000000              1973 ns/op
BenchmarkIndexGet-12             1000000              2027 ns/op
PASS
ok      go.etcd.io/etcd/server/v3/storage/mvcc  177.815s

(2) google/btree in the generic way
$ go test -bench='BenchmarkIndexPut$' -count=5
goos: darwin
goarch: amd64
pkg: go.etcd.io/etcd/server/v3/storage/mvcc
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkIndexPut-12             1000000              2477 ns/op
BenchmarkIndexPut-12             1000000              2380 ns/op
BenchmarkIndexPut-12             1000000              2360 ns/op
BenchmarkIndexPut-12             1000000              2396 ns/op
BenchmarkIndexPut-12             1000000              2382 ns/op
PASS
ok      go.etcd.io/etcd/server/v3/storage/mvcc  165.841s

$ go test -bench='BenchmarkIndexGet$' -count=5
goos: darwin
goarch: amd64
pkg: go.etcd.io/etcd/server/v3/storage/mvcc
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkIndexGet-12             1000000              1985 ns/op
BenchmarkIndexGet-12             1000000              1914 ns/op
BenchmarkIndexGet-12             1000000              1900 ns/op
BenchmarkIndexGet-12             1000000              1905 ns/op
BenchmarkIndexGet-12             1000000              1894 ns/op
PASS
ok      go.etcd.io/etcd/server/v3/storage/mvcc  177.573s

Signed-off-by: wathenjiang <wathenjiang@tencent.com>
 2022-09-27 14:33:02 +08:00
Spongecaptain
c53dfc7c5b upate:use google/btree in the genric way 
Signed-off-by: wathenjiang <wathenjiang@tencent.com>
 2022-09-27 10:16:15 +08:00
Benjamin Wang
1c20ed2cc5
Merge pull request #14521 from lovehhf/remove_pick_peer_url 
membership: Remove PickPeerURL Method
 2022-09-27 02:10:35 +08:00
Hongfei Huang
f6d808736c membership: Remove PickPeerURL Method 
PickPeerURL only used by unit test

Signed-off-by: Hongfei Huang <853885165@qq.com>
 2022-09-26 23:21:10 +08:00
Kafuu Chino
f1d4935e91 *: avoid closing a watch with ID 0 incorrectly 
Signed-off-by: Kafuu Chino <KafuuChinoQ@gmail.com>

add test
 2022-09-26 20:30:33 +08:00
Benjamin Wang
434c7c4309
Merge pull request #14506 from SimFG/main 
wal: Fix the `walWriteBytes` metric
 2022-09-26 09:36:07 +08:00
Jeremy Leach
cc1e245368 etcdserver: fix corruption check when server has just been compacted 
When a key-value store corruption check happens immediately after a
compaction, the revision at which the key-value store hash is computed,
is the compacted revision itself.
In that case, the hash computation logic was incorrect because it
returned an ErrCompacted error; this error should instead be returned when
the revision at which the key-value store is hashed, is strictly lower
than the compacted revision.

Fixes #14325

Signed-off-by: Jeremy Leach <44558776+jbml@users.noreply.github.com>
 2022-09-24 22:20:26 +10:00
SimFG
5702765729 wal: Fix the walWriteBytes metric 
Signed-off-by: SimFG <1142838399@qq.com>
 2022-09-22 19:23:06 +08:00
Benjamin Wang
9097e61b40 etcdserve: revert the etcdserver side change for the data loss on one node cluster 
Since the raft side change has been merged, so we need to revert the etcdserver
side change.
Refer to
  https://github.com/etcd-io/etcd/pull/14413
  https://github.com/etcd-io/etcd/pull/14400

Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-09-22 15:19:20 +08:00
Benjamin Wang
7f10dccbaf Bump go 1.19: update all the dependencies and go.sum files 
1. run ./scripts/fix.sh;
2. cd tools/mod; gofmt -w . & go mod tidy;

Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-09-22 08:47:46 +08:00
Benjamin Wang
cd0b1d0c66 Bump go 1.19: upgrade go version to 1.19 in all go.mod files 
Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-09-22 08:47:46 +08:00
Marek Siarkowicz
026794495f
Merge pull request #14494 from demoManito/remove/redundant-type-conversion 
etcd: remove redundant type conversion
 2022-09-21 11:34:19 +02:00
Benjamin Wang
2441a24cee
Merge pull request #14493 from demoManito/style/format-import-order 
etcd: format import order
 2022-09-21 06:03:31 +08:00
demoManito
f67ec10779 etcd: format import order 
golang CodeReviewComments:
https://github.com/golang/go/wiki/CodeReviewComments#imports

Signed-off-by: demoManito <1430482733@qq.com>
 2022-09-20 18:41:39 +08:00
demoManito
a9c3d56508 etcd: remove redundant type conversion 
Signed-off-by: demoManito <1430482733@qq.com>
 2022-09-20 11:26:02 +08:00
Benjamin Wang
5344085338
Merge pull request #14491 from ahrtr/bump_jwt_4.4.2 
etcd: Bump golang-jwt/jwt/ version to 4.4.2
 2022-09-20 10:18:44 +08:00
Sahdev Zala
ad4a465383
Merge pull request #14492 from cmurphy/bump-golang-x-net 
*: Update golang.org/x/net to latest
 2022-09-19 22:11:43 -04:00
Colleen Murphy
7ea2a3d7cb *: Update golang.org/x/net to latest 
Update golang.org/x/net to address CVE-2022-27664.

Signed-off-by: Colleen Murphy <colleen.murphy@suse.com>
 2022-09-19 16:01:45 -07:00
Benjamin Wang
159ed15afc
Merge pull request #14479 from demoManito/fix/declaring-empty-slice 
etcd: modify declaring empty slices
 2022-09-20 05:22:59 +08:00
Benjamin Wang
09db6ec1d7 etcd: Bump golang-jwt/jwt/ version to 4.4.2 
github.com/golang-jwt/jwt adds go mod support startig from 4.0.0,
and it's backwards-compatible with existing v3.x.y tags.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-09-20 04:06:47 +08:00
Hitoshi Mitake
c793f18238
Merge pull request #14322 from mitake/watch-auth-err 
*: handle auth invalid token and old revision errors in watch
 2022-09-19 23:33:49 +09:00
Hitoshi Mitake
2dcfa83094 *: handle auth invalid token and old revision errors in watch 
Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com>
 2022-09-17 21:51:36 +09:00
Sahdev Zala
cdf4228673
Merge pull request #14308 from dusk125/main 
server/etcdmain: add configurable cipher list to gRPC proxy listener
 2022-09-16 12:15:08 -04:00
demoManito
72cf0cc04a etcd: modify declaring empty slices 
declare an empty slice to var s []int replace  s :=[]int{}, https://github.com/golang/go/wiki/CodeReviewComments#declaring-empty-slices

Signed-off-by: demoManito <1430482733@qq.com>
 2022-09-16 14:41:14 +08:00
Benjamin Wang
b7ba0542f6
Merge pull request #14422 from kkkkun/remove-redundant-code 
remove redundant log messsages
 2022-09-16 12:18:43 +08:00
kkkkun
c4582aaaee remove redundant log messages 
Signed-off-by: kkkkun <scuzk373x@gmail.com>
 2022-09-16 11:45:43 +08:00
lovehhf
3b585e94fc mvcc: Remove unused revisions and change comment rev to modified 
Signed-off-by: Hongfei Huang <853885165@qq.com>
 2022-09-14 23:36:54 +08:00
Benjamin Wang
1d95b82b19
Merge pull request #14421 from vsvastey/usr/vsvastey/open-with-max-index-test-fix 
testing: fix TestOpenWithMaxIndex cleanup
 2022-09-08 06:41:36 +08:00
Vladimir Sokolov
a4f140c9fa testing: fix TestOpenWithMaxIndex cleanup 
A WAL object was closed by defer, however the WAL was rewritten afterwards,
so defer closed already closed WAL but not the new one. It caused a data
race between writing file and cleaning up a temporary test directory,
which led to a non-deterministic bug.

Fixes #14332

Signed-off-by: Vladimir Sokolov <vsvastey@gmail.com>
 2022-09-07 21:30:16 +03:00
Yingrong Zhao
57206f9dfb shutdown tracer provider first 
Signed-off-by: Yingrong Zhao <yingrong.zhao@gmail.com>
 2022-09-06 11:26:31 -04:00
Benjamin Wang
3dc5348d94
Merge pull request #14419 from ahrtr/alarm_list_ci 
Move consistent_index forward when executing alarmList operation
 2022-09-06 03:50:58 +08:00
Benjamin Wang
cc840336f0 move consistent_index forward when executing alarmList operation 
The alarm list is the only exception that doesn't move consistent_index
forward. The reproduction steps are as simple as,

```
etcd --snapshot-count=5 &
for i in {1..6}; do etcdctl  alarm list; done
kill -9 <etcd_pid>
etcd
```

Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-09-05 10:05:55 +08:00
Benjamin Wang
2a10049e47 fix the potential data loss for clusters with only one member 
For a cluster with only one member, the raft always send identical
unstable entries and committed entries to etcdserver, and etcd
responds to the client once it finishes (actually partially) the
applying workflow.

When the client receives the response, it doesn't mean etcd has already
successfully saved the data, including BoltDB and WAL, because:
   1. etcd commits the boltDB transaction periodically instead of on each request;
   2. etcd saves WAL entries in parallel with applying the committed entries.
Accordingly, it may run into a situation of data loss when the etcd crashes
immediately after responding to the client and before the boltDB and WAL
successfully save the data to disk.
Note that this issue can only happen for clusters with only one member.

For clusters with multiple members, it isn't an issue, because etcd will
not commit & apply the data before it being replicated to majority members.
When the client receives the response, it means the data must have been applied.
It further means the data must have been committed.
Note: for clusters with multiple members, the raft will never send identical
unstable entries and committed entries to etcdserver.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-08-30 15:29:20 +08:00
Allen Ray
c52108942b
Merge branch 'main' into main 2022-08-29 12:07:27 -04:00
Abirdcfly
08a9d1da07
chore: remove duplicate word in comments 
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
 2022-08-27 13:39:48 +08:00
Hitoshi Mitake
9c8326bb50
Merge pull request #14358 from vivekpatani/main 
server/auth: refresh cache on each NewAuthStore
 2022-08-27 11:13:59 +09:00
Benjamin Wang
dc4b810195
Merge pull request #14388 from niconorsk/fix/notify-systemd-when-cluster-ready-times-out 
etcdmain: Honour ExperimentalWaitClusterReadyTimeout in startEtcd
 2022-08-26 18:34:52 +08:00
Nicolai Moore
e15bdd9df1 etcdmain: Honour ExperimentalWaitClusterReadyTimeout in startEtcd 
When we can't reach quorum, we were waiting forever and never sending
the systemd notify message. As a result, systemd would eventually time out
and restart the etcd process which likely would make the unhealthy cluster
in an even worse state

Improves #13785

Signed-off-by: Nicolai Moore <niconorsk@gmail.com>
 2022-08-26 18:06:50 +10:00
Vitalii Levitskii
be58a2539c Added client-auto-sync-interval argument to the grpc-proxy 
Signed-off-by: Vitalii Levitskii <vitalii@uber.com>
 2022-08-25 15:33:38 +03:00
vivekpatani
ae608da7e6 server,test: refresh cache on each NewAuthStore 
- permissions were incorrectly loaded on restarts.
- https://github.com/etcd-io/etcd/issues/14355

Signed-off-by: vivekpatani <9080894+vivekpatani@users.noreply.github.com>
 2022-08-23 20:11:47 -07:00
Yingrong Zhao
1869fa9db2 have the tracingExporter to own resources it initializes 
Signed-off-by: Yingrong Zhao <yingrong.zhao@gmail.com>
 2022-08-19 11:31:52 -04:00
Benjamin Wang
1851316519
Merge pull request #14266 from stefanmonkey/feat/grpc-logging 
Add logging grpc request and response content with grpc-proxy mode
 2022-08-19 05:52:16 +08:00
Benjamin Wang
27ffd7e1cf
Merge pull request #14351 from spacewander/rca 
chore: log when an invalid watch request is received
 2022-08-18 09:45:29 +08:00
Benjamin Wang
74506738b8 Refactor the keepAliveListener and keepAliveConn 
Only `net.TCPConn` supports `SetKeepAlive` and `SetKeepAlivePeriod`
by default, so if you want to warp multiple layers of net.Listener,
the `keepaliveListener` should be the one which is closest to the
original `net.Listener` implementation, namely `TCPListener`.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-08-18 04:24:05 +08:00
Sam Batschelet
76a5902efa server/etcdmain: add configurable cipher list to gRPC proxy listener 
Signed-off-by: Allen Ray <alray@redhat.com>
 2022-08-17 10:56:27 -04:00