1929 Commits

Author SHA1 Message Date
Benjamin Wang
6071b1c523 Support configuring MaxConcurrentStreams for http2
Backport https://github.com/etcd-io/etcd/pull/14219 to 3.4

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-07-21 14:25:29 +08:00
Chao Chen
864006b72d print out applied index as uint64
Signed-off-by: Chao Chen <chaochn@amazon.com>
2022-07-20 12:07:51 -07:00
Pierre Zemb
3f9fba9112 etcdserver: add more detailed traces on linearized reading
To improve debuggability of `agreement among raft nodes before
linearized reading`, we added some tracing inside
`linearizableReadLoop`.

This will allow us to know the timing of `s.r.ReadIndex` vs
`s.applyWait.Wait(rs.Index)`.

Signed-off-by: Chao Chen <chaochn@amazon.com>
2022-07-20 12:07:51 -07:00
Benjamin Wang
07d2b1d626 support linearizable renew lease for 3.4
Cherry pick https://github.com/etcd-io/etcd/pull/13932 to 3.4.

When etcdserver receives a LeaseRenew request, it may be still in
progress of processing the LeaseGrantRequest on exact the same
leaseID. Accordingly it may return a TTL=0 to client due to the
leaseID not found error. So the leader should wait for the appliedID
to be available before processing client requests.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-07-19 13:34:55 +08:00
Benjamin Wang
f036529b5d Backport two lease related bug fixes to 3.4
The first bug fix is to resolve the race condition between goroutine
and channel on the same leases to be revoked. It's a classic mistake
in using Golang channel + goroutine. Please refer to
https://go.dev/doc/effective_go#channels

The second bug fix is to resolve the issue that etcd lessor may
continue to schedule checkpoint after stepping down the leader role.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-24 09:09:40 +08:00
Benjamin Wang
1abf085cfb fix all the pipeline failues for release 3.4
Items resolved:
1. fix the vet error: possible misuse of reflect.SliceHeader;
2. fix the vet error: call to (*T).Fatal from a non-test goroutine;
3. bump package golang.org/x/crypto, net and sys;
4. bump boltdb from 1.3.3 to 1.3.6;
5. remove the vendor directory;
6. remove go 1.12.17 and 1.15.15, add go 1.16.15 into pipeline;
7. bump go version to 1.16 in go.mod;
8. fix the issue: compile: version go1.16.15 does not match go tool version go1.17.11,
   refer to https://github.com/actions/setup-go/issues/107;
9. fix data race on compactMainRev and watcherGauge;
10. fix test failure for TestLeasingTxnOwnerGet in cluster_proxy mode.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-22 05:28:45 +08:00
Hitoshi Mitake
757a8e8f5b *: implement a retry logic for auth old revision in the client 2022-04-29 23:46:24 +09:00
Chao Chen
04d47a93f9 backport from #13467 exclude the same alarm type activated by multiple peers 2021-11-12 14:17:14 -08:00
Chao Chen
dbde4f2d5e Backport-3.4 exclude alarms from health check conditionally 2021-05-04 10:37:12 -07:00
Chao Chen
c4eb81af99 Backport-3.4 etcdserver/util.go: reduce memory when logging range requests 2021-04-22 15:07:44 -07:00
Lili Cosic
0b7e4184e8 etcdserver,wal: Convert int to string using rune() 2021-04-19 11:18:13 +02:00
Chris Wedgwood
656dc63eab etcdserver: fix incorrect metrics generated when clients cancel watches
Manual cherry-pick of 9571325fe85173a60c89d6ac6ce3491c7b1ec7a4 for
release-3.4.
2021-03-31 22:59:29 -07:00
Piotr Tabor
30799c97be
Merge pull request #12815 from dbavatar/release-3.4-peervalidation
etcdserver: Fix PeerURL validation
2021-03-30 12:54:32 +02:00
Sam Batschelet
9aeabe447d server: Added config parameter experimental-warning-apply-duration
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-03-03 12:14:30 -05:00
Chao Chen
f27ef4d343 [Backport-3.4] etcdserver/api/etcdhttp: log successful etcd server side health check in debug level
ref. #12677
ref. 0b9cfa8677
2021-02-08 21:44:44 -08:00
galal-hussein
3019246742 etcdserver: add ConfChangeAddLearnerNode to the list of config changes
To fix a panic that happens when trying to get ids of etcd members in
force new cluster mode, the issue happen if the cluster previously had
etcd learner nodes added to the cluster

Fixes #12285
2020-09-14 17:50:57 +02:00
Jordan Liggitt
b8878eac45 etcdserver: Avoid panics logging slow v2 requests in integration tests 2020-08-19 11:30:39 -04:00
Gyuho Lee
e71e0c5c88
Merge pull request #12226 from jingyih/fix_backport_PR12216
*: add plog logging to the backport of PR12216
2020-08-18 08:48:09 -07:00
Gyuho Lee
299e0f17aa Revert "etcdserver/api/v3rpc: "MemberList" never return non-empty ClientURLs"
This reverts commit 0372cfc7ab1052ac616ca34551f83657a8fd2e3e.
2020-08-18 08:45:38 -07:00
jingyih
75d5e78d1f *: fix backport of PR12216
Fix bugs introduced in commit c60dabf
2020-08-16 15:01:18 +08:00
jingyih
c60dabf2f3 *: add experimental flag for watch notify interval
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-08-15 10:24:25 -07:00
Gyuho Lee
008074187c etcdserver: add OS level FD metrics
Similar counts are exposed via Prometheus.
This adds the one that are perceived by etcd server.

e.g.

os_fd_limit 120000
os_fd_used 14
process_cpu_seconds_total 0.31
process_max_fds 120000
process_open_fds 17

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-08-12 18:38:35 -07:00
Gyuho Lee
0372cfc7ab etcdserver/api/v3rpc: "MemberList" never return non-empty ClientURLs
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

cr https://code.amazon.com/reviews/CR-29712724
2020-07-16 16:29:51 -07:00
Yuchen Zhou
ed28c768a3 etcdserver: change protobuf field type from int to int64 (#12000) 2020-07-08 10:21:10 +03:00
cfc4n
4488595e05 auth: Customize simpleTokenTTL settings.
see https://github.com/etcd-io/etcd/issues/11978 for more detail.
2020-06-25 19:58:26 +08:00
cfc4n
ee963470f4 etcdserver:FDUsage set ticker to 10 minute from 5 seconds. This ticker will check File Descriptor Requirements ,and count all fds in used. And recorded some logs when in used >= limit/5*4. Just recorded message. If fds was more than 10K,It's low performance due to FDUsage() works. So need to increase it.
see https://github.com/etcd-io/etcd/issues/11969 for more detail.
2020-06-24 13:28:40 +08:00
Gyuho Lee
7adbfa1144
Merge pull request #12038 from spzala/automated-cherry-pick-of-#11608-upstream-release-3.4
Automated cherry pick of #11608
2020-06-21 19:19:50 -07:00
Hitoshi Mitake
963b242846 etcdserver: don't let InternalAuthenticateRequest have password 2020-06-21 19:18:18 -04:00
Sahdev P. Zala
9a24f73f7b Discovery: do not allow passing negative cluster size
When an etcd instance attempts to perform service discovery, if a
cluster size with negative value  is provided, the etcd instance
will panic without recovery because of
2020-06-21 18:00:35 -04:00
David Crawshaw
78f67988aa
etcdserver, et al: add --unsafe-no-fsync flag
This makes it possible to run an etcd node for testing and development
without placing lots of load on the file system.

Fixes #11930.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
2020-06-04 20:19:28 -07:00
Gyuho Lee
cfe37de6c0 rafthttp: log snapshot download duration
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-20 11:37:01 -07:00
Gyuho Lee
a668adba78 rafthttp: improve snapshot send logging
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-18 11:39:24 -07:00
Gyuho Lee
9bad82fee5 *: make sure snapshot save downloads SHA256 checksum
ref. https://github.com/etcd-io/etcd/pull/11896

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-17 17:38:42 -07:00
Gyuho Lee
f1ea03a7c8 etcdserver/api/snap: exclude orphaned defragmentation files in snapNames
ref. https://github.com/etcd-io/etcd/pull/11900

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-17 14:21:02 -07:00
Ted Yu
4079deadb4 etcdserver: continue releasing snap db in case of error
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-17 14:15:26 -07:00
Viacheslav Biriukov
87fc3c9e57 etcdserver,wal: fix inconsistencies in WAL and snapshot
etcdserver/*, wal/*: changes to snapshots and wal logic
etcdserver/*: changes to snapshots and wal logic to fix #10219
etcdserver/*, wal/*: add Sync method
etcdserver/*, wal/*: find valid snapshots by cross checking snap files and wal snap entries
etcdserver/*, wal/*:Add comments, clean up error messages and tests
etcdserver/*, wal/*: Remove orphaned .snap.db files during Release

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-15 08:40:09 -07:00
Jingyi Hu
f1eca4e1fa
Merge pull request #11752 from tangcong/automated-cherry-pick-of-#11652-#11670-#11710-origin-release-3.4
Automated cherry pick of #11652 #11670 #11710
2020-04-10 23:21:45 +08:00
tangcong
eb80716532 etcdserver: print warn log when failed to apply request 2020-04-09 09:33:40 +08:00
tangcong
347c8dac3b *: fix auth revision corruption bug 2020-04-09 09:33:36 +08:00
Changxin Miao
9c8554573f etcdserver: watch stream got closed once one request is not permitted (#11708) 2020-04-06 07:06:57 -07:00
Gyuho Lee
e63db56cc9 etcdserver/api/v3rpc: handle api version metadata, add metrics
ref.
https://github.com/etcd-io/etcd/pull/11687

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-18 17:17:04 -07:00
Gyuho Lee
0b9cfa8677 etcdserver/api/etcdhttp: log server-side /health checks
ref.
https://github.com/etcd-io/etcd/pull/11704

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-18 16:29:24 -07:00
Rafael Fernández López
257319fb18 etcdserver: fix quorum calculation when promoting a learner member
When promoting a learner member we should not count already a voting
member, but take only into account the number of existing voting
members and their current status (started, unstarted) when taking the
decision whether a learner member can be promoted.

Before this change, it was impossible to grow from a quorum N to a N+1
through promoting a learning member.

Fixes: #11633
2020-02-21 23:14:55 -08:00
jingyih
770674e4a6 etcdserver: corruption check via http
During corruption check, get peer's hashKV via http call.
2020-02-18 14:12:19 -08:00
Joe Betz
6d646c442a
mvcc/backend: Delete orphaned db.tmp files before defrag 2020-02-13 12:26:54 -08:00
Jingyi Hu
b66203c0a1
Merge pull request #11299 from jingyih/automated-cherry-pick-of-#10468-upstream-release-3.4
Automated cherry pick of #10468 on release-3.4
2019-11-05 18:34:22 -08:00
Jingyi Hu
4f3c81d81d etcdserver: wait purge file loop during shutdown
To prevent the purge file loop from accidentally acquiring the file lock
and remove the files during server shutdowm.
2019-10-30 16:04:41 -07:00
Jingyi Hu
d6b30e43cd etcdserver: remove auth validation loop
Remove auth validation loop in v3_server.raftRequest(). Re-validation
when error ErrAuthOldRevision occurs should be handled on client side.
2019-10-28 15:03:44 -07:00
Gyuho Lee
6cf418ff6d
Merge pull request #11275 from YoyinZyc/stream-support-3.4
rafthttp: add 3.4 stream type
2019-10-18 10:25:53 -07:00
yoyinzyc
97e68cf4e7 rafthttp: add 3.4 stream type 2019-10-17 14:33:53 -07:00