17638 Commits

Author SHA1 Message Date
Sahdev Zala
55525e3cc5 Benchmark: remove redundant function call
Remove the redundancy as HashKV func is being called twice.
2021-08-12 09:56:38 -04:00
Sahdev Zala
ea24fb8507
Merge pull request #13282 from yuzhiquan/fix-typo-migrate
migrate_command: Fix typo in migrate
2021-08-09 11:48:10 -04:00
yuzhiquan
d0b9572fde fix typo in migrate 2021-08-09 17:55:43 +08:00
Sahdev Zala
706f256a05
Merge pull request #13279 from dengziming/typo
MINOR: Fix typos(hearbeat -> heartbeat)
2021-08-08 18:07:52 -04:00
dengziming
a286f5bb99 MINOR: Fix typos(hearbeat -> heartbeat) 2021-08-07 11:41:13 +08:00
Sahdev Zala
03ffa76db7
Merge pull request #13278 from mrueg/cleanup-todos
Cleanup remaining TODOs for/past 3.5 release
2021-08-06 10:51:51 -04:00
Manuel Rüger
c9b4c866ce Cleanup remaining TODOs for/past 3.5 release 2021-08-06 13:29:25 +02:00
Piotr Tabor
873f3691f1
Merge pull request #13216 from serathius/wal
Annotate proto messages with version and detect etcd version generated the wal
2021-08-06 10:58:57 +02:00
Sam Batschelet
b448daa698
Merge pull request #13275 from lilic/add-peer-dashboard
contrib/mixin/mixin.libsonnet: Add dashboard for peer round trip time
2021-08-05 08:27:38 -04:00
Lili Cosic
55b697c528 contrib/mixin/mixin.libsonnet: Add dashboard for peer round trip time
This helps users debug firing alerts.
2021-08-05 13:15:34 +02:00
Piotr Tabor
e591fcba33
Merge pull request #13220 from serathius/migrateforce
etcdutl: Implement migrate --force command
2021-08-04 22:50:46 +02:00
Marek Siarkowicz
b4b50f7852 etcdutl: Implement migrate --force command
Co-authored-by: nic-chen <33000667+nic-chen@users.noreply.github.com>
Co-authored-by: Heisenberg <yuzhiquanlong@gmail.com>
2021-08-04 17:58:18 +02:00
Sahdev Zala
c8c7fc49b8
Merge pull request #13270 from serathius/release-test
*: Bump release tests to use v3.5.0
2021-08-04 11:10:30 -04:00
Marek Siarkowicz
0ae9f0625c *: Bump release tests to use v3.5.0 2021-08-04 13:41:12 +02:00
Marek Siarkowicz
1b4e54c238 api: Annotate proto messages and use it to detect etcd version that generated wal
Co-authored-by: Lili Cosic <cosiclili@gmail.com>
2021-08-04 13:36:08 +02:00
Piotr Tabor
1e46145b29
Merge pull request #13268 from serathius/storage-min
Move storage bootstrap dependencies to storage package
2021-08-03 17:28:36 +02:00
Sahdev Zala
1da6c71321
Merge pull request #13256 from Haimantika/replacejwtgo
Replace github.com/form3tech-oss/jwt-go with https://github.com/golang-jwt/jwt
2021-08-03 10:17:43 -04:00
Sahdev Zala
55b7b74589
Merge pull request #13265 from spzala/missingprnum
CHANGELOG: add missing PR number
2021-08-03 09:11:47 -04:00
Marek Siarkowicz
83a325ac46 server: Move all functions needed for storage bootstrap to storage package
This is prerequestite to move storage bootstrap, splitted to separate PR
to make it easier to review.
2021-08-03 13:09:15 +02:00
Marek Siarkowicz
23b742cfd3 server: Remove Quota direct dependency on EtcdServer 2021-08-03 12:48:41 +02:00
Marek Siarkowicz
44b8ae145b etcdserver: Move datadir and wal to storage package 2021-08-03 12:47:37 +02:00
Haimantika Mitra
c10d50c4b3 Replace github.com/form3tech-oss/jwt-go with https://github.com/golang-jwt/jwt
Signed-off-by: Haimantika Mitra <haimantikamitra@gmail.com>

Made required adjustments to the go.sum file

Signed-off-by: Haimantika Mitra <haimantikamitra@gmail.com>

Changed go.sum file in the server directory

Signed-off-by: Haimantika Mitra <haimantikamitra@gmail.com>

Removed the white space

Signed-off-by: Haimantika Mitra <haimantikamitra@gmail.com>

Made required changes

Signed-off-by: Haimantika Mitra <haimantikamitra@gmail.com>

Trying to fix the fails

Signed-off-by: haimantika mitra <haimantikamitra@gmail.com>

Removed error

Signed-off-by: haimantika mitra <haimantikamitra@gmail.com>

Fixed bill-of-materials.json file

Signed-off-by: haimantika mitra <haimantikamitra@gmail.com>

Changed go.mod with recent version

Signed-off-by: haimantika mitra <haimantikamitra@gmail.com>

Newer version changes

Signed-off-by: haimantika mitra <haimantikamitra@gmail.com>

Changes to etcdutl directory

Signed-off-by: haimantika mitra <haimantikamitra@gmail.com>
2021-08-03 13:49:47 +05:30
Sahdev Zala
5beca68a11 Add missing PR number
Add missing PR number in the changelog entry.
2021-08-02 17:19:21 -04:00
Sam Batschelet
da34e9bf30
Merge pull request #13245 from serathius/makefile
*: Cleanup Makefile
2021-08-02 10:24:33 -04:00
Sam Batschelet
c58d7218f6
Merge pull request #13255 from discordianfish/mixin-configure-cluster-label
Mixin: Support configuring cluster label
2021-08-02 09:51:27 -04:00
Sahdev Zala
2526463e44
Merge pull request #13236 from roytman/expensiveRequest
etcdserver: configure "expensive" requests duration
2021-08-02 09:33:43 -04:00
Sergey Kacheev
eac75c28ae
Apply suggestions from code review
Co-authored-by: Lili Cosic <cosiclili@gmail.com>
2021-08-01 00:20:50 +07:00
nickhill
99182f5404 etcdserver,clientv3: server-side ignore sort-ascend-key for range requests
A client-side optimization was made in #6100 to filter ascending key sorts to avoid an unnecessary re-sort since this is the order already returned by the back-end logic.

It seems to me that this really belongs on the server side since it's tied to the server implementation and should apply for any caller of the kv api (for example non-go clients).

Related, the client/v3 syncer depends on this default sorting which isn't explicit in the kv api contract. So I'm proposing the required sort parameters be included explicitly; it will take the fast path either way.
2021-07-30 15:02:15 -07:00
Johannes 'fish' Ziemke
7885f2a951 Mixin: Support configuring cluster label 2021-07-29 17:54:14 +02:00
yuzhiquan
90773edb10 remove always true in if statement 2021-07-29 14:33:10 +08:00
Sahdev Zala
4cbb949595
Merge pull request #13203 from yishuT/client-race
client: call .Endpoints() in dial() in client/v3/client.go instead of accessing cfg.Endpoints directly
2021-07-29 00:06:58 -04:00
Sahdev Zala
9182a3c200
Merge pull request #13237 from tangcong/fix-auto-tls
fix self-signed-cert-validity parameter cannot be specified in the co…
2021-07-28 13:55:21 -04:00
tangcong
8929b8ca46 CHANGELOG: update for 13237 2021-07-29 00:29:11 +08:00
tangcong
8a3c43827b fix self-signed-cert-validity parameter cannot be specified in the config file 2021-07-29 00:13:13 +08:00
Sahdev Zala
f49ef071eb
Merge pull request #13214 from halleyshx/update_test_egrep
update makefile for test egrep
2021-07-28 09:44:42 -04:00
Sahdev Zala
def6f4ce90
Merge pull request #13246 from yuzhiquan/wrong-args
etcdctl|etcdutl: Invaild args
2021-07-28 09:05:23 -04:00
yuzhiquan
2a5dd39d79 invaild args 2021-07-28 11:39:54 +08:00
halleyshx
2bcfc1ae2e update makefile for test egrep
add 'FAIL:' to egrep
```shell
[root@LF-136-9 etcd]# egrep "(--- FAIL:|DATA RACE|panic: test timed out|appears to have leaked)" -B50 -A10 test-MTYyNjIyOTc0MQo.log
[root@LF-136-9 etcd]# egrep "(--- FAIL:|FAIL:|DATA RACE|panic: test timed out|appears to have leaked)" -B50 -A10 test-MTYyNjIyOTc0MQo.log
ok    go.etcd.io/etcd/server/v3/auth  3.247s
ok    go.etcd.io/etcd/server/v3/config  0.047s
ok    go.etcd.io/etcd/server/v3/datadir 0.035s
ok    go.etcd.io/etcd/server/v3/embed 1.944s
ok    go.etcd.io/etcd/server/v3/etcdmain  0.326s
FAIL  go.etcd.io/etcd/server/v3/etcdserver [build failed]
?     go.etcd.io/etcd/server/v3/etcdserver/api  [no test files]
ok    go.etcd.io/etcd/server/v3/etcdserver/api/etcdhttp 0.110s
ok    go.etcd.io/etcd/server/v3/etcdserver/api/membership 0.479s
ok    go.etcd.io/etcd/server/v3/etcdserver/api/rafthttp 0.251s
ok    go.etcd.io/etcd/server/v3/etcdserver/api/snap 0.045s
?     go.etcd.io/etcd/server/v3/etcdserver/api/snap/snappb  [no test files]
ok    go.etcd.io/etcd/server/v3/etcdserver/api/v2auth 1.470s
ok    go.etcd.io/etcd/server/v3/etcdserver/api/v2discovery  0.088s
ok    go.etcd.io/etcd/server/v3/etcdserver/api/v2error  0.034s
ok    go.etcd.io/etcd/server/v3/etcdserver/api/v2http 0.128s
ok    go.etcd.io/etcd/server/v3/etcdserver/api/v2http/httptypes 0.033s
?     go.etcd.io/etcd/server/v3/etcdserver/api/v2stats  [no test files]
ok    go.etcd.io/etcd/server/v3/etcdserver/api/v2store  0.068s
?     go.etcd.io/etcd/server/v3/etcdserver/api/v2v3 [no test files]
?     go.etcd.io/etcd/server/v3/etcdserver/api/v3alarm  [no test files]
?     go.etcd.io/etcd/server/v3/etcdserver/api/v3client [no test files]
ok    go.etcd.io/etcd/server/v3/etcdserver/api/v3compactor  1.793s
?     go.etcd.io/etcd/server/v3/etcdserver/api/v3election [no test files]
?     go.etcd.io/etcd/server/v3/etcdserver/api/v3election/v3electionpb  [no test files]
?     go.etcd.io/etcd/server/v3/etcdserver/api/v3election/v3electionpb/gw [no test files]
?     go.etcd.io/etcd/server/v3/etcdserver/api/v3lock [no test files]
?     go.etcd.io/etcd/server/v3/etcdserver/api/v3lock/v3lockpb  [no test files]
?     go.etcd.io/etcd/server/v3/etcdserver/api/v3lock/v3lockpb/gw [no test files]
ok    go.etcd.io/etcd/server/v3/etcdserver/api/v3rpc  0.089s
ok    go.etcd.io/etcd/server/v3/etcdserver/cindex 0.045s
ok    go.etcd.io/etcd/server/v3/lease 3.324s
ok    go.etcd.io/etcd/server/v3/lease/leasehttp 2.096s
?     go.etcd.io/etcd/server/v3/lease/leasepb [no test files]
?     go.etcd.io/etcd/server/v3/mock/mockstorage  [no test files]
?     go.etcd.io/etcd/server/v3/mock/mockstore  [no test files]
?     go.etcd.io/etcd/server/v3/mock/mockwait [no test files]
ok    go.etcd.io/etcd/server/v3/mvcc  8.805s
ok    go.etcd.io/etcd/server/v3/mvcc/backend  1.983s
?     go.etcd.io/etcd/server/v3/mvcc/backend/testing  [no test files]
?     go.etcd.io/etcd/server/v3/mvcc/buckets  [no test files]
?     go.etcd.io/etcd/server/v3/proxy/grpcproxy [no test files]
?     go.etcd.io/etcd/server/v3/proxy/grpcproxy/adapter [no test files]
?     go.etcd.io/etcd/server/v3/proxy/grpcproxy/cache [no test files]
ok    go.etcd.io/etcd/server/v3/proxy/httpproxy 0.046s
ok    go.etcd.io/etcd/server/v3/proxy/tcpproxy  0.035s
?     go.etcd.io/etcd/server/v3/verify  [no test files]
ok    go.etcd.io/etcd/server/v3/wal 0.513s
ok    go.etcd.io/etcd/server/v3/wal/walpb 0.045s
FAIL
FAIL: (code:2):
  % (cd server && env go test -short -timeout=3m --race --cpu=16 ./...)
FAIL: 'unit' failed at Wed Jul 14 10:29:37 CST 2021
```
2021-07-28 10:10:40 +08:00
Yi-Shu Tai
77a5072b69 client: call .Endpoints() in dial() in client/v3/client.go instead of accessing cfg.Endpoints directly
0cdd558361/client/v3/client.go (L299) accesses
endpoints without acquiring lock. Fix it to call Endpoints()

Fix #13201
2021-07-27 15:55:22 -07:00
Marek Siarkowicz
7b5f8fc71c *: Cleanup Makefile
* Move manual docker tests to ./tests/manual
* Move manual docker makefile targets to ./tests/manual/Makefile
* Remove unused makefile rules
2021-07-27 17:32:39 +02:00
Alexey Roytman
2a26f7ae4c
etcdserver: configure "expensive" requests duration
When a unary request takes more than predefined duration, this request
is defined as "expensive" and a warning is printed. The expensive request
duration is hard-coded to 300 ms. It can be not enough for example
for transactions with a lot of operations. The warnings just blow up
the log files and reduce throughput.

This fix allows user to configure the "expensive" request duration.

Signed-off-by: Alexey Roytman <roytman@il.ibm.com>
2021-07-27 08:33:44 +03:00
Piotr Tabor
53d234f1fe
Merge pull request #13190 from serathius/backend
Introduces Backend interfaces for alarm and auth bucket
2021-07-23 15:07:14 +02:00
Sergey Kacheev
baf594b24a etcdserver: add zap.AddCallerSkip in NewRaftLoggerZap
The file `zap_raft.go` adds the raft.Logger proxy logger on top of `*zap.Logger`.
Adding a proxy requires adding the option `zap.AddCallerSkip(1)`,
so that the logging message specifies the correct caller,
two of the three constructors in the `zap_raft.go` adds this option.
This commit fixes the third constructor so that it also adds `zap.AddCallerSkip`.

Before fix:
`{"level":"info","ts":"2021-07-22T17:46:01.435Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"bd07d29169ff0c5a [logterm: 2, index: 8, vote: 38447ba545569bbe] ignored MsgPreVote from c7baeaad79d6d5ed [logterm: 2, index: 8] at term 2: lease is not expired (remaining ticks: 10)"}`

After fix:
`{"level":"info","ts":"2021-07-22T17:46:51.227Z","logger":"raft","caller":"raft/raft.go:859","msg":"bd07d29169ff0c5a [logterm: 2, index: 8, vote: c7baeaad79d6d5ed] ignored MsgPreVote from 38447ba545569bbe [logterm: 2, index: 8] at term 2: lease is not expired (remaining ticks: 9)"}`
2021-07-23 00:36:15 +07:00
Sergey Kacheev
daf7e0350a etcdserver: remove code duplication from the peer.send func
During the refactoring process, duplicate logging
of the send buffer overflow event was added.
Each of these log lines logs exactly the same information, the logging
context is sufficient to distinguish the cause.
Additionally, the unnecessary context (in parentheses) in the log
message was removed, which was necessary without the zap context (with
the old logger), but now only confuses.
2021-07-22 23:15:20 +07:00
Marek Siarkowicz
bc16461995 server: Use zaptest in bucket tests and move backendMock to separate file 2021-07-20 18:12:02 +02:00
Marek Siarkowicz
a0554a6bd3 etcdserver: Create AuthBackend interface 2021-07-20 18:09:53 +02:00
Marek Siarkowicz
6cd3633543 etcdserver: Rename membershipStore to membershipBackend 2021-07-20 17:56:52 +02:00
Marek Siarkowicz
2f31cc3fbc etcdserver: Create AlarmBackend interface 2021-07-20 17:53:44 +02:00
Marek Siarkowicz
a97e48e08d Cleanup references to bucket module 2021-07-20 17:50:47 +02:00
Sergey Kacheev
b922a28622 netutil: add url comparison without resolver to URLStringsEqual
If one of the nodes in the cluster has lost a dns record,
restarting the second node will break it.
This PR makes an attempt to add a comparison without using a resolver,
which allows to protect cluster from dns errors and does not break
the current logic of comparing urls in the URLStringsEqual function.
You can read more in the issue #7798

Fixes #7798
2021-07-19 01:31:21 +07:00