595 Commits

Author SHA1 Message Date
Piotr Tabor
32bad8e9a8 integration/cluster_proxy.go: Fix broken build with tags=cov.
Prior to the fix:
```
go test -mod=mod -timeout 30m -tags cluster_proxy "./integration/..."
```

Was failing with:
```
integration/cluster_proxy.go:58:37: not enough arguments in call to grpcproxy.NewWatchProxy
   have (*clientv3.Client)
   want (*zap.Logger, *clientv3.Client)
FAIL    go.etcd.io/etcd/v3/integration [build failed]
```
2020-09-10 14:54:09 +02:00
Joe Betz
76e769ce95
Merge pull request #12273 from ptabor/2020-09-07-fix-grpc-proxy-tests
testing/e2e,grpcproxy: Fix: go test --tags "cluster_proxy" -v ./tests/e2e/...
2020-09-09 12:03:09 -07:00
jingyih
73817b53fd integration: add flag WatchProgressNotifyInterval in integration test 2020-09-07 08:32:54 -07:00
Piotr Tabor
2c93127c7b integration: Regenerate certificates, add client-nocn.crt
Executed:
(cd ./integration/fixtures && ./gencerts.sh)

This in particular cereated a new client-nocn.crt (and key) that can be
used for testing grpc-proxy -> etcd-server connections.
2020-09-07 11:48:38 +02:00
Piotr Tabor
966e8cecf0 integration: gencerts.sh cleanup and supports no-CN certs
integration/fixtures/gencerts.sh:
  - refactored common logic to a helper function
  - added definition for client-nocn certificate
    (used for grpc-proxy -> etcd-server) communication.
2020-09-07 11:47:24 +02:00
Gyuho Lee
7eac6bd497 *: upgrade zap logger to 1.15, replace global logger
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-08-12 09:50:00 -07:00
Piotr Tabor
830618e44d ./integration: Fix flakes due to .setupLogging race.
The source of problem was the fact that multiple tests were creating
their clusters (and some of them were setting global grpclog).
If the test was running after some other test that created HttpServer
(so accessed grpclog), this was reported as race.

Tested with:
  go test ./clientv3/. -v "--run=(Example).*" --count=2
  go test ./clientv3/. -v "--run=(Test).*" --count=2
  go test ./integration/embed/. -v "--run=(Test).*" --count=2
2020-08-07 13:54:41 +02:00
Jack Kleeman
ec13797407
mvcc: avoid negative watcher count metrics (#11882)
The watch count metrics are not robust to duplicate cancellations. These
cause the count to be decremented twice, leading eventually to negative
counts. We are seeing this in production. The duplicate cancellations
themselves are not themselves a big problem (except performance), but
they are caused by the new proactive cancellation logic (#11850) which
cancels proactively even immediately before initiating a Close, thus
nearly guaranteeing a Close-cancel race, as discussed in
watchable_store.go. We can avoid this in most cases by not sending a
cancellation when we are going to Close.
2020-05-12 16:50:53 -07:00
Jack Kleeman
87aa5a9e60 clientv3: cancel watches proactively on client context cancellation
Currently, watch cancel requests are only sent to the server after a
message comes through on a watch where the client has cancelled. This
means that cancelled watches that don't receive any new messages are
never cancelled; they persist for the lifetime of the client stream.
This has negative connotations for locking applications where a watch
may observe a key which might never change again after cancellation,
leading to many accumulating watches on the server.

By cancelling proactively, in most cases we simply move the cancel
request to happen earlier, and additionally we solve the case where the
cancel request would never be sent.

Fixes #9416
Heavy inspiration drawn from the solutions proposed there.
2020-05-07 16:50:26 +01:00
Brandon Philips
c9cdefaeac *: run gofmt and make fmt make test pass
gofmt tests were failing, fix.
2020-04-28 00:57:52 +00:00
Brandon Philips
96cce208c2 go.mod: use go.etcd.io/etcd/v3 versioning
This change makes the etcd package compatible with the existing Go
ecosystem for module versioning.

Used this tool to update package imports:
  https://github.com/KSubedi/gomove
2020-04-28 00:57:35 +00:00
Gyuho Lee
3ac7a11515
Merge pull request #11699 from tangcong/refactor-consistentindex
*: refactor consistentindex
2020-03-25 18:07:44 -07:00
tangcong
7b2018683a *: refactor consistent index 2020-03-25 10:59:15 +08:00
Gyuho Lee
33907477dd *: add "etcd_server_client_requests_total", tests
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-17 12:16:11 -07:00
Jingyi Hu
84ace97ad7
Merge pull request #11572 from lzhfromustc/API_Fatal_4
integration/clientv3: fix 4 API misusage in test functions
2020-02-14 00:27:41 -08:00
Ziheng Liu
f00394e384 integration/clientv3: fix 4 API misusage in test functions 2020-02-13 13:13:18 -05:00
Jingyi Hu
bb29615359
proxy: remove capnslog (#11614)
* proxy: remove capnslog

* CHANGELOG: function signature change
2020-02-12 10:30:09 -08:00
jingyih
725e09023a *: set zap as default logger, remove capnslog
Set zap as default logger. Remove capnslog and deprecated logging
flags.
2020-02-04 04:57:49 -08:00
Fansong Zeng
d70251835d integration: gracefully shut down gRPC server (#11437) 2019-12-11 13:07:16 -08:00
Gyuho Lee
9f81002a11
Merge pull request #11427 from YoyinZyc/migration-cluster-attr
Migrate cluster attributes to use v3 backend
2019-12-09 13:30:15 -08:00
Jingyi Hu
ed5a01a48d etcdserver: recover cluster version from backend 2019-12-05 16:25:13 -08:00
lzhfromustc
1f8764be3b integration: prevent goroutines leaks in test (#11318)
Some goroutines in test functions will be leaked in certain cases.
This patch stops these leaks no matter what happens in test, by
putting the blocking channel in select together with a new stopc
channel, or adding 1 buffer to the blocking channel.
2019-12-05 15:40:10 -08:00
Hitoshi Mitake
84e2788c2e
Merge pull request #10468 from jingyih/remove_auth_loop
etcdserver: remove infinite loop for auth in raftRequest
2019-10-29 00:11:40 +09:00
Jingyi Hu
41539df490 integration: disable TestV3AuthOldRevConcurrent
Disable TestV3AuthOldRevConcurrent for now. See
https://github.com/etcd-io/etcd/pull/10468#issuecomment-463253361
2019-10-25 18:39:07 -07:00
Joe Betz
6aae90325c
Merge pull request #11211 from jpbetz/ipv6-endpoints
Replace endpoint.ParseHostPort with net.SplitHostPort to fix IPv6 client endpoints
2019-10-08 15:15:16 -07:00
Joe Betz
594354b886
Replace endpoint.ParseHostPort with net.SplitHostPort to fix IPv6 client endpoints 2019-10-08 14:53:49 -07:00
Gyuho Lee
340f0ac797
Merge pull request #11179 from YoyinZyc/trace
Add tracing to range request in etcd server.
2019-10-08 13:23:53 -07:00
yoyinzyc
57aa68af5a etcdserver: trace compaction request; add return parameter 'trace' to applierV3.Compaction()
mvcc: trace compaction request; add input parameter 'trace' to KV.Compact()
2019-10-07 09:55:27 -07:00
vimalk78
04ddfa8b8d clientv3/concurrency: Added Mutex.TryLock()
TryLock locks the mutex if not already locked by another session.
If lock is held by another session, return immediately after attempting necessary cleanup

Added integration test

Fixes #10493
2019-09-09 20:16:07 -07:00
Gyuho Lee
a3f7202c5f *: use TLS.Config.MaxVersion to TLS 1.2
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-06 11:38:03 -07:00
Vimal K
2648a36f8c integration : fix TestTLSClientCipherSuitesMismatch in go1.13
In go1.13, the TLS13 is enablled by default, and as per go1.13 release notes :
TLS 1.3 cipher suites are not configurable. All supported cipher suites are safe,
and if PreferServerCipherSuites is set in Config the preference order is based
on the available hardware.

Fixing the test case for go1.13 by limiting the TLS version to TLS12
2019-09-06 01:12:32 +01:00
Joe Betz
9b51febaf5 *: Add experimental-compaction-batch-limit flag 2019-08-15 11:47:23 -07:00
Gyuho Lee
4a4629fd9f
Merge pull request #10957 from Hanaasagi/fix-metric-name-typo
test: fix metric name typo
2019-08-09 13:23:26 -07:00
Gyuho Lee
a4badc33a3 integration: test snapshot inflights metrics
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-08 13:33:52 -07:00
Hanaasagi
3ef1683bab
test: fix metric name typo 2019-07-31 12:43:46 +09:00
Gyuho Lee
8a2a951d79 integration: match code.Canceled in "TestV3KVInflightRangeRequests"
Match new error codes in gRPC v1.22.0

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-26 09:19:22 -07:00
Jingyi Hu
fe86a786a4
Merge pull request #10904 from yuzeming/fixed-10899
integration: fix a data race about `i` and `tt` in TestV3WatchFromCur…
2019-07-19 17:51:21 -07:00
Zeming YU
5f21b557e5 integration: fix a data race about i and tt in TestV3WatchFromCurrentRevision
don't references to loop variables from within go anonymous function

Fixes etcd-io#10899
2019-07-19 00:48:49 -07:00
yzm
3737979532 move wg.Wait() after loop 2019-07-17 16:31:48 -07:00
yzm
d87bd2c87c integration: add WaitGroup to prevent calling t.Fatalf after TestV3WatchCurrentPutOverlap function return
It could cause a panic when it happens

Fixes #10886
2019-07-16 10:25:35 -07:00
Xiang Li
2c5162af5c
Merge pull request #10523 from jingyih/fully_concurrent_reads
mvcc: fully concurrent read
2019-06-14 12:25:17 +08:00
Xiang Li
9a73013004
Merge pull request #10797 from jingyih/lease_checkpoint_enabled_by_experimental_flag
*: enable lease checkpoint via experimental flag
2019-06-05 22:56:54 -07:00
Jingyi Hu
e67b9829b6 *: enable lease checkpoint via experimental flag
Primary lessor persist lease remainingTTL only if experimental flag
"--experimental-enable-lease-checkpoint" is set.
2019-06-05 15:30:03 -07:00
Gyuho Lee
1caaa9ed4a test: test update for Go 1.12.5 and related changes
Update to Go 1.12.5 testing. Remove deprecated unused and gosimple
pacakges, and mask staticcheck 1006. Also, fix unconvert errors related
to unnecessary type conversions and following staticcheck errors:
- remove redundant return statements
- use for range instead of for select
- use time.Since instead of time.Now().Sub
- omit comparison to bool constant
- replace T.Fatal and T.Fatalf in tests with T.Error and T.Fatalf respectively because the goroutine calls T.Fatal must be called in the same goroutine as the test
- fix error strings that should not be capitalized
- use sort.Strings(...) instead of sort.Sort(sort.StringSlice(...))
- use he status code of Canceled instead of grpc.ErrClientConnClosing which is deprecated
- use use status.Errorf instead of grpc.Errorf which is deprecated

Related #10528 #10438
2019-06-05 17:02:05 -04:00
Hitoshi Mitake
5a67dd788d *: support creating a user without password
This commit adds a feature for creating a user without password. The
purpose of the feature is reducing attack surface by configuring bad
passwords (CN based auth will be allowed for the user).

The feature can be used with `--no-password` of `etcdctl user add`
command.

Fix https://github.com/coreos/etcd/issues/9590
2019-05-30 21:59:30 +09:00
Jingyi Hu
6bf609b96d integration: update TestMemberPromote test
Update TestMemberPromote to include both learner not-ready and learner
ready test cases.

Removed unit test TestPromoteMember, it requires underlying raft node to
be started and running. The member promote is covered by the integration
test.
2019-05-28 18:50:13 -07:00
Jingyi Hu
d0c1b3fa38 etcdserver: learner return Unavailable for unsupported RPC
Make learner return code.Unavailable when the request is not supported
by learner. Client balancer will retry a different endpoint.
2019-05-28 18:47:03 -07:00
Gyuho Lee
34bd797e67 *: revert module import paths
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-05-28 15:39:35 -07:00
Jingyi Hu
ac057951cc integration: remove unnecessary type conversion
Fixes go 'unconvert' test.
2019-05-15 13:48:54 -07:00
Jingyi Hu
bd7f42855b integration: add TestTransferLeadershipWithLearner
Adding integration test TestTransferLeadershipWithLearner, which ensures
that TransferLeadership does not timeout due to learner is automatically
picked by leader as transferee.
2019-05-15 13:27:42 -07:00