589 Commits

Author SHA1 Message Date
Kafuu Chino
ed10ca13f4 *: avoid closing a watch with ID 0 incorrectly
Signed-off-by: Kafuu Chino <KafuuChinoQ@gmail.com>

add test

1

1

1
2022-10-10 19:54:58 +08:00
Hitoshi Mitake
91365174b3 tests: a test case for watch with auth token expiration
Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com>
2022-10-04 22:55:36 +09:00
Benjamin Wang
b8bea91f22 fix the flaky test TestV3AuthRestartMember
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-09 09:37:25 +08:00
vivekpatani
c0ef7d52e0 server,test: refresh cache on each NewAuthStore
- permissions were incorrectly loaded on restarts.
- #14355
- Backport of https://github.com/etcd-io/etcd/pull/14358

Signed-off-by: vivekpatani <9080894+vivekpatani@users.noreply.github.com>
2022-08-31 13:08:11 -07:00
Benjamin Wang
e2b36f8879
Merge pull request #14253 from serathius/checkpoints-fix-3.4
[3.4] Checkpoints fix 3.4
2022-07-22 16:56:17 +08:00
Marek Siarkowicz
783e99cbfe Fix lease checkpointing tests by forcing a snapshot
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-07-22 10:28:44 +02:00
Marek Siarkowicz
8f4735dfd4 server: Require either cluster version v3.6 or --experimental-enable-lease-checkpoint-persist to persist lease remainingTTL
To avoid inconsistant behavior during cluster upgrade we are feature
gating persistance behind cluster version. This should ensure that
all cluster members are upgraded to v3.6 before changing behavior.

To allow backporting this fix to v3.5 we are also introducing flag
--experimental-enable-lease-checkpoint-persist that will allow for
smooth upgrade in v3.5 clusters with this feature enabled.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-07-22 10:28:29 +02:00
Chao Chen
96f69dee47 Backport: non mutating requests pass through quotaKVServer when NOSPACE
This is a backport of https://github.com/etcd-io/etcd/pull/13435 and is
part of the work for 3.4.20
https://github.com/etcd-io/etcd/issues/14232.

The original change had a second commit that modifies a changelog file.
The 3.4 branch does not include any changelog file, so that part was not
cherry-picked.

Local Testing:

- `make build`
- `make test`

Both succeed.

Signed-off-by: Ramsés Morales <ramses@gmail.com>
2022-07-21 15:06:09 -07:00
Michał Jasionowski
8d83691d53 etcdserver,integration: Store remaining TTL on checkpoint
To extend lease checkpointing mechanism to cases when the whole etcd
cluster is restarted.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-07-21 17:35:21 +02:00
Michał Jasionowski
a30aba8fc2 lease,integration: add checkpoint scheduling after leader change
Current checkpointing mechanism is buggy. New checkpoints for any lease
are scheduled only until the first leader change. Added fix for that
and a test that will check it.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-07-21 17:35:15 +02:00
Chao Chen
fd51434b54 backport 3.5: #13676 load all leases from backend
Signed-off-by: Chao Chen <chaochn@amazon.com>
2022-07-19 16:08:01 -07:00
Benjamin Wang
07d2b1d626 support linearizable renew lease for 3.4
Cherry pick https://github.com/etcd-io/etcd/pull/13932 to 3.4.

When etcdserver receives a LeaseRenew request, it may be still in
progress of processing the LeaseGrantRequest on exact the same
leaseID. Accordingly it may return a TTL=0 to client due to the
leaseID not found error. So the leader should wait for the appliedID
to be available before processing client requests.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-07-19 13:34:55 +08:00
Benjamin Wang
5b3f269159 replace all 3.4 certificates and keys with the files from 3.5
Fix the following error in integration pipeline,
```
=== RUN   TestTLSReloadCopy
    v3_grpc_test.go:1754: tls: failed to find any PEM data in key input
    v3_grpc_test.go:1754: tls: private key does not match public key
    v3_grpc_test.go:1754: tls: private key does not match public key
    v3_grpc_test.go:1754: tls: private key does not match public key
```

Refer to https://github.com/etcd-io/etcd/runs/7123775361?check_suite_focus=true

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-30 13:21:48 +08:00
Benjamin Wang
bb9113097a fix test failure in TestCtlV3WatchClientTLS
Also refer to the following commit in 3.5,
093282f5ea

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-30 10:19:03 +08:00
Benjamin Wang
6958ee8ff2 Skip WatchRequestProgress test in grpc-proxy mode.
We shouldn't fail the grpc-server (completely) by a not implemented RPC.
Failing whole server by remote request is anti-pattern and security
risk.

Refer to https://github.com/etcd-io/etcd/runs/7034342964?check_suite_focus=true#step:5:2284

```
=== RUN   TestWatchRequestProgress/1-watcher
panic: not implemented
goroutine 83024 [running]:
go.etcd.io/etcd/proxy/grpcproxy.(*watchProxyStream).recvLoop(0xc009232f00, 0x4a73e1, 0xc00e2406e0)
	/home/runner/work/etcd/etcd/proxy/grpcproxy/watch.go:265 +0xbf2
go.etcd.io/etcd/proxy/grpcproxy.(*watchProxy).Watch.func1(0xc0038a3bc0, 0xc009232f00)
	/home/runner/work/etcd/etcd/proxy/grpcproxy/watch.go:125 +0x70
created by go.etcd.io/etcd/proxy/grpcproxy.(*watchProxy).Watch
	/home/runner/work/etcd/etcd/proxy/grpcproxy/watch.go:123 +0x73b
FAIL	go.etcd.io/etcd/clientv3/integration	222.813s
FAIL
```

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-29 05:12:43 +08:00
Benjamin Wang
1c9fa07cd7 Fix deadlock in 'go test -tags cluster_proxy -v ./integration/... ./clientv3/...'
Cherry pick https://github.com/etcd-io/etcd/pull/12319 to 3.4.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-28 13:44:47 +08:00
Piotr Tabor
276ee962ec integration: Fix 'go test --tags cluster_proxy --timeout=30m -v ./integration/...'
grpc proxy opens additional 2 watching channels. The metric is shared
between etcd-server & grpc_proxy, so all assertions on number of open
watch channels need to take in consideration the additional "2"
channels.
2021-04-19 16:41:28 +02:00
Vimal K
df35086b6a integration : fix TestTLSClientCipherSuitesMismatch in go1.13
In go1.13, the TLS13 is enablled by default, and as per go1.13 release notes :
TLS 1.3 cipher suites are not configurable. All supported cipher suites are safe,
and if PreferServerCipherSuites is set in Config the preference order is based
on the available hardware.

Fixing the test case for go1.13 by limiting the TLS version to TLS12
2021-04-19 11:18:14 +02:00
Lili Cosic
35bd924596 integration,raft,tests: Comply with go v1.15 gofmt 2021-04-19 11:18:13 +02:00
Chris Wedgwood
c499d9b047 integration: relax leader timeout from 3s to 4s
The integration jobs fail with timeouts slightly over 3s, increase
this marginally so false failures are less prevalent.
2021-03-29 10:17:44 -07:00
jingyih
f44aaf8248 integration: add flag WatchProgressNotifyInterval in integration test 2020-09-09 12:39:42 -07:00
Jack Kleeman
36452a1c1d clientv3: cancel watches proactively on client context cancellation
Currently, watch cancel requests are only sent to the server after a
message comes through on a watch where the client has cancelled. This
means that cancelled watches that don't receive any new messages are
never cancelled; they persist for the lifetime of the client stream.
This has negative connotations for locking applications where a watch
may observe a key which might never change again after cancellation,
leading to many accumulating watches on the server.

By cancelling proactively, in most cases we simply move the cancel
request to happen earlier, and additionally we solve the case where the
cancel request would never be sent.

Fixes #9416
Heavy inspiration drawn from the solutions proposed there.
2020-06-23 19:50:21 +08:00
Jingyi Hu
478da3bf24 integration: disable TestV3AuthOldRevConcurrent
Disable TestV3AuthOldRevConcurrent for now. See
https://github.com/etcd-io/etcd/pull/10468#issuecomment-463253361
2019-10-28 15:03:44 -07:00
Gyuho Lee
2c36cab87d
Merge pull request #11223 from YoyinZyc/automated-cherry-pick-of-#11179-origin-release-3.4
Automated cherry pick of #11179
2019-10-09 13:28:04 -07:00
yoyinzyc
480d5510f9 etcdserver: trace compaction request; add return parameter 'trace' to applierV3.Compaction() mvcc: trace compaction request; add input parameter 'trace' to KV.Compact() 2019-10-09 12:40:12 -07:00
Joe Betz
0781c0327d
clientv3: Replace endpoint.ParseHostPort with net.SplitHostPort to fix IPv6 client endpoints 2019-10-08 18:27:03 -07:00
Joe Betz
1e213b7ab6 *: Add experimental-compaction-batch-limit flag
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-15 13:31:39 -07:00
Hanaasagi
8ff71c52db test: fix metric name typo 2019-08-09 13:24:27 -07:00
Gyuho Lee
1c8ab76333 integration: test snapshot inflights metrics
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-08 14:01:54 -07:00
Gyuho Lee
8a2a951d79 integration: match code.Canceled in "TestV3KVInflightRangeRequests"
Match new error codes in gRPC v1.22.0

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-26 09:19:22 -07:00
Jingyi Hu
fe86a786a4
Merge pull request #10904 from yuzeming/fixed-10899
integration: fix a data race about `i` and `tt` in TestV3WatchFromCur…
2019-07-19 17:51:21 -07:00
Zeming YU
5f21b557e5 integration: fix a data race about i and tt in TestV3WatchFromCurrentRevision
don't references to loop variables from within go anonymous function

Fixes etcd-io#10899
2019-07-19 00:48:49 -07:00
yzm
3737979532 move wg.Wait() after loop 2019-07-17 16:31:48 -07:00
yzm
d87bd2c87c integration: add WaitGroup to prevent calling t.Fatalf after TestV3WatchCurrentPutOverlap function return
It could cause a panic when it happens

Fixes #10886
2019-07-16 10:25:35 -07:00
Xiang Li
2c5162af5c
Merge pull request #10523 from jingyih/fully_concurrent_reads
mvcc: fully concurrent read
2019-06-14 12:25:17 +08:00
Xiang Li
9a73013004
Merge pull request #10797 from jingyih/lease_checkpoint_enabled_by_experimental_flag
*: enable lease checkpoint via experimental flag
2019-06-05 22:56:54 -07:00
Jingyi Hu
e67b9829b6 *: enable lease checkpoint via experimental flag
Primary lessor persist lease remainingTTL only if experimental flag
"--experimental-enable-lease-checkpoint" is set.
2019-06-05 15:30:03 -07:00
Gyuho Lee
1caaa9ed4a test: test update for Go 1.12.5 and related changes
Update to Go 1.12.5 testing. Remove deprecated unused and gosimple
pacakges, and mask staticcheck 1006. Also, fix unconvert errors related
to unnecessary type conversions and following staticcheck errors:
- remove redundant return statements
- use for range instead of for select
- use time.Since instead of time.Now().Sub
- omit comparison to bool constant
- replace T.Fatal and T.Fatalf in tests with T.Error and T.Fatalf respectively because the goroutine calls T.Fatal must be called in the same goroutine as the test
- fix error strings that should not be capitalized
- use sort.Strings(...) instead of sort.Sort(sort.StringSlice(...))
- use he status code of Canceled instead of grpc.ErrClientConnClosing which is deprecated
- use use status.Errorf instead of grpc.Errorf which is deprecated

Related #10528 #10438
2019-06-05 17:02:05 -04:00
Hitoshi Mitake
5a67dd788d *: support creating a user without password
This commit adds a feature for creating a user without password. The
purpose of the feature is reducing attack surface by configuring bad
passwords (CN based auth will be allowed for the user).

The feature can be used with `--no-password` of `etcdctl user add`
command.

Fix https://github.com/coreos/etcd/issues/9590
2019-05-30 21:59:30 +09:00
Jingyi Hu
6bf609b96d integration: update TestMemberPromote test
Update TestMemberPromote to include both learner not-ready and learner
ready test cases.

Removed unit test TestPromoteMember, it requires underlying raft node to
be started and running. The member promote is covered by the integration
test.
2019-05-28 18:50:13 -07:00
Jingyi Hu
d0c1b3fa38 etcdserver: learner return Unavailable for unsupported RPC
Make learner return code.Unavailable when the request is not supported
by learner. Client balancer will retry a different endpoint.
2019-05-28 18:47:03 -07:00
Gyuho Lee
34bd797e67 *: revert module import paths
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-05-28 15:39:35 -07:00
Jingyi Hu
ac057951cc integration: remove unnecessary type conversion
Fixes go 'unconvert' test.
2019-05-15 13:48:54 -07:00
Jingyi Hu
bd7f42855b integration: add TestTransferLeadershipWithLearner
Adding integration test TestTransferLeadershipWithLearner, which ensures
that TransferLeadership does not timeout due to learner is automatically
picked by leader as transferee.
2019-05-15 13:27:42 -07:00
Jingyi Hu
e8dc4c5c25 integration: add TestMoveLeaderToLearnerError
Adding integration test TestMoveLeaderToLearnerError, which ensures that
leader transfer to learner member will fail.
2019-05-15 13:27:42 -07:00
Jingyi Hu
57a11eb1e1 integration: add TestKVForLearner
Adding TestKVForLearner. Also adding test utility functions for clientv3
integration tests.
2019-05-15 13:27:38 -07:00
Jingyi Hu
a73fb85c0c mvcc: fully concurrent read 2019-05-08 19:11:23 -07:00
shivaramr
9150bf52d6 go modules: Fix module path version to include version number 2019-04-26 15:29:50 -07:00
caoming
97509833e2 integration: use cancel instead of close. 2019-03-14 11:14:43 +08:00
Gyuho Lee
dca0dec382 integration: use default log configuration
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-02-21 11:00:01 -08:00