Mirroristas/etcd
2
0
Fork 0
mirror of https://github.com/etcd-io/etcd.git synced 2024-09-27 06:25:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
17,432 Commits 15 Branches 491 Tags 148 MiB
Commit Graph

61 Commits

Author SHA1 Message Date
Piotr Tabor
9c28e07588
Merge pull request #13447 from Juneezee/deprecate-ioutil 
*: move from io/ioutil to io and os packages
 2021-10-29 23:18:36 +02:00
Sam Batschelet
a0bdfc4fc9 tests/integration/clientv3: allow time for endpoints to update 
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
 2021-10-27 17:26:30 -04:00
Eng Zer Jun
2a151c8982
*: move from io/ioutil to io and os packages 
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2021-10-28 00:05:28 +08:00
Marek Siarkowicz
ca9b720c1d tests: Move integration setup to separa framework package 2021-10-20 10:34:35 +02:00
Marek Siarkowicz
5364758181 tests: Cleanup member interface by exposing Bridge directly 2021-09-29 12:42:16 +02:00
Marek Siarkowicz
451eb5d711 tests: Make using bridge optional 2021-09-29 12:42:14 +02:00
Marek Siarkowicz
f2dd5d80a1 tests: Rename grpcAddr to grpcURL to imply that it includes schema 2021-09-28 16:45:06 +02:00
Eduardo Patrocinio
87f1dc7e40 Fix a few typos 2021-09-03 16:09:09 -04:00
Marek Siarkowicz
5e40a8b00c server: Create storage package and move mvcc files to it 2021-07-12 15:37:21 +02:00
Marek Siarkowicz
e21cf4ef02 tkest: Prevent TestMaintenanceSnapshotWithVersionVersion & TestSaveSnapshotVersion from flaking 
When running 100 times in row those tests flaked around 10-20%. Based on
some experimentation 10 keys was enough to ensure that wal snapshot is
created and prevented any flakes.
 2021-07-09 13:38:40 +02:00
Marek Siarkowicz
e2740b4afa server,etcdutl: Preserve etcd version in backend allowing etcdutl to read it from snapshot 2021-06-25 14:06:56 +02:00
Marek Siarkowicz
e1b1d93548 *: Snapshot returns local etcd version 
Co-authored-by: Lili Cosic <cosiclili@gmail.com>
 2021-06-14 16:36:50 +02:00
Piotr Tabor
f15e0b8237 integration.BeforeTest can be run without leak-detection. 2021-05-27 20:55:11 +02:00
Pavan BG
d9c5e1f0a2 *: Replace internal testutil AssertEqual function in favour of github.com/stretchr/testify/assert.Equals 
To maintain uniformity in testing methodology, assert.Equals has been used everywhere while replacing testutil.AssertEqual

Fixes #13015
 2021-05-20 19:55:45 +05:30
Piotr Tabor
3dd577ec08 Simplify client's logger configuration in tests. 2021-05-18 13:03:16 +02:00
Marek Siarkowicz
06afe87b34 client: Allow setting zap logger in config 2021-05-12 12:15:20 +02:00
Piotr Tabor
a78d072b9a Simplify KVstore dependency on cindex. 2021-05-04 18:21:24 +02:00
Piotr Tabor
3acf6e35b2 Finish migration zap.NewExample -> zaptest.NewLogger(t) in integration tests 2021-04-29 22:32:26 +02:00
Piotr Tabor
ffea1537d4 ClientV3 tests use integration.NewClient that configures proper logger. 2021-04-29 18:18:34 +02:00
Piotr Tabor
9a4b2bdccc Errors: context cancelled or context deadline exceeded are exposed as codes.Canceled, codes.DeadlineExceeded instead of 'codes.Unknown' 2021-04-22 14:35:24 +02:00
Piotr Tabor
b1c04ce043 Applying consistency fix: ClusterVersionSet (and co) might get no applied on v2store 
ClusterVersionSet, ClusterMemberAttrSet, DowngradeInfoSet functions are
writing both to V2store and backend. Prior this CL there were
in a branch not executed if shouldApplyV3 was false,
e.g. during restore when Backend is up-to-date (has high
consistency-index) while v2store requires replay from WAL log.

The most serious consequence of this bug was that v2store after restore
could have different index (revision) than the same exact store before restore,
so potentially different content between replicas.

Also this change is supressing double-applying of Membership
(ClusterConfig) changes on Backend (store v3) - that lackilly are not
part of MVCC/KeyValue store, so they didn't caused Revisions to be
bumped.

Inspired by jingyih@ comment:
https://github.com/etcd-io/etcd/pull/12820#issuecomment-815299406
 2021-04-12 09:43:48 +02:00
Piotr Tabor
e776efbb2a
Merge pull request #12828 from ptabor/20210404-embed-etcd 
embed: etcd.Close() is closing Errc() channel as well.
 2021-04-08 01:20:07 +02:00
Piotr Tabor
5da9cac193 embed: etcd.Close() is closing Errc() channel as well. 
Inspired by https://github.com/etcd-io/etcd/pull/9612 by purpleidea@.
 2021-04-08 01:19:13 +02:00
Piotr Tabor
3bb7acc8cf Migrate dependencies pkg/foo -> client/pkg/foo 2021-04-07 00:38:47 +02:00
Piotr Tabor
f290ab2e60 Update dependecies: 
github.com/grpc-ecosystem/grpc-gateway v1.14.6 -> grpc-gateway v1.16.0
  golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e->v0.0.0-20210220033141-f8bda1e9f3ba
 2021-03-27 20:48:33 +01:00
Piotr Tabor
d0962f10ae Adopt grpc_testing to grpc-1.36. 2021-03-24 23:31:03 +01:00
Piotr Tabor
8469108548
Merge pull request #12782 from ptabor/20210316-fixes 
Integration: Test flakiness fixes
 2021-03-19 20:10:11 +01:00
Piotr Tabor
18321a0c85
Merge pull request #12786 from ptabor/20210318-debug-env 
client: Bring back ETCD_CLIENT_DEBUG variable interpretation.
 2021-03-18 21:55:51 +01:00
Piotr Tabor
9312d1b077 clientv3: Bring back ETCD_CLIENT_DEBUG variable interpretation. 
env ETCD_CLIENT_DEBUG supports log levels (debug, info, warn, error, dpanic, panic, fatal).
Only when set, overrides application-wide grpc logging settings.
 2021-03-18 18:43:07 +01:00
wpedrak
dac6e37ea1 *: over 20 staticcheck fixes 2021-03-18 15:06:17 +01:00
Piotr Tabor
725a8c5e02 Enable configuring delegated zap-logging for embed server. 2021-03-17 08:17:36 +01:00
Piotr Tabor
809e7629ed Add integration.BeforeTest to all missing tests. 2021-03-16 23:50:01 +01:00
Piotr Tabor
a57e967d84 Integration test flakes fixes. 2021-03-16 16:08:18 +01:00
Piotr Tabor
5ddabfdb24 tests: Make tests operate in /tmp director instead of src. 
Thanks to this, unix sockets should be not longer
created by integration tests in the the source code directory,
so potentially trigger IDE reloads and unnecessery load (and mess).
 2021-03-09 18:19:52 +01:00
Piotr Tabor
fb1d48e98e Integration tests: Use BeforeTest(t) instead of defer AfterTest(). 
Thanks to this change, a single method BeforeTest(t) can handle
before-test logic as well as registration of cleanup code
(t.Cleanup(func)).
 2021-03-09 18:19:51 +01:00
Piotr Tabor
87258efd90 Integration tests: Use zaptest.Logger based testing.TB 
Thanks to this the logs:
  - are automatically printed if the test fails.
  - are in pretty consistent format.
  - are annotated by 'member' information of the cluster emitting them.

Side changes:
  - Set propert default got DefaultWarningApplyDuration (used to be '0')
  - Name the members based on their 'place' on the list (as opposed to
'random')
 2021-03-09 18:19:51 +01:00
Piotr Tabor
5e10d12996 tests: Fixes a few recently spotted test-flakes 
```
Unexpected goroutines running after all test(s).
1 instances of:
syscall.Syscall(...)
	/usr/local/go/src/syscall/asm_linux_386.s:19 +0x5
syscall.Close(...)
	/usr/local/go/src/syscall/zsyscall_linux_386.go:285 +0x3d
internal/poll.(*FD).destroy(...)
	/usr/local/go/src/internal/poll/fd_unix.go:77 +0x30
internal/poll.(*FD).decref(...)
	/usr/local/go/src/internal/poll/fd_mutex.go:213 +0x38
internal/poll.(*FD).Close(...)
	/usr/local/go/src/internal/poll/fd_unix.go:99 +0x43
net.(*netFD).Close(...)
	/usr/local/go/src/net/fd_posix.go:37 +0x49
FAIL	go.etcd.io/etcd/tests/v3/integration/client	0.039s
```

```
--- FAIL: TestServer_TCP_Secure_DelayTx (0.20s)
    server_test.go:110: took 128.026085ms with no latency
    server_test.go:125: took 62.980988ms with latency 50ms��5ms
    server_test.go:133: expected took1 128.026085ms < took2 62.980988ms (with latency)
```

https://github.com/etcd-io/etcd/issues/12372
 2021-03-01 18:07:38 +01:00
Piotr Tabor
a7f340216d Reformat code according to 'gotip' rules. 
In practices adds annotations in the new syntax:
```
+//go:build !linux
 // +build !linux
```

Fixes failing gotip PASSES='fmt' check:
https://travis-ci.com/github/etcd-io/etcd/jobs/486453806
 2021-02-26 10:14:46 +01:00
Piotr Tabor
45b1e6b470 ClientV3: Ordering: Fix the ordering test such it does not fail. 
The test depended on very subtle timing semantic and on properties of
'copied' clients.

https://travis-ci.com/github/etcd-io/etcd/jobs/486191449

Examplar failure:
```
{"level":"warn","ts":"2021-02-25T12:34:47.894Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0000d6fc0/#initially=[unix://localhost:86269902489114839060]","attempt":1,"error":"rpc error: code = Unavailable desc = etcdserver: rpc not supported for learner"}
{"level":"warn","ts":"2021-02-25T12:34:48.163Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc00035a000/#initially=[unix://localhost:78285857058450835940]","attempt":0,"error":"rpc error: code = FailedPrecondition desc = etcdserver: not leader"}
{"level":"info","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = Canceled desc = context canceled"}
{"level":"info","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"info","ts":"2021-02-25T12:34:50.255Z","caller":"v3/maintenance.go:219","msg":"completed snapshot read; closing"}
{"level":"info","ts":"2021-02-25T12:34:51.717Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:52.017Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = Canceled desc = context canceled"}
{"level":"info","ts":"2021-02-25T12:34:52.018Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:53.018Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
--- FAIL: TestEndpointSwitchResolvesViolation (10.12s)
    ordering_util_test.go:81: failed to resolve order violation etcdclient: no cluster members have a revision higher than the previously received revision
```
 2021-02-25 22:15:13 +01:00
Piotr Tabor
1a9c81abda Update grpc dependency to 1.32. 
Simplify grpc testing infrastructure to align with upstream changes.
 2021-02-23 11:31:50 +01:00
Piotr Tabor
4a1c24556c clientv3: PS: Replace balancer with upstream grpc solution 
Addresses comments from: https://github.com/etcd-io/etcd/pull/12671#pullrequestreview-593942302
 2021-02-23 10:03:15 +01:00
Piotr Tabor
a836a8045b Get rid of legacy client/v3/naming API. 
Update grpcproxy to use the new abstractions.
 2021-02-09 11:56:28 +01:00
limeng01
8feb55f65c *: implement Endpoint Watch and new Resolver 2021-02-08 20:05:45 +08:00
limeng01
571ed502d4 endpoints: implement Update method for EndpointManager. 
- Add integration test for endpoints and resolver.
 2021-02-04 23:30:22 +08:00
Piotr Tabor
d6d03beaea
Merge pull request #12538 from lzhfromustc/12_9_GoroutineLeak 
test: change channel operations to avoid potential goroutine leaks
 2021-02-01 21:16:43 +01:00
Piotr Tabor
0d7a671c75 Tests: Fix vet warnings about Fatal in sub-goroutines. 
% (cd tests && go vet ./...)
stderr: # go.etcd.io/etcd/tests/v3/integration/clientv3/concurrency_test
stderr: integration/clientv3/concurrency/election_test.go:74:6: call to (*T).Fatal from a non-test goroutine
stderr: integration/clientv3/concurrency/mutex_test.go:57:4: call to (*T).Fatal from a non-test goroutine
 2021-01-31 00:00:26 +01:00
Piotr Tabor
5d7c1db3a9 Introduce grpc-1.30+ compatible client/v3/naming API. 
This is not yet implementation, just API and tests to be filled
with implementation in next CLs,
tracked by: https://github.com/etcd-io/etcd/issues/12652

We propose here 3 packages:
 - clientv3/naming/endpoints ->
    That is abstraction layer over etcd that allows to write, read &
    watch Endpoints information. It's independent from GRPC API. It hides
    the storage details.

 - clientv3/naming/endpoints/internal ->
    That contains the grpc's compatible Update class to preserve the
    internal JSON mashalling format.

 - clientv3/naming/resolver ->
   That implements the GRPC resolver API, such that etcd can be
   used for connection.Dial in grpc.

Please see the grpc_naming.md document changes & grpcproxy/cluster.go
new integration, to see how the new abstractions work.
 2021-01-30 12:32:19 +01:00
Piotr Tabor
351bdb33c5 Split intengration/clientv3 tests into multiple packages 
They used to take >10min with coverage, so were causing interrupted
Travis runs.

Know thay fit in 100-150s (together), thanks also to parallel
execution.
 2021-01-23 11:12:44 +01:00
lzhfromustc
f2a912a4e6 test: change channel operations to avoid potential goroutine leaks 
In these unit tests, goroutines may leak if certain branches are chosen. This commit edits channel operations and buffer sizes, so no matter what branch is chosen, the test will end correctly. This commit doesn't change the semantics of unit tests.
 2020-12-09 22:23:21 -05:00
Piotr Tabor
aaf423e962 server: Update imports. 
find -name '*.go' | xargs sed -i --follow-symlinks 's|etcd/v3/|etcd/server/v3/|g'
 2020-10-26 13:02:32 +01:00