Mirroristas/etcd
2
0
Fork 0
mirror of https://github.com/etcd-io/etcd.git synced 2024-09-27 06:25:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
20,479 Commits 15 Branches 491 Tags 148 MiB
Commit Graph

1488 Commits

Author SHA1 Message Date
Piotr Tabor
b1c04ce043 Applying consistency fix: ClusterVersionSet (and co) might get no applied on v2store 
ClusterVersionSet, ClusterMemberAttrSet, DowngradeInfoSet functions are
writing both to V2store and backend. Prior this CL there were
in a branch not executed if shouldApplyV3 was false,
e.g. during restore when Backend is up-to-date (has high
consistency-index) while v2store requires replay from WAL log.

The most serious consequence of this bug was that v2store after restore
could have different index (revision) than the same exact store before restore,
so potentially different content between replicas.

Also this change is supressing double-applying of Membership
(ClusterConfig) changes on Backend (store v3) - that lackilly are not
part of MVCC/KeyValue store, so they didn't caused Revisions to be
bumped.

Inspired by jingyih@ comment:
https://github.com/etcd-io/etcd/pull/12820#issuecomment-815299406
 2021-04-12 09:43:48 +02:00
wpedrak
08ea9cb756 etcdserver: integration test covering first commit in current term notification 2021-04-09 16:05:02 +02:00
Piotr Tabor
63c25bf378
Merge pull request #12804 from ptabor/20210326-v3-publish 
server: v2store deprecation: Prepare to use publishV3 instead of publish V2.
 2021-04-08 09:22:22 +02:00
Piotr Tabor
e776efbb2a
Merge pull request #12828 from ptabor/20210404-embed-etcd 
embed: etcd.Close() is closing Errc() channel as well.
 2021-04-08 01:20:07 +02:00
Piotr Tabor
5da9cac193 embed: etcd.Close() is closing Errc() channel as well. 
Inspired by https://github.com/etcd-io/etcd/pull/9612 by purpleidea@.
 2021-04-08 01:19:13 +02:00
Piotr Tabor
d4a8093ea5 Switch release-test (upgrade test) to use etcd 3.4 (instead of 3.3) as upgrade-base. 2021-04-08 01:15:16 +02:00
Piotr Tabor
931af493cf Merge pull request #12830 from ptabor/20210405-split-pkg 
Split client/pkg as dedicated low-dependencies module for client
 2021-04-08 01:12:17 +02:00
Piotr Tabor
816d332d81
Merge pull request #12830 from ptabor/20210405-split-pkg 
Split client/pkg as dedicated low-dependencies module for client
 2021-04-08 00:48:41 +02:00
Piotr Tabor
3bb7acc8cf Migrate dependencies pkg/foo -> client/pkg/foo 2021-04-07 00:38:47 +02:00
garenchan
c047ed593c etcdctl: lock return exit code of exec-command 
Sometimes we expect to get the exit code of the command being
executed.
 2021-04-06 14:34:31 +08:00
Sam Batschelet
a40a6e9ad8
Merge pull request #12805 from ptabor/20210326-minor-test-fixes 
tests: logging & temp-dir fixes
 2021-03-28 19:14:13 -04:00
Piotr Tabor
f290ab2e60 Update dependecies: 
github.com/grpc-ecosystem/grpc-gateway v1.14.6 -> grpc-gateway v1.16.0
  golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e->v0.0.0-20210220033141-f8bda1e9f3ba
 2021-03-27 20:48:33 +01:00
Piotr Tabor
c49807f59e Update cmux to 1.5.0. 
Executed using:
```
./scripts/update_dep.sh github.com/soheilhy/cmux v0.1.5
```
 2021-03-27 11:18:13 +01:00
Piotr Tabor
03f55eeb2c Make NewTmpBackend use testing tmp location (so cleanup). 2021-03-26 13:54:55 +01:00
Piotr Tabor
d0962f10ae Adopt grpc_testing to grpc-1.36. 2021-03-24 23:31:03 +01:00
Piotr Tabor
a60676686b Update dep: grpc: 1.32.0 -> 1.36.0. 2021-03-24 22:27:55 +01:00
Piotr Tabor
45fb7b41d6 Update dep: github.com/golang/protobuf v1.3.5 ->  v1.5.1 
Thanks to https://go-review.googlesource.com/c/protobuf/+/300869/ its
feasible now.
 2021-03-24 22:27:12 +01:00
lzhfromustc
3de6b382df server&tests: Fix goroutine leaks 2021-03-23 12:52:59 -04:00
Piotr Tabor
8469108548
Merge pull request #12782 from ptabor/20210316-fixes 
Integration: Test flakiness fixes
 2021-03-19 20:10:11 +01:00
Piotr Tabor
18321a0c85
Merge pull request #12786 from ptabor/20210318-debug-env 
client: Bring back ETCD_CLIENT_DEBUG variable interpretation.
 2021-03-18 21:55:51 +01:00
Piotr Tabor
9312d1b077 clientv3: Bring back ETCD_CLIENT_DEBUG variable interpretation. 
env ETCD_CLIENT_DEBUG supports log levels (debug, info, warn, error, dpanic, panic, fatal).
Only when set, overrides application-wide grpc logging settings.
 2021-03-18 18:43:07 +01:00
wpedrak
dac6e37ea1 *: over 20 staticcheck fixes 2021-03-18 15:06:17 +01:00
Piotr Tabor
725a8c5e02 Enable configuring delegated zap-logging for embed server. 2021-03-17 08:17:36 +01:00
Piotr Tabor
6657d5907c Make sure all integration tests have BeforeTest. 
The CL disallows to create NewCluster in tests without BeforeTest.
 2021-03-16 23:50:01 +01:00
Piotr Tabor
809e7629ed Add integration.BeforeTest to all missing tests. 2021-03-16 23:50:01 +01:00
Piotr Tabor
a84bd093b0 Integration with grpc-settable logger. 2021-03-16 22:50:41 +01:00
Piotr Tabor
a57e967d84 Integration test flakes fixes. 2021-03-16 16:08:18 +01:00
Piotr Tabor
a47c18d30a Fix 2 remaining 'defer AfterTest' calls. 2021-03-13 23:41:29 +01:00
Piotr Tabor
b406647dd7 Fix/remove broken: TestMetricDbSizeDefragDebugging 2021-03-12 23:05:53 +01:00
Gyuho Lee
4eba403ccc
Merge pull request #12765 from ptabor/20210312-move-config 
Move config (ServerConfig) out of etcdserver package.
 2021-03-11 15:05:29 -08:00
Piotr Tabor
fd7fed1511 Move config (ServerConfig) out of etcdserver package. 
Motivation:
  - ServerConfig is part of 'embed' public API, while etcdserver is more 'internal'
  - EtcdServer is already too big and config is pretty wide-spread leaf
if we were to split etcdserver (e.g. into pre & post-apply part).
 2021-03-11 20:56:22 +01:00
Piotr Tabor
b9226d03f4
Merge pull request #12763 from hexfusion/bump-proto 
vendor: bump gogo/proto to v1.3.2
 2021-03-11 17:59:07 +01:00
Sam Batschelet
d3aa3fb486 vendor: bump gogo/proto to v1.3.2 
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
 2021-03-11 11:27:25 -05:00
Piotr Tabor
c8243a9927 Tests: Functional - in case of failure, log the exception. 2021-03-09 18:19:52 +01:00
Piotr Tabor
b6c2e87a74 Testing: Integration tests does not check whether t==nil 2021-03-09 18:19:52 +01:00
Piotr Tabor
5ddabfdb24 tests: Make tests operate in /tmp director instead of src. 
Thanks to this, unix sockets should be not longer
created by integration tests in the the source code directory,
so potentially trigger IDE reloads and unnecessery load (and mess).
 2021-03-09 18:19:52 +01:00
Piotr Tabor
bfe02c0526 tests: Cluster creation that failed shouldn't leak goroutines. 2021-03-09 18:19:52 +01:00
Piotr Tabor
41f6cc7234 Tests: Better isolation between store_v2v3 integration tests. 2021-03-09 18:19:51 +01:00
Piotr Tabor
fb1d48e98e Integration tests: Use BeforeTest(t) instead of defer AfterTest(). 
Thanks to this change, a single method BeforeTest(t) can handle
before-test logic as well as registration of cleanup code
(t.Cleanup(func)).
 2021-03-09 18:19:51 +01:00
Piotr Tabor
87258efd90 Integration tests: Use zaptest.Logger based testing.TB 
Thanks to this the logs:
  - are automatically printed if the test fails.
  - are in pretty consistent format.
  - are annotated by 'member' information of the cluster emitting them.

Side changes:
  - Set propert default got DefaultWarningApplyDuration (used to be '0')
  - Name the members based on their 'place' on the list (as opposed to
'random')
 2021-03-09 18:19:51 +01:00
Piotr Tabor
a46a358577 --experimental-memory-mlock support 
The flag protects etcd memory from being swapped out to disk.
This can happen in memory constrained systems where mmaped bbolt
area is natural condidate for swapping out.

This flag should provide better tail latency on the cost of higher RSS
ram usage. If the experiment is successful, the logic should get moved
into bbolt layer, where we can protect specific bbolt instances
(e.g. avoid protecting both during defragmentation).
 2021-03-07 12:32:57 +01:00
Piotr Tabor
f7a2389992 Update version of certifi/gocertifi to get rid of WTF Public license 
Seems old versions of https://github.com/certifi/gocertifi where
categorized as "Do What The F*ck You Want To Public License".

Update to newer version that is explicit `Mozilla Public License` 2.0 (MPL 2.0).
 2021-03-04 09:48:34 +01:00
Piotr Tabor
102c198444
Merge pull request #12705 from astromechza/bm_etcd_peer_server_cert 
etcdmain: added peer-client-{client,key}-file parameters for supporting separate client and server certs when communicating between peers
 2021-03-02 09:03:35 +01:00
Piotr Tabor
5e10d12996 tests: Fixes a few recently spotted test-flakes 
```
Unexpected goroutines running after all test(s).
1 instances of:
syscall.Syscall(...)
	/usr/local/go/src/syscall/asm_linux_386.s:19 +0x5
syscall.Close(...)
	/usr/local/go/src/syscall/zsyscall_linux_386.go:285 +0x3d
internal/poll.(*FD).destroy(...)
	/usr/local/go/src/internal/poll/fd_unix.go:77 +0x30
internal/poll.(*FD).decref(...)
	/usr/local/go/src/internal/poll/fd_mutex.go:213 +0x38
internal/poll.(*FD).Close(...)
	/usr/local/go/src/internal/poll/fd_unix.go:99 +0x43
net.(*netFD).Close(...)
	/usr/local/go/src/net/fd_posix.go:37 +0x49
FAIL	go.etcd.io/etcd/tests/v3/integration/client	0.039s
```

```
--- FAIL: TestServer_TCP_Secure_DelayTx (0.20s)
    server_test.go:110: took 128.026085ms with no latency
    server_test.go:125: took 62.980988ms with latency 50ms��5ms
    server_test.go:133: expected took1 128.026085ms < took2 62.980988ms (with latency)
```

https://github.com/etcd-io/etcd/issues/12372
 2021-03-01 18:07:38 +01:00
Ben Meier
3d44f5bf80
*: added client-{client,key}-file parameters for supporting separate client and server certs when communicating between peers 
In some environments, the CA is not able to sign certificates with both
'client auth' and 'server auth' extended usage parameters and so an operator
needs to be able to set a seperate client certificate to use when making
requests which is different to the certificate used for accepting requests.
This applies to both proxy and etcd member mode and is available as both a CLI
 flag and config file field for peer TLS.

Signed-off-by: Ben Meier <ben.meier@oracle.com>
 2021-02-28 14:37:56 +00:00
Piotr Tabor
a7f340216d Reformat code according to 'gotip' rules. 
In practices adds annotations in the new syntax:
```
+//go:build !linux
 // +build !linux
```

Fixes failing gotip PASSES='fmt' check:
https://travis-ci.com/github/etcd-io/etcd/jobs/486453806
 2021-02-26 10:14:46 +01:00
Piotr Tabor
45b1e6b470 ClientV3: Ordering: Fix the ordering test such it does not fail. 
The test depended on very subtle timing semantic and on properties of
'copied' clients.

https://travis-ci.com/github/etcd-io/etcd/jobs/486191449

Examplar failure:
```
{"level":"warn","ts":"2021-02-25T12:34:47.894Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0000d6fc0/#initially=[unix://localhost:86269902489114839060]","attempt":1,"error":"rpc error: code = Unavailable desc = etcdserver: rpc not supported for learner"}
{"level":"warn","ts":"2021-02-25T12:34:48.163Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc00035a000/#initially=[unix://localhost:78285857058450835940]","attempt":0,"error":"rpc error: code = FailedPrecondition desc = etcdserver: not leader"}
{"level":"info","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = Canceled desc = context canceled"}
{"level":"info","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"info","ts":"2021-02-25T12:34:50.255Z","caller":"v3/maintenance.go:219","msg":"completed snapshot read; closing"}
{"level":"info","ts":"2021-02-25T12:34:51.717Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:52.017Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = Canceled desc = context canceled"}
{"level":"info","ts":"2021-02-25T12:34:52.018Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:53.018Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
--- FAIL: TestEndpointSwitchResolvesViolation (10.12s)
    ordering_util_test.go:81: failed to resolve order violation etcdclient: no cluster members have a revision higher than the previously received revision
```
 2021-02-25 22:15:13 +01:00
Piotr Tabor
60d5159091 version: bump up to 3.5.0-alpha.0 2021-02-24 19:55:45 +00:00
Piotr Tabor
1a9c81abda Update grpc dependency to 1.32. 
Simplify grpc testing infrastructure to align with upstream changes.
 2021-02-23 11:31:50 +01:00
Piotr Tabor
4a1c24556c clientv3: PS: Replace balancer with upstream grpc solution 
Addresses comments from: https://github.com/etcd-io/etcd/pull/12671#pullrequestreview-593942302
 2021-02-23 10:03:15 +01:00