Mirroristas/etcd
2
0
Fork 0
mirror of https://github.com/etcd-io/etcd.git synced 2024-09-27 06:25:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
16,806 Commits 15 Branches 491 Tags 148 MiB
Commit Graph

473 Commits

Author SHA1 Message Date
Sam Batschelet
a40a6e9ad8
Merge pull request #12805 from ptabor/20210326-minor-test-fixes 
tests: logging & temp-dir fixes
 2021-03-28 19:14:13 -04:00
Piotr Tabor
f290ab2e60 Update dependecies: 
github.com/grpc-ecosystem/grpc-gateway v1.14.6 -> grpc-gateway v1.16.0
  golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e->v0.0.0-20210220033141-f8bda1e9f3ba
 2021-03-27 20:48:33 +01:00
Piotr Tabor
c49807f59e Update cmux to 1.5.0. 
Executed using:
```
./scripts/update_dep.sh github.com/soheilhy/cmux v0.1.5
```
 2021-03-27 11:18:13 +01:00
Piotr Tabor
55ccbe62a2 membership/cluster_test: Use zaptest logger. 2021-03-26 13:54:59 +01:00
Piotr Tabor
03f55eeb2c Make NewTmpBackend use testing tmp location (so cleanup). 2021-03-26 13:54:55 +01:00
Piotr Tabor
4d4c84e014 server: Written Snapshot's to WAL contains populated ConfState. 
This will (among others) allow etcd-3.6 to not depend on store_v2 .snap files at all,
as WAL + db file will be self-sufficient.
 2021-03-25 00:31:44 +01:00
Piotr Tabor
fce0c192eb Regenerate protos. 2021-03-25 00:31:44 +01:00
Piotr Tabor
9180e8e56e wal: Update WAL Snapshot format to contain ConfState. 2021-03-25 00:31:44 +01:00
Piotr Tabor
a60676686b Update dep: grpc: 1.32.0 -> 1.36.0. 2021-03-24 22:27:55 +01:00
Piotr Tabor
45fb7b41d6 Update dep: github.com/golang/protobuf v1.3.5 ->  v1.5.1 
Thanks to https://go-review.googlesource.com/c/protobuf/+/300869/ its
feasible now.
 2021-03-24 22:27:12 +01:00
Piotr Tabor
7d7a9c6f23 codec.go: should use google runtime 
golang-proto runtime can deal with both: gogo & golang generated
protobufs. It does not work vice-versa with protobuf-1.5.1.
 2021-03-24 22:06:47 +01:00
Joel Smith
19f7c6ef3e *: Update gogo/protobuf to v1.3.2, rerun ./scripts/genproto.sh 
While it appears that etcd is not vulnerable to CVE-2021-3121,
it is a good idea to update to the new generator so that new
vulnerable code isn't generated in any future APIs. Also, this
lays the issue to rest of whether there is any issue with
etcd and CVE-2021-3121.
 2021-03-23 11:48:06 -06:00
Piotr Tabor
30ce6067da
Merge pull request #12780 from wpedrak/read_index_retry 
Read index retry
 2021-03-23 13:47:41 +01:00
Gábor Lipták
49d19efd82
Correct ineffassign 2021-03-22 14:11:30 -04:00
Piotr Tabor
8469108548
Merge pull request #12782 from ptabor/20210316-fixes 
Integration: Test flakiness fixes
 2021-03-19 20:10:11 +01:00
wpedrak
dac6e37ea1 *: over 20 staticcheck fixes 2021-03-18 15:06:17 +01:00
Piotr Tabor
725a8c5e02 Enable configuring delegated zap-logging for embed server. 2021-03-17 08:17:36 +01:00
Piotr Tabor
a84bd093b0 Integration with grpc-settable logger. 2021-03-16 22:50:41 +01:00
Piotr Tabor
18382aa234 Fix 2 sources of leaked memory: embed server HTTP & v3_snapshot.leasser. 2021-03-16 22:20:00 +01:00
wpedrak
e9779231ec server: add 500ms retries to ReadIndex requests for l-reads 
It is second approach (with first being #12762) to solve #12680
 2021-03-16 16:34:15 +01:00
wpedrak
4b21e38381 refactored l-read loop in v3_server.go 2021-03-16 11:03:45 +01:00
Piotr Tabor
1e7c1805d8 Unify logic of building raft-loggers for etcd. 
1. We had the same code copied 3 times.
2. For no good reason the code was not reusing existing logger if this one is given.
 2021-03-14 16:02:50 +01:00
Piotr Tabor
44bd22307e Merge get_logger() & Logger() method. 2021-03-14 14:05:17 +01:00
Piotr Tabor
948e32ae15 Delete etcd_debug metrics scheduled for deletion in 3.5. 2021-03-12 16:30:47 +01:00
Piotr Tabor
54189f2f60 Enable --pre-vote=true by default in 3.5. 2021-03-12 16:23:23 +01:00
Gyuho Lee
4eba403ccc
Merge pull request #12765 from ptabor/20210312-move-config 
Move config (ServerConfig) out of etcdserver package.
 2021-03-11 15:05:29 -08:00
Piotr Tabor
fd7fed1511 Move config (ServerConfig) out of etcdserver package. 
Motivation:
  - ServerConfig is part of 'embed' public API, while etcdserver is more 'internal'
  - EtcdServer is already too big and config is pretty wide-spread leaf
if we were to split etcdserver (e.g. into pre & post-apply part).
 2021-03-11 20:56:22 +01:00
Piotr Tabor
b9226d03f4
Merge pull request #12763 from hexfusion/bump-proto 
vendor: bump gogo/proto to v1.3.2
 2021-03-11 17:59:07 +01:00
Sam Batschelet
d3aa3fb486 vendor: bump gogo/proto to v1.3.2 
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
 2021-03-11 11:27:25 -05:00
Gyuho Lee
3ead91ca3e
Merge pull request #12739 from LeoYang90/optimization_watch_prevkv 
create event do not need prevkv range
 2021-03-10 09:48:42 -08:00
Piotr Tabor
fb1d48e98e Integration tests: Use BeforeTest(t) instead of defer AfterTest(). 
Thanks to this change, a single method BeforeTest(t) can handle
before-test logic as well as registration of cleanup code
(t.Cleanup(func)).
 2021-03-09 18:19:51 +01:00
Gyuho Lee
94a371acd7
Merge pull request #12750 from ptabor/20210306-mlock 
--experimental-memory-mlock support
 2021-03-09 09:13:40 -08:00
Gyuho Lee
6fd85af641
Merge pull request #12702 from hexfusion/add-so 
*: add support for socket options
 2021-03-09 09:02:24 -08:00
Sam Batschelet
5b49fb41c8 fixup: add ListenerOptions 
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
 2021-03-08 11:27:03 -05:00
Piotr Tabor
a46a358577 --experimental-memory-mlock support 
The flag protects etcd memory from being swapped out to disk.
This can happen in memory constrained systems where mmaped bbolt
area is natural condidate for swapping out.

This flag should provide better tail latency on the cost of higher RSS
ram usage. If the experiment is successful, the logic should get moved
into bbolt layer, where we can protect specific bbolt instances
(e.g. avoid protecting both during defragmentation).
 2021-03-07 12:32:57 +01:00
Chris Wedgwood
b63d31e89b etcdserver: when using --unsafe-no-fsync write data 
There are situations where we don't wish to fsync but we do want to
write the data.

Typically this occurs in clusters where fsync latency (often the
result of firmware) transiently spikes.  For Kubernetes clusters this
causes (many) elections which have knock-on effects such that the API
server will transiently fail causing other components fail in turn.

By writing the data (buffered and asynchronously flushed, so in most
situations the write is fast) and avoiding the fsync we no longer
trigger this situation and opportunistically write out the data.

Anecdotally:
  Because the fsync is missing there is the argument that certain
  types of failure events will cause data corruption or loss, in
  testing this wasn't seen.  If this was to occur the expectation is
  the member can be readded to a cluster or worst-case restored from a
  robust persisted snapshot.

  The etcd members are deployed across isolated racks with different
  power feeds.  An instantaneous failure of all of them simultaneously
  is unlikely.

  Testing was usually of the form:
   * create (Kubernetes) etcd write-churn by creating replicasets of
     some 1000s of pods
   * break/fail the leader

  Failure testing included:
   * hard node power-off events
   * disk removal
   * orderly reboots/shutdown

  In all cases when the node recovered it was able to rejoin the
  cluster and synchronize.
 2021-03-05 10:58:04 -08:00
Piotr Tabor
f7a2389992 Update version of certifi/gocertifi to get rid of WTF Public license 
Seems old versions of https://github.com/certifi/gocertifi where
categorized as "Do What The F*ck You Want To Public License".

Update to newer version that is explicit `Mozilla Public License` 2.0 (MPL 2.0).
 2021-03-04 09:48:34 +01:00
leoyang.yl
d70f35f8d1 create event do not need prevkv range 2021-03-02 17:43:24 +08:00
Ben Meier
3d44f5bf80
*: added client-{client,key}-file parameters for supporting separate client and server certs when communicating between peers 
In some environments, the CA is not able to sign certificates with both
'client auth' and 'server auth' extended usage parameters and so an operator
needs to be able to set a seperate client certificate to use when making
requests which is different to the certificate used for accepting requests.
This applies to both proxy and etcd member mode and is available as both a CLI
 flag and config file field for peer TLS.

Signed-off-by: Ben Meier <ben.meier@oracle.com>
 2021-02-28 14:37:56 +00:00
Piotr Tabor
a7f340216d Reformat code according to 'gotip' rules. 
In practices adds annotations in the new syntax:
```
+//go:build !linux
 // +build !linux
```

Fixes failing gotip PASSES='fmt' check:
https://travis-ci.com/github/etcd-io/etcd/jobs/486453806
 2021-02-26 10:14:46 +01:00
Piotr Tabor
45b1e6b470 ClientV3: Ordering: Fix the ordering test such it does not fail. 
The test depended on very subtle timing semantic and on properties of
'copied' clients.

https://travis-ci.com/github/etcd-io/etcd/jobs/486191449

Examplar failure:
```
{"level":"warn","ts":"2021-02-25T12:34:47.894Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0000d6fc0/#initially=[unix://localhost:86269902489114839060]","attempt":1,"error":"rpc error: code = Unavailable desc = etcdserver: rpc not supported for learner"}
{"level":"warn","ts":"2021-02-25T12:34:48.163Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc00035a000/#initially=[unix://localhost:78285857058450835940]","attempt":0,"error":"rpc error: code = FailedPrecondition desc = etcdserver: not leader"}
{"level":"info","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = Canceled desc = context canceled"}
{"level":"info","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"info","ts":"2021-02-25T12:34:50.255Z","caller":"v3/maintenance.go:219","msg":"completed snapshot read; closing"}
{"level":"info","ts":"2021-02-25T12:34:51.717Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:52.017Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = Canceled desc = context canceled"}
{"level":"info","ts":"2021-02-25T12:34:52.018Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:53.018Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
--- FAIL: TestEndpointSwitchResolvesViolation (10.12s)
    ordering_util_test.go:81: failed to resolve order violation etcdclient: no cluster members have a revision higher than the previously received revision
```
 2021-02-25 22:15:13 +01:00
Piotr Tabor
60d5159091 version: bump up to 3.5.0-alpha.0 2021-02-24 19:55:45 +00:00
Piotr Tabor
1a9c81abda Update grpc dependency to 1.32. 
Simplify grpc testing infrastructure to align with upstream changes.
 2021-02-23 11:31:50 +01:00
Piotr Tabor
57dcb037c0
Merge pull request #12706 from ptabor/20210218 
clientv3: PS: Replace balancer with upstream grpc solution
 2021-02-23 10:56:20 +01:00
Piotr Tabor
4a1c24556c clientv3: PS: Replace balancer with upstream grpc solution 
Addresses comments from: https://github.com/etcd-io/etcd/pull/12671#pullrequestreview-593942302
 2021-02-23 10:03:15 +01:00
yangweiwei
7ef6ebd5eb mvcc: optimize watch logic of watchableStore 
Optimize watchableStore.watch func
 2021-02-22 14:10:25 +08:00
Gyuho Lee
3d7aac948b
Merge pull request #12196 from ironcladlou/metrics-watch-error-fix 
etcdserver: fix incorrect metrics generated when clients cancel watches
 2021-02-19 12:46:49 -08:00
Sam Batschelet
49078c683b *: add support for socket options 
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
 2021-02-19 13:31:23 -05:00
Piotr Tabor
b67ed4e4aa
Merge pull request #12671 from ptabor/20210207-grpc-del-balancer 
clientv3: Replace balancer with upstream grpc solution
 2021-02-17 22:45:13 +01:00
Piotr Tabor
47b2506fcb
Merge pull request #12670 from postgrespro/customizable_raft_connection_timeouts 
raft: makes 'ConnReadTimeout/ConnWriteTimeout' customizable
 2021-02-16 09:07:41 +01:00