Mirroristas/etcd
2
0
Fork 0
mirror of https://github.com/etcd-io/etcd.git synced 2024-09-27 06:25:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
17,115 Commits 15 Branches 491 Tags 148 MiB
Commit Graph

381 Commits

Author SHA1 Message Date
Piotr Tabor
404efd70f7 Upgrade of prometheous deps. 
For now we need to depend on unstable: prometheus/client_golang
 2021-06-03 16:07:01 +02:00
Piotr Tabor
edcfe575cf Update dependencies: 
- github.com/coreos/go-systemd/v22 v22.3.2
  - github.com/google/btree v1.0.1
  - github.com/json-iterator/go v1.1.11
  - github.com/mattn/go-runewidth v0.0.13
  - github.com/prometheus/client_golang v1.10.0
  - github.com/prometheus/common v0.26.0
  - github.com/sirupsen/logrus v1.8.1
 2021-06-03 13:03:01 +02:00
Piotr Tabor
b240625d21 Update bbolt to v1.3.6. 
`./scripts/update_dep.sh go.etcd.io/bbolt v1.3.6`
 2021-06-03 12:39:53 +02:00
Piotr Tabor
90cea7d4e7 Update zap to v1.17.0. 2021-06-03 12:37:43 +02:00
Piotr Tabor
573e055cd3
Merge pull request #13069 from lilic/log-errors 
client, raft: Log unhandled errors
 2021-06-02 18:05:58 +02:00
Lili Cosic
3aae7d7823 client/v3/README.md: Remove warning 2021-06-02 16:35:12 +02:00
Lili Cosic
65cdf43e48 client/v3/lease.go: Log unhandled errors 2021-06-02 11:41:26 +02:00
Piotr Tabor
f15e0b8237 integration.BeforeTest can be run without leak-detection. 2021-05-27 20:55:11 +02:00
Piotr Tabor
bda32db121 Expose clientv3.CreateDefaultZapLoggerConfig 
Need raised in
f3f4259d3f.

BTW: Alligned the implementation of grpclogs with what we use in embed
server, so reduced the client code dependencies.
 2021-05-25 22:59:08 +02:00
Pavan BG
facba6c663 client: Replace nil value testing with assert.Nil 
Multiple errors in previous commit where `nil` is not being detected by `assert.Equal`
 2021-05-21 14:18:01 +05:30
Pavan BG
25d3c1b068 *: Fixed go.sum 
Ran fix.sh script to correct for differences in go.sum, perhaps due to multiple Golang versions
 2021-05-21 13:28:43 +05:30
Pavan BG
ce5bb69c28 client: Added missing packages 
go.etcd.io/etcd/client/pkg/v3/testutil: package github.com/stretchr/testify/assert imported from implicitly required module; to add missing requirements, run:
 2021-05-20 21:44:12 +05:30
Pavan BG
319ef4aa42 client: Removed AssertEqual function 
Used github.com/stretchr/testify/assert to replace the AssertEqual function definition. Required the use of copyToInterface to copy a string slice to the empty interface slice
 2021-05-20 21:33:07 +05:30
Pavan BG
d9c5e1f0a2 *: Replace internal testutil AssertEqual function in favour of github.com/stretchr/testify/assert.Equals 
To maintain uniformity in testing methodology, assert.Equals has been used everywhere while replacing testutil.AssertEqual

Fixes #13015
 2021-05-20 19:55:45 +05:30
Lili Cosic
7957f4461d server/embed/config_test.go: Add unit test for socket options 2021-05-20 10:00:27 +02:00
Lili Cosic
73c530de83 client,server: Expose reuse-port and reuse-address so they can be 
configured via a config file.
 2021-05-19 18:41:21 +02:00
Piotr Tabor
16d51d8c26 Fix not retryable error codes from: Unavailable -> FailedPrecondition 
- ErrGRPCNotCapable("etcdserver: not capable") -> codes.FailedPrecondition  (it will not autofix, it requires new version of server)
 - ErrGPRCNotSupportedForLearner("etcdserver: rpc not supported for learner") -> codes.FailedPrecondition (as long as its learner, the call will not work)
 - ErrGRPCClusterVersionUnavailable("etcdserver: cluster version not found during downgrade") -> codes.FailedPrecondition (backend does not contain the version (old etcd?) so retry will not help)

https://github.com/etcd-io/etcd/runs/2599598633?check_suite_focus=true

```
{"level":"warn","ts":"2021-05-17T09:55:30.246Z","logger":"etcd-client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc000539880/#initially=[unix://localhost:m30]","attempt":0,"error":"rpc error: code = Unavailable desc = etcdserver: rpc not supported for learner"}
{"level":"warn","ts":"2021-05-17T09:55:30.270Z","logger":"etcd-client","caller":"v3/retry_interceptor.go:62","msg":"retrying
of unary invoker
failed","target":"etcd-endpoints://0xc000539880/#initially=[unix://localhost:m30]","attempt":1,"error":"rpc
error: code = Unavailable desc = etcdserver: rpc not supported for
learner"}`
```
 2021-05-19 02:08:53 +02:00
Piotr Tabor
3dd577ec08 Simplify client's logger configuration in tests. 2021-05-18 13:03:16 +02:00
Piotr Tabor
8981afb6f5 Fix unit tests logging config. 2021-05-14 22:45:57 +02:00
Piotr Tabor
f5c26814ab Loggers to catch the e2e flake. 2021-05-14 22:42:31 +02:00
Gyuho Lee
e2d67f2e3b
Merge pull request #12956 from gyuho/rename-to-main 
*: rename "master" branch references to "main" in source code
 2021-05-13 08:26:33 -07:00
Gyuho Lee
a8e1cf0433 client: rename "master" branch references to "main" 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2021-05-12 10:38:50 -07:00
Marek Siarkowicz
1189ee3f3d client: Add logger argument to NewCtxClient 2021-05-12 16:40:55 +02:00
Piotr Tabor
1929aa0a36
Merge pull request #12948 from serathius/logger 
client: Allow setting zap logger in config
 2021-05-12 13:49:48 +02:00
Marek Siarkowicz
06afe87b34 client: Allow setting zap logger in config 2021-05-12 12:15:20 +02:00
Piotr Tabor
3b24496c2a
Merge pull request #12925 from yangxuanjia/yxjetcd_check_resuming 
check resuming len to avoid panic
 2021-05-12 09:18:40 +02:00
yangxuanjia
68b1e9f728 check resuming len to avoid panic 2021-05-12 09:51:59 +08:00
Lili Cosic
1a718a958e Add initial Tracing with OpenTelemetry 2021-05-10 10:44:40 +02:00
刘兵
a8a3eea56d
fix doc demo bug 
the service parameter not used.
 2021-05-05 13:57:10 +08:00
Piotr Tabor
451f65d661
Merge pull request #12908 from ptabor/20210429-client-retry-logging 
Clientv3 (retry interceptor) logs should use the configured logger
 2021-04-29 19:25:04 +02:00
Piotr Tabor
ffea1537d4 ClientV3 tests use integration.NewClient that configures proper logger. 2021-04-29 18:18:34 +02:00
Piotr Tabor
bc8d3f6639 Clientv3 (retry) logs should use the configured logger. 
clientv3 logs (especially tests) were poluted with unattributed to testing.T log lines:

```
{"level":"warn","ts":"2021-04-29T12:42:11.055+0200","logger":"etcd-client","caller":"v3/retry_interceptor.go:64","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0000fafc0/#initially=[unix://localhost:m10]","attempt":0,"error":"rpc error: code = ResourceExhausted desc = etcdserver: mvcc: database space exceeded"}
```

The reasons were 2 fold:
  - Interceptors were copying logger before "WithLogger" could modify it.
  - We were not propagating the loggers in a few testing contexts.
 2021-04-29 12:57:09 +02:00
Piotr Tabor
f53b70facb Embed: In case KVStoreHash verification fails, close the backend. 
In case of failed verification, the server used to keep opened backend
(so the file was locked on OS level).
 2021-04-29 11:51:25 +02:00
Piotr Tabor
11249fdee9
Merge pull request #12874 from ptabor/20210417-go1.16.3 
Update go for 3.5: 1.15.x -> 1.16.3
 2021-04-19 16:51:49 +02:00
Piotr Tabor
3423a949c0 Update go for 3.5: 1.15 -> 1.16.(3). 
https://github.com/etcd-io/etcd/issues/12732
 2021-04-19 16:50:54 +02:00
Piotr Tabor
06ba0fc5a2
Merge pull request #12846 from pyiyun/fix-snaptmpfile-bug 
etcdserver: remove temp files in snap dir when etcdserver starting
 2021-04-17 12:58:46 +02:00
Piotr Tabor
17b982382e Fix TestSnapshotV3RestoreMultiMemberAdd flakes (leaks) 
- most important: unix's socket transport should not keep idle
connections. For top-level Transport we close them using:
    f3c518025e/server/etcdserver/api/rafthttp/transport.go (L226)
    but currently we don't have access to close them witing the nest (unix) transport. Short idle deadline is good enough.

  - Use dialContext (instead of dial) to make sure context is passed down the stack
  - Make sure Context is cancelled as soon as the operation is done in pipeline
  - nit: use dedicated method to yeld goroutines.

Tested with:
```
d=$(date +"%Y%m%d_%H%M")
(cd tests && go test --timeout=60m ./integration/snapshot -run TestSnapshotV3RestoreMultiMemberAdd -v --count=180 2>&1 | tee log_${d}.log)
```

There were transports & cmux leaked:

```
   leak.go:118: Test appears to have leaked a Transport:
        internal/poll.runtime_pollWait(0x7f6c5c3784c8, 0x72, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/runtime/netpoll.go:222 +0x55
        internal/poll.(*pollDesc).wait(0xc003296298, 0x72, 0x0, 0x18, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:87 +0x45
        internal/poll.(*pollDesc).waitRead(...)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:92
        internal/poll.(*FD).Read(0xc003296280, 0xc0031f60a8, 0x18, 0x18, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/internal/poll/fd_unix.go:166 +0x1d5
        net.(*netFD).Read(0xc003296280, 0xc0031f60a8, 0x18, 0x18, 0x18, 0xc0009056e2, 0x203000)
        	/usr/lib/google-golang/src/net/fd_posix.go:55 +0x4f
        net.(*conn).Read(0xc000010258, 0xc0031f60a8, 0x18, 0x18, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/net/net.go:183 +0x91
        github.com/soheilhy/cmux.(*bufferedReader).Read(0xc0003d24e0, 0xc0031f60a8, 0x18, 0x18, 0xc0003d24d0, 0xc0009056e2, 0xc000278400)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/buffer.go:53 +0x12d
        github.com/soheilhy/cmux.hasHTTP2Preface(0x1367e20, 0xc0003d24e0, 0x7f6c5c699f40)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/matchers.go:195 +0x8a
        github.com/soheilhy/cmux.matchersToMatchWriters.func1(0x7f6c5c699f40, 0xc000010258, 0x1367e20, 0xc0003d24e0, 0xc000010258)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:128 +0x39
        github.com/soheilhy/cmux.(*cMux).serve(0xc003228690, 0x138c410, 0xc000010258, 0xc00327f740, 0xc0059ba860)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:192 +0x1e7
        created by github.com/soheilhy/cmux.(*cMux).Serve
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:179 +0x191

        internal/poll.runtime_pollWait(0x7f6c5c60f3f0, 0x72, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/runtime/netpoll.go:222 +0x55
        internal/poll.(*pollDesc).wait(0xc000d53018, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:87 +0x45
        internal/poll.(*pollDesc).waitRead(...)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:92
        internal/poll.(*FD).Read(0xc000d53000, 0xc000cfd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/internal/poll/fd_unix.go:166 +0x1d5
        net.(*netFD).Read(0xc000d53000, 0xc000cfd000, 0x1000, 0x1000, 0x3, 0x3, 0x1000000000001)
        	/usr/lib/google-golang/src/net/fd_posix.go:55 +0x4f
        net.(*conn).Read(0xc00031a570, 0xc000cfd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/net/net.go:183 +0x91
        net/http.(*persistConn).Read(0xc00093b320, 0xc000cfd000, 0x1000, 0x1000, 0x577750, 0x60, 0x0)
        	/usr/lib/google-golang/src/net/http/transport.go:1933 +0x77
        bufio.(*Reader).fill(0xc005702fc0)
        	/usr/lib/google-golang/src/bufio/bufio.go:101 +0x108
        bufio.(*Reader).Peek(0xc005702fc0, 0x1, 0xc00077c660, 0xc003b082a0, 0xc000d08de0, 0x5ae586, 0x11dd6c0)
        	/usr/lib/google-golang/src/bufio/bufio.go:139 +0x4f
        net/http.(*persistConn).readLoop(0xc00093b320)
        	/usr/lib/google-golang/src/net/http/transport.go:2094 +0x1a8
        created by net/http.(*Transport).dialConn
        	/usr/lib/google-golang/src/net/http/transport.go:1754 +0xdaa

        net/http.(*persistConn).writeLoop(0xc00093b320)
        	/usr/lib/google-golang/src/net/http/transport.go:2393 +0xf7
        created by net/http.(*Transport).dialConn
        	/usr/lib/google-golang/src/net/http/transport.go:1755 +0xdcf

        sync.runtime_Semacquire(0xc0059ba868)
        	/usr/lib/google-golang/src/runtime/sema.go:56 +0x45
        sync.(*WaitGroup).Wait(0xc0059ba860)
        	/usr/lib/google-golang/src/sync/waitgroup.go:130 +0x65
        github.com/soheilhy/cmux.(*cMux).Serve.func1(0xc003228690, 0xc0059ba860)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:158 +0x56
        github.com/soheilhy/cmux.(*cMux).Serve(0xc003228690, 0x13698c0, 0xc00377a0f0)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:173 +0x115
        go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers.func1(0xc0007cc360, 0x122b75f)
        	/home/ptab/corp/etcd/server/embed/etcd.go:518 +0x2b9
        go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers.func3(0xc00036d080, 0xc0059330a0)
        	/home/ptab/corp/etcd/server/embed/etcd.go:549 +0x182
        created by go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers
        	/home/ptab/corp/etcd/server/embed/etcd.go:543 +0x73a
--- FAIL: TestSnapshotV3RestoreMultiMemberAdd (17.74s)
```
 2021-04-16 20:17:28 +02:00
pyiyun
28a490b09c etcdserver: remove temp files in snap dir when etcdServer starting 
When etcd exits abnormally, tmp files will remain in snap dir, so clean up tmp files in snap dir when etcdserver starting.

Fixes #12837
 2021-04-16 20:30:04 +08:00
Piotr Tabor
eafbc8c57e Update zap logging dependency. 
In particular bring up zapgrpc V2 code:
89e382035d
https://pkg.go.dev/google.golang.org/grpc/grpclog#LoggerV2
 2021-04-14 12:15:48 +02:00
大可
5db0070150
Update doc.go 2021-04-13 22:22:42 +08:00
大可
d7e971e8d8
client: fix doc typo 2021-04-13 16:33:24 +08:00
Piotr Tabor
3bb7acc8cf Migrate dependencies pkg/foo -> client/pkg/foo 2021-04-07 00:38:47 +02:00
Piotr Tabor
0b81fdf418 Move pkg/* used by client to client/pkg. 2021-04-05 23:47:27 +02:00
Piotr Tabor
f290ab2e60 Update dependecies: 
github.com/grpc-ecosystem/grpc-gateway v1.14.6 -> grpc-gateway v1.16.0
  golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e->v0.0.0-20210220033141-f8bda1e9f3ba
 2021-03-27 20:48:33 +01:00
Piotr Tabor
a60676686b Update dep: grpc: 1.32.0 -> 1.36.0. 2021-03-24 22:27:55 +01:00
Piotr Tabor
45fb7b41d6 Update dep: github.com/golang/protobuf v1.3.5 ->  v1.5.1 
Thanks to https://go-review.googlesource.com/c/protobuf/+/300869/ its
feasible now.
 2021-03-24 22:27:12 +01:00
Piotr Tabor
18321a0c85
Merge pull request #12786 from ptabor/20210318-debug-env 
client: Bring back ETCD_CLIENT_DEBUG variable interpretation.
 2021-03-18 21:55:51 +01:00
Piotr Tabor
9312d1b077 clientv3: Bring back ETCD_CLIENT_DEBUG variable interpretation. 
env ETCD_CLIENT_DEBUG supports log levels (debug, info, warn, error, dpanic, panic, fatal).
Only when set, overrides application-wide grpc logging settings.
 2021-03-18 18:43:07 +01:00
wpedrak
dac6e37ea1 *: over 20 staticcheck fixes 2021-03-18 15:06:17 +01:00
Piotr Tabor
a47c18d30a Fix 2 remaining 'defer AfterTest' calls. 2021-03-13 23:41:29 +01:00