Lili Cosic
7957f4461d
server/embed/config_test.go: Add unit test for socket options
2021-05-20 10:00:27 +02:00
Lili Cosic
73c530de83
client,server: Expose reuse-port and reuse-address so they can be
...
configured via a config file.
2021-05-19 18:41:21 +02:00
Piotr Tabor
16d51d8c26
Fix not retryable error codes from: Unavailable -> FailedPrecondition
...
- ErrGRPCNotCapable("etcdserver: not capable") -> codes.FailedPrecondition (it will not autofix, it requires new version of server)
- ErrGPRCNotSupportedForLearner("etcdserver: rpc not supported for learner") -> codes.FailedPrecondition (as long as its learner, the call will not work)
- ErrGRPCClusterVersionUnavailable("etcdserver: cluster version not found during downgrade") -> codes.FailedPrecondition (backend does not contain the version (old etcd?) so retry will not help)
https://github.com/etcd-io/etcd/runs/2599598633?check_suite_focus=true
```
{"level":"warn","ts":"2021-05-17T09:55:30.246Z","logger":"etcd-client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc000539880/#initially=[unix://localhost:m30]","attempt":0,"error":"rpc error: code = Unavailable desc = etcdserver: rpc not supported for learner"}
{"level":"warn","ts":"2021-05-17T09:55:30.270Z","logger":"etcd-client","caller":"v3/retry_interceptor.go:62","msg":"retrying
of unary invoker
failed","target":"etcd-endpoints://0xc000539880/#initially=[unix://localhost:m30]","attempt":1,"error":"rpc
error: code = Unavailable desc = etcdserver: rpc not supported for
learner"}`
```
2021-05-19 02:08:53 +02:00
Piotr Tabor
3dd577ec08
Simplify client's logger configuration in tests.
2021-05-18 13:03:16 +02:00
Piotr Tabor
8981afb6f5
Fix unit tests logging config.
2021-05-14 22:45:57 +02:00
Piotr Tabor
f5c26814ab
Loggers to catch the e2e flake.
2021-05-14 22:42:31 +02:00
Gyuho Lee
e2d67f2e3b
Merge pull request #12956 from gyuho/rename-to-main
...
*: rename "master" branch references to "main" in source code
2021-05-13 08:26:33 -07:00
Gyuho Lee
a8e1cf0433
client: rename "master" branch references to "main"
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-12 10:38:50 -07:00
Marek Siarkowicz
1189ee3f3d
client: Add logger argument to NewCtxClient
2021-05-12 16:40:55 +02:00
Piotr Tabor
1929aa0a36
Merge pull request #12948 from serathius/logger
...
client: Allow setting zap logger in config
2021-05-12 13:49:48 +02:00
Marek Siarkowicz
06afe87b34
client: Allow setting zap logger in config
2021-05-12 12:15:20 +02:00
Piotr Tabor
3b24496c2a
Merge pull request #12925 from yangxuanjia/yxjetcd_check_resuming
...
check resuming len to avoid panic
2021-05-12 09:18:40 +02:00
yangxuanjia
68b1e9f728
check resuming len to avoid panic
2021-05-12 09:51:59 +08:00
Lili Cosic
1a718a958e
Add initial Tracing with OpenTelemetry
2021-05-10 10:44:40 +02:00
刘兵
a8a3eea56d
fix doc demo bug
...
the service parameter not used.
2021-05-05 13:57:10 +08:00
Piotr Tabor
451f65d661
Merge pull request #12908 from ptabor/20210429-client-retry-logging
...
Clientv3 (retry interceptor) logs should use the configured logger
2021-04-29 19:25:04 +02:00
Piotr Tabor
ffea1537d4
ClientV3 tests use integration.NewClient that configures proper logger.
2021-04-29 18:18:34 +02:00
Piotr Tabor
bc8d3f6639
Clientv3 (retry) logs should use the configured logger.
...
clientv3 logs (especially tests) were poluted with unattributed to testing.T log lines:
```
{"level":"warn","ts":"2021-04-29T12:42:11.055+0200","logger":"etcd-client","caller":"v3/retry_interceptor.go:64","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0000fafc0/#initially=[unix://localhost:m10]","attempt":0,"error":"rpc error: code = ResourceExhausted desc = etcdserver: mvcc: database space exceeded"}
```
The reasons were 2 fold:
- Interceptors were copying logger before "WithLogger" could modify it.
- We were not propagating the loggers in a few testing contexts.
2021-04-29 12:57:09 +02:00
Piotr Tabor
f53b70facb
Embed: In case KVStoreHash verification fails, close the backend.
...
In case of failed verification, the server used to keep opened backend
(so the file was locked on OS level).
2021-04-29 11:51:25 +02:00
Piotr Tabor
11249fdee9
Merge pull request #12874 from ptabor/20210417-go1.16.3
...
Update go for 3.5: 1.15.x -> 1.16.3
2021-04-19 16:51:49 +02:00
Piotr Tabor
3423a949c0
Update go for 3.5: 1.15 -> 1.16.(3).
...
https://github.com/etcd-io/etcd/issues/12732
2021-04-19 16:50:54 +02:00
Piotr Tabor
06ba0fc5a2
Merge pull request #12846 from pyiyun/fix-snaptmpfile-bug
...
etcdserver: remove temp files in snap dir when etcdserver starting
2021-04-17 12:58:46 +02:00
Piotr Tabor
17b982382e
Fix TestSnapshotV3RestoreMultiMemberAdd flakes (leaks)
...
- most important: unix's socket transport should not keep idle
connections. For top-level Transport we close them using:
f3c518025e/server/etcdserver/api/rafthttp/transport.go (L226)
but currently we don't have access to close them witing the nest (unix) transport. Short idle deadline is good enough.
- Use dialContext (instead of dial) to make sure context is passed down the stack
- Make sure Context is cancelled as soon as the operation is done in pipeline
- nit: use dedicated method to yeld goroutines.
Tested with:
```
d=$(date +"%Y%m%d_%H%M")
(cd tests && go test --timeout=60m ./integration/snapshot -run TestSnapshotV3RestoreMultiMemberAdd -v --count=180 2>&1 | tee log_${d}.log)
```
There were transports & cmux leaked:
```
leak.go:118: Test appears to have leaked a Transport:
internal/poll.runtime_pollWait(0x7f6c5c3784c8, 0x72, 0xffffffffffffffff)
/usr/lib/google-golang/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc003296298, 0x72, 0x0, 0x18, 0xffffffffffffffff)
/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc003296280, 0xc0031f60a8, 0x18, 0x18, 0x0, 0x0, 0x0)
/usr/lib/google-golang/src/internal/poll/fd_unix.go:166 +0x1d5
net.(*netFD).Read(0xc003296280, 0xc0031f60a8, 0x18, 0x18, 0x18, 0xc0009056e2, 0x203000)
/usr/lib/google-golang/src/net/fd_posix.go:55 +0x4f
net.(*conn).Read(0xc000010258, 0xc0031f60a8, 0x18, 0x18, 0x0, 0x0, 0x0)
/usr/lib/google-golang/src/net/net.go:183 +0x91
github.com/soheilhy/cmux.(*bufferedReader).Read(0xc0003d24e0, 0xc0031f60a8, 0x18, 0x18, 0xc0003d24d0, 0xc0009056e2, 0xc000278400)
/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/buffer.go:53 +0x12d
github.com/soheilhy/cmux.hasHTTP2Preface(0x1367e20, 0xc0003d24e0, 0x7f6c5c699f40)
/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/matchers.go:195 +0x8a
github.com/soheilhy/cmux.matchersToMatchWriters.func1(0x7f6c5c699f40, 0xc000010258, 0x1367e20, 0xc0003d24e0, 0xc000010258)
/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:128 +0x39
github.com/soheilhy/cmux.(*cMux).serve(0xc003228690, 0x138c410, 0xc000010258, 0xc00327f740, 0xc0059ba860)
/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:192 +0x1e7
created by github.com/soheilhy/cmux.(*cMux).Serve
/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:179 +0x191
internal/poll.runtime_pollWait(0x7f6c5c60f3f0, 0x72, 0xffffffffffffffff)
/usr/lib/google-golang/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc000d53018, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc000d53000, 0xc000cfd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/lib/google-golang/src/internal/poll/fd_unix.go:166 +0x1d5
net.(*netFD).Read(0xc000d53000, 0xc000cfd000, 0x1000, 0x1000, 0x3, 0x3, 0x1000000000001)
/usr/lib/google-golang/src/net/fd_posix.go:55 +0x4f
net.(*conn).Read(0xc00031a570, 0xc000cfd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/lib/google-golang/src/net/net.go:183 +0x91
net/http.(*persistConn).Read(0xc00093b320, 0xc000cfd000, 0x1000, 0x1000, 0x577750, 0x60, 0x0)
/usr/lib/google-golang/src/net/http/transport.go:1933 +0x77
bufio.(*Reader).fill(0xc005702fc0)
/usr/lib/google-golang/src/bufio/bufio.go:101 +0x108
bufio.(*Reader).Peek(0xc005702fc0, 0x1, 0xc00077c660, 0xc003b082a0, 0xc000d08de0, 0x5ae586, 0x11dd6c0)
/usr/lib/google-golang/src/bufio/bufio.go:139 +0x4f
net/http.(*persistConn).readLoop(0xc00093b320)
/usr/lib/google-golang/src/net/http/transport.go:2094 +0x1a8
created by net/http.(*Transport).dialConn
/usr/lib/google-golang/src/net/http/transport.go:1754 +0xdaa
net/http.(*persistConn).writeLoop(0xc00093b320)
/usr/lib/google-golang/src/net/http/transport.go:2393 +0xf7
created by net/http.(*Transport).dialConn
/usr/lib/google-golang/src/net/http/transport.go:1755 +0xdcf
sync.runtime_Semacquire(0xc0059ba868)
/usr/lib/google-golang/src/runtime/sema.go:56 +0x45
sync.(*WaitGroup).Wait(0xc0059ba860)
/usr/lib/google-golang/src/sync/waitgroup.go:130 +0x65
github.com/soheilhy/cmux.(*cMux).Serve.func1(0xc003228690, 0xc0059ba860)
/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:158 +0x56
github.com/soheilhy/cmux.(*cMux).Serve(0xc003228690, 0x13698c0, 0xc00377a0f0)
/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:173 +0x115
go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers.func1(0xc0007cc360, 0x122b75f)
/home/ptab/corp/etcd/server/embed/etcd.go:518 +0x2b9
go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers.func3(0xc00036d080, 0xc0059330a0)
/home/ptab/corp/etcd/server/embed/etcd.go:549 +0x182
created by go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers
/home/ptab/corp/etcd/server/embed/etcd.go:543 +0x73a
--- FAIL: TestSnapshotV3RestoreMultiMemberAdd (17.74s)
```
2021-04-16 20:17:28 +02:00
pyiyun
28a490b09c
etcdserver: remove temp files in snap dir when etcdServer starting
...
When etcd exits abnormally, tmp files will remain in snap dir, so clean up tmp files in snap dir when etcdserver starting.
Fixes #12837
2021-04-16 20:30:04 +08:00
Piotr Tabor
eafbc8c57e
Update zap logging dependency.
...
In particular bring up zapgrpc V2 code:
89e382035d
https://pkg.go.dev/google.golang.org/grpc/grpclog#LoggerV2
2021-04-14 12:15:48 +02:00
大可
5db0070150
Update doc.go
2021-04-13 22:22:42 +08:00
大可
d7e971e8d8
client: fix doc typo
2021-04-13 16:33:24 +08:00
Piotr Tabor
3bb7acc8cf
Migrate dependencies pkg/foo -> client/pkg/foo
2021-04-07 00:38:47 +02:00
Piotr Tabor
0b81fdf418
Move pkg/* used by client to client/pkg.
2021-04-05 23:47:27 +02:00
Piotr Tabor
f290ab2e60
Update dependecies:
...
github.com/grpc-ecosystem/grpc-gateway v1.14.6 -> grpc-gateway v1.16.0
golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e->v0.0.0-20210220033141-f8bda1e9f3ba
2021-03-27 20:48:33 +01:00
Piotr Tabor
a60676686b
Update dep: grpc: 1.32.0 -> 1.36.0.
2021-03-24 22:27:55 +01:00
Piotr Tabor
45fb7b41d6
Update dep: github.com/golang/protobuf v1.3.5 -> v1.5.1
...
Thanks to https://go-review.googlesource.com/c/protobuf/+/300869/ its
feasible now.
2021-03-24 22:27:12 +01:00
Piotr Tabor
18321a0c85
Merge pull request #12786 from ptabor/20210318-debug-env
...
client: Bring back ETCD_CLIENT_DEBUG variable interpretation.
2021-03-18 21:55:51 +01:00
Piotr Tabor
9312d1b077
clientv3: Bring back ETCD_CLIENT_DEBUG variable interpretation.
...
env ETCD_CLIENT_DEBUG supports log levels (debug, info, warn, error, dpanic, panic, fatal).
Only when set, overrides application-wide grpc logging settings.
2021-03-18 18:43:07 +01:00
wpedrak
dac6e37ea1
*: over 20 staticcheck fixes
2021-03-18 15:06:17 +01:00
Piotr Tabor
a47c18d30a
Fix 2 remaining 'defer AfterTest' calls.
2021-03-13 23:41:29 +01:00
Piotr Tabor
783e26fcdf
Fix gogo to 1.3.2 in api/go.mod.
2021-03-11 19:34:34 +01:00
wpedrak
2c2456bf3d
*: partial staticcheck fix
2021-03-10 14:13:38 +00:00
Piotr Tabor
fb1d48e98e
Integration tests: Use BeforeTest(t) instead of defer AfterTest().
...
Thanks to this change, a single method BeforeTest(t) can handle
before-test logic as well as registration of cleanup code
(t.Cleanup(func)).
2021-03-09 18:19:51 +01:00
Piotr Tabor
a46a358577
--experimental-memory-mlock support
...
The flag protects etcd memory from being swapped out to disk.
This can happen in memory constrained systems where mmaped bbolt
area is natural condidate for swapping out.
This flag should provide better tail latency on the cost of higher RSS
ram usage. If the experiment is successful, the logic should get moved
into bbolt layer, where we can protect specific bbolt instances
(e.g. avoid protecting both during defragmentation).
2021-03-07 12:32:57 +01:00
Emily Selwood
9ebbf5f38b
Metrics example 404s - fix url
...
The metrics example link points to a file that appears to have moved. This change points it to what I think is the right place.
2021-03-04 08:17:48 +00:00
David Lanouette
6998c5641c
client v2: rename error var for revive
...
The revive tool complained durring the build. Error variable has been
renamed.
Fixes #12718
Signed-off-by: David Lanouette <David.Lanouette@GMail.com>
2021-03-01 17:09:16 -05:00
David Lanouette
7d02ce2073
client v2: check for empty request from the context
...
If the simpleHTTPClient.Do is called and the context has a nil request, return an error early.
Fixes #12718
Signed-off-by: David Lanouette <David.Lanouette@GMail.com>
2021-03-01 16:46:30 -05:00
Piotr Tabor
54b87505a3
Remove dead legacy logger code.
2021-02-26 09:13:09 +01:00
Piotr Tabor
45b1e6b470
ClientV3: Ordering: Fix the ordering test such it does not fail.
...
The test depended on very subtle timing semantic and on properties of
'copied' clients.
https://travis-ci.com/github/etcd-io/etcd/jobs/486191449
Examplar failure:
```
{"level":"warn","ts":"2021-02-25T12:34:47.894Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0000d6fc0/#initially=[unix://localhost:86269902489114839060]","attempt":1,"error":"rpc error: code = Unavailable desc = etcdserver: rpc not supported for learner"}
{"level":"warn","ts":"2021-02-25T12:34:48.163Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc00035a000/#initially=[unix://localhost:78285857058450835940]","attempt":0,"error":"rpc error: code = FailedPrecondition desc = etcdserver: not leader"}
{"level":"info","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = Canceled desc = context canceled"}
{"level":"info","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"info","ts":"2021-02-25T12:34:50.255Z","caller":"v3/maintenance.go:219","msg":"completed snapshot read; closing"}
{"level":"info","ts":"2021-02-25T12:34:51.717Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:52.017Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = Canceled desc = context canceled"}
{"level":"info","ts":"2021-02-25T12:34:52.018Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:53.018Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
--- FAIL: TestEndpointSwitchResolvesViolation (10.12s)
ordering_util_test.go:81: failed to resolve order violation etcdclient: no cluster members have a revision higher than the previously received revision
```
2021-02-25 22:15:13 +01:00
Piotr Tabor
60d5159091
version: bump up to 3.5.0-alpha.0
2021-02-24 19:55:45 +00:00
Piotr Tabor
1a9c81abda
Update grpc dependency to 1.32.
...
Simplify grpc testing infrastructure to align with upstream changes.
2021-02-23 11:31:50 +01:00
Piotr Tabor
4a1c24556c
clientv3: PS: Replace balancer with upstream grpc solution
...
Addresses comments from: https://github.com/etcd-io/etcd/pull/12671#pullrequestreview-593942302
2021-02-23 10:03:15 +01:00
Piotr Tabor
b67ed4e4aa
Merge pull request #12671 from ptabor/20210207-grpc-del-balancer
...
clientv3: Replace balancer with upstream grpc solution
2021-02-17 22:45:13 +01:00
Piotr Tabor
a836a8045b
Get rid of legacy client/v3/naming API.
...
Update grpcproxy to use the new abstractions.
2021-02-09 11:56:28 +01:00