644 Commits

Author SHA1 Message Date
Piotr Tabor
d0962f10ae Adopt grpc_testing to grpc-1.36. 2021-03-24 23:31:03 +01:00
Piotr Tabor
a60676686b Update dep: grpc: 1.32.0 -> 1.36.0. 2021-03-24 22:27:55 +01:00
Piotr Tabor
45fb7b41d6 Update dep: github.com/golang/protobuf v1.3.5 ->  v1.5.1
Thanks to https://go-review.googlesource.com/c/protobuf/+/300869/ its
feasible now.
2021-03-24 22:27:12 +01:00
Piotr Tabor
18321a0c85
Merge pull request #12786 from ptabor/20210318-debug-env
client: Bring back ETCD_CLIENT_DEBUG variable interpretation.
2021-03-18 21:55:51 +01:00
Piotr Tabor
e2b5835e64 Fix go-tip: (1.17) fmt error:
```
gofmt' started at Thu Mar 18 18:06:00 UTC 2021
transport/sockopt_unix.go
diff -u transport/sockopt_unix.go.orig transport/sockopt_unix.go
--- transport/sockopt_unix.go.orig	2021-03-18 18:06:01.667483834 +0000
+++ transport/sockopt_unix.go	2021-03-18 18:06:01.667483834 +0000
@@ -1,3 +1,4 @@
+//go:build !windows
 // +build !windows

 package transport
transport/sockopt_windows.go
diff -u transport/sockopt_windows.go.orig transport/sockopt_windows.go
--- transport/sockopt_windows.go.orig	2021-03-18 18:06:01.667483834 +0000
+++ transport/sockopt_windows.go	2021-03-18 18:06:01.667483834 +0000
@@ -1,3 +1,4 @@
+//go:build windows
 // +build windows
```
2021-03-18 19:44:28 +01:00
Piotr Tabor
9312d1b077 clientv3: Bring back ETCD_CLIENT_DEBUG variable interpretation.
env ETCD_CLIENT_DEBUG supports log levels (debug, info, warn, error, dpanic, panic, fatal).
Only when set, overrides application-wide grpc logging settings.
2021-03-18 18:43:07 +01:00
wpedrak
dac6e37ea1 *: over 20 staticcheck fixes 2021-03-18 15:06:17 +01:00
Piotr Tabor
de67806175 mend 2021-03-14 13:35:47 +01:00
Piotr Tabor
67491a00ea e2e/expect: In case of sut process failure, print last 40lines of logs. 2021-03-13 23:41:29 +01:00
wpedrak
2c2456bf3d *: partial staticcheck fix 2021-03-10 14:13:38 +00:00
Piotr Tabor
b6c2e87a74 Testing: Integration tests does not check whether t==nil 2021-03-09 18:19:52 +01:00
Piotr Tabor
5ddabfdb24 tests: Make tests operate in /tmp director instead of src.
Thanks to this, unix sockets should be not longer
created by integration tests in the the source code directory,
so potentially trigger IDE reloads and unnecessery load (and mess).
2021-03-09 18:19:52 +01:00
Piotr Tabor
fb1d48e98e Integration tests: Use BeforeTest(t) instead of defer AfterTest().
Thanks to this change, a single method BeforeTest(t) can handle
before-test logic as well as registration of cleanup code
(t.Cleanup(func)).
2021-03-09 18:19:51 +01:00
Piotr Tabor
efb584cc9b leak.go: Make the per-test AfterTest strictly wait for none of the unwanted rountines. 2021-03-09 18:19:51 +01:00
Gyuho Lee
94a371acd7
Merge pull request #12750 from ptabor/20210306-mlock
--experimental-memory-mlock support
2021-03-09 09:13:40 -08:00
Gyuho Lee
6fd85af641
Merge pull request #12702 from hexfusion/add-so
*: add support for socket options
2021-03-09 09:02:24 -08:00
Sam Batschelet
5b49fb41c8 fixup: add ListenerOptions
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-03-08 11:27:03 -05:00
Piotr Tabor
a46a358577 --experimental-memory-mlock support
The flag protects etcd memory from being swapped out to disk.
This can happen in memory constrained systems where mmaped bbolt
area is natural condidate for swapping out.

This flag should provide better tail latency on the cost of higher RSS
ram usage. If the experiment is successful, the logic should get moved
into bbolt layer, where we can protect specific bbolt instances
(e.g. avoid protecting both during defragmentation).
2021-03-07 12:32:57 +01:00
Piotr Tabor
792b7f57d3
Merge pull request #12747 from wilsonwang371/master
pkg/wait: change list from single element to an array.
2021-03-07 11:53:30 +01:00
Piotr Tabor
66cef61444 Detect leaked go-routines bases on pre-normalization syntax. 2021-03-04 22:28:44 +01:00
Wilson Wang
432fde88a9 pkg/wait: change list from single element to an array.
We found wait lock contention when a large amount of write operations. Converting wait from single element to an array helps to improve the performance.

Fixes #12731

Signed-off-by: Wilson Wang <wilsonny371@gmail.com>
2021-03-04 12:16:41 -08:00
Piotr Tabor
61fef348f8
Merge pull request #12742 from wilsonwang371/master
debugutil: Remove extra space in trace handler route
2021-03-04 08:55:44 +01:00
Wilson Wang
7fc447fb5c debugutil: Remove extra space in trace handler route
debugutil: Remove extra space in trace handler route. To use trace, user needed to escape the extra space and the extra space needs to be removed.
2021-03-03 16:01:04 -08:00
Piotr Tabor
102c198444
Merge pull request #12705 from astromechza/bm_etcd_peer_server_cert
etcdmain: added peer-client-{client,key}-file parameters for supporting separate client and server certs when communicating between peers
2021-03-02 09:03:35 +01:00
Piotr Tabor
5e10d12996 tests: Fixes a few recently spotted test-flakes
```
Unexpected goroutines running after all test(s).
1 instances of:
syscall.Syscall(...)
	/usr/local/go/src/syscall/asm_linux_386.s:19 +0x5
syscall.Close(...)
	/usr/local/go/src/syscall/zsyscall_linux_386.go:285 +0x3d
internal/poll.(*FD).destroy(...)
	/usr/local/go/src/internal/poll/fd_unix.go:77 +0x30
internal/poll.(*FD).decref(...)
	/usr/local/go/src/internal/poll/fd_mutex.go:213 +0x38
internal/poll.(*FD).Close(...)
	/usr/local/go/src/internal/poll/fd_unix.go:99 +0x43
net.(*netFD).Close(...)
	/usr/local/go/src/net/fd_posix.go:37 +0x49
FAIL	go.etcd.io/etcd/tests/v3/integration/client	0.039s
```

```
--- FAIL: TestServer_TCP_Secure_DelayTx (0.20s)
    server_test.go:110: took 128.026085ms with no latency
    server_test.go:125: took 62.980988ms with latency 50ms��5ms
    server_test.go:133: expected took1 128.026085ms < took2 62.980988ms (with latency)
```

https://github.com/etcd-io/etcd/issues/12372
2021-03-01 18:07:38 +01:00
Ben Meier
3d44f5bf80
*: added client-{client,key}-file parameters for supporting separate client and server certs when communicating between peers
In some environments, the CA is not able to sign certificates with both
'client auth' and 'server auth' extended usage parameters and so an operator
needs to be able to set a seperate client certificate to use when making
requests which is different to the certificate used for accepting requests.
This applies to both proxy and etcd member mode and is available as both a CLI
 flag and config file field for peer TLS.

Signed-off-by: Ben Meier <ben.meier@oracle.com>
2021-02-28 14:37:56 +00:00
Gyuho Lee
cb040801f9
Merge pull request #12730 from ptabor/20200208-client-no-logger
clientv3: Cleaning up dead logger code
2021-02-26 10:44:42 -08:00
Piotr Tabor
a7f340216d Reformat code according to 'gotip' rules.
In practices adds annotations in the new syntax:
```
+//go:build !linux
 // +build !linux
```

Fixes failing gotip PASSES='fmt' check:
https://travis-ci.com/github/etcd-io/etcd/jobs/486453806
2021-02-26 10:14:46 +01:00
Piotr Tabor
54b87505a3 Remove dead legacy logger code. 2021-02-26 09:13:09 +01:00
Mohammad Aziz
252dcc9bdb
Fix typo in DialJournal 2021-02-24 22:23:27 +05:30
Piotr Tabor
1a9c81abda Update grpc dependency to 1.32.
Simplify grpc testing infrastructure to align with upstream changes.
2021-02-23 11:31:50 +01:00
Sam Batschelet
49078c683b *: add support for socket options
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-02-19 13:31:23 -05:00
Piotr Tabor
60e44286fa Fix pkg/tlsutil (test) to not fail on 386.
In fact this commit rewrites the functionality to use upstream list of
ciphers instead of checking whether the lists are in sync using ast
analysis.
2021-02-08 23:03:43 +01:00
Piotr Tabor
dae29bb719
Merge pull request #12667 from lemonlinger/implement-endpoint-manager-and-resolver
Implement Update method for EndpointManager
2021-02-04 17:25:51 +01:00
limeng01
571ed502d4 endpoints: implement Update method for EndpointManager.
- Add integration test for endpoints and resolver.
2021-02-04 23:30:22 +08:00
Brad Davidson
603d975599 Fix cluster peer HTTP SRV discovery
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2021-02-03 03:08:13 -08:00
Piotr Tabor
d6d03beaea
Merge pull request #12538 from lzhfromustc/12_9_GoroutineLeak
test: change channel operations to avoid potential goroutine leaks
2021-02-01 21:16:43 +01:00
Piotr Tabor
351bdb33c5 Split intengration/clientv3 tests into multiple packages
They used to take >10min with coverage, so were causing interrupted
Travis runs.

Know thay fit in 100-150s (together), thanks also to parallel
execution.
2021-01-23 11:12:44 +01:00
Piotr Tabor
88ee005a0f Reduce verbosity of coverage collection. 2021-01-23 08:59:19 +01:00
Sahdev Zala
2c11bd5e19
Merge pull request #12586 from viviyww/clientconfigerror
transport: catch new cert error
2021-01-13 23:47:36 -05:00
Piotr Tabor
70b5ef1d3a Fix tests flakiness: in particular TestIssue6361.
The root reason of flakes, was that server was considered as ready to
early.
In particular:
```
../../bin/etcd-2456648: {"level":"info","ts":"2021-01-11T09:56:44.474+0100","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream Message","local-member-id":"ed5f620d34a8e61b","remote-peer-id":"ca50e9357181d758"}
../../bin/etcd-2456648: {"level":"warn","ts":"2021-01-11T09:56:49.040+0100","caller":"etcdserver/server.go:1942","msg":"failed to publish local member to cluster through raft","local-member-id":"ed5f620d34a8e61b","local-member-attributes":"{Name:infra2 ClientURLs:[http://localhost:20030]}","request-path":"/0/members/ed5f620d34a8e61b/attributes","publish-timeout":"7s","error":"etcdserver: request timed out, possibly due to connection lost"}
../../bin/etcd-2456648: {"level":"info","ts":"2021-01-11T09:56:49.049+0100","caller":"etcdserver/server.go:1921","msg":"published local member to cluster through raft","local-member-id":"ed5f620d34a8e61b","local-member-attributes":"{Name:infra2 ClientURLs:[http://localhost:20030]}","request-path":"/0/members/ed5f620d34a8e61b/attributes","cluster-id":"34f27e83b3bc2ff","publish-timeout":"7s"}
```
was taking 5s.   If this was happening concurrently with etcdctl, the
etcdctl could timeout.

The fix, requires servers to report 'ready to serve client requests' to consider them up.

Fixed also some whitelisted 'goroutines'.
2021-01-12 00:14:51 +01:00
Piotr Tabor
26f9b4be8f e2e tests were leaking 'defunc' etcdctl processes.
The commit ensures that spawned etcdctl processes are "closed",
so they perform proper os wait processing.
This might have contributed to file-descriptor/open-files limit being
exceeded.
2021-01-11 11:55:30 +01:00
Sahdev Zala
c632042bb9
Merge pull request #12489 from zhijianli88/cleanup-tmpfiles
Cleanup tmpfiles
2021-01-07 12:26:04 -05:00
yangweiwei
826573586f transport: catch new cert error
in pkg/transport, we should catch NewCert error.
2020-12-30 09:33:42 +08:00
lzhfromustc
f2a912a4e6 test: change channel operations to avoid potential goroutine leaks
In these unit tests, goroutines may leak if certain branches are chosen. This commit edits channel operations and buffer sizes, so no matter what branch is chosen, the test will end correctly. This commit doesn't change the semantics of unit tests.
2020-12-09 22:23:21 -05:00
Gyuho Lee
85b61c0f4d pkg/netutil: remove unused "iptables" wrapper
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-11-25 10:38:42 -08:00
Li Zhijian
63c284211d test: cleanup temporary files for TestZeroToEnd
Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
2020-11-25 17:33:43 +08:00
Kir Kolyshkin
4eb4250e6d pkg/fileutil: fix F_OFD_ constants
Use golang.org/x/sys/unix for F_OFD_* constants.

This fixes the issue that F_OFD_GETLK was defined incorrectly,
resulting in bugs such as https://github.com/moby/moby/issues/31182

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-02 19:37:25 -08:00
tangcong
a960d6b1c7 *: add self-signed-cert-validity flag 2020-10-30 10:10:26 +08:00
Gyuho Lee
eee8dec0c3
Merge pull request #12421 from ptabor/20201026-fix-ws-shell
Unify tabs vs. spaces in the shell scripts
2020-10-26 04:42:59 -07:00