Mirroristas/etcd
2
0
Fork 0
mirror of https://github.com/etcd-io/etcd.git synced 2024-09-27 06:25:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
17,841 Commits 15 Branches 491 Tags 148 MiB
Commit Graph

52 Commits

Author SHA1 Message Date
Piotr Tabor
68f2cb8c77 Fix ExampleAuth from integration/clientv3/examples (on OsX) 
The code now ensures that each of the test is running in its own directory as opposed to shared os.tempdir.
```
$  (cd tests && env go test -timeout=15m --race go.etcd.io/etcd/tests/v3/integration/clientv3/examples -run ExampleAuth)
2022/04/03 10:24:59 Running tests (examples): ...
2022/04/03 10:24:59 the function can be called only in the test context. Was integration.BeforeTest() called ?
2022/04/03 10:24:59 2022-04-03T10:24:59.462+0200	INFO	m0	LISTEN GRPC	{"member": "m0", "grpcAddr": "localhost:m0", "m.Name": "m0"}
```
 2022-04-03 14:16:45 +02:00
Piotr Tabor
f85cd0296f
Merge pull request #13872 from ptabor/20220402-osx-unit-test-pass 
Fix TestauthTokenBundleOnOverwrite on OsX:
 2022-04-02 20:03:38 +02:00
Piotr Tabor
3b589fb3b2 Fix TestauthTokenBundleOnOverwrite on OsX: 
```
% (cd client/v3 && env go test -short -timeout=3m --race ./...)
--- FAIL: TestAuthTokenBundleNoOverwrite (0.00s)
    client_test.go:210: listen unix /var/folders/t1/3m8z9xz93t9c3vpt7zyzjm6w00374n/T/TestAuthTokenBundleNoOverwrite3197524989/001/etcd-auth-test:0: bind: invalid argument
FAIL
FAIL	go.etcd.io/etcd/client/v3	4.270s
```

The reason was that the path exceeded 108 chars (that is too much for socket).
In the mitigation we first change chroot (working directory) to the tempDir... such the path is 'local'.
 2022-04-02 16:12:02 +02:00
Marek Siarkowicz
63346bfead server: Use default logging configuration instead of zap production one 
This fixes problem where logs json changes format of timestamp.
 2022-04-01 10:23:42 +02:00
eval-exec
8d01ac2816 remove endpoints length check in TestValidateSecureEndpoints() 2022-03-23 23:37:31 +08:00
EXEC
ad78a74c42 Update client/pkg/transport/tls_test.go 
Co-authored-by: Marek Siarkowicz <marek.siarkowicz@protonmail.com>
 2022-03-23 23:37:31 +08:00
eval-exec
88e1d6b126 using subtests for TestValidateSecureEndpoints() 2022-03-23 23:37:31 +08:00
EXEC
4786a72cfc Update client/pkg/transport/tls_test.go 
Co-authored-by: Marek Siarkowicz <marek.siarkowicz@protonmail.com>
 2022-03-23 23:37:31 +08:00
eval-exec
df71f59c0e close idle connections 2022-03-23 23:37:31 +08:00
eval-exec
983ee82c98 add test for transport/tls.go:ValidateSecureEndpoints() 2022-03-23 23:37:31 +08:00
EXEC
bd7d09255b Fix panic in etcd validate secure endpoints #13810 
`ValidateSecureEndpoints()` should call `t.DialContext()` instead of `t.Dial()`, because `t.Dial` is `nil`
 2022-03-23 23:37:31 +08:00
cuishuang
b7824b794b fix some typos 
Signed-off-by: cuishuang <imcusg@gmail.com>

fix some typos

Signed-off-by: cuishuang <imcusg@gmail.com>
 2022-03-07 20:28:53 +08:00
Piotr Tabor
f80f477073
Merge pull request #13644 from Juneezee/refactor/t.TempDir 
*: use `T.TempDir` to create temporary test directory
 2022-02-21 19:52:37 +01:00
ahrtr
5fc0092c87 trim the suffix dot from the srv.Target for etcd-client DNS lookup 2022-02-18 18:16:19 +08:00
Eng Zer Jun
e9edc5432c
*: use T.TempDir to create temporary test directory 
The directory created by `T.TempDir()` and is automatically removed when
the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2022-01-25 00:00:46 +08:00
Piotr Tabor
9c28e07588
Merge pull request #13447 from Juneezee/deprecate-ioutil 
*: move from io/ioutil to io and os packages
 2021-10-29 23:18:36 +02:00
Sam Batschelet
316e62b4e1 *: fixup go 1.17 bump 
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
 2021-10-27 14:26:55 -04:00
Eng Zer Jun
2a151c8982
*: move from io/ioutil to io and os packages 
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2021-10-28 00:05:28 +08:00
Arda Güçlü
d7fa8022e6
Merge branch 'main' into support-zap-console-encoding 2021-10-08 11:00:57 +03:00
Aleksandr Razumov
3644c9d67b
client/pkg/fileutil: add missing logger to {Create,Touch}DirAll 
Also populate it to every invocation.
 2021-10-07 17:53:59 +03:00
Arda Güçlü
3c6e09f932 Handle empty log-format gracefully 2021-09-23 08:17:29 +03:00
Arda Güçlü
ec252d06c9 Return error when log-format is invalid 2021-09-22 19:45:08 +03:00
Arda Güçlü
e647995a38 Add zap encoding configurable 
Json encoding is the default zap encoding value and can not be changeable.
This PR enables configuring zap encoding to console via new flag `log-format`.
 2021-09-22 15:48:47 +03:00
Tero Saarni
af626ebfde client/pkg/v3: fixes Solaris build of transport 
Add empty implementation for reuse port socket option since Solaris does not
support SO_REUSEPORT.
 2021-08-30 11:44:02 +03:00
Piotr Tabor
40da254d68
Merge pull request #13146 from tklauser/filelock-x-sys-windows 
client/pkg/fileutil: use `golang.org/x/sys/windows` for `FileLockEx`
 2021-07-01 10:44:08 +02:00
Marek Siarkowicz
86f68b9374 *: Add missing file licenses and Fix static analysis prevent skipping them in future 2021-06-29 12:52:02 +02:00
Tobias Klauser
c3fe63a658 client/pkg/fileutil: use golang.org/x/sys/windows for FileLockEx 
Use the FileLockEx wrapper and the corresponding LOCKFILE_* and error
constants from the golang.org/x/sys/windows package rather than
implementing these in the fileutil package.
 2021-06-25 14:44:30 +02:00
Piotr Tabor
404efd70f7 Upgrade of prometheous deps. 
For now we need to depend on unstable: prometheus/client_golang
 2021-06-03 16:07:01 +02:00
Piotr Tabor
edcfe575cf Update dependencies: 
- github.com/coreos/go-systemd/v22 v22.3.2
  - github.com/google/btree v1.0.1
  - github.com/json-iterator/go v1.1.11
  - github.com/mattn/go-runewidth v0.0.13
  - github.com/prometheus/client_golang v1.10.0
  - github.com/prometheus/common v0.26.0
  - github.com/sirupsen/logrus v1.8.1
 2021-06-03 13:03:01 +02:00
Piotr Tabor
b240625d21 Update bbolt to v1.3.6. 
`./scripts/update_dep.sh go.etcd.io/bbolt v1.3.6`
 2021-06-03 12:39:53 +02:00
Piotr Tabor
90cea7d4e7 Update zap to v1.17.0. 2021-06-03 12:37:43 +02:00
Piotr Tabor
f15e0b8237 integration.BeforeTest can be run without leak-detection. 2021-05-27 20:55:11 +02:00
Piotr Tabor
bda32db121 Expose clientv3.CreateDefaultZapLoggerConfig 
Need raised in
f3f4259d3f.

BTW: Alligned the implementation of grpclogs with what we use in embed
server, so reduced the client code dependencies.
 2021-05-25 22:59:08 +02:00
Pavan BG
facba6c663 client: Replace nil value testing with assert.Nil 
Multiple errors in previous commit where `nil` is not being detected by `assert.Equal`
 2021-05-21 14:18:01 +05:30
Pavan BG
25d3c1b068 *: Fixed go.sum 
Ran fix.sh script to correct for differences in go.sum, perhaps due to multiple Golang versions
 2021-05-21 13:28:43 +05:30
Pavan BG
ce5bb69c28 client: Added missing packages 
go.etcd.io/etcd/client/pkg/v3/testutil: package github.com/stretchr/testify/assert imported from implicitly required module; to add missing requirements, run:
 2021-05-20 21:44:12 +05:30
Pavan BG
319ef4aa42 client: Removed AssertEqual function 
Used github.com/stretchr/testify/assert to replace the AssertEqual function definition. Required the use of copyToInterface to copy a string slice to the empty interface slice
 2021-05-20 21:33:07 +05:30
Pavan BG
d9c5e1f0a2 *: Replace internal testutil AssertEqual function in favour of github.com/stretchr/testify/assert.Equals 
To maintain uniformity in testing methodology, assert.Equals has been used everywhere while replacing testutil.AssertEqual

Fixes #13015
 2021-05-20 19:55:45 +05:30
Lili Cosic
7957f4461d server/embed/config_test.go: Add unit test for socket options 2021-05-20 10:00:27 +02:00
Lili Cosic
73c530de83 client,server: Expose reuse-port and reuse-address so they can be 
configured via a config file.
 2021-05-19 18:41:21 +02:00
Piotr Tabor
8981afb6f5 Fix unit tests logging config. 2021-05-14 22:45:57 +02:00
Piotr Tabor
f5c26814ab Loggers to catch the e2e flake. 2021-05-14 22:42:31 +02:00
Lili Cosic
1a718a958e Add initial Tracing with OpenTelemetry 2021-05-10 10:44:40 +02:00
Piotr Tabor
451f65d661
Merge pull request #12908 from ptabor/20210429-client-retry-logging 
Clientv3 (retry interceptor) logs should use the configured logger
 2021-04-29 19:25:04 +02:00
Piotr Tabor
ffea1537d4 ClientV3 tests use integration.NewClient that configures proper logger. 2021-04-29 18:18:34 +02:00
Piotr Tabor
f53b70facb Embed: In case KVStoreHash verification fails, close the backend. 
In case of failed verification, the server used to keep opened backend
(so the file was locked on OS level).
 2021-04-29 11:51:25 +02:00
Piotr Tabor
06ba0fc5a2
Merge pull request #12846 from pyiyun/fix-snaptmpfile-bug 
etcdserver: remove temp files in snap dir when etcdserver starting
 2021-04-17 12:58:46 +02:00
Piotr Tabor
17b982382e Fix TestSnapshotV3RestoreMultiMemberAdd flakes (leaks) 
- most important: unix's socket transport should not keep idle
connections. For top-level Transport we close them using:
    f3c518025e/server/etcdserver/api/rafthttp/transport.go (L226)
    but currently we don't have access to close them witing the nest (unix) transport. Short idle deadline is good enough.

  - Use dialContext (instead of dial) to make sure context is passed down the stack
  - Make sure Context is cancelled as soon as the operation is done in pipeline
  - nit: use dedicated method to yeld goroutines.

Tested with:
```
d=$(date +"%Y%m%d_%H%M")
(cd tests && go test --timeout=60m ./integration/snapshot -run TestSnapshotV3RestoreMultiMemberAdd -v --count=180 2>&1 | tee log_${d}.log)
```

There were transports & cmux leaked:

```
   leak.go:118: Test appears to have leaked a Transport:
        internal/poll.runtime_pollWait(0x7f6c5c3784c8, 0x72, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/runtime/netpoll.go:222 +0x55
        internal/poll.(*pollDesc).wait(0xc003296298, 0x72, 0x0, 0x18, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:87 +0x45
        internal/poll.(*pollDesc).waitRead(...)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:92
        internal/poll.(*FD).Read(0xc003296280, 0xc0031f60a8, 0x18, 0x18, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/internal/poll/fd_unix.go:166 +0x1d5
        net.(*netFD).Read(0xc003296280, 0xc0031f60a8, 0x18, 0x18, 0x18, 0xc0009056e2, 0x203000)
        	/usr/lib/google-golang/src/net/fd_posix.go:55 +0x4f
        net.(*conn).Read(0xc000010258, 0xc0031f60a8, 0x18, 0x18, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/net/net.go:183 +0x91
        github.com/soheilhy/cmux.(*bufferedReader).Read(0xc0003d24e0, 0xc0031f60a8, 0x18, 0x18, 0xc0003d24d0, 0xc0009056e2, 0xc000278400)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/buffer.go:53 +0x12d
        github.com/soheilhy/cmux.hasHTTP2Preface(0x1367e20, 0xc0003d24e0, 0x7f6c5c699f40)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/matchers.go:195 +0x8a
        github.com/soheilhy/cmux.matchersToMatchWriters.func1(0x7f6c5c699f40, 0xc000010258, 0x1367e20, 0xc0003d24e0, 0xc000010258)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:128 +0x39
        github.com/soheilhy/cmux.(*cMux).serve(0xc003228690, 0x138c410, 0xc000010258, 0xc00327f740, 0xc0059ba860)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:192 +0x1e7
        created by github.com/soheilhy/cmux.(*cMux).Serve
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:179 +0x191

        internal/poll.runtime_pollWait(0x7f6c5c60f3f0, 0x72, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/runtime/netpoll.go:222 +0x55
        internal/poll.(*pollDesc).wait(0xc000d53018, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:87 +0x45
        internal/poll.(*pollDesc).waitRead(...)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:92
        internal/poll.(*FD).Read(0xc000d53000, 0xc000cfd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/internal/poll/fd_unix.go:166 +0x1d5
        net.(*netFD).Read(0xc000d53000, 0xc000cfd000, 0x1000, 0x1000, 0x3, 0x3, 0x1000000000001)
        	/usr/lib/google-golang/src/net/fd_posix.go:55 +0x4f
        net.(*conn).Read(0xc00031a570, 0xc000cfd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/net/net.go:183 +0x91
        net/http.(*persistConn).Read(0xc00093b320, 0xc000cfd000, 0x1000, 0x1000, 0x577750, 0x60, 0x0)
        	/usr/lib/google-golang/src/net/http/transport.go:1933 +0x77
        bufio.(*Reader).fill(0xc005702fc0)
        	/usr/lib/google-golang/src/bufio/bufio.go:101 +0x108
        bufio.(*Reader).Peek(0xc005702fc0, 0x1, 0xc00077c660, 0xc003b082a0, 0xc000d08de0, 0x5ae586, 0x11dd6c0)
        	/usr/lib/google-golang/src/bufio/bufio.go:139 +0x4f
        net/http.(*persistConn).readLoop(0xc00093b320)
        	/usr/lib/google-golang/src/net/http/transport.go:2094 +0x1a8
        created by net/http.(*Transport).dialConn
        	/usr/lib/google-golang/src/net/http/transport.go:1754 +0xdaa

        net/http.(*persistConn).writeLoop(0xc00093b320)
        	/usr/lib/google-golang/src/net/http/transport.go:2393 +0xf7
        created by net/http.(*Transport).dialConn
        	/usr/lib/google-golang/src/net/http/transport.go:1755 +0xdcf

        sync.runtime_Semacquire(0xc0059ba868)
        	/usr/lib/google-golang/src/runtime/sema.go:56 +0x45
        sync.(*WaitGroup).Wait(0xc0059ba860)
        	/usr/lib/google-golang/src/sync/waitgroup.go:130 +0x65
        github.com/soheilhy/cmux.(*cMux).Serve.func1(0xc003228690, 0xc0059ba860)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:158 +0x56
        github.com/soheilhy/cmux.(*cMux).Serve(0xc003228690, 0x13698c0, 0xc00377a0f0)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:173 +0x115
        go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers.func1(0xc0007cc360, 0x122b75f)
        	/home/ptab/corp/etcd/server/embed/etcd.go:518 +0x2b9
        go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers.func3(0xc00036d080, 0xc0059330a0)
        	/home/ptab/corp/etcd/server/embed/etcd.go:549 +0x182
        created by go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers
        	/home/ptab/corp/etcd/server/embed/etcd.go:543 +0x73a
--- FAIL: TestSnapshotV3RestoreMultiMemberAdd (17.74s)
```
 2021-04-16 20:17:28 +02:00
pyiyun
28a490b09c etcdserver: remove temp files in snap dir when etcdServer starting 
When etcd exits abnormally, tmp files will remain in snap dir, so clean up tmp files in snap dir when etcdserver starting.

Fixes #12837
 2021-04-16 20:30:04 +08:00
Piotr Tabor
eafbc8c57e Update zap logging dependency. 
In particular bring up zapgrpc V2 code:
89e382035d
https://pkg.go.dev/google.golang.org/grpc/grpclog#LoggerV2
 2021-04-14 12:15:48 +02:00