Mirroristas/etcd
2
0
Fork 0
mirror of https://github.com/etcd-io/etcd.git synced 2024-09-27 06:25:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
19,716 Commits 15 Branches 491 Tags 148 MiB
Commit Graph

594 Commits

Author SHA1 Message Date
Yi-Shu Tai
77a5072b69 client: call .Endpoints() in dial() in client/v3/client.go instead of accessing cfg.Endpoints directly 
0cdd558361/client/v3/client.go (L299) accesses
endpoints without acquiring lock. Fix it to call Endpoints()

Fix #13201
 2021-07-27 15:55:22 -07:00
Piotr Tabor
40da254d68
Merge pull request #13146 from tklauser/filelock-x-sys-windows 
client/pkg/fileutil: use `golang.org/x/sys/windows` for `FileLockEx`
 2021-07-01 10:44:08 +02:00
Piotr Tabor
7271adecc5
Merge pull request #12992 from awly/client-auth-bundle-overwrite 
client/v3: do not overwrite authTokenBundle on dial
 2021-07-01 10:40:18 +02:00
Marek Siarkowicz
86f68b9374 *: Add missing file licenses and Fix static analysis prevent skipping them in future 2021-06-29 12:52:02 +02:00
Tobias Klauser
c3fe63a658 client/pkg/fileutil: use golang.org/x/sys/windows for FileLockEx 
Use the FileLockEx wrapper and the corresponding LOCKFILE_* and error
constants from the golang.org/x/sys/windows package rather than
implementing these in the fileutil package.
 2021-06-25 14:44:30 +02:00
Andrew Lytvynov
ab8e5a4f8e client/v3: do not overwrite authTokenBundle on dial 
`Client.dial` can be called multiple times. For example, from regular
`NewClient` and [from the `Maintenance`
wrapper](6d451ab61d/client/v3/maintenance.go (L81-L84)).

This ends up creating two (if `Client.dial` is called twice) independent
`credentials.Bundle` instances:
- the first one is wired into `PerRPCCredentials` callback in gRPC
  client.
- the second one is in `Client.authTokenBundle` and receives the latest
  auth token updates.

When using username/password authentication and the server is configured
to issue JWTs, token rotation logic breaks because of the above
`credentials.Bundle` confusion.
 2021-06-23 09:44:27 -07:00
Lili Cosic
df696a7e76 go.mod: Bump etcd to 3.5.0 2021-06-22 12:25:39 +02:00
Piotr Tabor
5564476831
Merge pull request #13073 from serathius/snapshot-version 
[Version in Snapshot] SnapshotResponse includes local etcd version
 2021-06-21 16:56:00 +02:00
Brandon Philips
1a943350b0
client: README: update to new go.mod paths 
The paths in the readme and go.dev docs are out of date after v3.5
 2021-06-15 19:47:22 -07:00
Marek Siarkowicz
e1b1d93548 *: Snapshot returns local etcd version 
Co-authored-by: Lili Cosic <cosiclili@gmail.com>
 2021-06-14 16:36:50 +02:00
Lili Cosic
16477a8270 Update client_golang dependency to v1.11.0 2021-06-07 14:49:57 +02:00
Gyuho Lee
23e6bf3671
Merge pull request #13083 from lilic/add-downgrade-client 
client/v3/maintenance.go: Add Downgrade support to client
 2021-06-04 15:31:48 -07:00
Lili Cosic
d563c76e92 client/v3/maintenance.go: Add Downgrade support to client 2021-06-04 13:53:09 +02:00
Piotr Tabor
404efd70f7 Upgrade of prometheous deps. 
For now we need to depend on unstable: prometheus/client_golang
 2021-06-03 16:07:01 +02:00
Piotr Tabor
edcfe575cf Update dependencies: 
- github.com/coreos/go-systemd/v22 v22.3.2
  - github.com/google/btree v1.0.1
  - github.com/json-iterator/go v1.1.11
  - github.com/mattn/go-runewidth v0.0.13
  - github.com/prometheus/client_golang v1.10.0
  - github.com/prometheus/common v0.26.0
  - github.com/sirupsen/logrus v1.8.1
 2021-06-03 13:03:01 +02:00
Piotr Tabor
b240625d21 Update bbolt to v1.3.6. 
`./scripts/update_dep.sh go.etcd.io/bbolt v1.3.6`
 2021-06-03 12:39:53 +02:00
Piotr Tabor
90cea7d4e7 Update zap to v1.17.0. 2021-06-03 12:37:43 +02:00
Piotr Tabor
573e055cd3
Merge pull request #13069 from lilic/log-errors 
client, raft: Log unhandled errors
 2021-06-02 18:05:58 +02:00
Lili Cosic
3aae7d7823 client/v3/README.md: Remove warning 2021-06-02 16:35:12 +02:00
Lili Cosic
65cdf43e48 client/v3/lease.go: Log unhandled errors 2021-06-02 11:41:26 +02:00
Piotr Tabor
f15e0b8237 integration.BeforeTest can be run without leak-detection. 2021-05-27 20:55:11 +02:00
Piotr Tabor
bda32db121 Expose clientv3.CreateDefaultZapLoggerConfig 
Need raised in
f3f4259d3f.

BTW: Alligned the implementation of grpclogs with what we use in embed
server, so reduced the client code dependencies.
 2021-05-25 22:59:08 +02:00
Pavan BG
facba6c663 client: Replace nil value testing with assert.Nil 
Multiple errors in previous commit where `nil` is not being detected by `assert.Equal`
 2021-05-21 14:18:01 +05:30
Pavan BG
25d3c1b068 *: Fixed go.sum 
Ran fix.sh script to correct for differences in go.sum, perhaps due to multiple Golang versions
 2021-05-21 13:28:43 +05:30
Pavan BG
ce5bb69c28 client: Added missing packages 
go.etcd.io/etcd/client/pkg/v3/testutil: package github.com/stretchr/testify/assert imported from implicitly required module; to add missing requirements, run:
 2021-05-20 21:44:12 +05:30
Pavan BG
319ef4aa42 client: Removed AssertEqual function 
Used github.com/stretchr/testify/assert to replace the AssertEqual function definition. Required the use of copyToInterface to copy a string slice to the empty interface slice
 2021-05-20 21:33:07 +05:30
Pavan BG
d9c5e1f0a2 *: Replace internal testutil AssertEqual function in favour of github.com/stretchr/testify/assert.Equals 
To maintain uniformity in testing methodology, assert.Equals has been used everywhere while replacing testutil.AssertEqual

Fixes #13015
 2021-05-20 19:55:45 +05:30
Lili Cosic
7957f4461d server/embed/config_test.go: Add unit test for socket options 2021-05-20 10:00:27 +02:00
Lili Cosic
73c530de83 client,server: Expose reuse-port and reuse-address so they can be 
configured via a config file.
 2021-05-19 18:41:21 +02:00
Piotr Tabor
16d51d8c26 Fix not retryable error codes from: Unavailable -> FailedPrecondition 
- ErrGRPCNotCapable("etcdserver: not capable") -> codes.FailedPrecondition  (it will not autofix, it requires new version of server)
 - ErrGPRCNotSupportedForLearner("etcdserver: rpc not supported for learner") -> codes.FailedPrecondition (as long as its learner, the call will not work)
 - ErrGRPCClusterVersionUnavailable("etcdserver: cluster version not found during downgrade") -> codes.FailedPrecondition (backend does not contain the version (old etcd?) so retry will not help)

https://github.com/etcd-io/etcd/runs/2599598633?check_suite_focus=true

```
{"level":"warn","ts":"2021-05-17T09:55:30.246Z","logger":"etcd-client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc000539880/#initially=[unix://localhost:m30]","attempt":0,"error":"rpc error: code = Unavailable desc = etcdserver: rpc not supported for learner"}
{"level":"warn","ts":"2021-05-17T09:55:30.270Z","logger":"etcd-client","caller":"v3/retry_interceptor.go:62","msg":"retrying
of unary invoker
failed","target":"etcd-endpoints://0xc000539880/#initially=[unix://localhost:m30]","attempt":1,"error":"rpc
error: code = Unavailable desc = etcdserver: rpc not supported for
learner"}`
```
 2021-05-19 02:08:53 +02:00
Piotr Tabor
3dd577ec08 Simplify client's logger configuration in tests. 2021-05-18 13:03:16 +02:00
Piotr Tabor
8981afb6f5 Fix unit tests logging config. 2021-05-14 22:45:57 +02:00
Piotr Tabor
f5c26814ab Loggers to catch the e2e flake. 2021-05-14 22:42:31 +02:00
Gyuho Lee
e2d67f2e3b
Merge pull request #12956 from gyuho/rename-to-main 
*: rename "master" branch references to "main" in source code
 2021-05-13 08:26:33 -07:00
Gyuho Lee
a8e1cf0433 client: rename "master" branch references to "main" 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2021-05-12 10:38:50 -07:00
Marek Siarkowicz
1189ee3f3d client: Add logger argument to NewCtxClient 2021-05-12 16:40:55 +02:00
Piotr Tabor
1929aa0a36
Merge pull request #12948 from serathius/logger 
client: Allow setting zap logger in config
 2021-05-12 13:49:48 +02:00
Marek Siarkowicz
06afe87b34 client: Allow setting zap logger in config 2021-05-12 12:15:20 +02:00
Piotr Tabor
3b24496c2a
Merge pull request #12925 from yangxuanjia/yxjetcd_check_resuming 
check resuming len to avoid panic
 2021-05-12 09:18:40 +02:00
yangxuanjia
68b1e9f728 check resuming len to avoid panic 2021-05-12 09:51:59 +08:00
Lili Cosic
1a718a958e Add initial Tracing with OpenTelemetry 2021-05-10 10:44:40 +02:00
刘兵
a8a3eea56d
fix doc demo bug 
the service parameter not used.
 2021-05-05 13:57:10 +08:00
Piotr Tabor
451f65d661
Merge pull request #12908 from ptabor/20210429-client-retry-logging 
Clientv3 (retry interceptor) logs should use the configured logger
 2021-04-29 19:25:04 +02:00
Piotr Tabor
ffea1537d4 ClientV3 tests use integration.NewClient that configures proper logger. 2021-04-29 18:18:34 +02:00
Piotr Tabor
bc8d3f6639 Clientv3 (retry) logs should use the configured logger. 
clientv3 logs (especially tests) were poluted with unattributed to testing.T log lines:

```
{"level":"warn","ts":"2021-04-29T12:42:11.055+0200","logger":"etcd-client","caller":"v3/retry_interceptor.go:64","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0000fafc0/#initially=[unix://localhost:m10]","attempt":0,"error":"rpc error: code = ResourceExhausted desc = etcdserver: mvcc: database space exceeded"}
```

The reasons were 2 fold:
  - Interceptors were copying logger before "WithLogger" could modify it.
  - We were not propagating the loggers in a few testing contexts.
 2021-04-29 12:57:09 +02:00
Piotr Tabor
f53b70facb Embed: In case KVStoreHash verification fails, close the backend. 
In case of failed verification, the server used to keep opened backend
(so the file was locked on OS level).
 2021-04-29 11:51:25 +02:00
Piotr Tabor
11249fdee9
Merge pull request #12874 from ptabor/20210417-go1.16.3 
Update go for 3.5: 1.15.x -> 1.16.3
 2021-04-19 16:51:49 +02:00
Piotr Tabor
3423a949c0 Update go for 3.5: 1.15 -> 1.16.(3). 
https://github.com/etcd-io/etcd/issues/12732
 2021-04-19 16:50:54 +02:00
Piotr Tabor
06ba0fc5a2
Merge pull request #12846 from pyiyun/fix-snaptmpfile-bug 
etcdserver: remove temp files in snap dir when etcdserver starting
 2021-04-17 12:58:46 +02:00
Piotr Tabor
17b982382e Fix TestSnapshotV3RestoreMultiMemberAdd flakes (leaks) 
- most important: unix's socket transport should not keep idle
connections. For top-level Transport we close them using:
    f3c518025e/server/etcdserver/api/rafthttp/transport.go (L226)
    but currently we don't have access to close them witing the nest (unix) transport. Short idle deadline is good enough.

  - Use dialContext (instead of dial) to make sure context is passed down the stack
  - Make sure Context is cancelled as soon as the operation is done in pipeline
  - nit: use dedicated method to yeld goroutines.

Tested with:
```
d=$(date +"%Y%m%d_%H%M")
(cd tests && go test --timeout=60m ./integration/snapshot -run TestSnapshotV3RestoreMultiMemberAdd -v --count=180 2>&1 | tee log_${d}.log)
```

There were transports & cmux leaked:

```
   leak.go:118: Test appears to have leaked a Transport:
        internal/poll.runtime_pollWait(0x7f6c5c3784c8, 0x72, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/runtime/netpoll.go:222 +0x55
        internal/poll.(*pollDesc).wait(0xc003296298, 0x72, 0x0, 0x18, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:87 +0x45
        internal/poll.(*pollDesc).waitRead(...)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:92
        internal/poll.(*FD).Read(0xc003296280, 0xc0031f60a8, 0x18, 0x18, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/internal/poll/fd_unix.go:166 +0x1d5
        net.(*netFD).Read(0xc003296280, 0xc0031f60a8, 0x18, 0x18, 0x18, 0xc0009056e2, 0x203000)
        	/usr/lib/google-golang/src/net/fd_posix.go:55 +0x4f
        net.(*conn).Read(0xc000010258, 0xc0031f60a8, 0x18, 0x18, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/net/net.go:183 +0x91
        github.com/soheilhy/cmux.(*bufferedReader).Read(0xc0003d24e0, 0xc0031f60a8, 0x18, 0x18, 0xc0003d24d0, 0xc0009056e2, 0xc000278400)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/buffer.go:53 +0x12d
        github.com/soheilhy/cmux.hasHTTP2Preface(0x1367e20, 0xc0003d24e0, 0x7f6c5c699f40)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/matchers.go:195 +0x8a
        github.com/soheilhy/cmux.matchersToMatchWriters.func1(0x7f6c5c699f40, 0xc000010258, 0x1367e20, 0xc0003d24e0, 0xc000010258)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:128 +0x39
        github.com/soheilhy/cmux.(*cMux).serve(0xc003228690, 0x138c410, 0xc000010258, 0xc00327f740, 0xc0059ba860)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:192 +0x1e7
        created by github.com/soheilhy/cmux.(*cMux).Serve
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:179 +0x191

        internal/poll.runtime_pollWait(0x7f6c5c60f3f0, 0x72, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/runtime/netpoll.go:222 +0x55
        internal/poll.(*pollDesc).wait(0xc000d53018, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:87 +0x45
        internal/poll.(*pollDesc).waitRead(...)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:92
        internal/poll.(*FD).Read(0xc000d53000, 0xc000cfd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/internal/poll/fd_unix.go:166 +0x1d5
        net.(*netFD).Read(0xc000d53000, 0xc000cfd000, 0x1000, 0x1000, 0x3, 0x3, 0x1000000000001)
        	/usr/lib/google-golang/src/net/fd_posix.go:55 +0x4f
        net.(*conn).Read(0xc00031a570, 0xc000cfd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/net/net.go:183 +0x91
        net/http.(*persistConn).Read(0xc00093b320, 0xc000cfd000, 0x1000, 0x1000, 0x577750, 0x60, 0x0)
        	/usr/lib/google-golang/src/net/http/transport.go:1933 +0x77
        bufio.(*Reader).fill(0xc005702fc0)
        	/usr/lib/google-golang/src/bufio/bufio.go:101 +0x108
        bufio.(*Reader).Peek(0xc005702fc0, 0x1, 0xc00077c660, 0xc003b082a0, 0xc000d08de0, 0x5ae586, 0x11dd6c0)
        	/usr/lib/google-golang/src/bufio/bufio.go:139 +0x4f
        net/http.(*persistConn).readLoop(0xc00093b320)
        	/usr/lib/google-golang/src/net/http/transport.go:2094 +0x1a8
        created by net/http.(*Transport).dialConn
        	/usr/lib/google-golang/src/net/http/transport.go:1754 +0xdaa

        net/http.(*persistConn).writeLoop(0xc00093b320)
        	/usr/lib/google-golang/src/net/http/transport.go:2393 +0xf7
        created by net/http.(*Transport).dialConn
        	/usr/lib/google-golang/src/net/http/transport.go:1755 +0xdcf

        sync.runtime_Semacquire(0xc0059ba868)
        	/usr/lib/google-golang/src/runtime/sema.go:56 +0x45
        sync.(*WaitGroup).Wait(0xc0059ba860)
        	/usr/lib/google-golang/src/sync/waitgroup.go:130 +0x65
        github.com/soheilhy/cmux.(*cMux).Serve.func1(0xc003228690, 0xc0059ba860)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:158 +0x56
        github.com/soheilhy/cmux.(*cMux).Serve(0xc003228690, 0x13698c0, 0xc00377a0f0)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:173 +0x115
        go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers.func1(0xc0007cc360, 0x122b75f)
        	/home/ptab/corp/etcd/server/embed/etcd.go:518 +0x2b9
        go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers.func3(0xc00036d080, 0xc0059330a0)
        	/home/ptab/corp/etcd/server/embed/etcd.go:549 +0x182
        created by go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers
        	/home/ptab/corp/etcd/server/embed/etcd.go:543 +0x73a
--- FAIL: TestSnapshotV3RestoreMultiMemberAdd (17.74s)
```
 2021-04-16 20:17:28 +02:00