54 Commits

Author SHA1 Message Date
Chao Chen
af009be983 fix ts with no trailing zeros
Signed-off-by: Chao Chen <chaochn@amazon.com>
2024-07-06 10:52:11 +00:00
D Tripp
e2c6c4754a Bump Go version to 1.21.12: GO-2024-2963 fix
Signed-off-by: D Tripp <38776199+thedtripp@users.noreply.github.com>
2024-07-03 05:17:11 +00:00
Benjamin Wang
f7ab198fb6
Merge pull request #18160 from lhy1024/pick-cn
[3.5] Support multiple values for allowed client and peer TLS identities
2024-06-21 06:45:08 +01:00
Chun-Hung Tseng
49a0263016
dependency: dump golang.org/x/sys to the highest version v0.18.0
Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
2024-06-19 23:30:38 +02:00
Chun-Hung Tseng
e72a7994b2
dependency: bump golang.org/x/sys from v0.0.0-20210403161142-5e06dd20ab57 to v0.0.0-20220412211240-33da011f77ad
Extracted log from govulncheck, suggesting that we should bump the
version of golang.org/x/sys

Vulnerability #1: GO-2022-0493
    Incorrect privilege reporting in syscall and golang.org/x/sys/unix
  More info: https://pkg.go.dev/vuln/GO-2022-0493
  Module: golang.org/x/sys
    Found in: golang.org/x/sys@v0.0.0-20210403161142-5e06dd20ab57
    Fixed in: golang.org/x/sys@v0.0.0-20220412211240-33da011f77ad

Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
2024-06-14 11:33:21 +02:00
lhy1024
f8befd94f6 Support multiple values for allowed client and peer TLS identities(#18015)
Signed-off-by: lhy1024 <admin@liudos.us>
2024-06-13 14:32:24 +08:00
ArkaSaha30
a64ddefe00
Bump Go version to 1.21.11: CVE 2024-24790 fix
Signed-off-by: ArkaSaha30 <arkasaha30@gmail.com>
2024-06-05 13:48:25 +05:30
Madhav Jivrajani
474031588a .*: sync go toolchain version and add ability to verify versions
This commit adds a script to sync the version present in .go-version
across all go.mod files as the toolchain directive. As part of that,
this commit also modifies go.mod files that did not have synced toolchain
directives.

Additionally, this also adds a script to verify all toolchain and go
directives against the version present in .go-version as follows:
(1) The go directive <= version in .go-version
(2) The toolchain directive == version in .go-version

This script runs as part of the `make verify` target, making it run
as a presbumit by default.

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
2024-05-16 14:45:42 +05:30
Karen Almog
1a9062ec98 pkg/types: Support Unix sockets in NewURLS
Resolves #12450
This commits adds support to unix/unixs socket URLs, which currently
fail with the message "URL address does not have the form "host:port".
It also replaces the work started in #11747.

(cherry picked from commit d93b7c8cb185af5a8af8408fbfcc55a21e3d9e5a)
Signed-off-by: Hubert Zhang <hubert.zyk@gmail.com>
2024-02-19 12:21:16 +00:00
Ruize Tang
62a4bb3be9 etcd: Add OpenBSD-specific files for SetKeepAlivePeriod handling
Added keepalive_listener_unix.go and keepalive_listener_openbsd.go to
handle SetKeepAlivePeriod() specially on OpenBSD.

Signed-off-by: Ruize Tang <1466040111@qq.com>
2024-02-06 19:39:09 +01:00
Allen Ray
3d64877dc2 [3.5] Update to go1.21
Signed-off-by: Allen Ray <alray@redhat.com>
2024-02-02 14:25:53 -05:00
Marek Siarkowicz
f3a27b3745 Don't flock snapshot files
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-01-08 15:06:12 +01:00
sharath sivakumar
7f7b91508d upgrade github.com/stretchr/testify,google.golang.org/genproto/googleapis/api,google.golang.org/grpc to make it consistent
Signed-off-by: sharath sivakumar <sharath.sivakumar@mollie.com>
2023-11-15 17:10:40 +01:00
Wei Fu
7816df7f92 Fix panic in etcd validate secure endpoints
Backport: #13824

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-09 21:06:33 +08:00
James Blair
2a7bffc63e
Backport export reuse-port and reuse-address
so that they cabn be configured via config file.

Co-authored-by: Shawn Gerrard <shawn.gerrard@gmail.com>
Signed-off-by: James Blair <mail@jamesblair.net>
2023-08-18 12:05:19 +12:00
James Blair
9c7c8c6b3f
Backport update to golang 1.20 minor release.
Signed-off-by: James Blair <mail@jamesblair.net>
2023-08-11 21:16:01 +12:00
James Blair
358bcf3fb6
Backport tls 1.3 support.
Signed-off-by: James Blair <mail@jamesblair.net>
2023-03-15 14:10:14 +13:00
James Blair
f0aa228dbf
Updated go to 1.19.7.
Mitigates CVE-2023-24532.

Signed-off-by: James Blair <mail@jamesblair.net>
2023-03-08 22:33:13 +13:00
James Blair
183af509f6
Formatted source code for go 1.19.6.
Signed-off-by: James Blair <mail@jamesblair.net>
2023-02-20 21:33:59 +13:00
James Blair
b9568dba32
Fix regression in timestamp resolution
Historic capnslog timestamps are in microsecond resolution. We need to match that when we migrate to the zap logger.

Signed-off-by: James Blair <mail@jamesblair.net>
2023-02-03 21:45:45 +13:00
Benjamin Wang
78e739083e bump github.com/stretchr/testify to v1.8.1
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-01-31 07:12:46 +08:00
Benjamin Wang
f12f162587
Merge pull request #15018 from ahrtr/deps_3.5_20221219
[3.5] Security: address HIGH Vulnerabilities
2022-12-19 17:41:52 +08:00
Benjamin Wang
a612b9285f format the source code and tidy the dependencies using go 1.17.13
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-19 13:26:59 +08:00
Benjamin Wang
69ee8a83ab bump go version to 1.17.13
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-19 13:23:16 +08:00
Tero Saarni
96ca27a3f2 client/pkg/v3: fixes Solaris build of transport
Add empty implementation for reuse port socket option since Solaris does not
support SO_REUSEPORT.

(cherry picked from commit af626ebfdeb46c1025f9a717959b241fecc44d0a)

Conflicts:
	client/pkg/transport/sockopt_unix.go

Signed-off-by: Andrew Stormont <andyjstormont@gmail.com>
2022-12-13 00:12:15 +00:00
Aleksandr Razumov
c91978077b client/pkg/fileutil: add missing logger to {Create,Touch}DirAll
Also populate it to every invocation.

Signed-off-by: WangXiaoxiao <1141195807@qq.com>
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-11-17 14:08:30 +01:00
Benjamin Wang
ec6f0a74ba
Merge pull request #14500 from dusk125/release-3.5
Release-3.5: server/etcdmain: add configurable cipher list to gRPC proxy listener
2022-10-16 06:35:17 +08:00
Benjamin Wang
f6c4c84da3 etcdserver: added more debug log for the purgeFile goroutine
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-10-12 19:28:32 +08:00
Sahdev Zala
db55011d7c server/etcdmain: add configurable cipher list to gRPC proxy listener
Signed-off-by: Allen Ray <alray@redhat.com>
2022-09-29 11:41:57 -04:00
Benjamin Wang
9ea5b1ba22 Refactor the keepAliveListener and keepAliveConn
Only `net.TCPConn` supports `SetKeepAlive` and `SetKeepAlivePeriod`
by default, so if you want to warp multiple layers of net.Listener,
the `keepaliveListener` should be the one which is closest to the
original `net.Listener` implementation, namely `TCPListener`.

Also refer to https://github.com/etcd-io/etcd/pull/14356

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-08-20 15:03:15 +08:00
Benjamin Wang
db0b67e8a0 Add FileReader and FileBufReader utilities
The FileReader interface is the wrapper of io.Reader. It provides
the fs.FileInfo as well. The FileBufReader struct is the wrapper of
bufio.Reader, it also provides fs.FileInfo.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-17 09:00:43 +08:00
Jordan Liggitt
081b4e2a0f Add unit test for canonical SRV records 2022-04-15 15:33:31 -04:00
Jordan Liggitt
76564778a9 Revert "trim the suffix dot from the srv.Target for etcd-client DNS lookup"
This reverts commit 4f51cc1d9a6d31ae5622fe50c90f03962a7ee3a4.
2022-04-15 15:33:16 -04:00
Marek Siarkowicz
a060b42e47 server: Use default logging configuration instead of zap production one
This fixes problem where logs json changes format of timestamp.
2022-04-01 12:23:44 +02:00
ahrtr
4f51cc1d9a trim the suffix dot from the srv.Target for etcd-client DNS lookup 2022-02-19 05:55:19 +08:00
Lili Cosic
9fee8bf95e Update client_golang dependency to v1.11.0 2021-06-07 15:54:57 +02:00
Sam Batschelet
ab20aa29a0 version: 3.5.0-rc.0
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-06-03 21:39:37 -04:00
Piotr Tabor
a5523be415 Update of dependencies:
- zap-1.17.0
  - bbolt-1.3.6
  - grpc -1.38
  - github.com/coreos/go-systemd/v22 v22.3.2
  - github.com/google/btree v1.0.1
  - github.com/json-iterator/go v1.1.11
2021-06-03 15:19:33 +02:00
Sam Batschelet
6cd7c02873
Merge pull request #13055 from ptabor/20210528-beckport-integration-fixes 2021-05-30 16:35:15 -04:00
Piotr Tabor
3f13d3a2d5 integration.BeforeTest can be run without leak-detection. 2021-05-28 10:01:36 +02:00
Piotr Tabor
4af7fc393c Expose clientv3.CreateDefaultZapLoggerConfig
Need raised in
f3f4259d3f.

BTW: Alligned the implementation of grpclogs with what we use in embed server, so reduced the client code dependencies.
2021-05-26 12:52:46 +02:00
Sam Batschelet
b3f16d6691 version: 3.5.0-beta.4
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-05-25 20:48:48 -04:00
Piotr Tabor
8981afb6f5 Fix unit tests logging config. 2021-05-14 22:45:57 +02:00
Piotr Tabor
f5c26814ab Loggers to catch the e2e flake. 2021-05-14 22:42:31 +02:00
Lili Cosic
1a718a958e Add initial Tracing with OpenTelemetry 2021-05-10 10:44:40 +02:00
Piotr Tabor
451f65d661
Merge pull request #12908 from ptabor/20210429-client-retry-logging
Clientv3 (retry interceptor) logs should use the configured logger
2021-04-29 19:25:04 +02:00
Piotr Tabor
ffea1537d4 ClientV3 tests use integration.NewClient that configures proper logger. 2021-04-29 18:18:34 +02:00
Piotr Tabor
f53b70facb Embed: In case KVStoreHash verification fails, close the backend.
In case of failed verification, the server used to keep opened backend
(so the file was locked on OS level).
2021-04-29 11:51:25 +02:00
Piotr Tabor
06ba0fc5a2
Merge pull request #12846 from pyiyun/fix-snaptmpfile-bug
etcdserver: remove temp files in snap dir when etcdserver starting
2021-04-17 12:58:46 +02:00
Piotr Tabor
17b982382e Fix TestSnapshotV3RestoreMultiMemberAdd flakes (leaks)
- most important: unix's socket transport should not keep idle
connections. For top-level Transport we close them using:
    f3c518025e/server/etcdserver/api/rafthttp/transport.go (L226)
    but currently we don't have access to close them witing the nest (unix) transport. Short idle deadline is good enough.

  - Use dialContext (instead of dial) to make sure context is passed down the stack
  - Make sure Context is cancelled as soon as the operation is done in pipeline
  - nit: use dedicated method to yeld goroutines.

Tested with:
```
d=$(date +"%Y%m%d_%H%M")
(cd tests && go test --timeout=60m ./integration/snapshot -run TestSnapshotV3RestoreMultiMemberAdd -v --count=180 2>&1 | tee log_${d}.log)
```

There were transports & cmux leaked:

```
   leak.go:118: Test appears to have leaked a Transport:
        internal/poll.runtime_pollWait(0x7f6c5c3784c8, 0x72, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/runtime/netpoll.go:222 +0x55
        internal/poll.(*pollDesc).wait(0xc003296298, 0x72, 0x0, 0x18, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:87 +0x45
        internal/poll.(*pollDesc).waitRead(...)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:92
        internal/poll.(*FD).Read(0xc003296280, 0xc0031f60a8, 0x18, 0x18, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/internal/poll/fd_unix.go:166 +0x1d5
        net.(*netFD).Read(0xc003296280, 0xc0031f60a8, 0x18, 0x18, 0x18, 0xc0009056e2, 0x203000)
        	/usr/lib/google-golang/src/net/fd_posix.go:55 +0x4f
        net.(*conn).Read(0xc000010258, 0xc0031f60a8, 0x18, 0x18, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/net/net.go:183 +0x91
        github.com/soheilhy/cmux.(*bufferedReader).Read(0xc0003d24e0, 0xc0031f60a8, 0x18, 0x18, 0xc0003d24d0, 0xc0009056e2, 0xc000278400)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/buffer.go:53 +0x12d
        github.com/soheilhy/cmux.hasHTTP2Preface(0x1367e20, 0xc0003d24e0, 0x7f6c5c699f40)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/matchers.go:195 +0x8a
        github.com/soheilhy/cmux.matchersToMatchWriters.func1(0x7f6c5c699f40, 0xc000010258, 0x1367e20, 0xc0003d24e0, 0xc000010258)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:128 +0x39
        github.com/soheilhy/cmux.(*cMux).serve(0xc003228690, 0x138c410, 0xc000010258, 0xc00327f740, 0xc0059ba860)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:192 +0x1e7
        created by github.com/soheilhy/cmux.(*cMux).Serve
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:179 +0x191

        internal/poll.runtime_pollWait(0x7f6c5c60f3f0, 0x72, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/runtime/netpoll.go:222 +0x55
        internal/poll.(*pollDesc).wait(0xc000d53018, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:87 +0x45
        internal/poll.(*pollDesc).waitRead(...)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:92
        internal/poll.(*FD).Read(0xc000d53000, 0xc000cfd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/internal/poll/fd_unix.go:166 +0x1d5
        net.(*netFD).Read(0xc000d53000, 0xc000cfd000, 0x1000, 0x1000, 0x3, 0x3, 0x1000000000001)
        	/usr/lib/google-golang/src/net/fd_posix.go:55 +0x4f
        net.(*conn).Read(0xc00031a570, 0xc000cfd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/net/net.go:183 +0x91
        net/http.(*persistConn).Read(0xc00093b320, 0xc000cfd000, 0x1000, 0x1000, 0x577750, 0x60, 0x0)
        	/usr/lib/google-golang/src/net/http/transport.go:1933 +0x77
        bufio.(*Reader).fill(0xc005702fc0)
        	/usr/lib/google-golang/src/bufio/bufio.go:101 +0x108
        bufio.(*Reader).Peek(0xc005702fc0, 0x1, 0xc00077c660, 0xc003b082a0, 0xc000d08de0, 0x5ae586, 0x11dd6c0)
        	/usr/lib/google-golang/src/bufio/bufio.go:139 +0x4f
        net/http.(*persistConn).readLoop(0xc00093b320)
        	/usr/lib/google-golang/src/net/http/transport.go:2094 +0x1a8
        created by net/http.(*Transport).dialConn
        	/usr/lib/google-golang/src/net/http/transport.go:1754 +0xdaa

        net/http.(*persistConn).writeLoop(0xc00093b320)
        	/usr/lib/google-golang/src/net/http/transport.go:2393 +0xf7
        created by net/http.(*Transport).dialConn
        	/usr/lib/google-golang/src/net/http/transport.go:1755 +0xdcf

        sync.runtime_Semacquire(0xc0059ba868)
        	/usr/lib/google-golang/src/runtime/sema.go:56 +0x45
        sync.(*WaitGroup).Wait(0xc0059ba860)
        	/usr/lib/google-golang/src/sync/waitgroup.go:130 +0x65
        github.com/soheilhy/cmux.(*cMux).Serve.func1(0xc003228690, 0xc0059ba860)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:158 +0x56
        github.com/soheilhy/cmux.(*cMux).Serve(0xc003228690, 0x13698c0, 0xc00377a0f0)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:173 +0x115
        go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers.func1(0xc0007cc360, 0x122b75f)
        	/home/ptab/corp/etcd/server/embed/etcd.go:518 +0x2b9
        go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers.func3(0xc00036d080, 0xc0059330a0)
        	/home/ptab/corp/etcd/server/embed/etcd.go:549 +0x182
        created by go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers
        	/home/ptab/corp/etcd/server/embed/etcd.go:543 +0x73a
--- FAIL: TestSnapshotV3RestoreMultiMemberAdd (17.74s)
```
2021-04-16 20:17:28 +02:00