789 Commits

Author SHA1 Message Date
Marek Siarkowicz
1bb4c9558d tests: Update gofail to v0.1.0
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-12-10 14:49:20 +01:00
Benjamin Wang
1ba246e1d8 bump golang.org/x/ to v0.4.0
Found 1 known vulnerability.

Vulnerability #1: GO-2022-1144
  An attacker can cause excessive memory growth in a Go server
  accepting HTTP/2 requests. HTTP/2 server connections contain a
  cache of HTTP header keys sent by the client. While the total
  number of entries in this cache is capped, an attacker sending
  very large keys can cause the server to allocate approximately
  64 MiB per open connection.

  Call stacks in your code:
Error:       tools/etcd-dump-metrics/main.go:158:5: go.etcd.io/etcd/v3/tools/etcd-dump-metrics.main calls go.etcd.io/etcd/server/v3/embed.StartEtcd, which eventually calls golang.org/x/net/http2.Server.ServeConn

  Found in: golang.org/x/net/http2@v0.2.0
  Fixed in: golang.org/x/net/http2@v0.4.0
  More info: https://pkg.go.dev/vuln/GO-2022-1144
Error: Process completed with exit code 3.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-09 08:54:48 +08:00
Benjamin Wang
971e42d631 Resolve build and test errors
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 15:43:48 +08:00
Benjamin Wang
cc58edecf2 raft: add raft into the tools/mod
Previously etcdservers depends on raft/raftpb/raft.proto directly.
After moving raft to a separate repo, we need to add raft to the
tools/mod, and get raft included in the -I protc flags.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 15:36:41 +08:00
Benjamin Wang
ff337cc993 tools/etcd-dump-logs: generate go.mod and go.sum for the tool
Add go.mod and go.sum to fix issue below,
```
$ go build
../../server/etcdserver/api/snap/snapshotter.go:33:2: missing go.sum entry for module providing package go.etcd.io/raft/v3 (imported by go.etcd.io/etcd/server/v3/etcdserver/api/snap); to add:
	go get go.etcd.io/etcd/server/v3/etcdserver/api/snap@v3.6.0-alpha.0
../../server/storage/wal/walpb/record.pb.go:14:2: missing go.sum entry for module providing package go.etcd.io/raft/v3/raftpb (imported by go.etcd.io/etcd/v3/tools/etcd-dump-logs); to add:
	go get go.etcd.io/etcd/v3/tools/etcd-dump-logs
```

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 13:27:38 +08:00
Benjamin Wang
bb6e892763 tools/etcd-dump-logs: update tool to use the new raft module go.etcd.io/raft/v3
Just replaced all go.etcd.io/etcd/raft/v3 with go.etcd.io/raft/v3
under tools/etcd-dump-logs.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 13:12:19 +08:00
Benjamin Wang
3f86db5e53 bump golang.org/x imports to address CVEs
Reference: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-32149

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-30 05:03:43 +08:00
Marek Siarkowicz
dd4d69ca91 tests: Cleanup gofail
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-11-27 20:35:39 +01:00
Benjamin Wang
150ea9d880 tool: bump gofail version
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-18 15:19:18 +08:00
Sasha Melentyev
2c9c209eb6 all: Changing Printf and friends to Print if there is no formatting
Signed-off-by: Sasha Melentyev <sasha@melentyev.io>
2022-11-15 22:11:23 +03:00
Marek Siarkowicz
2a1055c7f3 raft: Remove dependency on etcd api
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-11-08 13:56:46 +01:00
qsyqian
946d120c7c etcd-dump-db: enhance metaDecoder
Signed-off-by: qsyqian <qsyqian@gmail.com>
2022-10-26 17:32:00 +08:00
Marek Siarkowicz
837819860b tests: Add linearizability tests scenario for #14370
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-10-24 13:36:12 +02:00
demoManito
7020e2fd2b fix lint
Signed-off-by: demoManito <1430482733@qq.com>
2022-09-29 13:57:55 +08:00
demoManito
0a08fa2b43 golangci: add import order check
Signed-off-by: demoManito <1430482733@qq.com>
2022-09-29 11:30:04 +08:00
Benjamin Wang
7f10dccbaf Bump go 1.19: update all the dependencies and go.sum files
1. run ./scripts/fix.sh;
2. cd tools/mod; gofmt -w . & go mod tidy;

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-22 08:47:46 +08:00
Benjamin Wang
cd0b1d0c66 Bump go 1.19: upgrade go version to 1.19 in all go.mod files
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-09-22 08:47:46 +08:00
demoManito
72cf0cc04a etcd: modify declaring empty slices
declare an empty slice to var s []int replace  s :=[]int{}, https://github.com/golang/go/wiki/CodeReviewComments#declaring-empty-slices

Signed-off-by: demoManito <1430482733@qq.com>
2022-09-16 14:41:14 +08:00
Cenk Alti
f8b65d53ba
Add revive to the list of linters.
Part of "Enhance the static-analysis workflow" issue. #14164

Signed-off-by: Cenk Alti <cenkalti@gmail.com>
2022-08-27 13:41:14 -04:00
SimFG
508fa22a91 benchmark: Fix the panic when running benchmark watch
Signed-off-by: SimFG <1142838399@qq.com>
2022-07-15 11:33:11 +08:00
Mikel Olasagasti Uranga
89637a4223 Tools/benchmark: migrate cheggaaa/pb.v1 to cheggaaa/pb/v3
etcdctl/ctlv3: migrate cheggaaa/pb.v1 to cheggaaa/pb/v3

This commit also changes the format of the progress bar, from using a
custom progress bar to the default provided by the library.

Old behaviour:

./benchmarkv1  put
 0 / 10000 B                                                   !   0.00%
 3987 / 10000 Boooooooooooooom                                 !  39.87%
 10000 / 10000 Boooooooooooooooooooooooooooooooooooooooooooo! 100.00% 1s

New behaviour:

./benchmark put
 6536 / 10000 [----------------------->________________] 65.36% 7053 p/s
10000 / 10000 [---------------------------------------] 100.00% 7581 p/s

Signed-off-by: Mikel Olasagasti Uranga <mikel@olasagasti.info>
2022-06-20 15:47:23 +02:00
ahrtr
c75bdef4d2 add etcd tool binaries into .gitignore 2022-05-21 06:58:05 +08:00
Marek Siarkowicz
d58f971895 tests: Use zaptest.NewLogger in places we missed 2022-05-06 12:23:20 +02:00
Sahdev Zala
02236b2773 Doc: add installtion instructions for tools
Add missing install commands. Update benchmark to go install as go
get usage is deprecated. Also, improve overall instructions and
experience.

related #13991
2022-04-28 01:13:02 -04:00
Piotr Tabor
55a7c91de9 test.sh: Tools run correctly on OsX
- The location of installed tools is found using `go list`
- ETCD_VERIFY env variable is printed in repro command line.
2022-04-17 23:28:26 +02:00
Manuel Rüger
dedb661d92 tools/mod: Update tools
github.com/google/addlicense v0.0.0-20210428195630-6d92264d7170 -> v1.0.0
github.com/gordonklaus/ineffassign v0.0.0-20200809085317-e36bfde3bb78 -> v0.0.0-20210914165742-4cc7213b9bc8
github.com/grpc-ecosystem/grpc-gateway v1.14.6 -> v1.16.0
github.com/hexfusion/schwag v0.0.0-20170606222847-b7d0fc9aadaa -> v0.0.0-20211117114134-3ceb0191ccbf
github.com/mgechev/revive v1.0.2 -> v1.2.0
github.com/mikefarah/yq/v3 v3.0.0-20201125113350-f42728eef735 -> v4.24.2
gotest.tools v2.2.0+incompatible -> v3.1.0
gotest.tools/gotestsum v0.3.5 -> v1.7.0
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc -> v0.3.0
mvdan.cc/unparam v0.0.0-20200501210554-b37ab49443f7 -> v0.0.0-20220316160445-06cc5682983b

Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2022-04-07 14:27:51 +02:00
Piotr Tabor
047e61df7a
Merge pull request #13880 from ahrtr/fix_dump_logs_panic
etcd-dump-logs will panic if there is no WAL entry after the snapshot
2022-04-06 09:25:17 +02:00
Marek Siarkowicz
73fc864247 tests: Pass logger to backend 2022-04-05 15:53:38 +02:00
Marek Siarkowicz
804fddf921 tests: Use zaptest.NewLogger in tests 2022-04-04 13:03:15 +02:00
ahrtr
543c87cc38 etcd-dump-logs will panic if there is no WAL entry after the snapshot 2022-04-04 18:58:18 +08:00
ahrtr
bbc81bb6b7 enhance etcd-dump-db to display keys in meta more friendly 2022-03-23 18:32:13 +08:00
Luiz Gustavo
b6a2d2d48a
tools: Add -wal-dir flag on etcd-dump-logs to make it compatible with ETCD_WAL_DIR variable (#13672) 2022-03-07 21:29:20 -08:00
Piotr Tabor
f80f477073
Merge pull request #13644 from Juneezee/refactor/t.TempDir
*: use `T.TempDir` to create temporary test directory
2022-02-21 19:52:37 +01:00
Sahdev Zala
bba3937f69
Merge pull request #13649 from serathius/scripts
Cleanup scripts in root directory
2022-02-11 15:40:53 -05:00
ahrtr
900fc8dd7e replace gobin with go install
The repository github.com/myitcv/gobin has already been archived,
and the `go install` command accepts arguments with version suffixs
starting from 1.16 (for example, go install example.com/cmd@v1.0.0).
So there is no reason to continue to use gobin.
2022-02-08 05:41:21 +08:00
Marek Siarkowicz
396efd9608 Cleanup scripts in root directory 2022-02-02 13:50:44 +01:00
Marek Siarkowicz
0c67c5ca49 tools: Move external packages constant to top level 2022-01-28 16:12:00 +01:00
Marek Siarkowicz
f4187b4460 tools: Improve proto annotation documentation 2022-01-28 11:40:55 +01:00
Marek Siarkowicz
d865bb96f1 server: Refactor wal version to use visitor pattern 2022-01-26 15:50:14 +01:00
Marek Siarkowicz
6d808e5d7d *: Add static validation to etcd_version proto annotation 2022-01-26 15:50:14 +01:00
Eng Zer Jun
e9edc5432c
*: use T.TempDir to create temporary test directory
The directory created by `T.TempDir()` and is automatically removed when
the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-01-25 00:00:46 +08:00
Piotr Tabor
1237b3576c
Merge pull request #13487 from mrueg/go-1.17.3
*: Bump to go 1.17.6
2022-01-15 17:01:17 +01:00
Marek Siarkowicz
4032d4f66a Remove CodeQL errors 2022-01-13 14:29:09 +01:00
Manuel Rüger
cf7e8b3535 go.mod: Bump golang.org/x/net dependency; regenerate go.sum
Bumps golang.org/x/net dependency due to fix CVE-2021-44716
as requested in https://github.com/etcd-io/etcd/pull/13487#issuecomment-997065540

Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2022-01-12 03:00:08 +01:00
Liang Zheng
0cc789d81d update dump db tool
Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
2022-01-01 00:13:33 +08:00
Liang Zheng
7eb4bed254 fix etcd benchmark binary not found
Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
2021-11-24 09:56:33 +00:00
Clivern
729af2ad2e fix undefined io 2021-11-10 20:12:28 +01:00
Eng Zer Jun
2a151c8982
*: move from io/ioutil to io and os packages
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-10-28 00:05:28 +08:00
Lili Cosic
e6e279a14d *: Bump go to go v1.17.2 2021-10-27 13:44:54 +02:00
Chao Chen
9084acceac tools/benchmark: add autoSync flag 2021-10-15 22:03:23 -07:00