117 Commits

Author SHA1 Message Date
Benjamin Wang
033c3f3d94 get golangci-lint's version from tools/mod for the Makefile target install-golangci-lint
Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
2023-11-16 14:12:58 +00:00
Etienne Simard
4ec3cf8cfa tests: fix failed prow test by adding golangci-lint to Makefile
Missing binary, fixes golangci-lint command not found on prow jobs

Signed-off-by: Etienne Simard <esimard@ripple.com>
2023-11-15 14:22:58 -06:00
Wei Fu
81fe610bd5 *: enable markdown_marker checker
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-10-13 14:00:25 +08:00
Wei Fu
46df6abb03 *: remove goimports(_fix)_pass and add lint_fix_pass
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-21 21:18:36 +08:00
Wei Fu
502879405f *: lint_pass should use global golangci.yaml
Disable failed linters and enable it by #16610.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-21 16:10:45 +08:00
Wei Fu
2b1e181c8b *: Use golangcilint_pass to run staticcheck and ineffassign
Copy the tools/.golangci.yaml and run the linters for which we have
already fixed. The temp .golangci.yaml will be removed when we fixes all
the linters' issues.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-21 12:34:17 +08:00
Wei Fu
df86cadd8b *: fix ineffassign lint
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-19 22:19:19 +08:00
Wei Fu
5e3910d96c *: fix govet-shadow lint
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-19 20:24:01 +08:00
Marek Siarkowicz
eb32d9cccc tests: Add support for lazyfs
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-07-27 13:42:38 +02:00
James Blair
2791422f77
Ensure goimports can be fixed individually.
Signed-off-by: James Blair <mail@jamesblair.net>
2023-06-16 21:47:46 +12:00
James Blair
8c483f31ad
Ensure Makefile PHONY targets match real target names.
Signed-off-by: James Blair <mail@jamesblair.net>
2023-06-16 21:20:45 +12:00
Jintao Zhang
0f32b588be ci: move .golangci.yaml to tools dir
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2023-06-06 19:08:31 +08:00
Jintao Zhang
8f887d3a79 ci: add fix-yamllint rule in Makefile
Two rules have been specified for yamlfmt here:
* Automatically add the header `---` to YAML files.
* Preserve line breaks.

ref:
https://github.com/google/yamlfmt/blob/main/docs/config-file.md#configuration-1

Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
Co-authored-by: Marek Siarkowicz <siarkowicz@google.com>
2023-06-01 09:06:36 +08:00
Jintao Zhang
b0b922cd71 ci: Introduce yamllint for actions workflow files
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2023-05-26 16:34:23 +08:00
James Blair
ab65ee3d01
Consolidate etcd dockerfiles.
We can consolidate by using docker build args to create the individual platform Dockerfile.

Signed-off-by: James Blair <mail@jamesblair.net>
2023-04-26 17:09:25 +12:00
Marek Siarkowicz
d0cd94a520
Merge pull request #15561 from serathius/makefile-entrypoint
Use makefile as entrypoint in CI
2023-04-01 09:02:09 +02:00
Marek Siarkowicz
f53594309a Use makefile as entrypoint in CI
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-30 11:15:19 +02:00
Chao Chen
a8c20f931e add etcd test analyzer build and integrate into measure-test-flakiness workflow
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-03-28 11:30:32 -07:00
Marek Siarkowicz
d03ac88b36 tests: Document robustness tests
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-28 15:08:43 +02:00
Hanning Lin
8bc6b289fb add new goimport_pass
Signed-off-by: Hanning Lin <liam.hanninglin@outlook.com>
2023-03-16 05:50:17 +00:00
Marek Siarkowicz
d475cf81a0 tests: Rename linearizability tests to robustness
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-02-26 14:36:18 +01:00
Benjamin Wang
1fe7b9793a etcdserver: add failpoints walBeforeSync and walAfterSync
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-02-08 09:08:09 +08:00
Benjamin Wang
b9b7c9328e tools: fix the CSV format error and add build target 'tools' in Makefile
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-01-30 16:35:41 +08:00
Thomas Jungblut
ab3c530b92 add linearizability nightlies for release 3.4/3.5
This CL refactors the tests to reuse a single workflow that has
parameters. This is then reused for PRs/pushes and the nightlies.

Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-01-16 11:00:16 +01:00
guiyong.ou
4f238837aa Make sure shellcheck exist
Signed-off-by: guiyong.ou <guiyong.ou@daocloud.io>
2022-12-05 15:36:16 +08:00
Bhargav Ravuri
dadd7358f4 Makefile: Read GO_BUILD_FLAGS from env and prepend
Build flags read from env GO_BUILD_FLAGS are prepended to
flags in Makefile and sent to scripts/build.sh for building
executables (etcd, etcdctl & etcdutl).

Signed-off-by: Bhargav Ravuri <bhargav.ravuri@infracloud.io>
2022-12-01 09:43:26 +05:30
Marek Siarkowicz
dd4d69ca91 tests: Cleanup gofail
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-11-27 20:35:39 +01:00
spacewander
e73a25a1d0 ci: ensure the generated code is up-to-date
See https://github.com/etcd-io/etcd/pull/14612#issue-1419792069
Signed-off-by: spacewander <spacewanderlzx@gmail.com>
2022-10-31 19:58:21 +08:00
Marek Siarkowicz
8ce81a1624
Merge pull request #14611 from serathius/issue14370
tests: Add linearizability tests scenario for #14370
2022-10-25 14:03:39 +02:00
Marek Siarkowicz
837819860b tests: Add linearizability tests scenario for #14370
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-10-24 13:36:12 +02:00
Samuele Resca
3d9c5c6166 Adding fuzz test on v3rpc interfaces.
Signed-off-by: Samuele Resca <sr7@ad.datcon.co.uk>
Signed-off-by: Samuele Resca <samuele.resca@gmail.com>
2022-10-23 13:46:10 +01:00
Marek Siarkowicz
069e26e284 tests: Validate etcd linearizability
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-10-23 06:41:38 +02:00
Marek Siarkowicz
bea478266e makefile: Raname targets update* to fix* to distinquish from update_dep
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-09-20 13:58:17 +02:00
Marek Siarkowicz
bb139b15f8 makefile: Don't run update_dep.sh as it's not a check
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-09-20 13:55:51 +02:00
Marek Siarkowicz
05104ee9a7 makefile: Remove verify-revive as it is already run by golangci
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-09-20 13:55:51 +02:00
Marek Siarkowicz
b8347edf3b makefile: Split fmt into separate verify commands
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-09-16 22:37:10 +02:00
Marek Siarkowicz
7bb114d171 makefile: Make static analysis commands easier to use
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-09-15 11:54:14 +02:00
Marek Siarkowicz
a36bcfff1f Improve contributor guidelines and simplify development commands
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-09-14 09:39:32 +02:00
Ramsés Morales
3cdc4225e1 Followup to "Add golangci-lint config file"
Addressed comments by @ldez and added `lint-fix` target.

Signed-off-by: Ramsés Morales <ramses@gmail.com>
2022-07-28 17:55:22 -07:00
Ramsés Morales
a7a48168a0 Add golangci-lint config file and make target.
Here we add file `.golangci.yaml`, to have a common configuration for
static analysis. If you use the following IDEs/editors, they will read
the file:

- `vim` with vim-go.
- VSCode.

The initial configuration file is based on Kubernetes' HEAD. We removed
a custom kubernetes-linter for custom kubernetes-logging. We also
changed to Golang version 1.17 (etcd's current target Golang version)
from 1.18.

Also, we added a new target to `Makefile`: `lint`. NOTE that auto-fixing
should be part of a later commit, once we are all happy with how the
configuration file is working for us.

As expected, this change fixes two issues found by this `.golangci.yaml`
in file `contrib/lock/client/client.go`:

- Dead code, removed.
- Innefective assignment, removed.

Finally, we are updating `CONTRIBUTING.md` to mention `golangci-lint`.

We will add a GitHub-action to run `golangci-lint` in a future change.

Local testing done:

- `make build`.
- `make test`.

Both are happy.

This is the initial step to fix
https://github.com/etcd-io/etcd/issues/14164.

Signed-off-by: Ramsés Morales <ramses@gmail.com>
2022-07-28 17:55:08 -07:00
Eduardo Patrocinio
5a93b087a6 Separate the build library functions and add a top level tools script
The current Makefile doesn't allow the compilation of the tools directory.
This commit creates a build library file, updates the Makefile
and a top level script fod building tools.

To build the tools, you can run make build_tools.

As before, you can run make build to build etcd binaries.
2022-06-10 11:43:10 -04:00
Eduardo Patrocinio
73766a92fb Update build file to run with an argument
The current Makefile doesn't allow the compilation of the tools directory.
This commit updates the Makefile and the build file to add an option
to build the tools.

To build the tools, you can run make build_tools.
2022-06-07 11:47:28 -04:00
Piotr Tabor
8a22d640a7 add OUTPUT_FILE='foo.log' env variable support to test.sh script.
Side consequence of this is that `make test-smoke` can become interactive again.
2022-04-18 22:54:51 +02:00
Piotr Tabor
47bb48dfdc
Merge pull request #13869 from wking/test-fail-exit-code-with-log-redirect-upstream
Makefile: Drop log tee calls
2022-04-07 09:12:19 +02:00
W. Trevor King
c59cae5aaa Makefile: Drop log tee calls
We've had these since in one form or another since 23a302364c
(Makefile: initial commit, 2017-09-29), but in at least some cases the
underlying shell does not pipefail, a test failure gets swallowed, and
the make call exits zero despite failing the tests [1]:

  $ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift_etcd/109/pull-ci-openshift-etcd-openshift-4.11-unit/1509260812278042624/artifacts/test/build-log.txt
  TEST_OPTS: PASSES='unit'
  log-file: test-MTY0ODY3MTA1MQo.log
  PASSES='unit' ./test.sh 2>&1 | tee test-MTY0ODY3MTA1MQo.log
  % env GO111MODULE=off go get github.com/myitcv/gobin
  Running with --race
  Starting at: Wed Mar 30 20:10:52 UTC 2022

  'unit' started at Wed Mar 30 20:10:52 UTC 2022
  % (cd api && env go test -short -timeout=3m --race ./...)
  stderr: authpb/auth.pb.go:12:2: open /go/pkg/mod/github.com/gogo/protobuf@v1.3.2/gogoproto: permission denied
  stderr: authpb/auth.pb.go:13:2: open /go/pkg/mod/github.com/golang/protobuf@v1.5.2/proto: permission denied
  stderr: etcdserverpb/rpc.pb.go:17:2: open /go/pkg/mod/google.golang.org/genproto@v0.0.0-20210602131652-f16073e35f0c/googleapis/api/annotations: permission denied
  stderr: etcdserverpb/rpc.pb.go:18:2: open /go/pkg/mod/google.golang.org/grpc@v1.38.0: permission denied
  stderr: etcdserverpb/rpc.pb.go:19:2: open /go/pkg/mod/google.golang.org/grpc@v1.38.0/codes: permission denied
  stderr: etcdserverpb/rpc.pb.go:20:2: open /go/pkg/mod/google.golang.org/grpc@v1.38.0/status: permission denied
  stderr: etcdserverpb/gw/rpc.pb.gw.go:17:2: open /go/pkg/mod/github.com/golang/protobuf@v1.5.2/descriptor: permission denied
  stderr: etcdserverpb/gw/rpc.pb.gw.go:19:2: open /go/pkg/mod/github.com/grpc-ecosystem/grpc-gateway@v1.16.0/runtime: permission denied
  stderr: etcdserverpb/gw/rpc.pb.gw.go:20:2: open /go/pkg/mod/github.com/grpc-ecosystem/grpc-gateway@v1.16.0/utilities: permission denied
  FAIL: (code:1):
    % (cd api && env go test -short -timeout=3m --race ./...)
  stderr: etcdserverpb/gw/rpc.pb.gw.go:23:2: open /go/pkg/mod/google.golang.org/grpc@v1.38.0/grpclog: permission denied
  stderr: version/version.go:23:2: open /go/pkg/mod/github.com/coreos/go-semver@v0.3.0/semver: permission denied
  FAIL: 'unit' failed at Wed Mar 30 20:10:52 UTC 2022
  ! egrep "(--- FAIL:|DATA RACE|panic: test timed out|appears to have leaked)" -B50 -A10 test-MTY0ODY3MTA1MQo.log

We can't drop the log aggregation, because the log files are used for
the panic/race grepping.  But I'm dropping the tee (so no more
synchronous updates, but we no longer have to worry about pipefail
handling).  And then if the test script fails, I'm dumping the log
file to stdout and exiting 1, so the overall run fails.

[1]: https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_etcd/109/pull-ci-openshift-etcd-openshift-4.11-unit/1509260812278042624
2022-04-01 11:06:50 -07:00
Manuel Rüger
29905029f6 Makefile: Additional logic fix
Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2022-03-31 11:18:36 +02:00
Manuel Rüger
ec29b9ee36 Makefile: Fix wrong target
Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2022-03-31 09:48:21 +02:00
Manuel Rüger
c544b2a2a5 Update go to 1.17.8 2022-03-23 20:11:12 +01:00
Marek Siarkowicz
396efd9608 Cleanup scripts in root directory 2022-02-02 13:50:44 +01:00
Manuel Rüger
ce086a4ba6 *: Bump to go 1.17.6
Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2022-01-12 03:13:27 +01:00