411 Commits

Author SHA1 Message Date
Benjamin Wang
9fc3b2ad40
Merge pull request #18590 from henrybear327/ci/fix
Add tools/mod to module_dirs
2024-09-19 10:36:12 +01:00
mevrin
5b2faec983
fea: Remane modules_exp to modules_for_bom to improve code and clarity
Signed-off-by: mevrin <matthieu.evrin@gmail.com>
2024-09-18 21:45:07 -04:00
Chun-Hung Tseng
fc901bd019 Add tools/mod to module_dirs
As `tools/mod` also contains the `go.mod` file. We should add it to the
`module_dirs` variable, so that when executing `./scripts/fix.sh`, the
proper checks and fixes can be applied.

To address the issue of broken unit tests and code coverage due to the
directory's lack of Go code, we've introduced a new doc.go file. This
file acts as a placeholder, enabling tools like golangci-lint and
go test to function correctly.

---

Discovered when working on https://github.com/etcd-io/etcd/pull/18575

The directories are checked against the following:
- Command: `find . -type f -name go.mod -exec dirname {} \;`
- Output:
```
./etcdutl
.
./tools/testgrid-analysis
./tools/rw-heatmaps
./tools/mod
./etcdctl
./tests
./server
./api
./client/internal/v2
./client/v3
./client/pkg
./pkg
```

Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
2024-09-18 14:33:13 +02:00
James Blair
7e8e0bf1f9
Ensure commits are signed in scripts/release.sh.
Signed-off-by: James Blair <mail@jamesblair.net>
2024-09-16 11:31:30 +12:00
Ivan Valdes
ce916fbb38
tests: remove Dockerfile
The tests/Dockerfile was outdated (using an obsolete Ubuntu
distribution), and it's not being used by any workflow or prow job. This
Dockerfile was the only one using `scripts/install-marker.sh`, which
used a vendorized binary generated by the etcd project. This is not
needed anymore, as marker releases now provide the binaries.

Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-07-26 15:31:38 -07:00
Ivan Valdes
4f9cd336a6
marker: use released binary when running markdown lint
The markdown linter, marker, now publishes a released version with the
tool. Given that this project is written in Rust, running this check can
now be done by using the statically compiled binary without having the
Rust local environment.

Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-07-13 15:54:46 -07:00
James Blair
059371a02f
Merge pull request #18250 from jmhbnz/main
Remove gsutil acl command for bucket permissions from release.sh
2024-07-04 13:43:54 +12:00
Ivan Valdes
ba4b2bffeb
make: Include tools/mod when checking dependency versions
As tools/mod is not part of the test library's modules, the check to verify
consistent dependency versions ignored it. Explicitly get the dependencies from
this module when running verify-dep.

Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-07-03 15:21:31 -07:00
James Blair
f0246a9f84
Remove gsutil acl command for bucket permissions.
The bucket is already fully public so new releases don't need to run this command.

This will prevent significant terminal spam during the release process as there are thousands of objects in the bucket.

Signed-off-by: James Blair <mail@jamesblair.net>
2024-06-29 17:39:06 +12:00
James Blair
29b4ff2fd8
Bump shellcheck to latest 2024 release.
Signed-off-by: James Blair <mail@jamesblair.net>
2024-06-24 11:24:31 +12:00
Ivan Valdes
dfda61bbbf
make: fix verify-dep
Makefile's target `verify-dep` current behavior is to use `go list` to
check consistent dependency versions from direct dependencies. Ignoring
indirect dependencies in a multi-module project could lead to version
mismatches. If module A imports module B, module B's dependency will be
an indirect dependency in module A. Which can potentially have a version
mismatch. Therefore, use `go mod edit` with indirect dependencies, too.
So it can work with all dependencies defined in go.mod.

Fix displaying dependencies with mismatches, as the old code was
searching with grep just for the prefix, which would show other
dependencies that shared the same prefix.

Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-06-19 09:02:17 -07:00
Ivan Valdes
043096067f
Add run-govulncheck Makefile target
Add a `Makefile` target to run govuln across the submodules. So, it can
be imported into a prow job.

Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-06-15 15:47:13 -07:00
Chun-Hung Tseng
1579c57729 Update codecov config to support commenting code coverage in PRs
Add missing directory fixing go.etcd.io/etcd/etcdutl/v3/::etcdutl/

Note: we have some of the tests written in a way that is
non-deterministic across runs, thus, even when there is no code changes
there might still have slight variation for test coverage [2]

Reference:
[1] https://github.com/etcd-io/etcd/issues/18131
[2] https://docs.codecov.com/docs/unexpected-coverage-changes#reasons-for-indirect-changes

Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
2024-06-12 11:23:31 +02:00
Ivan Valdes
4881e53677
scripts/release: shallow clone repository
Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-05-29 15:14:53 -07:00
Madhav Jivrajani
b915323be6 split out subroutine from fix.sh to makefile target
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
2024-05-14 15:18:30 +05:30
Chun-Hung Tseng
3375e89b11 Fix issue reported by static analyzer
Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
2024-05-09 13:58:29 +02:00
Madhav Jivrajani
f6a35f8786 .*: sync go toolchain version and add ability to verify versions
This commit adds a script to sync the version present in .go-version
across all go.mod files as the toolchain directive. As part of that,
this commit also modifies go.mod files that did not have synced toolchain
directives.

Additionally, this also adds a script to verify all toolchain and go
directives against the version present in .go-version as follows:
(1) The go directive <= version in .go-version
(2) The toolchain directive == version in .go-version

This script runs as part of the `make verify` target, making it run
as a presbumit by default.

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
2024-05-07 17:52:56 +05:30
Madhav Jivrajani
e094139b05 scripts: default to using .go-version's version for tests and builds
Additionally, provide ability to opt-out of the .go-version and use a
custom one via env vars: FORCE_HOST_GO and GO_VERSION.

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
2024-05-07 17:52:56 +05:30
Benjamin Wang
13c0f75b7d
Merge pull request #17877 from tjungblu/17872_status
add quota to endpoint status response
2024-05-03 13:03:23 +01:00
Benjamin Wang
2b1914c262
Merge pull request #17770 from tico88612/feat/update-tools
Test: add tools/rw-heatmaps & tools/testgrid-analysis to module_dirs
2024-04-29 18:35:05 +01:00
zgzhang4
0bd0c4a21b github workflow: drop measure test flakiness
Signed-off-by: zgzhang4 <zgzhang4@iflytek.com>
2024-04-29 23:20:45 +08:00
tico88612
a988f7edc5
script/test_lib: add description about modules
Signed-off-by: tico88612 <17496418+tico88612@users.noreply.github.com>
2024-04-26 20:08:05 +08:00
tico88612
aa53afdce2
test: add tools/{rw-heatmaps,testgrid-analysis} to module_dirs
Signed-off-by: tico88612 <17496418+tico88612@users.noreply.github.com>
2024-04-26 20:07:59 +08:00
Thomas Jungblut
bdcff246cb add quota to endpoint status response
This adds the configured backend quota bytes to the endpoint status response.
related discussion in #17821

Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
2024-04-25 16:44:20 +02:00
Siyuan Zhang
b03368485c testgrid: print out all failed tests for visibility.
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-04-12 12:39:38 -07:00
Benjamin Wang
0168c3e1ce
Merge pull request #17662 from siyuanfoundation/flaky
Add script and workflow to detect flaky tests in testgrid.
2024-04-05 16:58:45 +01:00
Ivan Valdes
5cd14a6031
Revert "tests: temporary disable coverage redesign Go experiment"
With the update to Go 1.22.2, this workaround is no longer needed.

This reverts commit da7ab15f80c06373e03267e11497da71a028a3aa.

Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-04-04 11:17:06 -07:00
Siyuan Zhang
3f37cd2d73 Add script to detect flaky tests in testgrid.
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-04-01 09:41:42 -07:00
James Blair
b82f882c3d
Update release script to use ssh.
From 2021-08-13, GitHub is no longer accepting account passwords when authenticating Git operations. You need to add a PAT (Personal Access Token) instead for HTTPS. We should intead keep things simple and just use ssh to push to GitHub.

Signed-off-by: James Blair <mail@jamesblair.net>
2024-03-31 10:09:51 +13:00
thirdkeyword
fbda591866 fix some typos
Signed-off-by: thirdkeyword <fliterdashen@gmail.com>
2024-03-25 10:34:44 +08:00
Ivan Valdes
da7ab15f80
tests: temporary disable coverage redisign Go experiment
Go v1.22 has an error generating the coverage output. Disable it
temporarily so GitHub workflows can run in the meantime.

Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-03-09 15:07:33 -08:00
Arka Saha
20005e0e84
Make CGO_ENABLED configurable
Signed-off-by: Arka Saha <arkas1@vmware.com>
2024-02-13 13:24:55 +05:30
=
dbfacb4260 Ensure bin directory exists for protoc install
Co-authored-by: James Blair <mail@jamesblair.net>
Signed-off-by: = <abby.crimlis@outlook.com>
2024-01-05 10:29:34 +13:00
=
580ace4d17 Automating install of protoc
Signed-off-by: = <abby.crimlis@outlook.com>
2023-12-10 19:42:30 +13:00
upodroid
f454342bfe fix missing json flag when using VERBOSE=1
Signed-off-by: upodroid <upodroid@users.noreply.github.com>
2023-11-23 12:33:29 +00:00
Tessa Pham
2b7e1c6f82 fix scripts
Signed-off-by: Tessa Pham <hpham111@bloomberg.net>
2023-11-08 00:27:13 -06:00
Marek Siarkowicz
649b365937
Merge pull request #16883 from mjlshen/16760
scripts: macOS support for scripts/genproto.sh
2023-11-08 01:56:45 +01:00
Tessa Pham
c4499adfd5 replace archived repo in BOM check
Signed-off-by: Tessa Pham <hpham111@bloomberg.net>
2023-11-07 17:41:11 -06:00
Michael Shen
27272c3ed0
scripts: macOS support for scripts/genproto.sh
This commit will allow scripts/genproto.sh to fall back to gsed if
available, which is required for the script to work on macOS.

Signed-off-by: Michael Shen <mishen@umich.edu>
2023-11-07 18:00:36 -05:00
Wei Fu
aea1cd0077 feat: enable unparam lint
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-10-17 21:24:13 +08:00
Wei Fu
81fe610bd5 *: enable markdown_marker checker
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-10-13 14:00:25 +08:00
Benjamin Wang
8e1713dda1
Merge pull request #16714 from fuweid/clean-unconvert_pass
*: cleanup unconvert_pass and remove revive tools
2023-10-09 19:38:19 +01:00
Wei Fu
0ec6181466 scripts: remove unconvert_pass
The `unconvert` has been migrated into golingci-lint.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-10-09 23:27:48 +08:00
Wei Fu
093ec7a11d scripts: remove the markdown_you_pass
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-10-08 15:20:30 +08:00
Wei Fu
5878f346ba *: migrate nakedret into golangci-lint
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-10-07 22:29:43 +08:00
Wei Fu
07effc4d0a *: fix revive linter
Remove old revive_pass in the bash scripts and migirate the revive.toml
into golangci linter_settings.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-24 14:21:11 +08:00
Wei Fu
46df6abb03 *: remove goimports(_fix)_pass and add lint_fix_pass
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-21 21:18:36 +08:00
Wei Fu
502879405f *: lint_pass should use global golangci.yaml
Disable failed linters and enable it by #16610.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-21 16:10:45 +08:00
Wei Fu
2b1e181c8b *: Use golangcilint_pass to run staticcheck and ineffassign
Copy the tools/.golangci.yaml and run the linters for which we have
already fixed. The temp .golangci.yaml will be removed when we fixes all
the linters' issues.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-21 12:34:17 +08:00
Wei Fu
9c3edfa0af *: fix staticcheck lint
Changed TraceKey/StartTimeKey/TokenFieldNameGRPCKey to struct{} to
follow the correct usage of context. Similar patch to #8901.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-21 11:24:26 +08:00