Backport of ba4b2bffeb
Related issue: https://github.com/etcd-io/etcd/issues/18180
As tools/mod is not part of the test library's modules, the check to verify
consistent dependency versions ignored it. Explicitly get the dependencies from
this module when running verify-dep.
Signed-off-by: D Tripp <38776199+thedtripp@users.noreply.github.com>
Include conditional logic to install shellcheck with correct architecture.
This is based on commit 4f23883 and pull request #14872.
Signed-off-by: D Tripp <38776199+thedtripp@users.noreply.github.com>
Makefile's target `verify-dep` current behavior is to use `go list` to
check consistent dependency versions from direct dependencies. Ignoring
indirect dependencies in a multi-module project could lead to version
mismatches. If module A imports module B, module B's dependency will be
an indirect dependency in module A. Which can potentially have a version
mismatch. Therefore, use `go mod edit` with indirect dependencies, too.
So it can work with all dependencies defined in go.mod.
Fix displaying dependencies with mismatches, as the old code was
searching with grep just for the prefix, which would show other
dependencies that shared the same prefix.
Reference:
- https://github.com/etcd-io/etcd/pull/18205
Signed-off-by: Ivan Valdes <ivan@vald.es>
Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
Motivation is as follows:
- etcdctl we only depend on clientv3 APIs, no dependencies of bolt, backend, mvcc, file-layout
- etcdctl can be officially supported across wide range of versions, while etcdutl is pretty specific to file format at particular version.
it's step towards desired modules layout, documented in: https://etcd.io/docs/next/dev-internal/modules/
Thanks to this the logs:
- are automatically printed if the test fails.
- are in pretty consistent format.
- are annotated by 'member' information of the cluster emitting them.
Side changes:
- Set propert default got DefaultWarningApplyDuration (used to be '0')
- Name the members based on their 'place' on the list (as opposed to
'random')
The purpose of this change is to learn more about flake cases like:
https://travis-ci.com/github/etcd-io/etcd/jobs/488324449
```
% (cd tests && 'env' 'go' 'test' '-timeout=30m' '--race=false' '--cpu=2' './integration/...')
stderr: go: downloading github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e
ok go.etcd.io/etcd/tests/v3/integration 197.295s
ok go.etcd.io/etcd/tests/v3/integration/client 0.089s
ok go.etcd.io/etcd/tests/v3/integration/client/examples 0.038s
ok go.etcd.io/etcd/tests/v3/integration/clientv3 70.365s
ok go.etcd.io/etcd/tests/v3/integration/clientv3/concurrency 3.169s
ok go.etcd.io/etcd/tests/v3/integration/clientv3/connectivity 100.535s
ok go.etcd.io/etcd/tests/v3/integration/clientv3/examples 1.341s
ok go.etcd.io/etcd/tests/v3/integration/clientv3/experimental/recipes 3.277s
No output has been received in the last 10m0s,
```
Prior to this PR, the e2e tests where creating dirs like:
```
/tmp/testname1.etcd030299846
/tmp/testname0.etcd039445123
/tmp/testname0.etcd206372065
```
and not cleaning them, that led to disk-space-exceeded flakes.
After the PR, the testing.TB tempdir mechanism is used and the names are
being cleaned and are more miningful:
```
../../bin/etcd --name test-TestCtlV3EndpointHashKV-2 --listen-client-urls http://localhost:20010 --advertise-client-urls http://localhost:20010 --listen-peer-urls https://localhost:20011 --initial-advertise-peer-urls https://localhost:20011 --initial-cluster-token new --data-dir /tmp/TestCtlV3EndpointHashKV429176179/003 --snapshot-count 100000 --experimental-initial-corrupt-check --peer-auto-tls --initial-cluster test-TestCtlV3EndpointHashKV-0=https://localhost:20001,test-TestCtlV3EndpointHashKV-1=https://localhost:20006,test-TestCtlV3EndpointHashKV-2=https://localhost:20011
```
They used to take >10min with coverage, so were causing interrupted
Travis runs.
Know thay fit in 100-150s (together), thanks also to parallel
execution.
Changes:
- signing tags.
- allows to override BRANCH and REPOSITORY using env variables.
Tested by a release in my private fork:
BRANCH="20201126-ptabor-release" REPOSITORY="git@github.com:ptabor/etcd.git" ./scripts/release 3.5.0-alpha.20
- build & test scripts deprecated. Call *.sh variants.
This will avoid delete the symlinks and get rid of
subtle dependency on 'sed --follow-symlinks' on OsX/BSD sed.
- Fix parameters to mktemp
Replace ./scripts/install_tool.sh with `gobin`, such that we have
consistent handling for all tools needed for build and consistent
versioning within ./tools/mod/go.mod.
Side changes:
- Expose /scripts/fix.sh that fixes formatting and bom across modules
- Expose *.sh variants of scripts like build and ./test (first step
towards replacement).
- Make stderr output of commands explicit and make commands use
different color than callouts.
- don't close ready channel until PeerServer is listening.
avoids possible panic in Stop() if PeerServer is nil.
- avoid data race in Run() (err variable was shared between 2 goroutines)
- avoid data race in PeerServer Start/Stop (PeerServer.closeChan)
use the third_party.go project to replace our update script. This
requires moving a few things around and gets rid of a few annoying bugs:
- You can now bump individual packages
- A new src directory isn't created on build
- Less shell scripting!
- Things get built into ./bin/
