18971 Commits

Author SHA1 Message Date
Chao Chen
008df94901 tests/common: migrate auth tests #1
Signed-off-by: Chao Chen <chaochn@amazon.com>
2022-12-15 10:49:20 -08:00
Benjamin Wang
ee9db729da
Merge pull request #14860 from ahrtr/fix_release_20221126
Trigger release in current branch for github workflow case
2022-12-12 17:46:19 +08:00
Benjamin Wang
638d0a016a
Merge pull request #14930 from ahrtr/secure_workflow_20221212
Secure the github workflows
2022-12-12 16:59:57 +08:00
Benjamin Wang
7aaef7616e
Merge pull request #14927 from ahrtr/fix_testMutexLock_20221212
test: fix nil pointer panic in testMutexLock
2022-12-12 16:50:59 +08:00
Benjamin Wang
bf5c094f3c secure the github workflow
https://app.stepsecurity.io/secureworkflow/etcd-io/etcd/tests.yaml/main?enable=pin
1. Copy the existing yaml file and paste into the textbox,
2. Click "SECURE WORKFLOW"
3. Copy the manifest from the textbox and paste into etcd repo.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-12 16:23:13 +08:00
Benjamin Wang
3c51c42417 test: fix nil pointer panic in testMutexLock
Refer to: https://github.com/etcd-io/etcd/actions/runs/3671847902/jobs/6207463700

```
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xedc388]

goroutine 5253 [running]:
go.etcd.io/etcd/client/v3/concurrency.(*Session).Client(...)
	/home/runner/work/etcd/etcd/client/v3/concurrency/session.go:76
go.etcd.io/etcd/client/v3/concurrency.(*Mutex).tryAcquire(0xc000133140, {0x18a8668, 0xc000050158})
	/home/runner/work/etcd/etcd/client/v3/concurrency/mutex.go:111 +0x88
go.etcd.io/etcd/client/v3/concurrency.(*Mutex).Lock(0xc000133140, {0x18a8668, 0xc000050158})
	/home/runner/work/etcd/etcd/client/v3/concurrency/mutex.go:74 +0x68
go.etcd.io/etcd/tests/v3/integration/clientv3/experimental/recipes_test.testMutexLock.func1()
	/home/runner/work/etcd/etcd/tests/integration/clientv3/experimental/recipes/v3_lock_test.go:65 +0x285
created by go.etcd.io/etcd/tests/v3/integration/clientv3/experimental/recipes_test.testMutexLock
	/home/runner/work/etcd/etcd/tests/integration/clientv3/experimental/recipes/v3_lock_test.go:59 +0xda
FAIL	go.etcd.io/etcd/tests/v3/integration/clientv3/experimental/recipes	7.070s
FAIL
```

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-12 10:18:45 +08:00
Benjamin Wang
5d78d6d4b1 release: support kick off release in current branch
Currently when triggering release, it always pull remote repo and
checkout main branch. Any changes which are merged into the target
release branch (e.g. release-3.5) will be ignored. It isn't
convenient for test, including in github workflow and local environment.
So we need to support triggering release in current branch.

Note: --current-branch should only be called with DRY_RUN=true

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-12 09:35:03 +08:00
Benjamin Wang
64599b4072
Merge pull request #14925 from serathius/gofail-v0.1.0
tests: Update gofail to v0.1.0
2022-12-11 05:31:34 +08:00
Marek Siarkowicz
1bb4c9558d tests: Update gofail to v0.1.0
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-12-10 14:49:20 +01:00
Benjamin Wang
142fa76a11
Merge pull request #14919 from ahrtr/govuln_version_20221209
Pin govulncheck to v0.0.0-20221208180742-f2dca5ff4cc3
2022-12-09 20:48:51 +08:00
Benjamin Wang
808099dc24 Pin govulncheck to v0.0.0-20221208180742-f2dca5ff4cc3
go install golang.org/x/vuln/cmd/govulncheck@latest && govulncheck ./...
  shell: /usr/bin/bash -e {0}
  env:
    GOROOT: /opt/hostedtoolcache/go/1.19.4/x64
go: golang.org/x/vuln/cmd/govulncheck@latest: no matching versions for query "latest"

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-09 18:23:53 +08:00
Marek Siarkowicz
dafc1ade48
Merge pull request #14917 from serathius/linearizability-nightly-timeout
tests: Increase test timeout for nighly runs to match job timeout min…
2022-12-09 10:10:11 +01:00
Marek Siarkowicz
a8bc8ba28b tests: Increase test timeout for nighly runs to match job timeout minus ten minutes
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-12-09 09:47:17 +01:00
Benjamin Wang
893a3e7356
Merge pull request #14916 from ahrtr/bump_go_1.19.4
Bump go to 1.19.4 and golang.org/x/net to v0.4.0 to address CVEs
2022-12-09 16:38:39 +08:00
Marek Siarkowicz
ebca62cc81
Merge pull request #14903 from serathius/linearizability-txn
tests: Add Txn operation to linearizability tests
2022-12-09 09:38:08 +01:00
Benjamin Wang
1ba246e1d8 bump golang.org/x/ to v0.4.0
Found 1 known vulnerability.

Vulnerability #1: GO-2022-1144
  An attacker can cause excessive memory growth in a Go server
  accepting HTTP/2 requests. HTTP/2 server connections contain a
  cache of HTTP header keys sent by the client. While the total
  number of entries in this cache is capped, an attacker sending
  very large keys can cause the server to allocate approximately
  64 MiB per open connection.

  Call stacks in your code:
Error:       tools/etcd-dump-metrics/main.go:158:5: go.etcd.io/etcd/v3/tools/etcd-dump-metrics.main calls go.etcd.io/etcd/server/v3/embed.StartEtcd, which eventually calls golang.org/x/net/http2.Server.ServeConn

  Found in: golang.org/x/net/http2@v0.2.0
  Fixed in: golang.org/x/net/http2@v0.4.0
  More info: https://pkg.go.dev/vuln/GO-2022-1144
Error: Process completed with exit code 3.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-09 08:54:48 +08:00
Benjamin Wang
dccc21bb69 bump go 1.19.4
$ govulncheck ./...
govulncheck is an experimental tool. Share feedback at https://go.dev/s/govulncheck-feedback.

Scanning for dependencies with known vulnerabilities...
Found 1 known vulnerability.

Vulnerability #1: GO-2022-1144
  An attacker can cause excessive memory growth in a Go server
  accepting HTTP/2 requests. HTTP/2 server connections contain a
  cache of HTTP header keys sent by the client. While the total
  number of entries in this cache is capped, an attacker sending
  very large keys can cause the server to allocate approximately
  64 MiB per open connection.

  Call stacks in your code:
      tools/etcd-dump-metrics/main.go:159:31: go.etcd.io/etcd/v3/tools/etcd-dump-metrics.main$4 calls go.etcd.io/etcd/server/v3/embed.StartEtcd, which eventually calls golang.org/x/net/http2.ConfigureServer$1

  Found in: golang.org/x/net/http2@v0.2.0
  Fixed in: golang.org/x/net/http2@v1.19.4
  More info: https://pkg.go.dev/vuln/GO-2022-1144

Vulnerability #2: GO-2022-1144
  An attacker can cause excessive memory growth in a Go server
  accepting HTTP/2 requests. HTTP/2 server connections contain a
  cache of HTTP header keys sent by the client. While the total
  number of entries in this cache is capped, an attacker sending
  very large keys can cause the server to allocate approximately
  64 MiB per open connection.

  Call stacks in your code:
      contrib/lock/storage/storage.go:106:28: go.etcd.io/etcd/v3/contrib/lock/storage.main calls net/http.ListenAndServe
      contrib/raftexample/httpapi.go:113:31: go.etcd.io/etcd/v3/contrib/raftexample.serveHTTPKVAPI$1 calls net/http.Server.ListenAndServe
      tools/etcd-dump-metrics/main.go:159:31: go.etcd.io/etcd/v3/tools/etcd-dump-metrics.main$4 calls go.etcd.io/etcd/server/v3/embed.StartEtcd, which eventually calls net/http.Serve
      tools/etcd-dump-metrics/main.go:159:31: go.etcd.io/etcd/v3/tools/etcd-dump-metrics.main$4 calls go.etcd.io/etcd/server/v3/embed.StartEtcd, which eventually calls net/http.Server.Serve

  Found in: net/http@go1.19.3
  Fixed in: net/http@go1.19.4
  More info: https://pkg.go.dev/vuln/GO-2022-1144

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-09 07:39:57 +08:00
Marek Siarkowicz
26bf2f81f5 tests: Add Txn operation to linearizability tests
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-12-08 23:57:42 +01:00
Marek Siarkowicz
dc9e422e28
Merge pull request #14902 from serathius/linearizability-issue14685
Refactor etcd model
2022-12-08 23:32:18 +01:00
Marek Siarkowicz
60200b37d3 tests: Refactor etcd model
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-12-08 10:01:43 +01:00
Sahdev Zala
bb528a6bf4
Merge pull request #14913 from spzala/updatepsc
Update PSC members
2022-12-07 18:46:01 -05:00
Benjamin Wang
4ba806a103
Merge pull request #14911 from tjungblu/linnight
Create a nightly job for linearizability tests
2022-12-08 07:19:00 +08:00
Sahdev Zala
e99e37e159 Update PSC members
Many of the existing PSC members are now emeritus maintainers.
Considering the current state of the project, we may want to keep
PSC with the current maintainers. We will list volunteers as we
add them.

Signed-off-by: Sahdev Zala <spzala@us.ibm.com>
2022-12-07 17:59:19 -05:00
Thomas Jungblut
7c9c1b6c1c Create a nightly job for linearizability tests
Start with a simple job against main that runs for 3h by repeating it
an order of magnitude more often than the PR job.

Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
2022-12-07 17:41:49 +01:00
Marek Siarkowicz
3e3be35f20
Merge pull request #14905 from joycebrum/main
Set Github workflows default permissions to read-only
2022-12-07 16:39:14 +01:00
Joyce Brum
a92e06f132
fix: revoke write permissions from all workflows
Signed-off-by: Joyce Brum <joycebrum@google.com>
2022-12-07 13:51:22 +00:00
Benjamin Wang
6d0bf24e54
Merge pull request #14907 from fuweid/chore-reduce-ifelse
chore: use Getter in WarnOfExpensiveReadOnlyTxnRequest
2022-12-07 19:57:39 +08:00
Wei Fu
f59896c735 chore: use Getter in WarnOfExpensiveReadOnlyTxnRequest
The pb provides an accessor method to get field and it will not panic if
the owner is nil. And add non-empty RangeRespone into the test case.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2022-12-07 17:54:52 +08:00
Benjamin Wang
d59c9b82d0
Merge pull request #14901 from ahrtr/changelog_readonly_txn_panic_20221206
changelog: add items for the nil pointer panic of readonly txn
2022-12-07 04:27:07 +08:00
Joyce Brum
6adc81c664
fix: remove workflow dispatch used for testing
Signed-off-by: Joyce Brum <joycebrum@google.com>
2022-12-06 18:10:50 +00:00
Joyce Brum
4bcf401b7f
Squashed commit of the following:
commit 9a3bf2c0ed6e63c718789679745fdaa24a2c2ba9
Author: Joyce Brum <joycebrum@google.com>
Date:   Tue Dec 6 17:59:42 2022 +0000

    fix: write permissions

    Signed-off-by: Joyce Brum <joycebrum@google.com>

commit 7716f3c00cd7cfe4debbbf97662b1cee7277ba00
Author: Joyce Brum <joycebrum@google.com>
Date:   Tue Dec 6 17:04:19 2022 +0000

    fix: typo on coverage workflow

    Signed-off-by: Joyce Brum <joycebrum@google.com>

commit cb5165401392f1a2de3683ec33ffe97dc0f1fe9f
Author: Joyce Brum <joycebrum@google.com>
Date:   Tue Dec 6 16:57:50 2022 +0000

    feat: test coverage workflow with write permissions

    Signed-off-by: Joyce Brum <joycebrum@google.com>

commit 235627f257d52139c9c73c2ca15c9ef7250cea2f
Author: Joyce Brum <joycebrum@google.com>
Date:   Tue Dec 6 16:44:21 2022 +0000

    fix: measure test read all and workflow dispatch

    Signed-off-by: Joyce Brum <joycebrum@google.com>

commit 81b1581f19945ba5ddd7fa74661910a457af7515
Author: Joyce Brum <joycebrum@google.com>
Date:   Tue Dec 6 14:50:12 2022 +0000

    feat: change from content read to read all

    Signed-off-by: Joyce Brum <joycebrum@google.com>

commit 95bd39f615924a9c0186e6d3e1ad6c205c7db428
Author: Joyce Brum <joycebrum@google.com>
Date:   Tue Dec 6 14:45:45 2022 +0000

    fix: add permission to write on e2e.yaml

    Signed-off-by: Joyce Brum <joycebrum@google.com>

commit f86661da253af3908cde9f5f71311fbca6b26c81
Author: Joyce Brum <joycebrum@google.com>
Date:   Mon Dec 5 17:04:44 2022 +0000

    feat: use read-only by default

    Signed-off-by: Joyce Brum <joycebrum@google.com>

Signed-off-by: Joyce Brum <joycebrum@google.com>
2022-12-06 18:03:50 +00:00
Benjamin Wang
b44c2d9afc changelog: add items for the nil pointer panic of readonly txn
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-06 19:26:56 +08:00
Marek Siarkowicz
a4c6d1bbce
Merge pull request #14880 from serathius/linearizability-failed
Improve support for failed requests in linearizability tests
2022-12-06 11:11:47 +01:00
Benjamin Wang
e6ef3c0bc5
Merge pull request #14895 from ahrtr/fix_readyonly_txn_panic_20221205
etcdserver: fix nil pointer panic for readonly txn
2022-12-06 17:29:39 +08:00
Marek Siarkowicz
5ff9202027 tests: Refactor append failed requests
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-12-06 10:21:07 +01:00
Marek Siarkowicz
619ca4f5cf tests: Add comments explaning handling failed requests
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-12-06 10:11:22 +01:00
Marek Siarkowicz
d8da98c49f tests: Fix return time for failed requests globally
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-12-06 10:11:22 +01:00
Marek Siarkowicz
45fdc2bbac tests: Extract history from client
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-12-06 10:11:22 +01:00
Marek Siarkowicz
c5c2ea6aad tests: Enforce that revision doesn't skips without error
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-12-06 10:11:22 +01:00
Marek Siarkowicz
fe6e42d411 tests: Improve tests by expending on scenarios
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-12-06 10:11:22 +01:00
Marek Siarkowicz
1d31e5e958 tests: Consider only latest failed request
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-12-06 10:11:22 +01:00
Marek Siarkowicz
b922afc0a3 tests: Make etcdRequest and etcdResponse public to make it serializable
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-12-06 10:11:22 +01:00
Marek Siarkowicz
68fd863c87 tests: Mark failed requests as timed out
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-12-06 10:11:22 +01:00
Marek Siarkowicz
6a68141db5 tests: Allow dynamic number of clients
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-12-06 10:11:22 +01:00
Marek Siarkowicz
42bb543315
Merge pull request #14897 from serathius/linearizability-trigger-exit
Allow failpoint requests to fail assuming that process exists within 1 second
2022-12-06 09:31:21 +01:00
Marek Siarkowicz
2263315871
Merge pull request #14802 from geetasg/delete_api
linearizability tests - Add support for delete api
2022-12-06 09:30:59 +01:00
Marek Siarkowicz
f2bc0823f6 tests: Make it explicit that we only suppurt panic failpoints
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-12-06 09:08:36 +01:00
Marek Siarkowicz
738ee3687a tests: Allow failpoint requests to fail assuming that process exists within 1 second
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-12-06 09:08:36 +01:00
Marek Siarkowicz
a5cfc089fa tests: Refactor picking a member function
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-12-06 09:08:36 +01:00
Benjamin Wang
daad3a2154 etcdserver: fix nil pointer panic for readonly txn
FYI. https://github.com/etcd-io/etcd/issues/14891#issuecomment-1337191993

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-06 14:53:47 +08:00