Mirroristas/etcd
2
0
Fork 0
mirror of https://github.com/etcd-io/etcd.git synced 2024-09-27 06:25:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
17,025 Commits 15 Branches 491 Tags 148 MiB
Commit Graph

247 Commits

Author SHA1 Message Date
Piotr Tabor
2edb08642c api: Make api/ a module that will contain proto-definitions. 
The module is supposed to contain minimal set of files that establish
public etcd server API. In particular client libraries for etcd built in
different languages might want to depend on this file.
 2020-10-06 11:54:50 +02:00
Piotr Tabor
0e5d81704f .travis.yml, scripts: Fix minor bugs in the test script. 
1. setting environment variable cannot be in quote
2. "--race" testing for unit tests is supposed to be part of linux-amd64-unit-4-cpu-race config.
3. 'run' function in test script should log_error in case of failed
command (wrong operator for ints comparison in bash).
 2020-10-01 14:34:58 +02:00
Piotr Tabor
f1d4593241 ./test: Refactoring of test script for modularization 
This refactoring offers following benefits:

  - Unified way how go test commands are being called (in terms of flags intepretation)
  - Uses standard go mechanisms (like go lists) to find files/packages that are subject for test. The mechanism are module aware.
  - Added instruction how to install tools needed for the tests/checkers.
  - Added colors to the output to make it easier to spot any failure.

Confirmed to work using:
- COVERDIR="./coverage" CPU="4" RACE=false COVER=false PASSES="build build_cov cov" ./test
- CPU="4" RACE=false COVER=false PASSES="e2e functional integration" ./test
- COVERDIR="./coverage" COVER="false" CPU="4" RACE="false" PASSES="fmt build unit build_cov integration e2e integration_e2e grpcproxy cov" ./test
- PASSES=unit PKG=./wal TIMEOUT=1m ./test
- PASSES=integration PKG=./clientv3 TIMEOUT=1m ./test
- PASSES=unit PKG=./wal TESTCASE=TestNew TIMEOUT=1m ./test
- PASSES=unit PKG=./wal TESTCASE="\bTestNew\b" TIMEOUT=1m ./test
- PASSES=integration PKG=./client/integration TESTCASE="\bTestV2NoRetryEOF\b" TIMEOUT=1m ./test
- COVERDIR=coverage PASSES="build_cov cov" ./test
 2020-09-28 11:07:50 +02:00
Piotr Tabor
138a312643 scripts: Removed updatedep.sh. 
As we get rid of ./vendor there is no need to update the dependencies.

If anyone needs up-to-date vendor directory locally, getting it is as
simple as:

```go mod vendor```
 2020-09-16 13:04:58 +02:00
Piotr Tabor
fb2e96e651 vendor: Get rid of ./vendor cleanup 
Updated scripts and documentation to not recommend vendoring.
Implemented best practices for tools installation.

Performed multiple tests to confirm its not breaking any workflows and
has no negative performance impact. Rather see 3x speedup.

1. PASSES="fmt unit integration e2e functional" ./test
2. ./scripts/updatebom.sh
3. ./scripts/updatedep.sh
4. ./scripts/genproto.sh - works - ca be simplified - in follow up PR
5. Installation without explicit GOPATH:

```
% unset GOPATH
% [sudo] rm -rf ~/go
% git clone https://github.com/etcd-io/etcd.git
% time ./build
go: downloading google.golang.org/grpc v1.26.0
go: downloading github.com/jonboulle/clockwork v0.1.0
go: downloading github.com/prometheus/client_golang v1.0.0
go: downloading github.com/soheilhy/cmux v0.1.4
go: downloading github.com/gogo/protobuf v1.2.1
go: downloading sigs.k8s.io/yaml v1.1.0
go: downloading golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2
go: downloading github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903
go: downloading go.etcd.io/bbolt v1.3.5
go: downloading go.uber.org/zap v1.15.0
go: downloading golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc
go: downloading github.com/golang/protobuf v1.3.2
go: downloading github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8
go: downloading github.com/beorn7/perks v1.0.0
go: downloading github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4
go: downloading github.com/coreos/go-systemd/v22 v22.0.0
go: downloading gopkg.in/yaml.v2 v2.2.2
go: downloading github.com/coreos/go-semver v0.2.0
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7
go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: downloading github.com/google/uuid v1.0.0
go: downloading github.com/modern-go/reflect2 v1.0.1
go: downloading github.com/prometheus/common v0.4.1
go: downloading github.com/spf13/cobra v0.0.3
go: downloading github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2
go: downloading github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c
go: downloading github.com/spf13/pflag v1.0.1
go: downloading github.com/json-iterator/go v1.1.7
go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: downloading github.com/google/btree v1.0.0
go: downloading go.uber.org/atomic v1.6.0
go: downloading github.com/prometheus/procfs v0.0.2
go: downloading go.uber.org/multierr v1.5.0
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5
go: downloading github.com/grpc-ecosystem/grpc-gateway v1.9.5
go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4
go: downloading github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: downloading google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading golang.org/x/text v0.3.3
go: downloading github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5
go: downloading github.com/bgentry/speakeasy v0.1.0
go: downloading gopkg.in/cheggaaa/pb.v1 v1.0.25
go: downloading github.com/urfave/cli v1.20.0
go: downloading github.com/mattn/go-runewidth v0.0.2
./build  8.22s user 2.31s system 117% cpu 8.961 total
```

Before:
```
% git clone https://github.com/etcd-io/etcd.git && cd etcd && time ./build
Cloning into 'etcd'...
remote: Enumerating objects: 97872, done.
remote: Total 97872 (delta 0), reused 0 (delta 0), pack-reused 97872
Receiving objects: 100% (97872/97872), 58.97 MiB | 19.85 MiB/s, done.
Resolving deltas: 100% (63091/63091), done.

./build  34.97s user 4.15s system 236% cpu 16.555 total
```

6. Rebuild without changes:

```
% time ./build
./build  1.43s user 0.83s system 168% cpu 1.336 total
```

7. Instantation of vendor directory (assuming ./build loaded them to
$GOPATH/pkg):

```
time go mod vendor
go: downloading github.com/inconshreveable/mousetrap v1.0.0
go: downloading github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa
go: downloading github.com/creack/pty v1.1.11
go: downloading github.com/etcd-io/gofail v0.0.0-20190801230047-ad7f989257ca
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1
go mod vendor  0.51s user 0.44s system 110% cpu 0.861 total
```

8. Fresh instantation of vendor:

```
% rm -rf vendor
% [sudo] rm -rf ~/go

% time go mod vendor
go: downloading github.com/coreos/go-systemd/v22 v22.0.0
go: downloading github.com/spf13/cobra v0.0.3
go: downloading github.com/prometheus/client_golang v1.0.0
go: downloading golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7
go: downloading github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4
go: downloading github.com/gogo/protobuf v1.2.1
go: downloading sigs.k8s.io/yaml v1.1.0
go: downloading google.golang.org/grpc v1.26.0
go: downloading github.com/urfave/cli v1.20.0
go: downloading go.uber.org/zap v1.15.0
go: downloading github.com/spf13/pflag v1.0.1
go: downloading github.com/soheilhy/cmux v0.1.4
go: downloading github.com/json-iterator/go v1.1.7
go: downloading github.com/coreos/go-semver v0.2.0
go: downloading github.com/prometheus/common v0.4.1
go: downloading github.com/prometheus/procfs v0.0.2
go: downloading go.uber.org/atomic v1.6.0
go: downloading github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5
go: downloading github.com/golang/protobuf v1.3.2
go: downloading github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa
go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4
go: downloading github.com/modern-go/reflect2 v1.0.1
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading go.uber.org/multierr v1.5.0
go: downloading github.com/creack/pty v1.1.11
go: downloading github.com/mattn/go-runewidth v0.0.2
go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: downloading golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc
go: downloading golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5
go: downloading github.com/jonboulle/clockwork v0.1.0
go: downloading gopkg.in/yaml.v2 v2.2.2
go: downloading github.com/etcd-io/gofail v0.0.0-20190801230047-ad7f989257ca
go: downloading github.com/grpc-ecosystem/grpc-gateway v1.9.5
go: downloading github.com/google/btree v1.0.0
go: downloading google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
go: downloading github.com/beorn7/perks v1.0.0
go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: downloading github.com/google/uuid v1.0.0
go: downloading golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2
go: downloading github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8
go: downloading go.etcd.io/bbolt v1.3.5
go: downloading golang.org/x/text v0.3.3
go: downloading gopkg.in/cheggaaa/pb.v1 v1.0.25
go: downloading github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2
go: downloading github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: downloading github.com/inconshreveable/mousetrap v1.0.0
go: downloading github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading github.com/bgentry/speakeasy v0.1.0
go: downloading github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1
go mod vendor  3.62s user 1.30s system 127% cpu 3.854 total
```

9. Size of the repository - before: 39M, after: 18M

Before:

```
% time git clone https://github.com/etcd-io/etcd.git
Cloning into 'etcd'...
remote: Enumerating objects: 97872, done.
remote: Total 97872 (delta 0), reused 0 (delta 0), pack-reused 97872
Receiving objects: 100% (97872/97872), 58.97 MiB | 20.53 MiB/s, done.
Resolving deltas: 100% (63091/63091), done.
git clone https://github.com/etcd-io/etcd.git  4.66s user 1.02s system 93% cpu 6.068 total

% du -h --exclude .git -d 1
944K	./clientv3
108K	./etcdmain
5.4M	./Documentation
384K	./security
384K	./mvcc
28K	./.github
8.0K	./version
144K	./contrib
240K	./proxy
2.5M	./etcdserver
112K	./embed
536K	./integration
332K	./tools
116K	./lease
108K	./logos
896K	./tests
960K	./raft
216K	./client
52K	./scripts
100K	./hack
464K	./etcdctl
3.0M	./pkg
620K	./functional
136K	./wal
152K	./auth
21M	./vendor
39M
```

After:
```
% time git clone https://github.com/ptabor/etcd.git -b 20200908-no-vendor
Cloning into 'etcd'...
remote: Enumerating objects: 38, done.
remote: Counting objects: 100% (38/38), done.
remote: Compressing objects: 100% (37/37), done.
remote: Total 98489 (delta 10), reused 8 (delta 1), pack-reused 98451
Receiving objects: 100% (98489/98489), 59.23 MiB | 21.26 MiB/s, done.
Resolving deltas: 100% (63572/63572), done.
git clone https://github.com/ptabor/etcd.git -b 20200908-no-vendor  5.56s user 1.05s system 105% cpu 6.260 total

% du -h --exclude .git -d 1
944K	./clientv3
108K	./etcdmain
5.4M	./Documentation
384K	./security
384K	./mvcc
28K	./.github
8.0K	./version
144K	./contrib
240K	./proxy
2.5M	./etcdserver
112K	./embed
536K	./integration
332K	./tools
116K	./lease
108K	./logos
896K	./tests
960K	./raft
216K	./client
56K	./scripts
100K	./hack
464K	./etcdctl
3.0M	./pkg
620K	./functional
136K	./wal
152K	./auth
19M	.
```
 2020-09-15 14:38:54 +02:00
Gyuho Lee
d93c36bbb9 *: run "gofmt -s" on genproto complete 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2020-05-09 23:14:32 -07:00
Brandon Philips
03248a9045 bill-of-materials: updates for /etcd/v3 packaging 
bill-of-materials:
- update to new package names

test/updatebom.sh scripts:
- Update to the right package names
- Don't add bom tool to go.mod
 2020-04-28 00:57:52 +00:00
Brandon Philips
2ea4d88299 *: use go.etcd.io/etcd/v3 where needed 
Found potential locations via

```
git grep 'go.etcd.io/etcd$'
```
 2020-04-28 00:57:52 +00:00
Gyuho Lee
3ffb3aed3a vendor: update bbolt v1.3.4, zap v1.14.1 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2020-03-19 13:36:43 -07:00
Nirman Narang
17a220eaee scripts: Added s390x support for docker image release 2020-01-20 07:09:59 -08:00
Jingyi Hu
dcd622b2c7 etcdserver: add v3 request type for cluster attr 
Added ClusterVersionSetRequest for setting cluster version via v3 apply.

Added ClusterMemberAttrSetRequest for setting clsuter member attributes
via v3 apply.
 2019-12-05 16:25:13 -08:00
Nirman Narang
cabf5942b2 scripts/build-binary: Added s390x support for binary release 2019-11-13 22:57:23 -08:00
Gyuho Lee
fd5a25fcec scripts/release: list GPG key only when tagging is needed 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-10-23 10:33:39 -07:00
Joe Betz
36696fe2cc
Add version, tag and branch checks to release script 2019-10-11 15:47:15 -07:00
Gyuho Lee
0824b08cab
Merge pull request #11237 from YoyinZyc/prevent-darwin-build 
scripts: avoid release builds on darwin machine.
 2019-10-11 10:39:14 -07:00
yoyinzyc
b9358a0bc8 scripts: avoid release builds on darwin machine. 2019-10-11 09:56:48 -07:00
yoyinzyc
55efdfe364 scripts: fix read failure prompt in release; use https for git clone. 2019-10-10 13:23:36 -07:00
Gyuho Lee
46f3e8349b scripts/build-binary: fix darwin tar commands 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-09-27 23:35:34 -07:00
Gyuho Lee
838315283d scripts/release: fix SHA256SUMS command 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-09-17 14:12:50 -07:00
Gyuho Lee
c327120ba0 scripts/release: fix docker push command 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-09-17 13:52:22 -07:00
Gyuho Lee
1dff148f69 scripts/updatedep: use Go 1.13 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-09-06 11:39:03 -07:00
Manuel Rüger
d981fe6b6b scripts/release: Apply shellcheck findings 
I run https://github.com/koalaman/shellcheck/ over scripts/* and fixed
the findings it returned.

Signed-off-by: Manuel Rüger <manuel@rueg.eu>
 2019-09-02 17:50:04 +02:00
Brandon Philips
f108a32393
scripts/release: rename SHA256SUM to SHA256SUMS 
These files are commonly called SHA256SUMS and with this change rget
works for v3.4.0 as well.
 2019-08-30 13:31:17 -07:00
Gyuho Lee
837fe89d0b scripts/release: fix sha256sum 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-29 09:39:18 -07:00
Sam Batschelet
2386b945c6 scripts/release: add sha256sum summary of release assets 
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
 2019-08-28 15:56:39 +00:00
Gyuho Lee
fd0d43dad9 scripts/release: clean up minor tag docker commands 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-13 22:00:07 -07:00
Gyuho Lee
8d823e70dd
Merge pull request #11018 from gyuho/yq 
scripts/release: update "yq" command
 2019-08-12 11:03:37 -07:00
Gyuho Lee
c72fa51b2a scripts: remove ".aci" commands 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-12 10:20:21 -07:00
Gyuho Lee
af38185f92 scripts/release: fix version check commands 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-12 10:20:13 -07:00
Gyuho Lee
ac613c481f scripts: fix build docker commands, add more logging 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-12 10:20:05 -07:00
Gyuho Lee
7299a6a106 *: remove "acbuild" 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-12 09:31:22 -07:00
Gyuho Lee
1f4f8ae731 scripts/release: update "yq" command 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-12 09:26:04 -07:00
Gyuho Lee
f7992237ce scripts/release: remove acbuild commands 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-04 23:20:41 -07:00
Tobias Schottdorf
14625b847c scripts: have genproto.sh clean up after itself 
We don't want it to leave gopath.proto around for reasons detailed in
the previous commit (messing up vgo).
 2019-07-16 14:01:04 +02:00
Tobias Schottdorf
f63984bb33 scripts: fail explicitly in updatedep.sh when gopath.proto exists 
I had been dealing with these intermittent failures for a while and
finally figured out why. The real solution is making genproto.sh less
ugly but that won't happen for a while.
 2019-07-16 13:54:09 +02:00
Gyuho Lee
b40597ce46 module: require 1.12, remove "v3" import paths 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-06-03 11:15:19 -07:00
Gyuho Lee
c8ffa36d9e scripts/genproto: bump up protoc 3.7.1 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-05-28 15:39:35 -07:00
Davanum Srinivas
86e3481ba2
scripts: Remove generated code and script 
Change-Id: Iac4601443bcad71920fd96b97bfe21c16116577a
 2019-04-23 16:54:44 -04:00
Gyuho Lee
3e0f0ba40e
Merge pull request #10401 from markmc/doc-drop-etcdctl-v3-flag 
Eliminate some ETCDCTL_API=3 usage
 2019-02-01 11:25:55 -08:00
Sam Batschelet
a011b2c4c4 scripts: disable go mod and bump protoc to 3.6.1 
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
 2019-01-22 16:55:01 +00:00
Mark McLoughlin
e53324db3b scripts/release: stop using ETCDCTL_API=3 
Note: v3 has been the default since 25bc65794.
 2019-01-14 14:46:16 +00:00
Gyuho Lee
d5967b40db scripts/updatedep: fix shellcheck errors 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2018-09-21 02:08:25 -07:00
Gyuho Lee
ad2d18aeff scripts/updatedep: use Go 1.11 module for dependency management 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2018-09-21 01:47:24 -07:00
Vimal K
bcde798fdd *: path changes for moving to github/etcd-io/etcd 2018-09-03 21:57:23 +05:30
Gyuho Lee
a8a3efd27a scripts: update import paths in "go.etcd.io/etcd" 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2018-08-28 17:47:55 -07:00
Gyuho Lee
981644af7d
scripts/updatedep: remove "dep prune" command 
"Pruning is now performed automatically by dep ensure."

Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-08-02 08:06:27 -07:00
Gyuho Lee
453db62d84 scripts/genproto: use protoc 3.6.0 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-07-10 10:16:37 -07:00
Joe Betz
a8990e2e18
scripts: Fix release script to always build etcd binary before checking version 2018-06-06 10:48:45 -07:00
Joe Betz
e2821a16ff
scripts/release: Fix docker push for 3.1 releases, remove inaccurate warning at the end of release script 2018-05-31 13:56:07 -07:00
Gyuho Lee
0db7ea7337 scripts/release: disable minor version docker push 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-05-31 12:08:01 -07:00