567 Commits

Author SHA1 Message Date
Piotr Tabor
e6baf6d751 Represent bucket as object instead of []byte name.
Thanks to this change:
  - all the maps bucket -> buffer are indexed by int's instead of
string. No need to do: byte[] -> string -> hash conversion on each
access.
  - buckets are strongly typed in backend/mvcc API.
2021-05-25 09:22:25 +02:00
Gyuho Lee
6bab63e96b tools: rename "master" branch references to "main"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-12 10:39:53 -07:00
Piotr Tabor
a78d072b9a Simplify KVstore dependency on cindex. 2021-05-04 18:21:24 +02:00
Piotr Tabor
ffea1537d4 ClientV3 tests use integration.NewClient that configures proper logger. 2021-04-29 18:18:34 +02:00
Piotr Tabor
aa6597384b etcd-dump-logs: Print full confState as json for debugging purposes. 2021-04-27 17:17:31 +02:00
Piotr Tabor
3423a949c0 Update go for 3.5: 1.15 -> 1.16.(3).
https://github.com/etcd-io/etcd/issues/12732
2021-04-19 16:50:54 +02:00
Piotr Tabor
3bb7acc8cf Migrate dependencies pkg/foo -> client/pkg/foo 2021-04-07 00:38:47 +02:00
Joel Smith
19f7c6ef3e *: Update gogo/protobuf to v1.3.2, rerun ./scripts/genproto.sh
While it appears that etcd is not vulnerable to CVE-2021-3121,
it is a good idea to update to the new generator so that new
vulnerable code isn't generated in any future APIs. Also, this
lays the issue to rest of whether there is any issue with
etcd and CVE-2021-3121.
2021-03-23 11:48:06 -06:00
Piotr Tabor
9312d1b077 clientv3: Bring back ETCD_CLIENT_DEBUG variable interpretation.
env ETCD_CLIENT_DEBUG supports log levels (debug, info, warn, error, dpanic, panic, fatal).
Only when set, overrides application-wide grpc logging settings.
2021-03-18 18:43:07 +01:00
Piotr Tabor
a7f340216d Reformat code according to 'gotip' rules.
In practices adds annotations in the new syntax:
```
+//go:build !linux
 // +build !linux
```

Fixes failing gotip PASSES='fmt' check:
https://travis-ci.com/github/etcd-io/etcd/jobs/486453806
2021-02-26 10:14:46 +01:00
Piotr Tabor
598ca6caab Modernize release script:
- making sure the DRY_RUN mode can finish e2e, so e.g. commits to
local copy of repository are OK in dry-run (while git pushes are NOT).
  - better interaction with ./test_lib.sh script.
  - more consistent logging
  - bringing back s390x architecture that on go 1.14.3 seems to work as
expected.
2021-01-21 17:47:36 +01:00
Piotr Tabor
577c898fee scripts: Integrate ./scripts/release with new code for tagging modules.
Changes:
  - signing tags.
  - allows to override BRANCH and REPOSITORY using env variables.

Tested by a release in my private fork:
  BRANCH="20201126-ptabor-release" REPOSITORY="git@github.com:ptabor/etcd.git" ./scripts/release 3.5.0-alpha.20
2021-01-15 12:31:44 +01:00
Gyuho Lee
3eea37cf31 tools/etcd-dump-metrics: validate exec cmd args
To prevent arbitrary command invocations.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-11-25 10:37:52 -08:00
Piotr Tabor
aaf423e962 server: Update imports.
find -name '*.go' | xargs sed -i --follow-symlinks 's|etcd/v3/|etcd/server/v3/|g'
2020-10-26 13:02:32 +01:00
Piotr Tabor
0ba16d8ee1 *: Convert tabulators to whitespaces in bash scripts.
Execution of `./scripts/fix.sh` that executed:
```
find ./ -name '*.sh' | xargs sed --follow-symlinks -i 's|\t|  |g'
```
2020-10-26 10:59:40 +01:00
Piotr Tabor
e33c6dd9df client/v3: Rename of imports 2020-10-20 10:13:06 +02:00
Piotr Tabor
e62417297d *: Rename of imports of raft (as its now a module)
% find -name '*.go' -o -name '*.md' -o -name '*.sh' | xargs sed -i --follow-symlinks 's|etcd/v3/raft|etcd/raft/v3|g'
2020-10-16 13:58:18 +02:00
Piotr Tabor
b7f0f52a16
*: Refresh of dependencies (#12399)
This PR focuses on dependencies that have new stable versions.
2020-10-15 15:32:00 -07:00
Piotr Tabor
da5ca2c0d5 tools: Update of go.mod for protos (gogo/protobuf 1.3.1, grpc 1.29.1, grpc-gateway 1.14.6)
This brings consistency between proto-generation code and actual versions of libraries being used in runtime:

github.com/gogo/protobuf                        v1.2.1,v1.0.0 -> v1.3.1
github.com/golang/protobuf                             v1.3.2 -> v1.3.5
github.com/grpc-ecosystem/grpc-gateway  v1.9.5,v1.4.1,v1.15.2 -> v1.14.6
google.golang.org/grpc                                v1.26.0 -> v1.29.1

Moved as far as possible, without bumping on grpc 1.30.0 "naming" decomissioning.
Please also notice that gogo/protobuf is likely to reach EOL: https://github.com/gogo/protobuf/issues/691
2020-10-14 18:46:38 +02:00
Piotr Tabor
bc9e433ca2 tools: Migrate remaining tools to gobin
Replace ./scripts/install_tool.sh with `gobin`, such that we have
consistent handling for all tools needed for build and consistent
versioning within ./tools/mod/go.mod.

Side changes:
  - Expose /scripts/fix.sh that fixes formatting and bom across modules
  - Expose *.sh variants of scripts like build and ./test (first step
towards replacement).
  - Make stderr output of commands explicit and make commands use
different color than callouts.
2020-10-13 19:33:01 +02:00
Piotr Tabor
de55bb6331 pkg: Rename imports after making 'pkg' a module
find -name '*.go' | xargs sed --follow-symlinks -i 's|go.etcd.io/etcd/v3/pkg/|go.etcd.io/etcd/pkg/v3/|g'
go fmt ./...
2020-10-13 00:09:27 +02:00
Piotr Tabor
dfdda47bd8 script/genproto.sh: Refactor to be explicit about versions.
Refactoring script/genproto.sh around state-of-the-art techniques of
managing tooling in go:
  - https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module
  - uses https://github.com/myitcv/gobin instead of customly created gopath.proto dir
  - caches tools between executions
  - guaratees hermetics runs (it was not guaranteed for protoc_grpc_gateway that used latest)

The change is no-op for the generated code.

The commit reveals a few 'worring things':
  1  We depend on : github.com/grpc-ecosystem/grpc-gateway/@v/v1.4.1/protoc-gen-grpc-gateway
  2. And also     : github.com/grpc-ecosystem/grpc-gateway/@v/v1.15.0/protoc-gen-swagger/protoc-gen-swagger
  3. And on extremely old: github.com/gogo/protobuf@v1.0.0 protoc-gen-gofast that is out of sync with the library linked to binaries: github.com/gogo/protobuf@v1.2.1
2020-10-08 19:52:27 +02:00
Piotr Tabor
28f2b07623 *: Update references to code moved to the api/ dir.
Follow up to file-moves done in the previous commit.

The commit contains purely mechanical consequences of execution (apart
of scripts/genproto.sh):

  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/etcdserver/api/v3rpc/rpctypes|v3/api/v3rpc/rpctypes|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/version|v3/api/version|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/mvcc/mvccpb|v3/api/mvccpb|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/etcdserver/etcdserverpb|v3/api/etcdserverpb|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/etcdserver/api/membership/membershippb|v3/api/membershippb|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/auth/authpb|v3/api/authpb|g'

  % find ./ -name '*.proto' -o -name '*.md'  | xargs -L 1 sed --follow-symlinks -i 's|/mvcc/mvccpb/kv.proto|/api/mvccpb/kv.proto|g'
  % find ./ -name '*.proto' -o -name '*.md'  | xargs -L 1 sed --follow-symlinks -i 's|/auth/authpb/auth.proto|/api/authpb/auth.proto|g'
  % find ./ -name '*.proto' -o -name '*.md'  | xargs -L 1 sed --follow-symlinks -i 's|/etcdserver/api/membership/membershippb/membership.proto|/api/membershippb/membership.proto|g'

  I also modified manually paths in scripts/genproto.sh.

  % go fmt ./...
2020-10-06 11:56:16 +02:00
Jingyi Hu
205a656cc5
Merge pull request #11853 from viviyww/dev1
tools: fix test case errors in etcd-dump-logs
2020-09-21 09:13:31 -07:00
yangweiwei
c88d1497ee tools: fix test case errors in etcd-dump-logs
Fix test case errors in etcd-dump-logs and the error is the time zone.
When the GOST time zone is CST or PST and the test case will be failed.
So we should set UTC as the standard time zone.
2020-09-21 15:54:47 +08:00
Brandon Philips
96cce208c2 go.mod: use go.etcd.io/etcd/v3 versioning
This change makes the etcd package compatible with the existing Go
ecosystem for module versioning.

Used this tool to update package imports:
  https://github.com/KSubedi/gomove
2020-04-28 00:57:35 +00:00
shawwang
08a8b80e32 tools/etcd-dump-db: add auth decoder, optimize print format 2020-02-29 13:31:44 +08:00
jingyih
725e09023a *: set zap as default logger, remove capnslog
Set zap as default logger. Remove capnslog and deprecated logging
flags.
2020-02-04 04:57:49 -08:00
Yuchen Zhou
6ffd9f659e tools/benchmark: exit when txn-ops is larger than key-space-size to avoid key duplicated error. (#11499) 2020-01-06 16:36:51 -08:00
yoyinzyc
57aa68af5a etcdserver: trace compaction request; add return parameter 'trace' to applierV3.Compaction()
mvcc: trace compaction request; add input parameter 'trace' to KV.Compact()
2019-10-07 09:55:27 -07:00
Vimal K
e98479ddfe tools/etcd-dump-logs: Fixed test cases
Fixed failing test cases due to changes in previous commit
2019-09-06 20:11:41 +01:00
Vimal K
4e551d06ed tools/etcd-dump-logs: Fixed default values for -entry-type flag
The tool takes default values but it was not visible which default
values were taken. Added default values in proper place, and added a
newline at the end of output.
2019-09-06 20:11:41 +01:00
Joe Betz
9b51febaf5 *: Add experimental-compaction-batch-limit flag 2019-08-15 11:47:23 -07:00
ethan
b5ee1de076
cleanup: correct summary message in put.go 2019-08-12 21:13:58 +08:00
ethan
867b31e01a
etcd-dump-logs: correct logging message word
Signed-off-by: Guangming Wang <guangming.wang@daocloud.io>
2019-08-10 17:01:57 +08:00
Gyuho Lee
3fc62ca586 tools,Documentation: move "etcd-dump-metrics" output to "Documentation"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-28 21:53:50 -07:00
Tobias Schottdorf
b9c051e7a7 raftpb: clean up naming in ConfChange 2019-07-23 10:40:03 +02:00
Hitoshi Mitake
5a67dd788d *: support creating a user without password
This commit adds a feature for creating a user without password. The
purpose of the feature is reducing attack surface by configuring bad
passwords (CN based auth will be allowed for the user).

The feature can be used with `--no-password` of `etcdctl user add`
command.

Fix https://github.com/coreos/etcd/issues/9590
2019-05-30 21:59:30 +09:00
Gyuho Lee
34bd797e67 *: revert module import paths
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-05-28 15:39:35 -07:00
shivaramr
9150bf52d6 go modules: Fix module path version to include version number 2019-04-26 15:29:50 -07:00
haroldHT
a1fb18a9fc benchmark: fix install docs 2019-02-18 18:15:40 +08:00
Gyuho Lee
f8a513ce65 tools/etcd-dump-db: add "--timeout" flag
By default, wait up to 10 seconds to obtain flocks.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2018-11-15 22:50:32 -08:00
Shin'ya Ueoka
aa4313a55a *: fix github links 2018-11-10 11:14:18 +09:00
Gyuho Lee
83304cfc80 tools/etcd-dump-metrics: add missing godoc
https://godoc.org/github.com/etcd-io/etcd/tools/etcd-dump-metrics

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2018-10-31 16:12:32 -07:00
Gyuho Lee
1f5aea320a
Merge pull request #10139 from DennisMao/patch-1
tools: fix building failures on Win
2018-10-02 13:30:49 -07:00
Mark Petrovic
95a282efb5 benchmark: util.go
allow client to setup TLS with cluster members, without the client having to offer TLS authentication itself

fixes #10142
2018-10-02 08:45:24 -07:00
Dennis
32a3a73755
tools: fix building failures on Windows
When building tools on Win,it shows `.\main.go:68:12: assignment mismatch: 2 variables but 1 values`.The reason is the return variables not match the calling from `main.go` and i try to fix it.
2018-09-29 11:17:49 +08:00
Sam Batschelet
d43f223e36 tools/etcd-dump-db: fix iterate-bucket example. 2018-09-05 20:41:42 -04:00
Gyuho Lee
8db439d693 *: use "go.etcd.io/bbolt"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2018-08-29 12:31:28 -07:00
Gyuho Lee
d37f1521b7 *: update import paths to "go.etcd.io/etcd"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2018-08-28 17:47:55 -07:00