552 Commits

Author SHA1 Message Date
Piotr Tabor
0880605772
Merge pull request #12551 from kolyshkin/3.4-fix-lock
[3.4 backport] pkg/fileutil: fix F_OFD_ constants
2021-01-15 23:16:49 +01:00
Kir Kolyshkin
bea35fd2c6 pkg/fileutil: fix F_OFD_ constants
Use golang.org/x/sys/unix for F_OFD_* constants.

This fixes the issue that F_OFD_GETLK was defined incorrectly,
resulting in bugs such as https://github.com/moby/moby/issues/31182

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-12-14 10:42:13 -08:00
Gyuho Lee
a4b43b388d pkg/netutil: remove unused "iptables" wrapper
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-11-25 11:31:17 -08:00
Sahdev P. Zala
d5ebbbceb8 pkg: file stat warning
Provide warning and doc instead of enforcing file permission.
2020-08-24 11:21:29 -04:00
Gyuho Lee
cf558ee8b7 pkg/runtime: optimize FDUsage by removing sort
No need sort when we just want the counts.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-08-12 18:38:17 -07:00
Sahdev P. Zala
319331192e pkg: consider umask when use MkdirAll
os.MkdirAll creates directory before umask so make sure that a desired
permission is set after creating a directory with MkdirAll. Use the
existing TouchDirAll function which checks for permission if dir is already
exist and when create a new dir.
2020-07-07 11:46:31 -04:00
tangcong
0207d1df66 pkg/fileutil: print desired file permission in error log 2020-06-29 09:59:19 +08:00
Changxin Miao
e5424fc474 pkg: Fix dir permission check on Windows 2020-06-25 20:20:55 -04:00
Sahdev P. Zala
434f7e83f0 pkg: check file stats
modify file util.
2020-06-20 16:29:47 -04:00
Viacheslav Biriukov
87fc3c9e57 etcdserver,wal: fix inconsistencies in WAL and snapshot
etcdserver/*, wal/*: changes to snapshots and wal logic
etcdserver/*: changes to snapshots and wal logic to fix #10219
etcdserver/*, wal/*: add Sync method
etcdserver/*, wal/*: find valid snapshots by cross checking snap files and wal snap entries
etcdserver/*, wal/*:Add comments, clean up error messages and tests
etcdserver/*, wal/*: Remove orphaned .snap.db files during Release

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-15 08:40:09 -07:00
Gyuho Lee
b68f8ff31d pkg/ioutil: add "FlushN"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-04-01 09:29:59 -07:00
Jingyi Hu
4f3c81d81d etcdserver: wait purge file loop during shutdown
To prevent the purge file loop from accidentally acquiring the file lock
and remove the files during server shutdowm.
2019-10-30 16:04:41 -07:00
yoyinzyc
480d5510f9 etcdserver: trace compaction request; add return parameter 'trace' to applierV3.Compaction() mvcc: trace compaction request; add input parameter 'trace' to KV.Compact() 2019-10-09 12:40:12 -07:00
yoyinzyc
9245518363 etcdserver: trace raft requests. 2019-10-09 12:40:12 -07:00
yoyinzyc
daa432cfa7 etcdserver: add put request steps. mvcc: add put request steps; add trace to KV.Write() as input parameter. 2019-10-09 12:40:12 -07:00
yoyinzyc
8717327697 pkg: use zap logger to format the structure log output. 2019-10-09 12:40:12 -07:00
yoyinzyc
4f1bbff888 pkg: add field to record additional detail of trace; add stepThreshold to reduce log volume. 2019-10-09 12:40:12 -07:00
yoyinzyc
28bb8037d9 pkg: create package traceutil for tracing. mvcc: add tracing steps:range from the in-memory index tree; range from boltdb. etcdserver: add tracing steps: agreement among raft nodes before linerized reading; authentication; filter and sort kv pairs; assemble the response. 2019-10-09 12:40:12 -07:00
Gyuho Lee
41a2cfa122 pkg/logutil: change to "MergeOutputPaths"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-27 09:50:26 -07:00
lzhfromustc
524278c187 pkg/types: Avoid potential double lock of tsafeSet.
(tsafeSet).Sub and (tsafeSet).Equals can cause double lock bug if ts and other is pointing the same variable

gofmt the code and add some comments
2019-08-07 16:02:24 -07:00
Gyuho Lee
b4fcaad87d pkg/adt: remove TODO
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-05 00:25:02 -07:00
xkey
bb7df24af4 pkg/adt: fix interval tree black-height property based on rbtree
Author: xkey <xk33430@ly.com>
ref. https://github.com/etcd-io/etcd/pull/10978

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-04 23:15:09 -07:00
Gyuho Lee
3b631e1bb6 pkg/adt: document textbook implementation with pseudo-code
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-01 12:58:47 -07:00
Gyuho Lee
3cc3affedd pkg/adt: mask test failure, add TODO
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-31 22:20:59 -07:00
Gyuho Lee
46f04b3c15 pkg/adt: add "IntervalTree.Delete" failure case
Described in https://github.com/etcd-io/etcd/issues/10877.

"black-height" property: Every path from a node to any descendant leaf node must have the same number of black nodes.

Expected

    After deleting 11 (requires rebalancing):
                            [510,511]
                             /      \
                   ----------        --------------------------
                  /                                            \
              [383,384]                                       [830,831]
              /       \                                      /          \
             /         \                                    /            \
      [261,262](red)  [410,411]                     [647,648]           [899,900](red)
          /               \                              \                      /    \
         /                 \                              \                    /      \
      [82,83]           [292,293]                      [815,816](red)   [888,889]    [972,973]
            \                                                           /
             \                                                         /
          [238,239](red)                                       [953,954](red)

Got

    After deleting 11 (requires rebalancing):
                            [510,511]
                             /      \
                   ----------        --------------------------
                  /                                            \
              [82,83]                                       [830,831]
                    \                                      /          \
                     \                                    /            \
                  [383,384]                        [647,648]            [899,900]
                  /       \                              \                  /    \
                 /         \                              \                /      \
           [261,262]      [410,411]                      [815,816]   [888,889]    [972,973]
             /   \                                                                  /
            /     \                                                                /
     [238,239]   [292,293]                                                  [953,954]

This violates "black-height" property.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-31 10:05:32 -07:00
Gyuho Lee
f2742d6cd4 pkg/adt: test node "11" deletion
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-31 10:05:32 -07:00
Gyuho Lee
1d638bad72 pkg/adt: README "IntervalTree.Delete" test case images
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-31 10:05:32 -07:00
Gyuho Lee
19d69d2563 pkg/adt: README initial commit
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-31 10:05:27 -07:00
Gyuho Lee
6917c495e8 pkg/adt: add "visitLevel", make "IntervalTree" interface, more tests
Make "IntervalTree" an interface to abstract range tree interface

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-30 15:59:59 -07:00
Martin Weindel
03fd396610 pkg/transport: Improved description of flag peer-skip-client-san-verification 2019-07-30 16:57:57 -04:00
Martin Weindel
2f476f2b5a pkg/transport: Added test for SkipClientVerify flag. 2019-07-30 16:57:57 -04:00
Martin Weindel
1b048c91ec etcdserver: Added configuration flag --peer-skip-client-verify=true 2019-07-30 16:57:57 -04:00
Gyuho Lee
c6e3401255 etcdserver: make raft log configured by top level logger
To make it consistent

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-29 15:43:19 -07:00
Gyuho Lee
e911f901a6 pkg/logutil: add log level utilities
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-29 15:43:19 -07:00
Gyuho Lee
46166ad733 vendor: update
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-24 14:09:50 -07:00
John Millikin
91472797ff
pkg: Remove stray printfs 2019-07-10 13:33:14 +09:00
John Millikin
5824421f8b
etcdman, pkg: Rename new flags to 'hostname' 2019-07-10 09:30:02 +09:00
John Millikin
9a53601a18
etcdmain, pkg: Support peer and client TLS auth based on SAN fields.
Etcd currently supports validating peers based on their TLS certificate's
CN field. The current best practice for creation and validation of TLS
certs is to use the Subject Alternative Name (SAN) fields instead, so that
a certificate might be issued with a unique CN and its logical
identities in the SANs.

This commit extends the peer validation logic to use Go's
`(*"crypto/x509".Certificate).ValidateHostname` function for name
validation, which allows SANs to be used for peer access control.

In addition, it allows name validation to be enabled on clients as well.
This is used when running Etcd behind an authenticating proxy, or as
an internal component in a larger system (like a Kubernetes master).
2019-07-10 09:30:02 +09:00
Gyuho Lee
1caaa9ed4a test: test update for Go 1.12.5 and related changes
Update to Go 1.12.5 testing. Remove deprecated unused and gosimple
pacakges, and mask staticcheck 1006. Also, fix unconvert errors related
to unnecessary type conversions and following staticcheck errors:
- remove redundant return statements
- use for range instead of for select
- use time.Since instead of time.Now().Sub
- omit comparison to bool constant
- replace T.Fatal and T.Fatalf in tests with T.Error and T.Fatalf respectively because the goroutine calls T.Fatal must be called in the same goroutine as the test
- fix error strings that should not be capitalized
- use sort.Strings(...) instead of sort.Sort(sort.StringSlice(...))
- use he status code of Canceled instead of grpc.ErrClientConnClosing which is deprecated
- use use status.Errorf instead of grpc.Errorf which is deprecated

Related #10528 #10438
2019-06-05 17:02:05 -04:00
Gyuho Lee
34bd797e67 *: revert module import paths
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-05-28 15:39:35 -07:00
shivaramr
9150bf52d6 go modules: Fix module path version to include version number 2019-04-26 15:29:50 -07:00
Gyuho Lee
52391e3be7 pkg/logutil: define default zap.Config
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-02-21 10:56:53 -08:00
Manjunath A Kumatagi
45d09f0508 pkg/transport: Limit InvalidCipherSuites to TLS12 2019-02-07 08:18:00 -06:00
Hitoshi Mitake
65887ae1b4 pkg, clientv3, etcdmain: let grpcproxy rise an error when its cert has non empty CN
Fix https://github.com/etcd-io/etcd/issues/9521
2019-01-25 00:43:57 +09:00
caoming
8f383852e2 pkg/testutil: add blankline between two functions 2019-01-04 09:44:57 +08:00
Shin'ya Ueoka
aa4313a55a *: fix github links 2018-11-10 11:14:18 +09:00
Sam Batschelet
fa35126ef8
*: add client support for discovery-srv-name
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2018-11-09 10:13:04 -05:00
Gyuho Lee
7b32c07899 pkg/flags: fix "TestSetFlagsFromEnvParsingError"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2018-10-31 10:44:38 -07:00
ping40
5a94f97a4f pkg/ioutil: n is equal or greater than cw.writeBytes 2018-10-27 16:13:33 +08:00
Gyuho Lee
07fcc26799 *: fix gofmt warnings with Go 1.11
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2018-08-28 21:45:24 -07:00