70 Commits

Author SHA1 Message Date
Benjamin Wang
1d7639f796 etcdserver: added more debug log for the purgeFile goroutine
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-10-12 19:39:20 +08:00
vivekpatani
e4deb09c9e etcdserver,pkg: remove temp files in snap dir when etcdserver starting
- Backporting: https://github.com/etcd-io/etcd/pull/12846
- Reference: https://github.com/etcd-io/etcd/issues/14232

Signed-off-by: vivekpatani <9080894+vivekpatani@users.noreply.github.com>
2022-07-21 15:50:27 -07:00
Kir Kolyshkin
bea35fd2c6 pkg/fileutil: fix F_OFD_ constants
Use golang.org/x/sys/unix for F_OFD_* constants.

This fixes the issue that F_OFD_GETLK was defined incorrectly,
resulting in bugs such as https://github.com/moby/moby/issues/31182

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-12-14 10:42:13 -08:00
Sahdev P. Zala
d5ebbbceb8 pkg: file stat warning
Provide warning and doc instead of enforcing file permission.
2020-08-24 11:21:29 -04:00
tangcong
0207d1df66 pkg/fileutil: print desired file permission in error log 2020-06-29 09:59:19 +08:00
Changxin Miao
e5424fc474 pkg: Fix dir permission check on Windows 2020-06-25 20:20:55 -04:00
Sahdev P. Zala
434f7e83f0 pkg: check file stats
modify file util.
2020-06-20 16:29:47 -04:00
Jingyi Hu
4f3c81d81d etcdserver: wait purge file loop during shutdown
To prevent the purge file loop from accidentally acquiring the file lock
and remove the files during server shutdowm.
2019-10-30 16:04:41 -07:00
Gyuho Lee
1caaa9ed4a test: test update for Go 1.12.5 and related changes
Update to Go 1.12.5 testing. Remove deprecated unused and gosimple
pacakges, and mask staticcheck 1006. Also, fix unconvert errors related
to unnecessary type conversions and following staticcheck errors:
- remove redundant return statements
- use for range instead of for select
- use time.Since instead of time.Now().Sub
- omit comparison to bool constant
- replace T.Fatal and T.Fatalf in tests with T.Error and T.Fatalf respectively because the goroutine calls T.Fatal must be called in the same goroutine as the test
- fix error strings that should not be capitalized
- use sort.Strings(...) instead of sort.Sort(sort.StringSlice(...))
- use he status code of Canceled instead of grpc.ErrClientConnClosing which is deprecated
- use use status.Errorf instead of grpc.Errorf which is deprecated

Related #10528 #10438
2019-06-05 17:02:05 -04:00
Gyuho Lee
d37f1521b7 *: update import paths to "go.etcd.io/etcd"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2018-08-28 17:47:55 -07:00
Gyuho Lee
fa6d2e2d71 pkg/fileutil: add "ReadDirOption"
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-05-18 10:28:27 -07:00
Gyuho Lee
2bb1a268b3 pkg/fileutil: clarify flock errors
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-05-03 14:00:42 -07:00
Gyuho Lee
0dad8abb6f pkg: support structured logger
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-04-16 17:36:00 -07:00
Gyuho Lee
b8bf42cc5a pkg/fileutil: test "Exist" on directory
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-04-08 14:26:30 -07:00
Gyu-Ho Lee
962976f2df pkg/fileutil: fix preallocate under OS X kernel
ftruncate changes st_blocks, and following fallocate
syscalls would return EINVAL when allocated block size
is already greater than requested block size
(e.g. st_blocks==8, requested blocks are 2).

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-12-14 10:36:57 -08:00
Gyu-Ho Lee
75110dd839 *: fix naked returns
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-11-10 18:46:15 -08:00
Anthony Romano
a36d62a30c Merge pull request #8049 from heyitsanthony/flock-base-test
fileutil: test some fallback functionality
2017-06-07 16:12:38 -07:00
Anthony Romano
c3fcf0f339 fileutil: test some fallback functionality
syscall.Flock fallback and preallocExtendTrunc
2017-06-07 11:22:40 -07:00
Anthony Romano
87a3c87e45 fileutil: return immediately if preallocating 0 bytes
fallocate will return EINVAL, causing zeroing to the end of a
0 byte file to fail.

Fixes #8045
2017-06-07 09:57:14 -07:00
Gyu-Ho Lee
aca2abd8fe *: use 'io.Seek*' for go1.7+
For https://github.com/coreos/etcd/issues/6174.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-03-20 15:15:24 -07:00
Anthony Romano
2f1542c06d *: use filepath.Join for files 2017-03-16 07:46:06 -07:00
Xiang Li
9473e9c30e pkg: use etcd as logging repo 2016-09-29 15:29:38 +08:00
Anthony Romano
c449da6ff9 fileutil: windows OpenDir
Windows needs to open a directory with write access to fsync but the go
runtime won't open directories that way.
2016-09-08 00:09:16 -07:00
Anthony Romano
58a36cb651 fileutil: add ZeroToEnd for zeroing files 2016-08-25 14:24:46 -07:00
Anthony Romano
c920ce0453 fileutil: rework purge tests so they don't poll
Fixes #5966
2016-07-18 14:19:09 -07:00
Gyu-Ho Lee
187faba3e0 pkg/fileutil: fix TouchDirAll, add CreateDirAll
os.MkdirAll never returns os.ErrExist.
And add another function to ensure deepest
directory is empty.
2016-06-22 15:54:17 -07:00
James Shubin
6480066054 *: misc typos and go vet fixes 2016-06-22 10:32:13 -07:00
Gyu-Ho Lee
47d5257622 pkg/fileutil: expose PrivateFileMode/DirMode 2016-06-10 15:22:14 -07:00
Jordan Liggitt
4f7622fb9a fileutil: avoid double preallocation 2016-06-10 00:27:59 -04:00
Anthony Romano
54aac4ab7e pkg/fileutil: fall back to truncate() if fallocate is interrupted
Fixes #5558
2016-06-06 09:52:34 -07:00
Gyu-Ho Lee
8b77de4e99 pkg: update LICENSE header 2016-05-12 20:48:53 -07:00
Gyu-Ho Lee
4ab1500a6d pkg/fileutil: wait up to 300ms for purge test
Fix https://github.com/coreos/etcd/issues/5231.

The issue shows that slow CI can take more than 200ms
for purging. This increase the loop iteration to wait
up to 300ms in case the disk is being slow.
2016-04-29 15:24:44 -07:00
Ajit Yagaty
8b6de5f85d fileutil: Sync on HFS/OSX needs to be handled differently.
A call file.Sync on OSX doesn't guarantee actual persistence on
physical drive media as the data can be cached in physical drive's
buffers. Hence calls to file.Sync need to be replaced with
fcntl(F_FULLFSYNC).
2016-04-18 21:49:04 -07:00
Anthony Romano
d4ff9364d4 Merge pull request #4861 from heyitsanthony/nfs-lock
pkg/fileutil: fix linux file locks over NFS
2016-04-16 08:59:10 -07:00
Gyu-Ho Lee
b0cc0e443c *: clean up if, bool comparison 2016-04-02 12:55:11 -07:00
Gyu-Ho Lee
3f1a1c3192 pkg/fileutil: lock file on Windows 2016-03-27 00:35:44 -07:00
Anthony Romano
877030ea9d pkg/fileutil: fix linux file locks over NFS
Fixes #4853
2016-03-25 16:28:29 -07:00
Anthony Romano
bd832e5b0a *: migrate Godeps to vendor/ 2016-03-22 17:10:28 -07:00
Anthony Romano
aafe717f2f fileutil: support file extending preallocate 2016-03-21 09:42:30 -07:00
Anthony Romano
7397e14c0a fileutil, wal: refactor file locking
File lock interface was more verbose than it needed to be while
simultaneously making it difficult to support systems (e.g., Windows)
that only permit locked writes on a single fd holding the lock.
2016-03-16 15:02:15 -07:00
Xiang Li
e59efe45a1 wal: support fadatasync on linux 2016-03-13 17:22:53 -07:00
Gyu-Ho Lee
22a8bbd3b1 pkg/fileutil: clean up interface, comments 2016-03-09 09:19:56 -08:00
Anthony Romano
20461ab11a *: fix many typos 2016-01-31 21:42:39 -08:00
Xiang Li
150e646b05 etcdserver: always check if the data dir is writable before starting etcd 2015-12-29 11:29:01 -08:00
Gyu-Ho Lee
8d368c4dba pkg/fileutil: fix error var shadow
Go tip complains about error variable shadowing at
https://travis-ci.org/coreos/etcd/jobs/98636879#L291-L292.
2015-12-23 23:56:26 -08:00
Akihiro Suda
058f1449d6 pkg/fileutil: skip TestIsDirWriteable when running as root 2015-12-24 14:52:40 +09:00
Xiang Li
96731f4525 Merge pull request #3991 from xiang90/fix_lock
pkg/fileutil: make TestLockAndUnlock less flaky on CI
2015-12-14 11:58:14 -08:00
Xiang Li
be3e87b238 pkg/fileutil: make TestLockAndUnlock less flaky on CI
CI is slow sometime. To make the test less flaky, we increases the
timeout. This does not affect the correctness, but the test might
take longer to finish or to fail.
2015-12-14 11:38:07 -08:00
Gyu-Ho Lee
4e06510280 pkg/fileutil: fix shadowed variables 2015-12-12 09:38:26 -08:00
Xiang Li
e93c07ba91 pkg/fileutil: make purge test more reliable 2015-12-09 10:34:38 -08:00