Mirroristas/etcd
2
0
Fork 0
mirror of https://github.com/etcd-io/etcd.git synced 2024-09-27 06:25:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
15,645 Commits 15 Branches 491 Tags 148 MiB
Commit Graph

228 Commits

Author SHA1 Message Date
Gyuho Lee
4571e528f4 wal: check out of range slice in "ReadAll", "decoder" 
wal: add slice bound checks in decoder

CHANGELOG-3.5: add wal slice bound check
CHANGELOG-3.5: add "decodeRecord"

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2020-06-22 11:57:43 -04:00
Sahdev P. Zala
7d1cf64049 wal: fix panic when decoder not set 
Handle the related panic and clarify doc.
 2020-06-21 16:21:34 -04:00
David Crawshaw
78f67988aa
etcdserver, et al: add --unsafe-no-fsync flag 
This makes it possible to run an etcd node for testing and development
without placing lots of load on the file system.

Fixes #11930.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
 2020-06-04 20:19:28 -07:00
tangcong
166b4473fa wal: add TestValidSnapshotEntriesAfterPurgeWal testcase 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2020-05-20 11:38:06 -07:00
tangcong
ed231df7c0 wal: fix crc mismatch crash bug 2020-05-20 11:37:04 -07:00
Viacheslav Biriukov
87fc3c9e57 etcdserver,wal: fix inconsistencies in WAL and snapshot 
etcdserver/*, wal/*: changes to snapshots and wal logic
etcdserver/*: changes to snapshots and wal logic to fix #10219
etcdserver/*, wal/*: add Sync method
etcdserver/*, wal/*: find valid snapshots by cross checking snap files and wal snap entries
etcdserver/*, wal/*:Add comments, clean up error messages and tests
etcdserver/*, wal/*: Remove orphaned .snap.db files during Release

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2020-05-15 08:40:09 -07:00
Gyuho Lee
e99399d0dc wal: add "etcd_wal_writes_bytes_total" 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2020-04-01 09:30:09 -07:00
Gyuho Lee
34bd797e67 *: revert module import paths 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-05-28 15:39:35 -07:00
Joshua Coutinho
a0c889d14b wal: add a test for wal cleanup, improve comments 
To add test coverage of wal cleanup.
 2019-05-10 22:36:26 +01:00
jcoutin
f7f7e9c762 wal: Improve cleanup for robustness and debuggability 
Rename wal with '.suffix.<timestamp>' instead of delete it and call cleanup when perr in a 'defer'ed statement.
 2019-05-07 21:38:40 +01:00
Joshua Coutinho
51035bfd84 wal: cleanup wal directory if creation fails 
delete <data-dir>/member/wal if any operation after the rename in
wal.Create fails to avoid reading an inconsistent WAL on restart.

Fixes #10688
 2019-05-04 01:58:57 +01:00
shivaramr
9150bf52d6 go modules: Fix module path version to include version number 2019-04-26 15:29:50 -07:00
Shreyas Rao
914e5edb00 wal: include logger in WAL returned by openAtIndex 
Signed-off-by: Shreyas Rao <shreyas.sriganesh.rao@sap.com>
 2019-04-02 13:09:10 +05:30
Shreyas Rao
bb3eb8fea9 wal: Add test for Verify 
Signed-off-by: Shreyas Rao <shreyas.sriganesh.rao@sap.com>
 2019-03-12 22:25:25 +05:30
shreyas-s-rao
3d6862fe0d wal: add Verify function to perform corruption check on wal contents 
Signed-off-by: Shreyas Rao <shreyas.sriganesh.rao@sap.com>
 2019-03-12 22:25:25 +05:30
Gyuho Lee
038fd844ac wal: update Go import paths to "go.etcd.io/etcd" 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2018-08-28 17:47:55 -07:00
Gyuho Lee
6fabe6d5ab wal: clean up code format 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-07-21 15:49:58 -07:00
Gyuho Lee
b0b966c43c wal: document, clean up fsync histogram 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-05-23 14:03:28 -07:00
Gyuho Lee
567b47fc3e wal: filter file names in WAL directory by ".wal" 
WAL never writes nor needs files without ".wal" suffix.
Thus, safe to filter out only ".wal" files.

Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-05-18 10:29:48 -07:00
Gyuho Lee
e15ce28168 wal: add missing logs, improve pipeline test coverage 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-05-17 11:01:31 -07:00
Gyuho Lee
f3d9a85697 wal: add warnings on fsync, flock fail paths 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-05-03 14:01:06 -07:00
Gyuho Lee
8a28e73862 wal: fix "unconvert" warnings 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-30 15:32:16 -07:00
Gyuho Lee
fdbedacc83 wal: support structured logger 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-16 17:36:00 -07:00
Gyuho Lee
bcd5390b35 *: regenerate protobuf, grpc-gateway 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2017-12-07 21:31:13 -08:00
Gyu-Ho Lee
75110dd839 *: fix naked returns 
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
 2017-11-10 18:46:15 -08:00
lorneli
7c50c06fb8 wal: tiny refactor 
a. add comment of reopening file in cut function.
b. add const frameSizeBytes in decoder.
c. return directly if locked files empty in ReleaseLockTo function.
 2017-09-07 02:50:37 +08:00
Anthony Romano
fe1ddab714 wal: fall back to closing wal if locked dir rename fails 
Detecting windows at compile time isn't enough since etcd might be
on linux but the fs is backed by windows.

Fixes: #8178
Fixes: #6984
 2017-07-20 13:30:41 -07:00
Anthony Romano
78a5eb79b5 *: add swagger and grpc-gateway assets for v3lock and v3election 2017-04-10 15:21:07 -07:00
Gyu-Ho Lee
aca2abd8fe *: use 'io.Seek*' for go1.7+ 
For https://github.com/coreos/etcd/issues/6174.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
 2017-03-20 15:15:24 -07:00
Tess Rinearson
39c733ebe7 wal: use path/filepath instead of path 
Use the path/filepath package instead of the path package. The
path package assumes slash-separated paths, which doesn't work
on Windows. But path/filepath manipulates filename paths in a way
that's compatible across OSes.
 2017-03-15 17:30:23 -07:00
Gyu-Ho Lee
3d75395875 *: remove never-unused vars, minor lint fix 
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
 2017-03-06 14:59:12 -08:00
Peter Mattis
ab03a42f06 raft: add Ready.MustSync 
Add Ready.MustSync which indicates that the hard state and raft log
entries in a Ready message must be synchronously written to persistent
storage.
 2017-02-13 15:13:21 -05:00
Gyu-Ho Lee
b8b72f80f9 *: revendor, update proto files 2016-11-10 12:02:00 -08:00
Gyu-Ho Lee
9b56e51ca7 *: regenerate proto + gofmt change 2016-10-03 15:34:34 -07:00
Anthony Romano
c26ebe3262 Merge pull request #6453 from vimalk78/wal-optimize-marshal-outside-lock 
wal/wal.go: optimized WAL.SaveSnapshot to do Marshal outside the mutex lock
 2016-10-03 11:50:11 -07:00
Gyu-Ho Lee
f5588526cc wal: set PageWriter offset in file encoder 2016-09-28 11:03:24 -07:00
Gyu-Ho Lee
231530e0c5 wal: fix minor wording in comment 2016-09-28 09:12:31 -07:00
Vimal Kumar
f4ec303d1b wal/wal.go: modified WAL.SaveSnapshot to do the Marshal before aquiring the mutex 2016-09-28 10:35:19 +05:30
ychen11
69f5b4ba79 Documentation:made watch request doc more clear 2016-09-23 23:13:55 +08:00
Vimal Kumar
064411b51c wal/wal.go : improved coverage by testing WAL.Save which causes a WAL.cut to happen 2016-09-21 16:50:55 +05:30
Gyu-Ho Lee
ccb46d2024 wal: simplify dir.Close call 2016-09-09 09:23:55 +09:00
Anthony Romano
bd7107bd4b wal: fsync directory after wal file rename 
Fixes #6368
 2016-09-08 00:09:16 -07:00
Anthony Romano
28277b5a65 wal: use page buffered writer for writing records 
Forces torn writes to only happen on sector boundaries.

Fixes #6271
 2016-08-30 15:49:07 -07:00
sharat
69ed560fae wal: document grammar correction 
Corrected grammar mistake for doc.go
 2016-08-30 00:50:02 +05:30
Aaron Lehmann
af4f82228c wal: hold file lock while renaming WAL directory on non-Windows 
Windows requires this lock to be released before the directory is
renamed. But on unix-like operating systems, releasing the lock and
trying to reacquire it immediately can be flaky if a process is forked
around the same time. The file descriptors are marked as close-on-exec
by the Go runtime, but there is a window between the fork and exec where
another process will be holding the lock.
 2016-08-26 09:27:51 -07:00
Anthony Romano
f1ead43482 wal: zero out wal tail past its first zero record 
Whenever the WAL is opened for writes, it should write zeroes to its tail
starting from the first zero record. Otherwise, if there are entries past
the first zero record due to a torn write, any new writes that overlap the
old entries will lead to a garbage record on the tail and cause a CRC
mismatch.
 2016-08-25 14:24:46 -07:00
Anthony Romano
0d8d9a374c wal: test for truncation on torn writes 2016-08-25 14:24:46 -07:00
Aaron Lehmann
2b996b6038 wal: Export SegmentSizeBytes as a variable 
In test situations, it's useful to create smaller than usual WAL files
to test rotation and to avoid the overhead of preallocation on old-style
filesystems that don't handle it efficiently. This commit changes
segmentSizeBytes to an exported variable so that tests can override it
from an init() function.
 2016-08-09 15:38:30 -07:00
Gyu-Ho Lee
982e18d80b *: regenerate proto with latest grpc-gateway 2016-07-27 13:21:03 -07:00
Xiang Li
fffa484a9f *: regenerate proto for adding deleterange 2016-07-23 16:17:44 -07:00