Mirroristas/etcd
2
0
Fork 0
mirror of https://github.com/etcd-io/etcd.git synced 2024-09-27 06:25:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
15,652 Commits 15 Branches 491 Tags 148 MiB
Commit Graph

77 Commits

Author SHA1 Message Date
Gyuho Lee
4571e528f4 wal: check out of range slice in "ReadAll", "decoder" 
wal: add slice bound checks in decoder

CHANGELOG-3.5: add wal slice bound check
CHANGELOG-3.5: add "decodeRecord"

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2020-06-22 11:57:43 -04:00
Sahdev P. Zala
7d1cf64049 wal: fix panic when decoder not set 
Handle the related panic and clarify doc.
 2020-06-21 16:21:34 -04:00
tangcong
166b4473fa wal: add TestValidSnapshotEntriesAfterPurgeWal testcase 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2020-05-20 11:38:06 -07:00
Viacheslav Biriukov
87fc3c9e57 etcdserver,wal: fix inconsistencies in WAL and snapshot 
etcdserver/*, wal/*: changes to snapshots and wal logic
etcdserver/*: changes to snapshots and wal logic to fix #10219
etcdserver/*, wal/*: add Sync method
etcdserver/*, wal/*: find valid snapshots by cross checking snap files and wal snap entries
etcdserver/*, wal/*:Add comments, clean up error messages and tests
etcdserver/*, wal/*: Remove orphaned .snap.db files during Release

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2020-05-15 08:40:09 -07:00
Gyuho Lee
34bd797e67 *: revert module import paths 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-05-28 15:39:35 -07:00
Joshua Coutinho
a0c889d14b wal: add a test for wal cleanup, improve comments 
To add test coverage of wal cleanup.
 2019-05-10 22:36:26 +01:00
shivaramr
9150bf52d6 go modules: Fix module path version to include version number 2019-04-26 15:29:50 -07:00
Shreyas Rao
bb3eb8fea9 wal: Add test for Verify 
Signed-off-by: Shreyas Rao <shreyas.sriganesh.rao@sap.com>
 2019-03-12 22:25:25 +05:30
Gyuho Lee
038fd844ac wal: update Go import paths to "go.etcd.io/etcd" 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2018-08-28 17:47:55 -07:00
Gyuho Lee
6fabe6d5ab wal: clean up code format 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-07-21 15:49:58 -07:00
Gyuho Lee
e15ce28168 wal: add missing logs, improve pipeline test coverage 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-05-17 11:01:31 -07:00
Gyuho Lee
fdbedacc83 wal: support structured logger 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-16 17:36:00 -07:00
lorneli
7c50c06fb8 wal: tiny refactor 
a. add comment of reopening file in cut function.
b. add const frameSizeBytes in decoder.
c. return directly if locked files empty in ReleaseLockTo function.
 2017-09-07 02:50:37 +08:00
Gyu-Ho Lee
aca2abd8fe *: use 'io.Seek*' for go1.7+ 
For https://github.com/coreos/etcd/issues/6174.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
 2017-03-20 15:15:24 -07:00
Tess Rinearson
39c733ebe7 wal: use path/filepath instead of path 
Use the path/filepath package instead of the path package. The
path package assumes slash-separated paths, which doesn't work
on Windows. But path/filepath manipulates filename paths in a way
that's compatible across OSes.
 2017-03-15 17:30:23 -07:00
Gyu-Ho Lee
3d75395875 *: remove never-unused vars, minor lint fix 
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
 2017-03-06 14:59:12 -08:00
Gyu-Ho Lee
f5588526cc wal: set PageWriter offset in file encoder 2016-09-28 11:03:24 -07:00
Vimal Kumar
064411b51c wal/wal.go : improved coverage by testing WAL.Save which causes a WAL.cut to happen 2016-09-21 16:50:55 +05:30
Aaron Lehmann
af4f82228c wal: hold file lock while renaming WAL directory on non-Windows 
Windows requires this lock to be released before the directory is
renamed. But on unix-like operating systems, releasing the lock and
trying to reacquire it immediately can be flaky if a process is forked
around the same time. The file descriptors are marked as close-on-exec
by the Go runtime, but there is a window between the fork and exec where
another process will be holding the lock.
 2016-08-26 09:27:51 -07:00
Anthony Romano
0d8d9a374c wal: test for truncation on torn writes 2016-08-25 14:24:46 -07:00
Gyu-Ho Lee
b4aa4607cb wal: use bytes.Equal, other minor updates 
- Replace reflect.Equal with bytes.Equal where possible
- Remove some TODOs
- Some minor simplifications
 2016-06-13 01:33:53 -07:00
Gyu-Ho Lee
4570eddc2c wal: PrivateFileMode/DirMode as in pkg/fileutil 
To make it consistent with pkg/fileutil
 2016-06-10 15:20:57 -07:00
Gyu-Ho Lee
4a5befc2de wal: update LICENSE header 2016-05-12 20:50:04 -07:00
Anthony Romano
17391336af wal: atomically initialize wal directory 
Fixes #5270
 2016-05-11 16:50:17 -07:00
Gyu-Ho Lee
641a1a66e1 *: fix govet -shadow in go tip 2016-04-15 07:39:52 -07:00
Anthony Romano
bfe3a3d08e wal: fix tail corruption 
On ReadAll, WAL seeks to the end of the last record in the tail. If the tail did not
end with preallocated space, the decoder would report 0 as the last offset and begin
writing at offset 0 of the tail.

Fixes #4903
 2016-04-01 15:05:52 -07:00
Anthony Romano
24b806d2ee wal: preallocate WAL files with initial size equal to segment size 
Avoids having to update file size metadata during fdatasync on common path.

Fixes #4755
 2016-03-21 11:56:53 -07:00
Anthony Romano
7397e14c0a fileutil, wal: refactor file locking 
File lock interface was more verbose than it needed to be while
simultaneously making it difficult to support systems (e.g., Windows)
that only permit locked writes on a single fd holding the lock.
 2016-03-16 15:02:15 -07:00
Gyu-Ho Lee
a4de207d53 wal: fix shadowed variables 
Fixes for https://github.com/coreos/etcd/issues/3954.
 2015-12-12 09:38:26 -08:00
Dmitry Smirnov
b2f4a5f587 *: fix spelling issues (codespell). 
Signed-off-by: Dmitry Smirnov <onlyjob@member.fsf.org>
 2015-09-11 10:22:29 +10:00
Xiang Li
f59da0e453 *:fix point-in-time backup 
Backup process should be able to read all WALs until io.EOF to
generate a point-in-time backup.

Our WAL file is append-only. And the backup process will lock all
files before start reading, which can prevent the gc routine from
removing any files in the middle.
 2015-06-15 11:12:28 -07:00
Yicheng Qin
3dd6e0b88f wal: fix missing import 2015-03-24 22:53:15 -07:00
Xiang Li
6e6669d696 wal: releastTo should work with large release index 2015-03-24 22:34:26 -07:00
Xiang Li
b66eb3d81c wal: fix ReleaseLockTo 
ReleaseLockTo should not release the lock on the WAL
segment that is right before the given index. When
restarting etcd, etcd needs to read from the WAL segment
that has a smaller index than the snapshot index.

The correct behavior is that ReleaseLockTo releases
the locks w is holding so that w only holds one lock
that has an index smaller than the given index.
 2015-03-09 19:52:54 -07:00
Xiang Li
86429264fb wal: support auto-cut in wal 
WAL should control the cut logic itself. We want to do falloc to
per allocate the space for a segmented wal file at the beginning
and cut it when it size reaches the limit.
 2015-02-28 11:18:59 -08:00
Barak Michener
92dca0af0f *: remove shadowing of variables from etcd and add travis test 
We've been bitten by this enough times that I wrote a tool so that
it never happens again.
 2015-02-17 16:31:42 -05:00
Jonathan Boulle
f1ed69e883 *: switch to line comments for copyright 
Build tags are not compatible with block comments.
Also adds copyright header to a few places it was missing.
 2015-01-26 09:53:30 -08:00
Yicheng Qin
9bdc343b7c wal: add ReleaseLockTo test 2015-01-09 14:59:41 -08:00
Yicheng Qin
270e67db84 wal: not export unnecessary public functions 2015-01-09 14:55:10 -08:00
Yicheng Qin
6460e49a33 wal: save empty snapshot when create 
So caller can open at empty snapshot to read all entries.
 2015-01-06 19:48:21 -08:00
Yicheng Qin
84f62f21ee wal: record and check snapshot 2015-01-06 16:27:40 -08:00
Xiang Li
53bf7e4b5e wal: rename openAtIndex -> open; OpenAtIndexUntilUsing -> openNotInUse 2014-12-14 19:33:06 -08:00
Xiang Li
ea94d19147 *: lock the in using files; do not purge locked the wal files 2014-12-14 19:27:22 -08:00
Yicheng Qin
b9bf957c6d wal: sync after writing data to disk in Cut function 2014-12-04 22:56:34 -08:00
Yicheng Qin
af0f34c595 wal: save latest state into new WAL 
So we could always read out state when open at valid index.
 2014-12-04 12:19:21 -08:00
Yicheng Qin
aa61009560 wal: not return ErrIndexNotFound in ReadAll 
This IndexNotFound case is reasonable now because we don't write dummy
entries into wals any more.
 2014-12-02 00:28:54 -08:00
Xiang Li
ff1f5a9d57 wal: sync before returning from create 2014-11-03 14:28:59 -08:00
Jonathan Boulle
7a4d42166b *: add license header to all source files 2014-10-17 15:41:22 -07:00
Yicheng Qin
447caf1afc etcdserver/wal: record info at the head of WAL file 2014-10-10 11:57:09 -07:00
Xiang Li
8bbbaa88b2 *: raft related int64 -> uint64 2014-10-09 14:29:21 +08:00