11 Commits

Author SHA1 Message Date
Benjamin Wang
bd9f1584d4 process the scenaro of the last WAL record being partially synced to disk
We need to return io.ErrUnexpectedEOF in the error chain, so that
etcdserver can repair it automatically.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-01-08 04:46:51 +08:00
Piotr Tabor
e571fb7baa Add --raw mode to ./etcd-dump-log
This mode allows to look at RAW protos for all entries in WAL logs in the given directory.

Signed-off-by: Piotr Tabor <ptab@google.com>
2022-12-30 09:22:34 +01:00
Piotr Tabor
58681d3feb Expose types of entries within the WAL log for access from the tools.
Signed-off-by: Piotr Tabor <ptab@google.com>
2022-12-30 09:19:23 +01:00
Piotr Tabor
0d8aad54ba wal: Expose Decoder as package visible interface.
Such that can be used by tools.

Signed-off-by: Piotr Tabor <ptab@google.com>
2022-12-30 09:19:23 +01:00
Piotr Tabor
bee2a08968 wal decoding: Add optional mode to ignore CRC errors.
Signed-off-by: Piotr Tabor <ptab@google.com>
2022-12-30 09:19:23 +01:00
Piotr Tabor
498619bdda wal decoder: report file & offset in case of CRC mismatch.
Signed-off-by: Piotr Tabor <ptab@google.com>
2022-12-30 09:19:23 +01:00
Benjamin Wang
faff80a2b3 etcdserve: format the source code
gofmt -w ./server

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 13:00:59 +08:00
Benjamin Wang
e9aa275b36 etcdserver: update etcdserver to use the new raft module go.etcd.io/raft/v3
Just replaced all go.etcd.io/etcd/raft/v3 with go.etcd.io/raft/v3
under directory server.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 09:33:45 +08:00
demoManito
72cf0cc04a etcd: modify declaring empty slices
declare an empty slice to var s []int replace  s :=[]int{}, https://github.com/golang/go/wiki/CodeReviewComments#declaring-empty-slices

Signed-off-by: demoManito <1430482733@qq.com>
2022-09-16 14:41:14 +08:00
Benjamin Wang
ccf477d12b restrict the max size of each WAL entry to the remaining size of the file
Currently the max size of each WAL entry is hard coded as 10MB. If users
set a value > 10MB for the flag --max-request-bytes, then etcd may run
into a situation that it successfully processes a big request, but fails
to decode it when replaying the WAL file on startup.

On the other hand, we can't just remove the limitation, because if a
WAL entry is somehow corrupted, and its recByte is a huge value, then
etcd may run out of memory. So the solution is to restrict the max size
of each WAL entry as a dynamic value, which is the remaining size of
the WAL file.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-17 05:41:49 +08:00
Marek Siarkowicz
44b8ae145b etcdserver: Move datadir and wal to storage package 2021-08-03 12:47:37 +02:00