12 Commits

Author SHA1 Message Date
Benjamin Wang
1fe7b9793a etcdserver: add failpoints walBeforeSync and walAfterSync
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-02-08 09:08:09 +08:00
Benjamin Wang
bd9f1584d4 process the scenaro of the last WAL record being partially synced to disk
We need to return io.ErrUnexpectedEOF in the error chain, so that
etcdserver can repair it automatically.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-01-08 04:46:51 +08:00
Piotr Tabor
58681d3feb Expose types of entries within the WAL log for access from the tools.
Signed-off-by: Piotr Tabor <ptab@google.com>
2022-12-30 09:19:23 +01:00
Piotr Tabor
0d8aad54ba wal: Expose Decoder as package visible interface.
Such that can be used by tools.

Signed-off-by: Piotr Tabor <ptab@google.com>
2022-12-30 09:19:23 +01:00
Benjamin Wang
faff80a2b3 etcdserve: format the source code
gofmt -w ./server

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 13:00:59 +08:00
Benjamin Wang
e9aa275b36 etcdserver: update etcdserver to use the new raft module go.etcd.io/raft/v3
Just replaced all go.etcd.io/etcd/raft/v3 with go.etcd.io/raft/v3
under directory server.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 09:33:45 +08:00
Benjamin Wang
686846ec65 add more debug info for opening WAL files failure
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-28 18:58:01 +08:00
Benjamin Wang
3dd7d3f9af enhance the WAL file related error
The `ErrFileNotFound` was used for for three cases:
1. There is no any WAL files (probably due to no read permission);
2. There is no WAL files matching the snapshot index;
3. The WAL file seqs do not increase continuously.

It's not good for debug when users see the `ErrFileNotFound` error,
so in this PR, a different error is returned for each case above.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-08-03 05:37:30 +08:00
Benjamin Wang
ccf477d12b restrict the max size of each WAL entry to the remaining size of the file
Currently the max size of each WAL entry is hard coded as 10MB. If users
set a value > 10MB for the flag --max-request-bytes, then etcd may run
into a situation that it successfully processes a big request, but fails
to decode it when replaying the WAL file on startup.

On the other hand, we can't just remove the limitation, because if a
WAL entry is somehow corrupted, and its recByte is a huge value, then
etcd may run out of memory. So the solution is to restrict the max size
of each WAL entry as a dynamic value, which is the remaining size of
the WAL file.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-06-17 05:41:49 +08:00
Marek Siarkowicz
431adc5878 server: Implement storage downgrades
By validating if WAL doesn't include any incompatible entries we can
implement storage downgrades.
2021-10-29 12:47:19 +02:00
Aleksandr Razumov
3644c9d67b
client/pkg/fileutil: add missing logger to {Create,Touch}DirAll
Also populate it to every invocation.
2021-10-07 17:53:59 +03:00
Marek Siarkowicz
44b8ae145b etcdserver: Move datadir and wal to storage package 2021-08-03 12:47:37 +02:00