34 Commits

Author SHA1 Message Date
Chun-Hung Tseng
5be397a6a8
Fix a linter issue
golangci-lint reports the following issue:
storage/mvcc/kvstore.go:312:27: (*store).restore - result 0 (error) is always nil (unparam)

It's due to the fact that both Attach() and compactLockfree() within the
function restore() are able to return an error, but we only log them in
the current implementation. Thus, the return value restore() is always
nil, hence the linter warning.

We have agreed to suppress the linter warning for now [1].

Reference:
[1] https://github.com/etcd-io/etcd/pull/18228#issuecomment-2187309957

Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
2024-06-25 11:45:53 +02:00
Cenk Alti
b107d2437f
cli: Add etcdutl snapshot hashkv command
Signed-off-by: Cenk Alti <cenkalti@gmail.com>

Apply suggestions from code review

Co-authored-by: Benjamin Wang <benjamin.wang@broadcom.com>
2024-06-02 00:07:50 -04:00
Wei Fu
94c83a962b server/storage/mvcc: should update currentRev in revMu
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-04-24 11:08:33 +08:00
Benjamin Wang
dd4e35a585
Merge pull request #17815 from fuweid/repro-17780
[RFC] fix revision loss issue caused by compaction - 17780
2024-04-22 18:15:44 +01:00
Benjamin Wang
29529c505b
Merge pull request #17791 from ahrtr/verify_revision_bootstrap_20240415
Add verification that the revision shouldn't decrease on bootstrap
2024-04-22 10:43:01 +01:00
Wei Fu
9ea234913a server/storage: update currentRev if scheduledCompact > currentRev
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-04-21 18:12:38 +08:00
Benjamin Wang
a54098dee3 Update the compaction log when bootstrap and update compact's signature
Actually the compact() never return an error, so remove the second return
parameter.

Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
2024-04-15 14:16:18 +01:00
Benjamin Wang
9ac964d1cc Add verification that the revision shouldn't decrease on bootstrap
Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
2024-04-15 13:56:19 +01:00
Allen Ray
395376d3ab Refactor common revision code to pkg
Signed-off-by: Allen Ray <alray@redhat.com>
2023-10-09 14:53:16 -04:00
Wei Fu
aa97484166 *: enable goimports in verify-lint
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-21 21:14:09 +08:00
caojiamingalan
bc97a94564 Follow up https://github.com/etcd-io/etcd/pull/16068#discussion_r1263664700
Replace unnecessary Lock()/Unlock()s with RLock()/RUnlock()s

Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com>
2023-07-14 20:08:25 -05:00
caojiamingalan
b9e30bf878 etcdserver: add e2e test to reproduce the incorrect hash issue when resuming scheduled compaction.
check ScheduledCompactKeyName and FinishedCompactKeyName
before writing hash to hashstore. If they do not match, then it means this compaction has once been interrupted and its hash value is invalid. In such cases, we won't write the hash values to the hashstore, and avoids the incorrect corruption alarm.

Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com>
2023-06-07 19:54:09 -05:00
Marek Siarkowicz
f96957adba tests: Add compact failpoints
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-11-15 14:59:03 +01:00
Jeremy Leach
cc1e245368 etcdserver: fix corruption check when server has just been compacted
When a key-value store corruption check happens immediately after a
compaction, the revision at which the key-value store hash is computed,
is the compacted revision itself.
In that case, the hash computation logic was incorrect because it
returned an ErrCompacted error; this error should instead be returned when
the revision at which the key-value store is hashed, is strictly lower
than the compacted revision.

Fixes #14325

Signed-off-by: Jeremy Leach <44558776+jbml@users.noreply.github.com>
2022-09-24 22:20:26 +10:00
SimFG
d83925e357 schedule: Provide logs when the fifo job panic happens
To make the fifo scheduler better debuggability.

Signed-off-by: SimFG <1142838399@qq.com>
2022-06-15 20:58:17 +08:00
Marek Siarkowicz
0e739da9a4 server: Cache compaction hash for HashByRev API
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:18 +02:00
Marek Siarkowicz
2b090e86a6 server: Extract hasher to separate interface
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:18 +02:00
Marek Siarkowicz
80828b593a server: Remove duplicated compaction revision
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:18 +02:00
Marek Siarkowicz
34a02ba621 server: Return revision range that hash was calcualted for
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:17 +02:00
Marek Siarkowicz
8f7b053480 server: Move adjusting revision to hasher
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
19941654fe server: Pass revision as int
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
887e53e61a server: Calculate hash during compaction
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
08a43178fa server: Move reading KV index inside scheduleCompaction function
Makes it easier to test hash match between scheduleCompaction and
HashByRev.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
638ca1006a server: Return error from scheduleCompaction
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
fcaf76dbca server: Move unsafeHashByRev to new hash.go file
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:25 +02:00
Marek Siarkowicz
0984878ae7 server: Extract unsafeHashByRev function
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:25 +02:00
ahrtr
a3650db574 use readTx in (*store).restore 2022-04-08 15:45:05 +08:00
ahrtr
e155e50886 rename LockWithoutHook to LockOutsideApply and add LockInsideApply 2022-04-07 05:35:13 +08:00
ahrtr
bfd5170f66 add a txPostLockHook into the backend
Previously the SetConsistentIndex() is called during the apply workflow,
but it's outside the db transaction. If a commit happens between SetConsistentIndex
and the following apply workflow, and etcd crashes for whatever reason right
after the commit, then etcd commits an incomplete transaction to db.
Eventually etcd runs into the data inconsistency issue.

In this commit, we move the SetConsistentIndex into a txPostLockHook, so
it will be executed inside the transaction lock.
2022-04-07 05:35:13 +08:00
kkkkun
124bc1ebbc mvcc: clean up duplicate metrics name 2022-03-01 18:00:07 +08:00
Liang Zheng
0cc789d81d update dump db tool
Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
2022-01-01 00:13:33 +08:00
Marek Siarkowicz
a97e48e08d Cleanup references to bucket module 2021-07-20 17:50:47 +02:00
Marek Siarkowicz
5b6f4579fb server: Rename buckets to schema 2021-07-12 15:37:21 +02:00
Marek Siarkowicz
5e40a8b00c server: Create storage package and move mvcc files to it 2021-07-12 15:37:21 +02:00