48 Commits

Author SHA1 Message Date
Marek Siarkowicz
2b090e86a6 server: Extract hasher to separate interface
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:18 +02:00
Marek Siarkowicz
80828b593a server: Remove duplicated compaction revision
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:18 +02:00
Marek Siarkowicz
34a02ba621 server: Return revision range that hash was calcualted for
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:17 +02:00
Marek Siarkowicz
76d3c527ae server: Store real rv range in hasher
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
8f7b053480 server: Move adjusting revision to hasher
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
19941654fe server: Pass revision as int
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
887e53e61a server: Calculate hash during compaction
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
7381c81a7e server: Fix range in mock not returning same number of keys and values
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
08a43178fa server: Move reading KV index inside scheduleCompaction function
Makes it easier to test hash match between scheduleCompaction and
HashByRev.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
638ca1006a server: Return error from scheduleCompaction
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
0f90359c4b server: Refactor hasher
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:25 +02:00
Marek Siarkowicz
e62c358793 server: Extract kvHash struct
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:25 +02:00
Marek Siarkowicz
fcaf76dbca server: Move unsafeHashByRev to new hash.go file
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:25 +02:00
Marek Siarkowicz
0984878ae7 server: Extract unsafeHashByRev function
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:25 +02:00
Marek Siarkowicz
39c6935c65 server: Test HashByRev values to make sure they don't change
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:25 +02:00
Marek Siarkowicz
7c35dadc25 server: Extract corruption detection to dedicated struct
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:24 +02:00
SimFG
b295cebc05 mvcc: improve the use of locks in index.go
To make the meaning of the RangeSince function in treeIndex clearer.
2022-06-09 20:35:02 +08:00
ahrtr
a3650db574 use readTx in (*store).restore 2022-04-08 15:45:05 +08:00
ahrtr
e155e50886 rename LockWithoutHook to LockOutsideApply and add LockInsideApply 2022-04-07 05:35:13 +08:00
ahrtr
bfd5170f66 add a txPostLockHook into the backend
Previously the SetConsistentIndex() is called during the apply workflow,
but it's outside the db transaction. If a commit happens between SetConsistentIndex
and the following apply workflow, and etcd crashes for whatever reason right
after the commit, then etcd commits an incomplete transaction to db.
Eventually etcd runs into the data inconsistency issue.

In this commit, we move the SetConsistentIndex into a txPostLockHook, so
it will be executed inside the transaction lock.
2022-04-07 05:35:13 +08:00
Marek Siarkowicz
ad03f2076a
Merge pull request #13886 from serathius/backend-logger
tests: Pass logger to backend
2022-04-05 16:35:07 +02:00
Marek Siarkowicz
73fc864247 tests: Pass logger to backend 2022-04-05 15:53:38 +02:00
Marek Siarkowicz
1d3517020b server: Add verification of whether lock was called within out outside of apply 2022-04-05 15:34:45 +02:00
Marek Siarkowicz
804fddf921 tests: Use zaptest.NewLogger in tests 2022-04-04 13:03:15 +02:00
ahrtr
9b3b383366 the file server/storage/mvcc/util.go isn't used at all, so removing it 2022-03-31 10:14:46 +08:00
kkkkun
124bc1ebbc mvcc: clean up duplicate metrics name 2022-03-01 18:00:07 +08:00
kkkkun
0ecd4de923 delete duplicate metrics rangeCounterDebug 2022-02-28 16:22:40 +08:00
AdamKorcz
029c354316 server/storage/mvcc: fix oss-fuzz issue 44449 2022-02-11 10:57:02 +00:00
Piotr Tabor
14a661e643
Merge pull request #13601 from lixd/op-put-lease
mvcc: add a fast return at put method
2022-01-19 14:47:26 +01:00
horizonzy
5b09de33a5 code clean: make rangeKeys close to storeTxnRead. 2022-01-17 11:12:25 +08:00
lixd
1d706179be mvcc: add test-case
a test-case for new code-path.
2022-01-16 10:43:15 +08:00
Piotr Tabor
0285f74aea
Merge pull request #13558 from gfuzz-asplos/main
fixing goroutine leaks
2022-01-15 16:58:19 +01:00
lixd
401cc1a575 mvcc: add a fast return at put method
return when two leaseID is equal, avoid invalid operations.

Fixes #13600
2022-01-15 13:16:44 +08:00
Piotr Tabor
68fa5dcf99
Merge pull request #13549 from songlh-psu/main
fixing the goroutine leaks in TestHashKVWhenCompacting
2022-01-14 13:58:16 +01:00
songlh
a9652b4b4e fixing the leaks in TestStressWatchCancelClose 2022-01-04 17:57:19 -05:00
Liang Zheng
0cc789d81d update dump db tool
Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
2022-01-01 00:13:33 +08:00
Linhai
98b0d901e8 fixing goroutine leaks 2021-12-24 15:57:38 -05:00
Linhai
a45c73d9b1 resolve the conflict 2021-12-21 17:49:47 -05:00
Linhai
246e7eba09 fixing the goroutine in two unit tests 2021-12-21 04:46:39 -05:00
Linhai Song
5e8f50bb09 remove the extra stop 2021-12-17 20:03:19 -05:00
Linhai Song
0098dbf350 fixing two goroutine leaks and one panic 2021-12-15 22:38:25 -05:00
leoyang.yl
7e6c29c198 fix runlock bug 2021-11-26 11:05:36 +08:00
Chao Chen
9038e65359 scripts/genproto.sh: fix module path of wal proto files 2021-10-15 22:00:16 -07:00
Sunny Liu
e7a09db019 mvcc: simplify watchable_store addVictim code 2021-09-24 17:49:03 +08:00
Manuel Rüger
c9b4c866ce Cleanup remaining TODOs for/past 3.5 release 2021-08-06 13:29:25 +02:00
Marek Siarkowicz
a97e48e08d Cleanup references to bucket module 2021-07-20 17:50:47 +02:00
Marek Siarkowicz
5b6f4579fb server: Rename buckets to schema 2021-07-12 15:37:21 +02:00
Marek Siarkowicz
5e40a8b00c server: Create storage package and move mvcc files to it 2021-07-12 15:37:21 +02:00