Mirroristas/etcd
2
0
Fork 0
mirror of https://github.com/etcd-io/etcd.git synced 2024-09-27 06:25:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
18,089 Commits 15 Branches 491 Tags 148 MiB
Commit Graph

104 Commits

Author SHA1 Message Date
Marek Siarkowicz
2b090e86a6 server: Extract hasher to separate interface 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:20:18 +02:00
Marek Siarkowicz
80828b593a server: Remove duplicated compaction revision 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:20:18 +02:00
Marek Siarkowicz
34a02ba621 server: Return revision range that hash was calcualted for 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:20:17 +02:00
Marek Siarkowicz
76d3c527ae server: Store real rv range in hasher 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:26 +02:00
Marek Siarkowicz
8f7b053480 server: Move adjusting revision to hasher 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:26 +02:00
Marek Siarkowicz
19941654fe server: Pass revision as int 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:26 +02:00
Marek Siarkowicz
887e53e61a server: Calculate hash during compaction 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:26 +02:00
Marek Siarkowicz
7381c81a7e server: Fix range in mock not returning same number of keys and values 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:26 +02:00
Marek Siarkowicz
08a43178fa server: Move reading KV index inside scheduleCompaction function 
Makes it easier to test hash match between scheduleCompaction and
HashByRev.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:26 +02:00
Marek Siarkowicz
638ca1006a server: Return error from scheduleCompaction 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:26 +02:00
Marek Siarkowicz
0f90359c4b server: Refactor hasher 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:25 +02:00
Marek Siarkowicz
e62c358793 server: Extract kvHash struct 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:25 +02:00
Marek Siarkowicz
fcaf76dbca server: Move unsafeHashByRev to new hash.go file 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:25 +02:00
Marek Siarkowicz
0984878ae7 server: Extract unsafeHashByRev function 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:25 +02:00
Marek Siarkowicz
39c6935c65 server: Test HashByRev values to make sure they don't change 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:25 +02:00
Marek Siarkowicz
7c35dadc25 server: Extract corruption detection to dedicated struct 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-06-13 18:19:24 +02:00
SimFG
b295cebc05 mvcc: improve the use of locks in index.go 
To make the meaning of the RangeSince function in treeIndex clearer.
 2022-06-09 20:35:02 +08:00
Piotr Tabor
b073129d03 Applier does not depend on EtcdServer any longer. 
All the depencies are explicily passed to the UberApplier factory method.
 2022-05-20 14:32:04 +02:00
Piotr Tabor
d69e07dd3a Verification framework and check whether cindex is not decreasing. 2022-04-22 12:32:05 +02:00
ahrtr
a3650db574 use readTx in (*store).restore 2022-04-08 15:45:05 +08:00
Marek Siarkowicz
1ea53d527e server: Save consistency index and term to backend even when they decrease 
Reason to store CI and term in backend was to make db fully independent
snapshot, it was never meant to interfere with apply logic. Skip of CI
was introduced for v2->v3 migration where we wanted to prevent it from
decreasing when replaying wal in
https://github.com/etcd-io/etcd/pull/5391. By mistake it was added to
apply flow during refactor in
https://github.com/etcd-io/etcd/pull/12855#commitcomment-70713670.

Consistency index and term should only be negotiated and used by raft to make
decisions. Their values should only driven by raft state machine and
backend should only be responsible for storing them.
 2022-04-07 19:00:03 +02:00
ahrtr
4033f5c2b9 move the consistentIdx and consistentTerm from Etcdserver to cindex package 
Removed the fields consistentIdx and consistentTerm from struct EtcdServer,
and added applyingIndex and applyingTerm into struct consistentIndex in
package cindex. We may remove the two fields completely if we decide to
remove the OnPreCommitUnsafe, and it will depend on the performance test
result.
 2022-04-07 15:16:49 +08:00
ahrtr
e155e50886 rename LockWithoutHook to LockOutsideApply and add LockInsideApply 2022-04-07 05:35:13 +08:00
ahrtr
7ac995cdde enhanced authBackend to support authReadTx 2022-04-07 05:35:13 +08:00
ahrtr
a4c5da844d added detailed comment to explain the difference between Lock and LockWithoutHook 2022-04-07 05:35:13 +08:00
ahrtr
bfd5170f66 add a txPostLockHook into the backend 
Previously the SetConsistentIndex() is called during the apply workflow,
but it's outside the db transaction. If a commit happens between SetConsistentIndex
and the following apply workflow, and etcd crashes for whatever reason right
after the commit, then etcd commits an incomplete transaction to db.
Eventually etcd runs into the data inconsistency issue.

In this commit, we move the SetConsistentIndex into a txPostLockHook, so
it will be executed inside the transaction lock.
 2022-04-07 05:35:13 +08:00
Marek Siarkowicz
ad03f2076a
Merge pull request #13886 from serathius/backend-logger 
tests: Pass logger to backend
 2022-04-05 16:35:07 +02:00
Marek Siarkowicz
73fc864247 tests: Pass logger to backend 2022-04-05 15:53:38 +02:00
Marek Siarkowicz
1d3517020b server: Add verification of whether lock was called within out outside of apply 2022-04-05 15:34:45 +02:00
Piotr Tabor
6c974a3e31
Merge pull request #13867 from serathius/logs-test 
tests: Use zaptest.NewLogger in tests
 2022-04-04 14:47:04 +02:00
Marek Siarkowicz
804fddf921 tests: Use zaptest.NewLogger in tests 2022-04-04 13:03:15 +02:00
ahrtr
836bd6bc3a fix WARNING: DATA RACE issue when multiple goroutines access the backend concurrently 2022-04-03 06:13:09 +08:00
ahrtr
9b3b383366 the file server/storage/mvcc/util.go isn't used at all, so removing it 2022-03-31 10:14:46 +08:00
ahrtr
3ecd8d2364 update incorrect migration log message 
It can be upgrade or downgrade, so we should use word "updated" here.
 2022-03-10 04:39:56 +08:00
ahrtr
d578a86a59 fix some log format typos 2022-03-10 03:26:34 +08:00
ahrtr
1ae5aa52de fix some typos related to downgrade 2022-03-09 16:07:18 +08:00
kkkkun
124bc1ebbc mvcc: clean up duplicate metrics name 2022-03-01 18:00:07 +08:00
kkkkun
0ecd4de923 delete duplicate metrics rangeCounterDebug 2022-02-28 16:22:40 +08:00
Piotr Tabor
f80f477073
Merge pull request #13644 from Juneezee/refactor/t.TempDir 
*: use `T.TempDir` to create temporary test directory
 2022-02-21 19:52:37 +01:00
Sahdev Zala
aa75fd0850
Merge pull request #13689 from AdamKorcz/fuzz7 
server/storage/wal: fix oss-fuzz issue 44433
 2022-02-14 15:48:55 -05:00
Marek Siarkowicz
09e35a44ec
Merge pull request #13676 from chaochn47/fix_12535 
Load all leases from backend
 2022-02-14 21:32:33 +01:00
Marek Siarkowicz
a0347261e4
Merge pull request #13679 from lavacat/defrag-bopts-fix 
server/storage/backend: restore original bolt db options after defrag
 2022-02-14 20:24:02 +01:00
AdamKorcz
0fcb2a628c server/storage/wal: fix oss-fuzz issue 44433 2022-02-14 11:50:27 +00:00
Chao Chen
fd79af9ee7 load all leases from backend 2022-02-13 20:30:20 -08:00
Bogdan Kanivets
01347a8f53 server/storage/backend: restore original bolt db options after defrag 
Problem: Defrag was implemented before custom bolt options were added.
Currently defrag doesn't restore backend options.
For example BackendFreelistType will be unset after defrag.

Solution: save bolt db options and use them in defrag.
 2022-02-11 11:01:27 -08:00
AdamKorcz
029c354316 server/storage/mvcc: fix oss-fuzz issue 44449 2022-02-11 10:57:02 +00:00
Marek Siarkowicz
f4187b4460 tools: Improve proto annotation documentation 2022-01-28 11:40:55 +01:00
Marek Siarkowicz
4bfcd84259 server: Handle V2Request when analysis WAL entries 2022-01-26 15:50:14 +01:00
Marek Siarkowicz
d865bb96f1 server: Refactor wal version to use visitor pattern 2022-01-26 15:50:14 +01:00
Marek Siarkowicz
6d808e5d7d *: Add static validation to etcd_version proto annotation 2022-01-26 15:50:14 +01:00