19153 Commits

Author SHA1 Message Date
Marek Siarkowicz
9612fc1194 tests: Unify TestCompactionHash and extend it to also Delete keys and Defrag
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:19 +02:00
Marek Siarkowicz
f5eadf5452 tests: Add tests for HashByRev HTTP API
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:19 +02:00
Marek Siarkowicz
1581eef888 tests: Add integration tests for compact hash
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:19 +02:00
Marek Siarkowicz
0e739da9a4 server: Cache compaction hash for HashByRev API
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:18 +02:00
Marek Siarkowicz
2b090e86a6 server: Extract hasher to separate interface
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:18 +02:00
Marek Siarkowicz
80828b593a server: Remove duplicated compaction revision
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:18 +02:00
Marek Siarkowicz
34a02ba621 server: Return revision range that hash was calcualted for
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:17 +02:00
Marek Siarkowicz
76d3c527ae server: Store real rv range in hasher
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
8f7b053480 server: Move adjusting revision to hasher
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
19941654fe server: Pass revision as int
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
887e53e61a server: Calculate hash during compaction
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
7381c81a7e server: Fix range in mock not returning same number of keys and values
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
08a43178fa server: Move reading KV index inside scheduleCompaction function
Makes it easier to test hash match between scheduleCompaction and
HashByRev.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
638ca1006a server: Return error from scheduleCompaction
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:26 +02:00
Marek Siarkowicz
0f90359c4b server: Refactor hasher
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:25 +02:00
Marek Siarkowicz
e62c358793 server: Extract kvHash struct
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:25 +02:00
Marek Siarkowicz
fcaf76dbca server: Move unsafeHashByRev to new hash.go file
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:25 +02:00
Marek Siarkowicz
0984878ae7 server: Extract unsafeHashByRev function
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:25 +02:00
Marek Siarkowicz
39c6935c65 server: Test HashByRev values to make sure they don't change
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:25 +02:00
Marek Siarkowicz
01e85be159 server: Cover corruptionMonitor with tests
Get 100% coverage on InitialCheck and PeriodicCheck functions to avoid
any mistakes.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:25 +02:00
Marek Siarkowicz
7c35dadc25 server: Extract corruption detection to dedicated struct
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:19:24 +02:00
Marek Siarkowicz
70fbc0b3e7 server: Extract triggerCorruptAlarm to function
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:04:24 +02:00
Marek Siarkowicz
fc690531e2
Merge pull request #14108 from serathius/tracked
Use single "stage/tracked" label to prevent issue from getting stale
2022-06-13 13:03:41 +02:00
Marek Siarkowicz
bce22b587c Use single "stage/tracked" label to prevent issue from getting stale
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 11:48:34 +02:00
Marek Siarkowicz
4ce7a85188
Merge pull request #14097 from patrocinio/buildtools
Update build file to run with an argument
2022-06-13 10:55:54 +02:00
Marek Siarkowicz
4592af25ba
Merge pull request #14107 from ahrtr/version_refactor_20220613
Get all constant etcd versions defined in a centralized place
2022-06-13 10:31:46 +02:00
Benjamin Wang
5f8cd5bd39 Replace all etcd versions with the centralized definitions
We have already defined all the constant etcd versions in the
centralized place api/version/version.go. So we should replace all
the versions with the centralized definitions.
2022-06-13 14:08:39 +08:00
Benjamin Wang
4f202a6429 Define all versions in a centralized place
The versions such as V3_5, V3_6, etc. are defined everywhere in
the repo. In this commit, we get all of the constant versions defined
in a centralized place, so that they can be reused by all cases.
2022-06-13 13:37:41 +08:00
Benjamin Wang
144f445838
Merge pull request #13747 from kkkkun/add-flag-for-del
add flag range for del in etcdctl
2022-06-13 08:22:50 +08:00
Eduardo Patrocinio
5a93b087a6 Separate the build library functions and add a top level tools script
The current Makefile doesn't allow the compilation of the tools directory.
This commit creates a build library file, updates the Makefile
and a top level script fod building tools.

To build the tools, you can run make build_tools.

As before, you can run make build to build etcd binaries.
2022-06-10 11:43:10 -04:00
Marek Siarkowicz
4b41f74ff8
Merge pull request #14103 from ahrtr/update_changelog_3.5.0
Update 3.5 changelog to add an entry to cover the new flag `---experimental-bootstrap-defrag-threshold-megabytes`
2022-06-10 11:10:59 +02:00
Benjamin Wang
a7aca4a7a2 update 3.5 changelog to add the --experimental-bootstrap-defrag-threshold-megabytes flag 2022-06-10 09:10:10 +08:00
Marek Siarkowicz
31cd4e7c2d
Merge pull request #14084 from SimFG/mvcc-range-since
mvcc: improve the use of locks in index.go
2022-06-09 15:39:18 +02:00
SimFG
b295cebc05 mvcc: improve the use of locks in index.go
To make the meaning of the RangeSince function in treeIndex clearer.
2022-06-09 20:35:02 +08:00
Benjamin Wang
05e4a5ad23
Merge pull request #14101 from serathius/fix-merge
server: Fix broken code during merge
2022-06-09 20:25:27 +08:00
Marek Siarkowicz
72a84ddbcf server: Fix broken code during merge 2022-06-09 13:34:33 +02:00
Benjamin Wang
5a7df03592
Merge pull request #14080 from ahrtr/lease_revoke_lock
Fix potential deadlock between Revoke and (Grant or Checkpoint)
2022-06-09 16:52:36 +08:00
Sahdev Zala
d9e5460ca8
Merge pull request #13829 from qsyqian/feature/skip_compact_when_rev_not_change
skip compact when rev not change at period compact mode
2022-06-08 22:54:39 -04:00
Benjamin Wang
29b964df25
Merge pull request #14099 from Dirc/main
Update link to tuning page
2022-06-08 16:46:16 +08:00
Dirc
ba405f64c1
Update link to tuning page 2022-06-08 10:16:04 +02:00
Benjamin Wang
e1fd5b1c35
Merge pull request #14095 from ahrtr/fix_raftexample_test_error
Fix the DATA RACE issue of TestCloseProposerInflight
2022-06-08 05:19:12 +08:00
Eduardo Patrocinio
73766a92fb Update build file to run with an argument
The current Makefile doesn't allow the compilation of the tools directory.
This commit updates the Makefile and the build file to add an option
to build the tools.

To build the tools, you can run make build_tools.
2022-06-07 11:47:28 -04:00
Benjamin Wang
8ad87be7a6 fix the DATA RACE issue of TestCloseProposerInflight 2022-06-07 18:59:00 +08:00
ahrtr
dabf6978cc Fix potential deadlock between Revoke and (Grant or Checkpoint) 2022-06-07 18:45:43 +08:00
Marek Siarkowicz
50d93ebea1
Merge pull request #14088 from ahrtr/3.5_changelog_revoke_lease_race
Update 3.5 changelog to cover the two bug fixes related to leases revoke and checkpoint
2022-06-07 11:32:05 +02:00
Benjamin Wang
800d3ec3ca update 3.5 changelog to cover the two bug fix related to leases revoke and checkpoint 2022-06-04 14:12:57 +08:00
Benjamin Wang
08f4c340eb
Merge pull request #14078 from ahrtr/checkpoint_reset
Clean up the local variable cps so that it will not continue to schedule checkpoint
2022-05-31 07:56:53 +08:00
ahrtr
1d482bfc08 Move the local variable cps into the loop so that it will not continue to schedule checkpoints after it steps down the leader 2022-05-29 05:08:41 +08:00
Benjamin Wang
ce77d83ee6
Merge pull request #14067 from ahrtr/lease_revoke
Fix the race condition between goroutine and channel on the same leases to be revoked
2022-05-29 05:04:26 +08:00
Benjamin Wang
2d6aadec24
Merge pull request #14079 from ahrtr/split_lease
Move lease related code into a separate file lease.go
2022-05-29 05:02:59 +08:00