Mirroristas/etcd
2
0
Fork 0
mirror of https://github.com/etcd-io/etcd.git synced 2024-09-27 06:25:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
21,066 Commits 15 Branches 491 Tags 148 MiB
Commit Graph

222 Commits

Author SHA1 Message Date
Benjamin Wang
cc32fa8559
Merge pull request #17618 from Akiqqqqqqq/feature-wal-write-metric 
feat: add wal write system call metrics observation (main)
 2024-03-31 16:54:20 +01:00
Marek Siarkowicz
a25497e97d
Merge pull request #17660 from ivanvc/address-server-storage-var-naming-lint-rule 
server/storage: address golangci var-naming issues
 2024-03-29 10:02:25 +01:00
Ivan Valdes
d43738a49b
server/storage: address golangci var-naming issues 
Signed-off-by: Ivan Valdes <ivan@vald.es>
 2024-03-27 10:12:42 -07:00
Siyuan Zhang
0a54362cca add key dedupe when a write buffer writeback to an empty read buffer bucket. 
Signed-off-by: Siyuan Zhang <sizhang@google.com>
 2024-03-27 17:03:31 +00:00
Siyuan Zhang
7be3606841 Add tx buffer test case of unordered key writes. 
Signed-off-by: Siyuan Zhang <sizhang@google.com>
 2024-03-27 17:03:31 +00:00
Siyuan Zhang
4346a43cee Add benchmark tests for buffer writeback function. 
Signed-off-by: Siyuan Zhang <sizhang@google.com>
 2024-03-27 17:03:26 +00:00
Qiuyu Wu
97efc2ade4 feat: observe wal write at one time 
Signed-off-by: Qiuyu Wu <qiuyu.wu@shopee.com>
 2024-03-26 17:26:02 +08:00
thirdkeyword
fbda591866 fix some typos 
Signed-off-by: thirdkeyword <fliterdashen@gmail.com>
 2024-03-25 10:34:44 +08:00
Ivan Valdes
66f56d71e4
server: address golangci var-naming issues 
Signed-off-by: Ivan Valdes <ivan@vald.es>
 2024-03-20 21:12:12 -07:00
Chao Chen
405862e807 Fix event loss after compaction 
Signed-off-by: Chao Chen <chaochn@amazon.com>
 2024-03-15 14:22:37 -07:00
Abhishek Kr Srivastav
fe113026fa Increased timeout to fix HashKV test 
Signed-off-by: Abhishek Kr Srivastav <Abhishek.kr.srivastav@ibm.com>
 2024-02-23 15:58:57 +05:30
Siyuan Zhang
3565a822de Add VerifyTxConsistency to backend. 
Signed-off-by: Siyuan Zhang <sizhang@google.com>

Update server/storage/backend/verify.go

Co-authored-by: Benjamin Wang <benjamin.wang@broadcom.com>

Update server/storage/backend/verify.go

Co-authored-by: Benjamin Wang <benjamin.wang@broadcom.com>
 2024-02-22 11:31:16 -08:00
Ivan Valdes
be28833212
etcdutl: Fix snapshot restore memory alloc issue 
When running the snapshot command, allow receiving an initial memory map
allocation for the database, avoiding future memory allocation issues.

Co-authored-by: Benjamin Wang <benjamin.wang@broadcom.com>
Co-authored-by: Fatih USTA <fatihusta86@gmail.com>
Signed-off-by: Ivan Valdes <ivan@vald.es>
 2024-02-13 13:36:13 -08:00
Ivan Valdes
d69adf45f9
server: Implement WithMmapSize option for backend config 
Accept a third argument for NewDefaultBackend for overrides to the
BackendConfig.
Add a new function, WithMmapSize, which modifies the backend config to
provide a custom InitiamMmapSize.

Signed-off-by: Ivan Valdes <ivan@vald.es>
 2024-02-13 13:24:44 -08:00
Benjamin Wang
c12e15d91a Add verification on keys: should be always mononically increasing 
Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
 2024-01-25 18:00:26 +00:00
Benjamin Wang
c647614b6a test: fix TestHashKVWhenCompacting: ensure all goroutine finished 
Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
 2024-01-25 13:30:27 +00:00
Benjamin Wang
dd0b0526e1
Merge pull request #17284 from ahrtr/test_TestScheduleCompaction_20240119 
Fix test issue in TestScheduleCompaction
 2024-01-20 13:29:04 +00:00
Siyuan Zhang
30b242f800 Revert "add test cases of duplicate keys in one tx." 
This reverts commit 8d29b245a39172bd72643106ea7d9e4df727bd7e.

Signed-off-by: Siyuan Zhang <sizhang@google.com>
 2024-01-19 08:58:48 -08:00
Siyuan Zhang
93afd8e430 Revert "add key dedupe when a write buffer writeback to an empty read buffer bucket." 
This reverts commit 971a4f57ccff71dfc10c1f9eb7a6a4d64bec1d2e.

Signed-off-by: Siyuan Zhang <sizhang@google.com>
 2024-01-19 08:58:27 -08:00
Benjamin Wang
82562abc3c Fix test issue in TestScheduleCompaction 
It always reuse the same memory space. Each next iteration
will always overwrite previous data

Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
 2024-01-19 15:29:51 +00:00
Benjamin Wang
690916f018 replace CreateBucket with CreateBucketIfNotExists 
Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
 2024-01-16 09:40:20 +00:00
Benjamin Wang
4bc55e5208
Merge pull request #17204 from rahulbapumore/branch1 
mvcc: Printing etcd backend database related metrics inside scheduleCompaction function
 2024-01-15 15:37:08 +00:00
Rahul More
21bbc82710 mvcc: Printing etcd backend database related metrics inside 
scheduleCompaction function

To improve traceability of backend database usage, Added below parameter
related to backend database usage metrics inside scheduledCompaction
function.
current-db-size-bytes
current-db-size
current-db-size-in-use-bytes
current-db-size-in-use

Signed-off-by: Rahul More <rahulbapumore@gmail.com>
 2024-01-15 17:36:38 +05:30
Benjamin Wang
9ded10ec76
Merge pull request #17237 from ahrtr/bbolt_logger_20240112 
Specify logger for bbolt
 2024-01-13 11:03:14 +00:00
Benjamin Wang
cfbf672b3c specify logger for bbolt 
Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
 2024-01-12 16:38:18 +00:00
Siyuan Zhang
971a4f57cc add key dedupe when a write buffer writeback to an empty read buffer bucket. 
Signed-off-by: Siyuan Zhang <sizhang@google.com>
 2024-01-11 11:31:09 -08:00
Siyuan Zhang
8d29b245a3 add test cases of duplicate keys in one tx. 
Signed-off-by: Siyuan Zhang <sizhang@google.com>
 2024-01-11 11:31:03 -08:00
Benjamin Wang
b3bf59a355
Merge pull request #17195 from siyuanfoundation/txBuf1 
Fix delete inconsistencies in read buffer
 2024-01-11 08:21:47 +00:00
Siyuan Zhang
db61c9695d commit bbolt transaction if there is any pending deleting operations 
Signed-off-by: Siyuan Zhang <sizhang@google.com>
 2024-01-10 09:33:21 -08:00
Siyuan Zhang
b3a5307640 add tests to test tx delete consistency. 
Signed-off-by: Siyuan Zhang <sizhang@google.com>
 2024-01-10 09:33:13 -08:00
Marek Siarkowicz
d39d86a214 Improve logs around recovering from snapshot backend 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-12-20 16:26:27 +01:00
Allen Ray
395376d3ab Refactor common revision code to pkg 
Signed-off-by: Allen Ray <alray@redhat.com>
 2023-10-09 14:53:16 -04:00
Wei Fu
4704a5af3a *: fix unused issue 
Signed-off-by: Wei Fu <fuweid89@gmail.com>
 2023-09-25 19:37:18 +08:00
Wei Fu
aa97484166 *: enable goimports in verify-lint 
Signed-off-by: Wei Fu <fuweid89@gmail.com>
 2023-09-21 21:14:09 +08:00
Wei Fu
5e3910d96c *: fix govet-shadow lint 
Signed-off-by: Wei Fu <fuweid89@gmail.com>
 2023-09-19 20:24:01 +08:00
Benjamin Wang
4ec581a6a3 re-generate all the *.pb.gw.go files 
Run ./scripts/genproto.sh

Signed-off-by: Benjamin Wang <wachao@vmware.com>
Signed-off-by: Wei Fu <fuweid89@gmail.com>
 2023-09-18 11:22:15 +01:00
Wei Fu
522b698928 upgrade grpc-gateway from v1 to v2 
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2023-09-18 11:22:15 +01:00
Benjamin Wang
700411d838
Merge pull request #16601 from fuweid/fix-nakedret-lint 
*: fix nakedret lint
 2023-09-18 10:00:25 +01:00
Wei Fu
e72c2c40d4 *: fix nakedret lint 
Signed-off-by: Wei Fu <fuweid89@gmail.com>
 2023-09-17 23:52:41 +08:00
chenyahui
c0aa3b613b Use any instead of interface{} 
Signed-off-by: chenyahui <cyhone@qq.com>
 2023-09-17 17:41:58 +08:00
redwrasse
b79cbcc3db mvcc: cleanup comment in HashStorage.Store 
Signed-off-by: redwrasse <mail@redwrasse.io>
 2023-09-11 21:54:42 -07:00
Benjamin Wang
a13fc7e51c
Merge pull request #16508 from new-dream/main-concurrentreadtxn 
server: optimizing memory overhead of copy operation in ConcurrentReadTxn
 2023-09-09 09:35:00 +01:00
Wei Fu
a6f7bbe560 server/storage/mvcc: deflake TestHashKVWhenCompacting 
The HashByRev-goroutines exit since receive `donec` notification. The
Check-computed-hashes goroutine could not have chance to get the hash
result and be stuck forever. We should add case for donec when we wait
for hash result.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
 2023-09-06 18:12:42 +08:00
new-dream
b05d75ab0b server: optimizing memory overhead of copy operation in ConcurrentReadTxn 
Signed-off-by: new-dream <111836360+new-dream@users.noreply.github.com>
 2023-08-30 23:03:15 +08:00
James Blair
cb0df72b70
Use crypto/rand.Read instead of deprecated math/rand.Read. 
Signed-off-by: James Blair <mail@jamesblair.net>
 2023-08-22 21:48:27 +12:00
Marek Siarkowicz
53cbd81009 Separate Writer interface from BatchTx interfaces 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-07-31 10:18:01 +02:00
Marek Siarkowicz
29769984e6 Remove RLock/RUnlock from BatchTx 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-07-28 11:39:50 +02:00
Marek Siarkowicz
b4f8a7be51 server: Remove Lock/Unlock from ReadTx 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2023-07-27 13:34:43 +02:00
caojiamingalan
bc97a94564 Follow up https://github.com/etcd-io/etcd/pull/16068#discussion_r1263664700 
Replace unnecessary Lock()/Unlock()s with RLock()/RUnlock()s

Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com>
 2023-07-14 20:08:25 -05:00
caojiamingalan
b9e30bf878 etcdserver: add e2e test to reproduce the incorrect hash issue when resuming scheduled compaction. 
check ScheduledCompactKeyName and FinishedCompactKeyName
before writing hash to hashstore. If they do not match, then it means this compaction has once been interrupted and its hash value is invalid. In such cases, we won't write the hash values to the hashstore, and avoids the incorrect corruption alarm.

Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com>
 2023-06-07 19:54:09 -05:00