233 Commits

Author SHA1 Message Date
Benjamin Wang
09f7d7ab92
Merge pull request #17875 from ivanvc/fix-deprecation-comments-added-with-var-naming-lint-rule
Fix deprecation comments from addressing var-naming work
2024-04-30 19:51:37 +01:00
Jes Cok
dafadd13c1 all: don't convert byte slice to string when using verb %s
This is unnecessary, as the documentation for 'go doc fmt' says:
%s the uninterpreted bytes of the string or slice

Signed-off-by: Jes Cok <xigua67damn@gmail.com>
2024-04-25 23:34:52 +08:00
Ivan Valdes
242f5d60a8
Fix deprecation comments from addressing var-naming work
The deprecation tag/comment must to be followed by an empty line.

Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-04-24 14:40:59 -07:00
Wei Fu
94c83a962b server/storage/mvcc: should update currentRev in revMu
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-04-24 11:08:33 +08:00
Ivan Valdes
a2bf8d7e80
server/config: address golangci var-naming issues
Addresses issues in V2 Deprecation constant names.

Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-04-22 17:12:15 -07:00
Benjamin Wang
dd4e35a585
Merge pull request #17815 from fuweid/repro-17780
[RFC] fix revision loss issue caused by compaction - 17780
2024-04-22 18:15:44 +01:00
Benjamin Wang
29529c505b
Merge pull request #17791 from ahrtr/verify_revision_bootstrap_20240415
Add verification that the revision shouldn't decrease on bootstrap
2024-04-22 10:43:01 +01:00
Wei Fu
9ea234913a server/storage: update currentRev if scheduledCompact > currentRev
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-04-21 18:12:38 +08:00
Benjamin Wang
a54098dee3 Update the compaction log when bootstrap and update compact's signature
Actually the compact() never return an error, so remove the second return
parameter.

Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
2024-04-15 14:16:18 +01:00
Benjamin Wang
9ac964d1cc Add verification that the revision shouldn't decrease on bootstrap
Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
2024-04-15 13:56:19 +01:00
idnandre
12fa1bb940 server/storage: change deprecated bolt error to new error package
Signed-off-by: idnandre <andre@idntimes.com>
2024-04-03 19:21:22 +07:00
Benjamin Wang
cc32fa8559
Merge pull request #17618 from Akiqqqqqqq/feature-wal-write-metric
feat: add wal write system call metrics observation (main)
2024-03-31 16:54:20 +01:00
Marek Siarkowicz
a25497e97d
Merge pull request #17660 from ivanvc/address-server-storage-var-naming-lint-rule
server/storage: address golangci var-naming issues
2024-03-29 10:02:25 +01:00
Ivan Valdes
d43738a49b
server/storage: address golangci var-naming issues
Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-03-27 10:12:42 -07:00
Siyuan Zhang
0a54362cca add key dedupe when a write buffer writeback to an empty read buffer bucket.
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-03-27 17:03:31 +00:00
Siyuan Zhang
7be3606841 Add tx buffer test case of unordered key writes.
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-03-27 17:03:31 +00:00
Siyuan Zhang
4346a43cee Add benchmark tests for buffer writeback function.
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-03-27 17:03:26 +00:00
Qiuyu Wu
97efc2ade4 feat: observe wal write at one time
Signed-off-by: Qiuyu Wu <qiuyu.wu@shopee.com>
2024-03-26 17:26:02 +08:00
thirdkeyword
fbda591866 fix some typos
Signed-off-by: thirdkeyword <fliterdashen@gmail.com>
2024-03-25 10:34:44 +08:00
Ivan Valdes
66f56d71e4
server: address golangci var-naming issues
Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-03-20 21:12:12 -07:00
Chao Chen
405862e807 Fix event loss after compaction
Signed-off-by: Chao Chen <chaochn@amazon.com>
2024-03-15 14:22:37 -07:00
Abhishek Kr Srivastav
fe113026fa Increased timeout to fix HashKV test
Signed-off-by: Abhishek Kr Srivastav <Abhishek.kr.srivastav@ibm.com>
2024-02-23 15:58:57 +05:30
Siyuan Zhang
3565a822de Add VerifyTxConsistency to backend.
Signed-off-by: Siyuan Zhang <sizhang@google.com>

Update server/storage/backend/verify.go

Co-authored-by: Benjamin Wang <benjamin.wang@broadcom.com>

Update server/storage/backend/verify.go

Co-authored-by: Benjamin Wang <benjamin.wang@broadcom.com>
2024-02-22 11:31:16 -08:00
Ivan Valdes
be28833212
etcdutl: Fix snapshot restore memory alloc issue
When running the snapshot command, allow receiving an initial memory map
allocation for the database, avoiding future memory allocation issues.

Co-authored-by: Benjamin Wang <benjamin.wang@broadcom.com>
Co-authored-by: Fatih USTA <fatihusta86@gmail.com>
Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-02-13 13:36:13 -08:00
Ivan Valdes
d69adf45f9
server: Implement WithMmapSize option for backend config
Accept a third argument for NewDefaultBackend for overrides to the
BackendConfig.
Add a new function, WithMmapSize, which modifies the backend config to
provide a custom InitiamMmapSize.

Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-02-13 13:24:44 -08:00
Benjamin Wang
c12e15d91a Add verification on keys: should be always mononically increasing
Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
2024-01-25 18:00:26 +00:00
Benjamin Wang
c647614b6a test: fix TestHashKVWhenCompacting: ensure all goroutine finished
Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
2024-01-25 13:30:27 +00:00
Benjamin Wang
dd0b0526e1
Merge pull request #17284 from ahrtr/test_TestScheduleCompaction_20240119
Fix test issue in TestScheduleCompaction
2024-01-20 13:29:04 +00:00
Siyuan Zhang
30b242f800 Revert "add test cases of duplicate keys in one tx."
This reverts commit 8d29b245a39172bd72643106ea7d9e4df727bd7e.

Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-01-19 08:58:48 -08:00
Siyuan Zhang
93afd8e430 Revert "add key dedupe when a write buffer writeback to an empty read buffer bucket."
This reverts commit 971a4f57ccff71dfc10c1f9eb7a6a4d64bec1d2e.

Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-01-19 08:58:27 -08:00
Benjamin Wang
82562abc3c Fix test issue in TestScheduleCompaction
It always reuse the same memory space. Each next iteration
will always overwrite previous data

Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
2024-01-19 15:29:51 +00:00
Benjamin Wang
690916f018 replace CreateBucket with CreateBucketIfNotExists
Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
2024-01-16 09:40:20 +00:00
Benjamin Wang
4bc55e5208
Merge pull request #17204 from rahulbapumore/branch1
mvcc: Printing etcd backend database related metrics inside scheduleCompaction function
2024-01-15 15:37:08 +00:00
Rahul More
21bbc82710 mvcc: Printing etcd backend database related metrics inside
scheduleCompaction function

To improve traceability of backend database usage, Added below parameter
related to backend database usage metrics inside scheduledCompaction
function.
current-db-size-bytes
current-db-size
current-db-size-in-use-bytes
current-db-size-in-use

Signed-off-by: Rahul More <rahulbapumore@gmail.com>
2024-01-15 17:36:38 +05:30
Benjamin Wang
9ded10ec76
Merge pull request #17237 from ahrtr/bbolt_logger_20240112
Specify logger for bbolt
2024-01-13 11:03:14 +00:00
Benjamin Wang
cfbf672b3c specify logger for bbolt
Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
2024-01-12 16:38:18 +00:00
Siyuan Zhang
971a4f57cc add key dedupe when a write buffer writeback to an empty read buffer bucket.
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-01-11 11:31:09 -08:00
Siyuan Zhang
8d29b245a3 add test cases of duplicate keys in one tx.
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-01-11 11:31:03 -08:00
Benjamin Wang
b3bf59a355
Merge pull request #17195 from siyuanfoundation/txBuf1
Fix delete inconsistencies in read buffer
2024-01-11 08:21:47 +00:00
Siyuan Zhang
db61c9695d commit bbolt transaction if there is any pending deleting operations
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-01-10 09:33:21 -08:00
Siyuan Zhang
b3a5307640 add tests to test tx delete consistency.
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-01-10 09:33:13 -08:00
Marek Siarkowicz
d39d86a214 Improve logs around recovering from snapshot backend
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-12-20 16:26:27 +01:00
Allen Ray
395376d3ab Refactor common revision code to pkg
Signed-off-by: Allen Ray <alray@redhat.com>
2023-10-09 14:53:16 -04:00
Wei Fu
4704a5af3a *: fix unused issue
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-25 19:37:18 +08:00
Wei Fu
aa97484166 *: enable goimports in verify-lint
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-21 21:14:09 +08:00
Wei Fu
5e3910d96c *: fix govet-shadow lint
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-19 20:24:01 +08:00
Benjamin Wang
4ec581a6a3 re-generate all the *.pb.gw.go files
Run ./scripts/genproto.sh

Signed-off-by: Benjamin Wang <wachao@vmware.com>
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-18 11:22:15 +01:00
Wei Fu
522b698928 upgrade grpc-gateway from v1 to v2
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-09-18 11:22:15 +01:00
Benjamin Wang
700411d838
Merge pull request #16601 from fuweid/fix-nakedret-lint
*: fix nakedret lint
2023-09-18 10:00:25 +01:00
Wei Fu
e72c2c40d4 *: fix nakedret lint
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-17 23:52:41 +08:00