203 Commits

Author SHA1 Message Date
jingyih
94673a6ba4 mvcc/backend: check for nil boltOpenOptions
Check if boltOpenOptions is nil before use it.
2020-02-15 00:18:26 -08:00
Joe Betz
6d646c442a
mvcc/backend: Delete orphaned db.tmp files before defrag 2020-02-13 12:26:54 -08:00
宇慕
8c10973820 mvcc/kvstore:fixcompactbug 2019-11-27 13:07:47 -08:00
yoyinzyc
f4669c3b62 mvcc: update to "etcd_debugging_mvcc_total_put_size_in_bytes" 2019-11-26 14:03:07 -08:00
yoyinzyc
55c3476abc mvcc: add "etcd_mvcc_put_size_in_bytes" to monitor the throughput of put request. 2019-11-26 14:03:07 -08:00
yoyinzyc
480d5510f9 etcdserver: trace compaction request; add return parameter 'trace' to applierV3.Compaction() mvcc: trace compaction request; add input parameter 'trace' to KV.Compact() 2019-10-09 12:40:12 -07:00
yoyinzyc
daa432cfa7 etcdserver: add put request steps. mvcc: add put request steps; add trace to KV.Write() as input parameter. 2019-10-09 12:40:12 -07:00
yoyinzyc
8717327697 pkg: use zap logger to format the structure log output. 2019-10-09 12:40:12 -07:00
yoyinzyc
4f1bbff888 pkg: add field to record additional detail of trace; add stepThreshold to reduce log volume. 2019-10-09 12:40:12 -07:00
yoyinzyc
28bb8037d9 pkg: create package traceutil for tracing. mvcc: add tracing steps:range from the in-memory index tree; range from boltdb. etcdserver: add tracing steps: agreement among raft nodes before linerized reading; authentication; filter and sort kv pairs; assemble the response. 2019-10-09 12:40:12 -07:00
Jingyi Hu
ccbbb2f8d6 mvcc: add store revision metrics
Add experimental metrics etcd_debugging_mvcc_current_revision and
etcd_debugging_mvcc_compact_revision.
2019-09-06 17:03:21 -07:00
Gyuho Lee
0161e72d8d mvcc: keep 64-bit alignment in "store" struct
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-15 13:31:52 -07:00
Joe Betz
1e213b7ab6 *: Add experimental-compaction-batch-limit flag
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-15 13:31:39 -07:00
Joe Betz
b30c1eb2c8 mvcc: Optimize compaction for short commit pauses 2019-08-15 13:29:28 -07:00
Gyuho Lee
8c8f6f4b01 mvcc: fix typo in test
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-06 15:09:55 -07:00
Gyuho Lee
328fdc2150 mvcc: add TODOs
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-01 14:45:21 -07:00
Gyuho Lee
f82e23ab52 mvcc: add "etcd_mvcc_range_total", "etcd_mvcc_txn_total"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-01 14:44:55 -07:00
Gyuho Lee
dde3c5fc40 mvcc: clean up metrics names, add missing register calls
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-01 14:44:55 -07:00
Jingyi Hu
456c91b63a
Merge pull request #10959 from gyuho/adt
pkg/adt: refactor + add more test cases
2019-08-01 12:22:15 -07:00
Gyuho Lee
6a0811a949 *: use new adt.IntervalTree interface
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-31 22:23:13 -07:00
Sam Batschelet
f46ee91863 metrics: promote etcd_debugging_mvcc put_total and delete_total
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2019-08-01 01:28:07 +00:00
Xiang Li
2c5162af5c
Merge pull request #10523 from jingyih/fully_concurrent_reads
mvcc: fully concurrent read
2019-06-14 12:25:17 +08:00
Jingyi Hu
55066ebdc0 mvcc: address comments 2019-06-13 18:05:50 -07:00
Jingyi Hu
2a9320e944 mvcc: add TestConcurrentReadTxAndWrite
Add TestConcurrentReadTxAndWrite which creates random reads and writes,
and ensures reads always see latest writes.
2019-06-11 17:05:41 -07:00
Jingyi Hu
b873fbd127 mvcc/backend: correct RLock in test
Should use RLock instead of Lock.
2019-06-11 16:24:44 -07:00
Jingyi Hu
693afd8e5e mvcc/backend: add unit test for ConcurrentReadTx
Add TestConcurrentReadTx to ensure concurrentReadTx can see all the
prior writes which are stored on the read buffer.
2019-06-10 18:30:21 -07:00
Jingyi Hu
ad80752715 mvcc: add metrics dbOpenReadTxn
Expose the number of currently open read transactions in backend to
metrics endpoint.
2019-06-10 17:20:04 -07:00
Gyuho Lee
1caaa9ed4a test: test update for Go 1.12.5 and related changes
Update to Go 1.12.5 testing. Remove deprecated unused and gosimple
pacakges, and mask staticcheck 1006. Also, fix unconvert errors related
to unnecessary type conversions and following staticcheck errors:
- remove redundant return statements
- use for range instead of for select
- use time.Since instead of time.Now().Sub
- omit comparison to bool constant
- replace T.Fatal and T.Fatalf in tests with T.Error and T.Fatalf respectively because the goroutine calls T.Fatal must be called in the same goroutine as the test
- fix error strings that should not be capitalized
- use sort.Strings(...) instead of sort.Sort(sort.StringSlice(...))
- use he status code of Canceled instead of grpc.ErrClientConnClosing which is deprecated
- use use status.Errorf instead of grpc.Errorf which is deprecated

Related #10528 #10438
2019-06-05 17:02:05 -04:00
Jingyi Hu
4345f74426 mvcc: revert change made by 10526
Revert #10526 and its followup #10699.
2019-05-29 17:41:33 -07:00
Gyuho Lee
34bd797e67 *: revert module import paths
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-05-28 15:39:35 -07:00
Max Lowther
9ab3572662 Doc: Fix typo in revision.go 2019-05-16 14:29:10 +01:00
Jingyi Hu
a73fb85c0c mvcc: fully concurrent read 2019-05-08 19:11:23 -07:00
Gyuho Lee
e899023f3f
Merge pull request #10640 from shrajfr12/gomodulecompat
Fix module path to have the major version to comply with go modules specification.
2019-05-01 22:46:03 -07:00
Jingyi Hu
88922b0d08 mvcc: protect tree clone with write lock 2019-05-01 12:34:09 -07:00
shivaramr
9150bf52d6 go modules: Fix module path version to include version number 2019-04-26 15:29:50 -07:00
Yingnan Zhang
c5cb5509ea mvcc: fix db_compaction_total_duration_milliseconds 2019-04-16 10:32:21 +08:00
Jingyi Hu
93732df3ef mvcc: release lock early when traversing index
Make a copy-on-write clone of index tree when traversing. So that lock
can be released right after the clone to improve backend concurrency.
2019-03-06 14:26:17 -08:00
Jingyi Hu
1c19f126cb mvcc/backend: rename ReadTx Lock() to RLock()
For better code readability, renaming Lock() to RLock() in ReadTx
interface.
2019-03-05 13:53:27 -08:00
WizardCXY
e6c6d8492e *: add flag to let etcd use the new boltdb freelistType feature 2019-02-14 11:07:08 +08:00
WizardCXY
6e8913b004 bugfix:dead lock on store.mu when store.Compact in store.Restore happens 2019-01-21 10:46:58 +08:00
caoming
cf309757d6 mvcc/backend: code format optimization 2018-10-17 14:18:09 +08:00
caoming
bf49b9a145 mvcc/backend: fix to use the backend create by snapshot instead of origin one. 2018-10-15 09:35:20 +08:00
Gyuho Lee
5adbc231f2 mvcc/backend: use "go.etcd.io/bbolt"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2018-08-29 12:31:04 -07:00
Gyuho Lee
d537b328cb mvcc: update import paths "go.etcd.io/etcd"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2018-08-28 17:47:55 -07:00
Gyuho Lee
6ab3cc0a2e mvcc: clean up code format
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-07-21 16:03:16 -07:00
Gyuho Lee
e388a4a1a1 mvcc: simplify increment "rrev"
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-07-05 10:28:10 -07:00
Gyuho Lee
bc18474029 mvcc: remove unnecessary type conversion
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-07-05 10:09:53 -07:00
Xiang Li
2f1730fcae backend: more metrics for bboltdb transcation 2018-06-11 14:05:04 -07:00
Gyuho Lee
f2db05a869 mvcc: server db size with "etcd_debugging" namespace for backward compatibility
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-06-07 10:23:12 -07:00
Gyuho Lee
21130d5fb6 mvcc: promote db size metrics to "etcd"
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-06-07 10:20:45 -07:00