38 Commits

Author SHA1 Message Date
Jingyi Hu
55066ebdc0 mvcc: address comments 2019-06-13 18:05:50 -07:00
Jingyi Hu
ad80752715 mvcc: add metrics dbOpenReadTxn
Expose the number of currently open read transactions in backend to
metrics endpoint.
2019-06-10 17:20:04 -07:00
Jingyi Hu
a73fb85c0c mvcc: fully concurrent read 2019-05-08 19:11:23 -07:00
Jingyi Hu
1c19f126cb mvcc/backend: rename ReadTx Lock() to RLock()
For better code readability, renaming Lock() to RLock() in ReadTx
interface.
2019-03-05 13:53:27 -08:00
WizardCXY
e6c6d8492e *: add flag to let etcd use the new boltdb freelistType feature 2019-02-14 11:07:08 +08:00
caoming
cf309757d6 mvcc/backend: code format optimization 2018-10-17 14:18:09 +08:00
Gyuho Lee
5adbc231f2 mvcc/backend: use "go.etcd.io/bbolt"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2018-08-29 12:31:04 -07:00
Gyuho Lee
d537b328cb mvcc: update import paths "go.etcd.io/etcd"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2018-08-28 17:47:55 -07:00
Gyuho Lee
e6a113cdcd mvcc/backend: clean up histogram variables
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-05-23 14:03:28 -07:00
Gyuho Lee
d326b2933c mvcc/backend: add "etcd_disk_backend_defrag_duration_seconds"
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-05-23 13:09:42 -07:00
Gyuho Lee
03ef9745a9 mvcc: add more structured logging
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-05-04 13:15:51 -07:00
Gyuho Lee
3df30b9c7f mvcc: fix "unconvert" warnings
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-04-30 15:32:16 -07:00
Gyuho Lee
c00c6cb685 mvcc: support structured logger
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-04-16 17:36:00 -07:00
Gyuho Lee
6c40b2b5d4 mvcc/backend: defrag to block concurrent read requests while resetting tx
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-16 03:29:18 -04:00
Gyuho Lee
8a518b01c4 *: revert "internal/mvcc" change
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-02-26 17:11:40 -08:00
Gyuho Lee
80d15948bc *: move "mvcc" to "internal/mvcc"
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-01-26 11:14:41 -08:00
Iwasaki Yudai
0b1b82aff2 mvcc: check null before set FillPercent not to panic
Since CreateBucketIfNotExists() can return nil when it gets an error,
accessing FillPercent must be done after a nil check, not to cause
a panic.
2018-01-08 11:34:34 -08:00
Anthony Romano
8b872196d0 backend: cache buckets in read tx
Saves an alloc and about 10% of Range() time.
2017-08-21 02:16:55 -07:00
Gyu-Ho Lee
318e9c766f *: replace 'boltdb' import paths with 'coreos/bbolt'
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-07-05 14:32:13 -07:00
Anthony Romano
8516d8ccc5 backend: force initial mmap size to 0 for windows
boltdb on windows allocates a file with the full mmap size even if the
db is empty. Force the initial mmap size to 0 so there's no huge initial
db file on windows.

Fixes #7910
2017-05-12 14:34:07 -07:00
fanmin shi
8468b38631 backend: dynamically set snapshotWarningTimeout based on db size 2017-05-11 15:25:35 -07:00
fanmin shi
230106dd3c backend: add prometheus metric for large snapshot duration.
FIXES #7878
2017-05-05 17:27:33 -07:00
fanmin shi
f7f30f2361 backend: print snapshotting duration warning every 30s
FIXES #7870
2017-05-04 16:41:03 -07:00
Gyu-Ho Lee
161c7f6bdf Merge pull request #7579 from gyuho/fix-defrage
*: fix panic during defrag operation
2017-03-23 10:08:33 -07:00
Anthony Romano
7ef75e373a Merge pull request #7525 from heyitsanthony/big-backend
etcdserver, backend: configure mmap size based on quota
2017-03-23 10:06:00 -07:00
Gyu-Ho Lee
26abd25cd3 mvcc/backend: hold 'readTx.Lock' until completing bolt.Tx reset
Fix https://github.com/coreos/etcd/issues/7526.

When resetting `bolt.Tx` in `defrag` and `batchTxBuffered.commit`
operation, we do not hold `readTx` lock, so the inflight range
requests can trigger panic in `mvcc.Range` paths. This fixes by
moving mutexes out and hold it while resetting the `readTx`.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-03-23 09:47:43 -07:00
Xiang Li
7698a2a546 Merge pull request #7553 from xiang90/fix_defrag
backend: add FillPercent option
2017-03-21 11:16:17 -07:00
Xiang
95870a21eb backend: add FillPercent option 2017-03-21 08:06:03 -07:00
Anthony Romano
8a3fee15a3 etcdserver, backend: only warn if exceeding max quota 2017-03-17 15:38:57 -07:00
Anthony Romano
5e4b008106 *: base initial mmap size on quota size 2017-03-17 15:38:49 -07:00
Anthony Romano
2f1542c06d *: use filepath.Join for files 2017-03-16 07:46:06 -07:00
Anthony Romano
8d438c2939 backend: readtx
ReadTxs are designed for read-only accesses to the backend using a
read-only boltDB transaction. Since BatchTx's are long-running
transactions, all writes to BatchTx will writeback to ReadTx, overlaying
the base read-only transaction.
2017-03-08 20:52:59 -08:00
Xiang Li
962433c17f *: set repo correctly for logging 2016-10-03 17:03:22 +08:00
Xiang Li
ef9754910e mvcc: do not hash consistent index 2016-06-28 09:36:26 -07:00
Nikita Vetoshkin
dbc7c2cf4e backend: reuse timer in run().
Benchmarks:

```
import (
	"testing"
	"time"
)

func BenchmarkTimeAfter(b *testing.B) {
	b.ReportAllocs()
	for n := 0; n < b.N; n++ {
		select {
		case <- time.After(1 * time.Millisecond):
		}
	}
}

func BenchmarkTimerReset(b *testing.B) {
	b.ReportAllocs()
	t := time.NewTimer(1 * time.Millisecond)
	for n := 0; n < b.N; n++ {
		select {
		case <- t.C:
		}
		t.Reset(1 * time.Millisecond)
	}
}
```

Running reveals that each loop results in 3 allocs:

```
BenchmarkTimeAfter-4 	    2000	   1112134 ns/op	     192 B/op	       3 allocs/op
BenchmarkTimerReset-4	    2000	   1109774 ns/op	       0 B/op	       0 allocs/op
```
2016-06-23 18:49:41 +05:00
Gyu-Ho Lee
bf8cf39daf mvcc: use capnslog 2016-05-20 22:31:22 -07:00
Gyu-Ho Lee
9d9f02c1ee mvcc: update LICENSE header 2016-05-12 20:50:33 -07:00
Anthony Romano
b7ac758969 *: rename storage package to mvcc 2016-04-25 15:25:51 -07:00