Gyu-Ho Lee
78b0b8a4a0
storage: range all unsynced at once
...
This is for https://github.com/coreos/etcd/issues/3848 .
It replaces RangeHistory method for more efficient event
sending.
2015-12-28 11:37:26 -08:00
Xiang Li
9f908ce67f
storage: add missing return for unsafeAddWatching
2015-12-01 14:55:20 -08:00
Yicheng Qin
deb1da5f57
storage: add watch ID to identify watchings
...
One watcher includes multiple watchings, and their events are
sent out through one channel. For the received event, user would like to
know which watching it belongs to.
Introduce a watch ID. When watching on some key, user will get a watch
ID. The watch ID is attached to all events that is observed by this
watch.
2015-11-21 11:19:17 -08:00
Gyu-Ho Lee
48aebd9b09
storage: use map for watchableStore synced
...
This is for coreos#3859 switching slice to map for synced watchings.
For a large amount of synced watchings, map implementation performs better.
When putting 1 million watchers on the same key and canceling them one by
one: original implementation takes 9m7.268221091s, while the one with map
takes only 430.531637ms.
2015-11-21 00:42:09 -08:00
Yicheng Qin
2f74f76025
storage: remove the event concept from key-value layer
...
The point is to decouple the key-value storage layer and the
event notification layer clearly. It gives the watchableKV the
flexibility to define whatever event structure it wants without
breaking the ondisk format at key-value storage layer.
Changes:
1. change the format of key and value stored in backend
Store KeyValue struct instead of Event struct in backend value for
better abstraction as xiang suggests. And record the corresponded
action in the backend key.
2. Remove word 'event' from functions
2015-11-17 20:35:49 -08:00
Yicheng Qin
dadfdf6af8
Merge pull request #3802 from yichengq/fix-storage-watch
...
storage: delete key instead of setting it to false
2015-11-05 11:40:46 -08:00
Gyu-Ho Lee
6e5eb03544
storage: move watcherGauge to watchable_store
...
watcherGauge should be increased everytime we creates Watcher, not per watch
method call.
2015-11-04 13:17:47 -08:00
Yicheng Qin
c8e622f517
storage: make putm/delm a set with empty value
...
This cleans the code, and reduces the allocation space.
2015-11-03 19:10:45 -08:00
Yicheng Qin
6dbfc21846
storage: delete key instead of setting it to false
...
When getting the watched events, it iterate all keys in putm and delm
to generate the events. If we don't delete the key from putm/delm,
it would range on the key that is not actually put or deleted. This is
incorrect.
Fix the panic that happens when single put/delete is watched.
2015-11-03 19:00:39 -08:00
Gyu-Ho Lee
bdc280c4a7
storage: apply same naming in metrics.go
...
This is PR following up with Xiang's https://github.com/coreos/etcd/pull/3795 ,
and to make the naming consistent with its interface change.
2015-11-03 15:19:18 -08:00
Xiang Li
a1129dd5a5
storage: support multiple watching per watcher
...
We want to support multiple watchings per one watcher chan. Then
we can have one single go routine to watch multiple keys/prefixs.
2015-11-03 12:36:11 -08:00
Gyu-Ho Lee
f73d0ed1d9
storage: use map for watchable store unsynced
...
This is for `TODO: use map to reduce cancel cost`.
I switched slice to map, and benchmark results show
that map implementation performs better, as follows:
```
[1]:
benchmark old ns/op new ns/op delta
BenchmarkWatchableStoreUnsyncedCancel 215212 1307 -99.39%
BenchmarkWatchableStoreUnsyncedCancel-2 120453 710 -99.41%
BenchmarkWatchableStoreUnsyncedCancel-4 120765 748 -99.38%
BenchmarkWatchableStoreUnsyncedCancel-8 121391 719 -99.41%
benchmark old allocs new allocs delta
BenchmarkWatchableStoreUnsyncedCancel 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8 0 0 +0.00%
benchmark old bytes new bytes delta
BenchmarkWatchableStoreUnsyncedCancel 200 1 -99.50%
BenchmarkWatchableStoreUnsyncedCancel-2 138 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4 138 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8 139 0 -100.00%
[2]:
benchmark old ns/op new ns/op delta
BenchmarkWatchableStoreUnsyncedCancel 212550 1117 -99.47%
BenchmarkWatchableStoreUnsyncedCancel-2 120927 691 -99.43%
BenchmarkWatchableStoreUnsyncedCancel-4 120752 699 -99.42%
BenchmarkWatchableStoreUnsyncedCancel-8 121012 688 -99.43%
benchmark old allocs new allocs delta
BenchmarkWatchableStoreUnsyncedCancel 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8 0 0 +0.00%
benchmark old bytes new bytes delta
BenchmarkWatchableStoreUnsyncedCancel 197 1 -99.49%
BenchmarkWatchableStoreUnsyncedCancel-2 138 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4 138 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8 139 0 -100.00%
[3]:
benchmark old ns/op new ns/op delta
BenchmarkWatchableStoreUnsyncedCancel 214268 1183 -99.45%
BenchmarkWatchableStoreUnsyncedCancel-2 120763 759 -99.37%
BenchmarkWatchableStoreUnsyncedCancel-4 120321 708 -99.41%
BenchmarkWatchableStoreUnsyncedCancel-8 121628 680 -99.44%
benchmark old allocs new allocs delta
BenchmarkWatchableStoreUnsyncedCancel 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8 0 0 +0.00%
benchmark old bytes new bytes delta
BenchmarkWatchableStoreUnsyncedCancel 200 1 -99.50%
BenchmarkWatchableStoreUnsyncedCancel-2 139 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4 138 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8 139 0 -100.00%
[4]:
benchmark old ns/op new ns/op delta
BenchmarkWatchableStoreUnsyncedCancel 208332 1089 -99.48%
BenchmarkWatchableStoreUnsyncedCancel-2 121011 691 -99.43%
BenchmarkWatchableStoreUnsyncedCancel-4 120678 681 -99.44%
BenchmarkWatchableStoreUnsyncedCancel-8 121303 721 -99.41%
benchmark old allocs new allocs delta
BenchmarkWatchableStoreUnsyncedCancel 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8 0 0 +0.00%
benchmark old bytes new bytes delta
BenchmarkWatchableStoreUnsyncedCancel 194 1 -99.48%
BenchmarkWatchableStoreUnsyncedCancel-2 139 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4 139 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8 139 0 -100.00%
[5]:
benchmark old ns/op new ns/op delta
BenchmarkWatchableStoreUnsyncedCancel 211900 1097 -99.48%
BenchmarkWatchableStoreUnsyncedCancel-2 121795 753 -99.38%
BenchmarkWatchableStoreUnsyncedCancel-4 123182 700 -99.43%
BenchmarkWatchableStoreUnsyncedCancel-8 122820 688 -99.44%
benchmark old allocs new allocs delta
BenchmarkWatchableStoreUnsyncedCancel 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8 0 0 +0.00%
benchmark old bytes new bytes delta
BenchmarkWatchableStoreUnsyncedCancel 198 1 -99.49%
BenchmarkWatchableStoreUnsyncedCancel-2 140 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4 141 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8 141 0 -100.00%
```
2015-10-21 15:30:15 -07:00
Yicheng Qin
0f7374ce89
storage: KV field -> store field in watchableStore
...
We need to access the underlying store to use its RangeEvents function.
It is not good to use unnecessary type conversion.
The underlying store is also needed for further store upon
watchableStore.
2015-10-20 19:23:20 -07:00
Xiang Li
6556bf1643
storage: remove the endRev of watcher
2015-10-17 15:59:49 -07:00
Xiang Li
09157d4f1a
storage: add metrics for watchers
2015-10-05 11:32:56 -07:00
Gyu-Ho Lee
4113509828
storage/watchable_store: defer to Unlock s.mu
...
New PR from https://github.com/coreos/etcd/pull/3575 .
This add `defer` to `s.mu`. Current code does not `Unlock`
in the correct scope, I think.
(Sorry, I accidentally deleted my fork so the changes
might not sound continuous from my previous pull requests.)
2015-09-22 23:25:07 -07:00
Jonathan Boulle
7848ac3979
*: add missing license headers
2015-09-15 14:09:01 -07:00
Yicheng Qin
ec43e0a4c3
storage: introduce WatchableKV and watch feature
...
WatchableKV is an interface upon KV, and supports watch feature.
2015-09-14 23:53:03 -07:00
