Xiang Li
9143329c85
storage: implement requestProgress
2016-03-03 09:39:29 -08:00
Anthony Romano
c0eac7ab72
storage: support watch on ranges
2016-02-29 15:20:41 -08:00
Anthony Romano
7c17665a1a
storage: limit total unique revisions in unsynced watcher event list
2016-02-18 11:36:22 -08:00
Anthony Romano
af225e7433
v3rpc: don't race on current watcher header revision
2016-02-17 14:03:52 -08:00
Anthony Romano
2cbf7cf6d1
storage: do not send outdated events to unsynced watchers
2016-02-17 14:03:51 -08:00
Anthony Romano
ee1a03167d
storage, v3: pass compaction revision through watchresponse
2016-02-12 14:06:59 -08:00
Xiang Li
b09214df32
storage: update watch.cur and fix tests
2016-02-03 00:54:07 -08:00
Xiang Li
52416fafb0
storage: send compaction
2016-02-02 23:17:27 -08:00
Xiang Li
5780497e18
storage: remove unncessary handle func
2016-02-02 23:02:15 -08:00
Xiang Li
ae5161382b
storage: release tx lock until finish using the readonly bytes
...
The backend will return read only bytes that are only vaild while
the tx is open. We should hold the lock until we get a full copy
by unmarshal.
2016-02-02 22:49:05 -08:00
Xiang Li
31c0c5181a
storage: make unsync a watcherSetByKey
2016-02-02 20:09:35 -08:00
Xiang Li
8dc6248aa7
storage: add set delete
2016-02-02 19:28:42 -08:00
Xiang Li
810c3e74a8
storage: remove unnecessary abstraction
2016-02-02 19:15:46 -08:00
Xiang Li
e5b35b82c5
storage: add watchSet and watchSetByKey type
2016-02-02 18:56:36 -08:00
Anthony Romano
20461ab11a
*: fix many typos
2016-01-31 21:42:39 -08:00
Xiang Li
611751aee2
storage: simplify watch store
...
We decided that we will not support modifing the same
key in one txn multiple times. That can simlify the current
code/design a lot.
2016-01-31 16:33:07 -08:00
Anthony Romano
5a967eb2a0
storage: publish delete events on lease revocation
2016-01-22 13:40:55 -08:00
Gyu-Ho Lee
4f427bca43
storage: check prefix in unsynced
...
Current syncWatchers method skips the events that have
prefixes that are being watched when the prefix is not
existent as a key. This fixes https://github.com/coreos/etcd/issues/4191
by adding prefix checking to not skip those events.
2016-01-13 11:21:47 -08:00
Xiang Li
f5753f2f51
*: support lease Attach
...
Now we can attach keys to leases. And revoking the lease removes all
the attached keys of that lease.
2016-01-09 11:01:58 -08:00
Gyu-Ho Lee
f76166a041
*: fix minor typos
2016-01-08 00:21:19 -08:00
Gyu-Ho Lee
366e7a879f
*: fill in WatchResponse.Header
...
Related to coreos#3848.
2016-01-06 15:12:53 -08:00
Xiang Li
5dd3f91903
*: make backend outside kv
...
KV and lease will share the same backend. Thus we need to make
backend outside KV.
2016-01-05 19:55:29 -08:00
Xiang Li
1e61243fd7
Merge pull request #4134 from xiang90/lease
...
lease: modify API and persist lease to disk
2016-01-05 10:30:52 -08:00
Xiang Li
09b420f08c
*: move leaseID typedef to lease pkg
2016-01-05 10:18:17 -08:00
Gyu-Ho Lee
7339abc79e
storage: fix watchable_store notify to hash-lookup once
...
We should just assign events and ok at first, and check the
boolean value, instead of looking up the map twice.
2016-01-04 22:34:48 -08:00
Anthony Romano
21a6ade53d
storage: change type of WatchIDs from int64 to WatchID
2016-01-04 19:52:22 -08:00
Gyu-Ho Lee
556d4a6932
*: remove CancelFunc return for Watch. Use Cancel for watch.
2016-01-04 16:17:55 -08:00
Xiang Li
4336278b44
*: support put with lease
2016-01-04 15:54:06 -08:00
Gyu-Ho Lee
6540f47dfa
*: WatchResponse for multiple Events with WatchID
...
storage/storagepb: remove watchID from Event
storage: add WatchResponse to watcher.go to wrap events, watchID
storage: watchableStore to use WatchResponse
storage: kv_test with WatchResponse
etcdserver/api/v3rpc: watch to receive storage.WatchResponse type
2016-01-03 16:32:48 -08:00
Xiang Li
eda0eefc25
*: support watcher cancellation inside watchStream
2016-01-03 00:20:21 -08:00
Xiang Li
807db7e2aa
storage: rename watching -> watcher
2016-01-02 20:20:22 -08:00
Xiang Li
ee0b3f42ed
*: rename watcher to watchStream
...
Watcher vs Watching in storage pkg is confusing. Watcher should be named
as watchStream since it contains a channel as stream to send out events.
Then we can rename watching to watcher, which actually watches on a key
and send watched events through watchStream.
This commits renames watcher to watchStram.
2016-01-02 16:03:57 -08:00
Gyu-Ho Lee
7dd599b69d
Merge pull request #4091 from gyuho/watch_events_slice
...
storage: watch events in slice
2015-12-31 23:54:08 -08:00
Gyu-Ho Lee
0b01acf131
storage: watchable_store.go events slice
2015-12-31 23:44:02 -08:00
Hitoshi Mitake
0b2d31f3bc
storage: decouple default parameters and storage creation
...
newStore() uses constants for some important parameters
e.g. batchInerval. It is not suitable for other storage users
(e.g. tools/benchmark). This commit decouples the default parameters
from storage creation. etcd should use a newly added function
newDefaultStore() for creating a store with default parameters.
2015-12-31 22:28:59 +09:00
Gyu-Ho Lee
78b0b8a4a0
storage: range all unsynced at once
...
This is for https://github.com/coreos/etcd/issues/3848 .
It replaces RangeHistory method for more efficient event
sending.
2015-12-28 11:37:26 -08:00
Xiang Li
9f908ce67f
storage: add missing return for unsafeAddWatching
2015-12-01 14:55:20 -08:00
Yicheng Qin
deb1da5f57
storage: add watch ID to identify watchings
...
One watcher includes multiple watchings, and their events are
sent out through one channel. For the received event, user would like to
know which watching it belongs to.
Introduce a watch ID. When watching on some key, user will get a watch
ID. The watch ID is attached to all events that is observed by this
watch.
2015-11-21 11:19:17 -08:00
Gyu-Ho Lee
48aebd9b09
storage: use map for watchableStore synced
...
This is for coreos#3859 switching slice to map for synced watchings.
For a large amount of synced watchings, map implementation performs better.
When putting 1 million watchers on the same key and canceling them one by
one: original implementation takes 9m7.268221091s, while the one with map
takes only 430.531637ms.
2015-11-21 00:42:09 -08:00
Yicheng Qin
2f74f76025
storage: remove the event concept from key-value layer
...
The point is to decouple the key-value storage layer and the
event notification layer clearly. It gives the watchableKV the
flexibility to define whatever event structure it wants without
breaking the ondisk format at key-value storage layer.
Changes:
1. change the format of key and value stored in backend
Store KeyValue struct instead of Event struct in backend value for
better abstraction as xiang suggests. And record the corresponded
action in the backend key.
2. Remove word 'event' from functions
2015-11-17 20:35:49 -08:00
Yicheng Qin
dadfdf6af8
Merge pull request #3802 from yichengq/fix-storage-watch
...
storage: delete key instead of setting it to false
2015-11-05 11:40:46 -08:00
Gyu-Ho Lee
6e5eb03544
storage: move watcherGauge to watchable_store
...
watcherGauge should be increased everytime we creates Watcher, not per watch
method call.
2015-11-04 13:17:47 -08:00
Yicheng Qin
c8e622f517
storage: make putm/delm a set with empty value
...
This cleans the code, and reduces the allocation space.
2015-11-03 19:10:45 -08:00
Yicheng Qin
6dbfc21846
storage: delete key instead of setting it to false
...
When getting the watched events, it iterate all keys in putm and delm
to generate the events. If we don't delete the key from putm/delm,
it would range on the key that is not actually put or deleted. This is
incorrect.
Fix the panic that happens when single put/delete is watched.
2015-11-03 19:00:39 -08:00
Gyu-Ho Lee
bdc280c4a7
storage: apply same naming in metrics.go
...
This is PR following up with Xiang's https://github.com/coreos/etcd/pull/3795 ,
and to make the naming consistent with its interface change.
2015-11-03 15:19:18 -08:00
Xiang Li
a1129dd5a5
storage: support multiple watching per watcher
...
We want to support multiple watchings per one watcher chan. Then
we can have one single go routine to watch multiple keys/prefixs.
2015-11-03 12:36:11 -08:00
Gyu-Ho Lee
f73d0ed1d9
storage: use map for watchable store unsynced
...
This is for `TODO: use map to reduce cancel cost`.
I switched slice to map, and benchmark results show
that map implementation performs better, as follows:
```
[1]:
benchmark old ns/op new ns/op delta
BenchmarkWatchableStoreUnsyncedCancel 215212 1307 -99.39%
BenchmarkWatchableStoreUnsyncedCancel-2 120453 710 -99.41%
BenchmarkWatchableStoreUnsyncedCancel-4 120765 748 -99.38%
BenchmarkWatchableStoreUnsyncedCancel-8 121391 719 -99.41%
benchmark old allocs new allocs delta
BenchmarkWatchableStoreUnsyncedCancel 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8 0 0 +0.00%
benchmark old bytes new bytes delta
BenchmarkWatchableStoreUnsyncedCancel 200 1 -99.50%
BenchmarkWatchableStoreUnsyncedCancel-2 138 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4 138 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8 139 0 -100.00%
[2]:
benchmark old ns/op new ns/op delta
BenchmarkWatchableStoreUnsyncedCancel 212550 1117 -99.47%
BenchmarkWatchableStoreUnsyncedCancel-2 120927 691 -99.43%
BenchmarkWatchableStoreUnsyncedCancel-4 120752 699 -99.42%
BenchmarkWatchableStoreUnsyncedCancel-8 121012 688 -99.43%
benchmark old allocs new allocs delta
BenchmarkWatchableStoreUnsyncedCancel 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8 0 0 +0.00%
benchmark old bytes new bytes delta
BenchmarkWatchableStoreUnsyncedCancel 197 1 -99.49%
BenchmarkWatchableStoreUnsyncedCancel-2 138 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4 138 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8 139 0 -100.00%
[3]:
benchmark old ns/op new ns/op delta
BenchmarkWatchableStoreUnsyncedCancel 214268 1183 -99.45%
BenchmarkWatchableStoreUnsyncedCancel-2 120763 759 -99.37%
BenchmarkWatchableStoreUnsyncedCancel-4 120321 708 -99.41%
BenchmarkWatchableStoreUnsyncedCancel-8 121628 680 -99.44%
benchmark old allocs new allocs delta
BenchmarkWatchableStoreUnsyncedCancel 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8 0 0 +0.00%
benchmark old bytes new bytes delta
BenchmarkWatchableStoreUnsyncedCancel 200 1 -99.50%
BenchmarkWatchableStoreUnsyncedCancel-2 139 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4 138 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8 139 0 -100.00%
[4]:
benchmark old ns/op new ns/op delta
BenchmarkWatchableStoreUnsyncedCancel 208332 1089 -99.48%
BenchmarkWatchableStoreUnsyncedCancel-2 121011 691 -99.43%
BenchmarkWatchableStoreUnsyncedCancel-4 120678 681 -99.44%
BenchmarkWatchableStoreUnsyncedCancel-8 121303 721 -99.41%
benchmark old allocs new allocs delta
BenchmarkWatchableStoreUnsyncedCancel 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8 0 0 +0.00%
benchmark old bytes new bytes delta
BenchmarkWatchableStoreUnsyncedCancel 194 1 -99.48%
BenchmarkWatchableStoreUnsyncedCancel-2 139 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4 139 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8 139 0 -100.00%
[5]:
benchmark old ns/op new ns/op delta
BenchmarkWatchableStoreUnsyncedCancel 211900 1097 -99.48%
BenchmarkWatchableStoreUnsyncedCancel-2 121795 753 -99.38%
BenchmarkWatchableStoreUnsyncedCancel-4 123182 700 -99.43%
BenchmarkWatchableStoreUnsyncedCancel-8 122820 688 -99.44%
benchmark old allocs new allocs delta
BenchmarkWatchableStoreUnsyncedCancel 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4 0 0 +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8 0 0 +0.00%
benchmark old bytes new bytes delta
BenchmarkWatchableStoreUnsyncedCancel 198 1 -99.49%
BenchmarkWatchableStoreUnsyncedCancel-2 140 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4 141 0 -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8 141 0 -100.00%
```
2015-10-21 15:30:15 -07:00
Yicheng Qin
0f7374ce89
storage: KV field -> store field in watchableStore
...
We need to access the underlying store to use its RangeEvents function.
It is not good to use unnecessary type conversion.
The underlying store is also needed for further store upon
watchableStore.
2015-10-20 19:23:20 -07:00
Xiang Li
6556bf1643
storage: remove the endRev of watcher
2015-10-17 15:59:49 -07:00
Xiang Li
09157d4f1a
storage: add metrics for watchers
2015-10-05 11:32:56 -07:00