vitalif
/
etcd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
7,213 Commits
21 Branches
406 Tags
79 MiB
Commit Graph

51 Commits (67472d1ee07b03957793a4ab36cef6d5130b030a)

Author SHA1 Message Date
Anthony Romano 7c17665a1a storage: limit total unique revisions in unsynced watcher event list 2016-02-18 11:36:22 -08:00
Anthony Romano af225e7433 v3rpc: don't race on current watcher header revision 2016-02-17 14:03:52 -08:00
Anthony Romano 2cbf7cf6d1 storage: do not send outdated events to unsynced watchers 2016-02-17 14:03:51 -08:00
Anthony Romano ee1a03167d storage, v3: pass compaction revision through watchresponse 2016-02-12 14:06:59 -08:00
Xiang Li b09214df32 storage: update watch.cur and fix tests 2016-02-03 00:54:07 -08:00
Xiang Li 52416fafb0 storage: send compaction 2016-02-02 23:17:27 -08:00
Xiang Li 5780497e18 storage: remove unncessary handle func 2016-02-02 23:02:15 -08:00
Xiang Li ae5161382b storage: release tx lock until finish using the readonly bytes 
The backend will return read only bytes that are only vaild while
the tx is open. We should hold the lock until we get a full copy
by unmarshal.
 2016-02-02 22:49:05 -08:00
Xiang Li 31c0c5181a storage: make unsync a watcherSetByKey 2016-02-02 20:09:35 -08:00
Xiang Li 8dc6248aa7 storage: add set delete 2016-02-02 19:28:42 -08:00
Xiang Li 810c3e74a8 storage: remove unnecessary abstraction 2016-02-02 19:15:46 -08:00
Xiang Li e5b35b82c5 storage: add watchSet and watchSetByKey type 2016-02-02 18:56:36 -08:00
Anthony Romano 20461ab11a *: fix many typos 2016-01-31 21:42:39 -08:00
Xiang Li 611751aee2 storage: simplify watch store 
We decided that we will not support modifing the same
key in one txn multiple times. That can simlify the current
code/design a lot.
 2016-01-31 16:33:07 -08:00
Anthony Romano 5a967eb2a0 storage: publish delete events on lease revocation 2016-01-22 13:40:55 -08:00
Gyu-Ho Lee 4f427bca43 storage: check prefix in unsynced 
Current syncWatchers method skips the events that have
prefixes that are being watched when the prefix is not
existent as a key. This fixes https://github.com/coreos/etcd/issues/4191
by adding prefix checking to not skip those events.
 2016-01-13 11:21:47 -08:00
Xiang Li f5753f2f51 *: support lease Attach 
Now we can attach keys to leases. And revoking the lease removes all
the attached keys of that lease.
 2016-01-09 11:01:58 -08:00
Gyu-Ho Lee f76166a041 *: fix minor typos 2016-01-08 00:21:19 -08:00
Gyu-Ho Lee 366e7a879f *: fill in WatchResponse.Header 
Related to coreos#3848.
 2016-01-06 15:12:53 -08:00
Xiang Li 5dd3f91903 *: make backend outside kv 
KV and lease will share the same backend. Thus we need to make
backend outside KV.
 2016-01-05 19:55:29 -08:00
Xiang Li 1e61243fd7 Merge pull request #4134 from xiang90/lease 
lease: modify API and persist lease to disk
 2016-01-05 10:30:52 -08:00
Xiang Li 09b420f08c *: move leaseID typedef to lease pkg 2016-01-05 10:18:17 -08:00
Gyu-Ho Lee 7339abc79e storage: fix watchable_store notify to hash-lookup once 
We should just assign events and ok at first, and check the
boolean value, instead of looking up the map twice.
 2016-01-04 22:34:48 -08:00
Anthony Romano 21a6ade53d storage: change type of WatchIDs from int64 to WatchID 2016-01-04 19:52:22 -08:00
Gyu-Ho Lee 556d4a6932 *: remove CancelFunc return for Watch. Use Cancel for watch. 2016-01-04 16:17:55 -08:00
Xiang Li 4336278b44 *: support put with lease 2016-01-04 15:54:06 -08:00
Gyu-Ho Lee 6540f47dfa *: WatchResponse for multiple Events with WatchID 
storage/storagepb: remove watchID from Event

storage: add WatchResponse to watcher.go to wrap events, watchID

storage: watchableStore to use WatchResponse

storage: kv_test with WatchResponse

etcdserver/api/v3rpc: watch to receive storage.WatchResponse type
 2016-01-03 16:32:48 -08:00
Xiang Li eda0eefc25 *: support watcher cancellation inside watchStream 2016-01-03 00:20:21 -08:00
Xiang Li 807db7e2aa storage: rename watching -> watcher 2016-01-02 20:20:22 -08:00
Xiang Li ee0b3f42ed *: rename watcher to watchStream 
Watcher vs Watching in storage pkg is confusing. Watcher should be named
as watchStream since it contains a channel as stream to send out events.
Then we can rename watching to watcher, which actually watches on a key
and send watched events through watchStream.

This commits renames watcher to watchStram.
 2016-01-02 16:03:57 -08:00
Gyu-Ho Lee 7dd599b69d Merge pull request #4091 from gyuho/watch_events_slice 
storage: watch events in slice
 2015-12-31 23:54:08 -08:00
Gyu-Ho Lee 0b01acf131 storage: watchable_store.go events slice 2015-12-31 23:44:02 -08:00
Hitoshi Mitake 0b2d31f3bc storage: decouple default parameters and storage creation 
newStore() uses constants for some important parameters
e.g. batchInerval. It is not suitable for other storage users
(e.g. tools/benchmark). This commit decouples the default parameters
from storage creation. etcd should use a newly added function
newDefaultStore() for creating a store with default parameters.
 2015-12-31 22:28:59 +09:00
Gyu-Ho Lee 78b0b8a4a0 storage: range all unsynced at once 
This is for https://github.com/coreos/etcd/issues/3848.
It replaces RangeHistory method for more efficient event
sending.
 2015-12-28 11:37:26 -08:00
Xiang Li 9f908ce67f storage: add missing return for unsafeAddWatching 2015-12-01 14:55:20 -08:00
Yicheng Qin deb1da5f57 storage: add watch ID to identify watchings 
One watcher includes multiple watchings, and their events are
sent out through one channel. For the received event, user would like to
know which watching it belongs to.

Introduce a watch ID. When watching on some key, user will get a watch
ID. The watch ID is attached to all events that is observed by this
watch.
 2015-11-21 11:19:17 -08:00
Gyu-Ho Lee 48aebd9b09 storage: use map for watchableStore synced 
This is for coreos#3859 switching slice to map for synced watchings.
For a large amount of synced watchings, map implementation performs better.
When putting 1 million watchers on the same key and canceling them one by
one: original implementation takes 9m7.268221091s, while the one with map
takes only 430.531637ms.
 2015-11-21 00:42:09 -08:00
Yicheng Qin 2f74f76025 storage: remove the event concept from key-value layer 
The point is to decouple the key-value storage layer and the
event notification layer clearly. It gives the watchableKV the
flexibility to define whatever event structure it wants without
breaking the ondisk format at key-value storage layer.

Changes:

1. change the format of key and value stored in backend

Store KeyValue struct instead of Event struct in backend value for
better abstraction as xiang suggests. And record the corresponded
action in the backend key.

2. Remove word 'event' from functions
 2015-11-17 20:35:49 -08:00
Yicheng Qin dadfdf6af8 Merge pull request #3802 from yichengq/fix-storage-watch 
storage: delete key instead of setting it to false
 2015-11-05 11:40:46 -08:00
Gyu-Ho Lee 6e5eb03544 storage: move watcherGauge to watchable_store 
watcherGauge should be increased everytime we creates Watcher, not per watch
method call.
 2015-11-04 13:17:47 -08:00
Yicheng Qin c8e622f517 storage: make putm/delm a set with empty value 
This cleans the code, and reduces the allocation space.
 2015-11-03 19:10:45 -08:00
Yicheng Qin 6dbfc21846 storage: delete key instead of setting it to false 
When getting the watched events, it iterate all keys in putm and delm
to generate the events. If we don't delete the key from putm/delm,
it would range on the key that is not actually put or deleted. This is
incorrect.

Fix the panic that happens when single put/delete is watched.
 2015-11-03 19:00:39 -08:00
Gyu-Ho Lee bdc280c4a7 storage: apply same naming in metrics.go 
This is PR following up with Xiang's https://github.com/coreos/etcd/pull/3795,
and to make the naming consistent with its interface change.
 2015-11-03 15:19:18 -08:00
Xiang Li a1129dd5a5 storage: support multiple watching per watcher 
We want to support multiple watchings per one watcher chan. Then
we can have one single go routine to watch multiple keys/prefixs.
 2015-11-03 12:36:11 -08:00
Gyu-Ho Lee f73d0ed1d9 storage: use map for watchable store unsynced 
This is for `TODO: use map to reduce cancel cost`.
I switched slice to map, and benchmark results show
that map implementation performs better, as follows:

```
[1]:
benchmark                                   old ns/op     new ns/op     delta
BenchmarkWatchableStoreUnsyncedCancel       215212        1307          -99.39%
BenchmarkWatchableStoreUnsyncedCancel-2     120453        710           -99.41%
BenchmarkWatchableStoreUnsyncedCancel-4     120765        748           -99.38%
BenchmarkWatchableStoreUnsyncedCancel-8     121391        719           -99.41%

benchmark                                   old allocs     new allocs     delta
BenchmarkWatchableStoreUnsyncedCancel       0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8     0              0              +0.00%

benchmark                                   old bytes     new bytes     delta
BenchmarkWatchableStoreUnsyncedCancel       200           1             -99.50%
BenchmarkWatchableStoreUnsyncedCancel-2     138           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4     138           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8     139           0             -100.00%

[2]:
benchmark                                   old ns/op     new ns/op     delta
BenchmarkWatchableStoreUnsyncedCancel       212550        1117          -99.47%
BenchmarkWatchableStoreUnsyncedCancel-2     120927        691           -99.43%
BenchmarkWatchableStoreUnsyncedCancel-4     120752        699           -99.42%
BenchmarkWatchableStoreUnsyncedCancel-8     121012        688           -99.43%

benchmark                                   old allocs     new allocs     delta
BenchmarkWatchableStoreUnsyncedCancel       0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8     0              0              +0.00%

benchmark                                   old bytes     new bytes     delta
BenchmarkWatchableStoreUnsyncedCancel       197           1             -99.49%
BenchmarkWatchableStoreUnsyncedCancel-2     138           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4     138           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8     139           0             -100.00%

[3]:
benchmark                                   old ns/op     new ns/op     delta
BenchmarkWatchableStoreUnsyncedCancel       214268        1183          -99.45%
BenchmarkWatchableStoreUnsyncedCancel-2     120763        759           -99.37%
BenchmarkWatchableStoreUnsyncedCancel-4     120321        708           -99.41%
BenchmarkWatchableStoreUnsyncedCancel-8     121628        680           -99.44%

benchmark                                   old allocs     new allocs     delta
BenchmarkWatchableStoreUnsyncedCancel       0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8     0              0              +0.00%

benchmark                                   old bytes     new bytes     delta
BenchmarkWatchableStoreUnsyncedCancel       200           1             -99.50%
BenchmarkWatchableStoreUnsyncedCancel-2     139           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4     138           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8     139           0             -100.00%

[4]:
benchmark                                   old ns/op     new ns/op     delta
BenchmarkWatchableStoreUnsyncedCancel       208332        1089          -99.48%
BenchmarkWatchableStoreUnsyncedCancel-2     121011        691           -99.43%
BenchmarkWatchableStoreUnsyncedCancel-4     120678        681           -99.44%
BenchmarkWatchableStoreUnsyncedCancel-8     121303        721           -99.41%

benchmark                                   old allocs     new allocs     delta
BenchmarkWatchableStoreUnsyncedCancel       0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8     0              0              +0.00%

benchmark                                   old bytes     new bytes     delta
BenchmarkWatchableStoreUnsyncedCancel       194           1             -99.48%
BenchmarkWatchableStoreUnsyncedCancel-2     139           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4     139           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8     139           0             -100.00%

[5]:
benchmark                                   old ns/op     new ns/op     delta
BenchmarkWatchableStoreUnsyncedCancel       211900        1097          -99.48%
BenchmarkWatchableStoreUnsyncedCancel-2     121795        753           -99.38%
BenchmarkWatchableStoreUnsyncedCancel-4     123182        700           -99.43%
BenchmarkWatchableStoreUnsyncedCancel-8     122820        688           -99.44%

benchmark                                   old allocs     new allocs     delta
BenchmarkWatchableStoreUnsyncedCancel       0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8     0              0              +0.00%

benchmark                                   old bytes     new bytes     delta
BenchmarkWatchableStoreUnsyncedCancel       198           1             -99.49%
BenchmarkWatchableStoreUnsyncedCancel-2     140           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4     141           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8     141           0             -100.00%
```
 2015-10-21 15:30:15 -07:00
Yicheng Qin 0f7374ce89 storage: KV field -> store field in watchableStore 
We need to access the underlying store to use its RangeEvents function.
It is not good to use unnecessary type conversion.

The underlying store is also needed for further store upon
watchableStore.
 2015-10-20 19:23:20 -07:00
Xiang Li 6556bf1643 storage: remove the endRev of watcher 2015-10-17 15:59:49 -07:00
Xiang Li 09157d4f1a storage: add metrics for watchers 2015-10-05 11:32:56 -07:00
Gyu-Ho Lee 4113509828 storage/watchable_store: defer to Unlock s.mu 
New PR from https://github.com/coreos/etcd/pull/3575.
This add `defer` to `s.mu`. Current code does not `Unlock`
in the correct scope, I think.

(Sorry, I accidentally deleted my fork so the changes
might not sound continuous from my previous pull requests.)
 2015-09-22 23:25:07 -07:00
Jonathan Boulle 7848ac3979 *: add missing license headers 2015-09-15 14:09:01 -07:00