300feea177
Merge pull request #8052 from heyitsanthony/watch-victim-test
...
mvcc: test watch victim/delay path
2017-06-08 11:10:33 -07:00
83b2ea2f60
mvcc: test watch victim/delay path
...
Current tests don't normally trigger the watch victim path because the
constants are too large; set the constants to small values and hammer
the store to cause watch delivery delays.
2017-06-07 17:02:00 -07:00
0352ce79b8
mvcc: count range/put/del operations for txns
...
Txns were previously only bumping the txn counter; now bumps all operation
counters.
2017-06-07 16:53:50 -07:00
fd71da47d1
mvcc: remove unused store.Equals function
2017-06-07 09:25:42 -07:00
ef63abdf7f
mvcc: don't use pointer for storeTxnRead in storeTxnWrite
...
Saves an allocation when creating a storeTxnWrite.
2017-06-06 09:51:57 -07:00
6846e49edf
Merge pull request #7859 from heyitsanthony/cache-consistent-get
...
mvcc: cache consistent index
2017-05-26 10:52:53 -07:00
ac4855e911
mvcc: benchmark ConsistentIndex
2017-05-26 09:49:40 -07:00
73dee0bec4
mvcc: cache consistentIndex
...
Called on every entry apply and boltdb requests aren't free.
2017-05-26 09:49:40 -07:00
0506f49f9e
backend: don't hold boltdb read txn lock on cursor scanning
...
Large fetches hold the lock when they do not need to do so.
2017-05-26 09:28:08 -07:00
343a018361
Merge pull request #7900 from heyitsanthony/chunk-restore
...
mvcc: chunk reads for restoring
2017-05-26 09:21:59 -07:00
8516d8ccc5
backend: force initial mmap size to 0 for windows
...
boltdb on windows allocates a file with the full mmap size even if the
db is empty. Force the initial mmap size to 0 so there's no huge initial
db file on windows.
Fixes #7910
2017-05-12 14:34:07 -07:00
8468b38631
backend: dynamically set snapshotWarningTimeout based on db size
2017-05-11 15:25:35 -07:00
1aca63e9e0
mvcc: time restore in restore benchmark
...
This never worked.
2017-05-09 20:14:58 -07:00
163fd2d76b
mvcc: chunk reads for restoring
...
Loading all keys at once would cause etcd to use twice as much
memory than it would need to serve the keys, causing RSS to spike on
boot. Instead, load the keys into the mvcc by chunk. Uses pipelining
for some concurrency.
Fixes #7822
2017-05-09 20:14:58 -07:00
230106dd3c
backend: add prometheus metric for large snapshot duration.
...
FIXES #7878
2017-05-05 17:27:33 -07:00
f7f30f2361
backend: print snapshotting duration warning every 30s
...
FIXES #7870
2017-05-04 16:41:03 -07:00
14d6ed9e5f
*: clear redundant return statement warnings (S1027)
2017-04-21 14:01:00 -07:00
5000d29b4a
mvcc: remove stopc select case in Hash
...
Revert change in 33acbb694b
2017-04-17 14:19:48 -07:00
8ffd58fb3b
mvcc/backend: remove t.tx.DB()==nil checks with GracefulStop
...
Revert https://github.com/coreos/etcd/pull/6662 .
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-04-17 14:17:00 -07:00
cd470f9ccd
Revert "mvcc: test inflight Hash to trigger Size on nil db"
...
This reverts commit 994e8e4f40
2017-04-17 14:15:43 -07:00
78a5eb79b5
*: add swagger and grpc-gateway assets for v3lock and v3election
2017-04-10 15:21:07 -07:00
f67bdc2eed
*: support checking that an interval tree's keys cover an entire interval
2017-04-03 15:38:07 -07:00
161c7f6bdf
Merge pull request #7579 from gyuho/fix-defrage
...
*: fix panic during defrag operation
2017-03-23 10:08:33 -07:00
7ef75e373a
Merge pull request #7525 from heyitsanthony/big-backend
...
etcdserver, backend: configure mmap size based on quota
2017-03-23 10:06:00 -07:00
26abd25cd3
mvcc/backend: hold 'readTx.Lock' until completing bolt.Tx reset
...
Fix https://github.com/coreos/etcd/issues/7526 .
When resetting `bolt.Tx` in `defrag` and `batchTxBuffered.commit`
operation, we do not hold `readTx` lock, so the inflight range
requests can trigger panic in `mvcc.Range` paths. This fixes by
moving mutexes out and hold it while resetting the `readTx`.
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-03-23 09:47:43 -07:00
7698a2a546
Merge pull request #7553 from xiang90/fix_defrag
...
backend: add FillPercent option
2017-03-21 11:16:17 -07:00
95870a21eb
backend: add FillPercent option
2017-03-21 08:06:03 -07:00
8a3fee15a3
etcdserver, backend: only warn if exceeding max quota
2017-03-17 15:38:57 -07:00
5e4b008106
*: base initial mmap size on quota size
2017-03-17 15:38:49 -07:00
2f1542c06d
*: use filepath.Join for files
2017-03-16 07:46:06 -07:00
33acbb694b
mvcc: txns and r/w views
...
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.
Adds support for concurrent read-only mvcc transactions.
Fixes #7083
2017-03-08 20:52:59 -08:00
8d438c2939
backend: readtx
...
ReadTxs are designed for read-only accesses to the backend using a
read-only boltDB transaction. Since BatchTx's are long-running
transactions, all writes to BatchTx will writeback to ReadTx, overlaying
the base read-only transaction.
2017-03-08 20:52:59 -08:00
3d75395875
*: remove never-unused vars, minor lint fix
...
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-03-06 14:59:12 -08:00
0914b8b707
test: Fix gosimple errors
...
Getting gosimple suggestion while running test script, so this PR is for fixing gosimple S1019 check.
raft/node_test.go:456:40: should use make([]raftpb.Entry, 1) instead (S1019)
raft/node_test.go:457:49: should use make([]raftpb.Entry, 1) instead (S1019)
raft/node_test.go:458:43: should use make([]raftpb.Message, 1) instead (S1019)
Refer https://github.com/dominikh/go-tools/blob/master/cmd/gosimple/README.md#checks for more information.
2017-02-09 08:01:28 -05:00
43078d3ced
mvcc: remove unused restore method
2016-11-18 23:04:39 +05:30
aa2b5aec1b
mvcc : Added benchmark for store.resotre
2016-11-17 04:01:15 +05:30
f014cca644
mvcc: TestStoreRestore fix
2016-11-16 16:58:42 +05:30
95fb41a923
mvcc: store.restore taking too long triggering snapshot cycle fix
2016-11-16 16:31:20 +05:30
b8b72f80f9
*: revendor, update proto files
2016-11-10 12:02:00 -08:00
425acb28c4
mvcc: return -1 for wrong watcher range key >= end
...
Fix https://github.com/coreos/etcd/issues/6819 .
2016-11-08 17:02:28 -08:00
4e1d3f0f52
mvcc: expose 'backend.IgnoreKey'
2016-10-25 10:07:08 -07:00
994e8e4f40
mvcc: test inflight Hash to trigger Size on nil db
2016-10-21 11:02:09 -07:00
7d30326968
backend: skip *bolt.DB.Size call when nil
...
Fix https://github.com/coreos/etcdlabs/issues/30 .
2016-10-21 11:01:23 -07:00
46716fe9fb
mvcc: fix gofmt issues from Go tip
2016-10-20 16:32:47 -07:00
93225ebafc
mvcc: fix rev inconsistency
...
Try:
./etcdctl put foo bar
./etcdctl del foo
./etcdctl compact 3
restart etcd
./etcdctl get foo
mvcc: required revision has been compacted
The error is unexpected when range over the head revision.
Internally, we incorrectly set current revision smaller than the
compacted revision when we remove all keys around compacted revision.
This commit fixes the issue by recovering the current revision at least
to compacted revision.
2016-10-12 10:42:57 -07:00
064e02f4b3
mvcc: Optimize updating key by storing lease in lessor
2016-10-12 09:37:09 +05:00
9970ded79f
mvcc: add BenchmarkWatchableStoreTxnPut benchmark
2016-10-06 22:44:25 +05:00
fa1e28102e
Merge pull request #5316 from ajityagaty/too_many_allocs
...
mvcc: Reduce number of allocs in PUT when watchableStore has no watchers.
2016-10-06 09:47:59 -07:00
9b56e51ca7
*: regenerate proto + gofmt change
2016-10-03 15:34:34 -07:00
962433c17f
*: set repo correctly for logging
2016-10-03 17:03:22 +08:00
69f5b4ba79
Documentation:made watch request doc more clear
2016-09-23 23:13:55 +08:00
1437388f77
mvcc: force commit and hash should be atomic for getting hash
2016-08-27 19:22:22 -07:00
e1789aa531
mvcc: only write txn should update index
2016-08-22 22:05:51 -07:00
de864d3b58
mvcc: fix count
2016-08-10 10:54:25 -07:00
bd62b0a646
mvcc: attach keys to leases after recover all state
...
The previous logic is wrong. When we have hisotry like Put(foo, bar, lease1),
and Put(foo, bar, lease2), we will end up with attaching foo to two leases 1 and
2. Similar things can happen for deattach by clearing the lease of a key.
Now we try to fix this by starting to attach leases at the end of the recovery.
We use a map to keep the last lease attachment state.
2016-08-04 11:17:58 -07:00
982e18d80b
*: regenerate proto with latest grpc-gateway
2016-07-27 13:21:03 -07:00
fffa484a9f
*: regenerate proto for adding deleterange
2016-07-23 16:17:44 -07:00
50be793f09
*: regenerate proto
2016-07-18 09:33:32 -07:00
ba2725c2d0
build, backend: add backend commit failpoints
2016-07-14 12:26:35 -07:00
c853704ac9
*: support get-old-kv in watch
2016-07-05 16:17:09 -07:00
bc6d7659af
Merge pull request #5795 from xiang90/filter
...
*: support watch with filters
2016-06-28 14:07:12 -07:00
dced92f8bd
*: support watch with filters
...
Now user can filter events with types. The API is also extensible.
It might make sense for the proxy to filter out events based on
more expensive/customized filter.
2016-06-28 13:46:57 -07:00
38035c8c13
Merge pull request #5794 from xiang90/fix_c
...
mvcc: do not hash consistent index
2016-06-28 12:25:32 -07:00
ef9754910e
mvcc: do not hash consistent index
2016-06-28 09:36:26 -07:00
9614dc6e71
etcdserver: check index of the kv when restarting
2016-06-27 10:27:27 -07:00
dbc7c2cf4e
backend: reuse timer in run().
...
Benchmarks:
```
import (
"testing"
"time"
)
func BenchmarkTimeAfter(b *testing.B) {
b.ReportAllocs()
for n := 0; n < b.N; n++ {
select {
case <- time.After(1 * time.Millisecond):
}
}
}
func BenchmarkTimerReset(b *testing.B) {
b.ReportAllocs()
t := time.NewTimer(1 * time.Millisecond)
for n := 0; n < b.N; n++ {
select {
case <- t.C:
}
t.Reset(1 * time.Millisecond)
}
}
```
Running reveals that each loop results in 3 allocs:
```
BenchmarkTimeAfter-4 2000 1112134 ns/op 192 B/op 3 allocs/op
BenchmarkTimerReset-4 2000 1109774 ns/op 0 B/op 0 allocs/op
```
2016-06-23 18:49:41 +05:00
def21f11a9
*: support count in range query
2016-06-21 16:20:55 -07:00
6f28b43806
*: fix pending events metrics
2016-06-19 23:00:39 -07:00
5a7b7f7595
main: add grpc-gateway support
...
Now etcd can serve HTTP json request at /v3alpha/
2016-06-14 17:09:06 -07:00
6bbd8b7efb
mvcc: add keyIndex benchmark test
...
Useful later when trying to optimize our restore operations.
2016-06-09 14:13:18 -07:00
1610391449
*: following changes for proto update
2016-06-07 13:33:03 -07:00
f57b4eb46d
mvcc: don't cancel watcher if stream is already closed
...
Close() already cancels all the watchers but doesn't bother to clear out
the bookkeeping maps so Cancel() may try to cancel twice.
Fixes #5533
2016-06-03 11:12:46 -07:00
9c767cbf98
Merge pull request #5464 from heyitsanthony/fix-victim-watchers
...
mvcc: tighten up watcher cancelation and revision handling
2016-05-30 20:09:39 -06:00
3266c809e4
mvcc: delete EXPIRE event type
...
Addressing https://github.com/coreos/etcd/pull/5484#discussion_r65005236 .
etcd v3 doesn't expire keys. It's either PUT of DELETE.
2016-05-29 14:54:38 -07:00
cfb3f96c2b
mvcc: tighten up watcher cancelation and revision handling
...
Makes w.cur into w.minrev, the minimum revision for the next update, and
retries cancelation if the watcher isn't found (because it's being processed
by moveVictims).
Fixes : #5459
2016-05-27 17:19:32 -07:00
bf8cf39daf
mvcc: use capnslog
2016-05-20 22:31:22 -07:00
394ce5f3b8
mvcc: move blocked unsynced watchers to victim list
2016-05-20 15:56:02 -07:00
5984e46364
mvcc: move blocked sync watcher work to victim list
...
Instead of holding the store lock while doing a lot of work like when syncung
unsynced watchers, the work from a blocked synced notify can be reused and
dispatched without holding the store lock for long.
2016-05-20 15:56:02 -07:00
0b34b236d6
mvcc: benchmark for synced watchers
2016-05-19 23:31:27 -07:00
aa85cf037f
mvcc: remove defer in watchable store
2016-05-19 13:51:51 -07:00
6f2e7875aa
etcdctl: add migrate command
...
Migrate command accepts a datadir and an optional user-provided
transformer function that transform v2 keys to v2 keys.
Migrate command then builds a v3 backend state based on the existing
v2 keys and the output of the transformer function.
2016-05-19 12:17:15 -07:00
77775e8e92
mvcc: preallocate bytes buffer for saveIndex
2016-05-18 10:01:57 -07:00
e4e4c9dc2c
mvcc: set bolt options to nil for non-linux systems
2016-05-17 12:46:44 -07:00
9d9f02c1ee
mvcc: update LICENSE header
2016-05-12 20:50:33 -07:00
080272be17
mvcc: limit total watchers synced per sync
...
Fixes #4567
2016-05-11 11:16:43 -07:00
0fb7cb8b00
*: add disk operation metrics for monitoring
2016-05-11 09:36:45 -07:00
34b0736f2c
mvcc: Reduce number of allocs when watchableStore if no watchers.
...
When there are no watchers the number of allocations made while handling
a PUT operation can be reduced by exiting early.
2016-05-11 00:51:00 -07:00
a288188001
*: typo, remove string type assertions
2016-05-03 10:59:57 -07:00
015acabdbb
*: rerun genproto -g
2016-05-02 23:02:31 -07:00
7a6d9ea01a
mvcc: Hash to return Revision
2016-05-02 15:04:24 -07:00
ad327e01d0
mvcc: add benchmark for watch put and improve it
2016-04-29 19:58:37 -07:00
3ddcc21179
mvcc: fix watch deleteRange
2016-04-29 09:40:28 -07:00
67645095e9
*: add debugging metrics
2016-04-26 09:52:56 -07:00
b7ac758969
*: rename storage package to mvcc
2016-04-25 15:25:51 -07:00
Anthony Romano