Anthony Romano
f58c0cfb66
mvcc: Revisions() method for index to avoid key allocation
...
Save another alloc on the one key path.
2017-08-21 11:30:02 -07:00
fengshaobao 00231050
13041c15ba
mvcc: sending events after restore
...
Fixes : #8411
2017-08-21 10:32:49 -07:00
Anthony Romano
8b872196d0
backend: cache buckets in read tx
...
Saves an alloc and about 10% of Range() time.
2017-08-21 02:16:55 -07:00
Anthony Romano
10b65c97dd
mvcc: benchmark Range() on a single key
2017-08-21 00:14:46 -07:00
Anthony Romano
ccd1bb1780
mvcc: test keys gauge is reloaded correctly on restore
2017-08-10 09:21:39 -07:00
Anthony Romano
32866572bf
mvcc: reset keys gauge on restore
...
Fixes #8388
2017-08-10 08:37:50 -07:00
fanmin shi
df5a3d15ce
mvcc: increase rev for TestHashKVWhenCompacting
2017-07-31 17:59:49 -07:00
fanmin shi
bb86c327e2
mvcc: HashKV gets keep from kvindex.Keep
2017-07-31 17:59:49 -07:00
fanmin shi
4c2c5b0084
mvcc: add tests for Keep
2017-07-31 17:59:42 -07:00
fanmin shi
7b8fb3cf0a
mvcc: add and implement Keep api to index
...
Keep finds all revisions to be kept for a Compaction at the given rev.
2017-07-31 14:04:03 -07:00
fanmin shi
451b062184
mvcc/backend: add TestBackendWritebackForEach to backend_test.go
2017-07-28 09:39:48 -07:00
fanmin shi
785deebd62
mvcc/backend: enforce ordering for UnsafeForEach in read_tx.go
...
This pr changes UnsafeForEach to traverse on boltdb before on the buffer.
This ordering guarantees that UnsafeForEach traverses in the same order
before or after the commit of buffer.
2017-07-28 09:30:23 -07:00
Xiang Li
2a348fb8e9
Merge pull request #8263 from fanminshi/hash_by_rev
...
api: hash by rev
2017-07-26 11:22:33 -07:00
fanmin shi
8609521ce2
mvcc: add TestHashKVWhenCompacting to kvstore_test
2017-07-26 09:48:29 -07:00
fanmin shi
deca9879c2
mvcc: add HashByRev to kv.go
...
HashByRev computes the hash of all MVCC keys up to a given revision.
2017-07-25 17:00:46 -07:00
Joe Betz
c06953ae08
mvcc: Add metric for count of db key revisions compacted.
...
When digging into etcd/boltdb "storage space exceeded" issues, this metric may help answer questions about if/when compactions occured and how much data was freed.
2017-07-20 10:07:56 -07:00
Anthony Romano
e9d096ae6b
mvcc: don't allocate end revision while computing range
...
Use 'nil' since it's only reading a single key. Also preallocates
the result slice based on limit / number of revisions fetched.
Fixes #8208
2017-07-06 15:59:27 -07:00
Gyu-Ho Lee
870302afa6
mvcc/backend: enable 'NoFreelistSync' by default (linux)
...
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-07-05 16:10:04 -07:00
Gyu-Ho Lee
318e9c766f
*: replace 'boltdb' import paths with 'coreos/bbolt'
...
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-07-05 14:32:13 -07:00
Anthony Romano
522e75cb4f
mvcc: use GaugeFunc metric to load db size when requested
...
Relying on mvcc to set the db size metric can cause it to
miss size changes when a txn commits after the last write
completes before a quiescent period. Instead, load the
db size on demand.
Fixes #8146
2017-06-21 23:58:37 -07:00
Anthony Romano
51a568aa81
mvcc: restore into tree index with one key index
...
Clobbering the mvcc kvindex with new keyIndexes for each restore
chunk would cause index corruption by dropping historical information.
2017-06-19 12:04:01 -07:00
Anthony Romano
02164874d9
mvcc: test restore and deletes with small chunk sizes
2017-06-19 12:04:01 -07:00
Anthony Romano
7f149d8fb6
mvcc: set db size metric on restore
...
Fixes #8080
2017-06-16 11:27:34 -07:00
Anthony Romano
da48f1feaf
mvcc: create TxnWrites from TxnRead with NewReadOnlyTxnWrite
...
Already used internally by mvcc, but needed by etcdserver txns.
2017-06-09 09:20:38 -07:00
Anthony Romano
300feea177
Merge pull request #8052 from heyitsanthony/watch-victim-test
...
mvcc: test watch victim/delay path
2017-06-08 11:10:33 -07:00
Anthony Romano
83b2ea2f60
mvcc: test watch victim/delay path
...
Current tests don't normally trigger the watch victim path because the
constants are too large; set the constants to small values and hammer
the store to cause watch delivery delays.
2017-06-07 17:02:00 -07:00
Anthony Romano
0352ce79b8
mvcc: count range/put/del operations for txns
...
Txns were previously only bumping the txn counter; now bumps all operation
counters.
2017-06-07 16:53:50 -07:00
Anthony Romano
fd71da47d1
mvcc: remove unused store.Equals function
2017-06-07 09:25:42 -07:00
Anthony Romano
ef63abdf7f
mvcc: don't use pointer for storeTxnRead in storeTxnWrite
...
Saves an allocation when creating a storeTxnWrite.
2017-06-06 09:51:57 -07:00
Anthony Romano
6846e49edf
Merge pull request #7859 from heyitsanthony/cache-consistent-get
...
mvcc: cache consistent index
2017-05-26 10:52:53 -07:00
Anthony Romano
ac4855e911
mvcc: benchmark ConsistentIndex
2017-05-26 09:49:40 -07:00
Anthony Romano
73dee0bec4
mvcc: cache consistentIndex
...
Called on every entry apply and boltdb requests aren't free.
2017-05-26 09:49:40 -07:00
Anthony Romano
0506f49f9e
backend: don't hold boltdb read txn lock on cursor scanning
...
Large fetches hold the lock when they do not need to do so.
2017-05-26 09:28:08 -07:00
Anthony Romano
343a018361
Merge pull request #7900 from heyitsanthony/chunk-restore
...
mvcc: chunk reads for restoring
2017-05-26 09:21:59 -07:00
Anthony Romano
8516d8ccc5
backend: force initial mmap size to 0 for windows
...
boltdb on windows allocates a file with the full mmap size even if the
db is empty. Force the initial mmap size to 0 so there's no huge initial
db file on windows.
Fixes #7910
2017-05-12 14:34:07 -07:00
fanmin shi
8468b38631
backend: dynamically set snapshotWarningTimeout based on db size
2017-05-11 15:25:35 -07:00
Anthony Romano
1aca63e9e0
mvcc: time restore in restore benchmark
...
This never worked.
2017-05-09 20:14:58 -07:00
Anthony Romano
163fd2d76b
mvcc: chunk reads for restoring
...
Loading all keys at once would cause etcd to use twice as much
memory than it would need to serve the keys, causing RSS to spike on
boot. Instead, load the keys into the mvcc by chunk. Uses pipelining
for some concurrency.
Fixes #7822
2017-05-09 20:14:58 -07:00
fanmin shi
230106dd3c
backend: add prometheus metric for large snapshot duration.
...
FIXES #7878
2017-05-05 17:27:33 -07:00
fanmin shi
f7f30f2361
backend: print snapshotting duration warning every 30s
...
FIXES #7870
2017-05-04 16:41:03 -07:00
Anthony Romano
14d6ed9e5f
*: clear redundant return statement warnings (S1027)
2017-04-21 14:01:00 -07:00
Gyu-Ho Lee
5000d29b4a
mvcc: remove stopc select case in Hash
...
Revert change in 33acbb694b
.
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-04-17 14:19:48 -07:00
Gyu-Ho Lee
8ffd58fb3b
mvcc/backend: remove t.tx.DB()==nil checks with GracefulStop
...
Revert https://github.com/coreos/etcd/pull/6662 .
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-04-17 14:17:00 -07:00
Gyu-Ho Lee
cd470f9ccd
Revert "mvcc: test inflight Hash to trigger Size on nil db"
...
This reverts commit 994e8e4f40
.
Since now etcdserver gracefully shuts down the gRPC server
2017-04-17 14:15:43 -07:00
Anthony Romano
78a5eb79b5
*: add swagger and grpc-gateway assets for v3lock and v3election
2017-04-10 15:21:07 -07:00
Anthony Romano
f67bdc2eed
*: support checking that an interval tree's keys cover an entire interval
2017-04-03 15:38:07 -07:00
Gyu-Ho Lee
161c7f6bdf
Merge pull request #7579 from gyuho/fix-defrage
...
*: fix panic during defrag operation
2017-03-23 10:08:33 -07:00
Anthony Romano
7ef75e373a
Merge pull request #7525 from heyitsanthony/big-backend
...
etcdserver, backend: configure mmap size based on quota
2017-03-23 10:06:00 -07:00
Gyu-Ho Lee
26abd25cd3
mvcc/backend: hold 'readTx.Lock' until completing bolt.Tx reset
...
Fix https://github.com/coreos/etcd/issues/7526 .
When resetting `bolt.Tx` in `defrag` and `batchTxBuffered.commit`
operation, we do not hold `readTx` lock, so the inflight range
requests can trigger panic in `mvcc.Range` paths. This fixes by
moving mutexes out and hold it while resetting the `readTx`.
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-03-23 09:47:43 -07:00
Xiang Li
7698a2a546
Merge pull request #7553 from xiang90/fix_defrag
...
backend: add FillPercent option
2017-03-21 11:16:17 -07:00