vitalif
/
etcd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
15,931 Commits
21 Branches
406 Tags
79 MiB
Commit Graph

63 Commits (b6d98719f6aae6c4628962d027a17d21d7b1d909)

Author SHA1 Message Date
jingyih 6b389bf23c mvcc: remove capnslog 2020-02-07 07:44:44 -08:00
Ted Yu 7f3dd59d22 mvcc: Log warning when compaction encounters error (#11447) 2019-12-13 15:59:49 -08:00
sun c5887cdc49 mvcc: go style err check (#11419) 2019-12-04 18:46:17 -08:00
Jingyi Hu cbc1340af5
Merge pull request #11330 from shenjiangc/master 
mvcc/kvstore: Optimize compaction, slove conflict for #11150
 2019-11-04 20:56:01 -08:00
shenjiangc 72cc5dcf39 mvcc/kvstore:when the number key-value is greater than one million, compact take too long and blocks other requests 2019-11-05 11:48:32 +08:00
Ziheng Liu 235d7c2e51 mvcc: Add Unlock before panic to prevent double lock 2019-10-28 22:38:34 -04:00
yoyinzyc 57aa68af5a etcdserver: trace compaction request; add return parameter 'trace' to applierV3.Compaction() 
mvcc: trace compaction request; add input parameter 'trace' to KV.Compact()
 2019-10-07 09:55:27 -07:00
yoyinzyc 401df4bb8e etcdserver: add put request steps. 
mvcc: add put request steps; add trace to KV.Write() as input parameter.
 2019-10-01 14:08:06 -07:00
Jingyi Hu 88d998be4c
Merge pull request #10975 from lzhfromustc/currentRev 
Add critical section to protect s.currentRev
 2019-09-06 12:27:53 -07:00
Ziheng Liu 077dd74827 mvcc: add a TODO in (*store).Hash, to warn against potential data race of currentRev and suggest feasible fix 2019-09-06 15:05:43 -04:00
Jingyi Hu 0f8c46a0f3 mvcc: add store revision metrics 
Add experimental metrics etcd_debugging_mvcc_current_revision and
etcd_debugging_mvcc_compact_revision.
 2019-09-05 15:40:06 -07:00
Gyuho Lee f279bfb775 mvcc: keep 64-bit alignment in "store" struct 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-15 13:01:32 -07:00
Joe Betz 9b51febaf5 *: Add experimental-compaction-batch-limit flag 2019-08-15 11:47:23 -07:00
Gyuho Lee dde3c5fc40 mvcc: clean up metrics names, add missing register calls 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-01 14:44:55 -07:00
Jingyi Hu ad80752715 mvcc: add metrics dbOpenReadTxn 
Expose the number of currently open read transactions in backend to
metrics endpoint.
 2019-06-10 17:20:04 -07:00
Jingyi Hu 1c19f126cb mvcc/backend: rename ReadTx Lock() to RLock() 
For better code readability, renaming Lock() to RLock() in ReadTx
interface.
 2019-03-05 13:53:27 -08:00
WizardCXY 6e8913b004 bugfix:dead lock on store.mu when store.Compact in store.Restore happens 2019-01-21 10:46:58 +08:00
Gyuho Lee d537b328cb mvcc: update import paths "go.etcd.io/etcd" 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2018-08-28 17:47:55 -07:00
Gyuho Lee f2db05a869 mvcc: server db size with "etcd_debugging" namespace for backward compatibility 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-06-07 10:23:12 -07:00
Gyuho Lee 1d91698268 mvcc: document, clean up histogram variables 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-05-23 14:03:28 -07:00
Gyuho Lee bc59f7b42f mvcc: add "etcd_mvcc_hash_(rev)_duration_seconds" 
etcd_mvcc_hash_duration_seconds
etcd_mvcc_hash_rev_duration_seconds

Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-05-23 13:09:42 -07:00
Gyuho Lee 03ef9745a9 mvcc: add more structured logging 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-05-04 13:15:51 -07:00
Gyuho Lee 4d863dac5a mvcc: support structured logging in compact restore 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-05-02 11:57:23 -07:00
jocalvert f176427791 mvcc: Clone the key index for compaction and lock on each item 
For compaction, clone the original Btree for traversal purposes, so as to
not hold the lock for the duration of compaction. This allows read/write
throughput by not blocking when the index tree is large (> 1M entries).

mvcc: add comment for index compaction lock
mvcc: explicitly unlock store to do index compaction synchronously
mvcc: formatting index bench
mvcc: add release note for index compaction changes
mvcc: add license header
 2018-04-18 13:29:27 -07:00
Gyuho Lee c00c6cb685 mvcc: support structured logger 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-16 17:36:00 -07:00
Gyuho Lee 8a518b01c4 *: revert "internal/mvcc" change 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-02-26 17:11:40 -08:00
Gyuho Lee 80d15948bc *: move "mvcc" to "internal/mvcc" 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-01-26 11:14:41 -08:00
Gyuho Lee 349a377a67 *: move "lease" to "internal/lease" 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-01-26 11:09:29 -08:00
Gyuho Lee 2e95ace82b mvcc: fetch revisions with current revision, not 0, in HashByRev 
It was getting revisions with "atRev==0", which makes
"available" from "keep" method always empty since
"walk" on "keyIndex" only returns true.

"available" should be populated with all revisions to be
kept if the compaction happens with the given revision.
But, "available" was being empty when "kvindex.Keep(0)"
since it's always the case that "rev.main > atRev==0".

Fix https://github.com/coreos/etcd/issues/9022.

Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2017-12-18 12:17:06 -08:00
Gyu-Ho Lee f65aee0759 *: replace 'golang.org/x/net/context' with 'context' 
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
 2017-09-07 13:39:42 -07:00
Anthony Romano 32866572bf mvcc: reset keys gauge on restore 
Fixes #8388
 2017-08-10 08:37:50 -07:00
fanmin shi bb86c327e2 mvcc: HashKV gets keep from kvindex.Keep 2017-07-31 17:59:49 -07:00
fanmin shi deca9879c2 mvcc: add HashByRev to kv.go 
HashByRev computes the hash of all MVCC keys up to a given revision.
 2017-07-25 17:00:46 -07:00
Anthony Romano e9d096ae6b mvcc: don't allocate end revision while computing range 
Use 'nil' since it's only reading a single key. Also preallocates
the result slice based on limit / number of revisions fetched.

Fixes #8208
 2017-07-06 15:59:27 -07:00
Anthony Romano 522e75cb4f mvcc: use GaugeFunc metric to load db size when requested 
Relying on mvcc to set the db size metric can cause it to
miss size changes when a txn commits after the last write
completes before a quiescent period. Instead, load the
db size on demand.

Fixes #8146
 2017-06-21 23:58:37 -07:00
Anthony Romano 51a568aa81 mvcc: restore into tree index with one key index 
Clobbering the mvcc kvindex with new keyIndexes for each restore
chunk would cause index corruption by dropping historical information.
 2017-06-19 12:04:01 -07:00
Anthony Romano 02164874d9 mvcc: test restore and deletes with small chunk sizes 2017-06-19 12:04:01 -07:00
Anthony Romano 7f149d8fb6 mvcc: set db size metric on restore 
Fixes #8080
 2017-06-16 11:27:34 -07:00
Anthony Romano fd71da47d1 mvcc: remove unused store.Equals function 2017-06-07 09:25:42 -07:00
Anthony Romano 73dee0bec4 mvcc: cache consistentIndex 
Called on every entry apply and boltdb requests aren't free.
 2017-05-26 09:49:40 -07:00
Anthony Romano 163fd2d76b mvcc: chunk reads for restoring 
Loading all keys at once would cause etcd to use twice as much
memory than it would need to serve the keys, causing RSS to spike on
boot. Instead, load the keys into the mvcc by chunk. Uses pipelining
for some concurrency.

Fixes #7822
 2017-05-09 20:14:58 -07:00
Gyu-Ho Lee 5000d29b4a mvcc: remove stopc select case in Hash 
Revert change in 33acbb694b.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
 2017-04-17 14:19:48 -07:00
Anthony Romano 33acbb694b mvcc: txns and r/w views 
Clean-up of the mvcc interfaces to use txn interfaces instead of an id.

Adds support for concurrent read-only mvcc transactions.

Fixes #7083
 2017-03-08 20:52:59 -08:00
Manjunath A Kumatagi 0914b8b707 test: Fix gosimple errors 
Getting gosimple suggestion while running test script, so this PR is for fixing gosimple S1019 check.
raft/node_test.go:456:40: should use make([]raftpb.Entry, 1) instead (S1019)
raft/node_test.go:457:49: should use make([]raftpb.Entry, 1) instead (S1019)
raft/node_test.go:458:43: should use make([]raftpb.Message, 1) instead (S1019)

Refer https://github.com/dominikh/go-tools/blob/master/cmd/gosimple/README.md#checks for more information.
 2017-02-09 08:01:28 -05:00
sharat aa2b5aec1b mvcc : Added benchmark for store.resotre 2016-11-17 04:01:15 +05:30
sharat 95fb41a923 mvcc: store.restore taking too long triggering snapshot cycle fix 2016-11-16 16:31:20 +05:30
Gyu-Ho Lee 4e1d3f0f52 mvcc: expose 'backend.IgnoreKey' 2016-10-25 10:07:08 -07:00
Xiang Li 93225ebafc mvcc: fix rev inconsistency 
Try:

./etcdctl put foo bar
./etcdctl del foo
./etcdctl compact 3

restart etcd

./etcdctl get foo
mvcc: required revision has been compacted

The error is unexpected when range over the head revision.

Internally, we incorrectly set current revision smaller than the
compacted revision when we remove all keys around compacted revision.

This commit fixes the issue by recovering the current revision at least
to compacted revision.
 2016-10-12 10:42:57 -07:00
Nikita Vetoshkin 064e02f4b3 mvcc: Optimize updating key by storing lease in lessor 2016-10-12 09:37:09 +05:00
Xiang Li 1437388f77 mvcc: force commit and hash should be atomic for getting hash 2016-08-27 19:22:22 -07:00