Commit Graph

63 Commits (ccdb850e1e0db7d9aa61ffbb9fc1a07076af7c24)

Author SHA1 Message Date
Yicheng Qin 0d38c13990 storage: use temp path to handle test file 2015-08-26 15:01:41 -07:00
Yicheng Qin 2d01eb4e11 storage: add tests for kvstore_compaction 2015-08-26 15:01:13 -07:00
Yicheng Qin 0813139140 storage: add more tests for index 2015-08-26 12:53:30 -07:00
Yicheng Qin 3723f01b48 storage: add more unit tests for keyIndex 2015-08-26 12:53:30 -07:00
Yicheng Qin ad8a291dc1 storage: return error when tombstone on new generation
It is not allowed to put tombstone on an empty generation.
2015-08-26 12:53:30 -07:00
Yicheng Qin ffa87f9678 storage: fix the comment in generation.walk 2015-08-26 12:53:30 -07:00
Xiang Li e8f40b0412 storage/backend: add commitAndStop
After the upgrade of boltdb, db.Close waits for all txn to finish.
CommitAndStop commits the current txn and stop creating new ones.
2015-08-25 10:57:25 -07:00
Xiang Li e3ef1d363a Merge pull request #3366 from xiang90/v3_proto
update v3 proto and doc
2015-08-24 11:22:29 -07:00
Xiang Li 3a60d490d1 storagepb: fix comment location 2015-08-24 10:42:16 -07:00
Yicheng Qin 9c0c314425 storage: add functional tests for the package
It adds and reorganize tests to construct functional tests.
2015-08-23 20:59:06 -07:00
Yicheng Qin 9960651c3f storage: let range work in the process of txn
range should work in the process of txn to help check the status during the
txn.
2015-08-23 20:59:06 -07:00
Yicheng Qin 6d97dcaf3f storage: ensure that desired compaction is persisted
It needs to persist the desired compaction, so it won't forget the compaction
if it crashes later.
2015-08-23 20:59:06 -07:00
Yicheng Qin 353f10ca2b storage: reject to compact on future rev
Compaction on future rev is unreasonable.
2015-08-23 20:59:06 -07:00
Yicheng Qin 47b243be5d storage: let TxnDeleteRange return rev if no error
If it doesn't return error, it should return valid rev.
2015-08-23 20:59:06 -07:00
Yicheng Qin 62f7481b19 storage: keyIndex.get returns err when key is tombstoned
Before this commit, it will return wrong create index, mod index.

It lets findGeneration return error when rev is at the gap of two
generations. This leads to the change of compact() code.
2015-08-23 20:59:02 -07:00
Yicheng Qin 3b2fa9f1de storage: fix TestKeyIndexCompact
It fails to pass before.
2015-08-23 17:22:49 -07:00
Cong Ding c09b667d57 *: fix go vet reported issues 2015-08-22 12:19:02 -05:00
Xiang Li 6b23a8131f *: test gofmt with -s and fix reported issues 2015-08-21 18:52:16 -07:00
Tyler Neely acd7a92f03 storage: reversion -> revision 2015-08-20 08:39:07 -07:00
Xiang Li 53a77fa519 *: tnx -> txn 2015-07-24 23:21:09 +08:00
Xiang Li 883bb47dcf Merge pull request #3074 from xiang90/storage_restore
storage: correctly restore create and ver
2015-06-30 09:20:19 -07:00
Xiang Li f8b947a00b storage: fix small issues 2015-06-29 22:02:21 -07:00
Xiang Li 581ef05bab *: resolve proto warnings 2015-06-29 18:39:46 -07:00
Xiang Li 13f44e4b79 *: update generated proto code 2015-06-29 16:45:25 -07:00
Xiang Li 433f2ee1bc storage: correctly restore create and ver
Add a restore func to correctly restore create reversion and
version of keys for the index.
2015-06-29 13:44:43 -07:00
Xiang Li ccca2b04da storage: save version 2015-06-29 13:15:09 -07:00
Xiang Li c069119abe Merge pull request #3067 from xiang90/storage_created_mod
storage: save created index and modified index
2015-06-27 23:11:05 -07:00
Xiang Li 4581064060 storage: save created index and modified index 2015-06-26 12:10:26 -07:00
Yicheng Qin 5787fabe5f Merge pull request #3008 from yichengq/storage-index-test
storage: add range and tombstone test for index
2015-06-18 19:29:31 -07:00
Yicheng Qin b20598eea0 storage: add range and tombstone test for index 2015-06-18 18:05:37 -07:00
Yicheng Qin 9f2e4c8a57 storage: remove unnecessary ForceCommit in kvstore.Close
s.b.Close will commit pending ops, so there is no need to FroceCommit
it in kvstore.Close()
2015-06-18 13:36:23 -07:00
Yicheng Qin 789e2f3426 Merge pull request #3003 from yichengq/storage-kvstore-test
storage: add restore test and fix some bug
2015-06-18 12:19:05 -07:00
Yicheng Qin 7cba42fb73 storage: wait for compact goroutine to exit before close backend
If backend is closed, the operations on backend in compact
goroutine will panic. So this PR waits for compact goroutine to exit
before close backend.

This fixes the TestWorkflow failure too.
2015-06-18 12:18:39 -07:00
Yicheng Qin 148394f66f storage: fix schedule compaction bug in recover process
It uses wrong schedule compaction reversion before.
2015-06-18 11:11:37 -07:00
Yicheng Qin 26a09d8479 storage: enhance TestRestore and kill TODO 2015-06-18 10:37:12 -07:00
Yicheng Qin 0ef53ee500 Merge pull request #2999 from yichengq/storage-rev-test
storage: add reversion test
2015-06-18 07:39:18 -07:00
Yicheng Qin 74fbf9d6a7 storage: add reversion test 2015-06-17 18:06:42 -07:00
Yicheng Qin 80a59f00b7 storage: fix limit mismatch in Range func 2015-06-17 17:43:08 -07:00
Yicheng Qin 93f477944b storage: return ErrFutureRev if rev is a future one 2015-06-17 17:42:43 -07:00
Yicheng Qin 94924d04db storage: add TestRangeBadRev 2015-06-17 16:22:28 -07:00
Yicheng Qin 9ad5e1e64f storage: kill TODO in TestRange 2015-06-17 15:58:28 -07:00
Yicheng Qin 500894dfe5 storage: add TestWorkflow 2015-06-17 14:38:21 -07:00
Yicheng Qin 7b1a93e1ef storage: put storage info keys into information bucket
They used to be in key bucket, and make recover failed because they
cannot be parsed as normal key.
2015-06-17 14:37:29 -07:00
Xiang Li ba9a46aa02 storage: initial snapshot and restore
Snapshot takes an io.Writer and writes the entire backend data to
the given writer. Snapshot writes a consistent view and does not
block other storage operations.

Restore restores the in-memory states (index and book keeping) of
the storage from the backend data.
2015-06-10 11:32:10 -07:00
Xiang Li f47ed4a364 storage: initial compact 2015-06-05 09:22:44 -07:00
Xiang Li d417b36e5d storage: refactor key_index 2015-05-31 15:24:04 -07:00
Xiang Li 6c207b9277 stroage: kill todo 2015-05-27 14:46:59 -07:00
Xiang Li 69d02410cf stroage: adopt KV interface 2015-05-27 14:24:23 -07:00
Xiang Li cbb8b9bb08 stroage: add tnx id 2015-05-27 10:35:51 -07:00
Xiang Li 9c1aec6877 storage: add rangeKeys func 2015-05-27 09:58:21 -07:00