Commit Graph

44 Commits (bf9cccfc3400d6a33aaa73347276298a2366196f)

Author SHA1 Message Date
Anthony Romano e838c26f8a etcdctl: use snapshot RPC in snapshot command 2016-04-11 12:32:53 -07:00
Gyu-Ho Lee 953a08d841 *: clean up from gosimple 2016-04-08 11:55:03 -07:00
Xiang Li 2b54b73b90 backend: reset count in defrg 2016-04-02 17:25:05 -07:00
Anthony Romano 9b2c963179 etcdserver: configurable backend size quota
Configurable with the flag --experimental-quota-backend-bytes and
through ServerConfig.QuotaBackendBytes.

Fixes #4894
2016-03-29 18:39:25 -07:00
Anthony Romano ae077a2183 backend: add UnsafeForEach to BatchTx
Useful for efficiently iterating over an entire bucket.
2016-03-28 14:56:26 -07:00
Anthony Romano bd832e5b0a *: migrate Godeps to vendor/ 2016-03-22 17:10:28 -07:00
Xiang Li d883d5acd3 storage: support seq put to make db more compact 2016-03-15 16:36:36 -07:00
Anthony Romano 6650db53a4 Merge pull request #4733 from heyitsanthony/backend-alignment
storage/backend: align fields used for atomic ops
2016-03-09 11:42:55 -08:00
Anthony Romano dd01ab6dc0 storage/backend: align fields used for atomic ops
Fixes crashes on 32-bit tests.
2016-03-09 11:17:27 -08:00
Xiang Li 6cf4c4b3fd backend: fix TestBackendClose by giving more time to wait for io 2016-03-09 09:44:16 -08:00
Anthony Romano 5f304b4dee *: build phony etcd server binary for unsupported architectures
We don't qualify etcdserver for anything other than amd64, so don't
build binaries that are untested and might be unreliable.
2016-03-08 13:12:11 -08:00
Xiang Li 558640d91e backend: support shrink db 2016-03-02 14:35:28 -08:00
Anthony Romano c5b51946eb *: exported godoc fixups 2016-02-21 20:36:44 -08:00
Xiang Li 26c645f049 storage/backend: set initial db size to avoid potential deadlock 2016-02-05 11:29:16 -08:00
Anthony Romano 20461ab11a *: fix many typos 2016-01-31 21:42:39 -08:00
Xiang Li e1de19bf75 backend: make test more reliable 2016-01-13 22:00:25 -08:00
Xiang Li 43a777b7a2 *: get snapshot from backend
We should get snapshot from backend, not just KV.
We plan to store the lease data into backend too.
2016-01-06 22:40:23 -08:00
Xiang Li 5dd8af444a backend: create bucket should increase pending 2016-01-06 20:25:50 -08:00
Xiang Li 5dd3f91903 *: make backend outside kv
KV and lease will share the same backend. Thus we need to make
backend outside KV.
2016-01-05 19:55:29 -08:00
Alexander Kolbasov 152dcdd04d storage/backend: disable MAP_POPULATE on Solaris 2016-01-04 14:42:57 -08:00
Xiang Li 8bc59b66d1 backend: do not commit unless there is a pending change
Reduce the nubmer of fsync etcd issues when the cluster is
idle.
2015-12-23 18:58:37 -08:00
Sean Russell 928c0d3601 storage/backend: fixes Windows compile error
The type is "Options," not "Option" -- at least, in the vendored version of boltdb in the repository.
2015-12-16 11:10:46 -08:00
Gyu-Ho Lee 81229dbea9 *: add missing package descriptions
This adds and updates package descriptions in etcd projects.
And also deletes some duplicate LICENSE statements.
2015-11-17 20:54:10 -08:00
Gyu-Ho Lee 7638423f85 storage/backend: support MAP_POPULATE for unix
This adds build constraints in order to pass memory-map flags to bolt.Option.
If backend package passes syscall.MAP_POPULATE flag, the boltdb does read-ahead
which speeds up entire-database read, which then leads to faster storage
Restore. Benchmark result shows for 4GB, it opens and loads 6x faster in SSD,
and 12x faster in HDD. For 2GB, 1.6x faster with MAP_POPULATE in SSD.
2015-11-15 16:47:38 -08:00
Xiang Li 5efdd7bc6d storage/backend: avoid creating new bolt.tx during a batchTx 2015-10-17 21:31:34 -07:00
Xiang Li 0aa2f1192a storage: add metrics for db total size 2015-10-05 16:56:30 -07:00
Yicheng Qin 36f4303fc3 storage/etcdserver: update KV.Snapshot function
When using Snapshot function, it is expected:
1. know the size of snapshot before writing data
2. split snapshot-ready phase and write-data phase. so we could cut
snapshot first and write data later.

Update its interface to fit the requirement of etcdserver.
2015-10-03 10:15:23 -07:00
Xiang Li 385e17583f storage: fix hash by iterating kv 2015-09-23 11:28:33 -07:00
Yicheng Qin f7efbe8b14 storage/backend: extend wait timeout for commit to finish
It needs to take more time on travis. Fix:

```
--- FAIL: TestBackendBatchIntervalCommit (0.01s)
		backend_test.go:113: bucket test does not exit
```
2015-09-16 14:14:51 -07:00
Jonathan Boulle 7848ac3979 *: add missing license headers 2015-09-15 14:09:01 -07:00
Yicheng Qin 44fd734038 storage/backend: add unit tests for backend and batchTx 2015-09-02 16:57:13 -07:00
Yicheng Qin 4b9b0cbcc1 storage: add newBackend and newBatchTx
This is for ease of testing.
2015-08-31 13:25:10 -07:00
Yicheng Qin 054fab84ee storage/backend: remove startc var
This makes start logic cleaner.
2015-08-28 13:52:31 -07:00
Yicheng Qin f04884f74d storage/backend: fix off-by-one error for pending var
Or it may commit until batchLimit + 1.
2015-08-27 22:51:32 -07:00
Yicheng Qin 7ed929fb3d storage/backend: fix limit doesn't effect in range 2015-08-27 22:51:32 -07:00
Matt Keller 19a28c8efd storage: Fixed backend test
./backend_test.go:23: multiple-value batchTx.UnsafeRange() in single-value context
2015-08-27 15:20:29 -04:00
Xiang Li e8f40b0412 storage/backend: add commitAndStop
After the upgrade of boltdb, db.Close waits for all txn to finish.
CommitAndStop commits the current txn and stop creating new ones.
2015-08-25 10:57:25 -07:00
Xiang Li 53a77fa519 *: tnx -> txn 2015-07-24 23:21:09 +08:00
Xiang Li ba9a46aa02 storage: initial snapshot and restore
Snapshot takes an io.Writer and writes the entire backend data to
the given writer. Snapshot writes a consistent view and does not
block other storage operations.

Restore restores the in-memory states (index and book keeping) of
the storage from the backend data.
2015-06-10 11:32:10 -07:00
Xiang Li f47ed4a364 storage: initial compact 2015-06-05 09:22:44 -07:00
Xiang Li fb12a4e412 storage: fix a deadlock in batch tx 2015-05-27 09:31:11 -07:00
Xiang Li 49da7b6556 storage: add boltdb as dependency 2015-05-27 09:24:49 -07:00
Xiang Li e332e86b5d storage: address barak's comments 2015-05-20 17:47:35 -07:00
Xiang Li 4b0d9f69c7 storage: add a simple backend and kv example 2015-05-14 20:43:32 -07:00