Commit Graph

103 Commits (1d02559ae5a0637499368243cd948cc7a1f1c90e)

Author SHA1 Message Date
Gyu-Ho Lee c1851dfca1 etcd-tester: add leader failure cases 2016-02-11 15:46:44 -08:00
Gyu-Ho Lee 78df258ea8 etcd-agent: cleans page cache when cleaning up
Reference:
- https://www.kernel.org/doc/Documentation/sysctl/vm.txt
- https://github.com/torvalds/linux/blob/master/fs/drop_caches.c
2016-02-11 13:48:54 -08:00
Xiang Li 4b68977851 Merge pull request #4481 from xiang90/testing-tool
tools/etcd-dump-logs: support parsing v3 log
2016-02-10 14:00:32 -08:00
Xiang Li 3611a9ad2e tools/etcd-dump-logs: support parsing v3 log 2016-02-10 13:52:02 -08:00
Gyu-Ho Lee 7b2ce70783 etcd-tester: count success for v3
needed for snapshot count comparison
2016-02-09 17:12:03 -08:00
Gyu-Ho Lee 7d2b7e0d23 etcd-tester: close leaky gRPC connections
when closed errors will be one of:

```
grpc.ErrorDesc(err) == context.Canceled.Error() ||
grpc.ErrorDesc(err) == context.DeadlineExceeded.Error() ||
grpc.ErrorDesc(err) == "transport is closing" ||
grpc.ErrorDesc(err) == "grpc: the client connection is closing"
```
2016-02-09 16:26:33 -08:00
Gyu-Ho Lee 16aa263d8c etcd-tester: continue after cleanup 2016-02-09 13:32:39 -08:00
Gyu-Ho Lee c8994aab62 Merge pull request #4466 from gyuho/f0
etcd-tester: close gRPC connection when canceling
2016-02-09 11:07:00 -08:00
Gyu-Ho Lee a46e20f92a etcd-tester: close gRPC connection when canceling
Currently gRPC connection just gets recreated
for every Stress call. When Stress ends or gets
canceled, gRPC connection must also be closed.

For https://github.com/coreos/etcd/issues/4464.
2016-02-09 11:04:16 -08:00
Gyu-Ho Lee e252c0c0ca etcd-tester: fix wrong error checking
Hash method returns either (nil, err) or (Hash, nil).
The current error checking is wrong. It only needs to check
the error is either nil or non-nil.

This causes panic in https://github.com/coreos/etcd/issues/4463
by allowing the case when resp is nil, but err is not nil.
2016-02-09 10:00:05 -08:00
Xiang Li fa71bec550 Merge pull request #4458 from xiang90/cl
etcd-tester: cleanup
2016-02-08 21:16:28 -08:00
Xiang Li 25834211a9 etcd-tester: cleanup 2016-02-08 20:54:25 -08:00
Gyu-Ho Lee 77f753ac32 etcd-agent: mkdir with read/write to all users 2016-02-08 20:50:11 -08:00
Gyu-Ho Lee 16543778f1 etcd-tester: avoid directory name conflict 2016-02-08 13:38:26 -08:00
Gyu-Ho Lee 08dbabdb5f etcd-tester: add compactKV
It compacts storage for every case.
For https://github.com/coreos/etcd/issues/4380.
2016-02-06 15:02:58 -08:00
Gyu-Ho Lee 044f7775e3 etcd-tester: log cancel in stresser
And fix some minor print formats.
2016-02-05 18:31:27 -08:00
Gyu-Ho Lee 09fc764552 functional-tester/etcd-tester: silent grpclog, check revs 2016-02-05 14:04:58 -08:00
Gyu-Ho Lee e2b5b1cd1a functional-tester/etcd-agent: configurable log path 2016-02-05 09:37:24 -08:00
Gyu-Ho Lee 8e1325d9e1 functional-tester/etcd-agent: configurable agent port
To make local testing easier.
2016-02-04 21:10:30 -08:00
Gyu-Ho Lee 220d0c3c14 etcd-agent: fix data-dir path check
Need one more dash to match 'data-dir' flag.
2016-02-04 16:53:01 -08:00
Gyu-Ho Lee 85a4a5b596 functional-tester: fix grpc endpoint, consistent check
And some clean ups.
2016-02-04 15:10:43 -08:00
Gyu-Ho Lee 7a3426a231 tools/functional-tester/etcd-tester: support v3 kv storage 2016-02-03 15:24:54 -08:00
Gyu-Ho Lee ed682c9f08 tools/functional-tester: minor cleanup 2016-02-03 14:30:34 -08:00
Anthony Romano b74a42b286 clientv3: support unix endpoints 2016-02-01 22:24:42 -08:00
Anthony Romano 4380617e1a tools/benchmark: support tls 2016-01-29 16:38:11 -08:00
Heungsub Lee c4a0159601 documentation: fix typo "a etcd" -> "an etcd"
"a" is not a correct article for "etcd".
2016-01-28 19:19:32 +09:00
Anthony Romano a7b6bbff3f tools/benchmark: use clientv3 2016-01-27 12:13:17 -08:00
Gyu-Ho Lee fa21946267 *: pass WithInsecure to grpc.Dial for now
Related to https://github.com/coreos/etcd/issues/4299.
2016-01-27 00:24:03 -08:00
Gyu-Ho Lee 652c01bffe tools/benchmark: update gRPC, proto interface 2016-01-26 17:41:32 -08:00
Xiang Li 6c82d768b2 Merge pull request #4201 from mitake/benchmark-pprof
tools/benchmark: add flags for pprof to storage put
2016-01-13 20:17:30 -08:00
Hitoshi Mitake 1c802e9266 tools/benchmark: add flags for pprof to storage put
This commit adds flags for profiling with runtime/pprof to storage
put:
- --cpuprofile: specify a path of CPU profiling result, if it is not
    empty, profiling is activated
- --memprofile: specify a path of heap profiling result, if it is not
    empty, profiling is activated

Of course, the flags should be added to RootCmd ideally. However,
adding common flags that shared by children command requires the
ongoing PR: https://github.com/spf13/cobra/pull/220 . Therefore this
commit adds the flags to storage put only.
2016-01-14 13:10:35 +09:00
Hitoshi Mitake 16b63310b2 tools/benchmark: remove deadcode
The Execute() function is a deadcode. Let's remove it.
2016-01-13 16:57:53 +09:00
Xiang Li f5753f2f51 *: support lease Attach
Now we can attach keys to leases. And revoking the lease removes all
the attached keys of that lease.
2016-01-09 11:01:58 -08:00
Gyu-Ho Lee a7287b6374 Merge pull request #4161 from gyuho/typo_in_benchmark
tools/benchmark/cmd: print error out to stderr
2016-01-07 18:50:09 -08:00
Anthony Romano 187064187c tools/benchmark: report standard deviation 2016-01-07 15:30:23 -08:00
Gyu-Ho Lee cc39cde319 tools/benchmark/cmd: print error out to stderr 2016-01-07 12:55:26 -08:00
Anthony Romano f9af744be3 lease: move storage.NoLease to lease package 2016-01-07 10:33:44 -08:00
Xiang Li 5dd3f91903 *: make backend outside kv
KV and lease will share the same backend. Thus we need to make
backend outside KV.
2016-01-05 19:55:29 -08:00
Xiang Li 4336278b44 *: support put with lease 2016-01-04 15:54:06 -08:00
Xiang Li ac330bb7c9 *: update watch related proto
1. Add watch/cancel request
2. Add necessary fields in response to return watch error
3. Add watch_id into watch response
2016-01-01 10:22:21 -08:00
Xiang Li a74147384d Merge pull request #4070 from mitake/storage-bench
tools: a new tool for benchmarking storage backends
2015-12-31 22:26:46 -08:00
Hitoshi Mitake 454865bd67 tools: a new tool for benchmarking storage backends
Current etcd repository has a test for benchmarking a storage backend
in storage/kvstore_bench_test.go. However, it is hard to test various
parameters (e.g. batch interval, a number of keys, etc) with the test.

This commit adds a new benchmarking subcommand "storage" to
tools/benchmark. It will encourage analysis of storage backends with
various parameter and complex workloads.

Exmaple usage:
$ ./benchmark storage put
total: 9.894173792s
average: 9.894173ms
minimum latency: 6.596991ms
maximum latency: 29.455695ms
2016-01-01 15:18:54 +09:00
Anthony Romano 1689bb3f02 tools/benchmark: support puts on sequential keys over a bounded keyspace
This patch makes it possible to use benchmark to generate n keys and
make random updates to only those n keys.
2015-12-30 16:47:24 -08:00
Hitoshi Mitake b97f78d356 tools/benchmark: ignore the binary "benchmark" 2015-12-30 16:02:52 +09:00
Anthony Romano 8e728afa62 tools/benchmark: support connecting to several endpoints
--endpoints is comma separated but gRPC blocks forever on comma
separated lists. Instead, round-robin select endpoints when
creating new connections.
2015-12-28 10:22:33 -08:00
Anthony Romano 382103af60 tools/benchmark: stream results into reports
Reports depended on writing all results to a large buffered channel and
reading from that synchronously. Similarly, requests were buffered the
same way which can take significant memory on big request strings. Instead,
have reports stream in results as they're produced then print when the
results channel closes.
2015-12-23 11:24:35 -08:00
Pawel Palucki 4f5f999847 tools/functional-test: add docker support
Commit adds docker bits to run functional tester within containers.

requires:
- docker 1.9 (networking)
- docker-compose
2015-12-18 15:56:18 +01:00
Gyu-Ho Lee 4289871cef tools/function-tester: fix shadowed variables
Fixes for https://github.com/coreos/etcd/issues/3954.
2015-12-12 09:38:20 -08:00
Xiang Li 8d4073d078 Merge pull request #3945 from xiang90/new_watch_bench
tools/benchmark: add watch subcommand.
2015-12-03 16:56:02 -08:00
Xiang Li c6430b3292 tools/benchmark: add watch subcommand.
Watch command run benchmark tests for watch releated operations:
1. watch keys 2. sending events to watchers.

To test 2, this test also puts keys into etcd cluster to trigger
event sending.

Besides the benchmark results showed at client side, the tester
can also monitor the server-side mem/cpu usage and also check
the metrics of slow watchers. If there are a lot of slow watchers,
it means etcd server is over-loaded by watchers.
2015-12-03 15:56:17 -08:00