Gyu-Ho Lee
e8adc24c32
functional-tester: build from repo root, vendor
2016-09-21 02:06:13 -07:00
Anthony Romano
d4eff5381c
etcd-tester: split out consistency checking code from tester
2016-09-20 10:26:58 -07:00
fanmin shi
8a63071463
etcd-tester: fix peer-port parsing bug with localhost url
...
The following format "http://localhost:1234 " causes existing port parser to fail. Add new logic to parse the host name first then extract port.
Fixes #6409
2016-09-12 13:29:52 -07:00
fanmin shi
a571655983
functional-tester: correct goreman command in readme
...
update readme file to have the correct goreman command to start the functional tester locally.
2016-09-09 14:56:23 -07:00
Anthony Romano
aa6b1e6a10
functional-tester: add Procfile
2016-09-08 16:35:55 -07:00
Anthony Romano
55ba3d95fb
etcd-tester: support per-agent client/peer/failpoint ports
2016-09-08 16:15:18 -07:00
Anthony Romano
2e25a772a5
etcd-agent: support rootless operation and configurable gofail ports
2016-09-08 16:12:00 -07:00
Gyu-Ho Lee
0bf2ef3c1b
etcd-tester: return error for mismatch rev/hash
2016-08-27 22:14:42 -07:00
Xiang Li
017aac88a8
etcd-tester: do not restart stresser on error
2016-08-27 20:47:45 -07:00
Gyu-Ho Lee
0c37df7265
etcd-tester: fix compact rev counting
2016-08-22 22:58:44 -07:00
Anthony Romano
47b6449934
functional-tester: put large keys
...
For testing writes that must span multiple pages.
2016-08-17 09:51:44 -07:00
Xiang Li
feaff17259
session: remove session manager and add ttl
2016-08-15 14:12:25 -07:00
Gyu-Ho Lee
c6cf015e26
etcd-tester: fix tester for 5-node cluster
...
1. fix failure case counting
2. match ErrClientConnClosing in stresser
3. longer timeout for set-health-key
4. fixed range for range/delete stresser
5. remove Limit in RangeRequest
2016-08-07 21:15:01 -07:00
Anthony Romano
978ae9de29
functional-tester: share limiter among stresser
...
Otherwise, adding more members stresses the cluster with more ops.
2016-08-07 19:15:00 -07:00
Xiang Li
7678b84f2c
Merge pull request #6123 from xiang90/fix_limiter
...
tools/functional-tester: fix limiter
2016-08-07 16:20:17 -07:00
Xiang Li
f6a1585902
functional-tester: reduce rate to 3000
2016-08-07 14:34:01 -07:00
Xiang Li
107a07563f
tools/functional-tester: fix limiter
2016-08-07 14:28:16 -07:00
Xiang Li
69204397ee
tools/functional-tester: better logging
2016-08-07 14:21:44 -07:00
Gyu-Ho Lee
854a215329
etcd-tester: delete/range with limit, clean up
2016-08-05 11:21:36 -07:00
Gyu-Ho Lee
755eee0d30
etcd-tester: remove unnecessary arg from stresser
2016-08-01 15:35:31 -07:00
Gyu-Ho Lee
969bcd282b
etcd-tester: stress with range, delete
2016-07-18 15:17:08 -07:00
Anthony Romano
0419d3ecf7
etcd-tester: add FailFast(false) to grpc calls
2016-07-14 19:16:41 -07:00
Gyu-Ho Lee
ceb9fe4822
etcd-tester: stop stress before compact, fix races
...
fix race condition between stresser cancel, start
2016-07-14 12:16:42 -07:00
Gyu-Ho Lee
8e59fb749c
etcd-tester: increase default qps, fix cleanup
2016-07-14 11:20:16 -07:00
Xiang Li
7114a27345
Merge pull request #5922 from xiang90/l_b
...
tools/benchmark: add benchmark for lease keepalive
2016-07-12 10:48:04 -07:00
Xiang Li
8273e1c07e
tools/benchmark: add benchmark for lease keepalive
2016-07-12 10:40:56 -07:00
Xiang Li
4bc29e2b9c
Merge pull request #5902 from mitake/bench-auth
...
tools: add --user for auth in benchmarks
2016-07-11 18:38:07 -07:00
Hitoshi Mitake
a153448b84
tools: add --user for auth in benchmarks
...
This commit adds --user for auth in benchmarks. Its purpose is
measuring overhead of authentication of v3 API. Of course the given
user must be granted permission of target keys before benchmarking.
Example of a case with no authentication:
% ./benchmark range k1
bench with linearizable range
10000 / 10000 Booooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00%2m10s
Summary:
Total: 130.1850 secs.
Slowest: 0.4071 secs.
Fastest: 0.0064 secs.
Average: 0.0130 secs.
Stddev: 0.0079 secs.
Requests/sec: 76.8138
Response time histogram:
0.006 [1] |
0.046 [9990] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
0.087 [3] |
0.127 [0] |
0.167 [3] |
0.207 [2] |
0.247 [0] |
0.287 [0] |
0.327 [0] |
0.367 [0] |
0.407 [1] |
Latency distribution:
10% in 0.0076 secs.
25% in 0.0086 secs.
50% in 0.0113 secs.
75% in 0.0146 secs.
90% in 0.0209 secs.
95% in 0.0272 secs.
99% in 0.0344 secs.
Example of a case with authentication:
% ./benchmark --user=u1:p range k1
bench with linearizable range
10000 / 10000 Booooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00%2m11s
Summary:
Total: 131.4923 secs.
Slowest: 0.1637 secs.
Fastest: 0.0065 secs.
Average: 0.0131 secs.
Stddev: 0.0070 secs.
Requests/sec: 76.0501
Response time histogram:
0.006 [1] |
0.022 [9075] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
0.038 [875] |∎∎∎
0.054 [36] |
0.069 [5] |
0.085 [1] |
0.101 [1] |
0.117 [0] |
0.132 [0] |
0.148 [5] |
0.164 [1] |
Latency distribution:
10% in 0.0076 secs.
25% in 0.0087 secs.
50% in 0.0114 secs.
75% in 0.0150 secs.
90% in 0.0215 secs.
95% in 0.0272 secs.
99% in 0.0347 secs.
It seems that current auth mechanism does not introduce visible overhead.
2016-07-08 16:53:05 +09:00
Gyu-Ho Lee
7c39f41e7c
etcd-tester: add rate limiter to stresser
2016-07-07 21:55:12 -07:00
Anthony Romano
a5f043c85b
etcd-tester: add failpoint cases
...
Fixes #5754
2016-07-01 15:31:49 -07:00
Anthony Romano
8d4701bb1d
etcd-agent: enable GOFAIL_HTTP endpoint
2016-07-01 14:39:48 -07:00
Gyu-Ho Lee
0d44947c11
etcd-tester: fix slow leader with injectLatency
2016-06-30 13:41:27 -07:00
Gyu-Ho Lee
dc49016987
etcd-tester: handle error in RevHash
2016-06-29 19:31:45 -07:00
Anthony Romano
5f459a64ce
etcd-tester: refactor cluster member handling
2016-06-29 11:25:33 -07:00
Anthony Romano
402df5bd03
etcd-tester: refactor failure code to reduce code duplication
2016-06-29 11:03:34 -07:00
Anthony Romano
63f78bf7c8
etcd-tester: refactor round loop
2016-06-29 11:03:34 -07:00
Gyu-Ho Lee
15f2fd0726
etcd-tester: match ErrTimeout in stresser
...
Fix https://github.com/coreos/etcd/issues/5804 .
2016-06-28 19:20:28 -07:00
Gyu-Ho Lee
ae057ec508
benchmark: fix Compact request
2016-06-28 13:58:28 -07:00
Gyu-Ho Lee
ac68f70843
etcd-tester: match ErrTimeoutDueToLeaderFail
...
stresser in followers should retry when failure is injected to
their leader.
2016-06-27 20:48:06 -07:00
Anthony Romano
244266708b
local-tester: support failpoints
2016-06-23 12:04:11 -07:00
Gyu-Ho Lee
a1c7a7df5e
*: use capnslog for grpclog
2016-06-20 20:35:03 -07:00
Xiang Li
2bd5d66596
benchmark: fix watch bench
2016-06-20 14:00:46 -07:00
Gyu-Ho Lee
50f2f984e4
etcd-agent: set up directory for etcd logs
2016-06-20 11:32:14 -07:00
Xiang Li
2a2dd1075f
benchmark: correctly count number of watchers
2016-06-20 10:37:17 -07:00
Anthony Romano
9feb3d0e51
etcd-tester: fix goword warnings
2016-06-17 13:37:35 -07:00
Anthony Romano
f7b84d69a4
etcd-agent/client: fixup godocs
2016-06-17 13:37:35 -07:00
Gyu-Ho Lee
63c13e8b98
etcd-agent: fix test
2016-06-17 10:47:15 -07:00
Anthony Romano
5385ca0a43
Merge pull request #5659 from heyitsanthony/bridge-more-errors
...
bridge: packet corruption and reordering
2016-06-15 19:23:22 -07:00
Anthony Romano
11869905ae
bridge: packet corruption and reordering
...
With bonus bridge connection code refactor.
2016-06-15 17:08:19 -07:00
Gyu-Ho Lee
bd604a029e
etcd-agent: SIGQUIT when cleanup
2016-06-15 16:03:25 -07:00
Gyu-Ho Lee
84953365a2
etcd-tester: cleanup in compact error, log level
2016-06-13 14:54:53 -07:00
Gyu-Ho Lee
5631acdb8f
etcd-tester: do not exit for compact timeout
...
Temporary fix for https://github.com/coreos/etcd/issues/5606 .
2016-06-10 09:44:45 -07:00
Gyu-Ho Lee
350673f1f8
etcd-tester: retry for 'etcdserver: not capable'
...
Fix https://github.com/coreos/etcd/issues/5573 .
Currently stresser starts at the same time as cluster start.
If the stresser got launched too fast/early, all stressers
exit from the error 'etcdserver: not capable', which
means the cluster is not ready yet. This adds additional
error checking, so stresser can retry.
2016-06-07 21:56:04 -07:00
Anthony Romano
51551abef5
concurrency, benchmark: read-committed STM isolation policy
2016-05-31 14:35:27 -07:00
Anthony Romano
f34a9350c3
benchmark: benchmark stm workload with distributed mutex
2016-05-31 14:35:27 -07:00
Anthony Romano
bb2a3ea8d8
benchmark: respect stm isolation mode flag
2016-05-31 14:35:27 -07:00
Gyu-Ho Lee
20fc3e968f
Merge pull request #5465 from gyuho/compact1
...
etcd-tester: log more for compact errors
2016-05-27 16:16:04 -07:00
Anthony Romano
fc7da09d67
*: add missing godoc package descriptions
...
Fixes #4074
2016-05-27 15:15:26 -07:00
Gyu-Ho Lee
04039eb006
etcd-tester: more logs for compact operations
2016-05-27 09:55:13 -07:00
Gyu-Ho Lee
3ed5d28e2e
etcd-tester: fix, clean up multiple things ( #5462 )
...
* etcd-tester: more logging, fix typo
* etcd-tester: fix prevCompactRev scope
Fix https://github.com/coreos/etcd/issues/5440 .
* etcd-tester: move utils to bottom, clean up logs
And remove stresser operation inside defrag
* etcd-tester: separate update revision call
* etcd-tester: fix cleanup when case is -1
2016-05-26 11:37:49 -07:00
Gyu-Ho Lee
4c7af825c7
etcd-tester: timeout per number of compact entries
...
Fix https://github.com/coreos/etcd/issues/5440 .
2016-05-25 18:37:13 -07:00
Gyu-Ho Lee
9ca84e814f
benchmark: fix watch command
...
Fix https://github.com/coreos/etcd/issues/5099 .
2016-05-19 09:57:35 -07:00
Anthony Romano
393725fe5f
clientv3/concurrency: ctx-ize Leader(), Resign(), and Unlock()
2016-05-13 10:07:35 -07:00
Anthony Romano
2e93c65c96
bridge: fix command line flag handling
...
flag package expects flags in Argv[1:] and stops on non-flag arguments
but bridge was expecting the forwarding address in os.Argv[1]
2016-05-13 10:07:35 -07:00
Anthony Romano
4612e2d59a
Merge pull request #5340 from heyitsanthony/etcd-runner-election
...
etcd-runner: election mode
2016-05-12 22:53:35 -07:00
Anthony Romano
4fe91ed1e2
etcd-runner: election mode
2016-05-12 22:32:33 -07:00
Anthony Romano
215afb9b1d
etcd-runner: refactor round code
2016-05-12 22:32:33 -07:00
Gyu-Ho Lee
3d523e34b1
tools: update LICENSE header
2016-05-12 20:50:17 -07:00
Anthony Romano
dcb3b7aecf
*: scrub legacy ports from code and scripts
2016-05-11 13:46:30 -07:00
Anthony Romano
f5165a0149
benchmark: make number of watcher streams configurable in watch-get
...
Each stream uses a client goroutine and a grpc stream; the setup causes
considerable client-side latency on the first get requests.
2016-05-11 11:16:43 -07:00
Anthony Romano
2aa4dd52cc
benchmark: use separate connection for get in watch-get
...
The watcher traffic interferes with the get latency when sharing connections.
2016-05-11 11:16:43 -07:00
Anthony Romano
467de8cb4f
benchmark: watch-get for testing unsynced watcher/get contention
2016-05-10 10:24:40 -07:00
Gyu-Ho Lee
cc7dd9b729
etcd-tester: refactor
2016-05-05 10:55:42 -07:00
Xiang Li
28f3cb0f14
Merge pull request #5171 from xiang90/runner
...
etcd-runner: initial commit
2016-04-30 19:39:53 -07:00
Xiang Li
73ecb61ff4
etcd-runner: initial commit
2016-04-30 17:24:03 -07:00
Anthony Romano
c0ff77e809
local-tester: procfile, faults, and network bridge
...
Creates a local fault injected cluster and stresser for etcd.
Usage: goreman -f tools/local-tester/Procfile start
2016-04-29 09:57:02 -07:00
Gyu-Ho Lee
de7c18909f
etcd-tester: match more grpc errors
...
To prevent stressers from returning from failure injections
2016-04-27 09:34:05 -07:00
Anthony Romano
b7ac758969
*: rename storage package to mvcc
2016-04-25 15:25:51 -07:00
Gyu-Ho Lee
151d0d3831
etcd-tester: flag consistency-check
2016-04-22 22:22:12 -07:00
Gyu-Ho Lee
6a0664d701
etcd-tester: wait more for slow network recovery
...
For https://github.com/coreos/etcd/issues/5121 .
2016-04-22 15:24:47 -07:00
Gyu-Ho Lee
0feb88cee1
etcd-tester: change var success->failed
...
Previous success overwrites the later failure.
Make it simpler by changing the variable to 'failed'.
2016-04-22 09:27:37 -07:00
Gyu-Ho Lee
e835d24bea
etcd-tester: skip compaction after different hash
...
When hashes don't match, there could be some nodes
falling behind and the compact request can then error
with 'future revision compact'.
2016-04-20 17:13:51 -07:00
Gyu-Ho Lee
ad226f2020
benchmark: ensure all watcher receivers to finish
...
Fix https://github.com/coreos/etcd/issues/5099 .
2016-04-15 11:11:14 -07:00
Gyu-Ho Lee
641a1a66e1
*: fix govet -shadow in go tip
2016-04-15 07:39:52 -07:00
Gyu-Ho Lee
8515ae30fb
functional-tester/etcd-tester: fix error check
2016-04-14 21:31:12 -07:00
Anthony Romano
21233416e8
etcd-tester: return error if first compaction times out
...
Fixes #5081
2016-04-14 17:11:53 -07:00
Gyu-Ho Lee
6af40ea1e1
functional-tester: add latency test (simulate slow network)
...
Fix https://github.com/coreos/etcd/issues/4666 .
2016-04-13 17:00:09 -07:00
Gyu-Ho Lee
2c6176b5f2
*: remove MemberLeader API in client side (fix examples)
2016-04-13 16:23:57 -07:00
Gyu-Ho Lee
cb280bae91
etcd-tester: fix govet
2016-04-13 11:12:31 -07:00
Gyu-Ho Lee
2aab6ff2eb
benchmark: return time series with missing periods filled in
2016-04-11 23:07:45 -07:00
Gyu-Ho Lee
953a08d841
*: clean up from gosimple
2016-04-08 11:55:03 -07:00
Gyu-Ho Lee
139f23fd13
etcd-agent: fix etcd agent tests, remove unused listener
2016-04-07 17:04:24 -07:00
Gyu-Ho Lee
b866337f25
functional-tester: defrag every 500 round
...
Fix https://github.com/coreos/etcd/issues/4665 .
2016-04-02 18:51:26 -07:00
Anthony Romano
307cb5167c
Merge pull request #4925 from heyitsanthony/wal-dump-lock
...
etcd-dump-logs: don't try to acquire wal file locks
2016-03-31 22:24:54 -07:00
Anthony Romano
7fffd6ffd2
etcd-dump-logs: don't try to acquire wal file locks
...
can now dump logs from a running etcd instance
2016-03-31 21:51:20 -07:00
Gyu-Ho Lee
f9b90e13ac
etcd-tester: no error for compact double-send
...
When compactKV request is halted before final acknowledgement,
it used to just continue on the next endpoint. But there could be
a case than compactKV is requested twice, and the first one is already
replicated and applied by the time the second request was to be
applied (returning compact revision error). This skips the case
by parsing the error message.
2016-03-31 21:29:02 -07:00
Xiang Li
14f146b9f7
Merge pull request #4908 from xiang90/c
...
*: simplify consistent index handling
2016-03-30 13:53:21 -07:00
Xiang Li
eddc741b5e
*: simplify consistent index handling
2016-03-30 13:38:28 -07:00
Anthony Romano
4c65f3fe7a
etcd-tester: check compaction revision
...
Faster than waiting 30 seconds between rounds.
2016-03-30 09:45:30 -07:00
Anthony Romano
3fbacf4be2
v3rpc: move Hash RPC to Maintenance service
2016-03-28 17:15:58 -07:00
Anthony Romano
4a17097d00
tools/functional-tester: --schedule-cases flag
...
Command line argument for specifying a schedule of test cases per round.
Default is run each test case once each round.
2016-03-24 19:43:23 -07:00
Xiang Li
d0d3b32210
Merge pull request #4850 from xiang90/rm_demo
...
*: enable v3 by default
2016-03-23 23:48:29 -07:00
Xiang Li
70a9391378
*: enable v3 by default
2016-03-23 17:01:36 -07:00
Gyu-Ho Lee
9fb60deb7c
functional-tester: add GRPCURLs for cluster config
...
GRPC and v2 client address share the same host(port)
but GRPC does not work with schema specified. This fixes
it by adding another member for GRPC without schema, as
we had before.
2016-03-23 16:28:05 -07:00
Gyu-Ho Lee
012143e703
functional-tester: set gRPC endpoint for stresser
2016-03-23 15:23:19 -07:00
Anthony Romano
bd832e5b0a
*: migrate Godeps to vendor/
2016-03-22 17:10:28 -07:00
Gyu-Ho Lee
bb9a7f5a7c
Godeps: semantic versioning cheggaaa/pb
...
Fix https://github.com/coreos/etcd/issues/4832 .
2016-03-21 22:06:16 -07:00
Xiang Li
d3809abe42
*: gRPC + HTTP on the same port
...
We use cmux to do this since we want to do http+https on the same
port in the near future too.
2016-03-21 14:29:25 -07:00
Anthony Romano
1383da1030
benchmark: STM benchmark
2016-03-20 12:21:29 -07:00
Gyu-Ho Lee
58792ca59b
bench/cmd: print csv-format timeseries
2016-03-16 11:06:36 -07:00
Gyu-Ho Lee
c9e4e2b6dc
benchmark: move sample flag to root command
...
Sample is configuration for reports. This should be
flag at top command.
2016-03-15 10:36:27 -07:00
Gyu-Ho Lee
5eefff12e1
benchmark: printSecondSample with time series
2016-03-15 08:35:03 -07:00
Gyu-Ho Lee
a932674a5b
benchmark: minor typos
2016-03-14 13:45:08 -07:00
Anthony Romano
78132c9b5b
clientv3: use tls.Config in clientv3.Config
...
Fixes #4648
2016-03-07 16:08:40 -08:00
Gyu-Ho Lee
b8912c9fb1
benchmark: change complete notifier first
...
Fix https://github.com/coreos/etcd/issues/4708 .
2016-03-07 14:54:11 -08:00
Gyu-Ho Lee
b3f0bcbeb4
functional-tester: less intensive stresser
2016-03-05 13:16:26 -08:00
Gyu-Ho Lee
b73c1223d8
Merge pull request #4638 from gyuho/compact
...
benchmark: add auto-compact flags
2016-03-04 10:15:02 -08:00
Gyu-Ho Lee
da0f77dc14
benchmark: measure Put with auto-compact
2016-03-04 09:34:24 -08:00
Gyu-Ho Lee
536b028831
benchmark: watch, key-space-size(max possible key)
...
By specifying 'key-space-size', we can test min/max-key-range
of keys to watch.
For https://github.com/coreos/etcd/issues/3863 .
2016-03-03 13:46:17 -08:00
Anthony Romano
4fb25d5f0e
Merge pull request #4613 from heyitsanthony/clientv3-composite
...
clientv3: compose API interfaces into client struct
2016-02-29 11:23:34 -08:00
Gyu-Ho Lee
64e276800f
benchmark: use endpoints for benchmark flag
2016-02-26 16:55:49 -08:00
Anthony Romano
298c1e2487
tools/benchmark: port to clientv3 API
2016-02-25 18:13:26 -08:00
Xiang Li
e316678a4d
tools/functional-tester: fix leaky issue by closing conn
2016-02-19 22:04:01 -08:00
Gyu-Ho Lee
8bcd823ed7
etcd-tester: continue for deadline exceeded
2016-02-19 13:48:58 -08:00
Gyu-Ho Lee
b2bad7bd79
etcd-agent: get base when renaming
...
Partially related to https://github.com/coreos/etcd/issues/4552 .
2016-02-18 09:03:34 -08:00
Gyu-Ho Lee
061e996998
functional-tester: remove log prefixes
...
capnslog already prefixes with its package name.
2016-02-17 19:01:16 -08:00
Gyu-Ho Lee
7aa62ec595
functional-tester: plog for milli-second timestamp
...
Standard log package by default only prints out the second-scale
so the 3rd party log feeder mixes the order of the events, which makes
the debugging hard. This replaces it with capnslog and make them consistent
with all other etcd log formats.
2016-02-17 18:39:05 -08:00
Gyu-Ho Lee
239a6d89c5
etcd-tester: 10-second timeout for stressers
...
For https://github.com/coreos/etcd/issues/4477 .
2016-02-17 15:44:05 -08:00
Gyu-Ho Lee
56e3ab0943
etcd-agent: tidy cleanup before SIGKILL
...
https://github.com/golang/go/blob/master/src/os/exec_posix.go#L18 shows that
cmd.Process.Kill calls syscall.SIGKILL to the command. But
http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_12_01.html explains
'If you send a SIGKILL to a process, you remove any chance for the process to
do a tidy cleanup and shutdown, which might have unfortunate consequences.'
This sends SIGTERM, SIGINT syscalls to the PID so that the process could
have more time to clean up the resources.
Related to https://github.com/coreos/etcd/issues/4517 .
2016-02-16 13:08:22 -08:00
Gyu-Ho Lee
20ac633059
Merge pull request #4516 from gyuho/path
...
etcd-agent: ignore error when no file to rename
2016-02-13 14:24:17 -08:00
Xiang Li
cff1208005
Merge pull request #4520 from xiang90/bench
...
tools/benchmark: support serializable range benchmark
2016-02-13 09:25:24 -08:00
Xiang Li
4d0a2b4215
Merge pull request #4462 from mitake/benchmark-watch
...
tools/benchmark: revive watch benchmark
2016-02-13 09:24:00 -08:00
Xiang Li
3a9a1c7d9b
tools/benchmark: support serializable range benchmark
2016-02-13 09:23:35 -08:00
Hitoshi Mitake
99e7449f44
tools/benchmark: revive watch benchmark
...
Current watch benchmark seems to be broken. This commit revives it.
2016-02-14 01:15:04 +09:00
Gyu-Ho Lee
205033d25f
etcd-agent: ignore error when no file to rename
...
Fixes https://github.com/coreos/etcd/issues/4512 .
When cluster fails before creation of log or data directory
the file does not exist and cannot be renamed. This skips such
error because there's no need to store empty logs in failure_archive.
2016-02-12 16:37:04 -08:00
Xiang Li
c15b2a5077
Merge pull request #4514 from xiang90/tester
...
tools/functional-tester: add metrics
2016-02-12 14:45:10 -08:00
Xiang Li
1bec0e6a0b
tools/functional-tester: add metrics
2016-02-12 14:42:41 -08:00
Gyu-Ho Lee
5b2847b338
etcd-tester: use Hash method to get both revision and hash
2016-02-12 14:40:51 -08:00
Gyu-Ho Lee
c1851dfca1
etcd-tester: add leader failure cases
2016-02-11 15:46:44 -08:00
Gyu-Ho Lee
78df258ea8
etcd-agent: cleans page cache when cleaning up
...
Reference:
- https://www.kernel.org/doc/Documentation/sysctl/vm.txt
- https://github.com/torvalds/linux/blob/master/fs/drop_caches.c
2016-02-11 13:48:54 -08:00
Xiang Li
4b68977851
Merge pull request #4481 from xiang90/testing-tool
...
tools/etcd-dump-logs: support parsing v3 log
2016-02-10 14:00:32 -08:00
Xiang Li
3611a9ad2e
tools/etcd-dump-logs: support parsing v3 log
2016-02-10 13:52:02 -08:00
Gyu-Ho Lee
7b2ce70783
etcd-tester: count success for v3
...
needed for snapshot count comparison
2016-02-09 17:12:03 -08:00
Gyu-Ho Lee
7d2b7e0d23
etcd-tester: close leaky gRPC connections
...
when closed errors will be one of:
```
grpc.ErrorDesc(err) == context.Canceled.Error() ||
grpc.ErrorDesc(err) == context.DeadlineExceeded.Error() ||
grpc.ErrorDesc(err) == "transport is closing" ||
grpc.ErrorDesc(err) == "grpc: the client connection is closing"
```
2016-02-09 16:26:33 -08:00
Gyu-Ho Lee
16aa263d8c
etcd-tester: continue after cleanup
2016-02-09 13:32:39 -08:00
Gyu-Ho Lee
c8994aab62
Merge pull request #4466 from gyuho/f0
...
etcd-tester: close gRPC connection when canceling
2016-02-09 11:07:00 -08:00
Gyu-Ho Lee
a46e20f92a
etcd-tester: close gRPC connection when canceling
...
Currently gRPC connection just gets recreated
for every Stress call. When Stress ends or gets
canceled, gRPC connection must also be closed.
For https://github.com/coreos/etcd/issues/4464 .
2016-02-09 11:04:16 -08:00
Gyu-Ho Lee
e252c0c0ca
etcd-tester: fix wrong error checking
...
Hash method returns either (nil, err) or (Hash, nil).
The current error checking is wrong. It only needs to check
the error is either nil or non-nil.
This causes panic in https://github.com/coreos/etcd/issues/4463
by allowing the case when resp is nil, but err is not nil.
2016-02-09 10:00:05 -08:00
Xiang Li
fa71bec550
Merge pull request #4458 from xiang90/cl
...
etcd-tester: cleanup
2016-02-08 21:16:28 -08:00
Xiang Li
25834211a9
etcd-tester: cleanup
2016-02-08 20:54:25 -08:00
Gyu-Ho Lee
77f753ac32
etcd-agent: mkdir with read/write to all users
2016-02-08 20:50:11 -08:00
Gyu-Ho Lee
16543778f1
etcd-tester: avoid directory name conflict
2016-02-08 13:38:26 -08:00
Gyu-Ho Lee
08dbabdb5f
etcd-tester: add compactKV
...
It compacts storage for every case.
For https://github.com/coreos/etcd/issues/4380 .
2016-02-06 15:02:58 -08:00
Gyu-Ho Lee
044f7775e3
etcd-tester: log cancel in stresser
...
And fix some minor print formats.
2016-02-05 18:31:27 -08:00
Gyu-Ho Lee
09fc764552
functional-tester/etcd-tester: silent grpclog, check revs
2016-02-05 14:04:58 -08:00
Gyu-Ho Lee
e2b5b1cd1a
functional-tester/etcd-agent: configurable log path
2016-02-05 09:37:24 -08:00
Gyu-Ho Lee
8e1325d9e1
functional-tester/etcd-agent: configurable agent port
...
To make local testing easier.
2016-02-04 21:10:30 -08:00
Gyu-Ho Lee
220d0c3c14
etcd-agent: fix data-dir path check
...
Need one more dash to match 'data-dir' flag.
2016-02-04 16:53:01 -08:00
Gyu-Ho Lee
85a4a5b596
functional-tester: fix grpc endpoint, consistent check
...
And some clean ups.
2016-02-04 15:10:43 -08:00
Gyu-Ho Lee
7a3426a231
tools/functional-tester/etcd-tester: support v3 kv storage
2016-02-03 15:24:54 -08:00
Gyu-Ho Lee
ed682c9f08
tools/functional-tester: minor cleanup
2016-02-03 14:30:34 -08:00
Anthony Romano
b74a42b286
clientv3: support unix endpoints
2016-02-01 22:24:42 -08:00
Anthony Romano
4380617e1a
tools/benchmark: support tls
2016-01-29 16:38:11 -08:00
Heungsub Lee
c4a0159601
documentation: fix typo "a etcd" -> "an etcd"
...
"a" is not a correct article for "etcd".
2016-01-28 19:19:32 +09:00
Anthony Romano
a7b6bbff3f
tools/benchmark: use clientv3
2016-01-27 12:13:17 -08:00
Gyu-Ho Lee
fa21946267
*: pass WithInsecure to grpc.Dial for now
...
Related to https://github.com/coreos/etcd/issues/4299 .
2016-01-27 00:24:03 -08:00
Gyu-Ho Lee
652c01bffe
tools/benchmark: update gRPC, proto interface
2016-01-26 17:41:32 -08:00
Xiang Li
6c82d768b2
Merge pull request #4201 from mitake/benchmark-pprof
...
tools/benchmark: add flags for pprof to storage put
2016-01-13 20:17:30 -08:00
Hitoshi Mitake
1c802e9266
tools/benchmark: add flags for pprof to storage put
...
This commit adds flags for profiling with runtime/pprof to storage
put:
- --cpuprofile: specify a path of CPU profiling result, if it is not
empty, profiling is activated
- --memprofile: specify a path of heap profiling result, if it is not
empty, profiling is activated
Of course, the flags should be added to RootCmd ideally. However,
adding common flags that shared by children command requires the
ongoing PR: https://github.com/spf13/cobra/pull/220 . Therefore this
commit adds the flags to storage put only.
2016-01-14 13:10:35 +09:00
Hitoshi Mitake
16b63310b2
tools/benchmark: remove deadcode
...
The Execute() function is a deadcode. Let's remove it.
2016-01-13 16:57:53 +09:00
Xiang Li
f5753f2f51
*: support lease Attach
...
Now we can attach keys to leases. And revoking the lease removes all
the attached keys of that lease.
2016-01-09 11:01:58 -08:00
Gyu-Ho Lee
a7287b6374
Merge pull request #4161 from gyuho/typo_in_benchmark
...
tools/benchmark/cmd: print error out to stderr
2016-01-07 18:50:09 -08:00
Anthony Romano
187064187c
tools/benchmark: report standard deviation
2016-01-07 15:30:23 -08:00
Gyu-Ho Lee
cc39cde319
tools/benchmark/cmd: print error out to stderr
2016-01-07 12:55:26 -08:00
Anthony Romano
f9af744be3
lease: move storage.NoLease to lease package
2016-01-07 10:33:44 -08:00
Xiang Li
5dd3f91903
*: make backend outside kv
...
KV and lease will share the same backend. Thus we need to make
backend outside KV.
2016-01-05 19:55:29 -08:00
Xiang Li
4336278b44
*: support put with lease
2016-01-04 15:54:06 -08:00
Xiang Li
ac330bb7c9
*: update watch related proto
...
1. Add watch/cancel request
2. Add necessary fields in response to return watch error
3. Add watch_id into watch response
2016-01-01 10:22:21 -08:00
Xiang Li
a74147384d
Merge pull request #4070 from mitake/storage-bench
...
tools: a new tool for benchmarking storage backends
2015-12-31 22:26:46 -08:00
Hitoshi Mitake
454865bd67
tools: a new tool for benchmarking storage backends
...
Current etcd repository has a test for benchmarking a storage backend
in storage/kvstore_bench_test.go. However, it is hard to test various
parameters (e.g. batch interval, a number of keys, etc) with the test.
This commit adds a new benchmarking subcommand "storage" to
tools/benchmark. It will encourage analysis of storage backends with
various parameter and complex workloads.
Exmaple usage:
$ ./benchmark storage put
total: 9.894173792s
average: 9.894173ms
minimum latency: 6.596991ms
maximum latency: 29.455695ms
2016-01-01 15:18:54 +09:00
Anthony Romano
1689bb3f02
tools/benchmark: support puts on sequential keys over a bounded keyspace
...
This patch makes it possible to use benchmark to generate n keys and
make random updates to only those n keys.
2015-12-30 16:47:24 -08:00
Hitoshi Mitake
b97f78d356
tools/benchmark: ignore the binary "benchmark"
2015-12-30 16:02:52 +09:00
Anthony Romano
8e728afa62
tools/benchmark: support connecting to several endpoints
...
--endpoints is comma separated but gRPC blocks forever on comma
separated lists. Instead, round-robin select endpoints when
creating new connections.
2015-12-28 10:22:33 -08:00
Anthony Romano
382103af60
tools/benchmark: stream results into reports
...
Reports depended on writing all results to a large buffered channel and
reading from that synchronously. Similarly, requests were buffered the
same way which can take significant memory on big request strings. Instead,
have reports stream in results as they're produced then print when the
results channel closes.
2015-12-23 11:24:35 -08:00
Pawel Palucki
4f5f999847
tools/functional-test: add docker support
...
Commit adds docker bits to run functional tester within containers.
requires:
- docker 1.9 (networking)
- docker-compose
2015-12-18 15:56:18 +01:00
Gyu-Ho Lee
4289871cef
tools/function-tester: fix shadowed variables
...
Fixes for https://github.com/coreos/etcd/issues/3954 .
2015-12-12 09:38:20 -08:00
Xiang Li
8d4073d078
Merge pull request #3945 from xiang90/new_watch_bench
...
tools/benchmark: add watch subcommand.
2015-12-03 16:56:02 -08:00
Xiang Li
c6430b3292
tools/benchmark: add watch subcommand.
...
Watch command run benchmark tests for watch releated operations:
1. watch keys 2. sending events to watchers.
To test 2, this test also puts keys into etcd cluster to trigger
event sending.
Besides the benchmark results showed at client side, the tester
can also monitor the server-side mem/cpu usage and also check
the metrics of slow watchers. If there are a lot of slow watchers,
it means etcd server is over-loaded by watchers.
2015-12-03 15:56:17 -08:00
Xiang Li
93240bd0f0
Merge pull request #3933 from ngaut/fix-shadow-variables
...
tools: Fix shadow variables and incorrect printing format.
2015-12-01 08:55:22 -08:00
Xiang Li
faff00d19e
tools: rewrite benchmark tool
2015-11-30 11:52:43 -08:00
ngaut
232439191c
tools: Fix shadow variables and incorrect printing format.
2015-11-30 20:13:29 +08:00
Yicheng Qin
dfc7cc7a62
tools/etcd-tester: extend timeout for stresser
...
Extend the timeout from 1s to defaultRequestTimeout 5s.
The 1s may bring unwanted burden to the target member. If the member is
busy at recovering, it has limited bandwidth for client requests. A
short timeout at client side will retry quickly while keeping the
on-going connections. Thus, etcd will queue lots of requests and
connections and takes long time to clear them. This finally causes the
timeout of member health check.
This problem is a general one that how etcd handles amounts of requests
at the same time in a good way. We don't plan to address it at current
stage.
2015-11-16 11:47:08 -08:00
Xiang Li
1a3f7f7fa4
*: rename etcd service to kv service in gRPC
2015-11-04 10:05:49 -08:00
Xiang Li
e1b2e7245b
tools/etcd-top: add copyright header
2015-11-01 18:19:32 -08:00
Tyler Neely
00557e96af
tools: add etcd-top
2015-11-01 18:07:27 -08:00
Xiang Li
c4b3ad72d9
Merge pull request #3544 from xiang90/bench
...
v3benchmark: add put benchmark
2015-09-17 15:10:13 -07:00
Xiang Li
f69582e1a2
v3benchmark: add put benchmark
2015-09-17 14:48:07 -07:00
Xiang Li
3a2700141e
Merge pull request #3539 from xiang90/bench
...
godep: use github.com/cheggaaa/pb
2015-09-15 16:12:34 -07:00
Xiang Li
38dd680f2e
godep: use github.com/cheggaaa/pb
2015-09-15 16:08:07 -07:00
Xiang Li
8bb50635ce
Merge pull request #3538 from xiang90/bench
...
benchmarkv3: refactoring the main logic
2015-09-15 15:58:32 -07:00
Xiang Li
4deb12fbbb
benchmarkv3: refactoring the main logic
2015-09-15 15:57:38 -07:00
Jonathan Boulle
7848ac3979
*: add missing license headers
2015-09-15 14:09:01 -07:00
Alex Kolbasov
2782418923
Godep: fixed missing dependencies
2015-09-04 04:51:44 +00:00
Yicheng Qin
3632a1b9b1
*: add initial read benchmark for etcd v3
...
It includes the initial read benchmark for etcd v3.
This is the first step to give some rough thoughts. I haven't digged
deeper to answer some questions, including why its performance is not
better than HTTP + json, why one put will cause performance downgrade.
2015-08-25 07:50:18 -07:00
Xiang Li
2b8abeb093
*: remove migration related stuff from 2.2
2015-08-01 19:37:20 +08:00
Yicheng Qin
e71d43b58e
tools/functional-testing: add help message for flags
...
Help users to understand what these flags are for.
2015-04-29 13:59:55 -07:00
Yicheng Qin
eafdd3b718
Merge pull request #2730 from yichengq/tester-key-param
...
main: parameterize stress key size and key suffix range
2015-04-27 17:02:36 -07:00
Yicheng Qin
057d21cf79
main: parameterize stress key size and key suffix range
...
It faciliates tester to adjust the size of each request, the number of
keys in the store and the size of snapshot.
2015-04-27 16:46:56 -07:00
Yicheng Qin
3127a3b659
tools/etcd-tester: wait longer for health
...
It dramatically reduce the probability that follower failed to catch up
the leader.
2015-04-21 17:55:24 -07:00
Xiang Li
a9157ce6d3
build: do not build internal debugging tool
...
We are still playing around with the dump-log tool.
Stop building it publicly until we are happy with its
ux and functionality.
2015-03-31 11:45:12 -07:00
Yicheng Qin
81750ab2d7
Merge pull request #2600 from yichengq/failure-isolate
...
tools/functional-tester: add isolate failures
2015-03-29 22:43:51 -07:00
Yicheng Qin
04a62dd54b
tools/functional-tester: add isolate failures
2015-03-29 00:29:47 -07:00
Yicheng Qin
097a56fe01
tools/etcd-agent: stop etcd only if it is running
...
Stop etcd only if it is running, and not report error when stopping etcd
which is not started.
2015-03-28 19:31:06 -07:00
Yicheng Qin
d920c5b801
tools/etcd-tester: stress cluster using 50MB snapshot
2015-03-19 14:52:27 -07:00
Xiang Li
46ebb83b90
tools/functional-tester/etcd-tester: report agent status
2015-03-13 15:29:57 -07:00
Xiang Li
83bb02e320
tools/funcational-tester/etcd-agent: log the error for dubgging
2015-03-13 12:08:08 -07:00
Xiang Li
e46beb75c8
tools/functional-tester/etcd-agent: add status rpc
2015-03-13 10:48:06 -07:00
Xiang Li
462f32a81b
tools/functional-tester: add http status reporter
2015-03-12 12:49:48 -07:00
Yicheng Qin
24a210ab20
tools/etcd-tester: add kill one member tests
2015-03-10 11:38:54 -07:00
Yicheng Qin
ba20016f0f
tools/etcd-tester: reorganize failures
2015-03-05 21:14:41 -08:00
Yicheng Qin
eba6daef4b
Merge pull request #2450 from yichengq/335
...
tools/functional-tester: add cleanup rpc
2015-03-05 16:36:16 -08:00
Yicheng Qin
181ee445c1
better dir name
2015-03-05 16:34:14 -08:00
Yicheng Qin
2152447361
tools/functional-tester: add cleanup rpc
2015-03-05 15:55:28 -08:00
Yicheng Qin
4314b19a2e
tools/etcd-agent: recycle etcd zombie when termination
2015-03-05 15:51:11 -08:00
Yicheng Qin
267313a3f8
tools/etcd-tester: add kill majority test
2015-03-05 15:14:14 -08:00
Yicheng Qin
3cffc910de
tools/etcd-tester: use stresser
2015-03-05 13:21:49 -08:00
Xiang Li
eec52738d8
etcd-tester: initial stresser
2015-03-05 11:06:43 -08:00
Yicheng Qin
d5957aebfd
tools/etcd-tester: add failure killall
2015-03-05 10:24:21 -08:00
Yicheng Qin
530dd891be
tools/etcd-tester: make it work
...
1. add cluster support
2. add failureNo case
3. add main func
2015-03-05 10:24:21 -08:00
Yicheng Qin
8d3d737993
tools/etcd-agent/client: fix rpc Dial
2015-03-05 10:24:21 -08:00
Yicheng Qin
061baad611
tools/etcd-agent: write etcd log into log file
2015-03-05 10:24:13 -08:00
Yicheng Qin
008bbd2b84
tools/etcd-agent: log rpc actions
2015-03-04 18:29:23 -08:00
Yicheng Qin
9e69aba7aa
tools/etcd-agent: add main func
2015-03-04 17:22:56 -08:00
Yicheng Qin
30e6d49bec
etcd-tester: fix build
2015-03-04 10:22:53 -08:00
Xiang Li
2bfd266a81
tools/functional-tester: inital commit
2015-03-03 20:12:20 -08:00
Xiang Li
d6f8a30f7c
tool: dump tool supports index
2015-02-18 09:13:47 -08:00
Xiang Li
161b1d2e2e
tools: etcd-dump-logs tool support dump from a given snapshot file
2015-02-11 10:50:04 -08:00
Jonathan Boulle
f1ed69e883
*: switch to line comments for copyright
...
Build tags are not compatible with block comments.
Also adds copyright header to a few places it was missing.
2015-01-26 09:53:30 -08:00
Yicheng Qin
84f62f21ee
wal: record and check snapshot
2015-01-06 16:27:40 -08:00
Yicheng Qin
9de4e36b6a
tools: move etcd-migrate to tools
2014-12-30 22:48:47 -08:00
Yicheng Qin
8b0c7bf652
tools: add etcd-dump-logs
...
The tool can dump the log from data directory.
It helps develop and debug.
2014-12-30 16:14:27 -08:00