Commit Graph

584 Commits (0461b3fa51ebcd3f72ef54296638ac2f19ea9071)

Author SHA1 Message Date
Anthony Romano 61abf25859 integration: close accepted connection on stopc path
Connection pausing added another exit condition in the listener
path, causing the bridge to leak connections instead of closing
them when signalled to close. Also adds some additional Close
paranoia.

Fixes #7823
2017-05-02 16:46:43 -07:00
Gyu-Ho Lee 01b6cdf13d integration: ensure revoke completes before TimeToLive
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-05-02 12:56:26 -07:00
Anthony Romano 6dd8fb6f24 Revert "Merge pull request #7732 from heyitsanthony/lease-err-ka"
This reverts commit fbbc4a4979, reversing
changes made to f254e38385.

Fixes #7851
2017-05-02 09:36:16 -07:00
Gyu-Ho Lee cfeab9324e integration: match more TLS errors for wrong certs
Fix https://github.com/coreos/etcd/issues/7835.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-04-28 10:03:29 -07:00
Gyu-Ho Lee 22943e7e06 integration: test TLS reload
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-04-27 13:32:09 -07:00
Anthony Romano c309d745a6 Merge pull request #7819 from heyitsanthony/fix-elect-compact
concurrency: use current revisions for election
2017-04-27 11:01:44 -07:00
Gyu-Ho Lee 7cb860a31b integration/fixtures: add expired certs
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-04-26 12:22:54 -07:00
Anthony Romano 6486be673b integration: test Observe can read leaders set prior to compaction 2017-04-25 20:03:49 -07:00
Anthony Romano 2fd6df922a integration: close proxy's lease client 2017-04-24 23:49:45 -07:00
Anthony Romano e6a789d541 integration: permit dropping intermediate leader values on observe
Weaken TestV3ElectionObserve so it only checks that it observes a strictly
monotonically ascending leader transition sequence following the first
observed leader. First, the Observe will issue the leader channel before
getting a response for its first get; the election revision is only bound
after returning the channel. So, Observe can't be expected to always
return the leader at the time it was started.  Second, Observe fetches
the current leader based on its create revision, but begins watching on its
ModRevision; this is important so that elections still work in case the
leader issues proclamations following a compaction that exceeds its
creation revision. So, Observe can't be expected to return the entire
proclamation sequence for a single leader.

Fixes #7749
2017-04-22 11:26:11 -07:00
Anthony Romano fe1ce3a2f0 integration: add pause/unpause to client bridge
Resetting connections sometimes isn't enough; need to stop/resume
accepting connections for some tests while keeping the member up.
2017-04-21 20:22:51 -07:00
Anthony Romano 14d6ed9e5f *: clear redundant return statement warnings (S1027) 2017-04-21 14:01:00 -07:00
Gyu-Ho Lee 99a2d6c4b1 integration: use 'time.Until' in Go 1.8
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-04-17 20:08:27 -07:00
Gyu-Ho Lee e771c6042b Merge pull request #7743 from gyuho/shutdown-grpc-server
*: use gRPC server GracefulStop
2017-04-17 17:12:52 -07:00
Gyu-Ho Lee 253e5a90bb integration: test auth API response header revision
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-04-17 14:26:30 -07:00
Gyu-Ho Lee 472a536052 integration: test 'inflight' range requests
- Test https://github.com/coreos/etcd/issues/7322.
- Remove test case added in https://github.com/coreos/etcd/pull/6662.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-04-17 14:15:36 -07:00
Anthony Romano 712f6cb0e1 integration: test requests with valid auth token but disabled auth
etcd was crashing since auth was assuming a token implies auth is enabled.
2017-04-12 13:17:33 -07:00
Anthony Romano bf047ed9d5 integration: v3 election rpc tests 2017-04-07 16:36:38 -07:00
Gyu-Ho Lee 1e3274dfa2 integration: use cluster adapter in tests
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-04-04 10:08:51 -07:00
Anthony Romano 24e4c94d98 Merge pull request #7640 from heyitsanthony/etcdserver-ctx
etcdserver: ctx-ize server initiated requests
2017-04-03 09:07:28 -07:00
Anthony Romano f5f4791023 integration: test cluster terminates quickly 2017-03-31 19:19:33 -07:00
Anthony Romano b55ea6a70b integration: test require leader for a lease stream 2017-03-30 20:18:33 -07:00
Gyu-Ho Lee c90a4b96d1 integration: use maintenance API adapter in tests
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-03-28 14:12:47 -07:00
Anthony Romano b9cfa4cef9 integration: add serialized range to TestV3CompactCurrentRev
To catch compaction bugs in the proxy key cache.
2017-03-24 13:13:38 -07:00
Gyu-Ho Lee b81cb999fb integration: retry TestNetworkPartition5MembersLeaderInMajority
Fix https://github.com/coreos/etcd/issues/7587.

Retry for possible leader election in majority.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-03-23 16:30:38 -07:00
Gyu-Ho Lee ed68bf89ff integration: test inflight range requests while defragmenting
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-03-23 09:48:04 -07:00
Anthony Romano f35d7d9608 integration: test namespacing on proxy layer
Hardcode a namespace over the testing grpcproxy.
2017-03-22 16:45:38 -07:00
Anthony Romano 9f34d3493d integration: wait on leader before progress check in TestRestartMember
In rare cases, the last member may not have the leader by the time the
final cluster progress check tries to open a watch, causing a timeout.
2017-03-22 12:48:31 -07:00
Gyu-Ho Lee 2d5f890091 integration: ensure 'StopNotify' on publish error
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-03-21 10:29:00 -07:00
fanmin shi f292a4c953 embed: don't return error when closing on embed etcd
FIXES #7019
2017-03-17 13:41:05 -07:00
Anthony Romano 2f1542c06d *: use filepath.Join for files 2017-03-16 07:46:06 -07:00
fanmin shi ace3a217b0 Merge pull request #7483 from fanminshi/add_tests_to_mutex
integration: add TestMutexWaitsOnCurrentHolder test
2017-03-14 13:01:47 -07:00
fanmin shi 276039e835 integration: add TestMutexWaitsOnCurrentHolder test
TestMutexWaitsOnCurrentHolder ensures a series of waiters
obtain lock only after the previous lock requests are gone.
2017-03-14 11:00:07 -07:00
Anthony Romano 300323fa50 integration: test grpc lock service 2017-03-13 15:23:26 -07:00
Anthony Romano b025cdd097 adapter, integration: split out grpc adapters from grpcproxy package
Break cyclic dependency:
clientv3/naming <-> integration <-> v3client <-> grpcproxy <-> clientv3/naming
2017-03-13 15:23:26 -07:00
Gyu-Ho Lee beb58c434c *: fix minor typos
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-03-08 16:21:52 -08:00
Hitoshi Mitake f8a290e7ca *: support jwt token in v3 auth API
This commit adds jwt token support in v3 auth API.

Remaining major ToDos:
- Currently token type isn't hidden from etcdserver. In the near
  future the information should be completely invisible from
  etcdserver package.
- Configurable expiration of token. Currently tokens can be valid
  until keys are changed.

How to use:
1. generate keys for signing and verfying jwt tokens:
 $ openssl genrsa -out app.rsa 1024
 $ openssl rsa -in app.rsa -pubout > app.rsa.pub
2.  add command line options to etcd like below:
--auth-token-type jwt \
--auth-jwt-pub-key app.rsa.pub --auth-jwt-priv-key app.rsa \
--auth-jwt-sign-method RS512
3. launch etcd cluster

Below is a performance comparison of serializable read w/ and w/o jwt
token. Every (3) etcd node is executed on a single machine. Signing
method is RS512 and key length is 1024 bit. As the results show, jwt
based token introduces a performance overhead but it would be
acceptable for a case that requires authentication.

w/o jwt token auth (no auth):

Summary:
  Total:        1.6172 secs.
  Slowest:      0.0125 secs.
  Fastest:      0.0001 secs.
  Average:      0.0002 secs.
  Stddev:       0.0004 secs.
  Requests/sec: 6183.5877

Response time histogram:
  0.000 [1]     |
  0.001 [9982]  |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.003 [1]     |
  0.004 [1]     |
  0.005 [0]     |
  0.006 [0]     |
  0.008 [6]     |
  0.009 [0]     |
  0.010 [1]     |
  0.011 [5]     |
  0.013 [3]     |

Latency distribution:
  10% in 0.0001 secs.
  25% in 0.0001 secs.
  50% in 0.0001 secs.
  75% in 0.0001 secs.
  90% in 0.0002 secs.
  95% in 0.0002 secs.
  99% in 0.0003 secs.

w/ jwt token auth:

Summary:
  Total:        2.5364 secs.
  Slowest:      0.0182 secs.
  Fastest:      0.0002 secs.
  Average:      0.0003 secs.
  Stddev:       0.0005 secs.
  Requests/sec: 3942.5185

Response time histogram:
  0.000 [1]     |
  0.002 [9975]  |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.004 [0]     |
  0.006 [1]     |
  0.007 [11]    |
  0.009 [2]     |
  0.011 [4]     |
  0.013 [5]     |
  0.015 [0]     |
  0.016 [0]     |
  0.018 [1]     |

Latency distribution:
  10% in 0.0002 secs.
  25% in 0.0002 secs.
  50% in 0.0002 secs.
  75% in 0.0002 secs.
  90% in 0.0003 secs.
  95% in 0.0003 secs.
  99% in 0.0004 secs.
2017-03-06 19:46:03 -08:00
Gyu-Ho Lee 3d75395875 *: remove never-unused vars, minor lint fix
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-03-06 14:59:12 -08:00
Anthony Romano 1753623f87 integration: don't set v3 capability since now default 2017-03-02 14:02:09 -08:00
fanmin shi 324d2383b8 integration: ensure leader is up in waitLeader() and clusterMustProgress()
The issue is caused by leader loss even after waitLeader() returns
which can happen if the test machine is flaky which triggers a leader loss
or the killed node is the leader since waitLeader() only scans followers in
TestRestartMember() and they can have the same older leader.
In those cases, clusterMustProgress() proceeds with no leader which triggers
the no leader error.

To get around that, use linearizable get in waitLeader() to ensure leader is up
and retries on kapi.create() in clusterMustProgress() to ensure it proceeds with
a leader.

FIX #7258
2017-02-28 09:17:03 -08:00
Gyu-Ho Lee f2e9936de5 integration: add 'TestV3HashRestart' 2017-02-21 16:20:56 -08:00
fanmin shi 65b59f4423 grpcproxy: incorporate lease proxy into existing proxy framework 2017-02-16 11:50:59 -08:00
fanmin shi 2925f02aac Merge pull request #7305 from fanminshi/return_header_for_timetolive
lease: LeaseTimeToLive returns TTL=-1 resp on lease not found
2017-02-13 11:24:36 -08:00
fanmin shi 0d08ffa282 integration: don't expect lease not found error for TestV3GetNonExistLease 2017-02-10 17:35:43 -08:00
Manjunath A Kumatagi 0914b8b707 test: Fix gosimple errors
Getting gosimple suggestion while running test script, so this PR is for fixing gosimple S1019 check.
raft/node_test.go:456:40: should use make([]raftpb.Entry, 1) instead (S1019)
raft/node_test.go:457:49: should use make([]raftpb.Entry, 1) instead (S1019)
raft/node_test.go:458:43: should use make([]raftpb.Message, 1) instead (S1019)

Refer https://github.com/dominikh/go-tools/blob/master/cmd/gosimple/README.md#checks for more information.
2017-02-09 08:01:28 -05:00
Ravi Gadde c586218ec6 clientv3: start a session with existing lease
This change is needed to handle process restarts with elections. When the
leader process is restarted, it should be able to hang on to the leadership
by using the existing lease.

Fixes #7166
2017-01-30 18:07:22 -08:00
Nick Miyake e3b325c196 test: fix failure message in TestEmbedEtcd 2017-01-26 14:00:32 -08:00
Anthony Romano 63572567b4 integration: test domain name URLs are rejected before binding 2017-01-26 12:37:34 -08:00
sharat 95edd1bc58 integration: put,txn with 'ignore_lease' flag 2017-01-25 03:07:23 +05:30
Anthony Romano 555b8047e6 integration: fix STM tests to compile against new interface 2017-01-20 16:30:58 -08:00
Gyu-Ho Lee a94d20d1e4 integration: test STM apply on concurrent deletion 2017-01-19 22:59:01 -08:00
Anthony Romano 8c0282ab24 grpcproxy, etcdmain, integration: add close channel to kv proxy
ccache launches goroutines that need to be explicitly stopped.

Fixes #7158
2017-01-18 11:51:16 -08:00
Gyu-Ho Lee a66f133209 integration: test Put,Txn with ignore_value flag 2017-01-13 15:13:18 -08:00
Anthony Romano 5278ea5ed0 integration: add grpc auth testing 2017-01-09 15:53:36 -08:00
Gyu-Ho Lee 6825ffe1a4 integration: use only digits in unix ports
Fix https://github.com/coreos/etcd/issues/6959.
2017-01-05 12:34:54 -08:00
Anthony Romano 75441390b6 integration: defer clus.Terminate in watch tests
Common pattern was defer cancel(), but clus.Terminate() at the end of
the test. This appears to lead to a deadlock that is only released
once the context times out, causing inflated test times.
2016-12-30 12:34:04 -08:00
Anthony Romano 9b5eb1ae5a grpcproxy, etcdmain, integration: return done channel with WatchServer
Makes it possible to synchronously close the watch server.

Fixes #7078
2016-12-30 12:09:48 -08:00
fanmin shi fef4a79528 lease: force leader to apply its pending committed index for lease operations
suppose a lease granting request from a follower goes through and followed by a lease look up or renewal, the leader might not apply the lease grant request locally. So the leader might not find the lease from the lease look up or renewal request which will result lease not found error. To fix this issue, we force the leader to apply its pending commited index before looking up lease.

FIX #6978
2016-12-22 14:24:38 -08:00
Gyu-Ho Lee f3cb93015c integration: simplify boolean comparison in resp.Created 2016-12-12 10:07:14 -08:00
Anthony Romano f6042890b7 integration: use RequireLeader for TestV3LeaseFailover
Giving Renew() the default request timeout causes TestV3LeaseFailover
to miss its timing constraints. Since it only needs to wait until the
leader recognizes the leader is lost, use RequireLeader to cancel the
keepalive stream before the request times out.
2016-12-06 14:09:57 -08:00
Anthony Romano 76bb33781f integration: cancel Watch when TestV3WatchWithPrevKV exits
Missing ctx cancel was causing goroutine leaks for the proxy tests.
2016-12-01 15:08:18 -08:00
Anthony Romano f1e0525c81 integration: use Range to wait for reboot in quota tests
Proxy client layer ignores call options so Put is always FailFast;
this can lead to connection errors when trying to issue the Put
following restarting the client's target server.
2016-11-30 13:56:30 -08:00
Anthony Romano 6d9168a2ec integration: don't expect recv to stop on CloseSend in waitResponse 2016-11-18 11:37:35 -08:00
Gyu-Ho Lee 396a71ee9e integration: test wrong watcher range 2016-11-08 17:02:32 -08:00
Gyu-Ho Lee ef9d55800f integration: test inflight Hash call on nil db 2016-10-21 11:02:54 -07:00
Gyu-Ho Lee f550af7ef4 integration: test sort ASCEND by default in range 2016-10-18 16:50:30 -07:00
Hongchao Deng c3948284a0 integration: add TestV3WatchWithPrevKV 2016-10-12 16:21:52 -07:00
Geoff Levand 84d2ff93b0 integration/v3_grpc_test: Fix quota tests
Use the system page size to set the test quota size.  Also, change
a comment related to setting the node quota to be more clear.

Signed-off-by: Geoff Levand <geoff@infradead.org>
2016-10-05 16:41:06 -07:00
fanmin shi 8ef6687018 etcdserver: fix a node panic bug caused LeaseTimeToLive call on a nonexistent lease
When the non Leader etcd server receives a LeaseTimeToLive on a nonexistent lease, it responds with a nil resp and a nil error The invoking function parses the nil resp and results a segmentation fault.
I fix the bug by making sure the lease not found error is returned so that the invoking function parses the the error message instead.

fix #6537
2016-09-27 17:46:30 -07:00
Anthony Romano e77baa3dcb Merge pull request #6424 from heyitsanthony/v3api-createminmax
etcdserver: range queries with min/max create revision
2016-09-14 19:10:52 -07:00
Anthony Romano aa7a35798d integration: add tests for MinCreateRev and MaxCreateRev 2016-09-14 15:31:45 -07:00
Anthony Romano 9b91e96510 integration: fix rwmutex test to check write locking 2016-09-13 14:09:59 -07:00
Anthony Romano c6bfdb909b Merge pull request #6412 from heyitsanthony/revert-domain-listener
embed: warn on domain name in listener
2016-09-13 10:25:18 -07:00
Anthony Romano fa2e9c2449 Revert "Merge pull request #6365 from heyitsanthony/fix-dns-bind"
This reverts commit af5ab7b351, reversing
changes made to da6a0f0594.
2016-09-12 19:45:35 -07:00
Anthony Romano 17e7f83212 integration: test MinModRev/MaxModRev 2016-09-12 19:44:14 -07:00
Gyu-Ho Lee 03b9d6f24c *: separate 'capnslog' log level setting 2016-09-10 20:26:51 +09:00
Anthony Romano 7644a8ad76 integration: test domain name URLs are rejected before binding 2016-09-06 15:33:47 -07:00
Anthony Romano 3489fa82fb integration: don't nest proxies in cluster_proxy mode 2016-09-01 15:21:52 -07:00
Xiang Li 5b60be9626 integration: fix live lock in issue3699
Do not restart the killed member immediately.
The member will advance its election timeout after restart
So it will have a better chance to become the leader again.
2016-08-31 12:25:24 -07:00
Gyu-Ho Lee 5c06fc9093 integration: change to 'NoValueOnSuccess' 2016-08-30 10:58:44 -07:00
Michael Fraenkel 82053f04b2 client: do not send previous node data (optional)
- Do not send back node data when specified
- remove node and prevNode when noDataOnSuccess is set
2016-08-30 10:04:09 -07:00
Xiang Li 9e9bbb829e Merge pull request #6289 from purpleidea/feat/move-readynotify
embed: Move the ReadyNotify() call to a more sane place
2016-08-29 20:06:17 -07:00
James Shubin 9aee3f01cd embed: Move the ReadyNotify() call to a better place
When using the embed functionality, you can't call the Server.Stop()
function until StartEtcd returns, which can block until there is a call
to Server.Stop() in error situations. Since we have a catch-22, the
ReadyNotify() can be called manually by the user if they wish to wait
for the server startup, or in parallel with a timeout if they wish to
cancel it after some time.

Chzz pointed out that this is also more consistent with the
etcdserver.Start() behaviour too.

purpleidea pointed out that this is actually more correct too, because
we can now register the stop interrupt handler before we block on
startup.
2016-08-29 22:45:41 -04:00
Jason E. Aten 9497e9678c clientv3/concurrency: allow election on prefixes of keys.
After winning an election or obtaining a lock, we
auto-append a slash after the provided key prefix.
This avoids the previous deadlock due to waiting
on the wrong key.

Fixes #6278
2016-08-29 18:34:14 -07:00
Gyu-Ho Lee eaa5d9772f integration: improve TestTransferLeader
so that it can check leader transition
2016-08-19 13:11:38 -07:00
Hongchao Deng 5630a76766 integration: NewClusterV3 should launch cluster before creating clients 2016-08-18 14:05:21 -07:00
Gyu-Ho Lee 10c9e238f0 integration: fix race in TestDoubleBarrierFailover 2016-08-17 11:56:49 -07:00
Gyu-Ho Lee f2fedbae9b integration: write to leader group first, or wait
Write to leader group first, or give more time to
acknowledge the leader after network partition recovery
2016-08-17 11:09:33 -07:00
Gyu-Ho Lee fb00a32b86 integration: fix races in global proxies 2016-08-16 19:43:31 -07:00
Xiang Li d5d2370fc8 Merge pull request #6172 from xiang90/session
session: remove session manager and add ttl
2016-08-15 15:20:19 -07:00
Xiang Li feaff17259 session: remove session manager and add ttl 2016-08-15 14:12:25 -07:00
Gyu-Ho Lee a205242ca5 integration: add 'TestTransferLeader/Stop' 2016-08-13 14:32:01 -07:00
Gyu-Ho Lee f975fe8068 Merge pull request #6140 from gyuho/network-partition
*: add network partition tests
2016-08-12 12:33:24 -07:00
Gyu-Ho Lee 0a00328a7c integration: add network partition tests 2016-08-12 12:15:29 -07:00
Xiang Li 82a3d90763 Merge pull request #6167 from xiang90/fix_txn_rev
etcdserver: fix wrong rev in header when nothing is actually got executed
2016-08-12 12:14:48 -07:00
Xiang Li 92a0f08722 etcdserver: fix wrong rev in header when nothing is actually got executed 2016-08-12 11:44:13 -07:00
Anthony Romano e742ff331f integration: test member removal which breaks active quorum is rejected 2016-08-11 11:14:14 -07:00
Anthony Romano c3c41234f1 integration: support querying member metrics 2016-08-08 23:45:50 -07:00
Gyu-Ho Lee f1f31f1015 integration: add more lease tests
Fix https://github.com/coreos/etcd/issues/6102.
2016-08-05 19:09:46 -07:00
Anthony Romano 9063ce5e3f etcdserver, embed: stricter reconfig checking
Make --strict-reconfig-check a default and check if cluster is healthy when
adding a member.
2016-08-05 16:59:25 -07:00
Anthony Romano 1cad722a6d integration: support watch apis in cluster_proxy build 2016-07-25 09:34:36 -07:00
Anthony Romano e001848270 Merge pull request #5772 from heyitsanthony/integration-proxy
integration: build tag for proxy
2016-07-20 16:28:12 -07:00
Anthony Romano 7b3d502b96 integration: build tag cluster_proxy for testing backed by proxy 2016-07-20 15:40:33 -07:00
Anthony Romano 15aa64eb3c Merge pull request #6009 from heyitsanthony/fix-progress-notify
v3rpc: don't elide next progress notification on progress notification
2016-07-20 13:46:11 -07:00
Anthony Romano 8c8742f43c integration: change timeouts for TestWatchWithProgressNotify
a) 2 * progress interval was passing with dropped notifies
b) waitResponse was waiting so long that it expected a dropped notify
2016-07-20 13:23:44 -07:00
Anthony Romano 82ddeb38b4 integration: fix race in TestV3WatchMultipleEventsTxnSynced
Writes between watcher creation request and reply were being dropped.

Fixes #5789
2016-07-20 09:55:39 -07:00
Anthony Romano 8d897fd51f integration: use unix sockets in TestEmbedEtcd
Was getting tcp port conflicts in semphore even after assigning unique ports.

Fixes #5953
2016-07-18 16:42:08 -07:00
Gyu-Ho Lee 4a5795b55f integration: new ports for embed test 2016-07-15 16:47:32 -07:00
Anthony Romano b3f8490660 integration: add FailFast(false) to failing tests 2016-07-14 17:58:58 -07:00
Anthony Romano 07ed4da2ff integration: test grpc error equivalence with Error() 2016-07-14 17:58:49 -07:00
Anthony Romano 41a98dbd66 Merge pull request #5925 from heyitsanthony/embed-etcdmain
embeddable etcdmain
2016-07-13 13:51:19 -07:00
Anthony Romano c0299ca6f4 integration: test embedded etcd 2016-07-13 10:40:03 -07:00
Xiang Li 6392ef5c44 integration: remove upgrade test for etcd0.4 2016-07-12 10:13:03 -07:00
Anthony Romano c30f89f1d0 client/integration: test v2 client one shot operations 2016-07-07 15:58:58 -07:00
Xiang Li 234c30c061 Merge pull request #5880 from xiang90/put_prev
add options to return prev_kv
2016-07-05 21:03:56 -07:00
Xiang Li 12bf1a3382 *: rename preserveKVs to prevKv 2016-07-05 20:45:01 -07:00
Anthony Romano c642430fae integration: use RSA certs for testing
Some systems don't support EC due to patent issues, but the tests
should still work.

Fixes #5744
2016-07-05 13:21:21 -07:00
Xiang Li 40c4a7894d *: support return prev deleted kv 2016-07-01 14:01:48 -07:00
Xiang Li dced92f8bd *: support watch with filters
Now user can filter events with types. The API is also extensible.
It might make sense for the proxy to filter out events based on
more expensive/customized filter.
2016-06-28 13:46:57 -07:00
Anthony Romano 13d0ea7f54 integration: use unix domain sockets for all connections 2016-06-24 21:18:19 -07:00
Anthony Romano 7abc8f21eb integration: update tests for new grpc reconnection interface 2016-06-08 01:04:59 -07:00
Anthony Romano 1823702cc6 integration: bridge connections to grpc server
Tests need to disconnect the network connection for the client to check
reconnection paths but closing a grpc connection closes the logical connection.
To disconnect the client, instead have a bridge between the server and
the client which can monitor and reset connections.
2016-06-08 00:34:53 -07:00
Gyu-Ho Lee 1610391449 *: following changes for proto update 2016-06-07 13:33:03 -07:00
Anthony Romano 7dfe7db243 clientv3: panic if ActiveConnection tries to return non-nil connection 2016-06-03 10:25:20 -07:00
Xiang Li 755567cb3d Merge pull request #5547 from xiang90/int
integration: always return active client
2016-06-02 15:52:38 -07:00
Xiang Li bbfe7f401f integration: always return active client
In the integration test, we sometimes stop/restart an etcd server.
Now our client has internal connection monitoring logic that might
set conn to nil when there is a connection failure and the redial
also fails.

Chaning randClient to always return a client with active connection
to make integration test reliable.
2016-06-02 14:49:32 -07:00
Anthony Romano 84a487f723 Revert "etcdserverpb: make RangeResponse.More an int64"
This reverts commit 84e1ab8765.
2016-06-02 13:43:40 -07:00
Anthony Romano 84e1ab8765 etcdserverpb: make RangeResponse.More an int64 2016-06-01 17:10:23 -07:00
Anthony Romano 7709cd84bb Merge pull request #5505 from heyitsanthony/v3rpc-watcher-close
v3rpc: fix race on ctrl channel when watcher stream closes
2016-05-31 14:24:10 -07:00
Gyu-Ho Lee b0d4a0a9bd integration: skip closed client in Terminate 2016-05-31 12:00:15 -07:00
Anthony Romano 41d3cea9b3 integration: test closing stream while creating watchers 2016-05-31 11:02:15 -07:00
Xiang Li ac2859057a integration: move cap enabling to init 2016-05-27 11:12:07 -07:00
Xiang Li 340df26883 Merge pull request #5435 from xiang90/cap
api: add v3rpc capability
2016-05-23 15:50:08 -07:00
Xiang Li 1c544c3ba5 api: add v3rpc capability 2016-05-23 14:45:08 -07:00
Gyu-Ho Lee ddbe46543d integration: add logs for debugging 2016-05-23 11:23:41 -07:00
Anthony Romano 73204e9637 etcdserver: wait for snapshots before closing raft
Fixes #5374
2016-05-17 15:04:25 -07:00
Anthony Romano 8ffbaef502 Merge pull request #5364 from heyitsanthony/fix-election-wait
integration: fix TestElectionWait
2016-05-16 10:30:17 -07:00
Anthony Romano c21b885dd5 integration: fix TestElectionWait
elections are now per-session so waiting on the same election with the
same client will not block like before

Fixes #5362
2016-05-16 07:32:42 -07:00
Xiang Li ec2ac72585 proxy: initial grpc kv service proxy 2016-05-13 23:00:29 -07:00
Xiang Li c7c0e1eb7a integration: remove parallel testing
We cannot do testing in parallel since leak testing will detect the goroutines
in other tests running in parallel.
2016-05-13 12:01:25 -07:00
Anthony Romano 120020fa9c clientv3/concurrency: use session id for election keys to avoid deadlock 2016-05-13 10:07:35 -07:00
Anthony Romano 393725fe5f clientv3/concurrency: ctx-ize Leader(), Resign(), and Unlock() 2016-05-13 10:07:35 -07:00
Gyu-Ho Lee ef44f71da9 *: update LICENSE header 2016-05-12 20:51:48 -07:00
Xiang Li a880e9c7cb Merge pull request #5332 from xiang90/sl
*: cancel required leader streams when memeber lost its leader
2016-05-12 20:24:34 -07:00
Xiang Li 9c103dd0de *: cancel required leader streams when memeber lost its leader 2016-05-12 19:42:21 -07:00
Gyu-Ho Lee 78ae4b92a6 integration: fix NewClientV3 error messages 2016-05-12 10:26:27 -07:00
Xiang Li 19221b33cc *: etcd member rejects unary call with leader requirement when it does not have leader 2016-05-11 16:34:34 -07:00
Anthony Romano 8b52fd0d2d clientv3: gracefully handle watcher resume on compacted revision
Fixes #5239
2016-05-03 09:30:53 -07:00
Gyu-Ho Lee ec1fdd3938 integration: test with new server errors 2016-04-29 12:00:26 -07:00
Gyu-Ho Lee bef5be42b5 integration: add quota backend bytes option 2016-04-28 12:15:31 -07:00
Anthony Romano 11ec94b7e8 Merge pull request #5218 from heyitsanthony/fix-issue-3699
integration: wait for ReadyNotify in Issue3699 test
2016-04-28 10:48:08 -07:00
Anthony Romano 7c666b533a Merge pull request #5221 from heyitsanthony/parallel-e2e-integration
test: run e2e and integration tests in parallel
2016-04-28 10:30:40 -07:00
Anthony Romano 85edd66c65 test: run e2e and integration tests in parallel 2016-04-28 10:17:40 -07:00
Xiang Li 6ee5f9c677 etcdserver: add timeout for processing v3 request 2016-04-28 08:52:17 -07:00
Anthony Romano d814e9dc35 integration: wait for ReadyNotify in Issue3699 test
Fixes #5147
2016-04-27 22:04:07 -07:00
Xiang Li 434f2c356d etcdserver: do not serve requests before finish the first internal proposal 2016-04-27 15:46:31 -07:00
Anthony Romano 22797c7185 clientv3/concurrency: use session lease id for mutex keys
With randomized keys, if the connection goes down, but the session remains,
the client would need complicated recovery logic to avoid deadlock.
Instead, bind the session's lease id to the lock entry; if a session tries
to reacquire the lock it will reassume its old place in the wait list.
2016-04-26 17:37:26 -07:00
Anthony Romano b7ac758969 *: rename storage package to mvcc 2016-04-25 15:25:51 -07:00
Anthony Romano 2927c90fae integration: wait for alarm in TestV3StorageQuotaApply
Fixes #4974
2016-04-21 20:53:43 -07:00
Anthony Romano d72bcdc156 storage: have Range on rev=0 work even if compacted to current revision 2016-04-13 21:00:35 -07:00
Gyu-Ho Lee 89f8e66682 *: fixes based on ineffassign 2016-04-13 10:41:58 -07:00
Anthony Romano c5b8e8dc88 etcdserver: set txn header revision to store revision following txn 2016-04-11 17:03:05 -07:00
Gyu-Ho Lee 9108af9046 *: clean up from go vet, misspell 2016-04-10 23:16:56 -07:00
Anthony Romano dc17eaace7 *: rename Lease Create to Grant
Creating a lease through the client API interface union looked like
"c.Create(...)"-- the method name wasn't very descriptive.
2016-04-07 12:28:14 -07:00
Xiang Li 21096bf27f Merge pull request #4963 from xiang90/ht
*: mv etcdhttp into api pkg
2016-04-04 18:40:29 -07:00
Hitoshi Mitake 88306c9fa7 etcdserver, clientv3: let progressReportIntervalMilliseconds be private
progressReportIntervalMilliseconds (old
ProgressReportIntervalMilliseconds) is accessed by multiple goroutines
and it is reported as race.

For avoiding this report, this commit wraps the variable with
functions. They access the variable with atomic operations so the race
won't be reported.
2016-04-05 09:12:17 +09:00
Xiang Li 2c50eb240e *: mv etcdhttp into api pkg 2016-04-04 16:31:35 -07:00
Gyu-Ho Lee b0cc0e443c *: clean up if, bool comparison 2016-04-02 12:55:11 -07:00
Anthony Romano 9b2c963179 etcdserver: configurable backend size quota
Configurable with the flag --experimental-quota-backend-bytes and
through ServerConfig.QuotaBackendBytes.

Fixes #4894
2016-03-29 18:39:25 -07:00
Anthony Romano d533c14881 Merge pull request #4876 from heyitsanthony/integration-races
*: fix races from clientv3/integration tests
2016-03-29 09:10:53 -07:00
Anthony Romano 3fbacf4be2 v3rpc: move Hash RPC to Maintenance service 2016-03-28 17:15:58 -07:00
Anthony Romano 91dc6b29a6 clientv3/integration: fix race when setting progress report interval 2016-03-28 16:08:18 -07:00
Anthony Romano 6e3a0948e4 Merge pull request #4868 from heyitsanthony/api-quota
etcdserver: storage quotas
2016-03-28 15:15:57 -07:00
Anthony Romano a403a94d7b etcdserver: cap new keys on space alarm 2016-03-28 14:56:26 -07:00
Anthony Romano 9c8253c543 etcdserver, v3rpc: space quotas 2016-03-28 14:56:26 -07:00
Anthony Romano 384c3ec907 integration: don't call rand.Intn in TestSTMConflict on 0
Fixes #4874
2016-03-28 13:06:07 -07:00
Xiang Li 70a9391378 *: enable v3 by default 2016-03-23 17:01:36 -07:00
Xiang Li 900a61b023 *: http and https on the same port 2016-03-23 10:28:38 -07:00
Anthony Romano bd832e5b0a *: migrate Godeps to vendor/ 2016-03-22 17:10:28 -07:00
Gyu-Ho Lee dae7e009b0 *: godoc clean up 2016-03-19 14:19:23 -07:00
Anthony Romano 58c8521920 Merge pull request #4792 from heyitsanthony/snip-snip
clientv3: break etcdserver dependency
2016-03-17 12:24:54 -07:00
Anthony Romano 44753594ec v3rpc: move errors to v3rpc/rpctypes
Fixes #4771
2016-03-17 11:52:34 -07:00
Xiang Li e6c39108a7 integration: add TestV3LeaseFailover test 2016-03-17 10:17:51 -07:00
Anthony Romano bc37a32062 clientv3/concurrency: software transactional memory
Repeatable read and serialized read STM implementations.
2016-03-16 11:23:06 -07:00
Xiang Li 39d307572e integration: fix TestV3WatchFutureRevision
Fix https://github.com/coreos/etcd/issues/4730.

Previously we put keys async and there might be a race when
the watch triggers before the put receives the response. When that
happens, put might fails to get the response since we shutdown the server
when watch triggers.
2016-03-09 09:55:52 -08:00
Anthony Romano 78132c9b5b clientv3: use tls.Config in clientv3.Config
Fixes #4648
2016-03-07 16:08:40 -08:00
Anthony Romano 632461cc50 Merge pull request #4706 from heyitsanthony/fix-client-close-deadlock
clientv3: don't deadlock on Close with broken connection
2016-03-07 14:02:12 -08:00
Anthony Romano d21d2e6624 clientv3: don't deadlock on Close with broken connection
Fixes #4679
2016-03-07 13:46:54 -08:00
Xiang Li 036ed87c6d *: support watch from future revision 2016-03-07 10:57:30 -08:00
Anthony Romano a31f84121b Merge pull request #4699 from heyitsanthony/fix-barrier
storage: use creation revision to compute txn event types
2016-03-05 19:17:56 -08:00
Anthony Romano 713f7c056f storage: use creation revision to compute txn event types
Fixes #4688
2016-03-05 19:03:07 -08:00
Xiang Li 633a0bdf55 integration: add test for full range watching 2016-03-05 18:52:41 -08:00
Anthony Romano 360aafec76 clientv3: include a context in Client
Useful for clean up tasks
2016-03-04 09:20:44 -08:00
Xiang Li 44151ba531 integration: add TestV3PutOnNonExistLease 2016-03-03 20:32:44 -08:00
Xiang Li b1521570b6 v3api: support progress 2016-03-03 13:58:15 -08:00
Anthony Romano 3327858a54 clientv3/concurrency: move election recipe into clientv3 2016-03-03 00:22:19 -08:00
Anthony Romano 8dbc6cfd43 etcdserver: ranges in watcher rpc protocol
protocol change so watch requests are ranges; server rejects non-prefix ranges
2016-02-29 14:03:27 -08:00
Anthony Romano 4fb25d5f0e Merge pull request #4613 from heyitsanthony/clientv3-composite
clientv3: compose API interfaces into client struct
2016-02-29 11:23:34 -08:00
Gyu-Ho Lee 4a0a83380e *: return -1 for canceled watch request 2016-02-26 14:26:46 -08:00
Anthony Romano 3e57bbf317 clientv3: use default client kv 2016-02-25 18:13:26 -08:00
Anthony Romano 5f62c05a6d clientv3: compose all clientv3 APIs into client struct 2016-02-25 18:13:26 -08:00
Gyu-Ho Lee 8f3981c651 Merge pull request #4612 from gyuho/watch_not_panic
*: watch true cancel, created for wrong rev
2016-02-24 22:07:03 -07:00
Gyu-Ho Lee a78604dacb *: watch true cancel, created for wrong rev
This sets Created and Cancel true in pb.WatchResponse
when it has received wrong start revision instead of
panic. So that clientv3 can set 'Canceled' in WatchResponse
as well.

Fix https://github.com/coreos/etcd/issues/4610.
2016-02-24 20:56:17 -08:00
Anthony Romano d4b2044eb1 clientv3/concurrency: Mutex 2016-02-24 17:23:40 -08:00
Anthony Romano 20b4336cdb clientv3/concurrency: Session
A client may bind itself to a session lease to signal its
continued in participation with the cluster.
2016-02-24 16:40:16 -08:00
Gyu-Ho Lee fc86e1ded1 *: return the number of deleted keys
For https://github.com/coreos/etcd/issues/4576.
2016-02-21 17:59:21 -08:00
Anthony Romano 3bb3351ca0 contrib/recipes: use clientv3 kv API 2016-02-21 14:43:41 -08:00
Anthony Romano 270fa00e54 integration: fix go vet -shadow error
breaking go tip
2016-02-18 12:26:35 -08:00
Anthony Romano ef2d3feca6 Merge pull request #4528 from heyitsanthony/fix-watchcurrev
fix several watcher races
2016-02-17 14:26:33 -08:00
Anthony Romano 155412bbfa integration: overlapped create and put v3 watcher test 2016-02-17 14:03:52 -08:00
Anthony Romano 019a145304 integration: put keys after watcher ack in TestV3WatchFromCurrentRevision
Watcher would miss events since the keys would be created after
sending the watcher request but before etcd registered the watcher.
2016-02-17 01:06:52 -08:00
Anthony Romano 0f7f375043 contrib/recipes: fix revision race in double barrier
current kv revision might be ahead of ready put event; watch using key's mod
revision instead.

Fixes #4425
2016-02-13 11:57:12 -08:00
Anthony Romano 6851fffdfb clientv3: support >= Range requests
Turns out grpc will convert an empty byte string to nil, so use "\0" to
indicate Range on >= key in v3 grpc protocol.
2016-02-12 14:06:59 -08:00
Xiang Li 1d02559ae5 Merge pull request #4501 from xiang90/lt
integration: test switch lease via put
2016-02-12 13:20:05 -08:00
Xiang Li be1534812a integration: give watch stream a timeout to fail the test fast 2016-02-12 09:58:15 -08:00
Xiang Li 8ed9ebf3e1 integration: WaitResponse -> waitResponse 2016-02-12 09:50:29 -08:00
Xiang Li 5f1d30b76a integration: move watch tests to v3_watch_test.go 2016-02-12 09:47:33 -08:00
Xiang Li fa45e13073 integration: test switch lease via put 2016-02-11 22:04:54 -08:00
Anthony Romano ed29bc3221 integration: add key usage to server.crt
curl handshake was failing; related: #209
2016-02-09 15:34:53 -08:00
Xiang Li 35567221a7 *: limit request size for v3 2016-02-08 12:54:03 -08:00
Anthony Romano 9523c2d29f clientv3/integration: txn tests 2016-02-04 08:40:24 -08:00
Anthony Romano 16dcade07d Merge pull request #4402 from heyitsanthony/minority-failure
rafthttp: add leader to transport if peer does not exist
2016-02-03 15:31:01 -08:00
Anthony Romano db0b505de5 rafthttp: add requester to transport if peer does not exist
cluster integration now supports adding members with stopped nodes, too

Fixes #3699
2016-02-03 14:16:46 -08:00
Gyu-Ho Lee 2d197ac9e8 *: add kv Hash method (for testing purpose) 2016-02-03 12:52:39 -08:00
Xiang Li cb30d6e6f8 Merge pull request #4393 from xiang90/fix_test
clientv3/integration: fix member remove
2016-02-02 21:10:40 -08:00
Xiang Li c7876d4111 clientv3/integration: fix member remove
Do not connect to the member to remove.
2016-02-02 20:49:00 -08:00
Anthony Romano 2919be91b9 Merge pull request #4387 from heyitsanthony/integration-cluster-speedup
integration: decrease timeout for isMemberBootstrapped
2016-02-02 14:45:18 -08:00
Anthony Romano 9ae8d85049 integration: decrease timeout for isMemberBootstrapped
Spending seconds(!) when it would fail anyway.

integration/TestV3 (before): 100.670
integration/TestV3 (after): 29.571
2016-02-02 14:34:58 -08:00
Xiang Li 6f72b31316 integration: add test promote and move lease tests to lease_test.go 2016-02-02 13:45:11 -08:00
Anthony Romano c5c5063efe etcdserver: reject v3 txns with duplicate put keys
An API check to support PR #4363; bad requests didn't return an error.
2016-02-02 12:32:33 -08:00
Anthony Romano b74a42b286 clientv3: support unix endpoints 2016-02-01 22:24:42 -08:00
Anthony Romano 64766e9d6a Merge pull request #4366 from heyitsanthony/fix-rejectinsecure
integration: accept transient failure in TestGRPCRejectInsecureClient
2016-02-01 11:58:41 -08:00
Anthony Romano 7d8ae8af78 integration: accept transient failure state in TestGRPCRejectInsecureClient 2016-02-01 11:32:22 -08:00
Anthony Romano 20461ab11a *: fix many typos 2016-01-31 21:42:39 -08:00
Gyu-Ho Lee f6215574f2 *: kv range to return current revision
This changes the behavior of KV's range and tx range to return
current revision rather than range revision. This makes populating
range response header easier.
2016-01-30 17:37:34 -08:00
Anthony Romano ca9bd575b1 integration: v3 grpc tls tests 2016-01-29 16:38:52 -08:00
Anthony Romano 60c037f1c3 integration: add client tls support 2016-01-29 16:38:11 -08:00
Anthony Romano 4634874d99 etcdmain, integration, v3rpc: consolidate grpc server setup 2016-01-29 16:38:11 -08:00
Shawn Smith 7abbd31fcd integration: add want IDs to Errorf calls 2016-01-29 15:57:02 +09:00
Gyu-Ho Lee 1767788074 *: expose integration functions for clientv3 2016-01-28 21:21:34 -08:00
Gyu-Ho Lee 599ecebdec integration: fix shadowed variables based on govet 2016-01-28 11:42:58 -08:00
Anthony Romano 6f0cc54541 contrib/recipes: add election and double barrier recipes
these recipes rely on leases so they weren't included in the last batch
2016-01-27 15:44:51 -08:00
Anthony Romano 56fce9f386 contrib/recipes, integration: use clientv3
updating both together since there's a circular dependency
2016-01-27 14:37:51 -08:00
Gyu-Ho Lee 51e62aa007 integration: update gRPC, proto interface 2016-01-26 17:41:27 -08:00
Xiang Li 59c6735c3c Merge pull request #4282 from xiang90/range_invalid
etcdserver: check invalid range in txn
2016-01-25 22:23:55 -08:00
Xiang Li c8bf77c722 v3rpc: check max ops in txn 2016-01-25 21:04:19 -08:00
Xiang Li 128b5e7387 etcdserver: check invalid range in txn 2016-01-25 20:21:17 -08:00
Gyu-Ho Lee 77cf05364d Merge pull request #4261 from gyuho/racey_e2e
*: detect leaky goroutines, fix leaks
2016-01-24 18:17:49 -08:00
Gyu-Ho Lee 96d2ee20e3 *: detect leaky goroutines, fix leaks
gexpect.Interact leaks. This adds ReadLine method to wait for the leaky
goroutine to accept an EOF.

Fixes https://github.com/coreos/etcd/issues/4258.

Reference: https://github.com/coreos/etcd/pull/4261#issuecomment-174198945.
2016-01-23 13:52:41 -08:00
Anthony Romano 5099bf6f7a Merge pull request #4269 from heyitsanthony/v3-reject-put-bogus-lease
etcdserver: return error when putting a key with a bad lease id
2016-01-22 21:25:17 -08:00