Commit Graph

161 Commits (887db5a3dbcbdc560b6de20202e784d686171321)

Author SHA1 Message Date
Anthony Romano 395cf7de51 grpcproxy: reject invalid watch ranges 2016-11-11 10:14:35 -08:00
Anthony Romano ec459c2185 grpcproxy: rework watcher organization
The single watcher / group watcher distinction limited and
complicated watcher coalescing more than necessary. Reworked:

Each server watcher is represented by a WatchBroadcast, each
client "Watcher" attaches to some WatchBroadcast. WatchBroadcasts
hold all WatchBroadcast instances for a range. WatchRanges holds
all WatchBroadcasts for the proxy.

WatchProxyStreams represent a grpc watch stream between the proxy and
a client. When a client requests a new watcher through its grpc stream,
the ProxyStream will allocate a Watcher and WatchRanges assigns it to
some WatchBroadcast based on its range.

Coalescing is done by WatchBroadcasts when it receives an update
notification from a WatchBroadcast.

Supports leader failure detection so watches on a bad member
can migrate to other members. Coincidentally, Fixes #6303.
2016-11-11 10:14:35 -08:00
Anthony Romano a4dcceb8aa grpcproxy: update cache based on txn response
Fixes more hangs in TestSTMConflict.
2016-11-09 12:11:38 -08:00
Anthony Romano bf08a6142c grpcproxy: invalidate comparison keys after txn
If the txn comparison block makes claims about a key's current
state, then it may say a key has been updated. Future range/txn
operations may expect this update to eventually be propagated through
the cluster and show up in serialized requests. To avoid spinning
forever on txn/serialized range loops, invalidate the comparison keys.
2016-11-04 09:46:43 -07:00
Anthony Romano 8ec4215279 grpcproxy: reliably track rid in watchergroups
Couldn't find watcher group from rid on server stream close, leading to
the watcher group sending on a closed channel.

Also got rid of send closing the watcher stream if the buffer is full,
this could lead to a send after close while broadcasting to all receivers.
Instead, if a send times out then the server stream is canceled.

Fixes #6739
2016-11-02 14:42:02 -07:00
Anthony Romano 7ef17d3e97 grpcproxy: add SetHeader support to ServerStream
Fixes #6726
2016-11-01 13:28:02 -07:00
Anthony Romano 06e2ce116c Merge pull request #6704 from heyitsanthony/proxy-broadcast-race
grpcproxy: fix race on watcher revision
2016-10-24 09:17:29 -07:00
Anthony Romano bd1985d84b grpcproxy: fix race on watcher revision
Was racing between broadcast setting the watchgroup revision
and joining single watchers.
2016-10-21 16:09:39 -07:00
Anthony Romano 65eb3038fe grpcproxy: respect {min,max}{create,mod} revision
Mutexes were breaking in proxy integration tests.
2016-10-21 15:02:00 -07:00
Xiang Li 962433c17f *: set repo correctly for logging 2016-10-03 17:03:22 +08:00
Anthony Romano 6ac284a577 grpcproxy: use valid progress notification in broadcast test 2016-09-29 10:45:25 -07:00
Gyu-Ho Lee 78cfc8db95 grpcproxy: implement 'LeaseTimeToLive' 2016-09-09 08:14:46 +09:00
Xiang Li 56cfe40184 grpcproxy: fix a data race 2016-09-03 07:53:18 -07:00
Xiang Li b56ee178d5 grpcproxy: handle overloaded stream 2016-09-03 07:49:20 -07:00
Xiang Li 0d07154926 Merge pull request #6340 from xiang90/fix_double_create
grpcproxy: fix double create event
2016-09-02 16:37:29 -07:00
Xiang Li 805d4cbd93 grpcproxy: fix double create event 2016-09-02 16:02:46 -07:00
Xiang Li eded62e60c grpcproxy: stop watchers in watch groups 2016-09-02 16:01:11 -07:00
Xiang Li 8cd47c4348 grpcproxy: fix more issues in watch path 2016-09-02 15:13:21 -07:00
Xiang Li 51b4d6b7a8 grpcproxy: support cancel watcher
We do not wait for the cancellation from actual etcd server,
but generate it at the proxy side. The rule is to return the
latest rev that the watcher has seen. This should be good
enough for most use cases if not all.
2016-09-02 12:36:47 -07:00
Anthony Romano d3ecebd14e grpcproxy: shut down watcher proxy when client context is done 2016-09-01 15:20:50 -07:00
Xiang Li 9ef0f5ef8a grpcproxy: fix stream closing issue 2016-09-01 09:35:56 -07:00
Xiang Li b982c80c14 grpcproxy: fix data race 2016-08-31 16:52:04 -07:00
Gyu-Ho Lee 86bf2bc443 grpcproxy: fix recursive Context method 2016-08-29 11:37:35 -07:00
Xiang Li c8bbb8c53e grpc-proxy: invalidate cache entries when there is a put/delete 2016-08-19 12:52:19 -07:00
Gyu-Ho Lee a4ef36c8bf proxy/grpcproxy: fix nil-map assign to 'singles' 2016-08-15 15:48:45 -07:00
Gyu-Ho Lee f14fd43548 proxy/httpproxy: fix httputil.RequestCanceler 2016-08-14 14:37:08 -07:00
Gyu-Ho Lee 19b35c939a proxy/grpcproxy: fix spell 'gropu' to 'group' 2016-08-13 20:55:15 -07:00
Xiang Li 57c68ab1db grpcproxy: handle create event 2016-08-02 20:51:30 -07:00
Anthony Romano 2b4c37f54a grpcproxy: don't leak goroutines on watch proxy shutdown 2016-07-25 09:34:36 -07:00
Anthony Romano 418bb5e176 grpcproxy: bind clientv3.Watcher on initialization 2016-07-25 09:34:36 -07:00
Anthony Romano 4fa9363aca grpcproxy: client watch adapter 2016-07-25 09:34:36 -07:00
Gyu-Ho Lee abbefc9e25 grpcproxy: define 'watchergroups' in pointer
To avoid copying mutex lock values
2016-07-22 16:54:11 -07:00
Anthony Romano 768ccb8c10 grpcproxy: respect prev_kv flag 2016-07-20 15:58:33 -07:00
Anthony Romano 54df540c2c grpcproxy: wrapper from pb.KVServer to pb.KVClient 2016-07-20 15:40:33 -07:00
Xiang Li 783675f91c grpcproxy: do not send duplicate events to watchers 2016-07-19 10:14:57 -07:00
Xiang Li a61862acc7 Merge pull request #5977 from xiang90/b_proxy
grpcproxy: return interface
2016-07-18 19:12:43 -07:00
Xiang Li 5271cf0160 grpcproxy: return interface 2016-07-18 16:47:58 -07:00
Xiang Li 7bfe75cbf3 Merge pull request #5963 from xiang90/p_filter
grpcproxy: add filter to watcher
2016-07-18 14:56:10 -07:00
Xiang Li e1892e264d grpcproxy: add auth 2016-07-18 14:26:22 -07:00
Xiang Li fd24340903 grpcproxy: add maintenance proxy 2016-07-18 13:31:03 -07:00
Xiang Li 58aa3483c3 grpcproxy: add filter to watcher 2016-07-18 13:02:34 -07:00
Xiang Li 3f74e9db0d *: add lease proxy 2016-07-18 12:06:59 -07:00
Xiang Li bdfbd26e94 *: add cluster proxy 2016-07-16 12:15:32 -07:00
Xiang Li 5f3aa43899 grpcproxy: initial watch proxy 2016-07-15 14:30:45 -07:00
Gyu-Ho Lee 244b1d7d20 tcpproxy: add start logging line 2016-07-06 14:21:26 -07:00
Xiang Li b7d0497c47 Merge pull request #5807 from xiang90/gproxy
*: initial implementation of grpc-proxy
2016-06-29 13:28:57 -07:00
Xiang Li 6d8c647db8 *: initial implementation of grpc-proxy 2016-06-29 12:06:04 -07:00
mqliang 2e2f405b1e proxy:replace c with client to improve readability 2016-06-29 11:30:03 +08:00
mqliang f28a87d835 proxy: implement compaction 2016-06-29 11:28:10 +08:00
Anthony Romano 74b13aab61 grpcproxy: fix go vet warnings 2016-06-17 11:41:49 -07:00
mqliang 5676c5cf26 proxy: serve range request from proxy cache if set serializable 2016-06-15 14:12:36 +08:00
Gyu-Ho Lee 1610391449 *: following changes for proto update 2016-06-07 13:33:03 -07:00
Anthony Romano fc7da09d67 *: add missing godoc package descriptions
Fixes #4074
2016-05-27 15:15:26 -07:00
Xiang Li c73e8fd946 httpproxy: init the rand that we use to randomize endpoints
This is actually does not change anything. The endpoints are already
randomized before feeding into proxy. But it makes the proxy more safe.
2016-05-27 10:28:03 -07:00
Gyu-Ho Lee dd8a36820e proxy/httpproxy: fix v2 proxy log header
Replace all with capnslog
2016-05-23 15:45:49 -07:00
Xiang Li ec2ac72585 proxy: initial grpc kv service proxy 2016-05-13 23:00:29 -07:00
Gyu-Ho Lee ef44f71da9 *: update LICENSE header 2016-05-12 20:51:48 -07:00
Gyu-Ho Lee a32aabc377 proxy/tcpproxy: add more logs 2016-05-12 17:48:36 -07:00
Gyu-Ho Lee 25d9169e9a httpproxy: fix capnslog log path
We changed the package path, so log paths needs to be updated as well.
2016-05-12 15:56:40 -07:00
Xiang Li a300be92dc *: initial support for gatway
etcd gatway is a simple l4 gateway that forwards tcp connections to
the given endpoints.
2016-05-11 09:44:50 -07:00
Anthony Romano a76f5f5ed2 httpproxy: fix race on getting close notifier channel
Fixes #5267
2016-05-04 09:32:26 -07:00
朱鹏 67db28f979 proxy: enable http2 for connecting to members
enable http2 when transport specified a custom TLS config, which was
not automatically enable.

Issue 5033
2016-04-15 10:16:26 +08:00
Xiang Li 4a913ae60a proxy: initial userspace tcp proxy 2016-04-14 10:14:30 -07:00
Xiang Li 66d2ae7a39 proxy: move http related thing to httpproxy 2016-04-13 21:09:26 -07:00
Gyu-Ho Lee 6abbdcdc06 proxy: simplify channel receive, add missing function call 2016-04-07 12:24:17 -07:00
Gyu-Ho Lee 6e6d64fb9b *: clean up unused vars, functions
With help from https://github.com/dominikh/go-unused.
IsNetTimeoutError seems useful, so moved to pkg/netutil.
2016-04-06 21:33:55 -07:00
Xiang Li 2c50eb240e *: mv etcdhttp into api pkg 2016-04-04 16:31:35 -07:00
Anthony Romano bd832e5b0a *: migrate Godeps to vendor/ 2016-03-22 17:10:28 -07:00
Gyu-Ho Lee a8e72b6285 proxy: prints out when endpoints are found 2016-02-01 13:08:21 -08:00
Gyu-Ho Lee 81229dbea9 *: add missing package descriptions
This adds and updates package descriptions in etcd projects.
And also deletes some duplicate LICENSE statements.
2015-11-17 20:54:10 -08:00
Gyu-Ho Lee ca25ed3ad2 proxy: expedite proxy refresh given no endpoints
This fixes coreos#3647 by giving shorter proxy refresh interval whenever
there is no endpoints found. Deleted sleep command in Procfile and proxy
documentation accordingly.
2015-11-07 07:27:39 -08:00
Xiang Li c9be719d92 Merge pull request #3579 from gyuho/etcdserver/etcdhttp/httptypes/errors.go-WriteTo-returns-error
httptypes: WriteTo to return error
2015-09-25 14:31:48 -07:00
Gyu-Ho Lee 85f4475f62 httptypes/errors: HTTPError.WriteTo returns error
Squashing all commits into this one
(from https://github.com/coreos/etcd/pull/357).

Thanks,
2015-09-25 08:06:26 -07:00
Guohua ouyang e35eeeae42 proxy: improve log for retrying an unavailable endpoint
Fixes #3541

Signed-off-by: Guohua ouyang <guohuaouyang@gmail.com>
2015-09-25 07:36:49 +08:00
Xiang Li 06180be154 Merge pull request #3533 from xiang90/proxy
proxy: expose proxy configuration
2015-09-18 14:18:06 -07:00
Xiang Li ac29432aab proxy: add a test for configHandler 2015-09-18 13:43:54 -07:00
Jonathan Boulle 7848ac3979 *: add missing license headers 2015-09-15 14:09:01 -07:00
Xiang Li e079f87410 proxy: expose proxy configuration 2015-09-15 10:27:51 -07:00
Yicheng Qin ece39c9462 proxy: always cancel in-flight request
This fits the way for go1.5 to cancel request.
2015-08-25 08:59:59 -07:00
Xiang Li 6b23a8131f *: test gofmt with -s and fix reported issues 2015-08-21 18:52:16 -07:00
Michal Witkowski 7bca757d09 *: add metrics to `store` and `proxy`. 2015-07-07 16:01:51 +01:00
Xiang Li cbe00e4415 Merge pull request #2967 from webner/feature/proxy-config
proxy: added endpoint refresh and timeout configuration values
2015-07-03 11:51:15 -07:00
Wolfgang Ebner 954e416bf6 proxy: fixed director.go formatting 2015-07-03 14:11:40 +02:00
Xiang Li 7f7e2cc79d Merge pull request #3034 from philips/replace-maximal-with-maximum
*: docs and code %s%maximal%maximum%g
2015-06-22 16:24:01 -07:00
Brandon Philips 740187f199 *: docs and code %s%maximal%maximum%g
maximum is a more common word, use it instead
2015-06-22 16:06:57 -07:00
Wolfgang Ebner a6e6186477 proxy: always set requestClosed flag when client closes the connection prematurely 2015-06-19 08:45:45 +02:00
Wolfgang Ebner 5e31854afd proxy: use atomic operations for requestCanceled flag 2015-06-18 20:56:28 +02:00
Wolfgang Ebner 864ce5f946 proxy: handle canceled proxy request gracefully
when a client of the proxy server cancels a request the proxy should not
set the endpoint state to unavailable
2015-06-18 20:52:52 +02:00
Wolfgang Ebner 5854d0e8a9 proxy: removed unused refreshInterval variable in director structure 2015-06-16 12:17:08 +02:00
Wolfgang Ebner 1264dbe24d proxy: added endpoint refresh and timeout configuration values
the default dial timeout was set to 30 seconds this made the proxy a pain to use
in failure scenarios.

fixes 2862
2015-06-13 09:42:18 +02:00
Xiang Li 612ecbc89d proxy: fix test 2015-06-10 16:31:42 -07:00
Xiang Li a8af787971 Merge pull request #2902 from BlueDragonX/bug-proxyreq-closed
Reuse a bytes buffer as proxy request body.
2015-06-02 10:37:48 -07:00
Ryan Bourgeois 4e85f932e0 proxy: Reuse a bytes buffer as proxy request body.
The call to transport.RoundTrip closes the request body regardless of
the value of request.Closed. This causes subsequent calls to RoundTrip
using the same request body to fail.

Fixes #2895
2015-06-02 10:27:20 -07:00
Yicheng Qin 0589afe605 etcdmain: increase maxIdleConnsPerHost in proxy transport
This PR set maxIdleConnsPerHost to 128 to let proxy handle 128 concurrent
requests in long term smoothly.
If the number of concurrent requests is bigger than this value,
proxy needs to create one new connection when handling each request in
the delta, which is bad because the creation consumes resource and may
eat up your ephemeral port.
2015-06-01 16:19:36 -07:00
Shota Fukumori (sora_h) a68efe7d1e proxy: Fix connection leak when client disconnect
established connections were leaked when client disconnected before
proxyreq completes. This happens all time for wait=true requests.
2015-04-30 11:41:42 +09:00
Xiang Li 1b4bcedf99 Merge pull request #2637 from bakins/proxy-randomize-endpoints
proxy: shuffle endpoints
2015-04-07 14:12:50 -07:00
Brian Akins e1622cd22c proxy: shuffle endpoints
Shuffle endpoitns to avoid being "stuck" to a single cluster member.
2015-04-07 15:40:29 -04:00
Alex Crawford d9ad6aa2a9 *: update to use IANA-assigned ports 2015-04-06 13:49:43 -07:00
Jonathan Boulle f1ed69e883 *: switch to line comments for copyright
Build tags are not compatible with block comments.
Also adds copyright header to a few places it was missing.
2015-01-26 09:53:30 -08:00
Xiang Li 424377f859 proxy: add a todo for logging 2014-11-10 16:37:15 -08:00