Commit Graph

72 Commits (5d6c6ad20e8fd144d0d22056009ffec0d2c66750)

Author SHA1 Message Date
Gyu-Ho Lee 334554f655 embed: wait up to request timeout for pending RPCs when closing
Both grpc.Server.Stop and grpc.Server.GracefulStop close the listeners
first, to stop accepting the new connections. GracefulStop blocks until
all clients close their open transports(connections). Unary RPCs
only take a few seconds to finish. Stream RPCs, like watch, might never
close the connections from client side, thus making gRPC server wait
forever.

This patch still calls GracefulStop, but waits up to 10s before manually
closing the open transports.

Address https://github.com/coreos/etcd/issues/8224.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-07-14 15:31:40 -07:00
Gyu-Ho Lee 9bb5ede659 embed: configure 'ListenMetricsUrls'
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-07-14 11:14:05 -07:00
Anthony Romano e29db923bc *: move v2http handlers without /v2 prefix to etcdhttp
Lets --enable-v2=false configurations provide /metrics, /health, etc.

Fixes #8167
2017-07-07 18:35:57 -07:00
Anthony Romano 63350f5ac1 embed: connect json gateway with user-provided listen address
net.Listener says its address is [::] when given 0.0.0.0, breaking
hosts that have ipv6 disabled.

Fixes #8151
Fixes #7961
2017-07-06 14:24:29 -07:00
Xiang Li 0fe8fdcb29 Merge pull request #8123 from yudai/revision_compactor
Compactor: Add Revisional compactor
2017-06-22 16:34:28 -07:00
Iwasaki Yudai a3f8f47422 *: add Revision compactor 2017-06-21 15:41:07 -07:00
Anthony Romano 87d16af2e2 embed: use transport TLS listener for client listener for CRLs 2017-06-19 15:23:41 -07:00
Anthony Romano dcf52bbfac etcdserver, embed, integration: don't use pointer for ServerConfig
ServerConfig is owned by etdcserver and unshared, so don't pass or store by
pointer. Also removes duplicated field 'snapCount'.
2017-06-15 13:02:13 -07:00
fanmin shi 2f84f3d8d8 Merge pull request #7968 from fanminshi/make_maxRequestBytes_configurable
etcd: make max request bytes configurable
2017-05-25 15:54:24 -07:00
Anthony Romano 212a1efd47 Merge pull request #7965 from heyitsanthony/shared-grpc-conn
embed: share grpc connection for grpc json services
2017-05-25 14:35:33 -07:00
fanmin shi 9e7740011b etcdserver: add --max-request-bytes flag 2017-05-25 11:01:38 -07:00
fanmin shi ae7ddfb483 etcdserver: add --max-txn-ops flag
--max-txn-ops allows users to define the maximum transaction operations
for each txn request. it defaults at 128.

Fixes #7826
2017-05-24 10:32:32 -07:00
Anthony Romano db7ab961bf embed: share grpc connection for grpc json services 2017-05-22 12:59:13 -07:00
Anthony Romano aac2292ab5 Merge pull request #7882 from heyitsanthony/srv-priority
gateway: DNS SRV priority
2017-05-08 14:17:04 -07:00
Gyu-Ho Lee 7d21d6c894 embed: gracefully close peer handlers on shutdown
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-05-06 07:47:23 -07:00
Anthony Romano 07ad18178d pkg/srv: package for SRV utilities
Trying to decouple the v2 client from SRV code. Can't move
into discovery/ since that creates a circular dependency. So,
give up and move all the SRV code into a new package.
2017-05-05 09:27:59 -07:00
Gyu-Ho Lee b0e9d24fb6 embed: fix blocking Close before gRPC server start
If 'StartEtcd' returns before starting gRPC server
(e.g. mismatch snapshot, misconfiguration),
receiving from grpcServerC blocks forever. This patch
just closes the channel to not block on grpcServerC,
and proceeds to next stop operations in Close.

This was masking the issues in https://github.com/coreos/etcd/issues/7834

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-05-01 16:41:13 -07:00
Anthony Romano 393e4335b7 *: put gateway stubs into their own packages
Fixes #7773
2017-04-19 13:09:06 -07:00
Gyu-Ho Lee ba299bcaaf embed: signal 'grpcServerC' before cmux serve
CMux.Serve blocks, so grpcServerC was never closed.

Fix https://github.com/coreos/etcdlabs/issues/216.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-04-18 17:49:50 -07:00
Gyu-Ho Lee f10a70401b embed: use '*url.URL.Hostname(),Port()' for Go 1.8
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-04-18 12:08:59 -07:00
Gyu-Ho Lee 8aaa1ed911 *: use '*tls.Config.Clone' in Go 1.8
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-04-17 20:08:27 -07:00
Gyu-Ho Lee c407e097e2 embed: gracefully shut down gRPC server
Fix https://github.com/coreos/etcd/issues/7322.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-04-17 14:12:40 -07:00
Anthony Romano 1153e1e7d9 Merge pull request #7687 from heyitsanthony/deny-tls-ipsan
transport: deny incoming peer certs with wrong IP SAN
2017-04-13 15:03:25 -07:00
Gyu-Ho Lee 537c7100b0 embed: remove ReadTimeout TODO
ref. https://github.com/golang/go/issues/9524#issuecomment-271937649

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-04-13 10:55:45 -07:00
Anthony Romano 780a7d359c discovery: remove dead token argument from SRVGetCluster
Can add the argument back when it's actually used something.
2017-04-12 16:49:44 -07:00
Anthony Romano cad1215b18 *: deny incoming peer certs with wrong IP SAN 2017-04-12 13:41:33 -07:00
Anthony Romano 93594006df embed: register grpc-gateway endpoints for v3lock and v3election 2017-04-10 15:21:07 -07:00
Anthony Romano 78422eaa17 embed: add Election service 2017-04-07 16:36:38 -07:00
Anthony Romano d42c1f5131 Merge pull request #7646 from andelf/fix-unix-socket-url
*: fix a bug in handling unix socket urls
2017-04-05 09:24:38 -07:00
andelf 4f27981c46 *: fix a bug in handling unix socket urls
Now use url.Host + url.Path as unix socket path

Fixes #7644
2017-04-05 14:33:13 +08:00
Xiang fd5984af56 *: add pprof flag to grpc proxy 2017-04-03 22:07:17 -07:00
Jonathan Sokolowski d0d4b1378b embed: Delay setting initial cluster for YAML
NewConfig() sets an initial cluster (potentially using a default name)
but we should clear it in the event another discovery option has been
specified.

PR #7517 attempted to address this however it only worked if the name
was left as "default".

(Completely) Fixes #7516
2017-03-31 10:01:42 +11:00
fanmin shi f292a4c953 embed: don't return error when closing on embed etcd
FIXES #7019
2017-03-17 13:41:05 -07:00
Xiang Li 5015480e0c Merge pull request #7517 from jsok/7516-discovery-flags
embed: Delay setting initial cluster
2017-03-16 09:17:42 -07:00
Anthony Romano 2f1542c06d *: use filepath.Join for files 2017-03-16 07:46:06 -07:00
Jonathan Sokolowski 1a91ed0e99 embed: Clear default initial cluster
NewConfig() should sets initial cluster from name but we should clear it
in the event that another discovery option has been specified.

Fixes #7516
2017-03-16 13:59:06 +11:00
Anthony Romano ad1a790116 embed: serve lock api 2017-03-13 15:23:26 -07:00
Hitoshi Mitake f8a290e7ca *: support jwt token in v3 auth API
This commit adds jwt token support in v3 auth API.

Remaining major ToDos:
- Currently token type isn't hidden from etcdserver. In the near
  future the information should be completely invisible from
  etcdserver package.
- Configurable expiration of token. Currently tokens can be valid
  until keys are changed.

How to use:
1. generate keys for signing and verfying jwt tokens:
 $ openssl genrsa -out app.rsa 1024
 $ openssl rsa -in app.rsa -pubout > app.rsa.pub
2.  add command line options to etcd like below:
--auth-token-type jwt \
--auth-jwt-pub-key app.rsa.pub --auth-jwt-priv-key app.rsa \
--auth-jwt-sign-method RS512
3. launch etcd cluster

Below is a performance comparison of serializable read w/ and w/o jwt
token. Every (3) etcd node is executed on a single machine. Signing
method is RS512 and key length is 1024 bit. As the results show, jwt
based token introduces a performance overhead but it would be
acceptable for a case that requires authentication.

w/o jwt token auth (no auth):

Summary:
  Total:        1.6172 secs.
  Slowest:      0.0125 secs.
  Fastest:      0.0001 secs.
  Average:      0.0002 secs.
  Stddev:       0.0004 secs.
  Requests/sec: 6183.5877

Response time histogram:
  0.000 [1]     |
  0.001 [9982]  |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.003 [1]     |
  0.004 [1]     |
  0.005 [0]     |
  0.006 [0]     |
  0.008 [6]     |
  0.009 [0]     |
  0.010 [1]     |
  0.011 [5]     |
  0.013 [3]     |

Latency distribution:
  10% in 0.0001 secs.
  25% in 0.0001 secs.
  50% in 0.0001 secs.
  75% in 0.0001 secs.
  90% in 0.0002 secs.
  95% in 0.0002 secs.
  99% in 0.0003 secs.

w/ jwt token auth:

Summary:
  Total:        2.5364 secs.
  Slowest:      0.0182 secs.
  Fastest:      0.0002 secs.
  Average:      0.0003 secs.
  Stddev:       0.0005 secs.
  Requests/sec: 3942.5185

Response time histogram:
  0.000 [1]     |
  0.002 [9975]  |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.004 [0]     |
  0.006 [1]     |
  0.007 [11]    |
  0.009 [2]     |
  0.011 [4]     |
  0.013 [5]     |
  0.015 [0]     |
  0.016 [0]     |
  0.018 [1]     |

Latency distribution:
  10% in 0.0002 secs.
  25% in 0.0002 secs.
  50% in 0.0002 secs.
  75% in 0.0002 secs.
  90% in 0.0003 secs.
  95% in 0.0003 secs.
  99% in 0.0004 secs.
2017-03-06 19:46:03 -08:00
Gyu-Ho Lee b7ee8f4967 embed: use machine default host only for default value, 0.0.0.0
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-03-03 14:25:34 -08:00
Anthony Romano c4f1e64de7 embed: enable debug endpoint if Debug is set and add net.trace events
/debug/ was only being enabled for Pprof.
2017-02-27 11:34:58 -08:00
felixoid 0f53ad0b84 netutil: add dualstack to linux_route
in v3.1.0 netutil couldn't get default interface for ipv6only hosts

Fixes #7219
2017-01-31 22:19:47 +03:00
Anthony Romano 5a67b0aba6 embed: reject binding listeners to domain names
Fixes #6336
2017-01-26 12:37:34 -08:00
disksing 861cb5cfa2 embed: add example for ServiceRegister. 2017-01-23 10:47:01 +08:00
disksing 4374d944d4 embed: support user defined grpc services.
Fixes #7200
2017-01-22 18:21:19 +08:00
Vimal Kumar f80914fba2 embed/etcd.go: make v2 endpoint optional. fixes #7100 2017-01-20 11:49:52 +05:30
Xiang Li e63059ec31 Merge pull request #7030 from crandles/grpc-histograms
etcdmain: add '--metrics' option
2016-12-28 12:03:53 -08:00
crandl201 36b2d3f5eb etcdmain: add --metrics flag for exposing histogram metrics
this adds a new flag, --metrics, that can be used to enable extensive (histogram) metrics.

Fixes #7024
2016-12-28 13:04:52 -05:00
Gyu-Ho Lee e58287f026 embed: only override default advertised client URL if the client listen URL is 0.0.0.0 2016-12-16 18:31:04 -08:00
Anthony Romano cc931a2319 embed: deep copy user handlers
Shallow copy of user handlers leads to a nil map assignment when
enabling pprof. Since the map is being modified, it should probably
be deep copied into the server context, which fixes the crash.
2016-12-14 10:17:32 -08:00
Anthony Romano 2c06def8ca etcdserver, embed, v2http: move pprof setup to embed
Seems like a better place for prof setup since it's not specific to v2.
2016-12-09 12:37:35 -08:00