Commit Graph

405 Commits (fd5a25fcec67995f1ae322bcaa5b27a39719da4b)

Author SHA1 Message Date
Gyuho Lee 1974785511 etcdmain: accept multiple values in "--log-output"
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-04-17 12:47:06 -07:00
Gyuho Lee c712e08a42 embed,etcdmain: support structured logger
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-04-16 17:36:00 -07:00
Gyuho Lee 041b9069a2 *: configure server logger
- Add/Document "logger" to support structured logging.
  - This makes functional tests run easier, since zap logger
    provides built-in log redirect to files.
  - "etcd --logger-option=zap" to enable structured logging.
- Current "capnslog" will still be used as "default".
  - We may switch the default or deprecate "capnslog" in v3.5.
  - Either way, will clearly be documented.

Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-04-16 17:36:00 -07:00
Gyuho Lee d5bcf66b6b etcdmain: pass logger to "transport.SelfCert"
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-04-02 13:42:17 -07:00
Gyuho Lee 57f036d617 etcdmain: update "help" page
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-27 09:00:13 -07:00
Gyuho Lee 3b1eaaee6e etcdmain: fix "validateClusteringFlags"
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-27 06:25:44 -07:00
Gyuho Lee 27ed129f44 etcdmain: update "CORS", "HostWhitelist" flag parsing
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-27 06:25:44 -07:00
Gyuho Lee 35509bf69c etcdmain: default "--host-whitelist '*'"
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-27 06:25:44 -07:00
Gyuho Lee 35b01b982c etcdmain: use NewUniqueURLsWithExceptions
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-27 06:25:44 -07:00
Gyuho Lee 940200f1ae etcdmain: move "unsafe" flags to bottom
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-23 19:44:24 -07:00
Gyuho Lee 4f1cf30c7d etcdmain: use "TrustedCAFile" in grpc_proxy
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-20 15:48:44 -07:00
Gyuho Lee 82ef3f83f6 etcdmain: deprecate 'ca-file' and 'peer-ca-file' flags
Has been deprecated since 2.1...

Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-20 15:31:32 -07:00
Gyuho Lee 706c760a3d etcdmain: remove deprecated flags from "etcd --help"
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-20 15:31:32 -07:00
Gyuho Lee a66e657cac *: update "pre-vote" flag
Disabled by default, anyway.

Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-19 07:18:57 -07:00
Gyuho Lee 1640cdb044 pkg/flags: clean up, add "SelectiveStringsValue"
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-19 02:23:06 -07:00
Gyuho Lee bec3d975e0 etcdmain: handle JSON "listen-metrics-urls"
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-15 01:24:53 -04:00
Gyuho Lee 4419661fb1 etcdmain: remove "listen-metrics-urls" manual parsing
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-15 01:23:33 -04:00
Gyuho Lee 79d3c93568 etcdmain: replace "--host-whitelist" with "flags.StringSlice"
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-14 21:38:20 -04:00
Gyuho Lee b48d3eb380 etcdmain: add "--experimental-pre-vote" flag
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-06 09:56:32 -08:00
Gyuho Lee 9f0027dfb2 *: do not whitelist empty hosts
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-02-28 19:20:10 -08:00
Gyuho Lee 0179d81f22 etcdmain: add "--host-whitelist" flag
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-02-28 18:23:16 -08:00
Gyuho Lee 0850ccbf45 *: revert "internal/version" change
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-02-26 17:11:40 -08:00
Gyuho Lee 0e65660548 *: revert "internal/discovery" change
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-02-26 17:11:40 -08:00
Gyuho Lee 5a62ceb003 etcdmain: define "defaultGRPCMaxCallSendMsgSize"
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-01-30 09:49:03 -08:00
Gyuho Lee 1b2d470b24
Merge pull request #9252 from hexfusion/fx_help
etcdmain: add discovery-srv-name
2018-01-30 09:32:21 -08:00
Sam Batschelet c2171f9bde etcdmain: add discovery-srv-name 2018-01-30 09:18:55 -05:00
Rene Zbinden 9aabe2a2b6 grpcproxy: configure --max-send-bytes and --max-recv-bytes for grpc proxy 2018-01-30 09:38:23 +01:00
Gyuho Lee 37546f74ab *: move "version" to "internal/version"
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-01-29 10:00:20 -08:00
Gyuho Lee 432581c7d0 *: move "discovery" to "internal/discovery"
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-01-25 15:41:17 -08:00
Tavish Armstrong b664b9176c pkg/srv, embed, etcdmain: Support multiple clusters in the same DNS discovery region. 2018-01-24 20:15:48 +00:00
Quentin MACHU a535c01050
embed: enable extensive metrics if specified 2018-01-06 20:40:30 +01:00
Gyuho Lee 936c4745c1 etcdmain: unset ETCD_UNSUPPORTED_ARCH after arch check
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-01-04 22:47:59 +00:00
Sahdev P. Zala 8a8aff198f Log: add information statement
Adding information that when config file is used other command line flags
and env variables will be ignored. This changes are a follow up of a
disucssion under PR,
https://github.com/coreos/etcd/pull/9066
2017-12-27 17:12:28 -05:00
Gyuho Lee 4f71941172 grpc-proxy: add "--debug" flag to "etcd grpc-proxy start" command
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2017-12-21 14:30:51 -08:00
Gyuho Lee 0133d77f0a etcdmain: display default --enable-v2, --strict-reconfig-check value ("true")
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2017-12-14 11:25:20 -08:00
Gyu-Ho Lee c8277e1b02 etcdmain: test wrong heartbeat-interval, election-timeout in TestConfigFileElectionTimeout
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-11-28 09:53:35 -08:00
Gyu-Ho Lee 3db5ad8d57 embed,etcdmain: add "--experimental-initial-corrupt-check"
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-11-22 15:27:14 -08:00
Gyu-Ho Lee 15bfc1b361
Merge pull request #8893 from dahefanteng/fix-typo
Documentation: change "key file" to "cert file"
2017-11-20 23:55:27 -08:00
Manjunath A Kumatagi e1b1ec8348 etcdmain: Fix go fmt for gotip 2017-11-21 11:37:09 +05:30
dahefanteng a4c407ece4 Documentation: change "key file" to "cert file"
when refered "--trusted-ca-file",what we need provide should be a CA cert file,not the CA private key file.
2017-11-20 00:44:32 -05:00
Frederic Branczyk 627cffd6f8
*: initialize gRPC server metrics with zero values 2017-11-15 11:21:29 +01:00
Gyu-Ho Lee 370ff6b670 etcdmain: do not embed structs (fix go vet warnings)
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-11-08 14:20:52 -08:00
Gyu-Ho Lee adc3cea8cf etcdmain: use embed.DefaultLogOutput for flags
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-11-02 14:43:06 -07:00
Gyu-Ho Lee 1fa295e3ba etcdmain: move SetupLogging to embed
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-11-02 13:19:49 -07:00
Gyu-Ho Lee 863dfd1f0e Merge pull request #8616 from mitake/peer-cn-auth
RFC: etcdmain, pkg: CN based auth for inter peer connection
2017-10-04 10:00:53 -07:00
Hitoshi Mitake 70018e9207 etcdmain, pkg: CN based auth for inter peer connection
This commit adds an authentication mechanism to inter peer connection
(rafthttp). If the cert based peer auth is enabled and a new option
`--peer-cert-allowed-cn` is passed, an etcd process denies a peer
connection whose CN doesn't match.
2017-10-02 15:59:17 +09:00
fanmin shi bcef78c665 Merge pull request #8563 from fanminshi/make_auto_compaction_granular
*: support auto-compaction with finer granularity
2017-09-29 11:18:51 -07:00
fanmin shi 733de98cfb *: modify etcd flags to support finner compaction retention 2017-09-28 17:22:44 -07:00
Gyu-Ho Lee 4b3d4000af etcdmain: add 'grpc-keepalive-*' flags
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-09-28 11:24:02 -07:00
Hitoshi Mitake 94b5071c30 etcdmain, proxy: handle authed watch in grpcproxy
This commit lets grpcproxy handle authed watch. The main changes are:
1. forwrading a token of a new broadcast client
2. checking permission of a new client that participates to an
   existing broadcast
2017-09-20 15:27:26 +09:00
Hitoshi Mitake e709f83253 etcdmain, proxy: support authed RPCs with grpcproxy
This commit lets grpcproxy support authed RPCs. Auth tokens supplied
by clients are now forwarded to etcdserver by grpcproxy.
2017-09-20 11:14:45 +09:00
Anthony Romano a0adee5209 etcdmain: add command line flag to etcdmain 2017-08-31 11:47:41 -07:00
blueblue 9b92e1b2d0 flag: improve StringFlags by support set default value when init (#8447)
* flag: improve StringFlags by support set default value when init

when init flagSet, set default value should be moved to StringFlags init
func, which is more friendly

personal proposal

* flag: code improved for StringFlags
2017-08-28 00:02:11 -07:00
Anthony Romano 35c5dcefc2 *: cockroachdb/cmux -> soheilhy/cmux
Has fixes not in fork. Includes SetReadTimeout.
2017-08-22 09:59:59 -07:00
Anthony Romano 86aeaad924 etcdmain: support experimental-corrupt-check-time flag 2017-08-22 09:59:59 -07:00
Anthony Romano 126e91c449 leasing, integration, etcdmain: closer function for leasing kv
Semaphore was seeing goroutine leaks
2017-08-18 14:05:57 -07:00
visheshnp 9be715bb66 etcdmain: support key leasing in grpcproxy 2017-08-04 11:35:36 -07:00
Gyu-Ho Lee b8fd5c3dba etcdmain: add '/health' endpoint to grpc-proxy
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-07-31 14:07:59 -07:00
Anthony Romano f6acd0316c etcdmain: add --experimental-serializable-ordering to grpc proxy
Connect to another endpoint on stale reads.
2017-07-27 12:39:30 -07:00
Gyu-Ho Lee 74c8050adc *: use etcdhttp.Handle* for health, prometheus handlers
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-07-26 06:23:55 -07:00
Anthony Romano 51d7786050 etcdmain: create self-signed certs when listening on https for httpproxy
Fixes failures from TestCtlV3PutClientAutoTLS in proxy coverage tests.
2017-07-24 15:37:05 -07:00
Anthony Romano c5447c2ec9 etcdmain: support crl in grpcproxy 2017-07-21 11:00:22 -07:00
Anthony Romano efbee9d8c7 etcdmain: support --auto-tls and --insecure-skip-verify in grpcproxy 2017-07-21 11:00:22 -07:00
Anthony Romano 1365f87d40 etcdmain: cleanup grpcproxy; support different certs for proxy/etcd
Enables TLS termination in grpcproxy.
2017-07-21 11:00:22 -07:00
Anthony Romano d5a0d4d696 etcdmain, embed: --auto-peer-tls and --auto-tls for v2 proxy
Fixes #7930
2017-07-21 11:00:22 -07:00
Anthony Romano 5d6c6ad20e etcdmain: use client tls info for v2 proxy client connections
Was defaulting to PeerTLSInfo for client connections to the etcd cluster.
Since proxy users may rely on this behavior, only use the client tls
info if given, and fall back to peer tls otherwise.
2017-07-21 11:00:22 -07:00
Anthony Romano 7c22d35dff etcdmain: support grpc-proxy/gateway compiled with -tags cov 2017-07-21 11:00:22 -07:00
Gyu-Ho Lee c47d4450c7 etcdmain/grpc-proxy: add 'metrics-addr' option
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-07-14 11:14:09 -07:00
Gyu-Ho Lee 8463b377d9 etcdmain: add 'listen-metrics-urls' option
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-07-14 11:14:09 -07:00
Xiang Li 0fe8fdcb29 Merge pull request #8123 from yudai/revision_compactor
Compactor: Add Revisional compactor
2017-06-22 16:34:28 -07:00
Iwasaki Yudai a3f8f47422 *: add Revision compactor 2017-06-21 15:41:07 -07:00
Anthony Romano 7d7d1ae6a0 etcdmain: configure CRL file through command line 2017-06-19 15:23:41 -07:00
fanmin shi 9e7740011b etcdserver: add --max-request-bytes flag 2017-05-25 11:01:38 -07:00
fanmin shi e9f464debc integration: creation of cluster now takes maxTxnOps 2017-05-24 14:48:44 -07:00
fanmin shi ae7ddfb483 etcdserver: add --max-txn-ops flag
--max-txn-ops allows users to define the maximum transaction operations
for each txn request. it defaults at 128.

Fixes #7826
2017-05-24 10:32:32 -07:00
Gyu-Ho Lee d690634bd6 *: remove unused, fix typos
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-05-18 12:11:18 -07:00
Anthony Romano fd01db9e60 grpcproxy, etcdmain: add lock and election services to proxy 2017-05-10 12:19:09 -07:00
Anthony Romano c232814003 etcdmain, tcpproxy: srv-priority policy
Adds DNS SRV weighting and priorities to gateway.

Partially addresses #4378
2017-05-08 11:35:18 -07:00
Anthony Romano 07ad18178d pkg/srv: package for SRV utilities
Trying to decouple the v2 client from SRV code. Can't move
into discovery/ since that creates a circular dependency. So,
give up and move all the SRV code into a new package.
2017-05-05 09:27:59 -07:00
Gyu-Ho Lee ea5f6dab6b etcdmain: trigger embed.Etcd.Close for OS interrupt
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-04-17 14:07:16 -07:00
Anthony Romano cad1215b18 *: deny incoming peer certs with wrong IP SAN 2017-04-12 13:41:33 -07:00
fanmin shi 2bc1dfd921 etcdmain: support SRV discovery for gRPC proxy
FIX #7562
2017-04-06 10:45:19 -07:00
Xiang fd5984af56 *: add pprof flag to grpc proxy 2017-04-03 22:07:17 -07:00
Jonathan Sokolowski 0472b2dc9f etcdmain: test config file clustering flags
A test to ensure that when clustering flags are correctly and
independently specified no errors are raised.
2017-03-31 10:01:46 +11:00
Anthony Romano 397a42efbe etcdmain: add prefixing support to grpc proxy
Fixes #6577
2017-03-22 16:45:38 -07:00
Gyu-Ho Lee 17e2e762b1 etcdmain: handle StopNotify when ErrStopped aborted publish
Fix https://github.com/coreos/etcd/issues/7512.

If a server starts and aborts due to config error,
it is possible to get stuck in ReadyNotify waits.
This adds select case to get notified on stop channel.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-03-21 10:22:39 -07:00
Anthony Romano 2f1542c06d *: use filepath.Join for files 2017-03-16 07:46:06 -07:00
Gyu-Ho Lee 01dd60c0f7 etcdmain: SdNotify when gateway, grpc-proxy are ready
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-03-09 10:10:21 -08:00
Gyu-Ho Lee 095407df58 etcdmain: add register,resolver flags
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-03-09 02:47:12 -08:00
Boris Dudelsack 0d48fc5511 gateway: fix the dns discovery method
strip the scheme from the endpoints to have a clean hostname for TCP proxy

Fixes #7452
2017-03-08 19:11:55 +01:00
Hitoshi Mitake f8a290e7ca *: support jwt token in v3 auth API
This commit adds jwt token support in v3 auth API.

Remaining major ToDos:
- Currently token type isn't hidden from etcdserver. In the near
  future the information should be completely invisible from
  etcdserver package.
- Configurable expiration of token. Currently tokens can be valid
  until keys are changed.

How to use:
1. generate keys for signing and verfying jwt tokens:
 $ openssl genrsa -out app.rsa 1024
 $ openssl rsa -in app.rsa -pubout > app.rsa.pub
2.  add command line options to etcd like below:
--auth-token-type jwt \
--auth-jwt-pub-key app.rsa.pub --auth-jwt-priv-key app.rsa \
--auth-jwt-sign-method RS512
3. launch etcd cluster

Below is a performance comparison of serializable read w/ and w/o jwt
token. Every (3) etcd node is executed on a single machine. Signing
method is RS512 and key length is 1024 bit. As the results show, jwt
based token introduces a performance overhead but it would be
acceptable for a case that requires authentication.

w/o jwt token auth (no auth):

Summary:
  Total:        1.6172 secs.
  Slowest:      0.0125 secs.
  Fastest:      0.0001 secs.
  Average:      0.0002 secs.
  Stddev:       0.0004 secs.
  Requests/sec: 6183.5877

Response time histogram:
  0.000 [1]     |
  0.001 [9982]  |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.003 [1]     |
  0.004 [1]     |
  0.005 [0]     |
  0.006 [0]     |
  0.008 [6]     |
  0.009 [0]     |
  0.010 [1]     |
  0.011 [5]     |
  0.013 [3]     |

Latency distribution:
  10% in 0.0001 secs.
  25% in 0.0001 secs.
  50% in 0.0001 secs.
  75% in 0.0001 secs.
  90% in 0.0002 secs.
  95% in 0.0002 secs.
  99% in 0.0003 secs.

w/ jwt token auth:

Summary:
  Total:        2.5364 secs.
  Slowest:      0.0182 secs.
  Fastest:      0.0002 secs.
  Average:      0.0003 secs.
  Stddev:       0.0005 secs.
  Requests/sec: 3942.5185

Response time histogram:
  0.000 [1]     |
  0.002 [9975]  |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.004 [0]     |
  0.006 [1]     |
  0.007 [11]    |
  0.009 [2]     |
  0.011 [4]     |
  0.013 [5]     |
  0.015 [0]     |
  0.016 [0]     |
  0.018 [1]     |

Latency distribution:
  10% in 0.0002 secs.
  25% in 0.0002 secs.
  50% in 0.0002 secs.
  75% in 0.0002 secs.
  90% in 0.0003 secs.
  95% in 0.0003 secs.
  99% in 0.0004 secs.
2017-03-06 19:46:03 -08:00
Gyu-Ho Lee 4aa68e0231 etcdmain: log machine default host after update check
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-03-03 14:25:39 -08:00
Anthony Romano 298d58841e etcdmain: enable grpc tracing with --debug 2017-02-27 11:18:13 -08:00
fanmin shi 2533c2a50c Merge pull request #7254 from fanminshi/rework_coverage_e2e
e2e: add code coverage to e2e
2017-02-17 15:51:47 -08:00
fanmin shi 65b59f4423 grpcproxy: incorporate lease proxy into existing proxy framework 2017-02-16 11:50:59 -08:00
fanmin shi 80ab321f9d etcdmain: whitelist etcd binary flags 2017-02-15 09:51:50 -08:00
Manjunath A Kumatagi 9767098331 etcdmain: ppc64le platform support 2017-01-26 21:08:07 -08:00
Anthony Romano dfb2ed07db etcdmain: add gateway and grpc-proxy commands to etcd help 2017-01-20 15:54:13 -08:00
Xiang Li a630735c29 Merge pull request #7170 from vimalk78/make-v2-endpoint-optional-#7100
embed/etcd.go: make v2 endpoint optional. fixes #7100
2017-01-20 11:14:20 -08:00
Vimal Kumar f80914fba2 embed/etcd.go: make v2 endpoint optional. fixes #7100 2017-01-20 11:49:52 +05:30