Commit Graph

1619 Commits (f5d02f02791716ed99c489ffa2441b8cc7925457)

Author SHA1 Message Date
Gyu-Ho Lee ce6bb4f1c9 etcdserver: replace /v3alpha with /v3beta in proto definitions
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-11-15 01:19:53 -08:00
Gyu-Ho Lee 75110dd839 *: fix naked returns
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-11-10 18:46:15 -08:00
tylerauerbeck 527d03e0d2 etcdserver: update metrics to use promhttp
Update api/etcdhttp/metrics.go to use promhttp.Handler() instead of prometheus.Handler()

fixes #8729
2017-11-10 09:47:49 -08:00
tamal c0c19465fc *: upgrade grpc-gateway to v1.3 2017-11-08 18:38:41 -08:00
Gyu-Ho Lee f48fe8ecda api/v3rpc: do not convert server context error to grpc/*status.statusError
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-11-06 17:32:23 -08:00
Gyu-Ho Lee ba233e2f4d etcdserver: preallocate slice in apply
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-11-06 09:13:12 -08:00
Gyu-Ho Lee 5d98710b2e api/v3rpc: deprecate grpc.Errorf
It's been deprecated as of grpc/grpc-go v1.6.x.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-11-04 22:08:17 -07:00
Gyu-Ho Lee 2feb8ba545 v3rpc/rpctypes: use codes.FailedPrecondition for ErrGRPCNotLeader
Changes ErrGRPCNotLeader error code to FailedPrecondition,
to disable retry with unavailable.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-10-20 14:28:17 -07:00
Gyu-Ho Lee 1109c6c321 etcdserver/api/etcdhttp: document package in doc.go
It was missing from godoc.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-10-06 10:47:47 -07:00
Gyu-Ho Lee 0199bdc266 *: fix 'ineffassign' issues
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-10-03 10:14:33 -07:00
fanmin shi bcef78c665 Merge pull request #8563 from fanminshi/make_auto_compaction_granular
*: support auto-compaction with finer granularity
2017-09-29 11:18:51 -07:00
fanmin shi 733de98cfb *: modify etcd flags to support finner compaction retention 2017-09-28 17:22:44 -07:00
Gyu-Ho Lee 32e15d790f api/rpc: accept grpc.ServerOption's for keepalive policy
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-09-28 10:55:00 -07:00
Xiang Li 554298d429 Merge pull request #8594 from mitake/auth-priority
RFC: etcdserver: swap priority of cert CN and username + password
2017-09-26 08:41:30 -07:00
Xiang 35e285674b etcdserver: make tick duration calculation clear 2017-09-23 15:43:12 -07:00
Xiang 230323255a etcdserver: make dial timeout longer 2017-09-22 14:56:41 -07:00
Hitoshi Mitake 1296281b27 etcdserver: swap priority of cert CN and username + password 2017-09-22 15:53:47 +09:00
Gyu-Ho Lee 9553afbb24 Merge pull request #8533 from gyuho/grpc
*: upgrade grpclog to LoggerV2
2017-09-12 03:53:04 -07:00
Gyu-Ho Lee 18ba4d60ec v3rpc/rpctypes: use grpc.status for errors
grpc.Code, grpc.ErrorDesc, grpc.Errorf have been deprecated.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-09-11 09:20:03 -07:00
Gyu-Ho Lee 0b2d8a6c96 *: fix minor typos
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-09-11 07:33:35 -07:00
Gyu-Ho Lee f37ff4a4e2 v3rpc: use grpclog.LoggerV2 for grpc logs
grpclog.Logger has been deprecated.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-09-08 15:25:36 -07:00
Gyu-Ho Lee 9a726b424d *: fix leaky context creation with cancel
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-09-07 13:39:42 -07:00
Gyu-Ho Lee f65aee0759 *: replace 'golang.org/x/net/context' with 'context'
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-09-07 13:39:42 -07:00
Anthony Romano 8091be6e97 v2v3: ServerV2 backed by clientv3 2017-08-31 11:47:40 -07:00
Anthony Romano 758c3c09fd etcdserver: refactor v2 request processing
Makes interfaces more reusable.
2017-08-31 11:47:40 -07:00
Anthony Romano 1d3afd4bb5 etcdhttp, v2http, etcdserver: use etcdserver.{Server,ServerV2} interfaces 2017-08-31 11:47:40 -07:00
fanmin shi d2ca782277 v3rpc: limit recv size using MaxRecvMsgSize and send using MaxSendMsgSize
grpc 1.3 uses MaxMsgSize() to limit received message size. However, grpc 1.4 introduces a 4mb default limit on send message size. In etcd, server shouldn't be limit size of message that it can be sent. Hence, set maximum size of send message using MaxSendMsgSize().
2017-08-22 14:31:01 -07:00
Anthony Romano 31381da53a etcdserver: raise alarm on cluster corruption
Fixes #7125
2017-08-22 09:59:59 -07:00
Anthony Romano 35dffc7bc1 rpctypes,v3rpc: add Corrupt error code 2017-08-22 09:59:59 -07:00
Anthony Romano b8bcc891a6 *: regenerate gRPC assets 2017-08-22 09:59:59 -07:00
Anthony Romano 6be5f9a841 etcdserverpb: add corrupt alarm 2017-08-22 09:59:59 -07:00
Gyu-Ho Lee d25ae50c02 etcdserver: implement LeaseLeases API
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-08-14 14:18:56 -07:00
Gyu-Ho Lee 8005f00bcf *: regenerate proto
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-08-14 14:18:56 -07:00
Gyu-Ho Lee a7413bbf28 etcdserverpb: define LeaseLeases API
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-08-14 14:18:56 -07:00
Gyu-Ho Lee e0843c691b Merge pull request #8322 from gyuho/health-grpc-proxy
*: add /health endpoint to grpc-proxy
2017-07-31 15:45:42 -07:00
Gyu-Ho Lee cd37ef2c1b *: expose etcdhttp.Health, define proxy health handler
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-07-31 14:07:54 -07:00
Anthony Romano ec4ca4408f etcdserver: support lease txn comparison 2017-07-31 13:00:04 -07:00
Anthony Romano 71e56a44b7 *: regenerate protobuf assets 2017-07-31 13:00:04 -07:00
Anthony Romano d8ca2bbffb etcdserverpb: add lease to txn comparison targets
Also shifts down fields following target_union in case there's any more
reason to expand. OK since range_end is still pre-release.
2017-07-31 13:00:04 -07:00
Gyu-Ho Lee 4267d368df api/etcdhttp: serve error information in '/health', marshal health in JSON
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-07-27 15:36:59 -07:00
Xiang Li 2a348fb8e9 Merge pull request #8263 from fanminshi/hash_by_rev
api: hash by rev
2017-07-26 11:22:33 -07:00
Gyu-Ho Lee 74c8050adc *: use etcdhttp.Handle* for health, prometheus handlers
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-07-26 06:23:55 -07:00
Gyu-Ho Lee 78432e3bd2 etcdhttp: add metrics.go for metrics, health handler
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-07-26 06:23:55 -07:00
Anthony Romano 16943f04e2 Merge pull request #8283 from heyitsanthony/cancel-compact-rpc
v3rpc: set Canceled=true on compacted watch
2017-07-25 19:15:18 -07:00
Anthony Romano 478ba2c4f2 etcdserver: consolidate error checking for v3_server functions
Duplicated error checking code moved into raftRequest/raftRequestOnce.
2017-07-25 14:28:39 -07:00
Anthony Romano 6fb08672d8 v3rpc: set canceled=true when stream is compacted
Fixes #8231
2017-07-25 12:36:01 -07:00
Gyu-Ho Lee 61a736a068 etcdserver: check alarms in health handler
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-07-18 15:51:28 -07:00
fanmin shi fbb75d24a4 v3rpc: add HashKV to server rpc 2017-07-14 16:44:00 -07:00
fanmin shi ed052ce9a3 proto: add HashKV grpc
HashKV rpc hash of all MVCC keys up to a given revision for a given node.
2017-07-14 16:41:23 -07:00
Xiang Li 0a2b580f73 Merge pull request #8238 from yudai/allow_more_streams
v3rpc: Let clients establish unlimited streams
2017-07-11 18:21:09 -07:00
Iwasaki Yudai 52101e6e93 v3rpc: Let clients establish unlimited streams
From go-grpc v1.2.0, the number of max streams per client is set to 100
by default by the server side. This change makes it impossible
for third party proxies and custom clients to establish many streams.
2017-07-11 13:02:01 -07:00
Anthony Romano e29db923bc *: move v2http handlers without /v2 prefix to etcdhttp
Lets --enable-v2=false configurations provide /metrics, /health, etc.

Fixes #8167
2017-07-07 18:35:57 -07:00
Gyu-Ho Lee a57405a958 Merge pull request #8153 from gyuho/leadership-transfer
*: expose Leadership Transfer API to clients
2017-07-06 13:00:08 -07:00
Hitoshi Mitake e0c33ef881 auth, etcdserver: allow users to know their roles and permissions
Current UserGet() and RoleGet() RPCs require admin permission. It
means that users cannot know which roles they belong to and what
permissions the roles have. This commit change the semantics and now
users can know their roles and permissions.
2017-06-26 22:20:41 -07:00
Gyu-Ho Lee 403ba1dfa7 etcdserver: expose 'transferLeadership' as 'MoveLeader'
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-06-23 12:51:28 -07:00
Gyu-Ho Lee b1a0ae3a3e etcdserver/api/v3rpc: add 'MoveLeader' to 'maintenanceServer'
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-06-23 12:51:24 -07:00
Gyu-Ho Lee 939bbd77c0 etcdserver/*: add 'ErrNotLeader'
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-06-23 12:46:07 -07:00
Gyu-Ho Lee 265303c19a *: regenerate proto with 'MoveLeader' RPC
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-06-23 12:45:53 -07:00
Gyu-Ho Lee d82f2572a4 etcdserver/etcdserverpb: define 'MoveLeader' RPC
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-06-23 12:43:29 -07:00
Xiang Li 0fe8fdcb29 Merge pull request #8123 from yudai/revision_compactor
Compactor: Add Revisional compactor
2017-06-22 16:34:28 -07:00
Iwasaki Yudai a3f8f47422 *: add Revision compactor 2017-06-21 15:41:07 -07:00
Anthony Romano 6ed51dc621 etcdserver, v3rpc: support nested txns 2017-06-21 14:33:15 -07:00
Anthony Romano 822473bc31 etcdserverpb: add txns to requestop/responseop 2017-06-21 14:33:15 -07:00
Gyu-Ho Lee c1e3172e3a etcdserver/api/v3rpc: add default grpc health service 2017-06-20 10:48:06 -07:00
Anthony Romano 8d7c29c732 etcdserver, etcdserverpb: Txn.Compare range_end support 2017-06-16 12:13:27 -07:00
Anthony Romano 1acc8090e3 Merge pull request #8110 from heyitsanthony/fix-test-sync-timeout
etcdserver: use RecorderStream for TestSyncTimeout to avoid missing action
2017-06-15 20:49:10 -07:00
Anthony Romano e962b0c849 Merge pull request #7909 from heyitsanthony/unptr-cfg
etcdserver, embed, integration: don't use pointer for ServerConfig
2017-06-15 20:47:30 -07:00
Gyu-Ho Lee 5e059fd8dc *: use metadata Incoming/OutgoingContext
Fix https://github.com/coreos/etcd/issues/7888.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-06-15 16:41:23 -07:00
Anthony Romano aa0e6b26c0 etcdserver: use RecorderStream for TestSyncTimeout to avoid missing action 2017-06-15 13:43:53 -07:00
Anthony Romano dcf52bbfac etcdserver, embed, integration: don't use pointer for ServerConfig
ServerConfig is owned by etdcserver and unshared, so don't pass or store by
pointer. Also removes duplicated field 'snapCount'.
2017-06-15 13:02:13 -07:00
Anthony Romano 4445996a38 Merge pull request #8084 from heyitsanthony/update-protobuf
vendor: update github.com/{gogo,golang}/protobuf
2017-06-12 19:09:49 -07:00
Anthony Romano 4ebeba0e18 *: regen protofiles with latest protobuf tools 2017-06-12 15:14:43 -07:00
Anthony Romano 7ff5b05004 etcdserver: better warning when initial-cluster doesn't match advertise urls
The old error was not clear about what URLs needed to be added, sometimes
truncating the list. To make it clearer, print out the missing entries
for --initial-cluster and print the full list of initial advertise peers.

Fixes #8079 and #7927
2017-06-12 14:14:16 -07:00
Anthony Romano d173b09a1b etcdserver: use same ReadView for read-only txns
A read-only txn isn't serialized by raft, but it uses a fresh
read txn for every mvcc access prior to executing its request ops.
If a write txn modifies the keys matching the read txn's comparisons,
the read txn may return inconsistent results.

To fix, use the same read-only mvcc txn for the duration of the etcd
txn. Probably gets a modest txn speedup as well since there are
fewer read txn allocations.
2017-06-09 09:20:38 -07:00
Anthony Romano 2caae60004 Merge pull request #8062 from heyitsanthony/revert-v2machines
v2http: put back /v2/machines and mark as non-deprecated
2017-06-08 12:01:58 -07:00
Gyu-Ho Lee 45fd8279f0 etcdserver: add leaseExpired debugging metrics
Fix https://github.com/coreos/etcd/issues/8050.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-06-08 10:36:25 -07:00
Anthony Romano c2dadbd9f8 v2http: put back /v2/machines and mark as non-deprecated
This reverts commit 2bb33181b6. python-etcd
seems to depend on /v2/machines and the maintainer vanished. Plus, it is
prefixed with /v2/ so it probably can't be deprecated anyway.
2017-06-08 09:39:11 -07:00
Hitoshi Mitake fa4903c83c Merge pull request #8031 from mitake/lease-revoke-auth
protecting lease revoking with auth
2017-06-08 13:34:14 +09:00
Hitoshi Mitake 0c655902f2 auth, etcdserver: protect revoking lease with auth
Currently clients can revoke any lease without permission. This commit
lets etcdserver protect revoking with write permission.

This commit adds a mechanism for generating internal token. It is used
for indicating that LeaseRevoke was issued internally so it should be
able to delete any attached keys.
2017-06-07 17:46:14 -07:00
Anthony Romano fb086ef13f v3rpc: dedup resp.Header == nil checks 2017-06-07 09:25:42 -07:00
Anthony Romano 8542f2e673 v3rpc: use map for translating errors to grpc errors
Switch statement had poor coverage, use a map instead
2017-06-06 16:55:44 -07:00
Anthony Romano 887db5a3db *: fix go tool vet -all -shadow errors 2017-06-03 21:32:36 -07:00
Anthony Romano 0c923bdf11 Merge pull request #8010 from heyitsanthony/json-txn
e2e: test txn over grpc json
2017-06-01 10:01:41 -07:00
Anthony Romano d8210da505 v3rpc: treat nil txn request op as error
Fixes #7889
2017-05-31 12:39:52 -07:00
Anthony Romano a20e667c5b Merge pull request #7967 from heyitsanthony/purge-snapdb
etcdserver: purge old snap.db files
2017-05-30 16:15:11 -07:00
fanmin shi 68a72c6b6e v3rpc: change grpc max recv size as needed. 2017-05-25 11:01:51 -07:00
fanmin shi 9e7740011b etcdserver: add --max-request-bytes flag 2017-05-25 11:01:38 -07:00
fanmin shi b003734be6 Merge pull request #7976 from fanminshi/make_maxOpsPerTxn_configurable
etcdserver: add --max-txn-ops flag
2017-05-25 10:34:17 -07:00
fanmin shi e9f464debc integration: creation of cluster now takes maxTxnOps 2017-05-24 14:48:44 -07:00
fanmin shi ae7ddfb483 etcdserver: add --max-txn-ops flag
--max-txn-ops allows users to define the maximum transaction operations
for each txn request. it defaults at 128.

Fixes #7826
2017-05-24 10:32:32 -07:00
Anthony Romano c1c9a2c96c etcdserver: close mvcc.KV on init error path
Scheduled compaction will panic if KV is not stopped before
closing the backend.
2017-05-23 10:41:37 -07:00
Anthony Romano ab16fa1f07 etcdserver: purge old snap.db files
Lots of garbage db files in #7957. Should purge.
2017-05-22 15:44:21 -07:00
Hitoshi Mitake 4cd5e7ebb2 Merge pull request #7809 from mitake/auth-watch
protect watch with auth
2017-05-20 13:23:30 +09:00
Hitoshi Mitake 939912c425 clientv3, etcdserver: support auth in Watch() 2017-05-20 11:34:45 +09:00
Anthony Romano 33c375dc44 *: fill out blank package godocs
Mostly one-liner short descriptions, but also includes some typo fixes
and some examples.
2017-05-18 09:41:13 -07:00
Xiang 32c252f003 etcdserver: more logging on snapshot close path 2017-05-17 14:48:52 -07:00
Anthony Romano f6cd4d4f5b snap, etcdserver: tighten up snapshot path handling
Computing the snapshot file path is error prone; snapshot recovery was
constructing file paths missing a path separator so the snapshot
would never be loaded. Instead, refactor the backend path handling
to use helper functions where possible.
2017-05-11 13:46:59 -07:00
fanmin shi 47f5b7c3ad Merge pull request #7876 from fanminshi/fix_7628
etcdserver: renaming db happens after snapshot persists to wal and snap files
2017-05-09 16:15:41 -07:00
fanmin shi dfdaf082c5 etcdserver: add a test to ensure renaming db happens before persisting wal and snap files 2017-05-09 14:00:22 -07:00
fanmin shi 8b7b7222dd etcdserver: renaming db happens after snapshot persists to wal and snap files
In the case that follower recieves a snapshot from leader
and crashes before renaming xxx.snap.db to db but after
snapshot has persisted to .wal and .snap, restarting
follower results loading old db, new .wal, and new .snap.
This will causes a index mismatch between snap metadata index
and consistent index from db.

This pr forces an ordering where saving/renaming db must
happen after snapshot is persisted to wal and snap file.
this guarantees wal and snap files are newer than db.
on server restart, etcd server checks if snap index > db consistent index.
if yes, etcd server attempts to load xxx.snap.db where xxx=snap index
if there is any and panic other wise.

FIXES #7628
2017-05-09 14:00:12 -07:00