Commit Graph

1295 Commits (06e2338108fdc694349aed923f4a7e45cf0cec1f)

Author SHA1 Message Date
Xiang Li 70bf768005 Merge pull request #5861 from xiang90/fix_watch
v3rpc: do not panic on user error for watch
2016-07-03 13:56:33 -07:00
Xiang Li 8a8a8253fa etcdserver: commit before sending snapshot 2016-07-03 13:54:05 -07:00
Xiang Li 13a4056327 v3rpc: do not panic on user error for watch 2016-07-03 08:57:48 -07:00
Gyu-Ho Lee ba023e539a etcdserver/api: print only major.minor version API
Before

2016-07-01 14:57:50.927170 I | api: enabled capabilities for version 3.0.0

After

2016-07-01 14:57:50.927170 I | api: enabled capabilities for version 3.0
2016-07-01 14:58:06 -07:00
Xiang Li c8c5f41a01 Merge pull request #5836 from xiang90/better_d_prev
*: support return prev deleted kv
2016-07-01 14:43:33 -07:00
Xiang Li 40c4a7894d *: support return prev deleted kv 2016-07-01 14:01:48 -07:00
Anthony Romano b7f5f8fc99 etcdserver: exit on missing backend only if semver is >= 3.0.0 2016-07-01 09:10:01 -07:00
Xiang Li bc6d7659af Merge pull request #5795 from xiang90/filter
*: support watch with filters
2016-06-28 14:07:12 -07:00
Xiang Li dced92f8bd *: support watch with filters
Now user can filter events with types. The API is also extensible.
It might make sense for the proxy to filter out events based on
more expensive/customized filter.
2016-06-28 13:46:57 -07:00
Xiang Li ddffdc3e37 Merge pull request #5725 from mitake/auth-not-enabled
auth, etcdserver: let Authenticate() fail if auth isn't enabled
2016-06-28 12:34:54 -07:00
Xiang Li 9614dc6e71 etcdserver: check index of the kv when restarting 2016-06-27 10:27:27 -07:00
Hitoshi Mitake 8df37d53d6 auth, etcdserver: let Authenticate() fail if auth isn't enabled
Successful Authenticate() would be confusing and make trouble shooting
harder if auth isn't enabled in a cluster.
2016-06-26 22:49:23 -07:00
Xiang Li 891ddcba6e etcdserver: refuse to restart if backend file is missing 2016-06-26 21:16:51 -07:00
Gyu-Ho Lee c363fd288b etcdserver: use CreateDirAll 2016-06-22 15:57:47 -07:00
Gyu-Ho Lee df9a52e53f Merge pull request #5702 from gyuho/vet
*: go vet, go lint fixes
2016-06-22 14:52:34 -07:00
Gyu-Ho Lee 725ded40f7 etcdserver: fix from go vet, go lint 2016-06-22 12:04:15 -07:00
Gyu-Ho Lee caffcb7fbb *: go vet fix in go tip 2016-06-22 11:10:59 -07:00
Anthony Romano 30cfa30490 etcdserver: make serialized txns auth-aware 2016-06-22 10:51:42 -07:00
Anthony Romano aafb2e9430 etcdserver: add lock to authApplier so serialized requests don't race 2016-06-22 10:51:42 -07:00
Xiang Li 0b5ea3ec94 Merge pull request #5742 from xiang90/count
*: support count in range query
2016-06-21 19:42:08 -07:00
Xiang Li def21f11a9 *: support count in range query 2016-06-21 16:20:55 -07:00
Anthony Romano de68818f03 etcdserver: add some failpoints 2016-06-21 14:43:20 -07:00
Gyu-Ho Lee a1c7a7df5e *: use capnslog for grpclog 2016-06-20 20:35:03 -07:00
Xiang Li e5583b26eb Merge pull request #5711 from xiang90/client_bytes
*: add client network metrics
2016-06-20 12:03:18 -07:00
Xiang Li 35fd81e465 *: add client network metrics 2016-06-20 11:18:06 -07:00
Xiang Li 729f5b45fd Merge pull request #5720 from xiang90/report_recv
*: fix pending events metrics
2016-06-20 06:44:16 -07:00
Xiang Li 6f28b43806 *: fix pending events metrics 2016-06-19 23:00:39 -07:00
Xiang Li 8111e0f7dc Merge pull request #5716 from ajityagaty/get_filtering
v3api: Add a flag to RangeRequest to return only the keys.
2016-06-19 14:50:15 -07:00
Ajit Yagaty ad5d55dd4c v3api: Add a flag to RangeRequest to return only the keys.
Currently the user can't list only the keys in a prefix search. In
order to support such operations the filtering will be done on the
server side to reduce the encoding and network transfer costs.
2016-06-19 14:18:39 -07:00
Gyu-Ho Lee d37e564eaa etcdserver: use TouchDirAll 2016-06-19 11:26:52 -07:00
Xiang Li 598fa7a10e *: add pending/failed proposal metrics 2016-06-17 13:09:38 -07:00
Xiang Li 57474697af etcdserver: add applied metrics 2016-06-17 11:52:50 -07:00
Gyu-Ho Lee 63901be674 *: regenerate proto 2016-06-17 10:22:28 -07:00
Hitoshi Mitake 18253e2723 *: support getting all users and roles in auth v3
This commit expands RPCs for getting user and role and support list up
all users and roles. etcdctl v3 is now support getting all users and
roles with the newly added option --all e.g. etcdctl user get --all
2016-06-17 16:22:41 +09:00
Xiang Li 2d08e093c1 Merge pull request #5696 from xiang90/fix_panic
etcdserver: fix panic when getting header of raft request
2016-06-16 13:58:50 -07:00
Xiang Li adff458895 etcdserver: fix panic when getting header of raft request 2016-06-16 13:42:10 -07:00
Gyu-Ho Lee b3558894f2 Merge pull request #5695 from gyuho/proto
*: use latest protodoc, regenerate
2016-06-16 12:35:44 -07:00
Xiang Li 5f5c3c8f82 Merge pull request #5694 from xiang90/comp
etcdserver: only pause compaction when sending snapshot
2016-06-16 12:26:55 -07:00
Gyu-Ho Lee 0bae7b635c *: regenerate proto, doc 2016-06-16 11:57:46 -07:00
Xiang Li 699e76b631 etcdserver: only pause compaction when sending snapshot 2016-06-16 08:57:02 -07:00
Xiang Li 49266dca2d Merge pull request #5690 from xiang90/fix_s
etcdserver: save state before save snapshot
2016-06-15 22:36:30 -07:00
Xiang Li 9c78cda088 etcdserver: save state before save snapshot 2016-06-15 22:00:33 -07:00
Hitoshi Mitake cdf1a2ee2c etcdserver: permission checking of Txn() in authApplierV3 2016-06-15 20:10:16 -07:00
Gyu-Ho Lee 32d766d749 etcdserver: preallocate slice 2016-06-15 13:03:10 -07:00
Anthony Romano 16db9e68a2 auth, etcdserver: separate auth checking apply from core apply 2016-06-15 09:03:27 -07:00
Xiang Li 5a7b7f7595 main: add grpc-gateway support
Now etcd can serve HTTP json request at /v3alpha/
2016-06-14 17:09:06 -07:00
Xiang Li c75fa6fdc9 *: support deleteRange perm checking 2016-06-13 17:49:13 -07:00
Gyu-Ho Lee 5d6af0b51f etcdserver: key, rangeEnd in []byte for auth 2016-06-13 14:21:25 -07:00
Hongchao Deng c11418b56c docs: v3 api, ErrCompaction -> ErrCompacted 2016-06-10 21:53:06 -07:00
Xiang Li 65ff76882b Merge pull request #5624 from xiang90/warn_apply
etcdserver: warn heavy apply
2016-06-10 15:28:27 -07:00
Xiang Li 64eccd519d etcdserver: warn heavy apply 2016-06-10 14:43:34 -07:00
Hitoshi Mitake ead5096fa9 auth, etcdserver: make auth tokens consistent for all nodes
Currently auth tokens are generated in the replicated state machine
layer randomly. It means one auth token generated in node A cannot be
used for node B. It is problematic for load balancing and fail
over. This commit moves the token generation logic from the state
machine to API layer (before raft) and let all nodes share a single
token.

Log index of Raft is also added to a token for ensuring uniqueness of
the token and detecting activation of the token in the cluster (some
nodes can receive the token before generating and installing the token
in its state machine).

This commit also lets authStore have simple token related things. It
is required because of unit test. The test requires cleaning of the
state of the simple token things after one test (succeeding test can
create duplicated token and it causes panic).
2016-06-10 13:55:37 -07:00
Anthony Romano b3a0b0502c etcdserver: respect auth on serialized Range 2016-06-10 11:43:05 -07:00
Xiang Li f1c6fa48f5 *: add admin permission checking 2016-06-09 15:25:09 -07:00
Xiang Li fb0df211f0 Merge pull request #5586 from xiang90/root
auth: add root user and root role
2016-06-09 00:23:45 -07:00
Xiang Li da2f2a5189 auth: add root user and root role 2016-06-08 19:55:08 -07:00
Hitoshi Mitake 253e313c09 *: support granting and revoking range
This commit adds a feature for granting and revoking range of keys,
not a single key.

Example:
$ ETCDCTL_API=3 bin/etcdctl role grant r1 readwrite k1 k3
Role r1 updated
$ ETCDCTL_API=3 bin/etcdctl role get r1
Role r1
KV Read:
        [a, b)
        [k1, k3)
        [k2, k4)
KV Write:
        [a, b)
        [k1, k3)
        [k2, k4)
$ ETCDCTL_API=3 bin/etcdctl --user u1:p get k1 k4
k1
v1
$ ETCDCTL_API=3 bin/etcdctl --user u1:p get k1 k5
Error:  etcdserver: permission denied
2016-06-08 14:58:25 -07:00
Hitoshi Mitake 6bb96074da auth, etcdserver: permission of range requests
Currently the auth mechanism doesn't support permissions of range
request. It just checks exact matching of key names even for range
queries. This commit adds a mechanism for setting permission to range
queries. Range queries are allowed if a range of the query is [begin1,
end1) and the user has a permission of reading [begin2, range2) and
[begin1, end2) is a subset of [begin2, range2). Range delete requests
will follow the same rule.
2016-06-08 11:57:32 -07:00
Anthony Romano ff2b24a8ac Merge pull request #5583 from heyitsanthony/grpc-nuke-waitstate
clientv3: use grpc balancer
2016-06-08 09:45:44 -07:00
Anthony Romano 62f8ec25c0 clientv3: use grpc reconnection logic 2016-06-08 01:04:59 -07:00
Xiang Li cc1155c93b etcdserver: make usernameFromCtx more go style 2016-06-07 21:17:32 -07:00
Gyu-Ho Lee 6e149e3485 etcdserver: following updates for proto change 2016-06-07 13:32:07 -07:00
Gyu-Ho Lee ca630a0803 etcdserverpb: RequestOp, ResponseOp
Fix https://github.com/coreos/etcd/issues/5504.
2016-06-07 13:31:10 -07:00
Xiang Li 83ce1051ff auth: make naming consistent 2016-06-07 10:54:50 -07:00
Hitoshi Mitake 94f22e8a07 *: rename RPCs and structs related to revoking
This commit renames RPCs and structs related to revoking.
1. UserRevoke -> UserRevokeRole
2. RoleRevoke -> RoleRevokePermission
2016-06-05 16:57:23 +09:00
Hitoshi Mitake 60fc1e4d4e auth, etcdserver: error codes for revoking non existing role and permission
This commit adds error codes for representing revoking non existing
role (from user) and permission (from role).
2016-06-05 16:41:10 +09:00
Hitoshi Mitake c7a1423d45 *: support deleting a role in auth v3
This commit implements RoleDelete() RPC for supporting deleting a role
in auth v3. It also adds a new subcommand "role delete" to etcdctl.
2016-06-04 13:42:45 +09:00
Hitoshi Mitake 0cb1343109 *: support revoking a key from a role in auth v3
This commit implements RoleRevoke() RPC for supporting revoking a key
from a role in auth v3. It also adds a new subcommand "role revoke" to
etcdctl.
2016-06-04 13:42:45 +09:00
Hitoshi Mitake 957b07c408 *: support revoking a role from a user in auth v3
This commit implements UserRevoke() RPC for supporting revoking a role
from a user in auth v3. It also adds a new subcommand "user revoke" to
etcdctl.
2016-06-04 13:39:26 +09:00
Hitoshi Mitake 10ee69b44c *: support getting role in auth v3
This commit implements RoleGet() RPC of etcdserver and adds a new
subcommand "role get" to etcdctl v3. It will list up permissions that
are granted to a given role.

$ ETCDCTL_API=3 bin/etcdctl role get r1
Role r1
KV Read:
        b
        d
KV Write:
        a
        c
        d
2016-06-03 13:03:54 +09:00
Anthony Romano 84a487f723 Revert "etcdserverpb: make RangeResponse.More an int64"
This reverts commit 84e1ab8765.
2016-06-02 13:43:40 -07:00
Hitoshi Mitake 5609fdb9a8 *: support getting user in etcdctl v3
This commit adds a new subcommand "user get" to etcdctl v3. It will
list up roles that are granted to a given user.

Example:
$ ETCDCTL_API=3 bin/etcdctl user get u1
User: u1
Roles: r1 r2 r3

This commit also modifies the layout of InternalRaftRequest for
frequent update of auth related members.
2016-06-02 12:10:19 +09:00
Anthony Romano 84e1ab8765 etcdserverpb: make RangeResponse.More an int64 2016-06-01 17:10:23 -07:00
Xiang Li 8528c8c599 *: more logging on critical state change
Add more logging for better debugging purpose.
2016-05-31 23:31:03 -07:00
Anthony Romano 7709cd84bb Merge pull request #5505 from heyitsanthony/v3rpc-watcher-close
v3rpc: fix race on ctrl channel when watcher stream closes
2016-05-31 14:24:10 -07:00
Gyu-Ho Lee 283318d547 v3rpc: add ErrConnClosed for closed client
For https://github.com/coreos/etcd/issues/5495.
2016-05-31 11:15:01 -07:00
Anthony Romano 09e8f5782e v3rpc: fix race on closing watcher stream ctrl channel
Sometimes close would race with the recvLoop, leading the
recvLoop to write to a close channel.
2016-05-31 11:07:31 -07:00
Xiang Li 6f8cc58214 Merge pull request #5490 from mitake/errcode
etcdserver, auth: not return grpc error code directly in the apply phase
2016-05-30 22:00:54 -07:00
Anthony Romano 9c767cbf98 Merge pull request #5464 from heyitsanthony/fix-victim-watchers
mvcc: tighten up watcher cancelation and revision handling
2016-05-30 20:09:39 -06:00
Hitoshi Mitake 5144318af0 etcdserver, auth: not return grpc error code directly in the apply phase
Current permission checking mechanism doesn't return its error code
well. The internal error (code = 13) is returned to client and the
retry mechanism doesn't work well. This commit fixes the problem.
2016-05-31 11:04:34 +09:00
Xiang Li 815bc5307f Merge pull request #5489 from linuxcer/master
etcdserver: fix typo in server.go
2016-05-30 15:20:02 -07:00
Chengfei Zhang 29cc568659 etcdserver: fix typo in server.go 2016-05-31 05:54:30 +08:00
Hitoshi Mitake 8e821cdc70 *: do permission check in raft log apply phase
This commit lets etcdserver check permission during its log applying
phase. With this change, permission checking of operations is
supported.

Currently, put and range are supported. In addition, multi key
permission check of range isn't supported yet.
2016-05-29 00:05:48 +09:00
Hitoshi Mitake 90e9652f70 etcdserver: return error of apply result without touching response
Current etcdserver tries to return result.resp even if result.err is
not nil. A situation of result.resp == nil and result.err != nil can
happen and it results an error like below:

18:49:57 etcd1 | interface conversion: proto.Message is nil, not *etcdserverpb.PutResponse

This commit lets the functions return result.err if it is not nil.
2016-05-29 00:05:48 +09:00
Anthony Romano c438310634 v3rpc: make watcher wait for its send goroutine to finish 2016-05-27 16:54:26 -07:00
Anthony Romano fc7da09d67 *: add missing godoc package descriptions
Fixes #4074
2016-05-27 15:15:26 -07:00
Gyu-Ho Lee 8a718f3e56 etcdserverpb: document prefix, range query
Fix https://github.com/coreos/etcd/issues/5401.
2016-05-25 16:53:36 -07:00
Xiang Li 1c544c3ba5 api: add v3rpc capability 2016-05-23 14:45:08 -07:00
Xiang Li 6f2e7875aa etcdctl: add migrate command
Migrate command accepts a datadir and an optional user-provided
transformer function that transform v2 keys to v2 keys.

Migrate command then builds a v3 backend state based on the existing
v2 keys and the output of the transformer function.
2016-05-19 12:17:15 -07:00
Anthony Romano fb8d12a9cd Merge pull request #5379 from heyitsanthony/fix-snapshot-close-wal
etcdserver: wait for snapshots before closing raft
2016-05-17 15:19:41 -07:00
Anthony Romano 73204e9637 etcdserver: wait for snapshots before closing raft
Fixes #5374
2016-05-17 15:04:25 -07:00
Anthony Romano ac2e3e43bf v3rpc: add sha trailer to snapshot 2016-05-16 11:15:03 -07:00
Gyu-Ho Lee abb4cd5646 etcdserver: update LICENSE header 2016-05-12 20:49:40 -07:00
Xiang Li a880e9c7cb Merge pull request #5332 from xiang90/sl
*: cancel required leader streams when memeber lost its leader
2016-05-12 20:24:34 -07:00
Xiang Li 9c103dd0de *: cancel required leader streams when memeber lost its leader 2016-05-12 19:42:21 -07:00
Gyu-Ho Lee b1f95c314b *: fix protodoc, consistent casing in api doc
There was a bug in protodoc.
This changes git SHA to use the latest protodoc.
And make the letter casing consistent with original
Protocol Buffer. Go capitalizes the member variables,
but the protocol buffer documentation should be same as
original proto files.
2016-05-12 16:23:29 -07:00
Xiang Li 6fc3106e68 Merge pull request #5324 from xiang90/partitioned
*: etcd member rejects unary call with leader requirement when it does not have leader
2016-05-11 17:48:06 -07:00
Xiang Li 19221b33cc *: etcd member rejects unary call with leader requirement when it does not have leader 2016-05-11 16:34:34 -07:00
Anthony Romano dcb3b7aecf *: scrub legacy ports from code and scripts 2016-05-11 13:46:30 -07:00
Xiang Li ab11415d25 *: add proposalsCommitted metrics 2016-05-10 10:56:25 -07:00