Commit Graph

531 Commits (1f206c027a37d2ed80b59465d37e8c22477209c1)

Author SHA1 Message Date
Anthony Romano a1ce07a321 etcdserver: reject member removal that breaks the current active quorum 2016-08-10 17:00:39 -07:00
Anthony Romano 9063ce5e3f etcdserver, embed: stricter reconfig checking
Make --strict-reconfig-check a default and check if cluster is healthy when
adding a member.
2016-08-05 16:59:25 -07:00
Xiang Li d69d438289 *: minor cleanup for lease 2016-08-04 20:39:32 -07:00
Xiang Li 29a077bdbe etcdserver: always recover lessor first 2016-08-04 08:06:19 -07:00
Anthony Romano bf71497537 etcdserver, lease: tie lease min ttl to election timeout 2016-08-02 13:06:57 -07:00
Anthony Romano 06da46c4ee etcdserver: apply serialized requests outside auth apply lock
Fixes #6010
2016-07-30 22:00:49 -07:00
Anthony Romano 13c2d32061 Merge pull request #6045 from heyitsanthony/fix-version-race
etcdserver, api, membership: don't race on setting version
2016-07-27 08:56:39 -07:00
Hitoshi Mitake 0090573749 etcdserver: skip range requests in txn if the result is needless
If a server isn't serving txn requests from a client, the server
doesn't need the result of range requests in the txn.

This is a succeeding commit of
https://github.com/coreos/etcd/pull/5689
2016-07-26 19:49:07 -07:00
Anthony Romano de2c3ec3db etcdserver, api, membership: don't race on setting version
Fixes #6029
2016-07-26 18:21:40 -07:00
Xiang Li 2d761d64a4 etcdserver: set applied index correctly 2016-07-16 11:44:18 -07:00
Xiang Li 27b03f0ed5 *: deny proposals when there is a huge gap between apply/commit 2016-07-14 10:02:55 -07:00
Xiang Li f65e75e4b3 *: remove unnecessary data upgrade code 2016-07-11 15:11:56 -07:00
Hitoshi Mitake c47689d98f Merge pull request #5689 from mitake/skip-apply
RFC: etcdserver, pkg: skip needless log entry applying
2016-07-10 01:23:35 +09:00
Jared Hulbert f78d4713ea etcdserver: atomic access alignment
Most fields accessed with sync/atomic functions are 64bit aligned, but a couple
are not.  This makes comments out of date and therefore misleading.

Affected fields reordered, comments scrubbed and updated.
2016-07-08 11:20:47 -07:00
Hitoshi Mitake abb20ec51f etcdserver, pkg: skip needless log entry applying
This commit lets etcdserver skip needless log entry applying. If the
result of log applying isn't required by the node (client that issued
the request isn't talking with the node) and the operation has no side
effects, applying can be skipped.

It would contribute to reduce disk I/O on followers and be useful for
a cluster that processes much serializable get.
2016-07-08 15:16:45 +09:00
Xiang Li 8a8a8253fa etcdserver: commit before sending snapshot 2016-07-03 13:54:05 -07:00
Anthony Romano b7f5f8fc99 etcdserver: exit on missing backend only if semver is >= 3.0.0 2016-07-01 09:10:01 -07:00
Xiang Li 9614dc6e71 etcdserver: check index of the kv when restarting 2016-06-27 10:27:27 -07:00
Xiang Li 891ddcba6e etcdserver: refuse to restart if backend file is missing 2016-06-26 21:16:51 -07:00
Gyu-Ho Lee d37e564eaa etcdserver: use TouchDirAll 2016-06-19 11:26:52 -07:00
Xiang Li 57474697af etcdserver: add applied metrics 2016-06-17 11:52:50 -07:00
Xiang Li 699e76b631 etcdserver: only pause compaction when sending snapshot 2016-06-16 08:57:02 -07:00
Anthony Romano 16db9e68a2 auth, etcdserver: separate auth checking apply from core apply 2016-06-15 09:03:27 -07:00
Xiang Li 65ff76882b Merge pull request #5624 from xiang90/warn_apply
etcdserver: warn heavy apply
2016-06-10 15:28:27 -07:00
Xiang Li 64eccd519d etcdserver: warn heavy apply 2016-06-10 14:43:34 -07:00
Hitoshi Mitake ead5096fa9 auth, etcdserver: make auth tokens consistent for all nodes
Currently auth tokens are generated in the replicated state machine
layer randomly. It means one auth token generated in node A cannot be
used for node B. It is problematic for load balancing and fail
over. This commit moves the token generation logic from the state
machine to API layer (before raft) and let all nodes share a single
token.

Log index of Raft is also added to a token for ensuring uniqueness of
the token and detecting activation of the token in the cluster (some
nodes can receive the token before generating and installing the token
in its state machine).

This commit also lets authStore have simple token related things. It
is required because of unit test. The test requires cleaning of the
state of the simple token things after one test (succeeding test can
create duplicated token and it causes panic).
2016-06-10 13:55:37 -07:00
Xiang Li 8528c8c599 *: more logging on critical state change
Add more logging for better debugging purpose.
2016-05-31 23:31:03 -07:00
Xiang Li 815bc5307f Merge pull request #5489 from linuxcer/master
etcdserver: fix typo in server.go
2016-05-30 15:20:02 -07:00
Chengfei Zhang 29cc568659 etcdserver: fix typo in server.go 2016-05-31 05:54:30 +08:00
Hitoshi Mitake 8e821cdc70 *: do permission check in raft log apply phase
This commit lets etcdserver check permission during its log applying
phase. With this change, permission checking of operations is
supported.

Currently, put and range are supported. In addition, multi key
permission check of range isn't supported yet.
2016-05-29 00:05:48 +09:00
Xiang Li 6f2e7875aa etcdctl: add migrate command
Migrate command accepts a datadir and an optional user-provided
transformer function that transform v2 keys to v2 keys.

Migrate command then builds a v3 backend state based on the existing
v2 keys and the output of the transformer function.
2016-05-19 12:17:15 -07:00
Anthony Romano 73204e9637 etcdserver: wait for snapshots before closing raft
Fixes #5374
2016-05-17 15:04:25 -07:00
Gyu-Ho Lee abb4cd5646 etcdserver: update LICENSE header 2016-05-12 20:49:40 -07:00
Xiang Li 9c103dd0de *: cancel required leader streams when memeber lost its leader 2016-05-12 19:42:21 -07:00
Gyu-Ho Lee 9bf601a921 etcdserver: log timeout 2016-05-03 13:39:31 -07:00
Xiang Li 434f2c356d etcdserver: do not serve requests before finish the first internal proposal 2016-04-27 15:46:31 -07:00
Anthony Romano b7ac758969 *: rename storage package to mvcc 2016-04-25 15:25:51 -07:00
Anthony Romano 08d879341d etcdserver: stop raft after stopping apply scheduler
Was causing a pipeline leak.
2016-04-22 17:15:13 -07:00
Xiang Li 4991cda202 etcdsever: fix the leaky snashot routine issue 2016-04-21 08:48:11 -07:00
Anthony Romano ebace2eb1b etcdserver: split out v2 Do() API from core server code 2016-04-20 10:29:22 -07:00
Anthony Romano 41382bc3f0 etcdserver: split out v2 raft apply interface 2016-04-20 10:29:22 -07:00
Gyu-Ho Lee 641a1a66e1 *: fix govet -shadow in go tip 2016-04-15 07:39:52 -07:00
Xiang Li e9735b7bd0 etcdserver: save cluster version into backend 2016-04-12 11:37:22 -07:00
Xiang Li 0a684c10ad Merge pull request #5025 from xiang90/no_dup_resp
etcdserver: do not send out out of date appResp
2016-04-11 14:41:52 -07:00
Xiang Li 3bad47d691 Merge pull request #5018 from xiang90/b
etcdserver: set backend to cluster
2016-04-11 13:02:57 -07:00
Anthony Romano 130567832f Merge pull request #4734 from luxas/32bit_alignments
etcdserver: align 64-bit atomics on 8-byte boundary
2016-04-10 19:18:15 -07:00
Xiang Li de7692b2b2 etcdserver: do not send out out of date appResp 2016-04-09 23:30:00 -07:00
Xiang Li 3c0ac9d600 etcdserver: set backend to cluster 2016-04-08 21:46:45 -07:00
Xiang Li b13b77f362 membership: update attr in membership pkg 2016-04-07 21:25:32 -07:00
Xiang Li bf2289ae00 etcdserver: move membership related code to membership pkg 2016-04-07 14:21:37 -07:00
Xiang Li 030865abe3 *: move Cluster interface to api 2016-04-07 08:05:47 -07:00
Xiang Li 2c50eb240e *: mv etcdhttp into api pkg 2016-04-04 16:31:35 -07:00
Anthony Romano 7ce5c2b9ff Merge pull request #4902 from heyitsanthony/alarm-ctl
etcdctl: alarm command
2016-03-30 13:55:29 -07:00
Xiang Li eddc741b5e *: simplify consistent index handling 2016-03-30 13:38:28 -07:00
Anthony Romano 96ee00a322 etcdserverpb: make alarm memberId uint64
To be consistent with Cluster API
2016-03-29 20:15:39 -07:00
Anthony Romano 3fbacf4be2 v3rpc: move Hash RPC to Maintenance service 2016-03-28 17:15:58 -07:00
Anthony Romano a403a94d7b etcdserver: cap new keys on space alarm 2016-03-28 14:56:26 -07:00
Anthony Romano 9e7f47c490 etcdserver: Alarm RPC
Alarms are events that nodes can use to relay health information to
the rest of the cluster. A node may Activate an alarm and that alarm
will stay set until Deactivated.
2016-03-28 14:56:26 -07:00
Anthony Romano 9c8253c543 etcdserver, v3rpc: space quotas 2016-03-28 14:56:26 -07:00
Anthony Romano 94e77cfa5d etcdserver: move v3 raft apply functions to interface 2016-03-28 13:16:21 -07:00
Xiang Li 70a9391378 *: enable v3 by default 2016-03-23 17:01:36 -07:00
Anthony Romano bd832e5b0a *: migrate Godeps to vendor/ 2016-03-22 17:10:28 -07:00
Xiang Li afb1bc242b Merge pull request #4822 from mitake/auth-backend
auth, etcdserver: add a method for updating backend during apply snap…
2016-03-21 23:34:46 -07:00
Hitoshi Mitake 4e39f690f2 auth, etcdserver: add a method for recoverying from backend during apply snapshot
This commit adds a new method Recovery() to auth.AuthStore for
recoverying auth state from backend during apply snapshot. It follows
a manner of the lessor.
2016-03-22 15:17:40 +09:00
Gyu-Ho Lee 67c2384bdf *: replace '-' with '--' in doc
Fix https://github.com/coreos/etcd/issues/4595.
2016-03-21 11:12:43 -07:00
Hitoshi Mitake 4eb1cfd658 etcdserver, auth: new package auth for the auth feature
This commit adds a new package auth. Its role is persisting auth
related metadata. This commit also connects its main interface
AuthStore and v3 server.
2016-03-14 13:57:41 +09:00
Xiang Li c6192d1d7a etcdserver: remove todo 2016-03-10 19:10:20 -08:00
Xiang Li adcba975cb *: recover lessor when applying snapshot 2016-03-10 17:06:58 -08:00
Xiang Li 2f12ea893b etcdserver: add maintain service to support defrag 2016-03-09 22:29:21 -08:00
Lucas Käldström a44645e13d etcdserver: align 64-bit atomics on 8-byte boundary 2016-03-10 07:24:33 +02:00
Xiang Li f0dbd0b856 Merge pull request #4646 from xiang90/starvation
etcdserver: detect raft stravation
2016-02-29 19:58:17 -08:00
Anthony Romano 1d6ebdd35c pkg/timeutil: removal
Overkill of a package for three lines of code.
2016-02-29 17:07:24 -08:00
Xiang Li d6520303c6 etcdserver: detect raft starvation caused by contention 2016-02-29 17:06:57 -08:00
Xiang Li d265fe000c *: support time based auto compaction.
Fix https://github.com/coreos/etcd/issues/3906.

We will have extensive doc to talk about what is compaction
and what is auto compaction soon.
2016-02-25 16:02:03 -08:00
Anthony Romano c5b51946eb *: exported godoc fixups 2016-02-21 20:36:44 -08:00
Anthony Romano 616f395920 etcdserver: use fifo scheduler for applier 2016-02-11 19:21:30 -08:00
Caleb Champlin 82778ed478 Add refresh parameter to allow TTL refreshes without firing watch/wait responses 2016-02-08 10:37:37 -07:00
Anthony Romano fe7cfe4d3d rafthttp: plumb local peer URLs through transport 2016-02-05 16:44:41 -08:00
Xiang Li e44e753e66 pkg/idutil: reduce conflict rate from 1% to 0.005%
Perviously, we only use 8bits from member identification
in id generation. The conflict rate is A(256,3)/256^3, which
is around 1%. Now we use 16bites to reduce the rate to 0.005%.

We can attach the full member id into id generation if needed...
2016-02-04 13:25:18 -08:00
Anthony Romano 9ae8d85049 integration: decrease timeout for isMemberBootstrapped
Spending seconds(!) when it would fail anyway.

integration/TestV3 (before): 100.670
integration/TestV3 (after): 29.571
2016-02-02 14:34:58 -08:00
Anthony Romano 20461ab11a *: fix many typos 2016-01-31 21:42:39 -08:00
Heungsub Lee c4a0159601 documentation: fix typo "a etcd" -> "an etcd"
"a" is not a correct article for "etcd".
2016-01-28 19:19:32 +09:00
Anthony Romano bd02d668c8 etcdserver: don't try to apply empty message list
If all messages have been applied, don't apply an empty messages list;
otherwise appliedi will update to 0 and etcd will panic.

Fixes #4278
2016-01-26 11:56:37 -08:00
Anthony Romano 12f6b8e72d etcdserver: complete stopWithDelay on server shutdown
Was causing goroutine leaks on my machine.
2016-01-25 19:45:29 -08:00
Xiang Li ef6320e638 etcdserver: make cluster checking interval shorter 2016-01-25 08:16:05 +08:00
Xiang Li 1aa312fcce *: lease forwarding should resue transport 2016-01-25 06:56:07 +08:00
Anthony Romano 2e157530a0 etcdhttp, lease, v3api: forward keepalives to leader
keepalives don't go through raft so let follower peers announce
keepalives to the leader through the peer http handler
2016-01-22 12:40:40 -08:00
Anthony Romano 9113a27bde lease: grant consistent lease IDs
When raft broadcasts a Grant to all nodes, all nodes must
agree on the same lease ID. Otherwise, attaching a key to
a lease will fail since the lease ID is node-dependent.
2016-01-22 09:43:39 -08:00
Hitoshi Mitake 588f655b4e etcdmain: add an option for pprof
This commit adds a new option for activating profiling based on pprof
in etcd process.
 - -enable-pprof: boolean type option which activates profiling

For example, if a client URL is http://localhost:12379, users and
developers access heap profiler with this URL:
http://localhost:12379/debug/pprof/heap
2016-01-13 16:12:26 +09:00
Xiang Li 6b1d9fb7ce *: stop lessor when etcdserver is stopped 2016-01-12 10:06:11 -08:00
Xiang Li f5753f2f51 *: support lease Attach
Now we can attach keys to leases. And revoking the lease removes all
the attached keys of that lease.
2016-01-09 11:01:58 -08:00
Xiang Li 2566699a48 *: revoke expired leases 2016-01-08 13:37:58 -08:00
Gyu-Ho Lee f76166a041 *: fix minor typos 2016-01-08 00:21:19 -08:00
Xiang Li d9ca929a33 *: add support for lease create and revoke
Basic support for lease operations like create and revoke.
We still need to:
1. attach keys to leases in KV implmentation if lease field is set
2. leader periodically removes expired leases
3. leader serves keepAlive requests and follower forwards keepAlive
requests to leader.
2016-01-07 16:39:39 -08:00
Xiang Li 1714290f4e storage: support recovering from backend
We want the KV to support recovering from backend to avoid
additional pointer swap. Or we have to do coordination between
etcdserver and API layer, since API layer might have access to
kv pointer and use a closed kv.
2016-01-06 21:16:55 -08:00
Xiang Li 5dd3f91903 *: make backend outside kv
KV and lease will share the same backend. Thus we need to make
backend outside KV.
2016-01-05 19:55:29 -08:00
Anthony Romano 8346a7c052 Merge pull request #4094 from heyitsanthony/send-merged-done-nowait
etcdserver: respect done channel when sleeping for snapshot backoff
2015-12-29 20:18:08 -08:00
Anthony Romano 942b5570bd Merge pull request #4096 from heyitsanthony/serialize-applier-snapmerge
etcdserver: serialize snapshot merger with applier
2015-12-29 19:37:11 -08:00
Anthony Romano 4cd86ae1ef etcdserver: serialize snapshot merger with applier
Avoids inconsistent snapshotting by only attempting to
create a snapshot after an apply completes.

Fixes #4061
2015-12-29 18:38:39 -08:00
Anthony Romano c7c3bda8b9 etcdserver: respect done channel when sleeping for snapshot backoff 2015-12-29 13:23:41 -08:00