Commit Graph

667 Commits (048a948eca5b7030e55f5c3ed1654faa24d4d922)

Author SHA1 Message Date
Xiang Li 3af4a45d7b etcdserver: make raft use leveled logger 2015-06-02 12:50:42 -07:00
Xiang Li 42fe370b35 Merge pull request #2848 from xiang90/metrics
*: use namespace and subsystem in metrics
2015-05-26 14:44:54 -07:00
Xiang Li 34ac145b38 *: use namespace and subsystem in metrics
Fix #2841.

From Prometheus developer:
```
the recommended way for etcd as an open source project and under
consideration of its size would be etcd_<subsystem>_<name>.
```

We made the naming change accordingly.
2015-05-26 14:39:04 -07:00
Xiang Li 3028edd7dc Merge pull request #2856 from xiang90/mrefactor
etcdserver: refactore member.go
2015-05-26 14:37:37 -07:00
Barak Michener 9ef098c5ed etcdserver: fix go vet. Fixes #2859 2015-05-22 13:54:54 -04:00
Xiang Li 58eefda72d Merge pull request #2840 from yichengq/revert-url-equal
Revert "Treat URLs have same IP address as same"
2015-05-21 19:27:19 -07:00
Xiang Li 4a72d3a8bb etcdserver: refactore member.go 2015-05-21 09:19:29 -07:00
Xiang Li 260aad5468 Merge pull request #2830 from xiang90/join_checking
checking cluster version compatibility before joining the existing cluster
2015-05-20 12:25:50 -07:00
Xiang Li aa417ab644 etcdserver: log the per endpoint error in getVersion 2015-05-20 12:10:10 -07:00
Xiang Li db7db689a6 etcdserver: check cluster version compability when joining 2015-05-19 10:19:41 -07:00
Barak Michener a88a53274f security: Lazily create the security directories. Fixes #2755, may find new instances for #2741
revert the kv integration test

fix nits

amend security mention of GUEST
2015-05-18 17:28:04 -04:00
Yicheng Qin 3153e635d5 Revert "Treat URLs have same IP address as same"
This reverts commit f8ce5996b0.

etcd no longer resolves TCP addresses passed in through flags,
so there is no need to compare hostname and IP slices anymore.
(for more details: a3892221ee)

Conflicts:
	etcdserver/cluster.go
	etcdserver/config.go
	pkg/netutil/netutil.go
	pkg/netutil/netutil_test.go
2015-05-16 03:21:10 -07:00
Xiang Li 9f8342dba4 etcdserver: do not get local version via HTTP 2015-05-13 17:19:32 -07:00
Xiang Li 988c30bfba etcdserver: getVersion returns both server and cluster version 2015-05-13 17:04:46 -07:00
Xiang Li 6296054ff6 etcdhttp: version endpoint also returns cluster version. 2015-05-13 15:48:10 -07:00
Yicheng Qin 75ee7f4aa1 Merge pull request #2821 from yichengq/private-cluster
etcdserver: stop exposing Cluster struct
2015-05-13 10:26:48 -07:00
Xiang Li 2690535f8a Merge pull request #2820 from xiang90/cap
version capability checking
2015-05-13 10:16:49 -07:00
Xiang Li d3b1d5c008 etcdhttp: support capability checking
etcdhttp will check the cluster version and update its
capability version periodically.

Any new handler's after 2.0 needs to wrap by capability handler
to ensure it is not accessable until rolling upgrade finished.
2015-05-13 10:11:35 -07:00
Yicheng Qin a6a649f1c3 etcdserver: stop exposing Cluster struct
After this PR, only cluster's interface Cluster is exposed, which makes
code much cleaner. And it avoids external packages to rely on cluster
struct in the future.
2015-05-13 10:01:25 -07:00
Xiang Li f2905f2828 etcdserver: remove unnecessary around detect datadir
The log is super unhelpful. When I have a 2.1.0 etcd, it prints out
`2.0.1 vaild dir`. I have no idea why the data dir of a 2.1.0 etcd is
2.0.1.
2015-05-12 22:06:42 -07:00
Yicheng Qin 032db5e396 *: extract types.Cluster from etcdserver.Cluster
The PR extracts types.Cluster from etcdserver.Cluster. types.Cluster
is used for flag parsing and etcdserver config.

There is no need to expose etcdserver.Cluster public, which contains
lots of etcdserver internal details and methods. This is the first step
for it.
2015-05-12 14:53:11 -07:00
Xiang Li e866314b94 etcdserver: support update cluster version through raft
1. Persist the cluster version change through raft. When the member is restarted, it can recover
the previous known decided cluster version.

2. When there is a new leader, it is forced to do a version checking immediately. This helps to
update the first cluster version fast.
2015-05-12 11:44:34 -07:00
Xiang Li 94ffd72c7e etcdserver: rename StoreAdminPrefix to StoreClusterPrefix
We store cluster related key in StoreAdminPrefix for some
historical reason. The previous API is called admin. But now,
the admin name is gone and `cluster` is a more clear and correct
name.
2015-04-29 12:05:51 -07:00
Xiang Li 6699107f61 *: add cluster version and cluster version detection.
Cluster version is the min major.minor of all members in
the etcd cluster. Cluster version is set to the min version
that a etcd member is compatible with when first bootstrapp.

During a rolling upgrades, the cluster version will be updated
automatically.

For example:

```
Cluster [a:1, b:1 ,c:1] -> clusterVersion 1

update a -> 2, b -> 2

after a detection

Cluster [a:2, b:2 ,c:1] -> clusterVersion 1, since c is still 1

update c -> 2

after a detection

Cluster [a:2, b:2 ,c:2] -> clusterVersion 2
```

The API/raft component can utilize clusterVersion to determine if
it can accept a client request or a raft RPC.

We choose polling rather than pushing since we want to use the same
logic for cluster version detection and (TODO) cluster version checking.

Before a member actually joins a etcd cluster, it should check the version
of the cluster. Push does not work since the other members cannot push
version info to it before it actually joins. Moreover, we do not want our
raft RPC system (which is doing the heartbeat pushing) to coordinate cluster version.
2015-04-29 11:31:59 -07:00
Yicheng Qin 1c1cccd236 rafthttp: stop etcd if it is found removed when stream dial
The original process is stopping etcd only when pipeline message finds itself
has been removed. After this PR, stream dial has this functionality too.
It helps fast etcd stop, which doesn't need to wait for stream break to
fall back to pipeline, and wait for election timeout to send out message
to detect self removal.
2015-04-27 15:10:00 -07:00
Yicheng Qin ebecee34e0 Merge pull request #2701 from yichengq/rafthttp-anon
rafthttp: add remotes
2015-04-24 13:04:37 -07:00
Yicheng Qin 9f19b5660f rafthttp: add AddRemote
Add remotes to rafthttp, who help newly joined members catch up the
progress of the cluster. It supports basic message sending to remote, and
has no stream connection for simplicity. remotes will not be used
after the latest peers have been added into rafthttp.
2015-04-24 11:49:23 -07:00
xiaost cab1e9a723 etcdserver: skip noop entry in apply 2015-04-24 12:15:51 +08:00
Barak Michener fa74e702d8 security: Improve the security api as per the suggestions list in #2384
Subcommits:

decouple root and security enable/disable

create root role

prefix matching

godep: bump go-etcd to include credentials

add godep for speakeasy and auth entry parsing

appropriate errors for security enable/disable

WIP adding to etcd/client all the security client methods

add guest access

minor ui return tweaks

revert client changes

respond to comments, log more security operations

fix major ensure() bug, add better UX

block recursive access

fix some boneheaded mistakes

fix integration test

last comments

fix up security_api.md

philips nits

fix docs
2015-04-23 16:11:38 -04:00
Yicheng Qin 1d96de459a etcdserver: init server stats before passing it as argument
It is more reasonable to init the variable before passing it as an
argument.

It fixes a bug that etcdserver may panic on server stats when processing
a message from rafthttp streamReader before server stats is initialized
in server.Start().
2015-04-22 08:28:08 -07:00
Xiang Li 5ad559b503 *: serve json version on both client and peer url 2015-04-20 16:23:51 -07:00
Yicheng Qin 1811701427 Revert "etcdserver: fix cluster fallback recovery"
This reverts commit cff005777a.

Conflicts:
	etcdserver/server.go
2015-04-19 11:34:33 -07:00
Yicheng Qin 88224f6f4e Revert "etcdserver: not apply stale conf change in cluster and transport"
This reverts commit 40197f0698.
2015-04-19 11:08:03 -07:00
Xiang Li 98f8dfbc9d etcdserver: prevExist=true + condition is compareAndSwap
PrevExist indicates the key should exist. Condition compares with
an existing key. So PrevExist+condition = CompareAndSwap not Update.
2015-04-14 23:44:06 -07:00
xiaost eab2c2224a etcdserver: fix minor bug in EtcdServer.send
it seems to nothing serious.
after deleted peers, the log may output:
"etcdserver: send message to unknown receiver %s"
2015-04-13 20:35:58 +08:00
Yicheng Qin 2141308524 Merge pull request #2631 from yichengq/metrics-fd
etcdserver: metrics and monitor number of file descriptor
2015-04-08 11:28:58 -07:00
Yicheng Qin 7a7e1f7a7c etcdserver: metrics and monitor number of file descriptor
It exposes the metrics of file descriptor limit and file descriptor used.
Moreover, it prints out warning when more than 80% of fd limit has been used.

```
2015/04/08 01:26:19 etcdserver: 80% of the file descriptor limit is open
[open = 969, limit = 1024]
```
2015-04-08 11:17:48 -07:00
Alex Crawford d9ad6aa2a9 *: update to use IANA-assigned ports 2015-04-06 13:49:43 -07:00
Xiang Li 471aa1aa89 Merge pull request #2622 from xiang90/fix_watcher
store: fix watcher removal
2015-04-03 10:39:03 -07:00
Xiang Li 999917010d store: fix watcher removal 2015-04-03 10:13:43 -07:00
Yicheng Qin 9e5743c816 etcdserver: stop raft node goroutine before stop server
Stop raftNode goroutine before stopping server goroutine, so
server.Stop does stop all underlying stuffs elegantly now. This fixes
the problem that previous-round lock on WAL may not be released when
etcd is restarted.
2015-04-01 11:20:51 -07:00
Xiang Li 77a04cda0c Merge pull request #2597 from xiang90/wal-repair
wal: fix the unexpectedEOF error in the last wal.
2015-03-30 13:49:05 -07:00
Xiang Li 253f7c4ae1 Merge pull request #2522 from xiang90/user_pw
etcdserver/etcdhttp: do not return back the password of a user
2015-03-30 13:42:41 -07:00
Xiang Li 0b9a318e68 etcdserver: make the wal repairing logic clear 2015-03-29 21:10:28 -07:00
Xiang Li 1231f82f22 etcdserver: save snapshot into wal first 2015-03-29 14:23:05 -07:00
Xiang Li 8b4eed29e5 wal: fix the unexpectedEOF error in the last wal.
It is safe to repair the unexpectedEOF error in the last wal. raft
will not send out message before the entry successfully comitted
into wal. Thus we can safely truncate the last entry in the wal
to repair.
2015-03-28 21:08:14 -07:00
Yicheng Qin 60efd4d96e Revert "etcdhttp: add internalVersion"
This reverts commit a77bf97c14.

Conflicts:
	version/version.go
2015-03-27 16:53:55 -07:00
Yicheng Qin dd92a2b484 Merge pull request #2556 from yichengq/fix-apply-conf
etcdserver: not apply stale conf change
2015-03-27 14:00:30 -07:00
Kelsey Hightower 538d624cfa etcdserver: add stats.LatencyStats and stats.CountsStats types 2015-03-27 13:42:44 -07:00
Yicheng Qin 40197f0698 etcdserver: not apply stale conf change in cluster and transport 2015-03-27 12:53:34 -07:00