Compare commits
53 Commits
Author | SHA1 | Date | |
---|---|---|---|
![]() |
13b0e72304 | ||
![]() |
43791a2f41 | ||
![]() |
a8d966d8f3 | ||
![]() |
ad7194d5d0 | ||
![]() |
084dcb5596 | ||
![]() |
8a0266a806 | ||
![]() |
2338481bb1 | ||
![]() |
ce1e19ae2f | ||
![]() |
a288333e6f | ||
![]() |
097aac79f5 | ||
![]() |
cd820269a6 | ||
![]() |
ac7e6bb002 | ||
![]() |
4b45cd4110 | ||
![]() |
fb426aec9e | ||
![]() |
774cb03f83 | ||
![]() |
4fb6087f4a | ||
![]() |
5524131a9e | ||
![]() |
3efb4d837b | ||
![]() |
20147c5357 | ||
![]() |
494d2c67aa | ||
![]() |
fb32a999a6 | ||
![]() |
d2f5934aa1 | ||
![]() |
4f3fb5a702 | ||
![]() |
9f5ec7732e | ||
![]() |
774cf34827 | ||
![]() |
92df44276d | ||
![]() |
eb00f200d3 | ||
![]() |
38d16775ab | ||
![]() |
690fd12b07 | ||
![]() |
b31483b2be | ||
![]() |
e9a21dda4b | ||
![]() |
2134036942 | ||
![]() |
6bd2ee4c49 | ||
![]() |
fcd429467e | ||
![]() |
e5e759b962 | ||
![]() |
d8a08f53e3 | ||
![]() |
3e95bf0fa3 | ||
![]() |
0d2512cb99 | ||
![]() |
a29f6fb799 | ||
![]() |
f4f429d4e3 | ||
![]() |
fc2afe1ed2 | ||
![]() |
24a442383b | ||
![]() |
f387bf8464 | ||
![]() |
83b06c0715 | ||
![]() |
75dc10c39d | ||
![]() |
66acf8a4e9 | ||
![]() |
1359d29fa4 | ||
![]() |
dc1f4adcd0 | ||
![]() |
e01a1f70c3 | ||
![]() |
2e4ea503b0 | ||
![]() |
c7aef5fdf2 | ||
![]() |
c4605160c5 | ||
![]() |
054de85da2 |
11
CHANGELOG
11
CHANGELOG
@@ -1,3 +1,14 @@
|
||||
v0.4.5
|
||||
* Flush headers immediatly on `wait=true` requests (#877)
|
||||
* Add `ETCD_HTTP_READ_TIMEOUT` and `ETCD_HTTP_WRITE_TIMEOUT` (#880)
|
||||
* Add `ETCDCTL_PEERS` configuration to etcdctl (#95)
|
||||
* etcdctl takes stdin for mk (#91)
|
||||
|
||||
v0.4.4
|
||||
* Fix `--no-sync` flag in etcdctl (#83)
|
||||
* Improved logging for machine removal (#844)
|
||||
* Various documentation improvements (#858, #851, #847)
|
||||
|
||||
v0.4.3
|
||||
* Avoid panic() on truncated or unexpected log data (#834, #833)
|
||||
* Fix missing stats field (#807)
|
||||
|
@@ -30,47 +30,32 @@ The coding style suggested by the Golang community is used in etcd. See [style d
|
||||
|
||||
Please follow this style to make etcd easy to review, maintain and develop.
|
||||
|
||||
### Format of the commit message
|
||||
### Format of the Commit Message
|
||||
|
||||
etcd follows a rough convention for commit messages borrowed from Angularjs. This is an example of a commit:
|
||||
We follow a rough convention for commit messages that is designed to answer two
|
||||
questions: what changed and why. The subject line should feature the what and
|
||||
the body of the commit should describe the why.
|
||||
|
||||
```
|
||||
feat(scripts/test-cluster): add a cluster test command
|
||||
scripts: add the test-cluster command
|
||||
|
||||
this uses tmux to setup a test cluster that you can easily kill and
|
||||
start for debugging.
|
||||
this uses tmux to setup a test cluster that you can easily kill and
|
||||
start for debugging.
|
||||
|
||||
Fixes #38
|
||||
```
|
||||
|
||||
The format can be more formally described as follows:
|
||||
The format can be described more formally as follows:
|
||||
|
||||
```
|
||||
<type>(<scope>): <subject>
|
||||
<subsystem>: <what changed>
|
||||
<BLANK LINE>
|
||||
<body>
|
||||
<why this change was made>
|
||||
<BLANK LINE>
|
||||
<footer>
|
||||
```
|
||||
|
||||
The first line is the subject and should be no longer than 70 characters, the second line is always blank, and other lines should be wrapped at 80 characters. This allows the message to be easier to read on github as well as
|
||||
in various git tools.
|
||||
|
||||
### Subject line
|
||||
|
||||
The subject line contains a succinct description of the change.
|
||||
|
||||
### Allowed <type>s
|
||||
- feat (feature)
|
||||
- fix (bug fix)
|
||||
- docs (documentation)
|
||||
- style (formatting, missing semi colons, …)
|
||||
- refactor
|
||||
- test (when adding missing tests)
|
||||
- chore (maintain)
|
||||
|
||||
### Allowed <scope>s
|
||||
|
||||
Scopes can be anything specifying the place of the commit change within the repository. For example, "store", "API", etc.
|
||||
|
||||
### More details on commits
|
||||
|
||||
For more details see the [angularjs commit style guide](https://docs.google.com/a/coreos.com/document/d/1QrDFcIiPjSLDn3EL15IJygNPiHORgU1_OOAqWjiDU5Y/edit#).
|
||||
The first line is the subject and should be no longer than 70 characters, the
|
||||
second line is always blank, and other lines should be wrapped at 80 characters.
|
||||
This allows the message to be easier to read on GitHub as well as in various
|
||||
git tools.
|
||||
|
@@ -2,7 +2,7 @@ FROM ubuntu:12.04
|
||||
# Let's install go just like Docker (from source).
|
||||
RUN apt-get update -q
|
||||
RUN DEBIAN_FRONTEND=noninteractive apt-get install -qy build-essential curl git
|
||||
RUN curl -s https://go.googlecode.com/files/go1.2.1.src.tar.gz | tar -v -C /usr/local -xz
|
||||
RUN curl -s https://storage.googleapis.com/golang/go1.3.src.tar.gz | tar -v -C /usr/local -xz
|
||||
RUN cd /usr/local/go/src && ./make.bash --no-clean 2>&1
|
||||
ENV PATH /usr/local/go/bin:$PATH
|
||||
ADD . /opt/etcd
|
||||
|
@@ -843,13 +843,13 @@ The client is told the write was successful and the keyspace is updated.
|
||||
Meanwhile F2 has partitioned from the network and will have an out-of-date version of the keyspace until the partition resolves.
|
||||
Since F2 missed the most recent write, a client reading from F2 will have an out-of-date version of the keyspace.
|
||||
|
||||
## Lock Module (*Deprecated*)
|
||||
## Lock Module (*Deprecated and Removed*)
|
||||
|
||||
The lock module is used to serialize access to resources used by clients.
|
||||
Multiple clients can attempt to acquire a lock but only one can have it at a time.
|
||||
Once the lock is released, the next client waiting for the lock will receive it.
|
||||
|
||||
**Warning:** This module is deprecated at v0.4. See [Modules][modules] for more details.
|
||||
**Warning:** This module is deprecated and removed at v0.4. See [Modules][modules] for more details.
|
||||
|
||||
|
||||
### Acquiring a Lock
|
||||
|
@@ -29,16 +29,16 @@ The v2 API has a lot of features, we will categorize them in a few categories:
|
||||
|
||||
### Supported features matrix
|
||||
|
||||
| Client| [go-etcd](https://github.com/coreos/go-etcd) | [jetcd](https://github.com/diwakergupta/jetcd) | [python-etcd](https://github.com/jplana/python-etcd) | [python-etcd-client](https://github.com/dsoprea/PythonEtcdClient) | [node-etcd](https://github.com/stianeikeland/node-etcd) | [nodejs-etcd](https://github.com/lavagetto/nodejs-etcd) | [etcd-ruby](https://github.com/ranjib/etcd-ruby) | [etcd-api](https://github.com/jdarcy/etcd-api) | [cetcd](https://github.com/dwwoelfel/cetcd) | [clj-etcd](https://github.com/rthomas/clj-etcd) | [etcetera](https://github.com/drusellers/etcetera)| [Etcd.jl](https://github.com/forio/Etcd.jl) |
|
||||
| --- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
||||
| **HTTPS Auth** | Y | Y | Y | Y | Y | Y | - | - | - | - | - | - |
|
||||
| **Reconnect** | Y | - | Y | Y | - | - | - | Y | - | - | - | - |
|
||||
| **Mod/Lock** | - | - | Y | Y | - | - | - | - | - | - | - | Y |
|
||||
| **Mod/Leader** | - | - | - | Y | - | - | - | - | - | - | - | Y |
|
||||
| **GET Features** | F | B | F | F | F | F | F | B | F | G | F | F |
|
||||
| **PUT Features** | F | B | F | F | F | F | F | G | F | G | F | F |
|
||||
| **POST Features** | F | - | F | F | - | F | F | - | - | - | F | F |
|
||||
| **DEL Features** | F | B | F | F | F | F | F | B | G | B | F | F |
|
||||
| Client| [go-etcd](https://github.com/coreos/go-etcd) | [jetcd](https://github.com/diwakergupta/jetcd) | [python-etcd](https://github.com/jplana/python-etcd) | [python-etcd-client](https://github.com/dsoprea/PythonEtcdClient) | [node-etcd](https://github.com/stianeikeland/node-etcd) | [nodejs-etcd](https://github.com/lavagetto/nodejs-etcd) | [etcd-ruby](https://github.com/ranjib/etcd-ruby) | [etcd-api](https://github.com/jdarcy/etcd-api) | [cetcd](https://github.com/dwwoelfel/cetcd) | [clj-etcd](https://github.com/rthomas/clj-etcd) | [etcetera](https://github.com/drusellers/etcetera)| [Etcd.jl](https://github.com/forio/Etcd.jl) | [p5-etcd](https://metacpan.org/release/Etcd)
|
||||
| --- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
||||
| **HTTPS Auth** | Y | Y | Y | Y | Y | Y | - | - | - | - | - | - | - |
|
||||
| **Reconnect** | Y | - | Y | Y | - | - | - | Y | - | - | - | - | - |
|
||||
| **Mod/Lock** | - | - | Y | Y | - | - | - | - | - | - | - | Y | - |
|
||||
| **Mod/Leader** | - | - | - | Y | - | - | - | - | - | - | - | Y | - |
|
||||
| **GET Features** | F | B | F | F | F | F | F | B | F | G | F | F | F |
|
||||
| **PUT Features** | F | B | F | F | F | F | F | G | F | G | F | F | F |
|
||||
| **POST Features** | F | - | F | F | - | F | F | - | - | - | F | F | F |
|
||||
| **DEL Features** | F | B | F | F | F | F | F | B | G | B | F | F | F |
|
||||
|
||||
**Legend**
|
||||
|
||||
|
@@ -8,6 +8,7 @@ For more information on how etcd can locate the cluster, see the [finding the cl
|
||||
|
||||
Please note - at least 3 nodes are required for [cluster availability][optimal-cluster-size].
|
||||
|
||||
[cluster-finding]: https://github.com/coreos/etcd/blob/master/Documentation/design/cluster-finding.md
|
||||
[optimal-cluster-size]: https://github.com/coreos/etcd/blob/master/Documentation/optimal-cluster-size.md
|
||||
|
||||
## Using discovery.etcd.io
|
||||
@@ -27,8 +28,8 @@ Here's a full example:
|
||||
```
|
||||
TOKEN=$(curl https://discovery.etcd.io/new)
|
||||
./etcd -name instance1 -peer-addr 10.1.2.3:7001 -addr 10.1.2.3:4001 -discovery $TOKEN
|
||||
./etcd -name instance2 -peer-addr 10.1.2.4:7002 -addr 10.1.2.4:4002 -discovery $TOKEN
|
||||
./etcd -name instance3 -peer-addr 10.1.2.5:7002 -addr 10.1.2.5:4002 -discovery $TOKEN
|
||||
./etcd -name instance2 -peer-addr 10.1.2.4:7001 -addr 10.1.2.4:4001 -discovery $TOKEN
|
||||
./etcd -name instance3 -peer-addr 10.1.2.5:7001 -addr 10.1.2.5:4001 -discovery $TOKEN
|
||||
```
|
||||
|
||||
## Running Your Own Discovery Endpoint
|
||||
@@ -38,8 +39,8 @@ The discovery API communicates with a separate etcd cluster to store and retriev
|
||||
```
|
||||
TOKEN="testcluster"
|
||||
./etcd -name instance1 -peer-addr 10.1.2.3:7001 -addr 10.1.2.3:4001 -discovery http://10.10.10.10:4001/v2/keys/$TOKEN
|
||||
./etcd -name instance2 -peer-addr 10.1.2.4:7002 -addr 10.1.2.4:4002 -discovery http://10.10.10.10:4001/v2/keys/$TOKEN
|
||||
./etcd -name instance3 -peer-addr 10.1.2.5:7002 -addr 10.1.2.5:4002 -discovery http://10.10.10.10:4001/v2/keys/$TOKEN
|
||||
./etcd -name instance2 -peer-addr 10.1.2.4:7001 -addr 10.1.2.4:4001 -discovery http://10.10.10.10:4001/v2/keys/$TOKEN
|
||||
./etcd -name instance3 -peer-addr 10.1.2.5:7001 -addr 10.1.2.5:4001 -discovery http://10.10.10.10:4001/v2/keys/$TOKEN
|
||||
```
|
||||
|
||||
If you're interested in how to discovery API works behind the scenes, read about the [Discovery Protocol](https://github.com/coreos/etcd/blob/master/Documentation/discovery-protocol.md).
|
||||
@@ -52,8 +53,6 @@ The Discovery API submits the `-peer-addr` of each etcd instance to the configur
|
||||
|
||||
The discovery API will automatically clean up the address of a stale peer that is no longer part of the cluster. The TTL for this process is a week, which should be long enough to handle any extremely long outage you may encounter. There is no harm in having stale peers in the list until they are cleaned up, since an etcd instance only needs to connect to one valid peer in the cluster to join.
|
||||
|
||||
[discovery-design]: https://github.com/coreos/etcd/blob/master/Documentation/design/cluster-finding.md
|
||||
|
||||
## Lifetime of a Discovery URL
|
||||
|
||||
A discovery URL identifies a single etcd cluster. Do not re-use discovery URLs for new clusters.
|
||||
|
@@ -1,6 +1,8 @@
|
||||
# Etcd Configuration
|
||||
|
||||
Configuration options can be set in three places:
|
||||
## Node Configuration
|
||||
|
||||
Individual node configuration options can be set in three places:
|
||||
|
||||
1. Command line flags
|
||||
2. Environment variables
|
||||
@@ -10,6 +12,16 @@ Options set on the command line take precedence over all other sources.
|
||||
Options set in environment variables take precedence over options set in
|
||||
configuration files.
|
||||
|
||||
## Cluster Configuration
|
||||
|
||||
Cluster-wide settings are configured via the `/config` admin endpoint and additionally in the configuration file. Values contained in the configuration file will seed the cluster setting with the provided value. After the cluster is running, only the admin endpoint is used.
|
||||
|
||||
The full documentation is contained in the [API docs](https://github.com/coreos/etcd/blob/master/Documentation/api.md#cluster-config).
|
||||
|
||||
* `activeSize` - the maximum number of peers that can participate in the consensus protocol. Other peers will join as standbys.
|
||||
* `removeDelay` - the minimum time in seconds that a machine has been observed to be unresponsive before it is removed from the cluster.
|
||||
* `syncInterval` - the amount of time in seconds between cluster sync when it runs in standby mode.
|
||||
|
||||
## Command Line Flags
|
||||
|
||||
### Required
|
||||
@@ -20,6 +32,8 @@ configuration files.
|
||||
|
||||
* `-addr` - The advertised public hostname:port for client communication. Defaults to `127.0.0.1:4001`.
|
||||
* `-discovery` - A URL to use for discovering the peer list. (i.e `"https://discovery.etcd.io/your-unique-key"`).
|
||||
* `-http-read-timeout` - The number of seconds before an HTTP read operation is timed out.
|
||||
* `-http-write-timeout` - The number of seconds before an HTTP write operation is timed out.
|
||||
* `-bind-addr` - The listening hostname for client communication. Defaults to advertised IP.
|
||||
* `-peers` - A comma separated list of peers in the cluster (i.e `"203.0.113.101:7001,203.0.113.102:7001"`).
|
||||
* `-peers-file` - The file path containing a comma separated list of peers in the cluster.
|
||||
@@ -31,7 +45,6 @@ configuration files.
|
||||
* `-cpuprofile` - The path to a file to output CPU profile data. Enables CPU profiling when present.
|
||||
* `-data-dir` - The directory to store log and snapshot. Defaults to the current working directory.
|
||||
* `-max-result-buffer` - The max size of result buffer. Defaults to `1024`.
|
||||
* `-max-cluster-size` - The max size of the cluster. Defaults to `9`.
|
||||
* `-max-retry-attempts` - The max retry attempts when trying to join a cluster. Defaults to `3`.
|
||||
* `-peer-addr` - The advertised public hostname:port for server communication. Defaults to `127.0.0.1:7001`.
|
||||
* `-peer-bind-addr` - The listening hostname for server communication. Defaults to advertised IP.
|
||||
@@ -41,6 +54,9 @@ configuration files.
|
||||
* `-peer-election-timeout` - The number of milliseconds to wait before the leader is declared unhealthy.
|
||||
* `-peer-heartbeat-interval` - The number of milliseconds in between heartbeat requests
|
||||
* `-snapshot=false` - Disable log snapshots. Defaults to `true`.
|
||||
* `-cluster-active-size` - The expected number of instances participating in the consensus protocol. Only applied if the etcd instance is the first peer in the cluster.
|
||||
* `-cluster-remove-delay` - The delay before one node is removed from the cluster since it cannot be connected at all. Only applied if the etcd instance is the first peer in the cluster.
|
||||
* `-cluster-sync-interval` - The interval between synchronization for standby-mode instance with the cluster. Only applied if the etcd instance is the first peer in the cluster.
|
||||
* `-v` - Enable verbose logging. Defaults to `false`.
|
||||
* `-vv` - Enable very verbose logging. Defaults to `false`.
|
||||
* `-version` - Print the version and exit.
|
||||
@@ -59,6 +75,8 @@ cors = []
|
||||
cpu_profile_file = ""
|
||||
data_dir = "."
|
||||
discovery = "http://etcd.local:4001/v2/keys/_etcd/registry/examplecluster"
|
||||
http_read_timeout = 10
|
||||
http_write_timeout = 10
|
||||
key_file = ""
|
||||
peers = []
|
||||
peers_file = ""
|
||||
@@ -76,6 +94,11 @@ bind_addr = "127.0.0.1:7001"
|
||||
ca_file = ""
|
||||
cert_file = ""
|
||||
key_file = ""
|
||||
|
||||
[cluster]
|
||||
active_size = 9
|
||||
remove_delay = 1800.0
|
||||
sync_interval = 5.0
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
@@ -89,6 +112,8 @@ key_file = ""
|
||||
* `ETCD_CPU_PROFILE_FILE`
|
||||
* `ETCD_DATA_DIR`
|
||||
* `ETCD_DISCOVERY`
|
||||
* `ETCD_CLUSTER_HTTP_READ_TIMEOUT`
|
||||
* `ETCD_CLUSTER_HTTP_WRITE_TIMEOUT`
|
||||
* `ETCD_KEY_FILE`
|
||||
* `ETCD_PEERS`
|
||||
* `ETCD_PEERS_FILE`
|
||||
@@ -105,3 +130,6 @@ key_file = ""
|
||||
* `ETCD_PEER_CERT_FILE`
|
||||
* `ETCD_PEER_KEY_FILE`
|
||||
* `ETCD_PEER_ELECTION_TIMEOUT`
|
||||
* `ETCD_CLUSTER_ACTIVE_SIZE`
|
||||
* `ETCD_CLUSTER_REMOVE_DELAY`
|
||||
* `ETCD_CLUSTER_SYNC_INTERVAL`
|
||||
|
@@ -17,7 +17,6 @@
|
||||
|
||||
**Python libraries**
|
||||
|
||||
- [transitorykris/etcd-py](https://github.com/transitorykris/etcd-py)
|
||||
- [jplana/python-etcd](https://github.com/jplana/python-etcd) - Supports v2
|
||||
- [russellhaering/txetcd](https://github.com/russellhaering/txetcd) - a Twisted Python library
|
||||
- [cholcombe973/autodock](https://github.com/cholcombe973/autodock) - A docker deployment automation tool
|
||||
@@ -90,3 +89,6 @@ A detailed recap of client functionalities can be found in the [clients compatib
|
||||
- [configdb](https://git.autistici.org/ai/configdb/tree/master) - A REST relational abstraction on top of arbitrary database backends, aimed at storing configs and inventories.
|
||||
- [scrz](https://github.com/scrz/scrz) - Container manager, stores configuration in etcd.
|
||||
- [fleet](https://github.com/coreos/fleet) - Distributed init system
|
||||
- [GoogleCloudPlatform/kubernetes](https://github.com/GoogleCloudPlatform/kubernetes) - Container cluster manager.
|
||||
- [mailgun/vulcand](https://github.com/mailgun/vulcand) - HTTP proxy that uses etcd as a configuration backend.
|
||||
- [duedil-ltd/discodns](https://github.com/duedil-ltd/discodns) - Simple DNS nameserver using etcd as a database for names and records.
|
||||
|
@@ -1,7 +1,7 @@
|
||||
## Modules
|
||||
|
||||
etcd has a number of modules that are built on top of the core etcd API.
|
||||
These modules provide things like dashboards, locks and leader election.
|
||||
These modules provide things like dashboards, locks and leader election (removed).
|
||||
|
||||
**Warning**: Modules are deprecated from v0.4 until we have a solid base we can apply them back onto.
|
||||
For now, we are choosing to focus on raft algorithm and core etcd to make sure that it works correctly and fast.
|
||||
@@ -81,7 +81,7 @@ curl -X DELETE http://127.0.0.1:4001/mod/v2/lock/customer1?value=bar
|
||||
```
|
||||
|
||||
|
||||
### Leader Election
|
||||
### Leader Election (Deprecated and Removed in 0.4)
|
||||
|
||||
The Leader Election module wraps the Lock module to allow clients to come to consensus on a single value.
|
||||
This is useful when you want one server to process at a time but allow other servers to fail over.
|
||||
|
@@ -28,8 +28,8 @@ The other important cluster optimization is to always have an odd active cluster
|
||||
|--------------|------------|-------------------|
|
||||
| 1 peers | 1 peers | None |
|
||||
| 3 peers | 2 peers | 1 peer |
|
||||
| 4 peers | 3 peers | 2 peers |
|
||||
| 5 peers | 3 peers | **3 peers** |
|
||||
| 4 peers | 3 peers | 1 peer |
|
||||
| 5 peers | 3 peers | **2 peers** |
|
||||
| 6 peers | 4 peers | 2 peers |
|
||||
| 7 peers | 4 peers | **3 peers** |
|
||||
| 8 peers | 5 peers | 3 peers |
|
||||
|
@@ -1,4 +1,4 @@
|
||||
ectd is being used successfully by many companies in production. It is,
|
||||
etcd is being used successfully by many companies in production. It is,
|
||||
however, under active development and systems like etcd are difficult to get
|
||||
correct. If you are comfortable with bleeding-edge software please use etcd and
|
||||
provide us with the feedback and testing young software needs.
|
||||
|
5
NOTICE
Normal file
5
NOTICE
Normal file
@@ -0,0 +1,5 @@
|
||||
CoreOS Project
|
||||
Copyright 2014 CoreOS, Inc
|
||||
|
||||
This product includes software developed at CoreOS, Inc.
|
||||
(http://www.coreos.com/).
|
@@ -1,6 +1,6 @@
|
||||
# etcd
|
||||
|
||||
README version 0.4.3
|
||||
README version 0.4.5
|
||||
|
||||
A highly-available key value store for shared configuration and service discovery.
|
||||
etcd is inspired by [Apache ZooKeeper][zookeeper] and [doozer][doozer], with a focus on being:
|
||||
@@ -95,7 +95,7 @@ You have successfully started an etcd on a single machine and written a key to t
|
||||
## Contact
|
||||
|
||||
- Mailing list: [etcd-dev](https://groups.google.com/forum/?hl=en#!forum/etcd-dev)
|
||||
- IRC: #[coreos](irc://irc.freenode.org:6667/#coreos) oon freenode.org
|
||||
- IRC: #[coreos](irc://irc.freenode.org:6667/#coreos) on freenode.org
|
||||
- Planning/Roadmap: [milestones](https://github.com/coreos/etcd/issues/milestones)
|
||||
- Bugs: [issues](https://github.com/coreos/etcd/issues)
|
||||
|
||||
|
@@ -36,7 +36,7 @@ var newFlagNameLookup = map[string]string{
|
||||
"d": "data-dir",
|
||||
"m": "max-result-buffer",
|
||||
"r": "max-retry-attempts",
|
||||
"maxsize": "max-cluster-size",
|
||||
"maxsize": "cluster-active-size",
|
||||
"clientCAFile": "ca-file",
|
||||
"clientCert": "cert-file",
|
||||
"clientKey": "key-file",
|
||||
@@ -45,6 +45,7 @@ var newFlagNameLookup = map[string]string{
|
||||
"serverKey": "peer-key-file",
|
||||
"snapshotCount": "snapshot-count",
|
||||
"peer-heartbeat-timeout": "peer-heartbeat-interval",
|
||||
"max-cluster-size": "cluster-active-size",
|
||||
}
|
||||
|
||||
// Config represents the server configuration.
|
||||
@@ -61,6 +62,8 @@ type Config struct {
|
||||
Discovery string `toml:"discovery" env:"ETCD_DISCOVERY"`
|
||||
Force bool
|
||||
KeyFile string `toml:"key_file" env:"ETCD_KEY_FILE"`
|
||||
HTTPReadTimeout float64 `toml:"http_read_timeout" env:"ETCD_HTTP_READ_TIMEOUT"`
|
||||
HTTPWriteTimeout float64 `toml:"http_write_timeout" env:"ETCD_HTTP_WRITE_TIMEOUT"`
|
||||
Peers []string `toml:"peers" env:"ETCD_PEERS"`
|
||||
PeersFile string `toml:"peers_file" env:"ETCD_PEERS_FILE"`
|
||||
MaxResultBuffer int `toml:"max_result_buffer" env:"ETCD_MAX_RESULT_BUFFER"`
|
||||
@@ -97,6 +100,8 @@ func New() *Config {
|
||||
c := new(Config)
|
||||
c.SystemPath = DefaultSystemConfigPath
|
||||
c.Addr = "127.0.0.1:4001"
|
||||
c.HTTPReadTimeout = server.DefaultReadTimeout
|
||||
c.HTTPWriteTimeout = server.DefaultWriteTimeout
|
||||
c.MaxResultBuffer = 1024
|
||||
c.MaxRetryAttempts = 3
|
||||
c.RetryInterval = 10.0
|
||||
@@ -254,6 +259,9 @@ func (c *Config) LoadFlags(arguments []string) error {
|
||||
f.StringVar(&c.Peer.CertFile, "peer-cert-file", c.Peer.CertFile, "")
|
||||
f.StringVar(&c.Peer.KeyFile, "peer-key-file", c.Peer.KeyFile, "")
|
||||
|
||||
f.Float64Var(&c.HTTPReadTimeout, "http-read-timeout", c.HTTPReadTimeout, "")
|
||||
f.Float64Var(&c.HTTPWriteTimeout, "http-write-timeout", c.HTTPReadTimeout, "")
|
||||
|
||||
f.StringVar(&c.DataDir, "data-dir", c.DataDir, "")
|
||||
f.IntVar(&c.MaxResultBuffer, "max-result-buffer", c.MaxResultBuffer, "")
|
||||
f.IntVar(&c.MaxRetryAttempts, "max-retry-attempts", c.MaxRetryAttempts, "")
|
||||
@@ -297,6 +305,8 @@ func (c *Config) LoadFlags(arguments []string) error {
|
||||
f.IntVar(&c.MaxRetryAttempts, "r", c.MaxRetryAttempts, "(deprecated)")
|
||||
f.IntVar(&c.SnapshotCount, "snapshotCount", c.SnapshotCount, "(deprecated)")
|
||||
f.IntVar(&c.Peer.HeartbeatInterval, "peer-heartbeat-timeout", c.Peer.HeartbeatInterval, "(deprecated)")
|
||||
f.IntVar(&c.Cluster.ActiveSize, "max-cluster-size", c.Cluster.ActiveSize, "(deprecated)")
|
||||
f.IntVar(&c.Cluster.ActiveSize, "maxsize", c.Cluster.ActiveSize, "(deprecated)")
|
||||
// END DEPRECATED FLAGS
|
||||
|
||||
if err := f.Parse(arguments); err != nil {
|
||||
|
@@ -27,9 +27,11 @@ func TestConfigTOML(t *testing.T) {
|
||||
max_result_buffer = 512
|
||||
max_retry_attempts = 5
|
||||
name = "test-name"
|
||||
http_read_timeout = 2.34
|
||||
snapshot = true
|
||||
verbose = true
|
||||
very_verbose = true
|
||||
http_write_timeout = 1.23
|
||||
|
||||
[peer]
|
||||
addr = "127.0.0.1:7002"
|
||||
@@ -52,6 +54,8 @@ func TestConfigTOML(t *testing.T) {
|
||||
assert.Equal(t, c.CorsOrigins, []string{"*"}, "")
|
||||
assert.Equal(t, c.DataDir, "/tmp/data", "")
|
||||
assert.Equal(t, c.Discovery, "http://example.com/foobar", "")
|
||||
assert.Equal(t, c.HTTPReadTimeout, 2.34, "")
|
||||
assert.Equal(t, c.HTTPWriteTimeout, 1.23, "")
|
||||
assert.Equal(t, c.KeyFile, "/tmp/file.key", "")
|
||||
assert.Equal(t, c.BindAddr, "127.0.0.1:4003", "")
|
||||
assert.Equal(t, c.Peers, []string{"coreos.com:4001", "coreos.com:4002"}, "")
|
||||
@@ -80,6 +84,8 @@ func TestConfigEnv(t *testing.T) {
|
||||
os.Setenv("ETCD_CORS", "localhost:4001,localhost:4002")
|
||||
os.Setenv("ETCD_DATA_DIR", "/tmp/data")
|
||||
os.Setenv("ETCD_DISCOVERY", "http://example.com/foobar")
|
||||
os.Setenv("ETCD_HTTP_READ_TIMEOUT", "2.34")
|
||||
os.Setenv("ETCD_HTTP_WRITE_TIMEOUT", "1.23")
|
||||
os.Setenv("ETCD_KEY_FILE", "/tmp/file.key")
|
||||
os.Setenv("ETCD_BIND_ADDR", "127.0.0.1:4003")
|
||||
os.Setenv("ETCD_PEERS", "coreos.com:4001,coreos.com:4002")
|
||||
@@ -107,6 +113,8 @@ func TestConfigEnv(t *testing.T) {
|
||||
assert.Equal(t, c.CorsOrigins, []string{"localhost:4001", "localhost:4002"}, "")
|
||||
assert.Equal(t, c.DataDir, "/tmp/data", "")
|
||||
assert.Equal(t, c.Discovery, "http://example.com/foobar", "")
|
||||
assert.Equal(t, c.HTTPReadTimeout, 2.34, "")
|
||||
assert.Equal(t, c.HTTPWriteTimeout, 1.23, "")
|
||||
assert.Equal(t, c.KeyFile, "/tmp/file.key", "")
|
||||
assert.Equal(t, c.BindAddr, "127.0.0.1:4003", "")
|
||||
assert.Equal(t, c.Peers, []string{"coreos.com:4001", "coreos.com:4002"}, "")
|
||||
@@ -553,19 +561,12 @@ func TestConfigClusterRemoveDelayFlag(t *testing.T) {
|
||||
assert.Equal(t, c.Cluster.RemoveDelay, 100.0, "")
|
||||
}
|
||||
|
||||
// Ensures that the cluster sync interval can be parsed from the environment.
|
||||
func TestConfigClusterSyncIntervalEnv(t *testing.T) {
|
||||
withEnv("ETCD_CLUSTER_SYNC_INTERVAL", "10", func(c *Config) {
|
||||
assert.Nil(t, c.LoadEnv(), "")
|
||||
assert.Equal(t, c.Cluster.SyncInterval, 10.0, "")
|
||||
})
|
||||
}
|
||||
|
||||
// Ensures that the cluster sync interval flag can be parsed.
|
||||
func TestConfigClusterSyncIntervalFlag(t *testing.T) {
|
||||
c := New()
|
||||
assert.Nil(t, c.LoadFlags([]string{"-cluster-sync-interval", "10"}), "")
|
||||
assert.Equal(t, c.Cluster.SyncInterval, 10.0, "")
|
||||
assert.Nil(t, c.LoadFlags([]string{"-http-read-timeout", "2.34"}), "")
|
||||
assert.Equal(t, c.HTTPReadTimeout, 2.34, "")
|
||||
assert.Nil(t, c.LoadFlags([]string{"-http-write-timeout", "1.23"}), "")
|
||||
assert.Equal(t, c.HTTPWriteTimeout, 1.23, "")
|
||||
}
|
||||
|
||||
// Ensures that a system config field is overridden by a custom config field.
|
||||
|
13
etcd/etcd.go
13
etcd/etcd.go
@@ -260,10 +260,19 @@ func (e *Etcd) Run() {
|
||||
|
||||
log.Infof("etcd server [name %s, listen on %s, advertised url %s]", e.Server.Name, e.Config.BindAddr, e.Server.URL())
|
||||
listener := server.NewListener(e.Config.EtcdTLSInfo().Scheme(), e.Config.BindAddr, etcdTLSConfig)
|
||||
e.server = &http.Server{Handler: &ModeHandler{e, serverHTTPHandler, standbyServerHTTPHandler}}
|
||||
|
||||
e.server = &http.Server{Handler: &ModeHandler{e, serverHTTPHandler, standbyServerHTTPHandler},
|
||||
ReadTimeout: time.Duration(e.Config.HTTPReadTimeout) * time.Second,
|
||||
WriteTimeout: time.Duration(e.Config.HTTPWriteTimeout) * time.Second,
|
||||
}
|
||||
|
||||
log.Infof("peer server [name %s, listen on %s, advertised url %s]", e.PeerServer.Config.Name, e.Config.Peer.BindAddr, e.PeerServer.Config.URL)
|
||||
peerListener := server.NewListener(e.Config.PeerTLSInfo().Scheme(), e.Config.Peer.BindAddr, peerTLSConfig)
|
||||
e.peerServer = &http.Server{Handler: &ModeHandler{e, peerServerHTTPHandler, http.NotFoundHandler()}}
|
||||
|
||||
e.peerServer = &http.Server{Handler: &ModeHandler{e, peerServerHTTPHandler, http.NotFoundHandler()},
|
||||
ReadTimeout: time.Duration(server.DefaultReadTimeout) * time.Second,
|
||||
WriteTimeout: time.Duration(server.DefaultWriteTimeout) * time.Second,
|
||||
}
|
||||
|
||||
wg := sync.WaitGroup{}
|
||||
wg.Add(2)
|
||||
|
@@ -1,85 +0,0 @@
|
||||
package leader
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/coreos/etcd/server"
|
||||
"github.com/coreos/etcd/tests"
|
||||
"github.com/coreos/etcd/third_party/github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
// Ensure that a leader can be set and read.
|
||||
func TestModLeaderSet(t *testing.T) {
|
||||
tests.RunServer(func(s *server.Server) {
|
||||
// Set leader.
|
||||
body, status, err := testSetLeader(s, "foo", "xxx", 10)
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, status, 200)
|
||||
assert.Equal(t, body, "2")
|
||||
|
||||
// Check that the leader is set.
|
||||
body, status, err = testGetLeader(s, "foo")
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, status, 200)
|
||||
assert.Equal(t, body, "xxx")
|
||||
|
||||
// Delete leader.
|
||||
body, status, err = testDeleteLeader(s, "foo", "xxx")
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, status, 200)
|
||||
assert.Equal(t, body, "")
|
||||
|
||||
// Check that the leader is removed.
|
||||
body, status, err = testGetLeader(s, "foo")
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, status, 200)
|
||||
assert.Equal(t, body, "")
|
||||
})
|
||||
}
|
||||
|
||||
// Ensure that a leader can be renewed.
|
||||
func TestModLeaderRenew(t *testing.T) {
|
||||
tests.RunServer(func(s *server.Server) {
|
||||
// Set leader.
|
||||
body, status, err := testSetLeader(s, "foo", "xxx", 2)
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, status, 200)
|
||||
assert.Equal(t, body, "2")
|
||||
|
||||
time.Sleep(1 * time.Second)
|
||||
|
||||
// Renew leader.
|
||||
body, status, err = testSetLeader(s, "foo", "xxx", 3)
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, status, 200)
|
||||
assert.Equal(t, body, "2")
|
||||
|
||||
time.Sleep(2 * time.Second)
|
||||
|
||||
// Check that the leader is set.
|
||||
body, status, err = testGetLeader(s, "foo")
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, status, 200)
|
||||
assert.Equal(t, body, "xxx")
|
||||
})
|
||||
}
|
||||
|
||||
func testSetLeader(s *server.Server, key string, name string, ttl int) (string, int, error) {
|
||||
resp, err := tests.PutForm(fmt.Sprintf("%s/mod/v2/leader/%s?name=%s&ttl=%d", s.URL(), key, name, ttl), nil)
|
||||
ret := tests.ReadBody(resp)
|
||||
return string(ret), resp.StatusCode, err
|
||||
}
|
||||
|
||||
func testGetLeader(s *server.Server, key string) (string, int, error) {
|
||||
resp, err := tests.Get(fmt.Sprintf("%s/mod/v2/leader/%s", s.URL(), key))
|
||||
ret := tests.ReadBody(resp)
|
||||
return string(ret), resp.StatusCode, err
|
||||
}
|
||||
|
||||
func testDeleteLeader(s *server.Server, key string, name string) (string, int, error) {
|
||||
resp, err := tests.DeleteForm(fmt.Sprintf("%s/mod/v2/leader/%s?name=%s", s.URL(), key, name), nil)
|
||||
ret := tests.ReadBody(resp)
|
||||
return string(ret), resp.StatusCode, err
|
||||
}
|
@@ -25,7 +25,6 @@ const (
|
||||
)
|
||||
|
||||
// ClusterConfig represents cluster-wide configuration settings.
|
||||
// These settings can only be changed through Raft.
|
||||
type ClusterConfig struct {
|
||||
// ActiveSize is the maximum number of node that can join as Raft followers.
|
||||
// Nodes that join the cluster after the limit is reached are standbys.
|
||||
|
@@ -3,10 +3,16 @@ package server
|
||||
import (
|
||||
"crypto/tls"
|
||||
"net"
|
||||
"time"
|
||||
|
||||
"github.com/coreos/etcd/log"
|
||||
)
|
||||
|
||||
const (
|
||||
DefaultReadTimeout = float64((5 * time.Minute) / time.Second)
|
||||
DefaultWriteTimeout = float64((5 * time.Minute) / time.Second)
|
||||
)
|
||||
|
||||
// TLSServerConfig generates tls configuration based on TLSInfo
|
||||
// If any error happens, this function will call log.Fatal
|
||||
func TLSServerConfig(info *TLSInfo) *tls.Config {
|
||||
|
@@ -853,7 +853,7 @@ func (s *PeerServer) monitorActiveSize() {
|
||||
// If we have more active nodes than we should then remove.
|
||||
if peerCount > activeSize {
|
||||
peer := peers[rand.Intn(len(peers))]
|
||||
log.Infof("%s: removing: %v", s.Config.Name, peer)
|
||||
log.Infof("%s: removing node: %v; peer number %d > expected size %d", s.Config.Name, peer, peerCount, activeSize)
|
||||
if _, err := s.raftServer.Do(&RemoveCommand{Name: peer}); err != nil {
|
||||
log.Infof("%s: warning: remove error: %v", s.Config.Name, err)
|
||||
}
|
||||
|
@@ -188,6 +188,7 @@ func (ps *PeerServer) RemoveHttpHandler(w http.ResponseWriter, req *http.Request
|
||||
|
||||
// Returns a JSON-encoded cluster configuration.
|
||||
func (ps *PeerServer) getClusterConfigHttpHandler(w http.ResponseWriter, req *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(ps.ClusterConfig())
|
||||
}
|
||||
|
||||
@@ -217,6 +218,7 @@ func (ps *PeerServer) setClusterConfigHttpHandler(w http.ResponseWriter, req *ht
|
||||
log.Debugf("[recv] Update Cluster Config Request")
|
||||
ps.server.Dispatch(c, w, req)
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(ps.ClusterConfig())
|
||||
}
|
||||
|
||||
@@ -230,6 +232,7 @@ func (ps *PeerServer) getMachinesHttpHandler(w http.ResponseWriter, req *http.Re
|
||||
}
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(&machines)
|
||||
}
|
||||
|
||||
@@ -237,6 +240,7 @@ func (ps *PeerServer) getMachinesHttpHandler(w http.ResponseWriter, req *http.Re
|
||||
func (ps *PeerServer) getMachineHttpHandler(w http.ResponseWriter, req *http.Request) {
|
||||
vars := mux.Vars(req)
|
||||
m := ps.getMachineMessage(vars["name"], ps.raftServer.Leader())
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(m)
|
||||
}
|
||||
|
||||
|
@@ -1,3 +1,3 @@
|
||||
package server
|
||||
|
||||
const ReleaseVersion = "0.4.3"
|
||||
const ReleaseVersion = "0.4.5"
|
||||
|
@@ -311,6 +311,7 @@ func (s *Server) GetPeersHandler(w http.ResponseWriter, req *http.Request) error
|
||||
|
||||
// Retrieves stats on the Raft server.
|
||||
func (s *Server) GetStatsHandler(w http.ResponseWriter, req *http.Request) error {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.Write(s.peerServer.Stats())
|
||||
return nil
|
||||
}
|
||||
@@ -318,6 +319,7 @@ func (s *Server) GetStatsHandler(w http.ResponseWriter, req *http.Request) error
|
||||
// Retrieves stats on the leader.
|
||||
func (s *Server) GetLeaderStatsHandler(w http.ResponseWriter, req *http.Request) error {
|
||||
if s.peerServer.RaftServer().State() == raft.Leader {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.Write(s.peerServer.PeerStats())
|
||||
return nil
|
||||
}
|
||||
@@ -333,6 +335,7 @@ func (s *Server) GetLeaderStatsHandler(w http.ResponseWriter, req *http.Request)
|
||||
|
||||
// Retrieves stats on the leader.
|
||||
func (s *Server) GetStoreStatsHandler(w http.ResponseWriter, req *http.Request) error {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.Write(s.store.JsonStats())
|
||||
return nil
|
||||
}
|
||||
|
@@ -53,9 +53,11 @@ Other Options:
|
||||
-max-result-buffer Max size of the result buffer.
|
||||
-max-retry-attempts Number of times a node will try to join a cluster.
|
||||
-retry-interval Seconds to wait between cluster join retry attempts.
|
||||
-max-cluster-size Maximum number of nodes in the cluster.
|
||||
-snapshot=false Disable log snapshots
|
||||
-snapshot-count Number of transactions before issuing a snapshot.
|
||||
-cluster-active-size Number of active nodes in the cluster.
|
||||
-cluster-remove-delay Seconds before one node is removed.
|
||||
-cluster-sync-interval Seconds between synchronizations for standby mode.
|
||||
`
|
||||
|
||||
// Usage returns the usage message for etcd.
|
||||
|
@@ -68,6 +68,7 @@ func handleWatch(key string, recursive, stream bool, waitIndex string, w http.Re
|
||||
closeChan := cn.CloseNotify()
|
||||
|
||||
writeHeaders(w, s)
|
||||
w.(http.Flusher).Flush()
|
||||
|
||||
if stream {
|
||||
// watcher hub will not help to remove stream watcher
|
||||
|
@@ -25,6 +25,7 @@ func TestClusterConfigSet(t *testing.T) {
|
||||
resp, _ = tests.Get("http://localhost:7002/v2/admin/config")
|
||||
body := tests.ReadBodyJSON(resp)
|
||||
assert.Equal(t, resp.StatusCode, 200)
|
||||
assert.Equal(t, resp.Header.Get("Content-Type"), "application/json")
|
||||
assert.Equal(t, body["activeSize"], 3)
|
||||
assert.Equal(t, body["removeDelay"], 60)
|
||||
}
|
||||
@@ -44,6 +45,7 @@ func TestClusterConfigReload(t *testing.T) {
|
||||
resp, _ = tests.Get("http://localhost:7002/v2/admin/config")
|
||||
body := tests.ReadBodyJSON(resp)
|
||||
assert.Equal(t, resp.StatusCode, 200)
|
||||
assert.Equal(t, resp.Header.Get("Content-Type"), "application/json")
|
||||
assert.Equal(t, body["activeSize"], 3)
|
||||
assert.Equal(t, body["removeDelay"], 60)
|
||||
|
||||
@@ -59,6 +61,7 @@ func TestClusterConfigReload(t *testing.T) {
|
||||
resp, _ = tests.Get("http://localhost:7002/v2/admin/config")
|
||||
body = tests.ReadBodyJSON(resp)
|
||||
assert.Equal(t, resp.StatusCode, 200)
|
||||
assert.Equal(t, resp.Header.Get("Content-Type"), "application/json")
|
||||
assert.Equal(t, body["activeSize"], 3)
|
||||
assert.Equal(t, body["removeDelay"], 60)
|
||||
}
|
||||
@@ -76,6 +79,7 @@ func TestGetMachines(t *testing.T) {
|
||||
t.FailNow()
|
||||
}
|
||||
assert.Equal(t, resp.StatusCode, 200)
|
||||
assert.Equal(t, resp.Header.Get("Content-Type"), "application/json")
|
||||
machines := make([]map[string]interface{}, 0)
|
||||
b := tests.ReadBody(resp)
|
||||
json.Unmarshal(b, &machines)
|
||||
|
Reference in New Issue
Block a user