Compare commits

..

53 Commits

Author SHA1 Message Date
Brandon Philips
13b0e72304 CHANGELOG: v0.4.5 2014-07-07 18:11:29 -07:00
Brandon Philips
43791a2f41 Merge pull request #877 from Rawn/flush-headers-on-wait-stream
server: Flush headers when using wait=true and stream=true
2014-07-07 17:04:02 -07:00
Brandon Philips
a8d966d8f3 Merge pull request #880 from philips/add-http-server-timeouts
etcd: add a read/write timeout to server
2014-07-07 16:02:27 -07:00
Mauricio de Abreu Antunes
ad7194d5d0 Documentation: remove etcd-py since it was removed
URL is no longer working
2014-07-07 14:26:56 -07:00
Brandon Philips
084dcb5596 etcd: add a read/write timeout to server
The default is for connections to last forever[1]. This leads to fds
leaking. I set the timeout so high by default so that watches don't have
to keep retrying but perhaps we should set it slower.

Tested on a cluster with lots of clients and it seems to have relieved
the problem.

[1] https://groups.google.com/forum/#!msg/golang-nuts/JFhGwh1q9xU/heh4J8pul3QJ
2014-07-07 11:42:56 -07:00
Yicheng Qin
8a0266a806 Merge pull request #867 from iand/json-headers
fix(server/server.go): /v2/stats endpoints emit application/json content type header
2014-07-07 10:33:27 -07:00
Christoffer Vikström
2338481bb1 server: Flush headers when using wait=true and stream=true
Many http clients will missbehave unless they get an initial http-
response, even when long-polling. It also saves the user/client from
having to handle headers on the first action of the watch, but rather
handle the response immediately.
2014-07-03 18:04:36 +02:00
Cole Gleason
ce1e19ae2f Merge pull request #849 from colegleason/max-cluster-size
docs(cluster-size): remove outdated references to flag max-cluster-size
2014-07-02 12:21:14 -07:00
Ian Davis
a288333e6f fix(server/server.go): /v2/stats endpoints emit application/json content type header 2014-06-30 10:50:43 +01:00
Cole Gleason
097aac79f5 flags: remove MaxClusterSize variable 2014-06-27 10:52:16 -07:00
Brandon Philips
cd820269a6 Merge branch 'master' of github.com:coreos/etcd 2014-06-25 08:57:54 -07:00
Brandon Philips
ac7e6bb002 Merge https://github.com/coreos/template-project
Conflicts:
	CONTRIBUTING.md
	LICENSE
2014-06-25 08:57:02 -07:00
Brandon Philips
4b45cd4110 Merge pull request #869 from sivaramsk/patch-2
Update optimal-cluster-size.md
2014-06-25 08:17:07 -07:00
sivaramsk
fb426aec9e Update optimal-cluster-size.md 2014-06-25 14:20:59 +05:30
Brandon Philips
774cb03f83 server: bump to 0.4.4+git 2014-06-24 10:58:11 -07:00
Brandon Philips
4fb6087f4a CHANGELOG: release 0.4.4 2014-06-24 10:56:53 -07:00
Yicheng Qin
5524131a9e Merge pull request #865 from robstrong/hotfix/contentType
fix(peer_server) set content type to application/json in admin
2014-06-23 16:31:49 -07:00
Brandon Philips
3efb4d837b Merge pull request #844 from unihorn/102
chore(peer_server): improve log for auto removal
2014-06-23 14:19:48 -07:00
Cole Gleason
20147c5357 flags: alias max-cluster-size to cluster-active-size as deprecated flag 2014-06-23 10:50:40 -07:00
Rob Strong
494d2c67aa fix(peer_server) set content type to application/json in admin 2014-06-21 13:13:10 -04:00
Blake Mizerany
fb32a999a6 doc: add note about removal of leader mod 2014-06-19 17:10:34 -07:00
Blake Mizerany
d2f5934aa1 mod: remove defunct leader test 2014-06-19 17:10:34 -07:00
Brandon Philips
4f3fb5a702 Merge pull request #861 from andybons/patch-2
Update to Go v1.3
2014-06-19 13:54:34 -07:00
Andrew Bonventre
9f5ec7732e Update to Go v1.3
Now I’m just being OCD about it.
2014-06-19 10:25:52 -04:00
Brandon Philips
774cf34827 CONTRIBUTING: add a section about not emailing maintainers
Setup some expectation for etiquette.
2014-06-18 18:38:28 -07:00
Brandon Philips
92df44276d CONTRIBUTING: remove the subject verbs
After talking to a few people on the team we have decided that the
feat/fix/etc tags aren't adding much value. Use a simpler format
instead.
2014-06-18 18:36:00 -07:00
Yicheng Qin
eb00f200d3 Merge pull request #856 from robn/patch-1
Add p5-etcd (Perl client lib) to clients-matrix
2014-06-18 10:45:11 -07:00
Jonathan Boulle
38d16775ab Merge pull request #858 from mikeumus/patch-1
docs(readme.md): spelling fix "oon" to "on"
2014-06-17 23:20:23 -07:00
Mikeumus
690fd12b07 docs(readme.md): spelling fix "oon" to "on" 2014-06-18 02:11:32 -04:00
Rob Szumski
b31483b2be Merge pull request #850 from robszumski/update-config
feat(docs): add cluster config section
2014-06-17 17:45:50 -07:00
Rob Szumski
e9a21dda4b Merge pull request #851 from unihorn/103
docs(configuration): add cluster configuration
2014-06-17 17:44:51 -07:00
Yicheng Qin
2134036942 Merge pull request #857 from tarnfeld/tools-discodns
Added discodns to the list of tools using etcd
2014-06-17 16:10:57 -07:00
Tom Arnfeld
6bd2ee4c49 Added discodns to the list of tools using etcd 2014-06-18 00:08:21 +01:00
Robert Norris
fcd429467e Add p5-etcd (Perl client lib) to clients-matrix 2014-06-18 08:15:03 +10:00
Yicheng Qin
e5e759b962 docs(config): refine cluster configuration 2014-06-17 09:31:08 -07:00
Rob Szumski
d8a08f53e3 feat(docs): add cluster config section 2014-06-16 22:31:13 -07:00
Jonathan Boulle
3e95bf0fa3 Merge pull request #854 from brianredbeard/moarthings
docs(libraries-and-tools.md) Add vulcan proxy and kubernetes
2014-06-16 20:05:43 -07:00
Brian 'Redbeard' Harrington
0d2512cb99 docs(libraries-and-tools.md) Add vulcan proxy and kubernetes
Both vulcan proxy (vulcand) and Google kubernetes utilize etcd as
as storage engine.
2014-06-16 20:03:47 -07:00
Yicheng Qin
a29f6fb799 docs(configuration): add cluster configuration 2014-06-16 13:58:00 -07:00
Cole Gleason
f4f429d4e3 docs(cluster-size): remove outdated refrences to flag max-cluster-size 2014-06-16 09:41:37 -07:00
Jonathan Boulle
fc2afe1ed2 Merge pull request #847 from pwaller/patch-1
docs(production-ready.md): Tiny typo fix
2014-06-13 08:58:33 -07:00
Peter Waller
24a442383b docs(production-ready.md): Tiny typo fix 2014-06-13 14:41:23 +01:00
Yicheng Qin
f387bf8464 chore(peer_server): improve log for auto removal 2014-06-12 10:02:56 -07:00
Brandon Philips
83b06c0715 Merge pull request #841 from andybons/patch-1
Update to Go v1.2.2
2014-06-11 09:43:25 -07:00
Andrew Bonventre
75dc10c39d Update to Go v1.2.2 2014-06-09 16:03:01 -04:00
Rob Szumski
66acf8a4e9 Merge pull request #839 from jonboulle/jonboulle-master
docs(cluster-discovery): fix bad link and confusing port references
2014-06-09 09:47:58 -07:00
Jonathan Boulle
1359d29fa4 docs(cluster-discovery): fix bad link and confusing port references 2014-06-08 23:58:14 -07:00
Brandon Philips
dc1f4adcd0 chore(server): bump to 0.4.3+git 2014-06-07 18:17:54 -07:00
Jonathan Boulle
e01a1f70c3 Merge pull request #2 from jonboulle/master
Clean up CONTRIBUTING.md and other bits of template-project
2014-04-04 10:41:40 -07:00
Jonathan Boulle
2e4ea503b0 chore(contributing): clean up CONTRIBUTING.md and split out DCO 2014-04-04 10:40:37 -07:00
Brandon Philips
c7aef5fdf2 Merge pull request #1 from bcwaldon/fix-case
fix(CONTRIBUTING.md): Fix title case
2014-02-05 15:52:24 -08:00
Brian Waldon
c4605160c5 fix(CONTRIBUTING.md): Fix title case 2014-02-05 15:51:24 -08:00
Brandon Philips
054de85da2 feat(*): initial commit 2014-01-19 12:25:11 -08:00
26 changed files with 148 additions and 164 deletions

View File

@@ -1,3 +1,14 @@
v0.4.5
* Flush headers immediatly on `wait=true` requests (#877)
* Add `ETCD_HTTP_READ_TIMEOUT` and `ETCD_HTTP_WRITE_TIMEOUT` (#880)
* Add `ETCDCTL_PEERS` configuration to etcdctl (#95)
* etcdctl takes stdin for mk (#91)
v0.4.4
* Fix `--no-sync` flag in etcdctl (#83)
* Improved logging for machine removal (#844)
* Various documentation improvements (#858, #851, #847)
v0.4.3
* Avoid panic() on truncated or unexpected log data (#834, #833)
* Fix missing stats field (#807)

View File

@@ -30,47 +30,32 @@ The coding style suggested by the Golang community is used in etcd. See [style d
Please follow this style to make etcd easy to review, maintain and develop.
### Format of the commit message
### Format of the Commit Message
etcd follows a rough convention for commit messages borrowed from Angularjs. This is an example of a commit:
We follow a rough convention for commit messages that is designed to answer two
questions: what changed and why. The subject line should feature the what and
the body of the commit should describe the why.
```
feat(scripts/test-cluster): add a cluster test command
scripts: add the test-cluster command
this uses tmux to setup a test cluster that you can easily kill and
start for debugging.
Fixes #38
```
The format can be more formally described as follows:
The format can be described more formally as follows:
```
<type>(<scope>): <subject>
<subsystem>: <what changed>
<BLANK LINE>
<body>
<why this change was made>
<BLANK LINE>
<footer>
```
The first line is the subject and should be no longer than 70 characters, the second line is always blank, and other lines should be wrapped at 80 characters. This allows the message to be easier to read on github as well as
in various git tools.
### Subject line
The subject line contains a succinct description of the change.
### Allowed <type>s
- feat (feature)
- fix (bug fix)
- docs (documentation)
- style (formatting, missing semi colons, …)
- refactor
- test (when adding missing tests)
- chore (maintain)
### Allowed <scope>s
Scopes can be anything specifying the place of the commit change within the repository. For example, "store", "API", etc.
### More details on commits
For more details see the [angularjs commit style guide](https://docs.google.com/a/coreos.com/document/d/1QrDFcIiPjSLDn3EL15IJygNPiHORgU1_OOAqWjiDU5Y/edit#).
The first line is the subject and should be no longer than 70 characters, the
second line is always blank, and other lines should be wrapped at 80 characters.
This allows the message to be easier to read on GitHub as well as in various
git tools.

View File

@@ -2,7 +2,7 @@ FROM ubuntu:12.04
# Let's install go just like Docker (from source).
RUN apt-get update -q
RUN DEBIAN_FRONTEND=noninteractive apt-get install -qy build-essential curl git
RUN curl -s https://go.googlecode.com/files/go1.2.1.src.tar.gz | tar -v -C /usr/local -xz
RUN curl -s https://storage.googleapis.com/golang/go1.3.src.tar.gz | tar -v -C /usr/local -xz
RUN cd /usr/local/go/src && ./make.bash --no-clean 2>&1
ENV PATH /usr/local/go/bin:$PATH
ADD . /opt/etcd

View File

@@ -843,13 +843,13 @@ The client is told the write was successful and the keyspace is updated.
Meanwhile F2 has partitioned from the network and will have an out-of-date version of the keyspace until the partition resolves.
Since F2 missed the most recent write, a client reading from F2 will have an out-of-date version of the keyspace.
## Lock Module (*Deprecated*)
## Lock Module (*Deprecated and Removed*)
The lock module is used to serialize access to resources used by clients.
Multiple clients can attempt to acquire a lock but only one can have it at a time.
Once the lock is released, the next client waiting for the lock will receive it.
**Warning:** This module is deprecated at v0.4. See [Modules][modules] for more details.
**Warning:** This module is deprecated and removed at v0.4. See [Modules][modules] for more details.
### Acquiring a Lock

View File

@@ -29,16 +29,16 @@ The v2 API has a lot of features, we will categorize them in a few categories:
### Supported features matrix
| Client| [go-etcd](https://github.com/coreos/go-etcd) | [jetcd](https://github.com/diwakergupta/jetcd) | [python-etcd](https://github.com/jplana/python-etcd) | [python-etcd-client](https://github.com/dsoprea/PythonEtcdClient) | [node-etcd](https://github.com/stianeikeland/node-etcd) | [nodejs-etcd](https://github.com/lavagetto/nodejs-etcd) | [etcd-ruby](https://github.com/ranjib/etcd-ruby) | [etcd-api](https://github.com/jdarcy/etcd-api) | [cetcd](https://github.com/dwwoelfel/cetcd) | [clj-etcd](https://github.com/rthomas/clj-etcd) | [etcetera](https://github.com/drusellers/etcetera)| [Etcd.jl](https://github.com/forio/Etcd.jl) |
| --- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| **HTTPS Auth** | Y | Y | Y | Y | Y | Y | - | - | - | - | - | - |
| **Reconnect** | Y | - | Y | Y | - | - | - | Y | - | - | - | - |
| **Mod/Lock** | - | - | Y | Y | - | - | - | - | - | - | - | Y |
| **Mod/Leader** | - | - | - | Y | - | - | - | - | - | - | - | Y |
| **GET Features** | F | B | F | F | F | F | F | B | F | G | F | F |
| **PUT Features** | F | B | F | F | F | F | F | G | F | G | F | F |
| **POST Features** | F | - | F | F | - | F | F | - | - | - | F | F |
| **DEL Features** | F | B | F | F | F | F | F | B | G | B | F | F |
| Client| [go-etcd](https://github.com/coreos/go-etcd) | [jetcd](https://github.com/diwakergupta/jetcd) | [python-etcd](https://github.com/jplana/python-etcd) | [python-etcd-client](https://github.com/dsoprea/PythonEtcdClient) | [node-etcd](https://github.com/stianeikeland/node-etcd) | [nodejs-etcd](https://github.com/lavagetto/nodejs-etcd) | [etcd-ruby](https://github.com/ranjib/etcd-ruby) | [etcd-api](https://github.com/jdarcy/etcd-api) | [cetcd](https://github.com/dwwoelfel/cetcd) | [clj-etcd](https://github.com/rthomas/clj-etcd) | [etcetera](https://github.com/drusellers/etcetera)| [Etcd.jl](https://github.com/forio/Etcd.jl) | [p5-etcd](https://metacpan.org/release/Etcd)
| --- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| **HTTPS Auth** | Y | Y | Y | Y | Y | Y | - | - | - | - | - | - | - |
| **Reconnect** | Y | - | Y | Y | - | - | - | Y | - | - | - | - | - |
| **Mod/Lock** | - | - | Y | Y | - | - | - | - | - | - | - | Y | - |
| **Mod/Leader** | - | - | - | Y | - | - | - | - | - | - | - | Y | - |
| **GET Features** | F | B | F | F | F | F | F | B | F | G | F | F | F |
| **PUT Features** | F | B | F | F | F | F | F | G | F | G | F | F | F |
| **POST Features** | F | - | F | F | - | F | F | - | - | - | F | F | F |
| **DEL Features** | F | B | F | F | F | F | F | B | G | B | F | F | F |
**Legend**

View File

@@ -8,6 +8,7 @@ For more information on how etcd can locate the cluster, see the [finding the cl
Please note - at least 3 nodes are required for [cluster availability][optimal-cluster-size].
[cluster-finding]: https://github.com/coreos/etcd/blob/master/Documentation/design/cluster-finding.md
[optimal-cluster-size]: https://github.com/coreos/etcd/blob/master/Documentation/optimal-cluster-size.md
## Using discovery.etcd.io
@@ -27,8 +28,8 @@ Here's a full example:
```
TOKEN=$(curl https://discovery.etcd.io/new)
./etcd -name instance1 -peer-addr 10.1.2.3:7001 -addr 10.1.2.3:4001 -discovery $TOKEN
./etcd -name instance2 -peer-addr 10.1.2.4:7002 -addr 10.1.2.4:4002 -discovery $TOKEN
./etcd -name instance3 -peer-addr 10.1.2.5:7002 -addr 10.1.2.5:4002 -discovery $TOKEN
./etcd -name instance2 -peer-addr 10.1.2.4:7001 -addr 10.1.2.4:4001 -discovery $TOKEN
./etcd -name instance3 -peer-addr 10.1.2.5:7001 -addr 10.1.2.5:4001 -discovery $TOKEN
```
## Running Your Own Discovery Endpoint
@@ -38,8 +39,8 @@ The discovery API communicates with a separate etcd cluster to store and retriev
```
TOKEN="testcluster"
./etcd -name instance1 -peer-addr 10.1.2.3:7001 -addr 10.1.2.3:4001 -discovery http://10.10.10.10:4001/v2/keys/$TOKEN
./etcd -name instance2 -peer-addr 10.1.2.4:7002 -addr 10.1.2.4:4002 -discovery http://10.10.10.10:4001/v2/keys/$TOKEN
./etcd -name instance3 -peer-addr 10.1.2.5:7002 -addr 10.1.2.5:4002 -discovery http://10.10.10.10:4001/v2/keys/$TOKEN
./etcd -name instance2 -peer-addr 10.1.2.4:7001 -addr 10.1.2.4:4001 -discovery http://10.10.10.10:4001/v2/keys/$TOKEN
./etcd -name instance3 -peer-addr 10.1.2.5:7001 -addr 10.1.2.5:4001 -discovery http://10.10.10.10:4001/v2/keys/$TOKEN
```
If you're interested in how to discovery API works behind the scenes, read about the [Discovery Protocol](https://github.com/coreos/etcd/blob/master/Documentation/discovery-protocol.md).
@@ -52,8 +53,6 @@ The Discovery API submits the `-peer-addr` of each etcd instance to the configur
The discovery API will automatically clean up the address of a stale peer that is no longer part of the cluster. The TTL for this process is a week, which should be long enough to handle any extremely long outage you may encounter. There is no harm in having stale peers in the list until they are cleaned up, since an etcd instance only needs to connect to one valid peer in the cluster to join.
[discovery-design]: https://github.com/coreos/etcd/blob/master/Documentation/design/cluster-finding.md
## Lifetime of a Discovery URL
A discovery URL identifies a single etcd cluster. Do not re-use discovery URLs for new clusters.

View File

@@ -1,6 +1,8 @@
# Etcd Configuration
Configuration options can be set in three places:
## Node Configuration
Individual node configuration options can be set in three places:
1. Command line flags
2. Environment variables
@@ -10,6 +12,16 @@ Options set on the command line take precedence over all other sources.
Options set in environment variables take precedence over options set in
configuration files.
## Cluster Configuration
Cluster-wide settings are configured via the `/config` admin endpoint and additionally in the configuration file. Values contained in the configuration file will seed the cluster setting with the provided value. After the cluster is running, only the admin endpoint is used.
The full documentation is contained in the [API docs](https://github.com/coreos/etcd/blob/master/Documentation/api.md#cluster-config).
* `activeSize` - the maximum number of peers that can participate in the consensus protocol. Other peers will join as standbys.
* `removeDelay` - the minimum time in seconds that a machine has been observed to be unresponsive before it is removed from the cluster.
* `syncInterval` - the amount of time in seconds between cluster sync when it runs in standby mode.
## Command Line Flags
### Required
@@ -20,6 +32,8 @@ configuration files.
* `-addr` - The advertised public hostname:port for client communication. Defaults to `127.0.0.1:4001`.
* `-discovery` - A URL to use for discovering the peer list. (i.e `"https://discovery.etcd.io/your-unique-key"`).
* `-http-read-timeout` - The number of seconds before an HTTP read operation is timed out.
* `-http-write-timeout` - The number of seconds before an HTTP write operation is timed out.
* `-bind-addr` - The listening hostname for client communication. Defaults to advertised IP.
* `-peers` - A comma separated list of peers in the cluster (i.e `"203.0.113.101:7001,203.0.113.102:7001"`).
* `-peers-file` - The file path containing a comma separated list of peers in the cluster.
@@ -31,7 +45,6 @@ configuration files.
* `-cpuprofile` - The path to a file to output CPU profile data. Enables CPU profiling when present.
* `-data-dir` - The directory to store log and snapshot. Defaults to the current working directory.
* `-max-result-buffer` - The max size of result buffer. Defaults to `1024`.
* `-max-cluster-size` - The max size of the cluster. Defaults to `9`.
* `-max-retry-attempts` - The max retry attempts when trying to join a cluster. Defaults to `3`.
* `-peer-addr` - The advertised public hostname:port for server communication. Defaults to `127.0.0.1:7001`.
* `-peer-bind-addr` - The listening hostname for server communication. Defaults to advertised IP.
@@ -41,6 +54,9 @@ configuration files.
* `-peer-election-timeout` - The number of milliseconds to wait before the leader is declared unhealthy.
* `-peer-heartbeat-interval` - The number of milliseconds in between heartbeat requests
* `-snapshot=false` - Disable log snapshots. Defaults to `true`.
* `-cluster-active-size` - The expected number of instances participating in the consensus protocol. Only applied if the etcd instance is the first peer in the cluster.
* `-cluster-remove-delay` - The delay before one node is removed from the cluster since it cannot be connected at all. Only applied if the etcd instance is the first peer in the cluster.
* `-cluster-sync-interval` - The interval between synchronization for standby-mode instance with the cluster. Only applied if the etcd instance is the first peer in the cluster.
* `-v` - Enable verbose logging. Defaults to `false`.
* `-vv` - Enable very verbose logging. Defaults to `false`.
* `-version` - Print the version and exit.
@@ -59,6 +75,8 @@ cors = []
cpu_profile_file = ""
data_dir = "."
discovery = "http://etcd.local:4001/v2/keys/_etcd/registry/examplecluster"
http_read_timeout = 10
http_write_timeout = 10
key_file = ""
peers = []
peers_file = ""
@@ -76,6 +94,11 @@ bind_addr = "127.0.0.1:7001"
ca_file = ""
cert_file = ""
key_file = ""
[cluster]
active_size = 9
remove_delay = 1800.0
sync_interval = 5.0
```
## Environment Variables
@@ -89,6 +112,8 @@ key_file = ""
* `ETCD_CPU_PROFILE_FILE`
* `ETCD_DATA_DIR`
* `ETCD_DISCOVERY`
* `ETCD_CLUSTER_HTTP_READ_TIMEOUT`
* `ETCD_CLUSTER_HTTP_WRITE_TIMEOUT`
* `ETCD_KEY_FILE`
* `ETCD_PEERS`
* `ETCD_PEERS_FILE`
@@ -105,3 +130,6 @@ key_file = ""
* `ETCD_PEER_CERT_FILE`
* `ETCD_PEER_KEY_FILE`
* `ETCD_PEER_ELECTION_TIMEOUT`
* `ETCD_CLUSTER_ACTIVE_SIZE`
* `ETCD_CLUSTER_REMOVE_DELAY`
* `ETCD_CLUSTER_SYNC_INTERVAL`

View File

@@ -17,7 +17,6 @@
**Python libraries**
- [transitorykris/etcd-py](https://github.com/transitorykris/etcd-py)
- [jplana/python-etcd](https://github.com/jplana/python-etcd) - Supports v2
- [russellhaering/txetcd](https://github.com/russellhaering/txetcd) - a Twisted Python library
- [cholcombe973/autodock](https://github.com/cholcombe973/autodock) - A docker deployment automation tool
@@ -90,3 +89,6 @@ A detailed recap of client functionalities can be found in the [clients compatib
- [configdb](https://git.autistici.org/ai/configdb/tree/master) - A REST relational abstraction on top of arbitrary database backends, aimed at storing configs and inventories.
- [scrz](https://github.com/scrz/scrz) - Container manager, stores configuration in etcd.
- [fleet](https://github.com/coreos/fleet) - Distributed init system
- [GoogleCloudPlatform/kubernetes](https://github.com/GoogleCloudPlatform/kubernetes) - Container cluster manager.
- [mailgun/vulcand](https://github.com/mailgun/vulcand) - HTTP proxy that uses etcd as a configuration backend.
- [duedil-ltd/discodns](https://github.com/duedil-ltd/discodns) - Simple DNS nameserver using etcd as a database for names and records.

View File

@@ -1,7 +1,7 @@
## Modules
etcd has a number of modules that are built on top of the core etcd API.
These modules provide things like dashboards, locks and leader election.
These modules provide things like dashboards, locks and leader election (removed).
**Warning**: Modules are deprecated from v0.4 until we have a solid base we can apply them back onto.
For now, we are choosing to focus on raft algorithm and core etcd to make sure that it works correctly and fast.
@@ -81,7 +81,7 @@ curl -X DELETE http://127.0.0.1:4001/mod/v2/lock/customer1?value=bar
```
### Leader Election
### Leader Election (Deprecated and Removed in 0.4)
The Leader Election module wraps the Lock module to allow clients to come to consensus on a single value.
This is useful when you want one server to process at a time but allow other servers to fail over.

View File

@@ -28,8 +28,8 @@ The other important cluster optimization is to always have an odd active cluster
|--------------|------------|-------------------|
| 1 peers | 1 peers | None |
| 3 peers | 2 peers | 1 peer |
| 4 peers | 3 peers | 2 peers |
| 5 peers | 3 peers | **3 peers** |
| 4 peers | 3 peers | 1 peer |
| 5 peers | 3 peers | **2 peers** |
| 6 peers | 4 peers | 2 peers |
| 7 peers | 4 peers | **3 peers** |
| 8 peers | 5 peers | 3 peers |

View File

@@ -1,4 +1,4 @@
ectd is being used successfully by many companies in production. It is,
etcd is being used successfully by many companies in production. It is,
however, under active development and systems like etcd are difficult to get
correct. If you are comfortable with bleeding-edge software please use etcd and
provide us with the feedback and testing young software needs.

5
NOTICE Normal file
View File

@@ -0,0 +1,5 @@
CoreOS Project
Copyright 2014 CoreOS, Inc
This product includes software developed at CoreOS, Inc.
(http://www.coreos.com/).

View File

@@ -1,6 +1,6 @@
# etcd
README version 0.4.3
README version 0.4.5
A highly-available key value store for shared configuration and service discovery.
etcd is inspired by [Apache ZooKeeper][zookeeper] and [doozer][doozer], with a focus on being:
@@ -95,7 +95,7 @@ You have successfully started an etcd on a single machine and written a key to t
## Contact
- Mailing list: [etcd-dev](https://groups.google.com/forum/?hl=en#!forum/etcd-dev)
- IRC: #[coreos](irc://irc.freenode.org:6667/#coreos) oon freenode.org
- IRC: #[coreos](irc://irc.freenode.org:6667/#coreos) on freenode.org
- Planning/Roadmap: [milestones](https://github.com/coreos/etcd/issues/milestones)
- Bugs: [issues](https://github.com/coreos/etcd/issues)

View File

@@ -36,7 +36,7 @@ var newFlagNameLookup = map[string]string{
"d": "data-dir",
"m": "max-result-buffer",
"r": "max-retry-attempts",
"maxsize": "max-cluster-size",
"maxsize": "cluster-active-size",
"clientCAFile": "ca-file",
"clientCert": "cert-file",
"clientKey": "key-file",
@@ -45,6 +45,7 @@ var newFlagNameLookup = map[string]string{
"serverKey": "peer-key-file",
"snapshotCount": "snapshot-count",
"peer-heartbeat-timeout": "peer-heartbeat-interval",
"max-cluster-size": "cluster-active-size",
}
// Config represents the server configuration.
@@ -61,6 +62,8 @@ type Config struct {
Discovery string `toml:"discovery" env:"ETCD_DISCOVERY"`
Force bool
KeyFile string `toml:"key_file" env:"ETCD_KEY_FILE"`
HTTPReadTimeout float64 `toml:"http_read_timeout" env:"ETCD_HTTP_READ_TIMEOUT"`
HTTPWriteTimeout float64 `toml:"http_write_timeout" env:"ETCD_HTTP_WRITE_TIMEOUT"`
Peers []string `toml:"peers" env:"ETCD_PEERS"`
PeersFile string `toml:"peers_file" env:"ETCD_PEERS_FILE"`
MaxResultBuffer int `toml:"max_result_buffer" env:"ETCD_MAX_RESULT_BUFFER"`
@@ -97,6 +100,8 @@ func New() *Config {
c := new(Config)
c.SystemPath = DefaultSystemConfigPath
c.Addr = "127.0.0.1:4001"
c.HTTPReadTimeout = server.DefaultReadTimeout
c.HTTPWriteTimeout = server.DefaultWriteTimeout
c.MaxResultBuffer = 1024
c.MaxRetryAttempts = 3
c.RetryInterval = 10.0
@@ -254,6 +259,9 @@ func (c *Config) LoadFlags(arguments []string) error {
f.StringVar(&c.Peer.CertFile, "peer-cert-file", c.Peer.CertFile, "")
f.StringVar(&c.Peer.KeyFile, "peer-key-file", c.Peer.KeyFile, "")
f.Float64Var(&c.HTTPReadTimeout, "http-read-timeout", c.HTTPReadTimeout, "")
f.Float64Var(&c.HTTPWriteTimeout, "http-write-timeout", c.HTTPReadTimeout, "")
f.StringVar(&c.DataDir, "data-dir", c.DataDir, "")
f.IntVar(&c.MaxResultBuffer, "max-result-buffer", c.MaxResultBuffer, "")
f.IntVar(&c.MaxRetryAttempts, "max-retry-attempts", c.MaxRetryAttempts, "")
@@ -297,6 +305,8 @@ func (c *Config) LoadFlags(arguments []string) error {
f.IntVar(&c.MaxRetryAttempts, "r", c.MaxRetryAttempts, "(deprecated)")
f.IntVar(&c.SnapshotCount, "snapshotCount", c.SnapshotCount, "(deprecated)")
f.IntVar(&c.Peer.HeartbeatInterval, "peer-heartbeat-timeout", c.Peer.HeartbeatInterval, "(deprecated)")
f.IntVar(&c.Cluster.ActiveSize, "max-cluster-size", c.Cluster.ActiveSize, "(deprecated)")
f.IntVar(&c.Cluster.ActiveSize, "maxsize", c.Cluster.ActiveSize, "(deprecated)")
// END DEPRECATED FLAGS
if err := f.Parse(arguments); err != nil {

View File

@@ -27,9 +27,11 @@ func TestConfigTOML(t *testing.T) {
max_result_buffer = 512
max_retry_attempts = 5
name = "test-name"
http_read_timeout = 2.34
snapshot = true
verbose = true
very_verbose = true
http_write_timeout = 1.23
[peer]
addr = "127.0.0.1:7002"
@@ -52,6 +54,8 @@ func TestConfigTOML(t *testing.T) {
assert.Equal(t, c.CorsOrigins, []string{"*"}, "")
assert.Equal(t, c.DataDir, "/tmp/data", "")
assert.Equal(t, c.Discovery, "http://example.com/foobar", "")
assert.Equal(t, c.HTTPReadTimeout, 2.34, "")
assert.Equal(t, c.HTTPWriteTimeout, 1.23, "")
assert.Equal(t, c.KeyFile, "/tmp/file.key", "")
assert.Equal(t, c.BindAddr, "127.0.0.1:4003", "")
assert.Equal(t, c.Peers, []string{"coreos.com:4001", "coreos.com:4002"}, "")
@@ -80,6 +84,8 @@ func TestConfigEnv(t *testing.T) {
os.Setenv("ETCD_CORS", "localhost:4001,localhost:4002")
os.Setenv("ETCD_DATA_DIR", "/tmp/data")
os.Setenv("ETCD_DISCOVERY", "http://example.com/foobar")
os.Setenv("ETCD_HTTP_READ_TIMEOUT", "2.34")
os.Setenv("ETCD_HTTP_WRITE_TIMEOUT", "1.23")
os.Setenv("ETCD_KEY_FILE", "/tmp/file.key")
os.Setenv("ETCD_BIND_ADDR", "127.0.0.1:4003")
os.Setenv("ETCD_PEERS", "coreos.com:4001,coreos.com:4002")
@@ -107,6 +113,8 @@ func TestConfigEnv(t *testing.T) {
assert.Equal(t, c.CorsOrigins, []string{"localhost:4001", "localhost:4002"}, "")
assert.Equal(t, c.DataDir, "/tmp/data", "")
assert.Equal(t, c.Discovery, "http://example.com/foobar", "")
assert.Equal(t, c.HTTPReadTimeout, 2.34, "")
assert.Equal(t, c.HTTPWriteTimeout, 1.23, "")
assert.Equal(t, c.KeyFile, "/tmp/file.key", "")
assert.Equal(t, c.BindAddr, "127.0.0.1:4003", "")
assert.Equal(t, c.Peers, []string{"coreos.com:4001", "coreos.com:4002"}, "")
@@ -553,19 +561,12 @@ func TestConfigClusterRemoveDelayFlag(t *testing.T) {
assert.Equal(t, c.Cluster.RemoveDelay, 100.0, "")
}
// Ensures that the cluster sync interval can be parsed from the environment.
func TestConfigClusterSyncIntervalEnv(t *testing.T) {
withEnv("ETCD_CLUSTER_SYNC_INTERVAL", "10", func(c *Config) {
assert.Nil(t, c.LoadEnv(), "")
assert.Equal(t, c.Cluster.SyncInterval, 10.0, "")
})
}
// Ensures that the cluster sync interval flag can be parsed.
func TestConfigClusterSyncIntervalFlag(t *testing.T) {
c := New()
assert.Nil(t, c.LoadFlags([]string{"-cluster-sync-interval", "10"}), "")
assert.Equal(t, c.Cluster.SyncInterval, 10.0, "")
assert.Nil(t, c.LoadFlags([]string{"-http-read-timeout", "2.34"}), "")
assert.Equal(t, c.HTTPReadTimeout, 2.34, "")
assert.Nil(t, c.LoadFlags([]string{"-http-write-timeout", "1.23"}), "")
assert.Equal(t, c.HTTPWriteTimeout, 1.23, "")
}
// Ensures that a system config field is overridden by a custom config field.

View File

@@ -260,10 +260,19 @@ func (e *Etcd) Run() {
log.Infof("etcd server [name %s, listen on %s, advertised url %s]", e.Server.Name, e.Config.BindAddr, e.Server.URL())
listener := server.NewListener(e.Config.EtcdTLSInfo().Scheme(), e.Config.BindAddr, etcdTLSConfig)
e.server = &http.Server{Handler: &ModeHandler{e, serverHTTPHandler, standbyServerHTTPHandler}}
e.server = &http.Server{Handler: &ModeHandler{e, serverHTTPHandler, standbyServerHTTPHandler},
ReadTimeout: time.Duration(e.Config.HTTPReadTimeout) * time.Second,
WriteTimeout: time.Duration(e.Config.HTTPWriteTimeout) * time.Second,
}
log.Infof("peer server [name %s, listen on %s, advertised url %s]", e.PeerServer.Config.Name, e.Config.Peer.BindAddr, e.PeerServer.Config.URL)
peerListener := server.NewListener(e.Config.PeerTLSInfo().Scheme(), e.Config.Peer.BindAddr, peerTLSConfig)
e.peerServer = &http.Server{Handler: &ModeHandler{e, peerServerHTTPHandler, http.NotFoundHandler()}}
e.peerServer = &http.Server{Handler: &ModeHandler{e, peerServerHTTPHandler, http.NotFoundHandler()},
ReadTimeout: time.Duration(server.DefaultReadTimeout) * time.Second,
WriteTimeout: time.Duration(server.DefaultWriteTimeout) * time.Second,
}
wg := sync.WaitGroup{}
wg.Add(2)

View File

@@ -1,85 +0,0 @@
package leader
import (
"fmt"
"testing"
"time"
"github.com/coreos/etcd/server"
"github.com/coreos/etcd/tests"
"github.com/coreos/etcd/third_party/github.com/stretchr/testify/assert"
)
// Ensure that a leader can be set and read.
func TestModLeaderSet(t *testing.T) {
tests.RunServer(func(s *server.Server) {
// Set leader.
body, status, err := testSetLeader(s, "foo", "xxx", 10)
assert.NoError(t, err)
assert.Equal(t, status, 200)
assert.Equal(t, body, "2")
// Check that the leader is set.
body, status, err = testGetLeader(s, "foo")
assert.NoError(t, err)
assert.Equal(t, status, 200)
assert.Equal(t, body, "xxx")
// Delete leader.
body, status, err = testDeleteLeader(s, "foo", "xxx")
assert.NoError(t, err)
assert.Equal(t, status, 200)
assert.Equal(t, body, "")
// Check that the leader is removed.
body, status, err = testGetLeader(s, "foo")
assert.NoError(t, err)
assert.Equal(t, status, 200)
assert.Equal(t, body, "")
})
}
// Ensure that a leader can be renewed.
func TestModLeaderRenew(t *testing.T) {
tests.RunServer(func(s *server.Server) {
// Set leader.
body, status, err := testSetLeader(s, "foo", "xxx", 2)
assert.NoError(t, err)
assert.Equal(t, status, 200)
assert.Equal(t, body, "2")
time.Sleep(1 * time.Second)
// Renew leader.
body, status, err = testSetLeader(s, "foo", "xxx", 3)
assert.NoError(t, err)
assert.Equal(t, status, 200)
assert.Equal(t, body, "2")
time.Sleep(2 * time.Second)
// Check that the leader is set.
body, status, err = testGetLeader(s, "foo")
assert.NoError(t, err)
assert.Equal(t, status, 200)
assert.Equal(t, body, "xxx")
})
}
func testSetLeader(s *server.Server, key string, name string, ttl int) (string, int, error) {
resp, err := tests.PutForm(fmt.Sprintf("%s/mod/v2/leader/%s?name=%s&ttl=%d", s.URL(), key, name, ttl), nil)
ret := tests.ReadBody(resp)
return string(ret), resp.StatusCode, err
}
func testGetLeader(s *server.Server, key string) (string, int, error) {
resp, err := tests.Get(fmt.Sprintf("%s/mod/v2/leader/%s", s.URL(), key))
ret := tests.ReadBody(resp)
return string(ret), resp.StatusCode, err
}
func testDeleteLeader(s *server.Server, key string, name string) (string, int, error) {
resp, err := tests.DeleteForm(fmt.Sprintf("%s/mod/v2/leader/%s?name=%s", s.URL(), key, name), nil)
ret := tests.ReadBody(resp)
return string(ret), resp.StatusCode, err
}

View File

@@ -25,7 +25,6 @@ const (
)
// ClusterConfig represents cluster-wide configuration settings.
// These settings can only be changed through Raft.
type ClusterConfig struct {
// ActiveSize is the maximum number of node that can join as Raft followers.
// Nodes that join the cluster after the limit is reached are standbys.

View File

@@ -3,10 +3,16 @@ package server
import (
"crypto/tls"
"net"
"time"
"github.com/coreos/etcd/log"
)
const (
DefaultReadTimeout = float64((5 * time.Minute) / time.Second)
DefaultWriteTimeout = float64((5 * time.Minute) / time.Second)
)
// TLSServerConfig generates tls configuration based on TLSInfo
// If any error happens, this function will call log.Fatal
func TLSServerConfig(info *TLSInfo) *tls.Config {

View File

@@ -853,7 +853,7 @@ func (s *PeerServer) monitorActiveSize() {
// If we have more active nodes than we should then remove.
if peerCount > activeSize {
peer := peers[rand.Intn(len(peers))]
log.Infof("%s: removing: %v", s.Config.Name, peer)
log.Infof("%s: removing node: %v; peer number %d > expected size %d", s.Config.Name, peer, peerCount, activeSize)
if _, err := s.raftServer.Do(&RemoveCommand{Name: peer}); err != nil {
log.Infof("%s: warning: remove error: %v", s.Config.Name, err)
}

View File

@@ -188,6 +188,7 @@ func (ps *PeerServer) RemoveHttpHandler(w http.ResponseWriter, req *http.Request
// Returns a JSON-encoded cluster configuration.
func (ps *PeerServer) getClusterConfigHttpHandler(w http.ResponseWriter, req *http.Request) {
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(ps.ClusterConfig())
}
@@ -217,6 +218,7 @@ func (ps *PeerServer) setClusterConfigHttpHandler(w http.ResponseWriter, req *ht
log.Debugf("[recv] Update Cluster Config Request")
ps.server.Dispatch(c, w, req)
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(ps.ClusterConfig())
}
@@ -230,6 +232,7 @@ func (ps *PeerServer) getMachinesHttpHandler(w http.ResponseWriter, req *http.Re
}
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(&machines)
}
@@ -237,6 +240,7 @@ func (ps *PeerServer) getMachinesHttpHandler(w http.ResponseWriter, req *http.Re
func (ps *PeerServer) getMachineHttpHandler(w http.ResponseWriter, req *http.Request) {
vars := mux.Vars(req)
m := ps.getMachineMessage(vars["name"], ps.raftServer.Leader())
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(m)
}

View File

@@ -1,3 +1,3 @@
package server
const ReleaseVersion = "0.4.3"
const ReleaseVersion = "0.4.5"

View File

@@ -311,6 +311,7 @@ func (s *Server) GetPeersHandler(w http.ResponseWriter, req *http.Request) error
// Retrieves stats on the Raft server.
func (s *Server) GetStatsHandler(w http.ResponseWriter, req *http.Request) error {
w.Header().Set("Content-Type", "application/json")
w.Write(s.peerServer.Stats())
return nil
}
@@ -318,6 +319,7 @@ func (s *Server) GetStatsHandler(w http.ResponseWriter, req *http.Request) error
// Retrieves stats on the leader.
func (s *Server) GetLeaderStatsHandler(w http.ResponseWriter, req *http.Request) error {
if s.peerServer.RaftServer().State() == raft.Leader {
w.Header().Set("Content-Type", "application/json")
w.Write(s.peerServer.PeerStats())
return nil
}
@@ -333,6 +335,7 @@ func (s *Server) GetLeaderStatsHandler(w http.ResponseWriter, req *http.Request)
// Retrieves stats on the leader.
func (s *Server) GetStoreStatsHandler(w http.ResponseWriter, req *http.Request) error {
w.Header().Set("Content-Type", "application/json")
w.Write(s.store.JsonStats())
return nil
}

View File

@@ -53,9 +53,11 @@ Other Options:
-max-result-buffer Max size of the result buffer.
-max-retry-attempts Number of times a node will try to join a cluster.
-retry-interval Seconds to wait between cluster join retry attempts.
-max-cluster-size Maximum number of nodes in the cluster.
-snapshot=false Disable log snapshots
-snapshot-count Number of transactions before issuing a snapshot.
-cluster-active-size Number of active nodes in the cluster.
-cluster-remove-delay Seconds before one node is removed.
-cluster-sync-interval Seconds between synchronizations for standby mode.
`
// Usage returns the usage message for etcd.

View File

@@ -68,6 +68,7 @@ func handleWatch(key string, recursive, stream bool, waitIndex string, w http.Re
closeChan := cn.CloseNotify()
writeHeaders(w, s)
w.(http.Flusher).Flush()
if stream {
// watcher hub will not help to remove stream watcher

View File

@@ -25,6 +25,7 @@ func TestClusterConfigSet(t *testing.T) {
resp, _ = tests.Get("http://localhost:7002/v2/admin/config")
body := tests.ReadBodyJSON(resp)
assert.Equal(t, resp.StatusCode, 200)
assert.Equal(t, resp.Header.Get("Content-Type"), "application/json")
assert.Equal(t, body["activeSize"], 3)
assert.Equal(t, body["removeDelay"], 60)
}
@@ -44,6 +45,7 @@ func TestClusterConfigReload(t *testing.T) {
resp, _ = tests.Get("http://localhost:7002/v2/admin/config")
body := tests.ReadBodyJSON(resp)
assert.Equal(t, resp.StatusCode, 200)
assert.Equal(t, resp.Header.Get("Content-Type"), "application/json")
assert.Equal(t, body["activeSize"], 3)
assert.Equal(t, body["removeDelay"], 60)
@@ -59,6 +61,7 @@ func TestClusterConfigReload(t *testing.T) {
resp, _ = tests.Get("http://localhost:7002/v2/admin/config")
body = tests.ReadBodyJSON(resp)
assert.Equal(t, resp.StatusCode, 200)
assert.Equal(t, resp.Header.Get("Content-Type"), "application/json")
assert.Equal(t, body["activeSize"], 3)
assert.Equal(t, body["removeDelay"], 60)
}
@@ -76,6 +79,7 @@ func TestGetMachines(t *testing.T) {
t.FailNow()
}
assert.Equal(t, resp.StatusCode, 200)
assert.Equal(t, resp.Header.Get("Content-Type"), "application/json")
machines := make([]map[string]interface{}, 0)
b := tests.ReadBody(resp)
json.Unmarshal(b, &machines)