vitalif
/
etcd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

*: eschew you from documentation 

Removed line wrapping in affected files as well.
release-3.2
Anthony Romano 2017-03-03 15:49:40 -08:00
parent 4e1ce81e17
commit c8a2c7f64f
15 changed files with 80 additions and 172 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.github/ISSUE_TEMPLATE.md vendored
View File

@ -1,7 +1,6 @@
# Bug reporting
A good bug report has some very specific qualities, so please read over our short document on
[reporting bugs][report_bugs] before you submit your bug report.
A good bug report has some very specific qualities, so please read over our short document on [reporting bugs][report_bugs] before submitting a bug report.
To ask a question, go ahead and ignore this.

26
CONTRIBUTING.md
View File

@ -1,6 +1,6 @@
# How to contribute
etcd is Apache 2.0 licensed and accepts contributions via GitHub pull requests. This document outlines some of the conventions on commit message formatting, contact points for developers and other resources to make getting your contribution into etcd easier.
etcd is Apache 2.0 licensed and accepts contributions via GitHub pull requests. This document outlines some of the conventions on commit message formatting, contact points for developers, and other resources to help get contributions into etcd.
# Email and chat
@ -14,24 +14,20 @@ etcd is Apache 2.0 licensed and accepts contributions via GitHub pull requests.
## Reporting bugs and creating issues
Reporting bugs is one of the best ways to contribute. However, a good bug report
has some very specific qualities, so please read over our short document on
[reporting bugs](https://github.com/coreos/etcd/blob/master/Documentation/reporting_bugs.md)
before you submit your bug report. This document might contain links known
issues, another good reason to take a look there, before reporting your bug.
Reporting bugs is one of the best ways to contribute. However, a good bug report has some very specific qualities, so please read over our short document on [reporting bugs](https://github.com/coreos/etcd/blob/master/Documentation/reporting_bugs.md) before submitting a bug report. This document might contain links to known issues, another good reason to take a look there before reporting a bug.
## Contribution flow
This is a rough outline of what a contributor's workflow looks like:
- Create a topic branch from where you want to base your work. This is usually master.
- Create a topic branch from where to base the contribution. This is usually master.
- Make commits of logical units.
- Make sure your commit messages are in the proper format (see below).
- Push your changes to a topic branch in your fork of the repository.
- Make sure commit messages are in the proper format (see below).
- Push changes in a topic branch to a personal fork of the repository.
- Submit a pull request to coreos/etcd.
- Your PR must receive a LGTM from two maintainers found in the MAINTAINERS file.
- The PR must receive a LGTM from two maintainers found in the MAINTAINERS file.
Thanks for your contributions!
Thanks for contributing!
### Code style
@ -48,8 +44,7 @@ the body of the commit should describe the why.
```
scripts: add the test-cluster command
this uses tmux to setup a test cluster that you can easily kill and
start for debugging.
this uses tmux to setup a test cluster that can easily be killed and started for debugging.
Fixes #38
```
@ -64,7 +59,4 @@ The format can be described more formally as follows:
<footer>
```
The first line is the subject and should be no longer than 70 characters, the
second line is always blank, and other lines should be wrapped at 80 characters.
This allows the message to be easier to read on GitHub as well as in various
git tools.
The first line is the subject and should be no longer than 70 characters, the second line is always blank, and other lines should be wrapped at 80 characters. This allows the message to be easier to read on GitHub as well as in various git tools.

2
Documentation/benchmarks/etcd-2-2-0-rc-memory-benchmarks.md
View File

@ -39,7 +39,7 @@ The length of key name is always 64 bytes, which is a reasonable length of avera
## Data Size Threshold
- When etcd reaches data size threshold, it may trigger leader election easily and drop part of proposals.
- At most cases, etcd cluster should work smoothly if it doesn't hit the threshold. If it doesn't work well due to insufficient resources, you need to decrease its data size.
- For most cases, the etcd cluster should work smoothly if it doesn't hit the threshold. If it doesn't work well due to insufficient resources, decrease its data size.
| value bytes | key number limitation | suggested data size threshold(MB) | consumed RSS(MB) |
|-------------|-----------------------|-----------------------------------|------------------|

2
Documentation/op-guide/container.md
View File

@ -130,7 +130,7 @@ docker exec etcd /bin/sh -c "export ETCDCTL_API=3 && /usr/local/bin/etcdctl put
## Bare Metal
To provision a 3 node etcd cluster on bare-metal, you might find the examples in the [baremetal repo](https://github.com/coreos/coreos-baremetal/tree/master/examples) useful.
To provision a 3 node etcd cluster on bare-metal, the examples in the [baremetal repo](https://github.com/coreos/coreos-baremetal/tree/master/examples) may be useful.
## Mounting a certificate volume

38
Documentation/platforms/freebsd.md
View File

@ -1,23 +1,18 @@
# FreeBSD
Starting with version 0.1.2 both etcd and etcdctl have been ported to FreeBSD and can
be installed either via packages or ports system. Their versions have been recently
updated to 0.2.0 so now you can enjoy using etcd and etcdctl on FreeBSD 10.0 (RC4 as
of now) and 9.x where they have been tested. They might also work when installed from
ports on earlier versions of FreeBSD, but your mileage may vary.
Starting with version 0.1.2 both etcd and etcdctl have been ported to FreeBSD and can be installed either via packages or ports system. Their versions have been recently updated to 0.2.0 so now etcd and etcdctl can be enjoyed on FreeBSD 10.0 (RC4 as of now) and 9.x, where they have been tested. They might also work when installed from ports on earlier versions of FreeBSD, but it is untested; caveat emptor.
## Installation
### Using pkgng package system
1. If you do not have pkg­ng installed, install it with command `pkg` and answering 'Y'
when asked
1. If pkg­ng is not installed, install it with command `pkg` and answering 'Y' when asked.
2. Update your repository data with `pkg update`
2. Update the repository data with `pkg update`.
3. Install etcd with `pkg install coreos-etcd coreos-etcdctl`
3. Install etcd with `pkg install coreos-etcd coreos-etcdctl`.
4. Verify successful installation with `pkg info | grep etcd` and you should get:
4. Verify successful installation by confirming `pkg info | grep etcd` matches:
```
r@fbsd­10:/ # pkg info | grep etcd
@ -26,21 +21,17 @@ coreos­etcdctl­0.2.0           Simple commandline client for et
r@fbsd­10:/ #
```
5. You’re ready to use etcd and etcdctl! For more information about using pkgng, please
see: http://www.freebsd.org/doc/handbook/pkgng­intro.html
5. etcd and etcdctl are ready to use! For more information about using pkgng, please see: http://www.freebsd.org/doc/handbook/pkgng­intro.html
 
### Using ports system
1. If you do not have ports installed, install with with `portsnap fetch extract` (it
may take some time depending on your hardware and network connection)
1. If ports is not installed, install with `portsnap fetch extract` (it may take some time depending on hardware and network connection).
2. Build etcd with `cd /usr/ports/devel/etcd && make install clean`, you
will get an option to build and install documentation and etcdctl with it.
2. Build etcd with `cd /usr/ports/devel/etcd && make install clean`. There will be an option to build and install documentation and etcdctl with it.
3. If you haven't installed it with etcdctl, and you would like to install it later, you can build it
with `cd /usr/ports/devel/etcdctl && make install clean`
3. If etcd wasn't installed with etcdctl, it can be built later with `cd /usr/ports/devel/etcdctl && make install clean`.
4. Verify successful installation with `pkg info | grep etcd` and you should get:
4. Verify successful installation by confirming `pkg info | grep etcd` matches:
 
```
@ -50,13 +41,8 @@ coreos­etcdctl­0.2.0           Simple commandline client for et
r@fbsd­10:/ #
```
5. You’re ready to use etcd and etcdctl! For more information about using ports system,
please see: https://www.freebsd.org/doc/handbook/ports­using.html
5. etcd and etcdctl are ready to use! For more information about using ports system, please see: https://www.freebsd.org/doc/handbook/ports­using.html
## Issues
If you find any issues with the build/install procedure or you've found a problem that
you've verified is local to FreeBSD version only (for example, by not being able to
reproduce it on any other platform, like OSX or Linux), please sent a
problem report using this page for more
information: http://www.freebsd.org/send­pr.html
If there are any issues with the build/install procedure or there's a problem that is local to FreeBSD only (for example, by not being able to reproduce it on any other platform, like OSX or Linux), please send a problem report using this page for more information: http://www.freebsd.org/send­pr.html

2
Documentation/production-users.md
View File

@ -7,7 +7,7 @@ This document tracks people and use cases for etcd in production. By creating a
- *Application*: https://kubernetes.io/
- *Environments*: AWS, OpenStack, Azure, Google Cloud, Bare Metal, etc
**This is a meta user if you want to document your specific Kubernetes cluster, please do!**
**This is a meta user; please feel free to document specific Kubernetes clusters!**
All Kubernetes clusters use etcd as their primary data store. This means etcd's users include such companies as [Niantic, Inc Pokemon Go](https://cloudplatform.googleblog.com/2016/09/bringing-Pokemon-GO-to-life-on-Google-Cloud.html), [Box](https://blog.box.com/blog/kubernetes-box-microservices-maximum-velocity/), [CoreOS](https://coreos.com/tectonic), [Ticketmaster](https://www.youtube.com/watch?v=wqXVKneP0Hg), and many many more.

15
Documentation/upgrades/upgrade_3_0.md
View File

@ -118,20 +118,11 @@ $ ETCDCTL_API=3 etcdctl endpoint health
## Further considerations
- etcdctl environment variables have been updated. If you find that
`ETCDCTL_API=2 etcdctl cluster-health` works properly but `ETCDCTL_API=3
etcdctl endpoints health` responds with `Error: grpc: timed out when
dialing`, make sure you are using the [new variable
names](https://github.com/coreos/etcd/tree/master/etcdctl#etcdctl).
- etcdctl environment variables have been updated. If `ETCDCTL_API=2 etcdctl cluster-health` works properly but `ETCDCTL_API=3 etcdctl endpoints health` responds with `Error: grpc: timed out when dialing`, be sure to use the [new variable names](https://github.com/coreos/etcd/tree/master/etcdctl#etcdctl).
## Known Issues
- etcd &lt; v3.1 does not work properly if built with Go &gt; v1.7. See [Issue
6951](https://github.com/coreos/etcd/issues/6951) for additional
information.
- If you see an error such as `transport: http2Client.notifyError got notified
that the client transport was broken unexpected EOF.` showing up in your
logs, make sure you are using a pre-built release, or building with (etcd
v3.1+ &amp; go v1.7+) or (etcd &lt;v3.1 &amp; go v1.6.x).
- etcd &lt; v3.1 does not work properly if built with Go &gt; v1.7. See [Issue 6951](https://github.com/coreos/etcd/issues/6951) for additional information.
- If an error such as `transport: http2Client.notifyError got notified that the client transport was broken unexpected EOF.` shows up in the etcd server logs, be sure etcd is a pre-built release or built with (etcd v3.1+ &amp; go v1.7+) or (etcd &lt;v3.1 &amp; go v1.6.x).
[etcd-contact]: https://groups.google.com/forum/#!forum/etcd-dev

9
README.md
View File

@ -39,10 +39,7 @@ See [etcdctl][etcdctl] for a simple command line client.
The easiest way to get etcd is to use one of the pre-built release binaries which are available for OSX, Linux, Windows, [rkt][rkt], and Docker. Instructions for using these binaries are on the [GitHub releases page][github-release].
For those wanting to try the very latest version, you can [build the latest version of etcd][dl-build] from the `master` branch.
You will first need [*Go*](https://golang.org/) installed on your machine (version 1.7+ is required).
All development occurs on `master`, including new features and bug fixes.
Bug fixes are first targeted at `master` and subsequently ported to release branches, as described in the [branch management][branch-management] guide.
For those wanting to try the very latest version, [build the latest version of etcd][dl-build] from the `master` branch. This first needs [*Go*](https://golang.org/) installed (version 1.7+ is required). All development occurs on `master`, including new features and bug fixes. Bug fixes are first targeted at `master` and subsequently ported to release branches, as described in the [branch management][branch-management] guide.
[rkt]: https://github.com/coreos/rkt/releases/
[github-release]: https://github.com/coreos/etcd/releases/
@ -96,7 +93,7 @@ Every cluster member and proxy accepts key value reads and key value writes.
### Running etcd on Kubernetes
If you want to run etcd cluster on Kubernetes, try [etcd operator](https://github.com/coreos/etcd-operator).
To run an etcd cluster on Kubernetes, try [etcd operator](https://github.com/coreos/etcd-operator).
### Next steps
@ -131,7 +128,7 @@ See [CONTRIBUTING](CONTRIBUTING.md) for details on submitting patches and the co
## Reporting bugs
See [reporting bugs](Documentation/reporting_bugs.md) for details about reporting any issue you may encounter.
See [reporting bugs](Documentation/reporting_bugs.md) for details about reporting any issues.
### License

4
contrib/raftexample/README.md
View File

@ -33,7 +33,7 @@ curl -L http://127.0.0.1:12380/my-key
First install [goreman](https://github.com/mattn/goreman), which manages Procfile-based applications.
The [Procfile script](./Procfile) will set up a local example cluster. You can start it with:
The [Procfile script](./Procfile) will set up a local example cluster. Start it with:
```sh
goreman start
@ -41,7 +41,7 @@ goreman start
This will bring up three raftexample instances.
You can write a key-value pair to any member of the cluster and likewise retrieve it from any member.
Now it's possible to write a key-value pair to any member of the cluster and likewise retrieve it from any member.
### Fault Tolerance

42
contrib/systemd/etcd2-backup-coreos/README.md
View File

@ -10,7 +10,7 @@ Remote backup and multi-node restore services for etcd2 clusters on CoreOS Linux
## Configuration
Before installing etcd2-backup, you need to configure `30-etcd2-backup-restore.conf`.
Before installing etcd2-backup, configure `30-etcd2-backup-restore.conf`:
```
[Service]
@ -18,7 +18,7 @@ Environment="ETCD_RESTORE_MASTER_ADV_PEER_URLS=<http://host:port>"
Environment="RCLONE_ENDPOINT=remote-name:path/to/backups"
```
Assuming you're deploying to CoreOS with etcd2, you should only need to change
Assuming a deployment to CoreOS with etcd2, only change:
* `ETCD_RESTORE_MASTER_ADV_PEER_URLS`
This is the new advertised peer url of the new etcd2 node that will be the founding member of the new restored cluster. We will call this node the **founding member**.
@ -31,13 +31,13 @@ Assuming you're deploying to CoreOS with etcd2, you should only need to change
* `./rclone.conf`
The rclone configuration file which will be installed. Must list a `[section]` which matches `RCLONE_ENDPOINT`'s remote-name component.
An easy way to generate this config file is to [install rclone](http://rclone.org/install/) on your local machine. Then follow the [configuration instructions](http://rclone.org/docs/) to generate an `rclone.conf` file.
An easy way to generate this config file is to [install rclone](http://rclone.org/install/) on a local machine. Then follow the [configuration instructions](http://rclone.org/docs/) to generate an `rclone.conf` file.
If you want to adjust backup frequency, edit `./etcd2-backup.timer`
To adjust backup frequency, edit `./etcd2-backup.timer`
## Installation
Once you've got those things configured, you can run `./build`.
Once those things are configured, run `./build`.
The `build` script generates a tarball for copying to CoreOS instances. The tarball contains the `etcd2-backup-install` script.
@ -54,7 +54,7 @@ After extracting the contents of the tar file and running the install script, th
## Recovery
This assumes that your cluster has lost quorum, and is not recoverable. Otherwise you should probably try to heal your cluster first.
This assumes that the cluster has lost quorum and is not recoverable. Otherwise try to heal the cluster first.
### Backup Freshness
@ -76,7 +76,7 @@ max-missed-seconds= (10000 transactions / (1000 transactions / second)) + 30 sec
3. Restore the rest of the cluster **one at a time**. Start `etcd2-join.service`, and then, if successful, `etcd2.service`. Please verify with `etcdctl cluster-health` that the expected set of nodes is present and healthy after each node joins.
4. Verify that your data is sane (enough). If so, kick off `etcd2-backup.timer` on all nodes and, hopefully, go back to bed.
4. Verify that the data is sane (enough). If so, kick off `etcd2-backup.timer` on all nodes and, hopefully, go back to bed.
## Retroactively change the founding member
@ -138,13 +138,13 @@ e1 $ exit
Now `e1`'s etcd data will be backed up to `s3://etcd2-backup-bucket/backups/<e1-machine-id>/` according to the schedule described in `etcd2-backup.timer`.
You should repeat the process for `e2` and `e3`. If you do not want a node to generate backups, omit enabling and starting `etcd2-backup.timer`.
Repeat the process for `e2` and `e3`. To stop a node from generating backups, omit enabling and starting `etcd2-backup.timer`.
## Restore the cluster
Let's assume that a mischievous friend decided it would be a good idea to corrupt the etcd2 data-dir on ALL of your nodes (`e1`,`e2`,`e3`). You simply want to restore the cluster from `e1`'s backup.
Let's assume that a mischievous friend decided it would be a good idea to corrupt the etcd2 data-dir on ALL of the nodes (`e1`,`e2`,`e3`). Simply restore the cluster from `e1`'s backup.
Here's how you would recover:
Here's how to recover:
```sh
# First, ENSURE etcd2 and etcd2-backup are not running on any nodes
@ -160,15 +160,15 @@ for node in e{2..3};do
done
```
After e2 and e3 finish catching up, your cluster should be back to normal.
After e2 and e3 finish catching up, the cluster should be back to normal.
## Migrate the cluster
The same friend who corrupted your etcd2 data-dirs decided that you have not had enough fun. This time, your friend dumps coffee on the machines hosting `e1`, `e2` and `e3`. There is a horrible smell, and the machines are dead.
The same friend who corrupted the etcd2 data-dirs decided that to have more fun. This time, the friend dumps coffee on the machines hosting `e1`, `e2` and `e3`. There is a horrible smell, and the machines are dead.
Luckily, you have a new 3-node etcd2 cluster ready to go, along with the S3 backup for `e1` from your old cluster.
Luckily, there's a new 3-node etcd2 cluster ready to go, along with the S3 backup for `e1` from the old cluster.
The new cluster configuration looks like this. Assume that etcd2-backup is not installed. (If it is, you NEED to make sure it's not running on any nodes)
The new cluster configuration looks like this. Assume that etcd2-backup is not installed. (If it is, make sure it's not running on any nodes)
| ETCD_NAME | ETCD_ADVERTISED_PEER_URL |
| ------------- |:-------------:|
@ -176,11 +176,11 @@ The new cluster configuration looks like this. Assume that etcd2-backup is not i
| q2 | http://172.17.8.202:2379 |
| q3 | http://172.17.8.203:2379 |
We will assume `q1` is the chosen founding member, though you can pick any node you like.
We will assume `q1` is the chosen founding member, though picking any node is fine.
## Migrate the remote backup
First, you need to copy your backup from `e1`'s backup folder to `q1`'s backup folder. I will show the S3 example.
First, copy the backup from `e1`'s backup folder to `q1`'s backup folder. I will show the S3 example.
```sh
# Make sure to remove q1's backup directory, if it exists already
@ -196,7 +196,7 @@ Environment="ETCD_RESTORE_MASTER_ADV_PEER_URLS=http://172.17.8.201:2379"
Environment="RCLONE_ENDPOINT=s3-testing-conf:s3://etcd2-backup-bucket/backups"
```
Since this is a new cluster, each new node will have new `machine-id` and will not clobber your backups from the old cluster, even though `RCLONE_ENDPOINT` is the same for both the old `e` cluster and the new `q` cluster.
Since this is a new cluster, each new node will have a new `machine-id` and will not clobber the backups from the old cluster, even though `RCLONE_ENDPOINT` is the same for both the old `e` cluster and the new `q` cluster.
## Installation
@ -234,18 +234,18 @@ for node in q{2..3};do
done
```
Once you've verifed the cluster has migrated properly, start and enable `etcd2-backup.timer` on at least one node.
After confirming the cluster has migrated properly, start and enable `etcd2-backup.timer` on at least one node.
```sh
ssh core@q1 "sudo systemctl enable etcd2-backup.service && sudo systemctl start etcd2-backup.service"
```
You should now have periodic backups going to: `s3://etcd2-backup-bucket/backups/<q1-machine-id>`
There should now be periodic backups going to: `s3://etcd2-backup-bucket/backups/<q1-machine-id>`
## Words of caution
1. Notice the `sleep 10` commands that follow starting `etcd2-join.service` and then `etcd2.service`. This sleep is there to allow the member that joined to cluster time to catch up on the cluster state before we attempt to add the next member. This involves sending the entire snapshot over the network. If you're dataset is large, or the network between nodes is slow, or your disks are already bogged down, etc- you may need to turn the sleep time up.
1. Notice the `sleep 10` commands that follow starting `etcd2-join.service` and then `etcd2.service`. This sleep is there to allow the member that joined to cluster time to catch up on the cluster state before we attempt to add the next member. This involves sending the entire snapshot over the network. If the dataset is large, the network between nodes is slow, disks are already bogged down, or the system is otherwise overutilized, try increasing the sleep.
In the case of large data sets, it is recommended that you copy the data directory produced by `etcd2-restore` on the founding member to the other nodes before running `etcd2-join` on them. This will avoid etcd transferring the entire snapshot to every node after it joins the cluster.
In the case of large data sets, it is recommended to copy the data directory produced by `etcd2-restore` on the founding member to the other nodes before running `etcd2-join` on them. This will avoid etcd transferring the entire snapshot to every node after it joins the cluster.
2. It is not recommended clients be allowed to access the etcd2 cluster **until** all members have been added and finished catching up.

13
etcdctl/READMEv2.md
View File

@ -8,7 +8,7 @@ It can be used in scripts or for administrators to explore an etcd cluster.
The latest release is available as a binary at [Github][github-release] along with etcd.
You can also build etcdctl from source using the build script found in the parent directory.
etcdctl can also be built from source using the build script found in the parent directory.
## Configuration
### --debug
@ -291,10 +291,7 @@ The following exit codes can be returned from etcdctl:
## Endpoint
If your etcd cluster isn't available on `http://127.0.0.1:2379` you can specify
a `--endpoint` flag or `ETCDCTL_ENDPOINT` environment variable. You can list one endpoint,
or a comma-separated list of endpoints. This option is ignored if the `--discovery-srv`
option is provided.
If the etcd cluster isn't available on `http://127.0.0.1:2379`, specify a `--endpoint` flag or `ETCDCTL_ENDPOINT` environment variable. One endpoint or a comma-separated list of endpoints can be listed. This option is ignored if the `--discovery-srv` option is provided.
```sh
ETCDCTL_ENDPOINT="http://10.0.28.1:4002" etcdctl set my-key to-a-value
@ -305,7 +302,7 @@ etcdctl --endpoint http://10.0.28.1:4002,http://10.0.28.2:4002,http://10.0.28.3:
## Username and Password
If your etcd cluster is protected by [authentication][authentication], you can specify username and password using the [`--username`][username-flag] or `ETCDCTL_USERNAME` environment variable. When `--username` flag or `ETCDCTL_USERNAME` environment variable doesn't contain password, etcdctl will prompt password in interactive mode.
If the etcd cluster is protected by [authentication][authentication], specify username and password using the [`--username`][username-flag] or `ETCDCTL_USERNAME` environment variable. When `--username` flag or `ETCDCTL_USERNAME` environment variable doesn't contain password, etcdctl will prompt password in interactive mode.
```sh
ETCDCTL_USERNAME="root:password" etcdctl set my-key to-a-value
@ -313,9 +310,7 @@ ETCDCTL_USERNAME="root:password" etcdctl set my-key to-a-value
## DNS Discovery
If you want to discover your etcd cluster through domain SRV records you can specify
a `--discovery-srv` flag or `ETCDCTL_DISCOVERY_SRV` environment variable. This option takes
precedence over the `--endpoint` flag.
To discover the etcd cluster through domain SRV records, specify a `--discovery-srv` flag or `ETCDCTL_DISCOVERY_SRV` environment variable. This option takes precedence over the `--endpoint` flag.
```sh
ETCDCTL_DISCOVERY_SRV="some-domain" etcdctl set my-key to-a-value

2
hack/insta-discovery/README.md
View File

@ -1 +1 @@
Starts a cluster via the discovery service on your local machine. Useful for testing.
Starts a cluster via the discovery service locally. Useful for testing.

2
hack/tls-setup/README.md
View File

@ -5,7 +5,7 @@ Defaults generate an ECDSA-384 root and leaf certificates for `localhost`. etcd
**Instructions**
1. Install git, go, and make
2. Amend https://github.com/coreos/etcd/blob/master/hack/tls-setup/config/req-csr.json - IP's currently in the config should be replaced/added to with IP addresses of each node in your cluster, please note 127.0.0.1 is always required for loopback purposes:
2. Amend https://github.com/coreos/etcd/blob/master/hack/tls-setup/config/req-csr.json - IP's currently in the config should be replaced/added with IP addresses of each cluster node, please note 127.0.0.1 is always required for loopback purposes:
```json
Example:
{

90
raft/README.md
View File

@ -13,9 +13,7 @@ To keep the codebase small as well as provide flexibility, the library only impl
In order to easily test the Raft library, its behavior should be deterministic. To achieve this determinism, the library models Raft as a state machine. The state machine takes a `Message` as input. A message can either be a local timer update or a network message sent from a remote peer. The state machine's output is a 3-tuple `{[]Messages, []LogEntries, NextState}` consisting of an array of `Messages`, `log entries`, and `Raft state changes`. For state machines with the same state, the same state machine input should always generate the same state machine output.
A simple example application, _raftexample_, is also available to help illustrate
how to use this package in practice:
https://github.com/coreos/etcd/tree/master/contrib/raftexample
A simple example application, _raftexample_, is also available to help illustrate how to use this package in practice: https://github.com/coreos/etcd/tree/master/contrib/raftexample
# Features
@ -54,8 +52,7 @@ This raft implementation also includes a few optional enhancements:
## Usage
The primary object in raft is a Node. You either start a Node from scratch
using raft.StartNode or start a Node from some initial state using raft.RestartNode.
The primary object in raft is a Node. Either start a Node from scratch using raft.StartNode or start a Node from some initial state using raft.RestartNode.
To start a three-node cluster
```go
@ -73,7 +70,7 @@ To start a three-node cluster
n := raft.StartNode(c, []raft.Peer{{ID: 0x02}, {ID: 0x03}})
```
You can start a single node cluster, like so:
Start a single node cluster, like so:
```go
// Create storage and config as shown above.
// Set peer list to itself, so this node can become the leader of this single-node cluster.
@ -81,7 +78,7 @@ You can start a single node cluster, like so:
n := raft.StartNode(c, peers)
```
To allow a new node to join this cluster, do not pass in any peers. First, you need add the node to the existing cluster by calling `ProposeConfChange` on any existing node inside the cluster. Then, you can start the node with empty peer list, like so:
To allow a new node to join this cluster, do not pass in any peers. First, add the node to the existing cluster by calling `ProposeConfChange` on any existing node inside the cluster. Then, start the node with an empty peer list, like so:
```go
// Create storage and config as shown above.
n := raft.StartNode(c, nil)
@ -110,46 +107,21 @@ To restart a node from previous state:
n := raft.RestartNode(c)
```
Now that you are holding onto a Node you have a few responsibilities:
After creating a Node, the user has a few responsibilities:
First, you must read from the Node.Ready() channel and process the updates
it contains. These steps may be performed in parallel, except as noted in step
2.
First, read from the Node.Ready() channel and process the updates it contains. These steps may be performed in parallel, except as noted in step 2.
1. Write HardState, Entries, and Snapshot to persistent storage if they are
not empty. Note that when writing an Entry with Index i, any
previously-persisted entries with Index >= i must be discarded.
1. Write HardState, Entries, and Snapshot to persistent storage if they are not empty. Note that when writing an Entry with Index i, any previously-persisted entries with Index >= i must be discarded.
2. Send all Messages to the nodes named in the To field. It is important that
no messages be sent until the latest HardState has been persisted to disk,
and all Entries written by any previous Ready batch (Messages may be sent while
entries from the same batch are being persisted). To reduce the I/O latency, an
optimization can be applied to make leader write to disk in parallel with its
followers (as explained at section 10.2.1 in Raft thesis). If any Message has type
MsgSnap, call Node.ReportSnapshot() after it has been sent (these messages may be
large). Note: Marshalling messages is not thread-safe; it is important that you
make sure that no new entries are persisted while marshalling.
The easiest way to achieve this is to serialise the messages directly inside
your main raft loop.
2. Send all Messages to the nodes named in the To field. It is important that no messages be sent until the latest HardState has been persisted to disk, and all Entries written by any previous Ready batch (Messages may be sent while entries from the same batch are being persisted). To reduce the I/O latency, an optimization can be applied to make leader write to disk in parallel with its followers (as explained at section 10.2.1 in Raft thesis). If any Message has type MsgSnap, call Node.ReportSnapshot() after it has been sent (these messages may be large). Note: Marshalling messages is not thread-safe; it is important to make sure that no new entries are persisted while marshalling. The easiest way to achieve this is to serialise the messages directly inside the main raft loop.
3. Apply Snapshot (if any) and CommittedEntries to the state machine.
If any committed Entry has Type EntryConfChange, call Node.ApplyConfChange()
to apply it to the node. The configuration change may be cancelled at this point
by setting the NodeID field to zero before calling ApplyConfChange
(but ApplyConfChange must be called one way or the other, and the decision to cancel
must be based solely on the state machine and not external information such as
the observed health of the node).
3. Apply Snapshot (if any) and CommittedEntries to the state machine. If any committed Entry has Type EntryConfChange, call Node.ApplyConfChange() to apply it to the node. The configuration change may be cancelled at this point by setting the NodeID field to zero before calling ApplyConfChange (but ApplyConfChange must be called one way or the other, and the decision to cancel must be based solely on the state machine and not external information such as the observed health of the node).
4. Call Node.Advance() to signal readiness for the next batch of updates.
This may be done at any time after step 1, although all updates must be processed
in the order they were returned by Ready.
4. Call Node.Advance() to signal readiness for the next batch of updates. This may be done at any time after step 1, although all updates must be processed in the order they were returned by Ready.
Second, all persisted log entries must be made available via an
implementation of the Storage interface. The provided MemoryStorage
type can be used for this (if you repopulate its state upon a
restart), or you can supply your own disk-backed implementation.
Second, all persisted log entries must be made available via an implementation of the Storage interface. The provided MemoryStorage type can be used for this (if repopulating its state upon a restart), or a custom disk-backed implementation can be supplied.
Third, when you receive a message from another node, pass it to Node.Step:
Third, after receiving a message from another node, pass it to Node.Step:
```go
func recvRaftRPC(ctx context.Context, m raftpb.Message) {
@ -157,10 +129,7 @@ Third, when you receive a message from another node, pass it to Node.Step:
}
```
Finally, you need to call `Node.Tick()` at regular intervals (probably
via a `time.Ticker`). Raft has two important timeouts: heartbeat and the
election timeout. However, internally to the raft package time is
represented by an abstract "tick".
Finally, call `Node.Tick()` at regular intervals (probably via a `time.Ticker`). Raft has two important timeouts: heartbeat and the election timeout. However, internally to the raft package time is represented by an abstract "tick".
The total state machine handling loop will look something like this:
@ -190,16 +159,13 @@ The total state machine handling loop will look something like this:
}
```
To propose changes to the state machine from your node take your application
data, serialize it into a byte slice and call:
To propose changes to the state machine from the node to take application data, serialize it into a byte slice and call:
```go
n.Propose(ctx, data)
```
If the proposal is committed, data will appear in committed entries with type
raftpb.EntryNormal. There is no guarantee that a proposed command will be
committed; you may have to re-propose after a timeout.
If the proposal is committed, data will appear in committed entries with type raftpb.EntryNormal. There is no guarantee that a proposed command will be committed; the command may have to be reproposed after a timeout.
To add or remove node in a cluster, build ConfChange struct 'cc' and call:
@ -207,8 +173,7 @@ To add or remove node in a cluster, build ConfChange struct 'cc' and call:
n.ProposeConfChange(ctx, cc)
```
After config change is committed, some committed entry with type
raftpb.EntryConfChange will be returned. You must apply it to node through:
After config change is committed, some committed entry with type raftpb.EntryConfChange will be returned. This must be applied to node through:
```go
var cc raftpb.ConfChange
@ -223,25 +188,8 @@ may be reused. Node IDs must be non-zero.
## Implementation notes
This implementation is up to date with the final Raft thesis
(https://ramcloud.stanford.edu/~ongaro/thesis.pdf), although our
implementation of the membership change protocol differs somewhat from
that described in chapter 4. The key invariant that membership changes
happen one node at a time is preserved, but in our implementation the
membership change takes effect when its entry is applied, not when it
is added to the log (so the entry is committed under the old
membership instead of the new). This is equivalent in terms of safety,
since the old and new configurations are guaranteed to overlap.
This implementation is up to date with the final Raft thesis (https://ramcloud.stanford.edu/~ongaro/thesis.pdf), although this implementation of the membership change protocol differs somewhat from that described in chapter 4. The key invariant that membership changes happen one node at a time is preserved, but in our implementation the membership change takes effect when its entry is applied, not when it is added to the log (so the entry is committed under the old membership instead of the new). This is equivalent in terms of safety, since the old and new configurations are guaranteed to overlap.
To ensure that we do not attempt to commit two membership changes at
once by matching log positions (which would be unsafe since they
should have different quorum requirements), we simply disallow any
proposed membership change while any uncommitted change appears in
the leader's log.
To ensure there is no attempt to commit two membership changes at once by matching log positions (which would be unsafe since they should have different quorum requirements), any proposed membership change is simply disallowed while any uncommitted change appears in the leader's log.
This approach introduces a problem when you try to remove a member
from a two-member cluster: If one of the members dies before the
other one receives the commit of the confchange entry, then the member
cannot be removed any more since the cluster cannot make progress.
For this reason it is highly recommended to use three or more nodes in
every cluster.
This approach introduces a problem when removing a member from a two-member cluster: If one of the members dies before the other one receives the commit of the confchange entry, then the member cannot be removed any more since the cluster cannot make progress. For this reason it is highly recommended to use three or more nodes in every cluster.

2
tools/functional-tester/README.md
View File

@ -34,7 +34,7 @@ Running the script requires:
Notes:
- Docker image is based on Alpine Linux OS running in privileged mode to allow iptables manipulation.
- To specify testing parameters (etcd-tester arguments) modify tools/functional-tester/docker/docker-compose.yml or start etcd-tester manually
- (OSX) make sure that etcd binary is built for linux/amd64 (eg. `rm bin/etcd;GOOS=linux GOARCH=amd64 ./tools/functional-tester/test`) otherwise you get `exec format error`
- (OSX) make sure that etcd binary is built for linux/amd64 (eg. `rm bin/etcd;GOOS=linux GOARCH=amd64 ./tools/functional-tester/test`) otherwise it will return `exec format error`
## with Goreman