etcdctlv3: add doc for mirror maker

release-2.3
Xiang Li 2016-02-19 13:43:48 -08:00
parent 825003a723
commit 13deb32447
3 changed files with 80 additions and 3 deletions

View File

@ -109,7 +109,6 @@ OK
./etcdctl range foo
```
### WATCH [options] [key or prefix]
Watch watches events stream on keys or prefixes. The watch command runs until it encounters an error or is terminated by the user.
@ -145,4 +144,38 @@ bar
#### Notes
TODO: doc interactive mode
TODO: doc interactive mode
## Utility Commands
### MAKE-MIRROR [options] \<destination\>
[make-mirror][mirror] mirrors a key prefix in an etcd cluster to a destination etcd cluster.
#### Options
- dest-cacert -- TLS certificate authority file for destination cluster
- dest-cert -- TLS certificate file for destination cluster
- dest-key -- TLS key file for destination cluster
- prefix -- The key-value prefix to mirror
#### Return value
Simple reply
- The approximate total number of keys transferred to the destination cluster, updated every 30 seconds.
- Error string if mirroring failed. Exit code is non-zero.
#### Examples
```
./etcdctl make-mirror mirror.example.com:2379
10
18
```
[mirror]: ./doc/mirror_maker.md

View File

@ -16,6 +16,9 @@ package command
import (
"errors"
"fmt"
"sync/atomic"
"time"
"github.com/coreos/etcd/Godeps/_workspace/src/github.com/spf13/cobra"
"github.com/coreos/etcd/Godeps/_workspace/src/golang.org/x/net/context"
@ -35,7 +38,7 @@ var (
// NewMakeMirrorCommand returns the cobra command for "makeMirror".
func NewMakeMirrorCommand() *cobra.Command {
c := &cobra.Command{
Use: "make-mirror [options] [destination]",
Use: "make-mirror [options] <destination>",
Short: "make-mirror makes a mirror at the destination etcd cluster",
Run: makeMirrorCommandFunc,
}
@ -62,6 +65,15 @@ func makeMirrorCommandFunc(cmd *cobra.Command, args []string) {
}
func makeMirror(ctx context.Context, c *clientv3.Client, dc *clientv3.Client) error {
total := int64(0)
go func() {
for {
time.Sleep(30 * time.Second)
fmt.Println(atomic.LoadInt64(&total))
}
}()
// TODO: remove the prefix of the destination cluster?
dkv := clientv3.NewKV(dc)
@ -75,6 +87,7 @@ func makeMirror(ctx context.Context, c *clientv3.Client, dc *clientv3.Client) er
if err != nil {
return err
}
atomic.AddInt64(&total, 1)
}
}
@ -105,8 +118,10 @@ func makeMirror(ctx context.Context, c *clientv3.Client, dc *clientv3.Client) er
switch ev.Type {
case storagepb.PUT:
ops = append(ops, clientv3.OpPut(string(ev.Kv.Key), string(ev.Kv.Value)))
atomic.AddInt64(&total, 1)
case storagepb.DELETE, storagepb.EXPIRE:
ops = append(ops, clientv3.OpDelete(string(ev.Kv.Key)))
atomic.AddInt64(&total, 1)
default:
panic("unexpected event type")
}

View File

@ -0,0 +1,29 @@
## Mirror Maker
Mirror maker mirrors a prefix in the key-value space of an etcd cluster into another prefix in another cluster. Mirroring is designed for copying configuration to various clusters distributed around the world. Mirroring usually has very low latency once it completes synchronizing with the initial state. Mirror maker utilizes the etcd watcher facility to immediately inform the mirror of any key modifications. Based on our experiments, the network latency between the mirror maker and the two clusters accounts for most of the latency. If the network is healthy, copying configuration held in etcd to the mirror should take under one second even for a world-wide deployment.
If the mirror maker fails to connect to one of the clusters, the mirroring will pause. Mirroring can be resumed automatically once connectivity is reestablished.
The mirroring mechanism is unidirectional. Data under the destination clusters mirroring prefix should be treated as read only. The mirror maker only mirrors key-value pairs; metadata, such as version number or modification revision, is discarded. However, mirror maker still attempts to preserve update ordering during normal operation, but there is no ordering guarantee during initial sync nor during failure recovery following network interruption. As a rule of thumb, the ordering of the updates on the mirror should not be considered reliable.
```
+-------------+
| |
| source | +-----------+
| cluster +----> | mirror |
| | | maker |
+-------------+ +---+-------+
|
v
+-------------+
| |
| mirror |
| cluster |
| |
+-------------+
```
Mirror-maker is a built-in feature of [etcdctl][etcdctl].
[etcdctl]: ../README.md