Commit Graph

6182 Commits (4ccbcb91c8d31670a88ac81fddd314cf6720f50d)

Author SHA1 Message Date
Kenji Kaneda ebd8cb04c1 raft: fix a description of MemoryStorage.Compact
The parameter name is compactIndex, not i.
2015-10-06 21:49:33 -07:00
Hitoshi Mitake 68dd3ee621 etcdserver, test: don't access testing.T in time.AfterFunc()'s own goroutine
time.AfterFunc() creates its own goroutine and calls the callback
function in the goroutine. It can cause datarace like the problem
fixed in the commit de1a16e0f1 . This
commit also fixes the potential dataraces of tests in
etcdserver/server_test.go .
2015-10-06 11:37:08 +09:00
Xiang Li 21179d929f Merge pull request #3616 from yichengq/storage-txn
storage: hold batchTx lock during KV txn
2015-10-05 17:12:52 -07:00
Xiang Li 4f2ada3f1e Merge pull request #3643 from xiang90/metrics_storage
storage: add metrics for db total size
2015-10-05 17:11:00 -07:00
Xiang Li 0aa2f1192a storage: add metrics for db total size 2015-10-05 16:56:30 -07:00
Yicheng Qin 522ee6ab3a Merge pull request #3635 from yichengq/parse-ipv6
pkg/types: fix unwanted unescape in NewURLsMap
2015-10-05 15:30:51 -07:00
Yicheng Qin 699e37562e Merge pull request #3637 from yichengq/run-snapshot
etcdserver: get existing snapshot instead of requesting one
2015-10-05 14:57:03 -07:00
Yicheng Qin e117f36e48 pkg/types: fix unwanted unescape in NewURLsMap
We use url.ParseQuery to parse names-to-urls string, but it has side
effect that unescape the string. If the initial-cluster string has ipv6
which contains `%25`, it will unescape it to `%` and make further url
parse failed.

Fix it by modifiying the parse process.

Go1.4 doesn't support literal IPv6 address w/ zone in
URI(https://github.com/golang/go/issues/6530), so we only enable tests
in Go1.5+.
2015-10-05 14:54:17 -07:00
Yicheng Qin 8c94ae0ee3 etcdserver: get existing snapshot instead of requesting one
This fixes the problem that proposal cannot be applied.

When start the etcdserver.run loop, it expects to get the latest
existing snapshot. It should not attempt to request one because the loop
is the entity to create the snapshot.
2015-10-05 14:32:16 -07:00
Xiang Li ba949ae2be Merge pull request #3640 from xiang90/watch_metrics
storage: add metrics for watchers
2015-10-05 11:46:50 -07:00
Xiang Li 09157d4f1a storage: add metrics for watchers 2015-10-05 11:32:56 -07:00
Xiang Li 432b1bc230 Merge pull request #3638 from gyuho/documentation_proxy
Documentation: proxy.md typo, line-breaks
2015-10-05 07:57:22 -07:00
Gyu-Ho Lee f2dae5a0d2 Documentation: proxy.md typo, line-breaks
1. I found a little typo (easily -> easy)
2. If you go to https://coreos.com/etcd/docs/2.0.9/proxy.html,
   the proxy flag command is out of width of the web-page. Can
   we have line-breaks between flags to make the command easier
   to read?

Thanks for the great documentation!
2015-10-04 12:25:25 -07:00
Yicheng Qin c97dda766e storage: hold batchTx lock during KV txn
One txn is treated as atomic, and might contain multiple Put/Delete/Range
operations. For now, between these operations, we might call forecCommit
to sync the change to disk, or backend may commit it in background.
Thus the snapshot state might contains an unfinished multiple objects
transaction, which is dangerous if database is restored from the snapshot.

This PR makes KV txn hold batchTx lock during the process and avoids
commit to happen.
2015-10-03 16:01:05 -07:00
Yicheng Qin 581cc5cff4 Merge pull request #3608 from yichengq/storage-snapshot
storage: update KV.Snapshot function
2015-10-03 15:32:14 -07:00
Yicheng Qin 36f4303fc3 storage/etcdserver: update KV.Snapshot function
When using Snapshot function, it is expected:
1. know the size of snapshot before writing data
2. split snapshot-ready phase and write-data phase. so we could cut
snapshot first and write data later.

Update its interface to fit the requirement of etcdserver.
2015-10-03 10:15:23 -07:00
Yicheng Qin 8c0db94fef Merge pull request #3631 from yichengq/create-snapshot
etcdserver: support to create raft snapshot at apply loop
2015-10-03 10:03:27 -07:00
Yicheng Qin 675d4306b0 Merge pull request #3634 from yichengq/fix-cluster-output
etcdserver: print out correct restored cluster info
2015-10-02 16:23:33 -07:00
Yicheng Qin 18c568bc82 etcdserver: print out correct restored cluster info
Before this PR, it always prints nil because cluster info has not been
covered when print:

```
2015-10-02 14:00:24.353631 I | etcdserver: loaded cluster information
from store: <nil>
```
2015-10-02 16:11:32 -07:00
Xiang Li 69ca0b8475 Merge pull request #3633 from xiang90/systemd_readiness
etcdmain: print out error and suggestion for fixing notify issue
2015-10-02 13:56:16 -07:00
Xiang Li 51043830d4 etcdmain: print out error and suggestion for fixing notify issue 2015-10-02 13:39:41 -07:00
Yicheng Qin bfe9502f4f etcdserver: support to create raft snapshot at apply loop
and snapStore could trigger it to create the latest raft snapshot.
2015-10-02 13:17:56 -07:00
Xiang Li f8a4d1f01b Merge pull request #3607 from xiang90/doc_name
doc: emphasize name should be unique
2015-10-02 12:23:06 -07:00
Xiang Li 2733e3f543 doc: emphasize name should be unique 2015-10-02 09:58:20 -07:00
Xiang Li f093559b1d Merge pull request #3632 from mickep76/master
docs/libraries-and-tools: add etcd-rest rest api daemon
2015-10-02 09:49:59 -07:00
Michael Persson 4835a411c7 docs/libraries-and-tools: add etcd-rest rest api daemon 2015-10-02 14:55:57 +02:00
Yicheng Qin ccce61bda9 Merge pull request #3614 from yichengq/snapshot-store
etcdserver: add snapshotStore and raftStorage
2015-10-01 19:35:34 -07:00
Yicheng Qin f47cbf3073 Merge pull request #3627 from jelmer/typofix
Fix typo: boostrapping -> bootstrapping.
2015-10-01 19:02:47 -07:00
Yicheng Qin 2276328720 etcdserver: add snapshotStore and raftStorage
snapshotStore is the store of snapshot, and it supports to get latest snapshot
and save incoming snapshot.

raftStorage supports to get latest snapshot when v3demo is open.
2015-10-01 19:00:59 -07:00
Jelmer Vernooij d70975e54c
Documentation/configuration.md: Fix typo.
boostrapping -> bootstrapping
2015-10-01 20:03:16 +00:00
Xiang Li 715fdfb669 Merge pull request #3093 from mwitkow-io/feature/httpd_metrics
add `events` metrics in etcdhttp.
2015-10-01 12:10:58 -07:00
Xiang Li 6e9943a037 Merge pull request #3629 from ccding/master
raft: fix typo in doc
2015-10-01 10:11:12 -07:00
Cong Ding b2edf1d24a raft: fix typo in doc 2015-10-01 11:21:23 -05:00
Michal Witkowski 1b2dc1c796 metrics: add `events` metrics in etcdhttp. 2015-10-01 08:11:42 +01:00
Yicheng Qin 46e5444d93 Merge pull request #3625 from yichengq/fix-race
pkg/transport: fix a data race in TestReadWriteTimeoutDialer
2015-09-30 17:50:48 -07:00
Yicheng Qin de1a16e0f1 pkg/transport: fix a data race in TestReadWriteTimeoutDialer
Accessing test.T async will cause data race.

Change to use select to coordinate the access of test.T.
2015-09-30 17:29:24 -07:00
Brandon Philips 036ea58a77 Documentation: 04 snapshot: add example with fleet 2015-09-30 16:35:28 -07:00
Brandon Philips b043635868 Documentation: fix-up the kubernetes github URL 2015-09-30 10:58:33 -07:00
Yicheng Qin 533e728b64 Merge pull request #3609 from yichengq/raft-snapshot
raft: kill TODO about behavior when snapshot fails
2015-09-29 19:32:31 -07:00
Yicheng Qin 4c82b481a5 raft: improve behavior when snapshot fails
etcd is going to support incremental snapshot, and we design to let it
send at most one snapshot out at first stage. So when one snapshot is in
flight, snapshot request will return error.

When failing to get snapshot when sending MsgSnap, raft prints out
related log and abort sending this message.
2015-09-29 19:15:15 -07:00
Yicheng Qin a535cf2cad Merge pull request #3610 from yichengq/load-storage
etcdserver: restore v3 storage when restart
2015-09-29 11:58:38 -07:00
Yicheng Qin 49d262185d Merge pull request #3590 from yichengq/discovery-log
etcdmain: improve log when join discovery fails
2015-09-29 08:02:18 -07:00
Hitoshi Mitake 33a0df3e33 etcdctl: use a context with -total-timeout in simple commands
Like the commit 8ebc933111, this commit lets simple etcdctl commands
use a context with timeout value passed via -total-timeout.

This commit doesn't change complex commands like watch,
cluster-health, and import because it is not obvious that using the
context in the commands is good or not.
2015-09-29 17:23:01 +09:00
Yicheng Qin 5d906a0acc etcdserver: restore v3 storage when restart
To load the previous data.
2015-09-29 00:14:27 -07:00
Yicheng Qin 939aa96a34 etcdmain: improve log when join discovery fails
Before this PR, the log is

```
2015/09/1 13:18:31 etcdmain: client: etcd cluster is unavailable or
misconfigured
```

It is quite hard for people to understand what happens.

Now we print out the exact reason for the failure, and explains the way
to handle it.
2015-09-28 23:23:50 -07:00
Xiang Li 783884a04e Merge pull request #3606 from kkaneda/kkaneda/tiny_fix
raft: remove an obsolete TODO comment on 4MB maxMsgSize hard coding
2015-09-28 21:44:45 -07:00
Kenji Kaneda f602767e50 raft: remove an obsolete TODO comment on 4MB maxMsgSize hard coding
The TODO comment was added by 7571b2cd, and it was addressed by d9b5b56c.
2015-09-28 21:31:12 -07:00
Xiang Li 6c05a01ec6 Merge pull request #3604 from gyuho/replace_netutil_BasicAuth
etcdhttp/auth: BasicAuth method in standard pkg
2015-09-28 15:55:46 -07:00
Gyu-Ho Lee 6264a41e22 ectd/Getting-etcd: update README to require Go1.4+
Notice `For those wanting to try the very latest version,`
2015-09-28 15:35:09 -07:00
Gyu-Ho Lee e16f81838b etcdhttp/auth: BasicAuth method in standard pkg
I created a new PR from https://github.com/coreos/etcd/pull/3598.
This is for `TODO: use the standard lib BasicAuth method when we move to
Go 1.4.` [1]. `BasicAuth` method got into Go standard package a year ago. [2]

---
1. https://github.com/coreos/etcd/blob/master/pkg/netutil/netutil.go#L126-L138
2. https://codereview.appspot.com/76540043/
2015-09-28 14:02:55 -07:00