Commit Graph

40 Commits (f602767e50661d665e9c5d965a67699bc4801326)

Author SHA1 Message Date
Yicheng Qin 8f6bf029f8 etcdserver: specify request timeout error due to connection lost
It specifies request timeout error possibly caused by connection lost,
and print out better log for user to understand.

It handles two cases:
1. the leader cannot connect to majority of cluster.
2. the connection between follower and leader is down for a while,
and it losts proposals.

log format:
```
20:04:19 etcd3 | 2015-08-25 20:04:19.368126 E | etcdhttp: etcdserver:
request timed out, possibly due to connection lost
20:04:19 etcd3 | 2015-08-25 20:04:19.368227 E | etcdhttp: etcdserver:
request timed out, possibly due to connection lost
```
2015-08-26 12:38:37 -07:00
Yicheng Qin ffae601af5 etcdmain: calculate dial timeout for peer transport
This helps peer communication in globally-deployed cluster.
2015-08-17 16:52:53 -07:00
Yicheng Qin fb4b0b5cf0 rafthttp: support to init term when adding peer
So it doesn't need to build term-0 stream with the remote first, then update it.
2015-06-29 10:20:48 -07:00
Yicheng Qin 7ff1fa36f2 rafthttp: always init streamReader before return from newPeer
Or etcd will panic if someone calls `setTerm()`, which uses streamReader
internally, before streamReader is inited.
2015-06-12 14:38:14 -07:00
Yicheng Qin 0de0e4b77c rafthttp: pretty print message drop info 2015-06-12 09:14:53 -07:00
Yicheng Qin 1dbe72bb74 rafthttp: pretty print connection error
1. print out the status change of connection with peer
2. only print the first error for repeated ones
2015-06-10 15:43:49 -07:00
Yicheng Qin f1e995b070 rafthttp: use leveled logger 2015-06-09 16:15:02 -07:00
Xiang Li 28878e34ff Merge pull request #2903 from xiang90/chord_rafthttp
rafhttp: clean up logging messages
2015-06-02 14:44:40 -07:00
Yicheng Qin c371d8c65c rafthttp: version enforcement on rafthttp messages
This PR sets etcd version and min cluster version in request header,
and let server check version compatibility. rafthttp server
will reject any message from peer with incompatible version(too low
version or too high version), and print out warning logs.
2015-06-02 13:33:18 -07:00
Xiang Li a7a4233f0b rafhttp: clean up logging messages 2015-06-01 17:18:37 -07:00
Yicheng Qin 19fc1a7137 rafthttp: update streamReader term in time
Because etcd 2.1 will build stream to any existing peers and etcd 2.0
requires the remote to provide most updated term, it is
necessary for streamReader to know the latest term.
2015-05-26 14:52:42 -07:00
Yicheng Qin 1c1cccd236 rafthttp: stop etcd if it is found removed when stream dial
The original process is stopping etcd only when pipeline message finds itself
has been removed. After this PR, stream dial has this functionality too.
It helps fast etcd stop, which doesn't need to wait for stream break to
fall back to pipeline, and wait for election timeout to send out message
to detect self removal.
2015-04-27 15:10:00 -07:00
Yicheng Qin 0d88e0d111 rafthttp: introduce msgappv2 stream format
msgappv2 stream is used to send all MsgApp, and replaces the
functionality of msgapp stream. Compared to v1, it has several
advantanges:
1. The output message is exactly the same with the input one, which
cannot be done in v1.
2. It uses one connection to stream persistently, which prevents message
reorder and saves the time to request stream.
3. It transmits 10 addiontional bytes in the procedure of committing one
proposal, which is trivia for idle time.
4. It transmits less bytes when committing mutliple proposals or keep
committing proposals.
2015-04-03 17:08:56 -07:00
Yicheng Qin d9cb77aad5 rafthttp: report snapshot failure when dropping MsgSnap 2015-03-12 13:06:43 -07:00
Yicheng Qin e41cbeda5d rafthttp: drop messages in channel when disconnection
The messages in channel are outdated, and there is no need to send
them in the future. It also reports unreachable if there are messages
in the channel.
2015-03-11 14:42:06 -07:00
Yicheng Qin a230003255 rafthttp: report unreachable when dropping messages 2015-03-11 14:11:41 -07:00
Xiang Li b1ff6ddd88 Merge pull request #2446 from xiang90/apply-routine
etcdserver: separate apply and raft routine
2015-03-10 18:40:52 -07:00
Xiang Li d015610da5 etcdserver: separate apply and raft routine 2015-03-10 13:34:24 -07:00
Yicheng Qin 51397a6423 rafthttp: use go-routine for MsgProp processing
MsgProp process is blocking when there is no leader, which blocks the peer
loop totally.
2015-03-09 14:11:16 -07:00
Yicheng Qin 933ab1e4f7 rafthttp: peer.newURLc -> peer.newURLsC 2015-03-04 15:00:47 -08:00
Yicheng Qin 0fe9861197 rafthttp: support multiple peer urls 2015-03-04 15:00:07 -08:00
Xiang Li cb105c626c rafthttp: kill connection timeout TODO 2015-03-03 09:49:01 -08:00
Xiang Li 115b045505 rafthttp: add comment for timeout 2015-03-02 16:52:19 -08:00
Xiang Li 88bde91716 rafhttp: refactor func peer.pick in peer.go 2015-03-02 15:17:14 -08:00
Yicheng Qin 399e3cdf81 rafthttp: add stream http tests 2015-03-02 13:24:50 -08:00
Yicheng Qin 78aa251ab2 rafthttp: only use pipeline to send MsgSnap
The size of MsgSnap may be very big, e.g., 1G.
If its size is big and general streaming is used to send it, it may block
the following messages for several ten seconds, which interrupts the
heartbeat heavily.
Only use pipeline to send MsgSnap.
2015-03-02 09:35:54 -08:00
Yicheng Qin 9b986fb4c1 rafthttp: report unreachable status of the peer
When it failed to send message to the remote peer, it reports unreachable
to raft.
2015-03-01 16:48:26 -08:00
Yicheng Qin 1c5a507761 rafthttp: refactor peer and add general stream 2015-02-28 11:35:16 -08:00
Yicheng Qin 2c94e2d771 *: make dial timeout configurable
Dial timeout is set shorter because
1. etcd is supposed to work in good environment, and the new value is long
enough
2. shorter dial timeout makes dial fail faster, which is good for
performance
2015-02-28 11:18:59 -08:00
Yicheng Qin 55cd03ff4b rafthttp: add run loop for peer 2015-02-28 11:18:59 -08:00
Yicheng Qin d8a9e11e22 rafthttp: extract pipeline from peer 2015-02-28 11:06:11 -08:00
Brandon Philips a4c4027dc7 rafthttp: becomes -> became in log line
Simple grammar fix.
2015-02-04 21:28:23 -08:00
Jonathan Boulle f1ed69e883 *: switch to line comments for copyright
Build tags are not compatible with block comments.
Also adds copyright header to a few places it was missing.
2015-01-26 09:53:30 -08:00
Xiang Li 0c55cfb21e Merge pull request #2036 from xiang90/trtr
refactor rafthttp
2015-01-05 14:08:51 -08:00
Xiang Li 4938e6bff5 rafthttp: a stopped peer does not accept any methods 2015-01-03 20:02:43 -08:00
Xiang Li 15be030aaa etcdserver: collect error from errorc 2015-01-02 20:13:46 -08:00
Xiang Li fe53ffd74d rafthttp: streamserver -> streamwriter 2014-12-31 21:11:24 -08:00
Yicheng Qin 2292da15d6 rafthttp: fix stream.open call 2014-12-31 10:02:45 -08:00
Xiang Li 2193b70fb3 rafthttp: add stream 2014-12-30 13:45:11 -08:00
Yicheng Qin 5bb8eeb5cf rafthttp: transport cleanup 2014-12-29 11:21:40 -08:00