1c1cccd236
rafthttp: stop etcd if it is found removed when stream dial
...
The original process is stopping etcd only when pipeline message finds itself
has been removed. After this PR, stream dial has this functionality too.
It helps fast etcd stop, which doesn't need to wait for stream break to
fall back to pipeline, and wait for election timeout to send out message
to detect self removal.
2015-04-27 15:10:00 -07:00
9f19b5660f
rafthttp: add AddRemote
...
Add remotes to rafthttp, who help newly joined members catch up the
progress of the cluster. It supports basic message sending to remote, and
has no stream connection for simplicity. remotes will not be used
after the latest peers have been added into rafthttp.
2015-04-24 11:49:23 -07:00
88224f6f4e
Revert "etcdserver: not apply stale conf change in cluster and transport"
...
This reverts commit 40197f0698
2015-04-19 11:08:03 -07:00
d9ad6aa2a9
*: update to use IANA-assigned ports
2015-04-06 13:49:43 -07:00
51548acb4f
rafthttp: reduce allocs in msgappv2
...
The patch decreases the allocs when sending one AppEntry in msgappv2
stream from 30 to 9. This helps reduce CPU load when etcd is under
high write load.
2015-04-06 09:45:39 -07:00
0d88e0d111
rafthttp: introduce msgappv2 stream format
...
msgappv2 stream is used to send all MsgApp, and replaces the
functionality of msgapp stream. Compared to v1, it has several
advantanges:
1. The output message is exactly the same with the input one, which
cannot be done in v1.
2. It uses one connection to stream persistently, which prevents message
reorder and saves the time to request stream.
3. It transmits 10 addiontional bytes in the procedure of committing one
proposal, which is trivia for idle time.
4. It transmits less bytes when committing mutliple proposals or keep
committing proposals.
2015-04-03 17:08:56 -07:00
40197f0698
etcdserver: not apply stale conf change in cluster and transport
2015-03-27 12:53:34 -07:00
d9cb77aad5
rafthttp: report snapshot failure when dropping MsgSnap
2015-03-12 13:06:43 -07:00
e41cbeda5d
rafthttp: drop messages in channel when disconnection
...
The messages in channel are outdated, and there is no need to send
them in the future. It also reports unreachable if there are messages
in the channel.
2015-03-11 14:42:06 -07:00
a230003255
rafthttp: report unreachable when dropping messages
2015-03-11 14:11:41 -07:00
b1ff6ddd88
Merge pull request #2446 from xiang90/apply-routine
...
etcdserver: separate apply and raft routine
2015-03-10 18:40:52 -07:00
d015610da5
etcdserver: separate apply and raft routine
2015-03-10 13:34:24 -07:00
51397a6423
rafthttp: use go-routine for MsgProp processing
...
MsgProp process is blocking when there is no leader, which blocks the peer
loop totally.
2015-03-09 14:11:16 -07:00
8e76ccf979
Merge pull request #2439 from xiang90/metrics
...
Metrics
2015-03-05 15:55:34 -08:00
a32abdbb0f
rafthttp: make metrics naming consistent
2015-03-04 16:12:53 -08:00
ab33c068b7
rafthttp: record the number of failed messages
2015-03-04 16:09:50 -08:00
933ab1e4f7
rafthttp: peer.newURLc -> peer.newURLsC
2015-03-04 15:00:47 -08:00
0fe9861197
rafthttp: support multiple peer urls
2015-03-04 15:00:07 -08:00
17aa3cf7db
rafthttp: add metrics for sending message
2015-03-04 11:18:16 -08:00
c7146bd5f2
Merge pull request #2421 from xiang90/cleanup-rafthttp
...
Cleanup rafthttp
2015-03-03 22:35:01 -08:00
44e53953c9
rafthttp: add comments for Transporter interface
2015-03-03 22:34:47 -08:00
cb105c626c
rafthttp: kill connection timeout TODO
2015-03-03 09:49:01 -08:00
3a132ad8ef
Merge pull request #2413 from xiang90/refactor-peer
...
rafthttp: add comment for timeout
2015-03-03 06:40:24 -08:00
1271b01069
Merge pull request #2406 from yichengq/333
...
rafthttp: add functional tests
2015-03-02 22:51:55 -08:00
115b045505
rafthttp: add comment for timeout
2015-03-02 16:52:19 -08:00
88bde91716
rafhttp: refactor func peer.pick in peer.go
2015-03-02 15:17:14 -08:00
81c67eed9c
rafthttp: add functional tests
2015-03-02 14:22:20 -08:00
fc2d7019e5
rafthttp: {nopProcessor, errProcessor} -> fakeRaft
2015-03-02 13:31:56 -08:00
f59b60671e
rafthttp: add peer tests
2015-03-02 13:30:30 -08:00
45d6b76eea
rafthttp: add stream tests
2015-03-02 13:29:05 -08:00
8ec28f27d1
rafthttp: streamReader roundtrip -> dial
2015-03-02 13:26:48 -08:00
a299f68e09
rafthttp: add transport benchmark test
2015-03-02 13:25:32 -08:00
9d445d2fcf
rafthttp: add transport tests
2015-03-02 13:25:30 -08:00
399e3cdf81
rafthttp: add stream http tests
2015-03-02 13:24:50 -08:00
b4b9b9118a
rafthttp: report MsgSnap status
2015-03-02 09:38:11 -08:00
78aa251ab2
rafthttp: only use pipeline to send MsgSnap
...
The size of MsgSnap may be very big, e.g., 1G.
If its size is big and general streaming is used to send it, it may block
the following messages for several ten seconds, which interrupts the
heartbeat heavily.
Only use pipeline to send MsgSnap.
2015-03-02 09:35:54 -08:00
9b986fb4c1
rafthttp: report unreachable status of the peer
...
When it failed to send message to the remote peer, it reports unreachable
to raft.
2015-03-01 16:48:26 -08:00
eeaf12beb1
rafthttp: use /raft/stream for MsgApp stream
...
New rafthttp uses /raft/stream/msgapp for MsgApp stream, but v2.0 rafthttp
cannot understand it. Use the old endpoint /raft/stream instead for backward
compatibility, and plan to move to new endpoint in the version after the
next one.
2015-02-28 11:35:16 -08:00
758ff26dd8
rafthttp: add copyright header
2015-02-28 11:35:16 -08:00
1fdbbb959f
rafthttp: add util, msgapp, message test
2015-02-28 11:35:16 -08:00
dee3001086
rafthttp: add back tests that commentted out
2015-02-28 11:35:16 -08:00
1c5a507761
rafthttp: refactor peer and add general stream
2015-02-28 11:35:16 -08:00
2c94e2d771
*: make dial timeout configurable
...
Dial timeout is set shorter because
1. etcd is supposed to work in good environment, and the new value is long
enough
2. shorter dial timeout makes dial fail faster, which is good for
performance
2015-02-28 11:18:59 -08:00
55cd03ff4b
rafthttp: add run loop for peer
2015-02-28 11:18:59 -08:00
7bf615aee0
*: drop old metrics pkg
2015-02-28 11:16:41 -08:00
d8a9e11e22
rafthttp: extract pipeline from peer
2015-02-28 11:06:11 -08:00
99840c9697
*: cleanup import
2015-02-28 10:12:35 -08:00
9b6fcfffb6
*: replace our own metrics with codahale/metrics
2015-02-28 10:11:53 -08:00
ca390560f9
rafthttp: fix panic on receiving empty ents
...
2.0 rc may send empty ents. Fix it for backward compatibility.
2015-02-20 15:07:27 -08:00
c16cc3a6a3
etcdserver: recover transport when recovering from a snapshot
2015-02-13 10:16:28 -08:00
317e57a8a8
rafthttp: Panic informatively when removing unknown peer ID
2015-02-12 14:43:44 -08:00
58112c4d2d
rafthttp: remove follower from leaderstats when it is removed from the cluster
2015-02-10 11:22:33 -08:00
96fde55a0f
rafthttp: not send 0-entry MsgApp using stream
...
It is not sent out because it is useless to let remote raft step the
message.
Moreover, MsgApp stream reader can always assume that the length
of entries sent is > 0.
2015-02-10 00:02:22 -08:00
a4c4027dc7
rafthttp: becomes -> became in log line
...
Simple grammar fix.
2015-02-04 21:28:23 -08:00
f1ed69e883
*: switch to line comments for copyright
...
Build tags are not compatible with block comments.
Also adds copyright header to a few places it was missing.
2015-01-26 09:53:30 -08:00
2120af8cfc
pkg: ioutils -> ioutil
2015-01-22 17:14:01 -08:00
99821579bf
metrics: add /rafthttp/stream metrics
2015-01-21 13:24:21 -08:00
84ceefbffc
rafthttp: write StatusOK before start streaming
2015-01-15 14:44:24 -08:00
0c55cfb21e
Merge pull request #2036 from xiang90/trtr
...
refactor rafthttp
2015-01-05 14:08:51 -08:00
1aa8f1eee6
rafthttp: clean up reader when failed
2015-01-05 12:04:25 -08:00
4938e6bff5
rafthttp: a stopped peer does not accept any methods
2015-01-03 20:02:43 -08:00
3319f716d9
rafthttp: a stopped stream does not accept any methods
2015-01-03 19:39:33 -08:00
15be030aaa
etcdserver: collect error from errorc
2015-01-02 20:13:46 -08:00
95a661251d
rafthttp: rename streamClient -> streamReader
2014-12-31 21:20:58 -08:00
fe53ffd74d
rafthttp: streamserver -> streamwriter
2014-12-31 21:11:24 -08:00
2292da15d6
rafthttp: fix stream.open call
2014-12-31 10:02:45 -08:00
d2c7a7e5cb
rafthttp: remove raftPrefix
2014-12-30 13:48:07 -08:00
2193b70fb3
rafthttp: add stream
2014-12-30 13:45:11 -08:00
c712dd682a
rafthttp: make Transport private
2014-12-29 12:20:52 -08:00
a14d13f724
rafthttp: make fields in Transport private
2014-12-29 12:08:13 -08:00
5bb8eeb5cf
rafthttp: transport cleanup
2014-12-29 11:21:40 -08:00
08f839e32c
rafthttp: set the API boundary of the package
2014-12-28 15:50:27 -08:00
04522baeee
etcdserver: fix leader stats
2014-12-15 16:50:03 -08:00
e3dbfefbe0
sender: set strmSrv to nil after stoping it
2014-12-14 20:00:32 -08:00
d86603840d
rafthttp: better logging
2014-12-14 09:50:59 -08:00
f2863e5279
Merge pull request #1911 from diffoperator/issue_1903
...
rafthttp: fixes issue 1903
2014-12-10 22:55:22 -08:00
123b3dd64c
rafthttp: fixes issue 1903
...
Record the URL being fetched in the log when we 404
2014-12-10 21:34:29 -08:00
e89cc25c50
Merge pull request #1901 from yichengq/260
...
rafthttp: batch MsgProp
2014-12-10 21:16:07 -08:00
8aba4caa72
rafthttp: batch MsgProp
...
If amounts of MsgProp goes to the follower, it could batch them and
forward to the leader. This can avoid dropping MsgProp in good path
due to exceed maximal serving in sender.
Moreover, batching MsgProp can increase the throughput of proposals that
come from follower.
2014-12-10 21:08:40 -08:00
07e876592b
rafthttp: log the type of message that is dropped when sending
2014-12-10 12:50:31 -08:00
2593914973
rafthttp: feedback
2014-12-10 10:18:01 -08:00
4094812b39
rafthttp: improve start/stop logging
2014-12-09 16:57:40 -08:00
b24d546bd0
rafthttp: increase sender buffer size
...
The buffer size is set big enough to buffer all messages generated in
one second as a follower in good path.
2014-12-07 22:35:50 -08:00
88e2fab572
Merge pull request #1859 from xiang90/pause_test
...
*: add pauseMember test
2014-12-04 15:11:59 -08:00
3de2ab2c04
*: typofixes
...
https://github.com/vlajos/misspell_fixer
2014-12-04 22:51:19 +00:00
151f043414
*: add pauseMember test
2014-12-04 14:22:43 -08:00
416b799ecf
Merge pull request #1788 from yichengq/233
...
rafthttp: increase the size of streaming buffer
2014-11-26 15:22:28 -08:00
00ce0702b9
rafthttp: increase the size of streaming buffer
...
Streaming buffer is used for:
1. hand over data to io goroutine in non-blocking way
2. hold pressure for temprorary network delay
3. be able to wait on I/O instead of data coming under high throughput
The old 1024 value is too small and is very likely to be full and
break the streaming when suffering temprorary network delay.
2014-11-26 14:46:52 -08:00
d5ceb26408
rafthttp: only batch good MsgAppResp
...
A MsgAppResp with Reject set should be sent back to the leader as soon
as possible instead of batching.
2014-11-25 17:53:26 -08:00
61ce494386
rafthttp: limit the buffer for every read correctly
2014-11-24 17:39:39 -08:00
1e797c1e38
rafthttp: limit the data size read from connection each time
2014-11-24 13:54:45 -08:00
3e55834c38
*: set read/write timeout for raft transport and listener
2014-11-24 13:46:44 -08:00
400e573013
rafthttp: log start and stop of streaming
2014-11-24 13:35:46 -08:00
91bfead9e9
rafthttp: add batcher
...
After we enable streaming, there will be a large amount of msgAppResp due to high
rate msgApp. We should batch msgAppResp in a meanful way.
2014-11-23 12:05:42 -08:00
264a63be80
rafthttp: fix import
2014-11-20 18:05:45 -08:00
c0fb1c8a00
Merge pull request #1755 from bcwaldon/golang.org-deps
...
Switch to golang.org/x/net/context
2014-11-20 16:26:14 -08:00
9d53b94546
rafthttp: add streaming server and client
2014-11-20 11:34:50 -08:00
9a728a127a
dep: bump golang.org/x/net/context
...
Move from code.google.com/p/go.net/context to
golang.org/x/net/context before bumping to latest.
2014-11-20 10:19:12 -08:00
1a72143ecb
rafthttp: send takes raft message instead of bytes
...
This gives streaming mechanism the chance to assemble and disassemble
raft messages.
2014-11-17 22:39:53 -08:00
04d416291a
*: add rafthttp pkg into test list
2014-11-17 16:45:54 -08:00
f24e214ee5
rafthttp: move server stats in raftHandler to etcdserver
2014-11-17 16:02:20 -08:00
5dc5f8145c
*: etcdhttp.raftHandler -> rafthttp.RaftHandler
2014-11-17 15:52:24 -08:00
84fbf7aab5
*: etcdserver.sender -> rafthttp.Sender
2014-11-17 15:35:10 -08:00
Yicheng Qin