Commit Graph

231 Commits (1917e5c543d6b6e3439a4308af1920d038797113)

Author SHA1 Message Date
Gyuho Lee 8aae8c1c9c raft: document disruptive rejoining server, add tests
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-06 09:54:29 -08:00
Gyuho Lee d808b4686c raft: fix typo in raft_test.go
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-02-26 10:03:25 -08:00
Gyuho Lee 38846c220a raft: use leader's term when candidate becomes follower
`raft.Step` already ensures that when `m.Term > r.Term`,
candidate reverts back to follower with its term being
reset with `m.Term`, thus it's always true that
`m.Term == r.Term` in `stepCandidate`.

This just makes `r.becomeFollower` calls consistent.

Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-02-21 16:10:52 -08:00
Xiang Li d54f281b26
Merge pull request #8525 from shuaili87/pre-vote-compatible
raft: fix deadlock during PreVote migration process
2018-01-26 16:34:59 -08:00
Xiang Li c5532ebbf6
Merge pull request #9067 from absolute8511/optimize-raft-drop
raft: let raft step return error when proposal is dropped to allow fail-fast
2018-01-11 19:54:52 -08:00
Vincent Lee 30ced5b2be raft: let raft step return error when proposal is dropped to allow fail-fast. 2018-01-12 10:16:47 +08:00
Vincent Lee 11fa4f0275 raft: raft learners should be returned after applyConfChange 2018-01-11 17:30:17 +08:00
Xiang Li ed1ff9e952
Merge pull request #9073 from bdarnell/pending-conf-index
raft: Avoid scanning raft log in becomeLeader
2018-01-08 16:37:36 -08:00
Nathan VanBenschoten e6dc57f708 raft: s/leaner/learner/g 2018-01-03 08:16:50 -05:00
Ben Darnell 8d8f3195e4 raft: Avoid scanning raft log in becomeLeader
Scanning the uncommitted portion of the raft log to determine whether
there are any pending config changes can be expensive. In
cockroachdb/cockroach#18601, we've seen that a new leader can spend so
much time scanning its log post-election that it fails to send
its first heartbeats in time to prevent a second election from
starting immediately.

Instead of tracking whether a pending config change exists with a
boolean, this commit tracks the latest log index at which a pending
config change *could* exist. This is a less expensive solution to
the problem, and the impact of false positives should be minimal since
a newly-elected leader should be able to quickly commit the tail of
its log.
2017-12-30 10:13:36 -05:00
siddontang c6f2db2e92 raft: support learner 2017-11-11 10:38:21 +08:00
Xiang 9801fd7297 raft: ensure CheckQuorum is enabled when readonlyoption is lease based 2017-09-17 10:46:12 -07:00
gladiator 58b98c6a14 raft: check leader request when becomeFollower 2017-09-15 08:23:18 +08:00
gladiator 8597361f01 raft: fix Pre-Vote migration 2017-09-09 09:12:39 +08:00
irfan sharif 248384a468 raft: (re-)introduce TestRecvMsgPreVote
TestRecvMsgPreVote was intended to be introduced in
github.com/coreos/etcd/pull/6624 but was uncapitalized (search for
testRecvMsgPreVote instead) and then subsequently removed due to it
being unused.
2017-09-01 10:45:47 -04:00
gladiator 42cc64a9e5 raft: add TestPreVoteWithSplitVote 2017-08-02 17:59:28 +08:00
irfan sharif a92ceeec25 raft: introduce/fix TestNodeWithSmallerTermCanCompleteElection
TestNodeWithSmallerTermCanCompleteElection tests the scenario where a
node that has been partitioned away (and fallen behind) rejoins the
cluster at about the same time the leader node gets partitioned away.
Previously the cluster would come to a standstill when run with PreVote
enabled.

When responding to Msg{Pre,}Vote messages we now include the term from
the message, not the local term. To see why consider the case where a
single node was previously partitioned away and it's local term is now
of date. If we include the local term (recall that for pre-votes we
don't update the local term), the (pre-)campaigning node on the other
end will proceed to ignore the message (it ignores all out of date
messages).
The term in the original message and current local term are the same in
the case of regular votes, but different for pre-votes.

NB: Had to change TestRecvMsgVote to include pb.Message.Term when
sending MsgVote messages. The new sanity checks on MsgVoteResp
(m.Term != 0) would panic with the old test as raft.Term would be equal
to 0 when responding with MsgVoteResp messages.
2017-07-21 02:26:02 -04:00
Aaron Lehmann 9451fa1f9c raft: Add unit test TestAddNodeCheckQuorum
This test verifies that adding a node does not cause the leader to step
down until at least one full ElectionTick cycle elapses.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-05-04 15:04:30 -07:00
fanmin shi f18ae033a7 raft: use rs.req.Entries[0].Data as the key for deletion in advance()
advance() should use rs.req.Entries[0].Data as the context instead of
req.Context for deletion. Since req.Context is never set, there won't be
any context being deleted from pendingReadIndex; results mem leak.

FIXES #7571
2017-03-24 12:31:21 -07:00
Dylan.Wen bc6bebe7b0 raft: revise test case and fix typo 2017-02-21 15:23:42 +08:00
Dylan.Wen 9342647e0c raft: fix read index request for #7331 2017-02-17 09:45:41 +08:00
Dylan.Wen 7bb768ba34 raft: fix test case for #7042 2017-01-09 16:52:02 +08:00
Xiang Li b940e0d514 Merge pull request #7042 from petermattis/pmattis/resume-after-heartbeat-resp
raft: resume paused followers on receipt of MsgHeartbeatResp
2016-12-27 21:15:53 -08:00
Dylan.Wen 920b155f17 raft: fix pre-vote tests 2016-12-26 14:31:59 +08:00
Peter Mattis e625400f1d raft: resume paused followers on receipt of MsgHeartbeatResp
Previously, paused followers were resumed upon sending a MsgHearbeat.

Fixes #7037
2016-12-20 08:22:09 -05:00
Gyu-Ho Lee 0c5d1d5641 raft: simplify boolean comparison, remove unused 2016-12-12 10:07:14 -08:00
Xiang Li 529806dba1 Merge pull request #6935 from bdarnell/election-test
raft: Fix election "logs converge" test
2016-12-06 16:45:39 -08:00
Ben Darnell f60a5d6025 raft: Export Progress.IsPaused
CockroachDB would like to use this method for monitoring.
2016-12-04 13:14:08 +08:00
Ben Darnell 340ba8353c raft: Fix election "logs converge" test
The "logs converge" case in TestLeaderElectionPreVote was incorrectly
passing because some nodes were not actually using the preVoteConfig.
This test case was more complex than its siblings and it was not
verifying what it wanted to verify, so pull it out into a separate test
where everything can be tested more explicitly.

Fixes #6895
2016-12-03 17:29:15 +08:00
Ben Darnell 2f34547d39 raft: Check promotable() in MsgTimeoutNow handling
If MsgTimeoutNow arrived after a node was removed, the node could start
and win an election, then panic in becomeLeader (see
cockroachdb/cockroach#8535)
2016-11-07 20:02:21 +08:00
Ben Darnell 8d5e969f12 raft: Separate test methods for vote and pre-vote tests 2016-10-25 23:31:44 +09:00
Ben Darnell cf93a74aa8 raft: Refactor vote handling
Move all vote handling from the per-state step functions to the
top-level Step(). This wasn't necessary before because MsgVote would
cause us to become a follower, but MsgPreVote needs to be handled
without changing the node's current state.
2016-10-19 19:35:21 +08:00
Ben Darnell 73cae7abd0 raft: Implement the PreVote RPC described in thesis section 9.6
This prevents disruption when a node that has been partitioned
away rejoins the cluster.

Fixes #6522
2016-10-19 19:35:20 +08:00
Ben Darnell ca87a13b18 raft: More realistic terms in tests
Some tests were starting nodes with a non-empty log but a term of zero,
which cannot happen in the real world. This was affecting the final term
being tested in TestLeaderElection.
2016-10-19 19:35:20 +08:00
Dylan.Wen a6eb2939b1 raft: add test cases to improve test coverage 2016-09-28 10:19:30 +08:00
Xiang Li cfe717e926 Merge pull request #6275 from xiang90/raft_l
raft: support safe readonly request
2016-09-13 01:36:04 -05:00
Xiang Li 710b14ce56 raft: support safe readonly request
Implement raft readonly request described in raft thesis 6.4
along with the existing clock/lease based approach.
2016-09-12 15:13:52 +08:00
Dylan.Wen 68f2fdc1ff raft: add test case for leader transfer from follower 2016-09-08 17:22:52 +08:00
Ben Darnell a7a867c1e6 raft: Allow an election immediately after start with checkQuorum
Previously, the checkQuorum flag required an election timeout to
expire before a node could cast its first vote. This change permits
the node to cast a vote at any time when the leader is not known,
including immediately after startup.
2016-08-30 08:28:41 +08:00
swingbach@gmail.com 41dee84733 raft: fix #6096 2016-08-04 18:31:22 +08:00
swingbach@gmail.com c0a8da7fd0 raft: minor refactor 2016-08-02 08:46:43 +08:00
swingbach@gmail.com 992f628e6e raft: fix #6068 2016-07-30 03:27:29 +08:00
Xiang Li 7432e9fbe9 Merge pull request #5809 from swingbach/master
raft: make leader transferring workable when quorum check is on
2016-07-12 09:46:18 -07:00
Gyu-Ho Lee 6f3a40cb53 raft: set leader id in stepFollower
Follower has already set its leader ID from
previous append messages from the leader, but
to be consistent,  this adds a line to set its
leader id from leader snapshot message.
2016-07-11 16:37:31 -07:00
swingbach@gmail.com 0d9b6ba0ab raft: fix a few problems 2016-07-11 14:59:53 +08:00
swingbach@gmail.com e020b2a228 raft: make leader transferring workable when quorum check is on 2016-06-29 18:24:58 +08:00
swingbach@gmail.com 0faae33ace raft: implemented read-only query when quorum check is on 2016-06-28 10:52:53 +08:00
Xiang Li 9fee7732f6 Merge pull request #5468 from swingbach/master
implemented leader lease when quorum check is on.
2016-06-01 16:10:41 -07:00
swingbach@gmail.com 337ef64ed5 raft: implemented leader lease when quorum check is on 2016-06-02 06:17:27 +08:00
swingbach@gmail.com ff9d16a2e0 raft: fix tiny mistake of message type 2016-05-20 14:04:08 +08:00