Commit Graph

1872 Commits (7cb2bb67bfc75feed2bc8a78cdbbf05a6c2dbf47)

Author SHA1 Message Date
Xiang Li 589ab747f7
Merge pull request #11014 from dbavatar/peervalidation
etcdserver: Fix PeerURL validation
2019-09-13 17:42:39 -07:00
Debabrata Banerjee 0dd10cf6b8 etcdserver: Fix PeerURL validation
In case of URLs that are synonyms, the current lexicographic sorting
and compare of the URLs fails with frustrating errors. Make sure to do
a full comparison between every set of PeerURLs before failing.

Fixes #11013
2019-09-13 17:53:40 -04:00
zhangjianweibj 81a34ab6d5 etcdserver: remove dup percentage sign in log 2019-09-05 11:24:39 +08:00
Joe Betz 9b51febaf5 *: Add experimental-compaction-batch-limit flag 2019-08-15 11:47:23 -07:00
Gyuho Lee 06b82c200f etcdserver: add "etcd_server_snapshot_apply_inflights_total"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-08 13:33:52 -07:00
Gyuho Lee 46bddacacb etcdserver/api: add "etcd_network_snapshot_send_inflights_total", "etcd_network_snapshot_receive_inflights_total"
Useful for deciding when to terminate the unhealthy follower.
If the follower is receiving a leader snapshot, operator may wait.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-08 13:33:48 -07:00
Zeming YU 3edb569ad3
v3rpc: fix a typo `err`
don't read return value in child goroutine which causes data race.
2019-08-06 14:04:58 -07:00
Gyuho Lee 7fbbb9c8bf *: add 3.5 capability for 3.5 dev tree
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-02 15:27:54 -07:00
Gyuho Lee 3658571e3a etcdserver/api: enable 3.4 capability
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-02 15:24:18 -07:00
Gyuho Lee 6a0811a949 *: use new adt.IntervalTree interface
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-31 22:23:13 -07:00
Gyuho Lee c6e3401255 etcdserver: make raft log configured by top level logger
To make it consistent

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-29 15:43:19 -07:00
Gyuho Lee 2f30e9ad7f etcdserver: document v2 usage in "publish" method
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-28 21:07:39 -05:00
Gyuho Lee 7cbe2f5dd6 etcdserver/api/v3rpc: use new "credentials" package
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-26 08:56:38 -07:00
Gyuho Lee 50babc16e7 etcdserver/api/v2v3: skip tests for CI
To fix in v3.5

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-26 05:54:58 -07:00
Gyuho Lee 388d15f521
Merge pull request #10622 from philips/add-v2v3-tests
etcdserver: api/v2v3: add initial tests
2019-07-25 10:05:29 -07:00
Sahdev P. Zala 1cef112a79 etcdserver: do not allow creating empty role
Like user, we should not allow creating empty role.

Related #10905
2019-07-24 17:41:24 -04:00
Tobias Schottdorf b9c051e7a7 raftpb: clean up naming in ConfChange 2019-07-23 10:40:03 +02:00
Tobias Schottdorf b67303c6a2 raft: allow use of joint quorums
This change introduces joint quorums by changing the Node and RawNode
API to accept pb.ConfChangeV2 (on top of pb.ConfChange).

pb.ConfChange continues to work as today: it allows carrying out a
single configuration change. A pb.ConfChange proposal gets added to
the Raft log as such and is thus also observed by the app during Ready
handling, and fed back to ApplyConfChange.

ConfChangeV2 allows joint configuration changes but will continue to
carry out configuration changes in "one phase" (i.e. without ever
entering a joint config) when this is possible.
2019-07-23 10:40:03 +02:00
Tobias Schottdorf eb4d9b640a etcdserver: fix createConfChangeEnts
It created a sequence of conf changes that could intermittently cause an
empty set of voters, which Raft asserts against as of #10889.

This fixes TestCtlV2BackupSnapshot and TestCtlV2BackupV3Snapshot, see:
https://github.com/etcd-io/etcd/issues/10700#issuecomment-512358126
2019-07-19 17:13:08 +02:00
Tobias Schottdorf c9491d7861 raft: clean up bootstrap
This is the first (maybe not last) step in cleaning up the bootstrap
code around StartNode.

Initializing a Raft group for the first time is awkward, since a
configuration has to be pulled from thin air. The way this is solved
today is unclean: The app is supposed to pass peers to StartNode(),
we add configuration changes for them to the log, immediately pretend
that they are applied, but actually leave them unapplied (to give the
app a chance to observe them, though if the app did decide to not apply
them things would really go off the rails), and then return control to
the app. The app will then process the initial Readys and as a result
the configuration will be persisted to disk; restarts of the node then
use RestartNode which doesn't take any peers.

The code that did this lived awkwardly in two places fairly deep down
the callstack, though it was really only necessary in StartNode(). This
commit refactors things to make this more obvious: only StartNode does
this dance now. In particular, RawNode does not support this at all any
more; it expects the app to set up its Storage correctly.

Future work may provide helpers to make this "preseeding" of the Storage
more user-friendly. It isn't entirely straightforward to do so since
the Storage interface doesn't provide the right accessors for this
purpose. Briefly speaking, we want to make sure that a non-bootstrapped
node can never catch up via the log so that we can implicitly use one
of the "skipped" log entries to represent the configuration change into
the bootstrap configuration. This is an invasive change that affects
all consumers of raft, and it is of lower urgency since the code (post
this commit) already encapsulates the complexity sufficiently.
2019-07-19 10:02:02 +02:00
lzhfromustc d35f6647bc
Use newbe instead of s.be to avoid potential race
`s.cluster.SetBackend(s.be)` is not in critical section. Using `newbe` instead of `s.be` can avoid potential data race.
2019-07-08 14:24:52 -07:00
Tobias Schottdorf f9c2d00fb3 raft: extract 'tracker' package
Mechanically extract `progressTracker`, `Progress`, and `inflights`
to their own package named `tracker`. Add lots of comments in the
progress, and take the opportunity to rename and clarify various
fields.
2019-06-21 22:15:00 +02:00
Nima Yahyazadeh b1812a410f Raft HTTP: fix pause/resume race condition 2019-06-17 11:45:25 -04:00
Jingyi Hu 48d144a3de
Merge pull request #10731 from WIZARD-CXY/learner_metric
etcdserver: add learner metrics
2019-06-08 22:43:03 -07:00
Xiang Li 9a73013004
Merge pull request #10797 from jingyih/lease_checkpoint_enabled_by_experimental_flag
*: enable lease checkpoint via experimental flag
2019-06-05 22:56:54 -07:00
Jingyi Hu e67b9829b6 *: enable lease checkpoint via experimental flag
Primary lessor persist lease remainingTTL only if experimental flag
"--experimental-enable-lease-checkpoint" is set.
2019-06-05 15:30:03 -07:00
Gyuho Lee 1caaa9ed4a test: test update for Go 1.12.5 and related changes
Update to Go 1.12.5 testing. Remove deprecated unused and gosimple
pacakges, and mask staticcheck 1006. Also, fix unconvert errors related
to unnecessary type conversions and following staticcheck errors:
- remove redundant return statements
- use for range instead of for select
- use time.Since instead of time.Now().Sub
- omit comparison to bool constant
- replace T.Fatal and T.Fatalf in tests with T.Error and T.Fatalf respectively because the goroutine calls T.Fatal must be called in the same goroutine as the test
- fix error strings that should not be capitalized
- use sort.Strings(...) instead of sort.Sort(sort.StringSlice(...))
- use he status code of Canceled instead of grpc.ErrClientConnClosing which is deprecated
- use use status.Errorf instead of grpc.Errorf which is deprecated

Related #10528 #10438
2019-06-05 17:02:05 -04:00
宇慕 0b8727b3f3 etcdserver: add learner metrics 2019-06-05 10:51:21 +08:00
Gyuho Lee cdca488d8b
Merge pull request #9817 from mitake/no-password
*: support creating a user without password
2019-05-31 22:41:29 -07:00
Xiang Li d8e2e47de5
Merge pull request #10693 from nolouch/fix-lease
lease/lessor: recheck if exprired lease is revoked
2019-05-31 12:13:36 -07:00
Hitoshi Mitake 5a67dd788d *: support creating a user without password
This commit adds a feature for creating a user without password. The
purpose of the feature is reducing attack surface by configuring bad
passwords (CN based auth will be allowed for the user).

The feature can be used with `--no-password` of `etcdctl user add`
command.

Fix https://github.com/coreos/etcd/issues/9590
2019-05-30 21:59:30 +09:00
nolouch dc8a31eaf0 lease/lessor: recheck if exprired lease is revoked
Signed-off-by: nolouch <nolouch@gmail.com>
2019-05-29 14:27:19 +08:00
Jingyi Hu 23511d21ec *: address comments 2019-05-28 18:50:13 -07:00
Jingyi Hu 6bf609b96d integration: update TestMemberPromote test
Update TestMemberPromote to include both learner not-ready and learner
ready test cases.

Removed unit test TestPromoteMember, it requires underlying raft node to
be started and running. The member promote is covered by the integration
test.
2019-05-28 18:50:13 -07:00
宇慕 3f94385fc6 etcdserver: update raftStatus 2019-05-28 18:50:13 -07:00
Jingyi Hu e994a4df01 etcdserver: check http StatusCode before unmarshal
Check http StatusCode. Only Unmarshal body if StatusCode is statusOK.
2019-05-28 18:50:13 -07:00
Jingyi Hu f8ad8ae4ad etcdserver: use etcdserver ErrLearnerNotReady
If learner is not ready to be promoted, use etcdserver.ErrLearnerNotReady
instead of using membership.ErrLearnerNotReady.
2019-05-28 18:50:13 -07:00
Jingyi Hu f5eaaaf440 etcdserver: forward member promote to leader 2019-05-28 18:50:10 -07:00
宇慕 dfe296ac3c etcdserver: add mayPromote check 2019-05-28 18:47:03 -07:00
Jingyi Hu aa4cda2f5c etcdserver: allow 1 learner in cluster
Hard-coded the maximum number of learners to 1.
2019-05-28 18:47:03 -07:00
Jingyi Hu c438f6db27 etcdserver: check IsMemberExist before IsLearner
If member does not exist in cluster, IsLearner will panic.
2019-05-28 18:47:03 -07:00
Jingyi Hu d0c1b3fa38 etcdserver: learner return Unavailable for unsupported RPC
Make learner return code.Unavailable when the request is not supported
by learner. Client balancer will retry a different endpoint.
2019-05-28 18:47:03 -07:00
Jingyi Hu 76a63f9f7d etcdserver: adjust StrictReconfigCheck
Adjust StrictReconfigCheck logic to accommodate learner members in the
cluster.
2019-05-28 18:47:03 -07:00
Gyuho Lee 34bd797e67 *: revert module import paths
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-05-28 15:39:35 -07:00
Jingyi Hu 23f1d02391 *: address comments 2019-05-15 15:58:46 -07:00
Jingyi Hu c836e37a83 etcdserver: remove unnecessary bool comparison
Fixes 'gosimple' test.
2019-05-15 13:48:54 -07:00
WizardCXY a039f2efb8 clientv3, etcdctl: MemberPromote for learner 2019-05-15 13:48:52 -07:00
Jingyi Hu 44d935e90a etcdserver: exclude learner from leader transfer
1. Maintenance API MoveLeader() returns ErrBadLeaderTransferee if
transferee does not exist or is raft learner.

2. etcdserver TransferLeadership() only choose voting member as
transferee.
2019-05-15 13:27:42 -07:00
WizardCXY 7f9479acc1 clientv3: add member promote 2019-05-15 13:27:42 -07:00
WizardCXY ba9fd620e8 etcdserver: support MemberPromote for learner 2019-05-15 13:27:42 -07:00