Commit Graph

1885 Commits (279fee668b24ca23bcb2828d6843a266adeb9950)

Author SHA1 Message Date
yoyinzyc 80a177292e rafthttp: add test stream support for current version. 2019-10-21 09:45:00 -07:00
Jingyi Hu 5dc12f2725
Merge pull request #11274 from YoyinZyc/fix-upgrade-failure
rafthttp: add 3.4.0,3.5.0 stream type
2019-10-20 19:20:06 -07:00
yoyinzyc a0e528e4b1 rafthttp: add 3.4.0,3.5.0 stream type 2019-10-17 14:25:56 -07:00
Jingyi Hu 444bfdff59 etcdserver: strip patch version in metrics
Strip patch version in cluster version metrics during node restart.
2019-10-16 12:39:17 -07:00
Jingyi Hu 1333abc606 etcdserver: strip patch version in cluster version
Strip patch version in cluster version metrics.
2019-10-14 16:59:09 -07:00
Jingyi Hu 9c4194f6ef etcdserver: unset old cluster version in metrics 2019-10-11 22:25:03 -07:00
Gyuho Lee 340f0ac797
Merge pull request #11179 from YoyinZyc/trace
Add tracing to range request in etcd server.
2019-10-08 13:23:53 -07:00
yoyinzyc 57aa68af5a etcdserver: trace compaction request; add return parameter 'trace' to applierV3.Compaction()
mvcc: trace compaction request; add input parameter 'trace' to KV.Compact()
2019-10-07 09:55:27 -07:00
yoyinzyc 3a3eb24c69 etcdserver: trace raft requests. 2019-10-01 15:38:52 -07:00
yoyinzyc 401df4bb8e etcdserver: add put request steps.
mvcc: add put request steps; add trace to KV.Write() as input parameter.
2019-10-01 14:08:06 -07:00
yoyinzyc 1d6ef8370e pkg: use zap logger to format the structure log output. 2019-09-30 13:11:21 -07:00
yoyinzyc 3830b3ef11 pkg: add field to record additional detail of trace; add stepThreshold
to reduce log volume.
2019-09-30 13:11:21 -07:00
yoyinzyc f4e7fc56a7 pkg: create package traceutil for tracing. mvcc: add tracing
steps:range from the in-memory index tree; range from boltdb.
etcdserver: add tracing steps: agreement among raft nodes before
linerized reading; authentication; filter and sort kv pairs; assemble
the response.
2019-09-30 13:06:02 -07:00
Xiang Li 589ab747f7
Merge pull request #11014 from dbavatar/peervalidation
etcdserver: Fix PeerURL validation
2019-09-13 17:42:39 -07:00
Debabrata Banerjee 0dd10cf6b8 etcdserver: Fix PeerURL validation
In case of URLs that are synonyms, the current lexicographic sorting
and compare of the URLs fails with frustrating errors. Make sure to do
a full comparison between every set of PeerURLs before failing.

Fixes #11013
2019-09-13 17:53:40 -04:00
zhangjianweibj 81a34ab6d5 etcdserver: remove dup percentage sign in log 2019-09-05 11:24:39 +08:00
Joe Betz 9b51febaf5 *: Add experimental-compaction-batch-limit flag 2019-08-15 11:47:23 -07:00
Gyuho Lee 06b82c200f etcdserver: add "etcd_server_snapshot_apply_inflights_total"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-08 13:33:52 -07:00
Gyuho Lee 46bddacacb etcdserver/api: add "etcd_network_snapshot_send_inflights_total", "etcd_network_snapshot_receive_inflights_total"
Useful for deciding when to terminate the unhealthy follower.
If the follower is receiving a leader snapshot, operator may wait.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-08 13:33:48 -07:00
Zeming YU 3edb569ad3
v3rpc: fix a typo `err`
don't read return value in child goroutine which causes data race.
2019-08-06 14:04:58 -07:00
Gyuho Lee 7fbbb9c8bf *: add 3.5 capability for 3.5 dev tree
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-02 15:27:54 -07:00
Gyuho Lee 3658571e3a etcdserver/api: enable 3.4 capability
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-02 15:24:18 -07:00
Gyuho Lee 6a0811a949 *: use new adt.IntervalTree interface
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-31 22:23:13 -07:00
Gyuho Lee c6e3401255 etcdserver: make raft log configured by top level logger
To make it consistent

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-29 15:43:19 -07:00
Gyuho Lee 2f30e9ad7f etcdserver: document v2 usage in "publish" method
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-28 21:07:39 -05:00
Gyuho Lee 7cbe2f5dd6 etcdserver/api/v3rpc: use new "credentials" package
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-26 08:56:38 -07:00
Gyuho Lee 50babc16e7 etcdserver/api/v2v3: skip tests for CI
To fix in v3.5

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-26 05:54:58 -07:00
Gyuho Lee 388d15f521
Merge pull request #10622 from philips/add-v2v3-tests
etcdserver: api/v2v3: add initial tests
2019-07-25 10:05:29 -07:00
Sahdev P. Zala 1cef112a79 etcdserver: do not allow creating empty role
Like user, we should not allow creating empty role.

Related #10905
2019-07-24 17:41:24 -04:00
Tobias Schottdorf b9c051e7a7 raftpb: clean up naming in ConfChange 2019-07-23 10:40:03 +02:00
Tobias Schottdorf b67303c6a2 raft: allow use of joint quorums
This change introduces joint quorums by changing the Node and RawNode
API to accept pb.ConfChangeV2 (on top of pb.ConfChange).

pb.ConfChange continues to work as today: it allows carrying out a
single configuration change. A pb.ConfChange proposal gets added to
the Raft log as such and is thus also observed by the app during Ready
handling, and fed back to ApplyConfChange.

ConfChangeV2 allows joint configuration changes but will continue to
carry out configuration changes in "one phase" (i.e. without ever
entering a joint config) when this is possible.
2019-07-23 10:40:03 +02:00
Tobias Schottdorf eb4d9b640a etcdserver: fix createConfChangeEnts
It created a sequence of conf changes that could intermittently cause an
empty set of voters, which Raft asserts against as of #10889.

This fixes TestCtlV2BackupSnapshot and TestCtlV2BackupV3Snapshot, see:
https://github.com/etcd-io/etcd/issues/10700#issuecomment-512358126
2019-07-19 17:13:08 +02:00
Tobias Schottdorf c9491d7861 raft: clean up bootstrap
This is the first (maybe not last) step in cleaning up the bootstrap
code around StartNode.

Initializing a Raft group for the first time is awkward, since a
configuration has to be pulled from thin air. The way this is solved
today is unclean: The app is supposed to pass peers to StartNode(),
we add configuration changes for them to the log, immediately pretend
that they are applied, but actually leave them unapplied (to give the
app a chance to observe them, though if the app did decide to not apply
them things would really go off the rails), and then return control to
the app. The app will then process the initial Readys and as a result
the configuration will be persisted to disk; restarts of the node then
use RestartNode which doesn't take any peers.

The code that did this lived awkwardly in two places fairly deep down
the callstack, though it was really only necessary in StartNode(). This
commit refactors things to make this more obvious: only StartNode does
this dance now. In particular, RawNode does not support this at all any
more; it expects the app to set up its Storage correctly.

Future work may provide helpers to make this "preseeding" of the Storage
more user-friendly. It isn't entirely straightforward to do so since
the Storage interface doesn't provide the right accessors for this
purpose. Briefly speaking, we want to make sure that a non-bootstrapped
node can never catch up via the log so that we can implicitly use one
of the "skipped" log entries to represent the configuration change into
the bootstrap configuration. This is an invasive change that affects
all consumers of raft, and it is of lower urgency since the code (post
this commit) already encapsulates the complexity sufficiently.
2019-07-19 10:02:02 +02:00
lzhfromustc d35f6647bc
Use newbe instead of s.be to avoid potential race
`s.cluster.SetBackend(s.be)` is not in critical section. Using `newbe` instead of `s.be` can avoid potential data race.
2019-07-08 14:24:52 -07:00
Tobias Schottdorf f9c2d00fb3 raft: extract 'tracker' package
Mechanically extract `progressTracker`, `Progress`, and `inflights`
to their own package named `tracker`. Add lots of comments in the
progress, and take the opportunity to rename and clarify various
fields.
2019-06-21 22:15:00 +02:00
Nima Yahyazadeh b1812a410f Raft HTTP: fix pause/resume race condition 2019-06-17 11:45:25 -04:00
Jingyi Hu 48d144a3de
Merge pull request #10731 from WIZARD-CXY/learner_metric
etcdserver: add learner metrics
2019-06-08 22:43:03 -07:00
Xiang Li 9a73013004
Merge pull request #10797 from jingyih/lease_checkpoint_enabled_by_experimental_flag
*: enable lease checkpoint via experimental flag
2019-06-05 22:56:54 -07:00
Jingyi Hu e67b9829b6 *: enable lease checkpoint via experimental flag
Primary lessor persist lease remainingTTL only if experimental flag
"--experimental-enable-lease-checkpoint" is set.
2019-06-05 15:30:03 -07:00
Gyuho Lee 1caaa9ed4a test: test update for Go 1.12.5 and related changes
Update to Go 1.12.5 testing. Remove deprecated unused and gosimple
pacakges, and mask staticcheck 1006. Also, fix unconvert errors related
to unnecessary type conversions and following staticcheck errors:
- remove redundant return statements
- use for range instead of for select
- use time.Since instead of time.Now().Sub
- omit comparison to bool constant
- replace T.Fatal and T.Fatalf in tests with T.Error and T.Fatalf respectively because the goroutine calls T.Fatal must be called in the same goroutine as the test
- fix error strings that should not be capitalized
- use sort.Strings(...) instead of sort.Sort(sort.StringSlice(...))
- use he status code of Canceled instead of grpc.ErrClientConnClosing which is deprecated
- use use status.Errorf instead of grpc.Errorf which is deprecated

Related #10528 #10438
2019-06-05 17:02:05 -04:00
宇慕 0b8727b3f3 etcdserver: add learner metrics 2019-06-05 10:51:21 +08:00
Gyuho Lee cdca488d8b
Merge pull request #9817 from mitake/no-password
*: support creating a user without password
2019-05-31 22:41:29 -07:00
Xiang Li d8e2e47de5
Merge pull request #10693 from nolouch/fix-lease
lease/lessor: recheck if exprired lease is revoked
2019-05-31 12:13:36 -07:00
Hitoshi Mitake 5a67dd788d *: support creating a user without password
This commit adds a feature for creating a user without password. The
purpose of the feature is reducing attack surface by configuring bad
passwords (CN based auth will be allowed for the user).

The feature can be used with `--no-password` of `etcdctl user add`
command.

Fix https://github.com/coreos/etcd/issues/9590
2019-05-30 21:59:30 +09:00
nolouch dc8a31eaf0 lease/lessor: recheck if exprired lease is revoked
Signed-off-by: nolouch <nolouch@gmail.com>
2019-05-29 14:27:19 +08:00
Jingyi Hu 23511d21ec *: address comments 2019-05-28 18:50:13 -07:00
Jingyi Hu 6bf609b96d integration: update TestMemberPromote test
Update TestMemberPromote to include both learner not-ready and learner
ready test cases.

Removed unit test TestPromoteMember, it requires underlying raft node to
be started and running. The member promote is covered by the integration
test.
2019-05-28 18:50:13 -07:00
宇慕 3f94385fc6 etcdserver: update raftStatus 2019-05-28 18:50:13 -07:00
Jingyi Hu e994a4df01 etcdserver: check http StatusCode before unmarshal
Check http StatusCode. Only Unmarshal body if StatusCode is statusOK.
2019-05-28 18:50:13 -07:00
Jingyi Hu f8ad8ae4ad etcdserver: use etcdserver ErrLearnerNotReady
If learner is not ready to be promoted, use etcdserver.ErrLearnerNotReady
instead of using membership.ErrLearnerNotReady.
2019-05-28 18:50:13 -07:00