Commit Graph

15933 Commits (3ffb3aed3a8f544da02fac84b12367820c90ee7c)

Author SHA1 Message Date
Gyuho Lee 3ffb3aed3a vendor: update bbolt v1.3.4, zap v1.14.1
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-19 13:36:43 -07:00
Sahdev Zala 45a45d3c0a
Merge pull request #11706 from gyuho/fix-race
clientv3: fix racy writes to context key
2020-03-19 10:34:08 -04:00
Gyuho Lee b6d98719f6 words: whitelist "racey"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-18 17:32:42 -07:00
Gyuho Lee eeb371b7c1 clientv3: fix racy writes to context key
=== RUN   TestWatchOverlapContextCancel

==================

WARNING: DATA RACE

Write at 0x00c42110dd40 by goroutine 99:

  runtime.mapassign()

      /usr/local/go/src/runtime/hashmap.go:485 +0x0

  github.com/coreos/etcd/clientv3.metadataSet()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/ctx.go:61 +0x8c

  github.com/coreos/etcd/clientv3.withVersion()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/ctx.go:47 +0x137

  github.com/coreos/etcd/clientv3.newStreamClientInterceptor.func1()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/client.go:309 +0x81

  google.golang.org/grpc.NewClientStream()

      /go/src/github.com/coreos/etcd/gopath/src/google.golang.org/grpc/stream.go:101 +0x10e

  github.com/coreos/etcd/etcdserver/etcdserverpb.(*watchClient).Watch()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/etcdserver/etcdserverpb/rpc.pb.go:3193 +0xe9

  github.com/coreos/etcd/clientv3.(*watchGrpcStream).openWatchClient()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:788 +0x143

  github.com/coreos/etcd/clientv3.(*watchGrpcStream).newWatchClient()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:700 +0x5c3

  github.com/coreos/etcd/clientv3.(*watchGrpcStream).run()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:431 +0x12b

Previous read at 0x00c42110dd40 by goroutine 130:

  reflect.maplen()

      /usr/local/go/src/runtime/hashmap.go:1165 +0x0

  reflect.Value.MapKeys()

      /usr/local/go/src/reflect/value.go:1090 +0x43b

  fmt.(*pp).printValue()

      /usr/local/go/src/fmt/print.go:741 +0x1885

  fmt.(*pp).printArg()

      /usr/local/go/src/fmt/print.go:682 +0x1b1

  fmt.(*pp).doPrintf()

      /usr/local/go/src/fmt/print.go:998 +0x1cad

  fmt.Sprintf()

      /usr/local/go/src/fmt/print.go:196 +0x77

  github.com/coreos/etcd/clientv3.streamKeyFromCtx()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:825 +0xc8

  github.com/coreos/etcd/clientv3.(*watcher).Watch()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:265 +0x426

  github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel.func1()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:959 +0x23e

Goroutine 99 (running) created at:

  github.com/coreos/etcd/clientv3.(*watcher).newWatcherGrpcStream()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:236 +0x59d

  github.com/coreos/etcd/clientv3.(*watcher).Watch()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:278 +0xbb6

  github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel.func1()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:959 +0x23e

Goroutine 130 (running) created at:

  github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:979 +0x76d

  github.com/coreos/etcd/clientv3/integration.TestWatchOverlapContextCancel()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:922 +0x44

  testing.tRunner()

      /usr/local/go/src/testing/testing.go:657 +0x107

==================

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-18 17:05:05 -07:00
Gyuho Lee bf883bd15b CHANGELOG*: add new patch releases
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-18 15:28:05 -07:00
Gyuho Lee b50c92cb73
Merge pull request #11704 from gyuho/log-health
*: log server-side /health checks
2020-03-18 12:33:13 -07:00
Gyuho Lee 92f180c574 *: log server-side /health checks
To make it easier to root-cause when /health check fails.
For example, we are using load balancer to health check
each etcd instance, and when one etcd node gets terminated,
it's hard to tell whether etcd "server" was really failing
or client (or load balancer") failed to reach the etcd cluster
which is also failure in load balancer health check.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-18 11:14:05 -07:00
Gyuho Lee 1c16c242db
Merge pull request #11687 from gyuho/embed-client-version
clientv3: fix "hasleader" metadata key, embed client version
2020-03-18 09:16:34 -07:00
Gyuho Lee 33907477dd *: add "etcd_server_client_requests_total", tests
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-17 12:16:11 -07:00
Gyuho Lee 58ba322bb4 clientv3: embed API version
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-17 12:16:11 -07:00
Gyuho Lee 5cb1e0b342 clientv3: fix metadata overwrites in "WithRequireLeader"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-17 12:16:11 -07:00
Jingyi Hu 07a74d61cb
Merge pull request #11675 from GSokol/master
etcdserver/api/etcdhttp: checkHealth refactoring
2020-03-17 22:43:27 +08:00
Grigorii Sokolik a33e1b5fae etcdserver/api/etcdhttp: checkHealth refactoring
Small refactoring of
`go.etcd.io/etcd/etcdserver/api/etcdhttp/metrics.go.checkHealth`
function just to avoid anoying repeatings of `if h.Health == "true" {`
2020-03-17 15:37:11 +02:00
Jingyi Hu c4d8bdc881
Merge pull request #11698 from hexfusion/cl-bump-11694
CHANGELOG: bump to include #11694
2020-03-17 14:07:06 +08:00
Jingyi Hu 31e7b34f71
Merge pull request #11700 from jingyih/correct_changelog_3p4
CHANGELOG-3.4: correction
2020-03-17 04:03:02 +08:00
jingyih 33fb62098b CHANGELOG: correction 2020-03-16 12:18:28 -07:00
Jingyi Hu 193e60ebd9
Merge pull request #11670 from tangcong/optimize-auth-store-log
*: optimize auth/etcdserver logs to facilitate troubleshooting data inconsistency
2020-03-16 00:15:14 +08:00
Sam Batschelet 8e35c0b867 CHANGELOG: bump to include #11694
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2020-03-15 07:41:41 -04:00
Sam Batschelet 7444d3ad53 proxy/grpcproxy: add return on error for metrics handler
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2020-03-14 16:34:16 -04:00
Sahdev Zala 23c2380105
Merge pull request #11666 from jingyih/update_changelog_from_11638
CHANGELOG: update from #11638
2020-03-11 19:23:59 -04:00
tangcong 5a17367923 etcdserver: print warn log when failed to apply request 2020-03-10 23:13:53 +08:00
tangcong 0d084d3a08 auth: cleanup saveConsistentIndex in NewAuthStore 2020-03-10 10:04:58 +08:00
Hitoshi Mitake 6d1982efe8
Merge pull request #11659 from wswcfan/add-auth-revision-status
etcdserver: add auth revision to AuthStatus to improve observability and testability
2020-03-09 23:08:55 +09:00
Joe Betz 221f0cc107
Merge pull request #11564 from eddycjy/balancer-error
clientv3: Fix grpc-go(v1.27.0) incompatible changes to balancer/resolver.
2020-03-06 10:35:22 -08:00
Xiang Li e0ff5ca318
RFC Documentation: enhance description of lock and lease (#11490)
* Documentation: enhance description of lock and lease

* Documentation: an executable implementation of fencing

* docs: api guarantees

cleanup lease grammar slightly

* docs: learning/lock/README.md improve grammar

Co-Authored-By: Steven E. Harris <seh@panix.com>

* docs: learning: improve locks and leases grammar

Co-authored-by: Brandon Philips <brandon@ifup.org>
Co-authored-by: Steven E. Harris <seh@panix.com>
2020-03-05 10:31:47 -08:00
shawwang 15eeb2c4ae etcdserver: add auth revision to AuthStatus to improve observability and testability 2020-03-04 22:37:24 +08:00
shawwang c6fce8c320 Documentation: generate *.swagger.json using latest protoc-gen-swagger 2020-03-04 22:36:13 +08:00
qupeng 6f850a65a1
raft: cleanup read index code (#11528)
Signed-off-by: qupeng <qupeng@pingcap.com>
2020-03-03 09:20:25 -08:00
tangcong ebe256d61c auth: print warning log when error is ErrAuthOldRevision 2020-03-03 23:21:25 +08:00
Sahdev Zala 3f4f7c5a6c
Merge pull request #11669 from jingyih/update_changelog_11621
CHANGELOG: update from #11621
2020-03-02 15:53:56 -05:00
jingyih cec9807a85 changelog: update from 11621
Also moving an previous update in changelog 3.4 to the right spot.
2020-03-02 10:11:13 -08:00
Jingyi Hu cb633418a2
Merge pull request #11652 from tangcong/fix-auth-store-corruption-bug
auth/store: save consistentIndex to fix a data corruption bug
2020-03-01 18:40:11 +08:00
jingyih e2ff769244 CHANGELOG: update from 11638 2020-02-29 07:33:55 -08:00
Jingyi Hu d774916f6d
Merge pull request #11638 from jingyih/fix_etcdctl_member_list
etcdctl: fix member add (again...)
2020-02-29 23:08:33 +08:00
shawwang f14d2a087f auth: add new metric 'etcd_debugging_auth_revision' 2020-02-29 13:31:44 +08:00
shawwang 08a8b80e32 tools/etcd-dump-db: add auth decoder, optimize print format 2020-02-29 13:31:44 +08:00
tangcong 06ad53321e *: fix auth revision corruption bug 2020-02-29 13:31:37 +08:00
Tobias Grieger 52fba431b6
Merge pull request #11046 from tbg/joint-auto-leave
raft: fix auto-transitioning out of joint config
2020-02-25 13:18:29 +01:00
Tobias Schottdorf 0544f33248 raft: clarify ApplyConfChange contract for rejected conf changes
Apps typically maintain the raft configuration as part of the state
machine. As a result, they want to be able to reject configuration change
entries at apply time based on the state on which the entry is supposed
to be applied. When this happens, the app should not call
ApplyConfChange, but the comments did not make this clear.

As a result, it was tempting to pass an empty pb.ConfChange or it's V2
version instead of not calling ApplyConfChange.

However, an empty V1 or V2 proto aren't noops when the configuration is
joint: an empty V1 change is treated internally as a single
configuration change for NodeID zero and will cause a panic when applied
in a joint state. An empty V2 proto is treated as a signal to leave a
joint state, which means that the app's config and raft's would diverge.

The comments updated in this commit now ask users to not call
ApplyConfState when they reject a conf change. Apps that never use joint
consensus can keep their old behavior since the distinction only matters
when in a joint state, but we don't want to encourage that.
2020-02-25 12:45:45 +01:00
Tobias Schottdorf 37c7e4d1d8 raft: fix auto-transitioning out of joint config
The code doing so was undertested and buggy: it would launch multiple
attempts to transition out when the conf change was not the last element
in the log.

This commit fixes the problem and adds a regression test. It also
reworks the code to handle a former untested edge case, in which the
auto-transition append is refused. This can't happen any more with the
current version of the code because this proposal has size zero and is
special cased in increaseUncommittedSize. Last but not least, the
auto-leave proposal now also bumps pendingConfIndex, which was not done
previously due to an oversight.
2020-02-25 12:35:51 +01:00
Tobias Schottdorf 2332705f10 raft: remove bogus tail end of membership change interaction test
The test was supposed to end earlier, but some old copy pasta
survived.
2020-02-25 12:35:51 +01:00
Gyuho Lee 5d7d8c09af CHANGELOG-3.4: update
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-02-24 13:42:21 -08:00
Jingyi Hu 085b19b5ea
Merge pull request #11645 from jingyih/update_changelog_11640
CHANGELOG: update from #11640 and #11644
2020-02-23 14:42:50 +08:00
jingyih d911a2b4d1 CHANGELOG: update from 11640 and 11644 2020-02-21 23:24:11 -08:00
Sahdev Zala 8c1a820a10
Merge pull request #11640 from ereslibre/fix-learner-promotion
Fix quorum calculation when promoting a learner member
2020-02-21 12:32:32 -05:00
Tobias Grieger 8756286fe8
Merge pull request #11568 from lzhfromustc/GL_test_7_bugs
transport: simple fix in test functions to prevent goroutine leak
2020-02-20 16:54:42 +01:00
Rafael Fernández López 6991f619f2
etcdserver: fix quorum calculation when promoting a learner member
When promoting a learner member we should not count already a voting
member, but take only into account the number of existing voting
members and their current status (started, unstarted) when taking the
decision whether a learner member can be promoted.

Before this change, it was impossible to grow from a quorum N to a N+1
through promoting a learning member.

Fixes: #11633
2020-02-20 12:06:35 +01:00
jingyih 8f5ce68adb etcdctl: fix member add command
Use members information from member add response, which is
guaranteed to be up to date.
2020-02-19 00:12:15 -08:00
Jingyi Hu d6a3c995cf
Merge pull request #11621 from jingyih/corruption_check_tls
etcdserver: make corruption check work under peer mTLS
2020-02-18 18:55:13 +08:00
EDDYCJY 4258cdd2ef clientv3: fix grpc-go(v1.27.0) incompatible changes to balancer/resolver. 2020-02-18 18:27:53 +08:00