Gyuho Lee
844091dda3
Merge pull request #12206 from ptabor/20200807-setLoggingDataRace
...
integration: Fix flakes due to .setupLogging race.
2020-08-11 11:58:47 -07:00
Piotr Tabor
9d182c2a70
clientv3/integration: Fix flaky TestGetTokenWithoutAuth ( #12200 )
...
The test is vary flaky on Travis.
Seems that since (https://github.com/etcd-io/etcd/issues/7724 ) the
client is expected to simply ignore whether server is in AuthDisabled
mode even if the user supplies credentials.
The tests used to:
* use very large cluster (10 nodes)
* set very low timeout (1 sec)
Such setup led to frequent deadlineExceed errors or following failures:
=== RUN TestGetTokenWithoutAuth
{"level":"warn","ts":"2020-08-04T16:50:48.686+0200","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-35573307-1ee5-441b-acc7-d073f0bd7de5/localhost:69820396562031027440","attempt":0,"error":"rpc error: code = Unavailable desc = etcdserver: leader changed"}
user_test.go:151: other errors:etcdserver: leader changed
--- FAIL: TestGetTokenWithoutAuth (10.91s)
2020-08-07 13:32:32 -07:00
Piotr Tabor
830618e44d
./integration: Fix flakes due to .setupLogging race.
...
The source of problem was the fact that multiple tests were creating
their clusters (and some of them were setting global grpclog).
If the test was running after some other test that created HttpServer
(so accessed grpclog), this was reported as race.
Tested with:
go test ./clientv3/. -v "--run=(Example).*" --count=2
go test ./clientv3/. -v "--run=(Test).*" --count=2
go test ./integration/embed/. -v "--run=(Test).*" --count=2
2020-08-07 13:54:41 +02:00
Jingyi Hu
cc564110bd
clientv3: remove excessive watch cancel logging ( #12187 )
2020-07-29 14:58:53 -07:00
Jan Kuri
7f27697df9
v3client: implement clientv3.Auth interface ( #12140 )
2020-07-20 16:53:25 -07:00
Denis Issoupov
701f02f90a
12125: panic: zap.Logger is nil in Embed client
2020-07-13 09:08:20 -07:00
CFC4N
b6d1987ccd
auth: return incorrect result 'ErrUserNotFound' when client request without username or username was empty. ( #12005 )
...
Fiexs https://github.com/etcd-io/etcd/issues/12004 .
2020-06-12 16:10:19 -07:00
Gyuho Lee
8af52dc0e7
Merge pull request #11935 from tangcong/save-cindex-for-lease
...
lease: ensure grant/revoke won't be applied repeatedly after restarting etcd
2020-05-31 00:56:12 -07:00
Ted Yu
dafd474677
clientv3: cancel the client if there is no endpoint
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-23 13:35:17 -07:00
tangcong
e9ae8eb5a1
*: ensure grant/revoke won't be applied repeatedly after restarting etcd
2020-05-22 21:47:24 +08:00
Gyuho Lee
a4ada8cb1f
Merge pull request #11922 from tedyu/non-recursive-watch
...
clientv3: non-recursive Watch()
2020-05-20 20:36:09 -07:00
Ted Yu
f976138186
clientv3: non-recursive watch
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-20 15:44:41 -07:00
Ted Yu
35d026246c
clientv3: use Mutex for watcher lock
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-19 19:20:44 -07:00
Gyuho Lee
39c43cfb3c
clientv3: make sure snapshot has integrity check hash
...
I've seen some cases SHA blobs are missing (still investigating).
Adding a check to make sure snapshot save always downloads
hash digests for integrity checks.
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-14 22:57:52 -07:00
Jack Kleeman
ec13797407
mvcc: avoid negative watcher count metrics ( #11882 )
...
The watch count metrics are not robust to duplicate cancellations. These
cause the count to be decremented twice, leading eventually to negative
counts. We are seeing this in production. The duplicate cancellations
themselves are not themselves a big problem (except performance), but
they are caused by the new proactive cancellation logic (#11850 ) which
cancels proactively even immediately before initiating a Close, thus
nearly guaranteeing a Close-cancel race, as discussed in
watchable_store.go. We can avoid this in most cases by not sending a
cancellation when we are going to Close.
2020-05-12 16:50:53 -07:00
Jack Kleeman
87aa5a9e60
clientv3: cancel watches proactively on client context cancellation
...
Currently, watch cancel requests are only sent to the server after a
message comes through on a watch where the client has cancelled. This
means that cancelled watches that don't receive any new messages are
never cancelled; they persist for the lifetime of the client stream.
This has negative connotations for locking applications where a watch
may observe a key which might never change again after cancellation,
leading to many accumulating watches on the server.
By cancelling proactively, in most cases we simply move the cancel
request to happen earlier, and additionally we solve the case where the
cancel request would never be sent.
Fixes #9416
Heavy inspiration drawn from the solutions proposed there.
2020-05-07 16:50:26 +01:00
Zhihong Yu
17ed0c6638
clientv3: Check the error return from Write in v3Manager#Status ( #11837 )
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-04 17:43:08 -07:00
Zhihong Yu
df1d2ca78e
clientv3: v3Manager#saveDB should always close db file ( #11838 )
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-04 15:46:36 -07:00
Brandon Philips
b28a6272b7
Merge pull request #11823 from philips/v3-mod-attempt2
...
go.mod: name packages with go.etcd.io/etcd/v3
2020-04-28 15:24:29 -07:00
Gaurav Singh
86a97533bb
clientv3: fix DialTimeout race condition
2020-04-28 10:56:28 -04:00
Brandon Philips
7a5fefedcd
*: fix imports missed by gomove
2020-04-28 00:57:52 +00:00
Brandon Philips
96cce208c2
go.mod: use go.etcd.io/etcd/v3 versioning
...
This change makes the etcd package compatible with the existing Go
ecosystem for module versioning.
Used this tool to update package imports:
https://github.com/KSubedi/gomove
2020-04-28 00:57:35 +00:00
Xiang Li
b41711276e
clientv3/concurrency: do not swallow error ( #11458 )
2020-04-06 12:07:59 -07:00
zhanwang
2092b5b1a9
pkg, clientv3: etcdctl snaprestore when data-dir empty ( #11648 )
2020-04-04 14:41:19 -07:00
Sahdev Zala
a04b74d4b4
Merge pull request #11639 from jingyih/linearizable_memberlist
...
*: serve member list API with linearizable guarantee
2020-03-27 11:13:58 -04:00
jingyih
0344b70906
*: make MemberList linearizable
...
- Add linearizable field to etcdserverpb.MemberListRequest.
- Change behavior of clienv3 MemberList API. Now it is served with
linearizable guarantee.
2020-03-25 20:16:20 -07:00
Gyuho Lee
3ac7a11515
Merge pull request #11699 from tangcong/refactor-consistentindex
...
*: refactor consistentindex
2020-03-25 18:07:44 -07:00
tangcong
804070ce87
clientv3/snapshot: clean up initIndex
2020-03-25 10:59:19 +08:00
tangcong
7b2018683a
*: refactor consistent index
2020-03-25 10:59:15 +08:00
yoyinzyc
d8b9b54348
etcdserver: add downgrade rpc proto api.
2020-03-20 17:37:26 -07:00
Gyuho Lee
eeb371b7c1
clientv3: fix racy writes to context key
...
=== RUN TestWatchOverlapContextCancel
==================
WARNING: DATA RACE
Write at 0x00c42110dd40 by goroutine 99:
runtime.mapassign()
/usr/local/go/src/runtime/hashmap.go:485 +0x0
github.com/coreos/etcd/clientv3.metadataSet()
/go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/ctx.go:61 +0x8c
github.com/coreos/etcd/clientv3.withVersion()
/go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/ctx.go:47 +0x137
github.com/coreos/etcd/clientv3.newStreamClientInterceptor.func1()
/go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/client.go:309 +0x81
google.golang.org/grpc.NewClientStream()
/go/src/github.com/coreos/etcd/gopath/src/google.golang.org/grpc/stream.go:101 +0x10e
github.com/coreos/etcd/etcdserver/etcdserverpb.(*watchClient).Watch()
/go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/etcdserver/etcdserverpb/rpc.pb.go:3193 +0xe9
github.com/coreos/etcd/clientv3.(*watchGrpcStream).openWatchClient()
/go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:788 +0x143
github.com/coreos/etcd/clientv3.(*watchGrpcStream).newWatchClient()
/go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:700 +0x5c3
github.com/coreos/etcd/clientv3.(*watchGrpcStream).run()
/go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:431 +0x12b
Previous read at 0x00c42110dd40 by goroutine 130:
reflect.maplen()
/usr/local/go/src/runtime/hashmap.go:1165 +0x0
reflect.Value.MapKeys()
/usr/local/go/src/reflect/value.go:1090 +0x43b
fmt.(*pp).printValue()
/usr/local/go/src/fmt/print.go:741 +0x1885
fmt.(*pp).printArg()
/usr/local/go/src/fmt/print.go:682 +0x1b1
fmt.(*pp).doPrintf()
/usr/local/go/src/fmt/print.go:998 +0x1cad
fmt.Sprintf()
/usr/local/go/src/fmt/print.go:196 +0x77
github.com/coreos/etcd/clientv3.streamKeyFromCtx()
/go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:825 +0xc8
github.com/coreos/etcd/clientv3.(*watcher).Watch()
/go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:265 +0x426
github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel.func1()
/go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:959 +0x23e
Goroutine 99 (running) created at:
github.com/coreos/etcd/clientv3.(*watcher).newWatcherGrpcStream()
/go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:236 +0x59d
github.com/coreos/etcd/clientv3.(*watcher).Watch()
/go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:278 +0xbb6
github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel.func1()
/go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:959 +0x23e
Goroutine 130 (running) created at:
github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel()
/go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:979 +0x76d
github.com/coreos/etcd/clientv3/integration.TestWatchOverlapContextCancel()
/go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:922 +0x44
testing.tRunner()
/usr/local/go/src/testing/testing.go:657 +0x107
==================
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-18 17:05:05 -07:00
Gyuho Lee
33907477dd
*: add "etcd_server_client_requests_total", tests
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-17 12:16:11 -07:00
Gyuho Lee
58ba322bb4
clientv3: embed API version
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-17 12:16:11 -07:00
Gyuho Lee
5cb1e0b342
clientv3: fix metadata overwrites in "WithRequireLeader"
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-17 12:16:11 -07:00
Joe Betz
221f0cc107
Merge pull request #11564 from eddycjy/balancer-error
...
clientv3: Fix grpc-go(v1.27.0) incompatible changes to balancer/resolver.
2020-03-06 10:35:22 -08:00
EDDYCJY
4258cdd2ef
clientv3: fix grpc-go(v1.27.0) incompatible changes to balancer/resolver.
2020-02-18 18:27:53 +08:00
Jingyi Hu
84ace97ad7
Merge pull request #11572 from lzhfromustc/API_Fatal_4
...
integration/clientv3: fix 4 API misusage in test functions
2020-02-14 00:27:41 -08:00
Ziheng Liu
f00394e384
integration/clientv3: fix 4 API misusage in test functions
2020-02-13 13:13:18 -05:00
Vern Burton
071e70cdc4
*: add a new API and command for checking auth status ( #11536 )
...
This changes have started at etcdctl under auth.go, and make changes to stub out everything down into the internal raft. Made changes to the .proto files and regenerated them so that the local version would build successfully.
2020-02-05 19:27:42 -08:00
Jingyi Hu
7395ed8e5d
Merge pull request #11578 from jingyih/set_zap_as_default_logger
...
*: set zap as default logger, remove capnslog
2020-02-04 22:58:45 -08:00
jingyih
725e09023a
*: set zap as default logger, remove capnslog
...
Set zap as default logger. Remove capnslog and deprecated logging
flags.
2020-02-04 04:57:49 -08:00
Gyuho Lee
a698ad65f5
Merge pull request #11574 from YoyinZyc/simpligy-grpc
...
clientv3: simplify grpc dialer usage.
2020-02-03 12:41:41 -08:00
yoyinzyc
b90d801beb
clientV3: simplify grpc dialer usage. Remove workaround #11184 after bumping grpc to 1.26.0.
2020-01-30 14:51:24 -08:00
sfzhu93
467e08c32a
in multiple packages: fixed goroutine leak bugs in tests (cont.d) ( #11570 )
2020-01-30 10:55:59 -08:00
sfzhu93
cad92706cf
in multiple packages: fixed goroutine leak bugs in tests ( #11569 )
2020-01-30 10:45:59 -08:00
Jingyi Hu
342c2464ae
Documentation: specify starting revision ( #11559 )
2020-01-27 10:18:27 -08:00
Gyuho Lee
993e82a85b
clientv3/integration: fix a typo in "TestLeasingTxnRangeCmp"
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-01-19 23:56:22 -08:00
Gyuho Lee
2669b83c68
clientv3/integration: fix typo in "TestLeasingRevGet"
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-01-19 10:20:23 -08:00
Gyuho Lee
091b84f154
clientv3/integration: fix "TestLeaseKeepAliveNotFound"
...
with "default" select, the failure case will never be selected
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-01-17 12:04:10 -08:00
Gyuho Lee
255944bf51
clientv3/integration: expect "ErrCompacted" in "TestKVCompact"
...
Making tests more strict
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-01-17 12:03:13 -08:00