Marek Siarkowicz
63346bfead
server: Use default logging configuration instead of zap production one
...
This fixes problem where logs json changes format of timestamp.
2022-04-01 10:23:42 +02:00
Chris Ayoub
125f3c3f9a
clientv3: filter learners members during autosync
...
This change is to ensure that all members returned during the client's
AutoSync are started and are not learners, which are not valid
etcd members to make requests to.
2022-03-29 13:38:21 -04:00
Marek Siarkowicz
0d55a1ca2a
Merge pull request #13821 from ahrtr/configspec_config
...
Move the newClientCfg into clientv3 package so as to be reused by both etcdctl and v3discovery
2022-03-24 10:12:55 +01:00
ahrtr
49e9a14580
migrate unit test to cover the logic of converting ConfigSpec to Config
2022-03-24 07:24:22 +08:00
ahrtr
f978da4f4f
move the newClientCfg into clientv3 package so as to be reused by both etcdctl and v3discovery
2022-03-24 06:18:25 +08:00
eval-exec
8d01ac2816
remove endpoints length check in TestValidateSecureEndpoints()
2022-03-23 23:37:31 +08:00
EXEC
ad78a74c42
Update client/pkg/transport/tls_test.go
...
Co-authored-by: Marek Siarkowicz <marek.siarkowicz@protonmail.com>
2022-03-23 23:37:31 +08:00
eval-exec
88e1d6b126
using subtests for TestValidateSecureEndpoints()
2022-03-23 23:37:31 +08:00
EXEC
4786a72cfc
Update client/pkg/transport/tls_test.go
...
Co-authored-by: Marek Siarkowicz <marek.siarkowicz@protonmail.com>
2022-03-23 23:37:31 +08:00
eval-exec
df71f59c0e
close idle connections
2022-03-23 23:37:31 +08:00
eval-exec
983ee82c98
add test for transport/tls.go:ValidateSecureEndpoints()
2022-03-23 23:37:31 +08:00
EXEC
bd7d09255b
Fix panic in etcd validate secure endpoints #13810
...
`ValidateSecureEndpoints()` should call `t.DialContext()` instead of `t.Dial()`, because `t.Dial` is `nil`
2022-03-23 23:37:31 +08:00
ahrtr
1a3822f2c3
Rename ClientConfig to ConfigSpec
...
The ClientConfig is a fully declarive configuration, so it makes more
sense to rename it to ConfigSpec. It can also mitigate the confusion
between Config and ClientConfig.
2022-03-13 05:41:49 +08:00
ahrtr
3dcbbf62d9
Move clientconfig into clientv3 so that it can be reused by both etcdctl and v3 discovery
2022-03-12 06:38:41 +08:00
cuishuang
b7824b794b
fix some typos
...
Signed-off-by: cuishuang <imcusg@gmail.com>
fix some typos
Signed-off-by: cuishuang <imcusg@gmail.com>
2022-03-07 20:28:53 +08:00
Marek Siarkowicz
fb55910500
version: bump up to 3.6.0-alpha.0
2022-02-28 13:20:27 +01:00
Marek Siarkowicz
6af760131e
Merge pull request #13687 from serathius/etcdctl
...
Add downgrade commands
2022-02-22 17:12:23 +01:00
Piotr Tabor
f80f477073
Merge pull request #13644 from Juneezee/refactor/t.TempDir
...
*: use `T.TempDir` to create temporary test directory
2022-02-21 19:52:37 +01:00
Marek Siarkowicz
d0c1c3a1fd
client: Alias downgrade action enum
2022-02-21 17:58:20 +01:00
ahrtr
8681888012
fix typo, renamed ErrGPRCNotSupportedForLearner to ErrGRPCNotSupportedForLearner
2022-02-21 14:46:58 +08:00
ahrtr
5fc0092c87
trim the suffix dot from the srv.Target for etcd-client DNS lookup
2022-02-18 18:16:19 +08:00
Eng Zer Jun
e9edc5432c
*: use `T.TempDir` to create temporary test directory
...
The directory created by `T.TempDir()` and is automatically removed when
the test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-01-25 00:00:46 +08:00
ahrtr
15568f4c00
add protection code for Range when the sortTarget is an invalid value
2022-01-18 07:46:37 +08:00
Piotr Tabor
fdd98477ef
Update dep: require gopkg.in/yaml.v2 v2.2.8 -> v2.4.0 due to: CVE-2019-11254
2022-01-17 16:52:58 +01:00
Jille Timmermans
3710083dc3
retry_interceptor.go: Log the method that failed
...
When getting warnings from the retry_interceptors that RPCs failed, they
provide very little context as to what exactly failed.
I used this patch to try debugging
https://groups.google.com/g/etcd-dev/c/3hxxr9ktdpM .
2022-01-16 16:46:50 +01:00
Piotr Tabor
b2f6ffdd81
Merge pull request #13595 from serathius/enable-v2
...
server: --enable-v2 and --enable-v2v3 is decomissioned
2022-01-15 16:48:29 +01:00
Marek Siarkowicz
ee5ef42c5c
server: --enable-v2 and --enable-v2v3 is decomissioned
2022-01-14 13:19:30 +01:00
Marek Siarkowicz
4032d4f66a
Remove CodeQL errors
2022-01-13 14:29:09 +01:00
Sahdev Zala
96a9fd0a1e
Merge pull request #13574 from cunnie/defer_cancel
...
Golang Client docs: defer `cancel()`, avoid erroring
2022-01-03 20:03:22 -05:00
Yap Sok Ann
17fd2e7282
Disable auth gracefully without impacting existing watchers
...
This attempts to fix a special case of the problem described in #12385 ,
where trying to do `clientv3.Watch` with an expired token would result
in `ErrGRPCPermissionDenied`, due to the failing authorization check in
`isWatchPermitted`. Furthermore, the client can't auto recover, since
`shouldRefreshToken` rightly returns false for the permission denied
error.
In this case, we would like to have a runbook to dynamically disable
auth, without causing any disruption. Doing so would immediately expire
all existing tokens, which would then cause the behavior described
above. This means existing watchers would still work for a period of
time after disabling auth, until they have to reconnect, e.g. due to a
rolling restart of server nodes.
This commit adds a client-side fix and a server-side fix, either of
which is sufficient to get the added test case to pass. Note that it is
an e2e test case instead of an integration one, as the reconnect only
happens if the server node is stopped via SIGINT or SIGTERM.
A generic fix for the problem described in #12385 would be better, as
that shall also fix this special case. However, the fix would likely be
a lot more involved, as some untangling of authn/authz is required.
2021-12-31 14:39:46 +07:00
Brian Cunnie
5620a9c227
Golang Client docs: defer cancel(), avoid erroring
...
In the sample code demonstrating how to specify a client request
timeout, the `cancel()` is called immediately after the Put, but it
should be deferred instead, giving the Put enough time to complete.
In the canonical Golang context
[docs](https://pkg.go.dev/context#WithTimeout ), the sample code sets a
`defer cancel()` immediately after context creation, and with this
commit we adhere to that convention.
fixes:
```json
{
"level": "warn",
"ts": "2021-12-29T09:56:42.439-0800",
"logger": "etcd-client",
"caller": "v3@v3.5.1/retry_interceptor.go:62",
"msg": "retrying of unary invoker failed",
"target": "etcd-endpoints://0xc000213340/localhost:2379",
"attempt": 0,
"error": "rpc error: code = Canceled desc = context canceled"
}
```
2021-12-29 14:10:36 -08:00
Piotr Tabor
69279532f4
Merge pull request #13540 from songlh-psu/fixing-3
...
fixing one panic and two goroutine leaks
2021-12-21 11:03:59 +01:00
Linhai Song
0098dbf350
fixing two goroutine leaks and one panic
2021-12-15 22:38:25 -05:00
Linhai
d1194977eb
fix potential goroutine leaks in TestTxnPanics
2021-12-15 01:22:56 -05:00
Kushal Pisavadia
71493bde3e
*: Upgrade to use go.opentelemetry.io/otel@v1.2.0
...
Upgrading from v1.0.1.
Upgrading related dependencies
------------------------------
The following dependencies also had to be upgraded:
- go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.26.1
From v0.25.0. This gets rid of a transitive dependency on go.opentelemetry.io/otel@v1.0.1.
- google.golang.org/genproto@v0.0.0-20211118181313-81c1377c94b1
2021-11-24 16:03:33 +00:00
Chao Chen
b5e4c2d3c4
client/v2: remove unsafe json-iterator/reflect2
2021-11-09 11:16:40 -08:00
Piotr Tabor
9c28e07588
Merge pull request #13447 from Juneezee/deprecate-ioutil
...
*: move from io/ioutil to io and os packages
2021-10-29 23:18:36 +02:00
Sam Batschelet
316e62b4e1
*: fixup go 1.17 bump
...
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-10-27 14:26:55 -04:00
Eng Zer Jun
2a151c8982
*: move from io/ioutil to io and os packages
...
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil . This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-10-28 00:05:28 +08:00
Lili Cosic
e6e279a14d
*: Bump go to go v1.17.2
2021-10-27 13:44:54 +02:00
Piotr Tabor
8aa6ae9bbb
Merge pull request #13361 from vooon/otel-1.0.0
...
server: update OpenTelemetry to 1.0.1
2021-10-20 14:14:00 +02:00
Vladimir Ermakov
49a8aa7f49
run fix.sh
...
To fix dependencies.
Signed-off-by: Vladimir Ermakov <vooon341@gmail.com>
2021-10-13 14:08:27 +03:00
Arda Güçlü
d7fa8022e6
Merge branch 'main' into support-zap-console-encoding
2021-10-08 11:00:57 +03:00
Aleksandr Razumov
3644c9d67b
client/pkg/fileutil: add missing logger to {Create,Touch}DirAll
...
Also populate it to every invocation.
2021-10-07 17:53:59 +03:00
Karen Almog
d93b7c8cb1
pkg/types: Support Unix sockets in NewURLS
...
Resolves #12450
This commits adds support to unix/unixs socket URLs, which currently
fail with the message "URL address does not have the form "host:port".
It also replaces the work started in #11747 .
2021-10-07 11:59:32 +02:00
Marek Siarkowicz
c929a917b6
client: Use first endpoint as http2 authority header
2021-09-29 14:41:35 +02:00
Piotr Tabor
ca5bad0e83
Merge pull request #13261 from njhill/sorting
...
etcdserver,clientv3: server-side ignore sort-ascend-key for range reqs
2021-09-25 17:47:10 +02:00
Arda Güçlü
3c6e09f932
Handle empty log-format gracefully
2021-09-23 08:17:29 +03:00
Arda Güçlü
ec252d06c9
Return error when log-format is invalid
2021-09-22 19:45:08 +03:00
Arda Güçlü
e647995a38
Add zap encoding configurable
...
Json encoding is the default zap encoding value and can not be changeable.
This PR enables configuring zap encoding to console via new flag `log-format`.
2021-09-22 15:48:47 +03:00
Tero Saarni
78463237d4
Merge branch 'main' into etcd-client-solaris-build
2021-09-10 11:31:44 +03:00
leoyang.yl
ad69fe0f40
fix IsOptsWithFromKey
...
fix IsOptsWithFromKey
fix IsOptsWithFromKey
fix IsOptsWithFromKey
fix IsOptsWithFromKey
fix IsOptsWithFromKey
2021-09-10 00:44:52 +08:00
Hitoshi Mitake
2a750a8dba
*: implement a retry logic for auth old revision in the client
2021-09-05 01:13:52 +09:00
Tero Saarni
af626ebfde
client/pkg/v3: fixes Solaris build of transport
...
Add empty implementation for reuse port socket option since Solaris does not
support SO_REUSEPORT.
2021-08-30 11:44:02 +03:00
Ashish Ranjan
6d300fd38b
client/v3: refresh the token when ErrUserEmpty is received while retrying
...
To fix a bug in the retry logic caused when the auth token is cleared after receiving `ErrInvalidAuthToken` from the server and the subsequent call to `getToken` also fails due to some reason (eg. context deadline exceeded).
This leaves the client without a token and the retry will continue to fail with `ErrUserEmpty` unless the token is refreshed.
2021-08-25 11:27:19 +08:00
nickhill
99182f5404
etcdserver,clientv3: server-side ignore sort-ascend-key for range requests
...
A client-side optimization was made in #6100 to filter ascending key sorts to avoid an unnecessary re-sort since this is the order already returned by the back-end logic.
It seems to me that this really belongs on the server side since it's tied to the server implementation and should apply for any caller of the kv api (for example non-go clients).
Related, the client/v3 syncer depends on this default sorting which isn't explicit in the kv api contract. So I'm proposing the required sort parameters be included explicitly; it will take the fast path either way.
2021-07-30 15:02:15 -07:00
Yi-Shu Tai
77a5072b69
client: call .Endpoints() in dial() in client/v3/client.go instead of accessing cfg.Endpoints directly
...
0cdd558361/client/v3/client.go (L299)
accesses
endpoints without acquiring lock. Fix it to call Endpoints()
Fix #13201
2021-07-27 15:55:22 -07:00
Piotr Tabor
40da254d68
Merge pull request #13146 from tklauser/filelock-x-sys-windows
...
client/pkg/fileutil: use `golang.org/x/sys/windows` for `FileLockEx`
2021-07-01 10:44:08 +02:00
Piotr Tabor
7271adecc5
Merge pull request #12992 from awly/client-auth-bundle-overwrite
...
client/v3: do not overwrite authTokenBundle on dial
2021-07-01 10:40:18 +02:00
Marek Siarkowicz
86f68b9374
*: Add missing file licenses and Fix static analysis prevent skipping them in future
2021-06-29 12:52:02 +02:00
Tobias Klauser
c3fe63a658
client/pkg/fileutil: use golang.org/x/sys/windows for FileLockEx
...
Use the FileLockEx wrapper and the corresponding LOCKFILE_* and error
constants from the golang.org/x/sys/windows package rather than
implementing these in the fileutil package.
2021-06-25 14:44:30 +02:00
Andrew Lytvynov
ab8e5a4f8e
client/v3: do not overwrite authTokenBundle on dial
...
`Client.dial` can be called multiple times. For example, from regular
`NewClient` and [from the `Maintenance`
wrapper](6d451ab61d/client/v3/maintenance.go (L81-L84)
).
This ends up creating two (if `Client.dial` is called twice) independent
`credentials.Bundle` instances:
- the first one is wired into `PerRPCCredentials` callback in gRPC
client.
- the second one is in `Client.authTokenBundle` and receives the latest
auth token updates.
When using username/password authentication and the server is configured
to issue JWTs, token rotation logic breaks because of the above
`credentials.Bundle` confusion.
2021-06-23 09:44:27 -07:00
Lili Cosic
df696a7e76
go.mod: Bump etcd to 3.5.0
2021-06-22 12:25:39 +02:00
Piotr Tabor
5564476831
Merge pull request #13073 from serathius/snapshot-version
...
[Version in Snapshot] SnapshotResponse includes local etcd version
2021-06-21 16:56:00 +02:00
Brandon Philips
1a943350b0
client: README: update to new go.mod paths
...
The paths in the readme and go.dev docs are out of date after v3.5
2021-06-15 19:47:22 -07:00
Marek Siarkowicz
e1b1d93548
*: Snapshot returns local etcd version
...
Co-authored-by: Lili Cosic <cosiclili@gmail.com>
2021-06-14 16:36:50 +02:00
Lili Cosic
16477a8270
Update client_golang dependency to v1.11.0
2021-06-07 14:49:57 +02:00
Gyuho Lee
23e6bf3671
Merge pull request #13083 from lilic/add-downgrade-client
...
client/v3/maintenance.go: Add Downgrade support to client
2021-06-04 15:31:48 -07:00
Lili Cosic
d563c76e92
client/v3/maintenance.go: Add Downgrade support to client
2021-06-04 13:53:09 +02:00
Piotr Tabor
404efd70f7
Upgrade of prometheous deps.
...
For now we need to depend on unstable: prometheus/client_golang
2021-06-03 16:07:01 +02:00
Piotr Tabor
edcfe575cf
Update dependencies:
...
- github.com/coreos/go-systemd/v22 v22.3.2
- github.com/google/btree v1.0.1
- github.com/json-iterator/go v1.1.11
- github.com/mattn/go-runewidth v0.0.13
- github.com/prometheus/client_golang v1.10.0
- github.com/prometheus/common v0.26.0
- github.com/sirupsen/logrus v1.8.1
2021-06-03 13:03:01 +02:00
Piotr Tabor
b240625d21
Update bbolt to v1.3.6.
...
`./scripts/update_dep.sh go.etcd.io/bbolt v1.3.6`
2021-06-03 12:39:53 +02:00
Piotr Tabor
90cea7d4e7
Update zap to v1.17.0.
2021-06-03 12:37:43 +02:00
Piotr Tabor
573e055cd3
Merge pull request #13069 from lilic/log-errors
...
client, raft: Log unhandled errors
2021-06-02 18:05:58 +02:00
Lili Cosic
3aae7d7823
client/v3/README.md: Remove warning
2021-06-02 16:35:12 +02:00
Lili Cosic
65cdf43e48
client/v3/lease.go: Log unhandled errors
2021-06-02 11:41:26 +02:00
Piotr Tabor
f15e0b8237
integration.BeforeTest can be run without leak-detection.
2021-05-27 20:55:11 +02:00
Piotr Tabor
bda32db121
Expose clientv3.CreateDefaultZapLoggerConfig
...
Need raised in
f3f4259d3f
.
BTW: Alligned the implementation of grpclogs with what we use in embed
server, so reduced the client code dependencies.
2021-05-25 22:59:08 +02:00
Pavan BG
facba6c663
client: Replace nil value testing with assert.Nil
...
Multiple errors in previous commit where `nil` is not being detected by `assert.Equal`
2021-05-21 14:18:01 +05:30
Pavan BG
25d3c1b068
*: Fixed go.sum
...
Ran fix.sh script to correct for differences in go.sum, perhaps due to multiple Golang versions
2021-05-21 13:28:43 +05:30
Pavan BG
ce5bb69c28
client: Added missing packages
...
go.etcd.io/etcd/client/pkg/v3/testutil: package github.com/stretchr/testify/assert imported from implicitly required module; to add missing requirements, run:
2021-05-20 21:44:12 +05:30
Pavan BG
319ef4aa42
client: Removed AssertEqual function
...
Used github.com/stretchr/testify/assert to replace the AssertEqual function definition. Required the use of copyToInterface to copy a string slice to the empty interface slice
2021-05-20 21:33:07 +05:30
Pavan BG
d9c5e1f0a2
*: Replace internal testutil AssertEqual function in favour of github.com/stretchr/testify/assert.Equals
...
To maintain uniformity in testing methodology, assert.Equals has been used everywhere while replacing testutil.AssertEqual
Fixes #13015
2021-05-20 19:55:45 +05:30
Lili Cosic
7957f4461d
server/embed/config_test.go: Add unit test for socket options
2021-05-20 10:00:27 +02:00
Lili Cosic
73c530de83
client,server: Expose reuse-port and reuse-address so they can be
...
configured via a config file.
2021-05-19 18:41:21 +02:00
Piotr Tabor
16d51d8c26
Fix not retryable error codes from: Unavailable -> FailedPrecondition
...
- ErrGRPCNotCapable("etcdserver: not capable") -> codes.FailedPrecondition (it will not autofix, it requires new version of server)
- ErrGPRCNotSupportedForLearner("etcdserver: rpc not supported for learner") -> codes.FailedPrecondition (as long as its learner, the call will not work)
- ErrGRPCClusterVersionUnavailable("etcdserver: cluster version not found during downgrade") -> codes.FailedPrecondition (backend does not contain the version (old etcd?) so retry will not help)
https://github.com/etcd-io/etcd/runs/2599598633?check_suite_focus=true
```
{"level":"warn","ts":"2021-05-17T09:55:30.246Z","logger":"etcd-client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc000539880/#initially=[unix://localhost:m30]","attempt":0,"error":"rpc error: code = Unavailable desc = etcdserver: rpc not supported for learner"}
{"level":"warn","ts":"2021-05-17T09:55:30.270Z","logger":"etcd-client","caller":"v3/retry_interceptor.go:62","msg":"retrying
of unary invoker
failed","target":"etcd-endpoints://0xc000539880/#initially=[unix://localhost:m30]","attempt":1,"error":"rpc
error: code = Unavailable desc = etcdserver: rpc not supported for
learner"}`
```
2021-05-19 02:08:53 +02:00
Piotr Tabor
3dd577ec08
Simplify client's logger configuration in tests.
2021-05-18 13:03:16 +02:00
Piotr Tabor
8981afb6f5
Fix unit tests logging config.
2021-05-14 22:45:57 +02:00
Piotr Tabor
f5c26814ab
Loggers to catch the e2e flake.
2021-05-14 22:42:31 +02:00
Gyuho Lee
e2d67f2e3b
Merge pull request #12956 from gyuho/rename-to-main
...
*: rename "master" branch references to "main" in source code
2021-05-13 08:26:33 -07:00
Gyuho Lee
a8e1cf0433
client: rename "master" branch references to "main"
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-12 10:38:50 -07:00
Marek Siarkowicz
1189ee3f3d
client: Add logger argument to NewCtxClient
2021-05-12 16:40:55 +02:00
Piotr Tabor
1929aa0a36
Merge pull request #12948 from serathius/logger
...
client: Allow setting zap logger in config
2021-05-12 13:49:48 +02:00
Marek Siarkowicz
06afe87b34
client: Allow setting zap logger in config
2021-05-12 12:15:20 +02:00
Piotr Tabor
3b24496c2a
Merge pull request #12925 from yangxuanjia/yxjetcd_check_resuming
...
check resuming len to avoid panic
2021-05-12 09:18:40 +02:00
yangxuanjia
68b1e9f728
check resuming len to avoid panic
2021-05-12 09:51:59 +08:00
Lili Cosic
1a718a958e
Add initial Tracing with OpenTelemetry
2021-05-10 10:44:40 +02:00
刘兵
a8a3eea56d
fix doc demo bug
...
the service parameter not used.
2021-05-05 13:57:10 +08:00
Piotr Tabor
451f65d661
Merge pull request #12908 from ptabor/20210429-client-retry-logging
...
Clientv3 (retry interceptor) logs should use the configured logger
2021-04-29 19:25:04 +02:00
Piotr Tabor
ffea1537d4
ClientV3 tests use integration.NewClient that configures proper logger.
2021-04-29 18:18:34 +02:00
Piotr Tabor
bc8d3f6639
Clientv3 (retry) logs should use the configured logger.
...
clientv3 logs (especially tests) were poluted with unattributed to testing.T log lines:
```
{"level":"warn","ts":"2021-04-29T12:42:11.055+0200","logger":"etcd-client","caller":"v3/retry_interceptor.go:64","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0000fafc0/#initially=[unix://localhost:m10]","attempt":0,"error":"rpc error: code = ResourceExhausted desc = etcdserver: mvcc: database space exceeded"}
```
The reasons were 2 fold:
- Interceptors were copying logger before "WithLogger" could modify it.
- We were not propagating the loggers in a few testing contexts.
2021-04-29 12:57:09 +02:00
Piotr Tabor
f53b70facb
Embed: In case KVStoreHash verification fails, close the backend.
...
In case of failed verification, the server used to keep opened backend
(so the file was locked on OS level).
2021-04-29 11:51:25 +02:00
Piotr Tabor
11249fdee9
Merge pull request #12874 from ptabor/20210417-go1.16.3
...
Update go for 3.5: 1.15.x -> 1.16.3
2021-04-19 16:51:49 +02:00
Piotr Tabor
3423a949c0
Update go for 3.5: 1.15 -> 1.16.(3).
...
https://github.com/etcd-io/etcd/issues/12732
2021-04-19 16:50:54 +02:00
Piotr Tabor
06ba0fc5a2
Merge pull request #12846 from pyiyun/fix-snaptmpfile-bug
...
etcdserver: remove temp files in snap dir when etcdserver starting
2021-04-17 12:58:46 +02:00
Piotr Tabor
17b982382e
Fix TestSnapshotV3RestoreMultiMemberAdd flakes (leaks)
...
- most important: unix's socket transport should not keep idle
connections. For top-level Transport we close them using:
f3c518025e/server/etcdserver/api/rafthttp/transport.go (L226)
but currently we don't have access to close them witing the nest (unix) transport. Short idle deadline is good enough.
- Use dialContext (instead of dial) to make sure context is passed down the stack
- Make sure Context is cancelled as soon as the operation is done in pipeline
- nit: use dedicated method to yeld goroutines.
Tested with:
```
d=$(date +"%Y%m%d_%H%M")
(cd tests && go test --timeout=60m ./integration/snapshot -run TestSnapshotV3RestoreMultiMemberAdd -v --count=180 2>&1 | tee log_${d}.log)
```
There were transports & cmux leaked:
```
leak.go:118: Test appears to have leaked a Transport:
internal/poll.runtime_pollWait(0x7f6c5c3784c8, 0x72, 0xffffffffffffffff)
/usr/lib/google-golang/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc003296298, 0x72, 0x0, 0x18, 0xffffffffffffffff)
/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc003296280, 0xc0031f60a8, 0x18, 0x18, 0x0, 0x0, 0x0)
/usr/lib/google-golang/src/internal/poll/fd_unix.go:166 +0x1d5
net.(*netFD).Read(0xc003296280, 0xc0031f60a8, 0x18, 0x18, 0x18, 0xc0009056e2, 0x203000)
/usr/lib/google-golang/src/net/fd_posix.go:55 +0x4f
net.(*conn).Read(0xc000010258, 0xc0031f60a8, 0x18, 0x18, 0x0, 0x0, 0x0)
/usr/lib/google-golang/src/net/net.go:183 +0x91
github.com/soheilhy/cmux.(*bufferedReader).Read(0xc0003d24e0, 0xc0031f60a8, 0x18, 0x18, 0xc0003d24d0, 0xc0009056e2, 0xc000278400)
/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/buffer.go:53 +0x12d
github.com/soheilhy/cmux.hasHTTP2Preface(0x1367e20, 0xc0003d24e0, 0x7f6c5c699f40)
/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/matchers.go:195 +0x8a
github.com/soheilhy/cmux.matchersToMatchWriters.func1(0x7f6c5c699f40, 0xc000010258, 0x1367e20, 0xc0003d24e0, 0xc000010258)
/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:128 +0x39
github.com/soheilhy/cmux.(*cMux).serve(0xc003228690, 0x138c410, 0xc000010258, 0xc00327f740, 0xc0059ba860)
/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:192 +0x1e7
created by github.com/soheilhy/cmux.(*cMux).Serve
/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:179 +0x191
internal/poll.runtime_pollWait(0x7f6c5c60f3f0, 0x72, 0xffffffffffffffff)
/usr/lib/google-golang/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc000d53018, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc000d53000, 0xc000cfd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/lib/google-golang/src/internal/poll/fd_unix.go:166 +0x1d5
net.(*netFD).Read(0xc000d53000, 0xc000cfd000, 0x1000, 0x1000, 0x3, 0x3, 0x1000000000001)
/usr/lib/google-golang/src/net/fd_posix.go:55 +0x4f
net.(*conn).Read(0xc00031a570, 0xc000cfd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/lib/google-golang/src/net/net.go:183 +0x91
net/http.(*persistConn).Read(0xc00093b320, 0xc000cfd000, 0x1000, 0x1000, 0x577750, 0x60, 0x0)
/usr/lib/google-golang/src/net/http/transport.go:1933 +0x77
bufio.(*Reader).fill(0xc005702fc0)
/usr/lib/google-golang/src/bufio/bufio.go:101 +0x108
bufio.(*Reader).Peek(0xc005702fc0, 0x1, 0xc00077c660, 0xc003b082a0, 0xc000d08de0, 0x5ae586, 0x11dd6c0)
/usr/lib/google-golang/src/bufio/bufio.go:139 +0x4f
net/http.(*persistConn).readLoop(0xc00093b320)
/usr/lib/google-golang/src/net/http/transport.go:2094 +0x1a8
created by net/http.(*Transport).dialConn
/usr/lib/google-golang/src/net/http/transport.go:1754 +0xdaa
net/http.(*persistConn).writeLoop(0xc00093b320)
/usr/lib/google-golang/src/net/http/transport.go:2393 +0xf7
created by net/http.(*Transport).dialConn
/usr/lib/google-golang/src/net/http/transport.go:1755 +0xdcf
sync.runtime_Semacquire(0xc0059ba868)
/usr/lib/google-golang/src/runtime/sema.go:56 +0x45
sync.(*WaitGroup).Wait(0xc0059ba860)
/usr/lib/google-golang/src/sync/waitgroup.go:130 +0x65
github.com/soheilhy/cmux.(*cMux).Serve.func1(0xc003228690, 0xc0059ba860)
/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:158 +0x56
github.com/soheilhy/cmux.(*cMux).Serve(0xc003228690, 0x13698c0, 0xc00377a0f0)
/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:173 +0x115
go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers.func1(0xc0007cc360, 0x122b75f)
/home/ptab/corp/etcd/server/embed/etcd.go:518 +0x2b9
go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers.func3(0xc00036d080, 0xc0059330a0)
/home/ptab/corp/etcd/server/embed/etcd.go:549 +0x182
created by go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers
/home/ptab/corp/etcd/server/embed/etcd.go:543 +0x73a
--- FAIL: TestSnapshotV3RestoreMultiMemberAdd (17.74s)
```
2021-04-16 20:17:28 +02:00
pyiyun
28a490b09c
etcdserver: remove temp files in snap dir when etcdServer starting
...
When etcd exits abnormally, tmp files will remain in snap dir, so clean up tmp files in snap dir when etcdserver starting.
Fixes #12837
2021-04-16 20:30:04 +08:00
Piotr Tabor
eafbc8c57e
Update zap logging dependency.
...
In particular bring up zapgrpc V2 code:
89e382035d
https://pkg.go.dev/google.golang.org/grpc/grpclog#LoggerV2
2021-04-14 12:15:48 +02:00
大可
5db0070150
Update doc.go
2021-04-13 22:22:42 +08:00
大可
d7e971e8d8
client: fix doc typo
2021-04-13 16:33:24 +08:00
Piotr Tabor
3bb7acc8cf
Migrate dependencies pkg/foo -> client/pkg/foo
2021-04-07 00:38:47 +02:00
Piotr Tabor
0b81fdf418
Move pkg/* used by client to client/pkg.
2021-04-05 23:47:27 +02:00
Piotr Tabor
f290ab2e60
Update dependecies:
...
github.com/grpc-ecosystem/grpc-gateway v1.14.6 -> grpc-gateway v1.16.0
golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e->v0.0.0-20210220033141-f8bda1e9f3ba
2021-03-27 20:48:33 +01:00
Piotr Tabor
a60676686b
Update dep: grpc: 1.32.0 -> 1.36.0.
2021-03-24 22:27:55 +01:00
Piotr Tabor
45fb7b41d6
Update dep: github.com/golang/protobuf v1.3.5 -> v1.5.1
...
Thanks to https://go-review.googlesource.com/c/protobuf/+/300869/ its
feasible now.
2021-03-24 22:27:12 +01:00
Piotr Tabor
18321a0c85
Merge pull request #12786 from ptabor/20210318-debug-env
...
client: Bring back ETCD_CLIENT_DEBUG variable interpretation.
2021-03-18 21:55:51 +01:00
Piotr Tabor
9312d1b077
clientv3: Bring back ETCD_CLIENT_DEBUG variable interpretation.
...
env ETCD_CLIENT_DEBUG supports log levels (debug, info, warn, error, dpanic, panic, fatal).
Only when set, overrides application-wide grpc logging settings.
2021-03-18 18:43:07 +01:00
wpedrak
dac6e37ea1
*: over 20 staticcheck fixes
2021-03-18 15:06:17 +01:00
Piotr Tabor
a47c18d30a
Fix 2 remaining 'defer AfterTest' calls.
2021-03-13 23:41:29 +01:00
Piotr Tabor
783e26fcdf
Fix gogo to 1.3.2 in api/go.mod.
2021-03-11 19:34:34 +01:00
wpedrak
2c2456bf3d
*: partial staticcheck fix
2021-03-10 14:13:38 +00:00
Piotr Tabor
fb1d48e98e
Integration tests: Use BeforeTest(t) instead of defer AfterTest().
...
Thanks to this change, a single method BeforeTest(t) can handle
before-test logic as well as registration of cleanup code
(t.Cleanup(func)).
2021-03-09 18:19:51 +01:00
Piotr Tabor
a46a358577
--experimental-memory-mlock support
...
The flag protects etcd memory from being swapped out to disk.
This can happen in memory constrained systems where mmaped bbolt
area is natural condidate for swapping out.
This flag should provide better tail latency on the cost of higher RSS
ram usage. If the experiment is successful, the logic should get moved
into bbolt layer, where we can protect specific bbolt instances
(e.g. avoid protecting both during defragmentation).
2021-03-07 12:32:57 +01:00
Emily Selwood
9ebbf5f38b
Metrics example 404s - fix url
...
The metrics example link points to a file that appears to have moved. This change points it to what I think is the right place.
2021-03-04 08:17:48 +00:00
David Lanouette
6998c5641c
client v2: rename error var for revive
...
The revive tool complained durring the build. Error variable has been
renamed.
Fixes #12718
Signed-off-by: David Lanouette <David.Lanouette@GMail.com>
2021-03-01 17:09:16 -05:00
David Lanouette
7d02ce2073
client v2: check for empty request from the context
...
If the simpleHTTPClient.Do is called and the context has a nil request, return an error early.
Fixes #12718
Signed-off-by: David Lanouette <David.Lanouette@GMail.com>
2021-03-01 16:46:30 -05:00
Piotr Tabor
54b87505a3
Remove dead legacy logger code.
2021-02-26 09:13:09 +01:00
Piotr Tabor
45b1e6b470
ClientV3: Ordering: Fix the ordering test such it does not fail.
...
The test depended on very subtle timing semantic and on properties of
'copied' clients.
https://travis-ci.com/github/etcd-io/etcd/jobs/486191449
Examplar failure:
```
{"level":"warn","ts":"2021-02-25T12:34:47.894Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0000d6fc0/#initially=[unix://localhost:86269902489114839060]","attempt":1,"error":"rpc error: code = Unavailable desc = etcdserver: rpc not supported for learner"}
{"level":"warn","ts":"2021-02-25T12:34:48.163Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc00035a000/#initially=[unix://localhost:78285857058450835940]","attempt":0,"error":"rpc error: code = FailedPrecondition desc = etcdserver: not leader"}
{"level":"info","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = Canceled desc = context canceled"}
{"level":"info","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"info","ts":"2021-02-25T12:34:50.255Z","caller":"v3/maintenance.go:219","msg":"completed snapshot read; closing"}
{"level":"info","ts":"2021-02-25T12:34:51.717Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:52.017Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = Canceled desc = context canceled"}
{"level":"info","ts":"2021-02-25T12:34:52.018Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:53.018Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
--- FAIL: TestEndpointSwitchResolvesViolation (10.12s)
ordering_util_test.go:81: failed to resolve order violation etcdclient: no cluster members have a revision higher than the previously received revision
```
2021-02-25 22:15:13 +01:00
Piotr Tabor
60d5159091
version: bump up to 3.5.0-alpha.0
2021-02-24 19:55:45 +00:00
Piotr Tabor
1a9c81abda
Update grpc dependency to 1.32.
...
Simplify grpc testing infrastructure to align with upstream changes.
2021-02-23 11:31:50 +01:00
Piotr Tabor
4a1c24556c
clientv3: PS: Replace balancer with upstream grpc solution
...
Addresses comments from: https://github.com/etcd-io/etcd/pull/12671#pullrequestreview-593942302
2021-02-23 10:03:15 +01:00
Piotr Tabor
b67ed4e4aa
Merge pull request #12671 from ptabor/20210207-grpc-del-balancer
...
clientv3: Replace balancer with upstream grpc solution
2021-02-17 22:45:13 +01:00
Piotr Tabor
a836a8045b
Get rid of legacy client/v3/naming API.
...
Update grpcproxy to use the new abstractions.
2021-02-09 11:56:28 +01:00
Piotr Tabor
0b75fede64
Replace client/v3/balancer with standard components: resolver + round_robin LB
...
This commit significantly reduces volume of custom code
in etcd client v3, while preserving full existing functionality.
2021-02-08 18:50:31 +01:00
limeng01
8feb55f65c
*: implement Endpoint Watch and new Resolver
2021-02-08 20:05:45 +08:00
Piotr Tabor
dae29bb719
Merge pull request #12667 from lemonlinger/implement-endpoint-manager-and-resolver
...
Implement Update method for EndpointManager
2021-02-04 17:25:51 +01:00
limeng01
571ed502d4
endpoints: implement Update method for EndpointManager.
...
- Add integration test for endpoints and resolver.
2021-02-04 23:30:22 +08:00
Sahdev Zala
66777cf1e8
Doc: fix inconsistency and broken path
...
1) Fix the path for rpc errors
2) The README says 2 types of client errors vs doc mentioning 3
types of client errors. grpc erros should be considered single time
as covered in the README.
2021-02-03 12:02:49 -05:00
Danny Hermes
ea34f8dbc6
Prepare `balancer` interfaces for `>=google.golang.org/grpc@1.30.0` upgrade.
2021-02-02 15:54:33 -06:00
Piotr Tabor
5d7c1db3a9
Introduce grpc-1.30+ compatible client/v3/naming API.
...
This is not yet implementation, just API and tests to be filled
with implementation in next CLs,
tracked by: https://github.com/etcd-io/etcd/issues/12652
We propose here 3 packages:
- clientv3/naming/endpoints ->
That is abstraction layer over etcd that allows to write, read &
watch Endpoints information. It's independent from GRPC API. It hides
the storage details.
- clientv3/naming/endpoints/internal ->
That contains the grpc's compatible Update class to preserve the
internal JSON mashalling format.
- clientv3/naming/resolver ->
That implements the GRPC resolver API, such that etcd can be
used for connection.Dial in grpc.
Please see the grpc_naming.md document changes & grpcproxy/cluster.go
new integration, to see how the new abstractions work.
2021-01-30 12:32:19 +01:00
Piotr Tabor
92458228e1
Merge pull request #12423 from Murphylu1993/master
...
client: replace dial with dialContext
2021-01-25 20:28:40 +01:00
Piotr Tabor
84b5b87fb2
Make test.sh scripts OSX/BSD compatible:
...
- build & test scripts deprecated. Call *.sh variants.
This will avoid delete the symlinks and get rid of
subtle dependency on 'sed --follow-symlinks' on OsX/BSD sed.
- Fix parameters to mktemp
2021-01-12 16:29:00 +01:00
Sahdev Zala
c632042bb9
Merge pull request #12489 from zhijianli88/cleanup-tmpfiles
...
Cleanup tmpfiles
2021-01-07 12:26:04 -05:00
Sahdev Zala
a4570a60e7
Merge pull request #12549 from bbiao/bugfix/refresh_token
...
client/v3: clear auth token when encounter ErrInvalidAuthToken
2020-12-29 22:08:14 -05:00
bbiao
af4ef4ec04
client/v3: clear auth token when encounter ErrInvalidAuthToken
...
Old etcdserver which have not apply pr of #12165 will check auth token
even if the request is an Authenticate request.
If the client has a invalid auth token, it will not able to update it's
token, since the Authenticate has a invalid auth token.
This fix clear the auth token when encounter an ErrInvalidAuthToken to
talk with old version etcd servers.
Fix #12385 with #12165 and #12264
2020-12-27 11:08:22 +08:00
ianwoolf
fc5b5fa31e
fix godoc link in clientv3 readme
2020-12-23 10:59:13 +08:00
琚致远
bd97953de3
Update README.md
2020-12-21 17:40:40 +08:00
琚致远
549bff5980
docs: update client v3's README
2020-12-20 00:06:10 +08:00
Piotr Tabor
4d173231d1
Documentation: Developer's docs about etcd-3.5.x modules
2020-12-02 10:00:09 +01:00
Li Zhijian
fb3f91c141
test: cleanup temporary file for TestConfigFromFile
...
Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
2020-11-25 17:40:43 +08:00
Piotr Tabor
eeafcef0d2
Use "v3.5.0-pre" to reference within-etcd modules
...
instead of v3.0.0-000101010000000-00000000000,
that might be misleading as we don't develop etcd v3.0.0 any longer.
This version is a virtual version and is not supposed to be tagged
within the repository. We should tag real versions like: 3.5.0-alpha.0.
Please notice that go.etcd.io/etcd/client/v2 will be versioned as `v2.305.0-pre`.
The reason is that client v2 must have v2 version. I propose a
convention to envode the major version as 100x in minor version to make
the association to the underlying repository clear, staying within v2
version family.
The change was generated using:
```
DRY_RUN=false TARGET_VERSION="v3.5.0-pre" ./scripts/release_mod.sh update_versions
```
2020-11-04 18:28:43 +01:00
meadlai
ec37e15caf
Update README.md
2020-10-28 14:34:03 +08:00
luqi
ed81d2e2db
client: replace dial with dialContext
2020-10-27 12:43:14 +08:00
Piotr Tabor
45b007b8b4
contrib,clientv3: Move contrib/recipies to clientv3/experimental/recipies/...
...
Recipies is set of patterns / primitives implementation on top of clientv3.
It's used by integration tests. It shouldn't be considered "server" code.
2020-10-22 11:10:07 +02:00
Piotr Tabor
e33c6dd9df
client/v3: Rename of imports
2020-10-20 10:13:06 +02:00
Piotr Tabor
cf795c0ae6
client/v3: Create and update go.mod
2020-10-20 10:09:12 +02:00
Piotr Tabor
8a7f15511c
clientv3: Move to client/v3 (just file move)
...
Mechanical:
% git mv clientv3 client/v3
% git mv client/mock client/v3/
2020-10-20 10:08:35 +02:00
Piotr Tabor
b7f0f52a16
*: Refresh of dependencies ( #12399 )
...
This PR focuses on dependencies that have new stable versions.
2020-10-15 15:32:00 -07:00
Piotr Tabor
45e4306508
client: Move client to client/v2 as a module.
...
We make v2 client code a module go.etcd.io/etcd/client/v2.
Pretty mechanical change that can be summarized as:
mkdir client/v2
cd client/v2 && git mod init go.etcd.io/etcd/client/v2
git mv client/*.go client/v2/
find -name '*.go' | xargs sed -i --follow-symlinks 's|/v3/client["]|/client/v2\"|g'
+ fixing changelog, bom, go.mod etc.
2020-10-15 14:39:43 +02:00
Piotr Tabor
de55bb6331
pkg: Rename imports after making 'pkg' a module
...
find -name '*.go' | xargs sed --follow-symlinks -i 's|go.etcd.io/etcd/v3/pkg/|go.etcd.io/etcd/pkg/v3/|g'
go fmt ./...
2020-10-13 00:09:27 +02:00
Piotr Tabor
28036db6f0
Mechanical: Move mock packages out of ./pkg
...
Mechanical execution of:
```
git mv ./pkg/mock/mockserver ./client/pkg/mock
git mv ./pkg/mock/{mockstorage,mockstore,mockwait} ./server/pkg/mock
```
The packages depend on etcd API / protos - so they are NOT etcd-dependencies.
In such situation thay should be placed in 'pkg' folder.
2020-10-12 23:58:09 +02:00
Piotr Tabor
c5ccebf792
tests/integration: Simplify the testMain for examples.
...
We introduce a LazyCluster abstraction (instead of copy-pasted logic)
that makes clusters to be created only if there are runnable tests
in need for the infrastructure.
2020-10-07 15:42:35 +02:00
Piotr Tabor
73b92fe688
tests: Make examples (for not client) to be both: documentation and integration-runnable
...
This CL tries to connect 2 objectives:
- Examples should be close (the same package) to the original code,
such that they can participate in documentation.
- Examples should be runnable - such that they are not getting out of
sync with underlying API/implementation.
In case of etcd-client, the examples are assuming running 'integration'
style, i.e. thay do connect to fully functional etcd-server.
That would lead to a cyclic dependencies between modules:
- server depends on client (as client need to be lightweight)
- client (for test purposes) depend on server.
Go modules does not allow to distingush testing dependency from
prod-code dependency.
Thus to meet the objective:
- The examples are getting executed within testing/integration packages against real etcd
- The examples are symlinked to 'unit' tests, such that they included in documentation.
- Long-term the unit examples should get rewritten to use 'mocks' instead of real integration tests.
2020-10-07 15:38:40 +02:00
Piotr Tabor
3153038ffb
tests/integration: Moving integration tests to tests/integration directory
...
git mv integration ./tests
git mv client/integration/* ./tests/integration/client
git mv clientv3/integration/* ./tests/integration/clientv3
git mv client/example_keys_test.go client/main_test.go tests/integration/client/examples
git mv clientv3/concurrency/*_test.go tests/integration/clientv3/concurrency
git mv etcdserver/api/v2store/store_v2v3_test.go etcdserver/api/v2v3/*_test.go tests/integration/v2store
git mv tests/integration/v2store/store_v2v3_test.go tests/integration/v2store/store_v2v3.go
git mv tests/integration/v2store/store_test.go tests/integration/v2store/store_v2v3_test.go
git mv etcdserver/api/v2store/store_test.go tests/integration/v2store
git mv etcdserver/api/v2store/store_v2_test.go tests/integration/v2store
git mv proxy/grpcproxy/*_test.go tests/integration/proxy/grpcproxy
git mv ./clientv3/snapshot/testdata ./clientv3/snapshot/*_test.go ./tests/integration/snapshot
2020-10-07 15:36:21 +02:00
Piotr Tabor
28f2b07623
*: Update references to code moved to the api/ dir.
...
Follow up to file-moves done in the previous commit.
The commit contains purely mechanical consequences of execution (apart
of scripts/genproto.sh):
% find ./ -name '*.go' | xargs sed --follow-symlinks -i 's|v3/etcdserver/api/v3rpc/rpctypes|v3/api/v3rpc/rpctypes|g'
% find ./ -name '*.go' | xargs sed --follow-symlinks -i 's|v3/version|v3/api/version|g'
% find ./ -name '*.go' | xargs sed --follow-symlinks -i 's|v3/mvcc/mvccpb|v3/api/mvccpb|g'
% find ./ -name '*.go' | xargs sed --follow-symlinks -i 's|v3/etcdserver/etcdserverpb|v3/api/etcdserverpb|g'
% find ./ -name '*.go' | xargs sed --follow-symlinks -i 's|v3/etcdserver/api/membership/membershippb|v3/api/membershippb|g'
% find ./ -name '*.go' | xargs sed --follow-symlinks -i 's|v3/auth/authpb|v3/api/authpb|g'
% find ./ -name '*.proto' -o -name '*.md' | xargs -L 1 sed --follow-symlinks -i 's|/mvcc/mvccpb/kv.proto|/api/mvccpb/kv.proto|g'
% find ./ -name '*.proto' -o -name '*.md' | xargs -L 1 sed --follow-symlinks -i 's|/auth/authpb/auth.proto|/api/authpb/auth.proto|g'
% find ./ -name '*.proto' -o -name '*.md' | xargs -L 1 sed --follow-symlinks -i 's|/etcdserver/api/membership/membershippb/membership.proto|/api/membershippb/membership.proto|g'
I also modified manually paths in scripts/genproto.sh.
% go fmt ./...
2020-10-06 11:56:16 +02:00
Piotr Tabor
528f5315d6
auth: Fix "race" - auth unit tests leaking goroutines
...
- We were leaking goroutines in auth-test
- The go-routines were depending / modifying global test environment
variables (simpleTokenTTLDefault) leading to races
Removed the leaked go-routines, and expanded 'auth' package to
be covered we leaked go-routines detection.
2020-10-03 19:38:30 +02:00
Piotr Tabor
fb2e96e651
vendor: Get rid of ./vendor cleanup
...
Updated scripts and documentation to not recommend vendoring.
Implemented best practices for tools installation.
Performed multiple tests to confirm its not breaking any workflows and
has no negative performance impact. Rather see 3x speedup.
1. PASSES="fmt unit integration e2e functional" ./test
2. ./scripts/updatebom.sh
3. ./scripts/updatedep.sh
4. ./scripts/genproto.sh - works - ca be simplified - in follow up PR
5. Installation without explicit GOPATH:
```
% unset GOPATH
% [sudo] rm -rf ~/go
% git clone https://github.com/etcd-io/etcd.git
% time ./build
go: downloading google.golang.org/grpc v1.26.0
go: downloading github.com/jonboulle/clockwork v0.1.0
go: downloading github.com/prometheus/client_golang v1.0.0
go: downloading github.com/soheilhy/cmux v0.1.4
go: downloading github.com/gogo/protobuf v1.2.1
go: downloading sigs.k8s.io/yaml v1.1.0
go: downloading golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2
go: downloading github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903
go: downloading go.etcd.io/bbolt v1.3.5
go: downloading go.uber.org/zap v1.15.0
go: downloading golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc
go: downloading github.com/golang/protobuf v1.3.2
go: downloading github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8
go: downloading github.com/beorn7/perks v1.0.0
go: downloading github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4
go: downloading github.com/coreos/go-systemd/v22 v22.0.0
go: downloading gopkg.in/yaml.v2 v2.2.2
go: downloading github.com/coreos/go-semver v0.2.0
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7
go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: downloading github.com/google/uuid v1.0.0
go: downloading github.com/modern-go/reflect2 v1.0.1
go: downloading github.com/prometheus/common v0.4.1
go: downloading github.com/spf13/cobra v0.0.3
go: downloading github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2
go: downloading github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c
go: downloading github.com/spf13/pflag v1.0.1
go: downloading github.com/json-iterator/go v1.1.7
go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: downloading github.com/google/btree v1.0.0
go: downloading go.uber.org/atomic v1.6.0
go: downloading github.com/prometheus/procfs v0.0.2
go: downloading go.uber.org/multierr v1.5.0
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5
go: downloading github.com/grpc-ecosystem/grpc-gateway v1.9.5
go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4
go: downloading github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: downloading google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading golang.org/x/text v0.3.3
go: downloading github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5
go: downloading github.com/bgentry/speakeasy v0.1.0
go: downloading gopkg.in/cheggaaa/pb.v1 v1.0.25
go: downloading github.com/urfave/cli v1.20.0
go: downloading github.com/mattn/go-runewidth v0.0.2
./build 8.22s user 2.31s system 117% cpu 8.961 total
```
Before:
```
% git clone https://github.com/etcd-io/etcd.git && cd etcd && time ./build
Cloning into 'etcd'...
remote: Enumerating objects: 97872, done.
remote: Total 97872 (delta 0), reused 0 (delta 0), pack-reused 97872
Receiving objects: 100% (97872/97872), 58.97 MiB | 19.85 MiB/s, done.
Resolving deltas: 100% (63091/63091), done.
./build 34.97s user 4.15s system 236% cpu 16.555 total
```
6. Rebuild without changes:
```
% time ./build
./build 1.43s user 0.83s system 168% cpu 1.336 total
```
7. Instantation of vendor directory (assuming ./build loaded them to
$GOPATH/pkg):
```
time go mod vendor
go: downloading github.com/inconshreveable/mousetrap v1.0.0
go: downloading github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa
go: downloading github.com/creack/pty v1.1.11
go: downloading github.com/etcd-io/gofail v0.0.0-20190801230047-ad7f989257ca
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1
go mod vendor 0.51s user 0.44s system 110% cpu 0.861 total
```
8. Fresh instantation of vendor:
```
% rm -rf vendor
% [sudo] rm -rf ~/go
% time go mod vendor
go: downloading github.com/coreos/go-systemd/v22 v22.0.0
go: downloading github.com/spf13/cobra v0.0.3
go: downloading github.com/prometheus/client_golang v1.0.0
go: downloading golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7
go: downloading github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4
go: downloading github.com/gogo/protobuf v1.2.1
go: downloading sigs.k8s.io/yaml v1.1.0
go: downloading google.golang.org/grpc v1.26.0
go: downloading github.com/urfave/cli v1.20.0
go: downloading go.uber.org/zap v1.15.0
go: downloading github.com/spf13/pflag v1.0.1
go: downloading github.com/soheilhy/cmux v0.1.4
go: downloading github.com/json-iterator/go v1.1.7
go: downloading github.com/coreos/go-semver v0.2.0
go: downloading github.com/prometheus/common v0.4.1
go: downloading github.com/prometheus/procfs v0.0.2
go: downloading go.uber.org/atomic v1.6.0
go: downloading github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5
go: downloading github.com/golang/protobuf v1.3.2
go: downloading github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa
go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4
go: downloading github.com/modern-go/reflect2 v1.0.1
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading go.uber.org/multierr v1.5.0
go: downloading github.com/creack/pty v1.1.11
go: downloading github.com/mattn/go-runewidth v0.0.2
go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: downloading golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc
go: downloading golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5
go: downloading github.com/jonboulle/clockwork v0.1.0
go: downloading gopkg.in/yaml.v2 v2.2.2
go: downloading github.com/etcd-io/gofail v0.0.0-20190801230047-ad7f989257ca
go: downloading github.com/grpc-ecosystem/grpc-gateway v1.9.5
go: downloading github.com/google/btree v1.0.0
go: downloading google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
go: downloading github.com/beorn7/perks v1.0.0
go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: downloading github.com/google/uuid v1.0.0
go: downloading golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2
go: downloading github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8
go: downloading go.etcd.io/bbolt v1.3.5
go: downloading golang.org/x/text v0.3.3
go: downloading gopkg.in/cheggaaa/pb.v1 v1.0.25
go: downloading github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2
go: downloading github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: downloading github.com/inconshreveable/mousetrap v1.0.0
go: downloading github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading github.com/bgentry/speakeasy v0.1.0
go: downloading github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1
go mod vendor 3.62s user 1.30s system 127% cpu 3.854 total
```
9. Size of the repository - before: 39M, after: 18M
Before:
```
% time git clone https://github.com/etcd-io/etcd.git
Cloning into 'etcd'...
remote: Enumerating objects: 97872, done.
remote: Total 97872 (delta 0), reused 0 (delta 0), pack-reused 97872
Receiving objects: 100% (97872/97872), 58.97 MiB | 20.53 MiB/s, done.
Resolving deltas: 100% (63091/63091), done.
git clone https://github.com/etcd-io/etcd.git 4.66s user 1.02s system 93% cpu 6.068 total
% du -h --exclude .git -d 1
944K ./clientv3
108K ./etcdmain
5.4M ./Documentation
384K ./security
384K ./mvcc
28K ./.github
8.0K ./version
144K ./contrib
240K ./proxy
2.5M ./etcdserver
112K ./embed
536K ./integration
332K ./tools
116K ./lease
108K ./logos
896K ./tests
960K ./raft
216K ./client
52K ./scripts
100K ./hack
464K ./etcdctl
3.0M ./pkg
620K ./functional
136K ./wal
152K ./auth
21M ./vendor
39M
```
After:
```
% time git clone https://github.com/ptabor/etcd.git -b 20200908-no-vendor
Cloning into 'etcd'...
remote: Enumerating objects: 38, done.
remote: Counting objects: 100% (38/38), done.
remote: Compressing objects: 100% (37/37), done.
remote: Total 98489 (delta 10), reused 8 (delta 1), pack-reused 98451
Receiving objects: 100% (98489/98489), 59.23 MiB | 21.26 MiB/s, done.
Resolving deltas: 100% (63572/63572), done.
git clone https://github.com/ptabor/etcd.git -b 20200908-no-vendor 5.56s user 1.05s system 105% cpu 6.260 total
% du -h --exclude .git -d 1
944K ./clientv3
108K ./etcdmain
5.4M ./Documentation
384K ./security
384K ./mvcc
28K ./.github
8.0K ./version
144K ./contrib
240K ./proxy
2.5M ./etcdserver
112K ./embed
536K ./integration
332K ./tools
116K ./lease
108K ./logos
896K ./tests
960K ./raft
216K ./client
56K ./scripts
100K ./hack
464K ./etcdctl
3.0M ./pkg
620K ./functional
136K ./wal
152K ./auth
19M .
```
2020-09-15 14:38:54 +02:00
Brandon Philips
d88d765ba4
Documentation, CHANGELOG: use new go.etcd.io/etcd/v3 pkg
...
Use the new package path in the docs and announce it in the CHANGELOG
2020-04-28 22:02:19 +00:00
Brandon Philips
96cce208c2
go.mod: use go.etcd.io/etcd/v3 versioning
...
This change makes the etcd package compatible with the existing Go
ecosystem for module versioning.
Used this tool to update package imports:
https://github.com/KSubedi/gomove
2020-04-28 00:57:35 +00:00
sfzhu93
467e08c32a
in multiple packages: fixed goroutine leak bugs in tests (cont.d) ( #11570 )
2020-01-30 10:55:59 -08:00
Gyuho Lee
1caaa9ed4a
test: test update for Go 1.12.5 and related changes
...
Update to Go 1.12.5 testing. Remove deprecated unused and gosimple
pacakges, and mask staticcheck 1006. Also, fix unconvert errors related
to unnecessary type conversions and following staticcheck errors:
- remove redundant return statements
- use for range instead of for select
- use time.Since instead of time.Now().Sub
- omit comparison to bool constant
- replace T.Fatal and T.Fatalf in tests with T.Error and T.Fatalf respectively because the goroutine calls T.Fatal must be called in the same goroutine as the test
- fix error strings that should not be capitalized
- use sort.Strings(...) instead of sort.Sort(sort.StringSlice(...))
- use he status code of Canceled instead of grpc.ErrClientConnClosing which is deprecated
- use use status.Errorf instead of grpc.Errorf which is deprecated
Related #10528 #10438
2019-06-05 17:02:05 -04:00
Gyuho Lee
34bd797e67
*: revert module import paths
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-05-28 15:39:35 -07:00
Xiang Li
caee28a88e
Merge pull request #10666 from mkumatag/fix_tests
...
Fix tests for latest golang
2019-05-03 11:18:58 -07:00
shivaramr
9150bf52d6
go modules: Fix module path version to include version number
2019-04-26 15:29:50 -07:00
Davanum Srinivas
daee668b75
client: Switch to case sensitive unmarshalling to be compatible with ugorji
...
Using lessons learned from k8s changes:
https://github.com/kubernetes/kubernetes/pull/65034
Change-Id: Ia17a8f94ae6ed00c5af2595c2b48d3c9a0344427
2019-04-23 16:54:44 -04:00
Davanum Srinivas
86e3481ba2
scripts: Remove generated code and script
...
Change-Id: Iac4601443bcad71920fd96b97bfe21c16116577a
2019-04-23 16:54:44 -04:00
Davanum Srinivas
90108a2e61
client: Replace ugorji/codec with json-iterator/go
...
We need to use the stdlib-compatible one that is case-sensitive, etc
Change-Id: Id0df573a70e09967ac7d8c0a63d99d6a49ce82f1
2019-04-23 16:54:44 -04:00
Manjunath A Kumatagi
867b45d865
client: Fix tests for latest golang
2019-04-22 08:14:10 -05:00
zhoulin xie
a943ad0ee4
client/keys_bench_test.go: Fix some misspells
...
Signed-off-by: zhoulin xie <zhoulin.xie@daocloud.io>
2019-02-28 14:36:06 -05:00
Sophos
6543273666
client: generate new keys and remove yynn2 = 0
2019-02-25 12:07:40 +08:00
Sam Batschelet
f3fbedc88f
client: update generated ugorji codec, manual remove "yynn2=0"
...
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2018-12-21 18:16:55 -05:00
Sam Batschelet
fa35126ef8
*: add client support for discovery-srv-name
...
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2018-11-09 10:13:04 -05:00
vcaesar
fc7ef659cc
client,clientv3: update client docs to "go.etcd.io"
2018-08-30 19:26:12 -04:00
Gyuho Lee
0ef9ef3c74
*: rerun "gofmt"
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2018-08-28 18:25:39 -07:00
Gyuho Lee
d37f1521b7
*: update import paths to "go.etcd.io/etcd"
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2018-08-28 17:47:55 -07:00
Gyuho Lee
42bef8460c
*: clean up code format
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-07-21 16:34:56 -07:00
Gyuho Lee
91664ec5ac
client: manually delete "yynn2 = 0"
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-04-09 13:42:50 -07:00
Gyuho Lee
13c440dea4
client: regenerate with f3cacc17c85ecb7f1b6a9e373ee85d1480919868
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-04-09 13:42:50 -07:00
Gyuho Lee
ebba9ed29e
client: regenerate
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-04-09 13:42:50 -07:00
Gyuho Lee
0850ccbf45
*: revert "internal/version" change
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-02-26 17:11:40 -08:00
Gyuho Lee
ebfba72715
client: fix "ineffassign" warning
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-02-26 10:07:18 -08:00
Gyuho Lee
37546f74ab
*: move "version" to "internal/version"
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-01-29 10:00:20 -08:00
Gyuho Lee
97cd2a41c8
client,clientv3: remove "cmd/vendor" in README.md
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-01-24 15:26:17 -08:00
Gyu-Ho Lee
75110dd839
*: fix naked returns
...
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-11-10 18:46:15 -08:00
Gyu-Ho Lee
81ca10f991
client/keys.generated.go: remove ineffassign yynn2 = 0
...
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-10-20 08:52:48 -07:00
Gyu-Ho Lee
1b2a62d9d0
client/keys.generated.go: remove redundant and: x.Expiration != nil
...
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-10-20 08:47:01 -07:00
Andrew Meissner
12a6efb74b
update: client
...
Updating the codec required codecgen to be reran on the client/keys.go file. This is the result of that run.
2017-10-20 09:23:23 -06:00
Gyu-Ho Lee
0199bdc266
*: fix 'ineffassign' issues
...
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-10-03 10:14:33 -07:00
Gyu-Ho Lee
fbc7acde95
client: permute endpoints manually (for Go 1.9>)
...
To keep backward compatibility, use old algorithm of
rand.Rand.Perm.
Reference: caae0917bf (diff-d4a72c5ba8515eae95a093e0aec62635)
.
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-09-22 10:19:30 +09:00
Alexandre Beslic
8dc4833a3e
client: regenerate sources for etcd/client with new codec version
...
Major updates to ugorji/go changed the signature of some
methods, resulting in the build failing for etcd/client
with default installation of the codec.
We regenerate the sources using codecgen with the new version
to reflect on the new changes.
Fixes #8573
Signed-off-by: Alexandre Beslic <abeslic@abronan.com>
2017-09-19 15:14:58 +02:00