vitalif
/
etcd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
16,311 Commits
21 Branches
406 Tags
79 MiB
Commit Graph

243 Commits (d19758b50101e2ea24d40a2357246efe6169dcc2)

Author SHA1 Message Date
Piotr Tabor fb2e96e651 vendor: Get rid of ./vendor cleanup 
Updated scripts and documentation to not recommend vendoring.
Implemented best practices for tools installation.

Performed multiple tests to confirm its not breaking any workflows and
has no negative performance impact. Rather see 3x speedup.

1. PASSES="fmt unit integration e2e functional" ./test
2. ./scripts/updatebom.sh
3. ./scripts/updatedep.sh
4. ./scripts/genproto.sh - works - ca be simplified - in follow up PR
5. Installation without explicit GOPATH:

```
% unset GOPATH
% [sudo] rm -rf ~/go
% git clone https://github.com/etcd-io/etcd.git
% time ./build
go: downloading google.golang.org/grpc v1.26.0
go: downloading github.com/jonboulle/clockwork v0.1.0
go: downloading github.com/prometheus/client_golang v1.0.0
go: downloading github.com/soheilhy/cmux v0.1.4
go: downloading github.com/gogo/protobuf v1.2.1
go: downloading sigs.k8s.io/yaml v1.1.0
go: downloading golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2
go: downloading github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903
go: downloading go.etcd.io/bbolt v1.3.5
go: downloading go.uber.org/zap v1.15.0
go: downloading golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc
go: downloading github.com/golang/protobuf v1.3.2
go: downloading github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8
go: downloading github.com/beorn7/perks v1.0.0
go: downloading github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4
go: downloading github.com/coreos/go-systemd/v22 v22.0.0
go: downloading gopkg.in/yaml.v2 v2.2.2
go: downloading github.com/coreos/go-semver v0.2.0
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7
go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: downloading github.com/google/uuid v1.0.0
go: downloading github.com/modern-go/reflect2 v1.0.1
go: downloading github.com/prometheus/common v0.4.1
go: downloading github.com/spf13/cobra v0.0.3
go: downloading github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2
go: downloading github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c
go: downloading github.com/spf13/pflag v1.0.1
go: downloading github.com/json-iterator/go v1.1.7
go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: downloading github.com/google/btree v1.0.0
go: downloading go.uber.org/atomic v1.6.0
go: downloading github.com/prometheus/procfs v0.0.2
go: downloading go.uber.org/multierr v1.5.0
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5
go: downloading github.com/grpc-ecosystem/grpc-gateway v1.9.5
go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4
go: downloading github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: downloading google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading golang.org/x/text v0.3.3
go: downloading github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5
go: downloading github.com/bgentry/speakeasy v0.1.0
go: downloading gopkg.in/cheggaaa/pb.v1 v1.0.25
go: downloading github.com/urfave/cli v1.20.0
go: downloading github.com/mattn/go-runewidth v0.0.2
./build  8.22s user 2.31s system 117% cpu 8.961 total
```

Before:
```
% git clone https://github.com/etcd-io/etcd.git && cd etcd && time ./build
Cloning into 'etcd'...
remote: Enumerating objects: 97872, done.
remote: Total 97872 (delta 0), reused 0 (delta 0), pack-reused 97872
Receiving objects: 100% (97872/97872), 58.97 MiB | 19.85 MiB/s, done.
Resolving deltas: 100% (63091/63091), done.

./build  34.97s user 4.15s system 236% cpu 16.555 total
```

6. Rebuild without changes:

```
% time ./build
./build  1.43s user 0.83s system 168% cpu 1.336 total
```

7. Instantation of vendor directory (assuming ./build loaded them to
$GOPATH/pkg):

```
time go mod vendor
go: downloading github.com/inconshreveable/mousetrap v1.0.0
go: downloading github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa
go: downloading github.com/creack/pty v1.1.11
go: downloading github.com/etcd-io/gofail v0.0.0-20190801230047-ad7f989257ca
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1
go mod vendor  0.51s user 0.44s system 110% cpu 0.861 total
```

8. Fresh instantation of vendor:

```
% rm -rf vendor
% [sudo] rm -rf ~/go

% time go mod vendor
go: downloading github.com/coreos/go-systemd/v22 v22.0.0
go: downloading github.com/spf13/cobra v0.0.3
go: downloading github.com/prometheus/client_golang v1.0.0
go: downloading golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7
go: downloading github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4
go: downloading github.com/gogo/protobuf v1.2.1
go: downloading sigs.k8s.io/yaml v1.1.0
go: downloading google.golang.org/grpc v1.26.0
go: downloading github.com/urfave/cli v1.20.0
go: downloading go.uber.org/zap v1.15.0
go: downloading github.com/spf13/pflag v1.0.1
go: downloading github.com/soheilhy/cmux v0.1.4
go: downloading github.com/json-iterator/go v1.1.7
go: downloading github.com/coreos/go-semver v0.2.0
go: downloading github.com/prometheus/common v0.4.1
go: downloading github.com/prometheus/procfs v0.0.2
go: downloading go.uber.org/atomic v1.6.0
go: downloading github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5
go: downloading github.com/golang/protobuf v1.3.2
go: downloading github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa
go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4
go: downloading github.com/modern-go/reflect2 v1.0.1
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading go.uber.org/multierr v1.5.0
go: downloading github.com/creack/pty v1.1.11
go: downloading github.com/mattn/go-runewidth v0.0.2
go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: downloading golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc
go: downloading golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5
go: downloading github.com/jonboulle/clockwork v0.1.0
go: downloading gopkg.in/yaml.v2 v2.2.2
go: downloading github.com/etcd-io/gofail v0.0.0-20190801230047-ad7f989257ca
go: downloading github.com/grpc-ecosystem/grpc-gateway v1.9.5
go: downloading github.com/google/btree v1.0.0
go: downloading google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
go: downloading github.com/beorn7/perks v1.0.0
go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: downloading github.com/google/uuid v1.0.0
go: downloading golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2
go: downloading github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8
go: downloading go.etcd.io/bbolt v1.3.5
go: downloading golang.org/x/text v0.3.3
go: downloading gopkg.in/cheggaaa/pb.v1 v1.0.25
go: downloading github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2
go: downloading github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: downloading github.com/inconshreveable/mousetrap v1.0.0
go: downloading github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading github.com/bgentry/speakeasy v0.1.0
go: downloading github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1
go mod vendor  3.62s user 1.30s system 127% cpu 3.854 total
```

9. Size of the repository - before: 39M, after: 18M

Before:

```
% time git clone https://github.com/etcd-io/etcd.git
Cloning into 'etcd'...
remote: Enumerating objects: 97872, done.
remote: Total 97872 (delta 0), reused 0 (delta 0), pack-reused 97872
Receiving objects: 100% (97872/97872), 58.97 MiB | 20.53 MiB/s, done.
Resolving deltas: 100% (63091/63091), done.
git clone https://github.com/etcd-io/etcd.git  4.66s user 1.02s system 93% cpu 6.068 total

% du -h --exclude .git -d 1
944K	./clientv3
108K	./etcdmain
5.4M	./Documentation
384K	./security
384K	./mvcc
28K	./.github
8.0K	./version
144K	./contrib
240K	./proxy
2.5M	./etcdserver
112K	./embed
536K	./integration
332K	./tools
116K	./lease
108K	./logos
896K	./tests
960K	./raft
216K	./client
52K	./scripts
100K	./hack
464K	./etcdctl
3.0M	./pkg
620K	./functional
136K	./wal
152K	./auth
21M	./vendor
39M
```

After:
```
% time git clone https://github.com/ptabor/etcd.git -b 20200908-no-vendor
Cloning into 'etcd'...
remote: Enumerating objects: 38, done.
remote: Counting objects: 100% (38/38), done.
remote: Compressing objects: 100% (37/37), done.
remote: Total 98489 (delta 10), reused 8 (delta 1), pack-reused 98451
Receiving objects: 100% (98489/98489), 59.23 MiB | 21.26 MiB/s, done.
Resolving deltas: 100% (63572/63572), done.
git clone https://github.com/ptabor/etcd.git -b 20200908-no-vendor  5.56s user 1.05s system 105% cpu 6.260 total

% du -h --exclude .git -d 1
944K	./clientv3
108K	./etcdmain
5.4M	./Documentation
384K	./security
384K	./mvcc
28K	./.github
8.0K	./version
144K	./contrib
240K	./proxy
2.5M	./etcdserver
112K	./embed
536K	./integration
332K	./tools
116K	./lease
108K	./logos
896K	./tests
960K	./raft
216K	./client
56K	./scripts
100K	./hack
464K	./etcdctl
3.0M	./pkg
620K	./functional
136K	./wal
152K	./auth
19M	.
```
 2020-09-15 14:38:54 +02:00
Piotr Tabor c32180d772 tests/e2e,etcdctl,etcdmain: Fix go test --tags cov -v ./tests/e2e 
This CL fixes:
  COVERDIR=./coverage PASSES="build_cov" && go test --tags cov -v ./tests/e2e
and is part of the effort to make:
  COVERDIR=coverage PASSES="build_cov cov" ./test
fully pass.

The args passed to ./bin/etcd_test and ./bin/etcdctl_test binaries were
mismatched. The protocol of passing the arguments using
environment variables has been replaces with proper passing of flags.

How the measurement of coverage by e2e tests works:
  1. COVERDIR=./coverage PASSES="build_cov" are generating
./bin/etcd_test and ./bin/etcdctl_test binaries.

  2. These binaries are tests (as coverage can be computed only for
tests) [see ./main_test.go ./etcdctl/main_test.go], but this tests are
running the main logic of the server and uppon termination (or SIGTERM
signal) are writting proper .coverprofile files in the $COVERDIR folder.
The binaries used to take arguments using env variables, but its not
needed any longer. The binaries can consume any command line arguments
that either test (so --test.fooo) or the original binary can consume.

 3.  The tests/e2e (when compiled with the --tags cov) are starting the
_test binaries instead of the original binaries, such that the coverage
is being collected.
 2020-09-09 12:56:15 +02:00
Hitoshi Mitake 76539cee57 test: avoid non existing package for integration test 2020-08-03 00:13:47 +09:00
Gyuho Lee 6874b9a76b test: add "mod" to build tests 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2020-05-10 18:36:38 -07:00
Gyuho Lee e94201c9fd test: set mod flags for bom test 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2020-05-10 13:37:18 -07:00
Gyuho Lee 027c26a0fd test: test '-mod=mod' 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2020-05-10 12:39:11 -07:00
Gyuho Lee b6ea32ee0f test: ignore vendor directory in govet test 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2020-05-10 12:35:15 -07:00
Brandon Philips 03248a9045 bill-of-materials: updates for /etcd/v3 packaging 
bill-of-materials:
- update to new package names

test/updatebom.sh scripts:
- Update to the right package names
- Don't add bom tool to go.mod
 2020-04-28 00:57:52 +00:00
lucperkins 1be2f4b8e2 Documentation: Restructure directory to accommodate new site generation system 
Signed-off-by: lucperkins <lucperkins@gmail.com>
 2020-01-21 14:29:54 -08:00
Gyuho Lee d4d57c5422 test: skip govet shadow tests for now 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-09-06 12:51:10 -07:00
Gyuho Lee 158354755a test: output etcd server logs when functional tests fail 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-07 10:16:15 -07:00
Tobias Schottdorf 362dfb4d08 test: s/20m/30m/g 
Every other test build times out due to the 20 minute test timeout. I
doesn't seem like tests are actually hanging, it's more that 20 minutes
just isn't enough to run the tests any more.
 2019-06-20 23:44:25 +02:00
Gyuho Lee 1caaa9ed4a test: test update for Go 1.12.5 and related changes 
Update to Go 1.12.5 testing. Remove deprecated unused and gosimple
pacakges, and mask staticcheck 1006. Also, fix unconvert errors related
to unnecessary type conversions and following staticcheck errors:
- remove redundant return statements
- use for range instead of for select
- use time.Since instead of time.Now().Sub
- omit comparison to bool constant
- replace T.Fatal and T.Fatalf in tests with T.Error and T.Fatalf respectively because the goroutine calls T.Fatal must be called in the same goroutine as the test
- fix error strings that should not be capitalized
- use sort.Strings(...) instead of sort.Sort(sort.StringSlice(...))
- use he status code of Canceled instead of grpc.ErrClientConnClosing which is deprecated
- use use status.Errorf instead of grpc.Errorf which is deprecated

Related #10528 #10438
 2019-06-05 17:02:05 -04:00
shivaramr 9150bf52d6 go modules: Fix module path version to include version number 2019-04-26 15:29:50 -07:00
Manjunath A Kumatagi 474cea1cd6 test: Add shadow tool 2019-02-06 23:20:06 -06:00
Sam Batschelet 2001786f02 *: Use -n instead of ! -z. [SC2236] 
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
 2019-01-08 13:45:40 -05:00
Gyuho Lee c58f5cfeda test: disable "unparam" for now 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2018-12-17 11:30:28 -08:00
Sam Batschelet 78d01140ff *: resolve shellcheck errors. 2018-10-25 19:41:11 -04:00
Vimal K bcde798fdd *: path changes for moving to github/etcd-io/etcd 2018-09-03 21:57:23 +05:30
Gyuho Lee 02b94fcc0d test: fix "license-bill-of-materials" command 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2018-08-28 20:53:28 -07:00
Gyuho Lee 1c04820026 tests: integrate "github.com/mgechev/revive" to fmt tests 
https://github.com/mgechev/revive

Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-07-21 15:49:53 -07:00
Joe Betz 12acfc057a vendor: upgrade grpc/grpc-go to v1.11.3 2018-06-15 13:41:28 -07:00
Gyuho Lee 4cbfcfeede test: fix test "etcdserver/api/v2store" 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-05-14 14:33:49 -07:00
Gyuho Lee 775e9d5ba6 test: add "unused_pass" 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-05-09 15:39:44 -07:00
Gyuho Lee 18e1686bdf test: only print test targets in verbose mode 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-05-01 10:07:25 -07:00
Gyuho Lee 1af2228fca test: add "unconvert" test 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-30 15:32:16 -07:00
Gyuho Lee 358640de46 test: fix "markdown_you_pass" 
"find" command was matching "vendor" directory in
my local machine.

Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-30 11:42:35 -07:00
Gyuho Lee cdadc4638d test: enable static analysis "SA4004" 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-30 10:04:44 -07:00
Gyuho Lee 5fd9270d6c test: disable SA4004 temporarily 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-27 14:21:17 -07:00
Gyuho Lee ad4b747ed4 test: configure go test -cpu flag 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-20 14:10:30 -07:00
Gyuho Lee fdf4a12d3c test: rename etcd functional test data directories 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-11 19:52:23 -07:00
Gyuho Lee 7ed4803fb0 test: remove "-a" GO_BUILD_FLAGS 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-11 01:45:06 -07:00
Gyuho Lee 7e56c3e737 test: remove backup data from functional tests 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-11 01:18:15 -07:00
Gyuho Lee 6cf9979cf0 test: remove wait commands 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-10 18:38:01 -07:00
Gyuho Lee eabb40f0ae test: combine "compile_pass" into "build_pass" 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-10 17:07:42 -07:00
Gyuho Lee 0e7f1100e3 cover: move coverage test script to "tests" 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-09 10:47:32 -07:00
Gyuho Lee 0c327ea456 test: move "e2e" and "functional" 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-09 10:47:25 -07:00
Gyuho Lee 85e050a120 *: rename, clean up functional tests 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-07 10:00:00 -07:00
Gyuho Lee d7b468ec6f test: bump up test timeout to 20m 
Recently, we've added bunch of tests...

Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-04 13:45:12 -07:00
Gyuho Lee 173fdc37a3 test: run shellcheck only on bash scripts 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-02 11:45:04 -07:00
Gyuho Lee 048a1b14d5 test: bump up default timeout for now 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-03-30 14:42:45 -07:00
Gyuho Lee 28766fb002 test: run test script with new "functional-tester" 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-03-30 11:25:23 -07:00
Gyuho Lee 9c995fab9b test: fix shellcheck 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-03-22 10:39:23 -07:00
Wenjia Zhang 3571f406aa scripts: add TESTCASE flag to run single unit test 
Enable user to run specified test case with different TIMEOUT expectation
when needed.
 2018-03-13 13:54:20 -07:00
Gyuho Lee 1da0818186 build: use "bash" syntax, clean up 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-02-27 11:07:36 -08:00
Gyuho Lee 18f8b1b1e1 build,test: set up GOPATH in build script 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-02-27 10:23:14 -08:00
Gyuho Lee 2dd8af3503 test: fix "internal/store" path 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-02-26 17:12:45 -08:00
Gyuho Lee d4aeb3b8cf
Merge pull request #9356 from mkumatag/fix_gopath 
test, build: Fix gopath creation
 2018-02-26 07:38:22 -08:00
Manjunath A Kumatagi adf4e05079 test, build: Fix gopath creation 2018-02-26 04:21:06 -05:00
Gyuho Lee 3e13424bb8 test: fix "internal/store" 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-02-12 09:21:11 -08:00