vitalif
/
etcd
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
16,418 Commits
21 Branches
406 Tags
79 MiB
Commit Graph

45 Commits (bc9e433ca26be0314a37c8c6ead66429a584bbc9)

Author SHA1 Message Date
Piotr Tabor dfdda47bd8 script/genproto.sh: Refactor to be explicit about versions. 
Refactoring script/genproto.sh around state-of-the-art techniques of
managing tooling in go:
  - https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module
  - uses https://github.com/myitcv/gobin instead of customly created gopath.proto dir
  - caches tools between executions
  - guaratees hermetics runs (it was not guaranteed for protoc_grpc_gateway that used latest)

The change is no-op for the generated code.

The commit reveals a few 'worring things':
  1  We depend on : github.com/grpc-ecosystem/grpc-gateway/@v/v1.4.1/protoc-gen-grpc-gateway
  2. And also     : github.com/grpc-ecosystem/grpc-gateway/@v/v1.15.0/protoc-gen-swagger/protoc-gen-swagger
  3. And on extremely old: github.com/gogo/protobuf@v1.0.0 protoc-gen-gofast that is out of sync with the library linked to binaries: github.com/gogo/protobuf@v1.2.1
 2020-10-08 19:52:27 +02:00
Piotr Tabor 313087f8c5 ./test,./scripts: Update go.sum and bill-of-materials generation logic (and refresh bom). 2020-10-07 15:38:39 +02:00
Piotr Tabor f1d4593241 ./test: Refactoring of test script for modularization 
This refactoring offers following benefits:

  - Unified way how go test commands are being called (in terms of flags intepretation)
  - Uses standard go mechanisms (like go lists) to find files/packages that are subject for test. The mechanism are module aware.
  - Added instruction how to install tools needed for the tests/checkers.
  - Added colors to the output to make it easier to spot any failure.

Confirmed to work using:
- COVERDIR="./coverage" CPU="4" RACE=false COVER=false PASSES="build build_cov cov" ./test
- CPU="4" RACE=false COVER=false PASSES="e2e functional integration" ./test
- COVERDIR="./coverage" COVER="false" CPU="4" RACE="false" PASSES="fmt build unit build_cov integration e2e integration_e2e grpcproxy cov" ./test
- PASSES=unit PKG=./wal TIMEOUT=1m ./test
- PASSES=integration PKG=./clientv3 TIMEOUT=1m ./test
- PASSES=unit PKG=./wal TESTCASE=TestNew TIMEOUT=1m ./test
- PASSES=unit PKG=./wal TESTCASE="\bTestNew\b" TIMEOUT=1m ./test
- PASSES=integration PKG=./client/integration TESTCASE="\bTestV2NoRetryEOF\b" TIMEOUT=1m ./test
- COVERDIR=coverage PASSES="build_cov cov" ./test
 2020-09-28 11:07:50 +02:00
Jingyi Hu 14566556f3
Merge pull request #12283 from teddylear/feature/FixPortOnTest 
embed: TestStartEtcdWrongToken now uses dynamic ports instead of default
 2020-09-20 08:30:48 -07:00
teddylear 3b92b9f884 embed: TestStartEtcdWrongToken now uses dynamic ports instead of default 
To avoid issues with a test failing due to port conflict when etcd is
already running, this test now uses dynamic ports instead.

Fixes #11956
 2020-09-19 11:29:22 -04:00
Piotr Tabor fb2e96e651 vendor: Get rid of ./vendor cleanup 
Updated scripts and documentation to not recommend vendoring.
Implemented best practices for tools installation.

Performed multiple tests to confirm its not breaking any workflows and
has no negative performance impact. Rather see 3x speedup.

1. PASSES="fmt unit integration e2e functional" ./test
2. ./scripts/updatebom.sh
3. ./scripts/updatedep.sh
4. ./scripts/genproto.sh - works - ca be simplified - in follow up PR
5. Installation without explicit GOPATH:

```
% unset GOPATH
% [sudo] rm -rf ~/go
% git clone https://github.com/etcd-io/etcd.git
% time ./build
go: downloading google.golang.org/grpc v1.26.0
go: downloading github.com/jonboulle/clockwork v0.1.0
go: downloading github.com/prometheus/client_golang v1.0.0
go: downloading github.com/soheilhy/cmux v0.1.4
go: downloading github.com/gogo/protobuf v1.2.1
go: downloading sigs.k8s.io/yaml v1.1.0
go: downloading golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2
go: downloading github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903
go: downloading go.etcd.io/bbolt v1.3.5
go: downloading go.uber.org/zap v1.15.0
go: downloading golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc
go: downloading github.com/golang/protobuf v1.3.2
go: downloading github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8
go: downloading github.com/beorn7/perks v1.0.0
go: downloading github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4
go: downloading github.com/coreos/go-systemd/v22 v22.0.0
go: downloading gopkg.in/yaml.v2 v2.2.2
go: downloading github.com/coreos/go-semver v0.2.0
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7
go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: downloading github.com/google/uuid v1.0.0
go: downloading github.com/modern-go/reflect2 v1.0.1
go: downloading github.com/prometheus/common v0.4.1
go: downloading github.com/spf13/cobra v0.0.3
go: downloading github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2
go: downloading github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c
go: downloading github.com/spf13/pflag v1.0.1
go: downloading github.com/json-iterator/go v1.1.7
go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: downloading github.com/google/btree v1.0.0
go: downloading go.uber.org/atomic v1.6.0
go: downloading github.com/prometheus/procfs v0.0.2
go: downloading go.uber.org/multierr v1.5.0
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5
go: downloading github.com/grpc-ecosystem/grpc-gateway v1.9.5
go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4
go: downloading github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: downloading google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading golang.org/x/text v0.3.3
go: downloading github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5
go: downloading github.com/bgentry/speakeasy v0.1.0
go: downloading gopkg.in/cheggaaa/pb.v1 v1.0.25
go: downloading github.com/urfave/cli v1.20.0
go: downloading github.com/mattn/go-runewidth v0.0.2
./build  8.22s user 2.31s system 117% cpu 8.961 total
```

Before:
```
% git clone https://github.com/etcd-io/etcd.git && cd etcd && time ./build
Cloning into 'etcd'...
remote: Enumerating objects: 97872, done.
remote: Total 97872 (delta 0), reused 0 (delta 0), pack-reused 97872
Receiving objects: 100% (97872/97872), 58.97 MiB | 19.85 MiB/s, done.
Resolving deltas: 100% (63091/63091), done.

./build  34.97s user 4.15s system 236% cpu 16.555 total
```

6. Rebuild without changes:

```
% time ./build
./build  1.43s user 0.83s system 168% cpu 1.336 total
```

7. Instantation of vendor directory (assuming ./build loaded them to
$GOPATH/pkg):

```
time go mod vendor
go: downloading github.com/inconshreveable/mousetrap v1.0.0
go: downloading github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa
go: downloading github.com/creack/pty v1.1.11
go: downloading github.com/etcd-io/gofail v0.0.0-20190801230047-ad7f989257ca
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1
go mod vendor  0.51s user 0.44s system 110% cpu 0.861 total
```

8. Fresh instantation of vendor:

```
% rm -rf vendor
% [sudo] rm -rf ~/go

% time go mod vendor
go: downloading github.com/coreos/go-systemd/v22 v22.0.0
go: downloading github.com/spf13/cobra v0.0.3
go: downloading github.com/prometheus/client_golang v1.0.0
go: downloading golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7
go: downloading github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4
go: downloading github.com/gogo/protobuf v1.2.1
go: downloading sigs.k8s.io/yaml v1.1.0
go: downloading google.golang.org/grpc v1.26.0
go: downloading github.com/urfave/cli v1.20.0
go: downloading go.uber.org/zap v1.15.0
go: downloading github.com/spf13/pflag v1.0.1
go: downloading github.com/soheilhy/cmux v0.1.4
go: downloading github.com/json-iterator/go v1.1.7
go: downloading github.com/coreos/go-semver v0.2.0
go: downloading github.com/prometheus/common v0.4.1
go: downloading github.com/prometheus/procfs v0.0.2
go: downloading go.uber.org/atomic v1.6.0
go: downloading github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5
go: downloading github.com/golang/protobuf v1.3.2
go: downloading github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa
go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4
go: downloading github.com/modern-go/reflect2 v1.0.1
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading go.uber.org/multierr v1.5.0
go: downloading github.com/creack/pty v1.1.11
go: downloading github.com/mattn/go-runewidth v0.0.2
go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: downloading golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc
go: downloading golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5
go: downloading github.com/jonboulle/clockwork v0.1.0
go: downloading gopkg.in/yaml.v2 v2.2.2
go: downloading github.com/etcd-io/gofail v0.0.0-20190801230047-ad7f989257ca
go: downloading github.com/grpc-ecosystem/grpc-gateway v1.9.5
go: downloading github.com/google/btree v1.0.0
go: downloading google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
go: downloading github.com/beorn7/perks v1.0.0
go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: downloading github.com/google/uuid v1.0.0
go: downloading golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2
go: downloading github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8
go: downloading go.etcd.io/bbolt v1.3.5
go: downloading golang.org/x/text v0.3.3
go: downloading gopkg.in/cheggaaa/pb.v1 v1.0.25
go: downloading github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2
go: downloading github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: downloading github.com/inconshreveable/mousetrap v1.0.0
go: downloading github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading github.com/bgentry/speakeasy v0.1.0
go: downloading github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1
go mod vendor  3.62s user 1.30s system 127% cpu 3.854 total
```

9. Size of the repository - before: 39M, after: 18M

Before:

```
% time git clone https://github.com/etcd-io/etcd.git
Cloning into 'etcd'...
remote: Enumerating objects: 97872, done.
remote: Total 97872 (delta 0), reused 0 (delta 0), pack-reused 97872
Receiving objects: 100% (97872/97872), 58.97 MiB | 20.53 MiB/s, done.
Resolving deltas: 100% (63091/63091), done.
git clone https://github.com/etcd-io/etcd.git  4.66s user 1.02s system 93% cpu 6.068 total

% du -h --exclude .git -d 1
944K	./clientv3
108K	./etcdmain
5.4M	./Documentation
384K	./security
384K	./mvcc
28K	./.github
8.0K	./version
144K	./contrib
240K	./proxy
2.5M	./etcdserver
112K	./embed
536K	./integration
332K	./tools
116K	./lease
108K	./logos
896K	./tests
960K	./raft
216K	./client
52K	./scripts
100K	./hack
464K	./etcdctl
3.0M	./pkg
620K	./functional
136K	./wal
152K	./auth
21M	./vendor
39M
```

After:
```
% time git clone https://github.com/ptabor/etcd.git -b 20200908-no-vendor
Cloning into 'etcd'...
remote: Enumerating objects: 38, done.
remote: Counting objects: 100% (38/38), done.
remote: Compressing objects: 100% (37/37), done.
remote: Total 98489 (delta 10), reused 8 (delta 1), pack-reused 98451
Receiving objects: 100% (98489/98489), 59.23 MiB | 21.26 MiB/s, done.
Resolving deltas: 100% (63572/63572), done.
git clone https://github.com/ptabor/etcd.git -b 20200908-no-vendor  5.56s user 1.05s system 105% cpu 6.260 total

% du -h --exclude .git -d 1
944K	./clientv3
108K	./etcdmain
5.4M	./Documentation
384K	./security
384K	./mvcc
28K	./.github
8.0K	./version
144K	./contrib
240K	./proxy
2.5M	./etcdserver
112K	./embed
536K	./integration
332K	./tools
116K	./lease
108K	./logos
896K	./tests
960K	./raft
216K	./client
56K	./scripts
100K	./hack
464K	./etcdctl
3.0M	./pkg
620K	./functional
136K	./wal
152K	./auth
19M	.
```
 2020-09-15 14:38:54 +02:00
Piotr Tabor 2d76f9d17d vendor: Remove /vendor and add to .gitignore 2020-09-15 14:38:47 +02:00
Gyuho Lee a48706ecfb *: remove old gitignore and build functions 
Not needed since we are now using go module.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2020-05-30 14:35:34 -07:00
Jingguo Yao e86a12bad0 gitignore: ignore build result and runtime files 2019-01-01 19:14:48 +08:00
Gyuho Lee 11028331e6 gitignore: add "*.coverprofile" 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-05-03 17:22:34 -07:00
Gyuho Lee 8e00d764be gitignore: remove outdated lists 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-13 11:01:19 -07:00
Gyuho Lee 0ca37a0c1d gitignore: add ".Dockerfile-functional-tester" 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-01-24 17:37:31 -08:00
Gyuho Lee 4fc2db7d65 gitignore: ignore test files in "vendor" 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-01-24 15:26:17 -08:00
Joe Betz 1bca2e969f test: Clean agent directories on disk before functional test runs, not after 
This is primarily so CI tooling can capture the agent logs after the functional tester runs.
 2017-11-14 13:09:52 -08:00
Gyu-Ho Lee 736b9f0be3 gitignore: ignore hidden Dockerfiles for docker build 
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
 2017-11-02 06:58:58 -07:00
Gyu-Ho Lee fa7c8f3f83 gitignore: add covdir 2017-10-23 10:34:30 -07:00
Gyu-Ho Lee 5790ffde7c gitignore: ignore *.log, release directory 
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
 2017-10-17 09:05:10 -07:00
rpatel af20ba21cb .gitignore: Adding .idea to .gitignore 
This will keep  all intellij IDEA IDE related files out of git.
This helps contributors using IDEA IDE for development.
 2017-01-24 22:14:20 -08:00
Gyu-Ho Lee 8174fcf201 scripts/genproto: use 'gopath.proto' for $GOPATH 2016-10-03 15:34:31 -07:00
Jeff Zellner 0938c861f0 hack: add tls-setup example generated certs to gitignore 2016-06-30 10:11:28 -06:00
Pawel Palucki 4f5f999847 tools/functional-test: add docker support 
Commit adds docker bits to run functional tester within containers.

requires:
- docker 1.9 (networking)
- docker-compose
 2015-12-18 15:56:18 +01:00
Brian Waldon 677d9d1bea etcd: use old default data-dir format 2014-10-24 16:41:42 -07:00
Brian Waldon 9af5b74a8d gitignore: be more specific when ignoring /etcd 2014-10-22 17:52:40 -07:00
Ben Darnell fe0e168b3b Add *.test to gitignore. 
These files are left by some go commands.
 2014-10-07 18:22:09 -04:00
Brandon Philips ef0ed31210 hack: introduce a insta-discovery script 
This sets up a localhost cluster via a new discovery token. Very useful
for testing.
 2014-10-07 10:52:40 -07:00
Jonathan Boulle 255e62dcdd gitignore: ignore vim swap files 2014-09-10 12:06:28 -07:00
Brandon Philips 6b647fd481 gitignore: add _etcd_data directory ignores 
After 9a57d1067d we want to ignore all of
these _etcd_data directories.
 2014-09-09 14:30:09 -07:00
Jonathan Boulle 491362f5db scripts: add build, cover and update test 
This adds a build script that attempts to be as user friendly as
possible: if they have already set $GOPATH and/or $GOBIN, use those
environment variables. If not, create a gopath for them in this
directory. This should facilitate both `go get` and `git clone` usage.

The `test` script is updated, and the new `cover` script facilitates
easy coverage generation for the repo's constituent packages by setting
the PKG environment variable.
 2014-09-08 23:09:49 -07:00
Blake Mizerany 8f3544ece8 server of one is alive! 2014-09-03 09:20:19 -07:00
Yicheng Qin 84ad6ddd79 chore(build): set build path to gopath/ 
With this, pkg/ could be used as normal directory, and it doesn't need
to gitignore pkg/, which shields pkg/http and pkg/strings.
 2014-04-09 09:51:58 -07:00
Blake Mizerany 99c8b097d1 ignore .etcd dirs 2014-03-26 12:19:20 -07:00
Slawosz Slawinski 838a945862 feat(Vagrantfile): Add Vagrantfile for easy start 2014-03-15 12:48:48 -07:00
Brandon Philips 33be0e09fe fix(build/test.sh): use new GOPATH setup 2014-02-02 16:57:36 -08:00
Brandon Philips e2e0853492 fix(server/release_version): checkin to git 2014-01-14 22:14:47 -08:00
Brandon Philips 0f97e3528a chore(build): use third_party.go 
use the third_party.go project to replace our update script. This
requires moving a few things around and gets rid of a few annoying bugs:

- You can now bump individual packages
- A new src directory isn't created on build
- Less shell scripting!
- Things get built into ./bin/
 2014-01-14 22:14:47 -08:00
Brandon Philips cc88215b46 fix(bench): initial commit 2013-12-20 15:19:02 -08:00
Brandon Philips b2d72b3e55 bump(github.com/jteeuwen/go-bindata): 79847ab3e91ae5d2e1b18796c1795c78f29565d7 2013-11-08 14:05:16 -08:00
Ben Johnson eb78d96a20 Intermediate commit. 2013-10-12 00:28:46 -06:00
Brandon Philips cbd8a4fb9c feat(scripts/test-cluster): add a cluster test command 
this uses tmux to setup a test cluster that you can easily kill and
start for debugging.
 2013-09-26 10:40:33 -07:00
Brandon Philips 0ef9d944f6 fix(gitignore): ignore the actual binary 2013-09-25 17:04:34 -07:00
Brandon Philips d7c6d0c672 fix(gitignore): be more specific on dirs 2013-08-06 18:39:23 -07:00
Brandon Philips a6b7b8ce3f fix(gitignore): ignore the etcd binary not everything 
fixes a problem with go-etcd not being added to third_party
 2013-08-06 18:38:00 -07:00
Brandon Philips dc6480dda8 fix(gitignore): ignore the pkg dir 2013-08-06 17:21:43 -07:00
Brandon Philips a7deba0f90 feat(version): generate version from git tags 
This will generate the version based on git tags. So if it is built
directly on a tag it will use the tag name and if it is built off a
commit that isn't a tag it will look like:

    $ curl localhost:7001
    0-267-ga39cf1c
 2013-08-06 15:39:30 -07:00
Brandon Philips 1eb0be10fe feat(build): add initial build script 
add a simple build script that sets up a gopath and uses the current git
directory for the github.com/coreos/etcd packages.

There aren't a lot of great alternatives to doing it this way unless we
want to check in all of the dependencies into the repo (which is
actually a good practice probably).
 2013-07-10 15:45:15 -07:00