80ccb27ea5
Merge pull request #12997 from hexfusion/fix-etcdutl-build
...
scripts: add missing etcdutl to release pipeline
2021-05-18 07:36:22 -04:00
86393ecfcd
scripts: add missing etcdutl to release pipeline
...
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-05-18 07:16:28 -04:00
3f7a038656
Fixed documentation and scripts.
2021-05-17 11:54:03 +02:00
c09aca1ba4
Split etcdctl into etcdctl (public API access) & etcdutl (direct surgery on files)
...
Motivation is as follows:
- etcdctl we only depend on clientv3 APIs, no dependencies of bolt, backend, mvcc, file-layout
- etcdctl can be officially supported across wide range of versions, while etcdutl is pretty specific to file format at particular version.
it's step towards desired modules layout, documented in: https://etcd.io/docs/next/dev-internal/modules/
2021-05-17 11:54:03 +02:00
2712c787ab
Fix release script: add `go mod tidy`
...
Seems that additinal refresh of go.sum files is needed during the
release process after editing the versions of modules.
Discovered the problem while I was testing whether etcdutl is
properly included in the target binaries using:
```
DRY_RUN=true REPOSITORY=\`pwd\` BRANCH='branch-with-etcdutl' ./scripts/release 3.5.0-foobar.2
```
2021-05-16 20:43:12 +02:00
3d4629c1fb
hack,scripts: rename "master" branch references to "main"
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-12 10:40:28 -07:00
3a04b2b61f
github action: add ARM64 test with README
...
README for self-hosted-linux-arm64-graviton2-tests.yml
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-10 00:28:41 -07:00
3bb7acc8cf
Migrate dependencies pkg/foo -> client/pkg/foo
2021-04-07 00:38:47 +02:00
f290ab2e60
Update dependecies:
...
github.com/grpc-ecosystem/grpc-gateway v1.14.6 -> grpc-gateway v1.16.0
golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e->v0.0.0-20210220033141-f8bda1e9f3ba
2021-03-27 20:48:33 +01:00
7665bd764e
scripts/update_dep.sh: Script to update selected versions of dependencies.
2021-03-27 11:17:56 +01:00
9180e8e56e
wal: Update WAL Snapshot format to contain ConfState.
2021-03-25 00:31:44 +01:00
19f7c6ef3e
*: Update gogo/protobuf to v1.3.2, rerun ./scripts/genproto.sh
...
While it appears that etcd is not vulnerable to CVE-2021-3121,
it is a good idea to update to the new generator so that new
vulnerable code isn't generated in any future APIs. Also, this
lays the issue to rest of whether there is any issue with
etcd and CVE-2021-3121.
2021-03-23 11:48:06 -06:00
a84bd093b0
Integration with grpc-settable logger.
2021-03-16 22:50:41 +01:00
f7a2389992
Update version of certifi/gocertifi to get rid of WTF Public license
...
Seems old versions of https://github.com/certifi/gocertifi where
categorized as "Do What The F*ck You Want To Public License".
Update to newer version that is explicit `Mozilla Public License` 2.0 (MPL 2.0).
2021-03-04 09:48:34 +01:00
a7f340216d
Reformat code according to 'gotip' rules.
...
In practices adds annotations in the new syntax:
```
+//go:build !linux
// +build !linux
```
Fixes failing gotip PASSES='fmt' check:
https://travis-ci.com/github/etcd-io/etcd/jobs/486453806
2021-02-26 10:14:46 +01:00
60a669762f
Improve release scripts:
...
- Fix script that creates manifest-list based multi-arch-images.
The images need to be pushed first.
- Use docker instead of gcloud docker helper
- Make sure docker pushes are properly 'dry run'
- Added preparation instruction to the release script.
2021-02-25 12:28:52 +00:00
c640957e2d
Release scripts: Minor fixes discovered during attempt for 3.5.0-alpha.0
2021-02-24 19:16:27 +00:00
1a9c81abda
Update grpc dependency to 1.32.
...
Simplify grpc testing infrastructure to align with upstream changes.
2021-02-23 11:31:50 +01:00
6bbef10721
Clarify the root directory comment
...
A contributor had an issue when ran script changing the directory
name after clone. This was discussed on the etcd slack channel.
2021-02-01 15:41:36 -05:00
90d1b838ad
Fix shellcheck warnings: Escaping.
...
```
build:3:65: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
test:3:64: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/fix.sh:23:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/fix.sh:23:46: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/fix.sh:30:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/genproto.sh:38:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/genproto.sh:53:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/genproto.sh:73:23: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
scripts/genproto.sh:73:33: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
scripts/genproto.sh:74:69: note: Backslash is literal in "\1". Prefer explicit escaping: "\\1". [SC1117]
scripts/genproto.sh:74:78: note: Backslash is literal in "\2". Prefer explicit escaping: "\\2". [SC1117]
scripts/genproto.sh:75:40: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
scripts/genproto.sh:75:52: note: Backslash is literal in "\1". Prefer explicit escaping: "\\1". [SC1117]
scripts/genproto.sh:91:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/genproto.sh:115:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
./test.sh:71:29: warning: Quote to prevent word splitting, or split robustly with mapfile or read -a. [SC2206]
./test.sh:400:28: note: Backslash is literal in "\t". Prefer explicit escaping: "\\t". [SC1117]
./test.sh:406:33: note: Backslash is literal in "\t". Prefer explicit escaping: "\\t". [SC1117]
./test.sh:415:36: note: Backslash is literal in "\*". Prefer explicit escaping: "\\*". [SC1117]
./test.sh:415:176: note: Backslash is literal in "\'". Prefer explicit escaping: "\\'". [SC1117]
./test.sh:493:56: note: Backslash is literal in "\.". Prefer explicit escaping: "\\.". [SC1117]
./test.sh:601:57: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
./test.sh:601:64: note: Backslash is literal in "\)". Prefer explicit escaping: "\\)". [SC1117]
./test.sh:601:67: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
./test.sh:601:74: note: Backslash is literal in "\)". Prefer explicit escaping: "\\)". [SC1117]
./test.sh:601:77: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
./test.sh:601:84: note: Backslash is literal in "\)". Prefer explicit escaping: "\\)". [SC1117]
./test.sh:601:91: note: Backslash is literal in "\1". Prefer explicit escaping: "\\1". [SC1117]
./test.sh:601:94: note: Backslash is literal in "\2". Prefer explicit escaping: "\\2". [SC1117]
./test.sh:601:103: note: Backslash is literal in "\3". Prefer explicit escaping: "\\3". [SC1117]
```
2021-01-29 22:32:43 +00:00
88ee005a0f
Reduce verbosity of coverage collection.
2021-01-23 08:59:19 +01:00
598ca6caab
Modernize release script:
...
- making sure the DRY_RUN mode can finish e2e, so e.g. commits to
local copy of repository are OK in dry-run (while git pushes are NOT).
- better interaction with ./test_lib.sh script.
- more consistent logging
- bringing back s390x architecture that on go 1.14.3 seems to work as
expected.
2021-01-21 17:47:36 +01:00
1d73d2e85e
Merge pull request #12632 from ptabor/20210119-fix-coverage-report
...
Fix code coverage measurement
2021-01-19 10:19:39 -08:00
486687843b
Fix code coverage measurement.
2021-01-19 15:41:36 +01:00
2529fee958
Use /build instead of deprecated /build.sh
2021-01-19 12:46:56 +11:00
7f79ffb409
Generate and push multi-arch docker manifest list
...
Change amd64 docker image to use a `-amd64` suffix, consistent with
other architectures. Introduce a multiarch docker manifest list with
the original (no-arch) docker tag.
Fixes #10380
2021-01-19 12:46:56 +11:00
577c898fee
scripts: Integrate ./scripts/release with new code for tagging modules.
...
Changes:
- signing tags.
- allows to override BRANCH and REPOSITORY using env variables.
Tested by a release in my private fork:
BRANCH="20201126-ptabor-release" REPOSITORY="git@github.com:ptabor/etcd.git" ./scripts/release 3.5.0-alpha.20
2021-01-15 12:31:44 +01:00
84b5b87fb2
Make test.sh scripts OSX/BSD compatible:
...
- build & test scripts deprecated. Call *.sh variants.
This will avoid delete the symlinks and get rid of
subtle dependency on 'sed --follow-symlinks' on OsX/BSD sed.
- Fix parameters to mktemp
2021-01-12 16:29:00 +01:00
23340bb62a
Refresh proto generation script after moving modules files.
...
With modulatiozation server protos get moved into ./server directory,
but it was not reflected in scripts/genproto.sh.
2021-01-08 16:33:12 +01:00
fd2f34fd13
Release: Scripts to change versions in all go.mod files and push tags upstream.
...
Examplar invocations:
Edit go.mod files such that all etcd modules are pointing on given version:
```
% DRY_RUN=false TARGET_VERSION="v3.5.13" ./scripts/release_mod.sh update_versions
```
Tag latest commit with current version number for all the modules and push upstream:
```
% DRY_RUN=true REMOTE_REPO="origin" ./scripts/release_mod.sh push_mod_tags
```
2020-11-04 15:16:36 +01:00
6e800b9b01
20201103 no commit title check ( #12447 )
...
* Turn off checking of format of commit message.
* scripts/fix.sh: Fix fixing whitespaces in *.sh scripts
Aparently there is a difference between:
find ./ -print0 -name *.sh and
find ./ -name *.sh -print0
* etcdserver unit tests: Do not call .Fatalf(...) from not test's goroutine.
Fixes following test failures:
https://travis-ci.com/github/etcd-io/etcd/jobs/425920416
```
% (cd server && go vet ./...)
stderr: # go.etcd.io/etcd/server/v3/etcdserver
stderr: etcdserver/server_test.go:1002:4: call to (*T).Fatalf from a non-test goroutine
stderr: etcdserver/server_test.go:1166:4: call to (*T).Fatalf from a non-test goroutine
FAIL: (code:2):
% (cd server && go vet ./...)
FAIL: 'run go vet ./...' checking failed (!=0 return code)
FAIL: 'govet' failed at Tue Nov 3 04:07:47 UTC 2020
```
2020-11-03 07:59:42 -08:00
2e55875cc7
scripts: install github.com/myitcv/gobin while gobin doesn't exist.
2020-10-30 20:58:00 +08:00
7da5182f1d
Merge pull request #12422 from tangcong/fix-realpath
...
scripts: fix realpath command not found in mac os
2020-10-26 10:42:26 -07:00
8277395e1b
scripts: use manual scripts to replace realpath
2020-10-27 00:36:28 +08:00
6c1efd6ba5
server: Update go.mod
2020-10-26 13:02:32 +01:00
0ba16d8ee1
*: Convert tabulators to whitespaces in bash scripts.
...
Execution of `./scripts/fix.sh` that executed:
```
find ./ -name '*.sh' | xargs sed --follow-symlinks -i 's|\t| |g'
```
2020-10-26 10:59:40 +01:00
c035df5317
test: Detect indention done using tab (\t) in *.sh
2020-10-26 10:59:40 +01:00
09679d29ad
etcdctl: Rename of imports after making etcdctl a module.
...
```
find -name '*.go' | xargs sed -i --follow-symlinks 's|etcd/v3/etcdctl|etcd/etcdctl/v3|g'
```
2020-10-21 11:15:35 +02:00
df48e499ea
etcdctl: Make etcdctl a module (go.mod, LICENSE)
2020-10-20 12:07:27 +02:00
cf795c0ae6
client/v3: Create and update go.mod
2020-10-20 10:09:12 +02:00
bea5f379bc
raft: Make raft a module
...
Establishing a module: Creating a go.mod files
2020-10-16 09:15:48 +02:00
45e4306508
client: Move client to client/v2 as a module.
...
We make v2 client code a module go.etcd.io/etcd/client/v2.
Pretty mechanical change that can be summarized as:
mkdir client/v2
cd client/v2 && git mod init go.etcd.io/etcd/client/v2
git mv client/*.go client/v2/
find -name '*.go' | xargs sed -i --follow-symlinks 's|/v3/client["]|/client/v2\"|g'
+ fixing changelog, bom, go.mod etc.
2020-10-15 14:39:43 +02:00
da5ca2c0d5
tools: Update of go.mod for protos (gogo/protobuf 1.3.1, grpc 1.29.1, grpc-gateway 1.14.6)
...
This brings consistency between proto-generation code and actual versions of libraries being used in runtime:
github.com/gogo/protobuf v1.2.1,v1.0.0 -> v1.3.1
github.com/golang/protobuf v1.3.2 -> v1.3.5
github.com/grpc-ecosystem/grpc-gateway v1.9.5,v1.4.1,v1.15.2 -> v1.14.6
google.golang.org/grpc v1.26.0 -> v1.29.1
Moved as far as possible, without bumping on grpc 1.30.0 "naming" decomissioning.
Please also notice that gogo/protobuf is likely to reach EOL: https://github.com/gogo/protobuf/issues/691
2020-10-14 18:46:38 +02:00
bc9e433ca2
tools: Migrate remaining tools to gobin
...
Replace ./scripts/install_tool.sh with `gobin`, such that we have
consistent handling for all tools needed for build and consistent
versioning within ./tools/mod/go.mod.
Side changes:
- Expose /scripts/fix.sh that fixes formatting and bom across modules
- Expose *.sh variants of scripts like build and ./test (first step
towards replacement).
- Make stderr output of commands explicit and make commands use
different color than callouts.
2020-10-13 19:33:01 +02:00
bfa91d48af
pkg: Estabilish pkg as a module
2020-10-12 23:58:09 +02:00
0b95e8cef1
Merge pull request #12374 from ptabor/20201009-new-genproto
...
script/genproto.sh: Refactor to be explicit about versions.
2020-10-12 14:25:43 -07:00
00e49d0c10
build: Fix updating 'go.etcd.io/etcd/api/v3/version.GitSHA' + test ( #12382 )
...
During move of code to 'api' (0aab02e7b5
2020-10-09 17:32:15 -07:00
dfdda47bd8
script/genproto.sh: Refactor to be explicit about versions.
...
Refactoring script/genproto.sh around state-of-the-art techniques of
managing tooling in go:
- https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module
- uses https://github.com/myitcv/gobin instead of customly created gopath.proto dir
- caches tools between executions
- guaratees hermetics runs (it was not guaranteed for protoc_grpc_gateway that used latest)
The change is no-op for the generated code.
The commit reveals a few 'worring things':
1 We depend on : github.com/grpc-ecosystem/grpc-gateway/@v/v1.4.1/protoc-gen-grpc-gateway
2. And also : github.com/grpc-ecosystem/grpc-gateway/@v/v1.15.0/protoc-gen-swagger/protoc-gen-swagger
3. And on extremely old: github.com/gogo/protobuf@v1.0.0 protoc-gen-gofast that is out of sync with the library linked to binaries: github.com/gogo/protobuf@v1.2.1
2020-10-08 19:52:27 +02:00
313087f8c5
./test,./scripts: Update go.sum and bill-of-materials generation logic (and refresh bom).
2020-10-07 15:38:39 +02:00
7886ec8574
test: Modules: PASSES="integration" ./test understand new file locations
2020-10-07 15:38:39 +02:00
4fb48b7c14
tests: Estabilishing module for tests
...
For now it contains only e2e tests, but integration & functional will follow.
2020-10-07 15:03:51 +02:00
28f2b07623
*: Update references to code moved to the api/ dir.
...
Follow up to file-moves done in the previous commit.
The commit contains purely mechanical consequences of execution (apart
of scripts/genproto.sh):
% find ./ -name '*.go' | xargs sed --follow-symlinks -i 's|v3/etcdserver/api/v3rpc/rpctypes|v3/api/v3rpc/rpctypes|g'
% find ./ -name '*.go' | xargs sed --follow-symlinks -i 's|v3/version|v3/api/version|g'
% find ./ -name '*.go' | xargs sed --follow-symlinks -i 's|v3/mvcc/mvccpb|v3/api/mvccpb|g'
% find ./ -name '*.go' | xargs sed --follow-symlinks -i 's|v3/etcdserver/etcdserverpb|v3/api/etcdserverpb|g'
% find ./ -name '*.go' | xargs sed --follow-symlinks -i 's|v3/etcdserver/api/membership/membershippb|v3/api/membershippb|g'
% find ./ -name '*.go' | xargs sed --follow-symlinks -i 's|v3/auth/authpb|v3/api/authpb|g'
% find ./ -name '*.proto' -o -name '*.md' | xargs -L 1 sed --follow-symlinks -i 's|/mvcc/mvccpb/kv.proto|/api/mvccpb/kv.proto|g'
% find ./ -name '*.proto' -o -name '*.md' | xargs -L 1 sed --follow-symlinks -i 's|/auth/authpb/auth.proto|/api/authpb/auth.proto|g'
% find ./ -name '*.proto' -o -name '*.md' | xargs -L 1 sed --follow-symlinks -i 's|/etcdserver/api/membership/membershippb/membership.proto|/api/membershippb/membership.proto|g'
I also modified manually paths in scripts/genproto.sh.
% go fmt ./...
2020-10-06 11:56:16 +02:00
2edb08642c
api: Make api/ a module that will contain proto-definitions.
...
The module is supposed to contain minimal set of files that establish
public etcd server API. In particular client libraries for etcd built in
different languages might want to depend on this file.
2020-10-06 11:54:50 +02:00
0e5d81704f
.travis.yml, scripts: Fix minor bugs in the test script.
...
1. setting environment variable cannot be in quote
2. "--race" testing for unit tests is supposed to be part of linux-amd64-unit-4-cpu-race config.
3. 'run' function in test script should log_error in case of failed
command (wrong operator for ints comparison in bash).
2020-10-01 14:34:58 +02:00
f1d4593241
./test: Refactoring of test script for modularization
...
This refactoring offers following benefits:
- Unified way how go test commands are being called (in terms of flags intepretation)
- Uses standard go mechanisms (like go lists) to find files/packages that are subject for test. The mechanism are module aware.
- Added instruction how to install tools needed for the tests/checkers.
- Added colors to the output to make it easier to spot any failure.
Confirmed to work using:
- COVERDIR="./coverage" CPU="4" RACE=false COVER=false PASSES="build build_cov cov" ./test
- CPU="4" RACE=false COVER=false PASSES="e2e functional integration" ./test
- COVERDIR="./coverage" COVER="false" CPU="4" RACE="false" PASSES="fmt build unit build_cov integration e2e integration_e2e grpcproxy cov" ./test
- PASSES=unit PKG=./wal TIMEOUT=1m ./test
- PASSES=integration PKG=./clientv3 TIMEOUT=1m ./test
- PASSES=unit PKG=./wal TESTCASE=TestNew TIMEOUT=1m ./test
- PASSES=unit PKG=./wal TESTCASE="\bTestNew\b" TIMEOUT=1m ./test
- PASSES=integration PKG=./client/integration TESTCASE="\bTestV2NoRetryEOF\b" TIMEOUT=1m ./test
- COVERDIR=coverage PASSES="build_cov cov" ./test
2020-09-28 11:07:50 +02:00
138a312643
scripts: Removed updatedep.sh.
...
As we get rid of ./vendor there is no need to update the dependencies.
If anyone needs up-to-date vendor directory locally, getting it is as
simple as:
```go mod vendor```
2020-09-16 13:04:58 +02:00
fb2e96e651
vendor: Get rid of ./vendor cleanup
...
Updated scripts and documentation to not recommend vendoring.
Implemented best practices for tools installation.
Performed multiple tests to confirm its not breaking any workflows and
has no negative performance impact. Rather see 3x speedup.
1. PASSES="fmt unit integration e2e functional" ./test
2. ./scripts/updatebom.sh
3. ./scripts/updatedep.sh
4. ./scripts/genproto.sh - works - ca be simplified - in follow up PR
5. Installation without explicit GOPATH:
```
% unset GOPATH
% [sudo] rm -rf ~/go
% git clone https://github.com/etcd-io/etcd.git
% time ./build
go: downloading google.golang.org/grpc v1.26.0
go: downloading github.com/jonboulle/clockwork v0.1.0
go: downloading github.com/prometheus/client_golang v1.0.0
go: downloading github.com/soheilhy/cmux v0.1.4
go: downloading github.com/gogo/protobuf v1.2.1
go: downloading sigs.k8s.io/yaml v1.1.0
go: downloading golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2
go: downloading github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903
go: downloading go.etcd.io/bbolt v1.3.5
go: downloading go.uber.org/zap v1.15.0
go: downloading golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc
go: downloading github.com/golang/protobuf v1.3.2
go: downloading github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8
go: downloading github.com/beorn7/perks v1.0.0
go: downloading github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4
go: downloading github.com/coreos/go-systemd/v22 v22.0.0
go: downloading gopkg.in/yaml.v2 v2.2.2
go: downloading github.com/coreos/go-semver v0.2.0
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7
go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: downloading github.com/google/uuid v1.0.0
go: downloading github.com/modern-go/reflect2 v1.0.1
go: downloading github.com/prometheus/common v0.4.1
go: downloading github.com/spf13/cobra v0.0.3
go: downloading github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2
go: downloading github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c
go: downloading github.com/spf13/pflag v1.0.1
go: downloading github.com/json-iterator/go v1.1.7
go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: downloading github.com/google/btree v1.0.0
go: downloading go.uber.org/atomic v1.6.0
go: downloading github.com/prometheus/procfs v0.0.2
go: downloading go.uber.org/multierr v1.5.0
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5
go: downloading github.com/grpc-ecosystem/grpc-gateway v1.9.5
go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4
go: downloading github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: downloading google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading golang.org/x/text v0.3.3
go: downloading github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5
go: downloading github.com/bgentry/speakeasy v0.1.0
go: downloading gopkg.in/cheggaaa/pb.v1 v1.0.25
go: downloading github.com/urfave/cli v1.20.0
go: downloading github.com/mattn/go-runewidth v0.0.2
./build 8.22s user 2.31s system 117% cpu 8.961 total
```
Before:
```
% git clone https://github.com/etcd-io/etcd.git && cd etcd && time ./build
Cloning into 'etcd'...
remote: Enumerating objects: 97872, done.
remote: Total 97872 (delta 0), reused 0 (delta 0), pack-reused 97872
Receiving objects: 100% (97872/97872), 58.97 MiB | 19.85 MiB/s, done.
Resolving deltas: 100% (63091/63091), done.
./build 34.97s user 4.15s system 236% cpu 16.555 total
```
6. Rebuild without changes:
```
% time ./build
./build 1.43s user 0.83s system 168% cpu 1.336 total
```
7. Instantation of vendor directory (assuming ./build loaded them to
$GOPATH/pkg):
```
time go mod vendor
go: downloading github.com/inconshreveable/mousetrap v1.0.0
go: downloading github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa
go: downloading github.com/creack/pty v1.1.11
go: downloading github.com/etcd-io/gofail v0.0.0-20190801230047-ad7f989257ca
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1
go mod vendor 0.51s user 0.44s system 110% cpu 0.861 total
```
8. Fresh instantation of vendor:
```
% rm -rf vendor
% [sudo] rm -rf ~/go
% time go mod vendor
go: downloading github.com/coreos/go-systemd/v22 v22.0.0
go: downloading github.com/spf13/cobra v0.0.3
go: downloading github.com/prometheus/client_golang v1.0.0
go: downloading golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7
go: downloading github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4
go: downloading github.com/gogo/protobuf v1.2.1
go: downloading sigs.k8s.io/yaml v1.1.0
go: downloading google.golang.org/grpc v1.26.0
go: downloading github.com/urfave/cli v1.20.0
go: downloading go.uber.org/zap v1.15.0
go: downloading github.com/spf13/pflag v1.0.1
go: downloading github.com/soheilhy/cmux v0.1.4
go: downloading github.com/json-iterator/go v1.1.7
go: downloading github.com/coreos/go-semver v0.2.0
go: downloading github.com/prometheus/common v0.4.1
go: downloading github.com/prometheus/procfs v0.0.2
go: downloading go.uber.org/atomic v1.6.0
go: downloading github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5
go: downloading github.com/golang/protobuf v1.3.2
go: downloading github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa
go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4
go: downloading github.com/modern-go/reflect2 v1.0.1
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading go.uber.org/multierr v1.5.0
go: downloading github.com/creack/pty v1.1.11
go: downloading github.com/mattn/go-runewidth v0.0.2
go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: downloading golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc
go: downloading golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5
go: downloading github.com/jonboulle/clockwork v0.1.0
go: downloading gopkg.in/yaml.v2 v2.2.2
go: downloading github.com/etcd-io/gofail v0.0.0-20190801230047-ad7f989257ca
go: downloading github.com/grpc-ecosystem/grpc-gateway v1.9.5
go: downloading github.com/google/btree v1.0.0
go: downloading google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
go: downloading github.com/beorn7/perks v1.0.0
go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: downloading github.com/google/uuid v1.0.0
go: downloading golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2
go: downloading github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8
go: downloading go.etcd.io/bbolt v1.3.5
go: downloading golang.org/x/text v0.3.3
go: downloading gopkg.in/cheggaaa/pb.v1 v1.0.25
go: downloading github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2
go: downloading github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: downloading github.com/inconshreveable/mousetrap v1.0.0
go: downloading github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading github.com/bgentry/speakeasy v0.1.0
go: downloading github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1
go mod vendor 3.62s user 1.30s system 127% cpu 3.854 total
```
9. Size of the repository - before: 39M, after: 18M
Before:
```
% time git clone https://github.com/etcd-io/etcd.git
Cloning into 'etcd'...
remote: Enumerating objects: 97872, done.
remote: Total 97872 (delta 0), reused 0 (delta 0), pack-reused 97872
Receiving objects: 100% (97872/97872), 58.97 MiB | 20.53 MiB/s, done.
Resolving deltas: 100% (63091/63091), done.
git clone https://github.com/etcd-io/etcd.git 4.66s user 1.02s system 93% cpu 6.068 total
% du -h --exclude .git -d 1
944K ./clientv3
108K ./etcdmain
5.4M ./Documentation
384K ./security
384K ./mvcc
28K ./.github
8.0K ./version
144K ./contrib
240K ./proxy
2.5M ./etcdserver
112K ./embed
536K ./integration
332K ./tools
116K ./lease
108K ./logos
896K ./tests
960K ./raft
216K ./client
52K ./scripts
100K ./hack
464K ./etcdctl
3.0M ./pkg
620K ./functional
136K ./wal
152K ./auth
21M ./vendor
39M
```
After:
```
% time git clone https://github.com/ptabor/etcd.git -b 20200908-no-vendor
Cloning into 'etcd'...
remote: Enumerating objects: 38, done.
remote: Counting objects: 100% (38/38), done.
remote: Compressing objects: 100% (37/37), done.
remote: Total 98489 (delta 10), reused 8 (delta 1), pack-reused 98451
Receiving objects: 100% (98489/98489), 59.23 MiB | 21.26 MiB/s, done.
Resolving deltas: 100% (63572/63572), done.
git clone https://github.com/ptabor/etcd.git -b 20200908-no-vendor 5.56s user 1.05s system 105% cpu 6.260 total
% du -h --exclude .git -d 1
944K ./clientv3
108K ./etcdmain
5.4M ./Documentation
384K ./security
384K ./mvcc
28K ./.github
8.0K ./version
144K ./contrib
240K ./proxy
2.5M ./etcdserver
112K ./embed
536K ./integration
332K ./tools
116K ./lease
108K ./logos
896K ./tests
960K ./raft
216K ./client
56K ./scripts
100K ./hack
464K ./etcdctl
3.0M ./pkg
620K ./functional
136K ./wal
152K ./auth
19M .
```
2020-09-15 14:38:54 +02:00
d93c36bbb9
*: run "gofmt -s" on genproto complete
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-09 23:14:32 -07:00
03248a9045
bill-of-materials: updates for /etcd/v3 packaging
...
bill-of-materials:
- update to new package names
test/updatebom.sh scripts:
- Update to the right package names
- Don't add bom tool to go.mod
2020-04-28 00:57:52 +00:00
2ea4d88299
*: use go.etcd.io/etcd/v3 where needed
...
Found potential locations via
```
git grep 'go.etcd.io/etcd$'
```
2020-04-28 00:57:52 +00:00
3ffb3aed3a
vendor: update bbolt v1.3.4, zap v1.14.1
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-19 13:36:43 -07:00
17a220eaee
scripts: Added s390x support for docker image release
2020-01-20 07:09:59 -08:00
dcd622b2c7
etcdserver: add v3 request type for cluster attr
...
Added ClusterVersionSetRequest for setting cluster version via v3 apply.
Added ClusterMemberAttrSetRequest for setting clsuter member attributes
via v3 apply.
2019-12-05 16:25:13 -08:00
cabf5942b2
scripts/build-binary: Added s390x support for binary release
2019-11-13 22:57:23 -08:00
fd5a25fcec
scripts/release: list GPG key only when tagging is needed
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-10-23 10:33:39 -07:00
36696fe2cc
Add version, tag and branch checks to release script
2019-10-11 15:47:15 -07:00
0824b08cab
Merge pull request #11237 from YoyinZyc/prevent-darwin-build
...
scripts: avoid release builds on darwin machine.
2019-10-11 10:39:14 -07:00
b9358a0bc8
scripts: avoid release builds on darwin machine.
2019-10-11 09:56:48 -07:00
55efdfe364
scripts: fix read failure prompt in release; use https for git clone.
2019-10-10 13:23:36 -07:00
46f3e8349b
scripts/build-binary: fix darwin tar commands
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-27 23:35:34 -07:00
838315283d
scripts/release: fix SHA256SUMS command
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-17 14:12:50 -07:00
c327120ba0
scripts/release: fix docker push command
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-17 13:52:22 -07:00
1dff148f69
scripts/updatedep: use Go 1.13
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-06 11:39:03 -07:00
d981fe6b6b
scripts/release: Apply shellcheck findings
...
I run https://github.com/koalaman/shellcheck/ over scripts/* and fixed
the findings it returned.
Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2019-09-02 17:50:04 +02:00
f108a32393
scripts/release: rename SHA256SUM to SHA256SUMS
...
These files are commonly called SHA256SUMS and with this change rget
works for v3.4.0 as well.
2019-08-30 13:31:17 -07:00
837fe89d0b
scripts/release: fix sha256sum
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-29 09:39:18 -07:00
2386b945c6
scripts/release: add sha256sum summary of release assets
...
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2019-08-28 15:56:39 +00:00
fd0d43dad9
scripts/release: clean up minor tag docker commands
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-13 22:00:07 -07:00
8d823e70dd
Merge pull request #11018 from gyuho/yq
...
scripts/release: update "yq" command
2019-08-12 11:03:37 -07:00
c72fa51b2a
scripts: remove ".aci" commands
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-12 10:20:21 -07:00
af38185f92
scripts/release: fix version check commands
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-12 10:20:13 -07:00
ac613c481f
scripts: fix build docker commands, add more logging
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-12 10:20:05 -07:00
7299a6a106
*: remove "acbuild"
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-12 09:31:22 -07:00
1f4f8ae731
scripts/release: update "yq" command
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-12 09:26:04 -07:00
f7992237ce
scripts/release: remove acbuild commands
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-04 23:20:41 -07:00
14625b847c
scripts: have genproto.sh clean up after itself
...
We don't want it to leave gopath.proto around for reasons detailed in
the previous commit (messing up vgo).
2019-07-16 14:01:04 +02:00
f63984bb33
scripts: fail explicitly in updatedep.sh when gopath.proto exists
...
I had been dealing with these intermittent failures for a while and
finally figured out why. The real solution is making genproto.sh less
ugly but that won't happen for a while.
2019-07-16 13:54:09 +02:00
b40597ce46
module: require 1.12, remove "v3" import paths
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-06-03 11:15:19 -07:00
c8ffa36d9e
scripts/genproto: bump up protoc 3.7.1
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-05-28 15:39:35 -07:00
86e3481ba2
scripts: Remove generated code and script
...
Change-Id: Iac4601443bcad71920fd96b97bfe21c16116577a
2019-04-23 16:54:44 -04:00
3e0f0ba40e
Merge pull request #10401 from markmc/doc-drop-etcdctl-v3-flag
...
Eliminate some ETCDCTL_API=3 usage
2019-02-01 11:25:55 -08:00
a011b2c4c4
scripts: disable go mod and bump protoc to 3.6.1
...
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2019-01-22 16:55:01 +00:00
e53324db3b
scripts/release: stop using ETCDCTL_API=3
...
Note: v3 has been the default since 25bc65794
2019-01-14 14:46:16 +00:00
d5967b40db
scripts/updatedep: fix shellcheck errors
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2018-09-21 02:08:25 -07:00
ad2d18aeff
scripts/updatedep: use Go 1.11 module for dependency management
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2018-09-21 01:47:24 -07:00
bcde798fdd
*: path changes for moving to github/etcd-io/etcd
2018-09-03 21:57:23 +05:30
a8a3efd27a
scripts: update import paths in "go.etcd.io/etcd"
...
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2018-08-28 17:47:55 -07:00
981644af7d
scripts/updatedep: remove "dep prune" command
...
"Pruning is now performed automatically by dep ensure."
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-08-02 08:06:27 -07:00
453db62d84
scripts/genproto: use protoc 3.6.0
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-07-10 10:16:37 -07:00
a8990e2e18
scripts: Fix release script to always build etcd binary before checking version
2018-06-06 10:48:45 -07:00
e2821a16ff
scripts/release: Fix docker push for 3.1 releases, remove inaccurate warning at the end of release script
2018-05-31 13:56:07 -07:00
0db7ea7337
scripts/release: disable minor version docker push
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-05-31 12:08:01 -07:00
fe15c3250e
scripts/genproto: use latest "grpc-ecosystem/grpc-gateway"
2018-05-23 08:36:58 -04:00
05554119c9
vendor: use latest "grpc-ecosystem/grpc-gateway"
2018-05-22 17:35:15 -04:00
d29a0c7172
scripts/genproto: use latest "gogo/protobuf"
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-05-22 13:42:43 -07:00
2d542d336a
scripts/genproto.sh: rename
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-05-21 07:45:34 -07:00
4319a8a2ba
scripts: Fix remote tag check, gcloud login and umask in release script
2018-05-09 10:37:24 -07:00
bee6f34197
scripts: Fix a few etcd release script bugs and make it reenterant.
2018-04-24 17:03:14 -07:00
7750974760
scripts: Add scripts/release that performs 'etcd-release-runbook' ( https://goo.gl/Gxwysq ) style release workflow
2018-04-18 13:35:41 -07:00
140374e25d
*: move "scripts/docker-*" to "tests"
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-04-09 10:31:00 -07:00
6f2adc8a3d
scripts: add dev scripts
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-04-02 11:27:39 -07:00
f5c56401d7
scripts/update-clientv2: initial commit
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-27 10:33:27 -07:00
906a40bced
scripts/genproto: fix shellcheck
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-22 10:39:31 -07:00
27f0aa9cf0
scripts: document "updatedep.sh" script
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-03-07 13:54:56 -08:00
b24d81c30d
scripts: remove "internal"
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-02-26 17:11:40 -08:00
dac6948774
scripts/genproto: run "protodoc" by default
...
Do not skip "protodoc" by default.
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-02-13 13:51:13 -08:00
264f8f6577
*: regenerate proto
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-01-28 23:43:15 -08:00
2c91007591
*: regenerate proto
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-01-26 11:21:40 -08:00
698ec7f7d5
scripts/updatebom.sh: remove "./cmd/vendor"
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-01-24 15:26:17 -08:00
505d24c463
scripts/updatedep.sh: use "golang/dep"
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-01-24 15:26:16 -08:00
2b6fd3094f
*: rename "snap" to "raftsnap" package
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-01-24 10:26:07 -08:00
fd16656af1
scripts/genproto: use "grpc-gateway" v1.3.1
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-01-09 12:41:49 -08:00
7800fd9fff
scripts/genproto: require protoc 3.5.1
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2017-12-31 13:20:40 -08:00
3518351a24
scripts: remove ACI build scripts
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2017-12-20 15:35:11 -08:00
b1c6b98f3d
scripts/genproto: require protoc 3.5, update gogo/proto
...
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2017-12-07 21:31:13 -08:00
aca39e2ae1
scripts/build-docker: build both gcr.io and quay.io images
...
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-11-28 15:02:09 -08:00
ec881b0507
scripts/genproto: upgrade protoc to 3.4
...
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-11-08 18:50:29 -08:00
c0c19465fc
*: upgrade grpc-gateway to v1.3
2017-11-08 18:38:41 -08:00
9abe9da9db
*: fix shellcheck warnings
...
Fixes scripts and removes shellcheck warning suppressions.
* regexp warnings
* use ./*glob* so names don't become options
* use $(..) instead of legacy `..`
* read with -r to avoid mangling backslashes
* double quote to prevent globbing and word splitting
2017-09-06 19:18:04 -07:00
e5aa938fec
scripts: generate swagger with authorization support
2017-06-15 13:41:43 -04:00
e402606f02
build-aci: Fix ACI image name
...
The appc discovery spec states that the architecture specifier in the ACI
image file name will be an ACI architecture value. Our build scripts were
using GOARCH in the image name, which is incorrect for arm64/aarch64.
See: https://github.com/appc/spec/blob/master/spec/discovery.md
Fixes errors like these on arm64 machines:
$ rkt --debug --insecure-options=image fetch coreos.com/etcd:v3.2.0-rc.1
image: remote fetching from URL "https://github.com/coreos/etcd/releases/download/v3.2.0-rc.1/etcd-v3.2.0-rc.1-linux-aarch64.aci "
fetch: bad HTTP status code: 404
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-06-13 13:09:02 -07:00
4ebeba0e18
*: regen protofiles with latest protobuf tools
2017-06-12 15:14:43 -07:00
85f433232a
*: clear rarer shellcheck errors on scripts
...
Clean up the tail of the warnings
2017-06-06 09:36:25 -07:00
1be245269e
scripts, Documentation: fix swagger generation
...
Changes to the genproto to support splitting out the grpc-gateway broke
swagger generation.
2017-06-02 10:54:05 -07:00
156612bb25
Documentation, scripts: regen RPC docs
...
Was missing the new cancel_reason field. Also includes updated protodoc
sha to fix generating documentation for upcoming txn compare range patchset.
2017-06-02 10:15:12 -07:00
10b1ba7886
Documentation, scripts: regenerate protobuf docs with updated protodoc
2017-05-19 14:57:16 -07:00
8c9f01ef53
scripts: add 'BOM' update script
...
Need this script when we add external dependencies.
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-05-08 17:59:11 -07:00
e9a63473a0
scripts,travis: install 'marker' for CI tests
...
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-05-04 08:26:00 -07:00
6c1ce697a6
scripts: annotate with acbuild with supports-systemd-notify
...
Fixes #7840
2017-05-01 12:59:08 -07:00
76fb6ebcbb
scripts: remove testify hack in updatedep
2017-04-22 20:29:58 -07:00
07c07cea25
release: Add multi arch support
...
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-04-21 10:04:41 -07:00
0c8988aa07
build-docker: Updates for multi-arch release
...
o Set -e to abort script if a command fails.
o Allow custom docker 'TAG' from the environment.
o Move arch suffix to version to allow all images to
be put into a single repository.
o Enable cross builds. When doing cross builds where the
host and target architectures are different 'RUN mkdir'
will fail since the target container cannot be run on
the host. To work around this, create the directories
in build-docker, then use ADD in the Dockerfile.
o Add Dockerfile-release.arm64
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-04-21 10:04:41 -07:00
8309ca92d7
build-aci: Add multi arch support
...
Uses GOARCH to build for a targeted arch.
Usage: GOARCH=... BINARYDIR=... BUILDDIR=... ./scripts/build-aci version
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-04-21 10:04:41 -07:00
fb6287240f
build-binary: Add arm64
...
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-04-21 10:04:41 -07:00
fd11523af9
scripts: move gateway stubs into gw/ packages
2017-04-19 12:50:04 -07:00
78a5eb79b5
*: add swagger and grpc-gateway assets for v3lock and v3election
2017-04-10 15:21:07 -07:00
9ba69ff317
scripts: update genproto.sh to include v3election
2017-04-07 16:36:38 -07:00
1ff0b71b30
*: use protoc 3.2.0
...
Fixes #7631
2017-03-30 13:43:10 -07:00
c737bf3d2a
scripts: generate lock service rpc stubs
2017-03-13 15:23:26 -07:00
633cfbe241
script: Build docker image for ppc64le
2017-02-27 19:04:32 -05:00
Sam Batschelet