Commit Graph

28 Commits (c4d055fe7b0aa79c0be5e05e756743f90ee87d08)

Author SHA1 Message Date
Marek Siarkowicz 804fddf921 tests: Use zaptest.NewLogger in tests 2022-04-04 13:03:15 +02:00
Piotr Tabor f80f477073
Merge pull request #13644 from Juneezee/refactor/t.TempDir
*: use `T.TempDir` to create temporary test directory
2022-02-21 19:52:37 +01:00
Marek Siarkowicz 8c91d60a6f server: Switch to publishV3 2022-02-14 23:06:45 +01:00
Eng Zer Jun e9edc5432c
*: use `T.TempDir` to create temporary test directory
The directory created by `T.TempDir()` and is automatically removed when
the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-01-25 00:00:46 +08:00
Eng Zer Jun 2a151c8982
*: move from io/ioutil to io and os packages
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-10-28 00:05:28 +08:00
Marek Siarkowicz 83a325ac46 server: Move all functions needed for storage bootstrap to storage package
This is prerequestite to move storage bootstrap, splitted to separate PR
to make it easier to review.
2021-08-03 13:09:15 +02:00
Marek Siarkowicz a0554a6bd3 etcdserver: Create AuthBackend interface 2021-07-20 18:09:53 +02:00
Marek Siarkowicz 5b6f4579fb server: Rename buckets to schema 2021-07-12 15:37:21 +02:00
Marek Siarkowicz 5e40a8b00c server: Create storage package and move mvcc files to it 2021-07-12 15:37:21 +02:00
Marek Siarkowicz bf3e7033e9 etcdserver: Move Read/Update methods on Meta bucket to one place
There are still some left like compact keys, but they will require more
work to avoid circular dependency.
2021-07-05 13:23:53 +02:00
Marek Siarkowicz 50507d5f3c etcdserver: Membership uses MembershipStorage interface instead of directly accessing Backend 2021-06-29 16:14:06 +02:00
Piotr Tabor 0a972a3f05 applyV2 should reapply on backend only once
During review of:  https://github.com/etcd-io/etcd/pull/12988 spotted
that PUT is actially writing to v3-backend.
If we are replaying WAL log, it might happened that backend's
applied_index is > than the WAL's log entry. In such situation we should
skip applying on backend V3.
I think both the methods (setVersion, setMembersAttributes) are in
practice idempotent so its not that 'serious' problem, but for
formal correctness adding the proper checks.
2021-05-18 20:55:05 +02:00
Chao Chen 783c5ad2d2 use v2 api to update cluster version 2021-05-17 11:45:54 -07:00
Piotr Tabor ab586cd463 Persists Term in the (bbolt) Backend.
Additional layer of protection, that allows to validate whether we
start replaying log not only from the proper 'index', but also of the
right 'term'.
2021-05-13 21:29:01 +02:00
Piotr Tabor 865df75714 Save raftpb.ConfState in the backend.
This makes (bbolt) backend a full feature snapshot in term of WAL/raft,
i.e. carries:
  - commit : (applied_index)
  - confState

Benefits:
  - Backend will be a sufficient point in time definition sufficient to
start replaying WAL. We have applied_index & confState in consistent
state.
  - In case of emergency a backend state can be used for recovery
2021-05-13 14:29:36 +02:00
Piotr Tabor fe3254aee3 Remove explicit authStore->ConsistencyIndex updates, as they are taken care by hook. 2021-05-04 15:38:23 +02:00
Piotr Tabor a70386a1a4 Simplify membership interface: Does not pass the 'unused' token. 2021-04-27 17:17:31 +02:00
Piotr Tabor d7d110b5a8 mvcc/backend tests: Refactor: Do not mix testing&prod code. 2021-04-21 09:43:13 +02:00
Piotr Tabor b1c04ce043 Applying consistency fix: ClusterVersionSet (and co) might get no applied on v2store
ClusterVersionSet, ClusterMemberAttrSet, DowngradeInfoSet functions are
writing both to V2store and backend. Prior this CL there were
in a branch not executed if shouldApplyV3 was false,
e.g. during restore when Backend is up-to-date (has high
consistency-index) while v2store requires replay from WAL log.

The most serious consequence of this bug was that v2store after restore
could have different index (revision) than the same exact store before restore,
so potentially different content between replicas.

Also this change is supressing double-applying of Membership
(ClusterConfig) changes on Backend (store v3) - that lackilly are not
part of MVCC/KeyValue store, so they didn't caused Revisions to be
bumped.

Inspired by jingyih@ comment:
https://github.com/etcd-io/etcd/pull/12820#issuecomment-815299406
2021-04-12 09:43:48 +02:00
Piotr Tabor cc0f812f51 server_test.go: Use context.Background() instead of TODO in tests.
Suggected in: https://golang.org/pkg/context/#Background
2021-04-08 01:15:16 +02:00
Piotr Tabor 931af493cf Merge pull request #12830 from ptabor/20210405-split-pkg
Split client/pkg as dedicated low-dependencies module for client
2021-04-08 01:12:17 +02:00
Piotr Tabor 3bb7acc8cf Migrate dependencies pkg/foo -> client/pkg/foo 2021-04-07 00:38:47 +02:00
Piotr Tabor 03f55eeb2c Make NewTmpBackend use testing tmp location (so cleanup). 2021-03-26 13:54:55 +01:00
Piotr Tabor fd7fed1511 Move config (ServerConfig) out of etcdserver package.
Motivation:
  - ServerConfig is part of 'embed' public API, while etcdserver is more 'internal'
  - EtcdServer is already too big and config is pretty wide-spread leaf
if we were to split etcdserver (e.g. into pre & post-apply part).
2021-03-11 20:56:22 +01:00
lzhfromustc f2a912a4e6 test: change channel operations to avoid potential goroutine leaks
In these unit tests, goroutines may leak if certain branches are chosen. This commit edits channel operations and buffer sizes, so no matter what branch is chosen, the test will end correctly. This commit doesn't change the semantics of unit tests.
2020-12-09 22:23:21 -05:00
Piotr Tabor 6e800b9b01
20201103 no commit title check (#12447)
* Turn off checking of format of commit message.

* scripts/fix.sh: Fix fixing whitespaces in *.sh scripts

Aparently there is a difference between:
  find ./ -print0 -name *.sh and
  find ./ -name *.sh -print0

* etcdserver unit tests: Do not call .Fatalf(...) from not test's goroutine.

Fixes following test failures:
https://travis-ci.com/github/etcd-io/etcd/jobs/425920416
```
% (cd server && go vet ./...)
stderr: # go.etcd.io/etcd/server/v3/etcdserver
stderr: etcdserver/server_test.go:1002:4: call to (*T).Fatalf from a non-test goroutine
stderr: etcdserver/server_test.go:1166:4: call to (*T).Fatalf from a non-test goroutine
FAIL: (code:2):
  % (cd server && go vet ./...)
FAIL: 'run go vet ./...' checking failed (!=0 return code)
FAIL: 'govet' failed at Tue Nov  3 04:07:47 UTC 2020
```
2020-11-03 07:59:42 -08:00
Piotr Tabor aaf423e962 server: Update imports.
find -name '*.go' | xargs sed -i --follow-symlinks 's|etcd/v3/|etcd/server/v3/|g'
2020-10-26 13:02:32 +01:00
Piotr Tabor 4a5e9d1261 server: Move server files to 'server' directory.
26  git mv mvcc wal auth etcdserver etcdmain proxy embed/ lease/ server
   36  git mv go.mod go.sum server
2020-10-26 12:57:19 +01:00